Microsoft incident EX394347
June 21, 2022 8:31 PM (UTC)
Title: Users were unable to access their Exchange Online mailboxes via any connection method
Users were unable to access their Exchange Online mailboxes via any connection method.
More info: Outlook on the web was also impacted. Affected users would have encountered delays and failures when creating, sending, receiving, or accessing email messages. Additionally, affected users also experienced reduced search functionality within the impacted services, including Outlook Room Finder.
Final status: We’ve confirmed from our updated service monitoring that all services remain healthy following the targeted restarts. Additionally, we completed our investigation into the potential remaining impact to Outlook on the web and confirmed that this is a known issue which is unrelated to this event.
Scope of impact: Impact was most severe in Western Europe but would also have impacted a small percentage of users located throughout EMEA, North America, and the Asia-Pacific regions. Start time: Tuesday, June 21, 2022, 1:00 AM (6/20/2022, 11:00 PM UTC) End time: Tuesday, June 21, 2022, 5:27 PM (3:27 PM UTC)
Root cause: An infrastructure power outage necessitated failing over Microsoft 365 traffic management servicing users primarily in Western Europe. This action failed to properly complete, leading to functional delays and access failures for several Microsoft 365 services. This is the final update for the event A Post-Incident Report will be published within five business days.
June 21, 2022 6:12 PM (UTC)
Title: Some users may be unable to access their Exchange Online mailboxes via any connection method.
User Impact: Users may be unable to access their Exchange Online mailboxes via any connection method.
More info: Outlook on the web is also affected. Users may encounter delays and experience failures when creating, sending, receiving, or accessing email messages. Additionally, Users may also experience reduced search functionality within the impacted services, which includes Outlook Room Finder.
Current status: We've confirmed that restarting the underlying CAFE infrastructure has resolved the majority of impact; however, monitoring indicates some residual impact for Outlook on the web. We're investigating the underlying cause of this suboptimal service health so we can ensure all users' Outlook on the web functionality is restored.
Scope of impact: Impact is specific to some users located in EMEA, North America and Asia-Pacific. Start time: Tuesday, June 21, 2022, 1:00 AM (6/20/2022, 11:00 PM UTC) Preliminary root cause: CAFE requests routed to a specific traffic management system weren't being processed as expected, inhibiting users' ability to access some Microsoft 365 services.
Next update by: Tuesday, June 21, 2022, 10:00 PM (8:00 PM UTC)
June 21, 2022 4:02 PM (UTC)
Title: Some users may be unable to access their Exchange Online mailboxes via any connection method User Impact: Users may be unable to access their Exchange Online mailboxes via any connection method.
More info: Outlook on the web is also affected. Users may encounter delays and experience failures when creating, sending, receiving, or accessing email messages. Additionally, Users may also experience reduced search functionality within the impacted services, which includes Outlook Room Finder.
Current status: We've restarted the affected CAFE infrastructure responsible for facilitating users' requests to access the impacted Exchange Online services and Outlook on the web. While many users should already be experiencing relief, we're continuing to monitor service telemetry to ensure resolution is achieved for all affected users.
Scope of impact: Impact is specific to some users located in EMEA, North America and Asia-Pacific. Start time: Tuesday, June 21, 2022, 1:00 AM (6/20/2022, 11:00 PM UTC) Preliminary root cause: Requests routed to a specific traffic management system weren't processing CAFE requests as expected, inhibiting users' ability to access some Microsoft 365 services.
Next update by: Tuesday, June 21, 2022, 8:00 PM (6:00 PM UTC)
June 21, 2022 1:59 PM (UTC)
Title: Some users may be unable to access their Exchange Online mailboxes via any connection method.
User Impact: Users may be unable to access their Exchange Online mailboxes via any connection method.
More info: Outlook on the web is also affected. Users may encounter delays and experience failures when creating, sending, receiving or accessing email messages. Additionally, Users may also experience reduced search functionality within the impacted services, which includes Outlook Room Finder.
Current status: We're continuing to see improvement in service availability, but some users are still experiencing impact. We're restarting the infrastructure that serves user requests, to resolve the remaining impact and fully restore service.
Scope of impact: Impact is specific to some users located in EMEA, North America and Asia-Pacific.
Next update by: Tuesday, June 21, 2022, 6:00 PM (4:00 PM UTC)
June 21, 2022 12:57 PM (UTC)
Title: Some users may be unable to access their Exchange Online mailboxes via any connection method.
User Impact: Users may be unable to access their Exchange Online mailboxes via any connection method.
More info: Outlook on the web is also affected. Users may encounter delays and experience failures when creating, sending, receiving or accessing email messages. Additionally, Users may also experience reduced search functionality within the impacted services, which includes Outlook Room Finder.
Current status: We're continuing to see improvement in service availability across all impacted services which we're monitoring to ensure the impact is no longer occurring.
Scope of impact: Impact is specific to some users located in EMEA, North America and Asia-Pacific.
Next update by: Tuesday, June 21, 2022, 4:00 PM (2:00 PM UTC)
June 21, 2022 11:59 AM (UTC
Title: Some users may be unable to access their Exchange Online mailboxes via any connection method.
User Impact: Users may be unable to access their Exchange Online mailboxes via any connection method.
More info: Users may encounter delays and experience failures when creating, sending, receiving or accessing email messages. Additionally, Users may also experience reduced search functionality within the impacted services, which includes Outlook Room Finder.
Current status: We've identified that our traffic management infrastructure is not working as expected. We've successfully routed traffic to an alternate traffic management method, and we're seeing an improvement in service availability since this process completed. We're continuing to monitor the service availability as it improves, while we determine the root cause.
Scope of impact: Impact is specific to some users located in EMEA, North America and Asia-Pacific.
Next update by: Tuesday, June 21, 2022, 3:00 PM (1:00 PM UTC)
June 21, 2022 10:56 AM (UTC)
Title: Some users may be unable to access their Exchange Online mailboxes via any connection method.
User Impact: Users may be unable to access their Exchange Online mailboxes via any connection method.
More info: Users may encounter delays and experience failures when creating, sending, receiving or accessing email messages.
Current status: We're redirecting traffic to alternate, healthy infrastructure to mitigate impact, while we continue to investigate the root cause of this issue.
Scope of impact: Impact is specific to some users located in Europe and North America.
Next update by: Tuesday, June 21, 2022, 2:00 PM (12:00 PM UTC)
June 21, 2022 09:56 AM (UTC)
Title: Some users may be unable to access their Exchange Online mailboxes via any connection method User Impact: Users may be unable to access their Exchange Online mailboxes via any connection method. Current status: We're continuing to analyze service monitoring telemetry to identify the next troubleshooting steps to mitigate the impact.
Scope of impact: Impact is specific to some users located in Europe.
Next update by: Tuesday, June 21, 2022, 1:00 PM (11:00 AM UTC)
June 21, 2022 7:49 AM (UTC)
Title: Some users may experience delays or access issues in the Exchange Online service via any connection method.
User impact: Users may experience delays or access issues in the Exchange Online service via any connection method.
More info: Users may encounter delays sending, receiving or accessing email messages.
Current status: Microsoft is analyzing service monitoring telemetry to identify the root cause and develop a remediation plan.
Scope of impact: Impact is specific to some users located in Europe.
Next update by: Tuesday, June 21, 2022, 12:00 PM (10:00 AM UTC)
June 21, 2022 7:07 AM (UTC)
Title: We're looking into a potential problemUser Impact: We're investigating the potential end-user impact.
Title: We're looking into a potential problemUser Impact: We're investigating the potential end-user impact.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 60 minutes. (edited)
Comments
0 comments
Article is closed for comments.