An error in routine maintenance terminated some backend and user facing infrastructure. The rollback procedure failed, so engineers intervened to bring back system components in a priority order based on our DR planning. Mission critical components, such as traffic delivery POPs, were unaffected during the incident.
Posted May 11, 2021 - 21:24 UTC
Resolved
This incident has been resolved.
Posted May 11, 2021 - 15:42 UTC
Monitoring
We have implemented workarounds for the issue and are currently monitoring the system.
Posted May 11, 2021 - 07:47 UTC
Update
We are continuing to work on restoring the portal.
Posted May 11, 2021 - 07:12 UTC
Update
Cache clear API has been restored and is operational.
Posted May 11, 2021 - 06:31 UTC
Update
We are continuing to work on resolving the cache clear API availability.
Posted May 11, 2021 - 06:14 UTC
Update
We are continuing to resolve the issue.
Posted May 11, 2021 - 05:43 UTC
Update
Service to logs and metrics is being restored. They may take some time to appear in the portal. We are still investigating the cache clear API.
Posted May 11, 2021 - 04:52 UTC
Update
We are continuing to restore the system. We are currently working on restoring the availability of logs and the cache clear API.
Posted May 11, 2021 - 04:11 UTC
Update
Access to the management portal has been restored. We are still investigating issues with the API functionality. Metrics/Logs are not available at the moment.
Posted May 11, 2021 - 03:40 UTC
Investigating
Section portal is currently down. We are currently investigating.
Posted May 11, 2021 - 02:35 UTC
This incident affected: Management Console - Aperture and API.