PyPI Outage

Incident Report for Python Infrastructure

Resolved

Incident is resolved. During the database failover 37,889 requests received an error at our CDN edge and 3,723 requests to backend servers resulted in an error.

No data loss or ongoing impact to PyPI was detected.
Posted Dec 23, 2017 - 14:45 UTC

Monitoring

PyPI service is restored and we are actively monitoring and ensuring data consistency. All looks good and this incident will be marked resolved shortly.
Posted Dec 23, 2017 - 14:42 UTC

Identified

Our high availability database system underwent a failover event causing all backend web servers to stop serving requests.

The database system failover took approximately six minutes but has completed and servers are coming back online.
Posted Dec 23, 2017 - 14:40 UTC

Investigating

We are investigating hard outage of PyPI's backend servers.

Simple Index and Package installs should continue to be served by the CDN while we investigate.
Posted Dec 23, 2017 - 14:38 UTC