PyPI is experiencing 503 Backend Unhealthy Errors

Incident Report for Python Infrastructure

Resolved

Back online, going to dig and see if we can alleviate some memory usage issues.
Posted 12 years ago. Aug 30, 2013 - 17:45 UTC

Monitoring

We've brought pypi back online and are keeping tabs to make sure we're clear.
Posted 12 years ago. Aug 30, 2013 - 17:38 UTC

Update

We've restored the backing data store and are working to bring the application back up.
Posted 12 years ago. Aug 30, 2013 - 17:18 UTC

Identified

A VM crash due to out of memory errors took down our primary Pypi application server.

Unfortunately this confused DRBD which is running to keep Pypi's data safe. We are re syncing the DRBD device and should be back online soon.
Posted 12 years ago. Aug 30, 2013 - 17:07 UTC

Investigating

We received automated notification of issues with the PyPI backend. Service will be degraded while we investigate.
Posted 12 years ago. Aug 30, 2013 - 16:42 UTC