The mail.python.org service is now in a stable and reliable state. This incident is resolved.
Posted Oct 29, 2015 - 01:44 UTC
Update
The new mail.python.org infrastructure is continuing to function as expected. There may be some continued delays for users who's ISP enforces IP based constraints on mail delivery. The postmaster team is working to notify ISPs of our change in IP.
Posted Oct 27, 2015 - 16:56 UTC
Update
Outbound mail has caught up for the vast majority of recipients. We are monitoring as the backlog of greylisted mail empties. This may take some time. Additionally we'll be keeping track of any functionality which may have been affected by migrating to the new infrastructure.
Posted Oct 27, 2015 - 06:08 UTC
Update
An issue without outbound mail on the new mail.python.org has been identified and resolved. We are working through the backlog of messages quickly now. We'll continue to monitor all aspects of mail delivery and mailing list activity looking out for any issues. Please don't hesitate to reach out to postmaster@python.org with any unexpected behavior.
Posted Oct 27, 2015 - 03:55 UTC
Update
The restored mail server continues to operate as expected. We have tuned the Mail Handling Services to more fairly service the backlog incurred during downtime and continue to monitor as it works through the backlog.
Posted Oct 26, 2015 - 06:15 UTC
Update
Outbound mail from the new mail.python.org server may experience abnormally long delays in delivery due to an IP Address change which was required during restoration. We are continuing to monitor that greylisting retries are occurring, as well as monitoring for whitelisting notifications from other ISPs.
Posted Oct 25, 2015 - 22:34 UTC
Update
all of mail.python.org's core services are now running and available. We are continuing to monitor for issues. If unexpected behavior is seen, please notify postmaster@python.org
Posted Oct 25, 2015 - 21:44 UTC
Update
We are continuing to work on bringing Mail Handling and Mailing List processing services fully online.
Posted Oct 25, 2015 - 20:42 UTC
Monitoring
Required DNS updates have been rolled out to point Mail Handling Services and Mailing List Web traffic to the restored server. We are monitoring and will be working to verify all systems are operational.
Posted Oct 25, 2015 - 19:01 UTC
Update
Restoration is progressing. The postmaster team is continuing to recover configurations and services to a working state. During this time we are also beginning to validate functionality before updating requisite DNS entries.
Posted Oct 25, 2015 - 17:14 UTC
Update
The postmaster team is continuing efforts to restore mail.python.org's functionality. We will continue to update with any new information as it is available.
Posted Oct 25, 2015 - 03:22 UTC
Update
The backup has been transferred to the new instance created for recovery. We are working with the postmaster team to restore services on the new host.
Posted Oct 24, 2015 - 16:44 UTC
Update
We've been informed that the server hosting mail.python.org has failed. We're brining up a new mail server and restoring from backup.
Posted Oct 23, 2015 - 23:23 UTC
Identified
The hardware hosting mail.python.org has apparently failed and is unable to boot. We are awaiting more information from the data center facility.
Posted Oct 23, 2015 - 21:33 UTC
Investigating
Beginning 2015-10-23 10:00, the server hosting mail.python.org became unresponsive and inaccessible. We are currently working to identify the cause of the crash and to get a technician in the data center to ascertain the state of the hardware.
This outage affects all @python.org email accounts, mailing lists, and access to mailing list archives.
Posted Oct 23, 2015 - 20:03 UTC
This incident affected: mail.python.org (Message Handling Services, Mailing Lists and Archives - Mailman).