We are now at 100 days since the decision to disable the XMLRPC search endpoint.
Traffic to the endpoint has not subsided in any substantial way and we have not heard from any of the parties who continue to issue hundreds of thousands of search calls per hour.
As such, XMLRPC search will be permanently disabled.
Posted 4 years ago. Mar 24, 2021 - 14:22 UTC
Update
The XMLRPC Search endpoint remains disabled due to ongoing request volume. As of this update, there has been no reduction in inbound traffic to the endpoint from abusive IPs and we are unable to re-enable the endpoint, as it would immediately cause PyPI service to degrade again.
Posted 4 years ago. Jan 12, 2021 - 16:04 UTC
Update
We are continuing to monitor for any further issues.
Posted 4 years ago. Dec 28, 2020 - 13:51 UTC
Update
The XMLRPC Search endpoint remains disabled due to ongoing request volume. As of this update, there has been no reduction in inbound traffic to the endpoint from abusive IPs and we are unable to re-enable the endpoint, as it would immediately cause PyPI service to degrade again.
Posted 4 years ago. Dec 28, 2020 - 13:50 UTC
Update
The XMLRPC Search endpoint is still disabled due to ongoing request volume. As of this update, there has been no reduction in inbound traffic to the endpoint from abusive IPs and we are unable to re-enable the endpoint, as it would immediately cause PyPI service to degrade again. We are working with the abuse contact at the owner of the IPs and trying to make contact with the maintainers of whatever tool is flooding us via other channels.
Posted 4 years ago. Dec 23, 2020 - 14:54 UTC
Update
The XMLRPC Search endpoint is still disabled due to ongoing request volume. As of this update, there has been no reduction in inbound traffic to the endpoint from abusive IPs and we are unable to re-enable the endpoint, as it would immediately cause PyPI service to degrade again. We are working with the abuse contact at the owner of the IPs and trying to make contact with the maintainers of whatever tool is flooding us via other channels.
Posted 4 years ago. Dec 15, 2020 - 20:59 UTC
Monitoring
With the temporary disabling of XMLRPC we are hoping that the mass consumer that is causing us trouble will make contact. Due to the huge swath of IPs we were unable to make a more targeted block without risking more severe disruption, and were not able to receive a response from their abuse contact or direct outreach in an actionable time frame.
Posted 4 years ago. Dec 14, 2020 - 17:46 UTC
Update
Due to the overwhelming surges of inbound XMLRPC search requests (and growing) we will be temporarily disabling the XMLRPC search endpoint until further notice.
Posted 4 years ago. Dec 14, 2020 - 17:30 UTC
Identified
We've identified that the issue is with excess volume to our XLMRPC search endpoint that powers `pip search` among other tools. We are working to try to identify patterns and prohibit abusive clients to retain service health.
Posted 4 years ago. Dec 14, 2020 - 15:09 UTC
Investigating
PyPI's search backends are experiencing an outage causing the backends to timeout and fail, leading to degradation of service for the web app. Uploads and installs are currently unaffected but logged in actions and search via the web app and API access via XMLRPC are currently experiencing partial outages.
Posted 4 years ago. Dec 14, 2020 - 09:41 UTC
This incident affected: PyPI (pypi.org - CDN, pypi.org - Backends).