One of our web servers in the load balancer stopped communicating with Memcached and was querying the database on every request. At 30 Million requests per day, that’s a bit much for a database to handle.
Resolved - We believe all users experiencing issues have been able to connect at this time. (21:20 — Aug 21)
Update - Slow and intermittent connections are still being experienced and we continue to investigate. (19:14 — Aug 21)
Monitoring - After stopping the Package Validator and Verifier we will continue to monitor the status of the services. (18:57 — Aug 21)
Update - We will be temporarily stopping the Package Validator and Package Verifier to reduce load on the website and database. (18:40 — Aug 21)
Monitoring - After performing maintenance on the packages database we will continue to monitor the status of the services. (17:02 — Aug 21)
Update - The database has been taken down for a short period while we perform maintenance. (16:30 — Aug 21)
Investigating - We’re aware of users experiencing slow and intermittent failed connections to the website and to both package repositories. We’re currently investigating. (16:01 — Aug 21)