Yesterday, March 11th, at 6:23 PM, a Dreamhost staffer alerted me that Philly Future was incurring too much database load and that they were disabling it:

Hello,

I am sorry but I had to disable your database (it was single-handedly killing the database server – the load was around 20 and is now down
under 1 (4 is optimal) after disablement.

We can re-enable it if you can assure us you’ll take proper steps to curb the usage (you need to be sure you are indexing things if you weren’t and perhaps re-evaluate the efficiency of your code).

Thanks!
John

Immediately upon getting the notice we emailed Dreamhost our intent to help them resolve any issues they may have.

As of 12:03PM EST I still cannot access our database to attempt to improve performance or do a dump of it to migrate us. I have been emailing them all night.

This situation comes as a surprise since back on February 3rd we received a thank you from Dreamhost stating we had successfully reduced resource utilization:

You have successfully reduced your resource usage on machine limbo-spunky.

Therefore we have moved your account (#109016) back to its original home machine foothill.
Thank you for your efforts and we are glad to have your account in good standing again.

Moving you back to the original server will change any IPs associated with your web services. Please get the new ones if you set up any custom scripts or DNS that relies on the old IP to function.

Happy Dreamhost Support Team!

While we looked for and prepared a new hosting solution, Dreamhost’s staff worked with us to reduce our CPU utilization to an acceptable level over the course of January and as you can see, we had a successful resolution.

That experience strengthened my trust in Dreamhost, I came away impressed by them, and it enabled me to not worry about our hosting situation while my daughter was being born.

I assure you there were *no* warnings and *no* indications of a current problem.

I want to apologize to our community. This is an unacceptable situation that we hope to have resolved sometime today.

Thank you for your patience and understanding.

More updates as they come.

Karl