Identi.ca down [UPDATE: not any more]
Identi.ca is currently down; I just got off an airplane. We had a flood of new users and posters over the last 24 hours, and the extra traffic seems to have sunk our database server.
I'm trying to revive the server (it's currently re-starting), but I might take the opportunity to move to our new data centre. If we're down, maybe it's time to do the move. Watch this space for details.
Update 14:50 PM PST: I can't seem to revive the main database server, so I'm copying data to another server in the same data centre to see if it wants to wake up.
Update 23:40 PM PST: The main and backup database servers are down, so I've started the process of moving to the new data centre. It should take about 10-12 more hours.
Update 01:30 AM PST: The import from backup is going nicely. It looks like we're going to lose a few hours of posts from Monday night, but I might be able to recover them from the old servers. I've consequently changed the DNS for identi.ca to point to the new data centre; that's going to take a few hours to propagate, maybe up to 24 hours.
Update 03:40 AM PST: The import stopped because I did something dumb (had the database dump uncompressed on the same drive as /var/lib/mysql); I need to clean up the problem, and import the rest of the data. Good news is that it's not too difficult to do; bad news is that it will take some time to get everything moved around again.
Update 09:50 AM PST: Looks like I'm getting the last of the data imported (needed to move hundreds of gigs of data from one server to another, munge it with Perl, then move part of it back again). If things go well, we should see identi.ca back up within a few hours. DNS is still propagating for the new data centre, though; some people might see the old "server down" message for a while.
Update 17:00 PST: We've been back up for a few hours. DNS is still propagating; it may take a while for things to get out. Those of you who are comfy editing /etc/hosts may want to map "identi.ca" to 188.8.131.52 there. Otherwise, things will gradually come online soon.