Server downtime

Author	Message
TJM Project administrator Project developer Project scientist Send message Joined: 25 Aug 07 Posts: 843 Credit: 267,994,998 RAC: 0	Message 1532 - Posted: 9 Mar 2010, 10:25:45 UTC Last modified: 9 Mar 2010, 10:28:48 UTC During this weekend the server will be taken down for O/S upgrade. The old distro (Debian 4) will be replaced by Debian 5 amd64, since the older version has a limited support and probably slows down the hardware a bit. It's hard to estimate how long the upgrade process will take, but I'd say that 8-12 hours minimum, because I have to do a complete backup first, so in the worst case I'll be able to rollback everything to the previous state. M4 Project homepage M4 Project wiki ID: 1532 · Rating: 0 · rate: / Reply Quote

Tom_unoduetre Send message Joined: 18 Sep 07 Posts: 14 Credit: 2,149,222 RAC: 0	Message 1533 - Posted: 9 Mar 2010, 16:41:36 UTC Good luck and hopefully all goes well. ID: 1533 · Rating: 0 · rate: / Reply Quote

thinking_goose Send message Joined: 12 Nov 07 Posts: 119 Credit: 2,750,621 RAC: 0	Message 1534 - Posted: 9 Mar 2010, 18:04:50 UTC I hope everything goes well. I've upped the work unit cache to a whole day on a couple of machines that I expect to be running through the weekend. ID: 1534 · Rating: 0 · rate: / Reply Quote

TJM Project administrator Project developer Project scientist Send message Joined: 25 Aug 07 Posts: 843 Credit: 267,994,998 RAC: 0	Message 1535 - Posted: 14 Mar 2010, 18:15:04 UTC Last modified: 14 Mar 2010, 20:10:44 UTC So far everything looks fine. The upgrade process took longer than I expected, mainly due to lots of minor configuration differences between etch and lenny. Also, I had to rebuild the server software, so I decided to upgrade all the way to the latest trunk. As a result, the server runs on revision 20902 which is the latest one and might be unstable. So far only one bug has been found and it's just a minor glitch on the main site - missing character encoding. EDIT: There was a bug in work generator and it crashed soon after I restarted the server. I didn't notice but someone left me a message at google chat - thanks. M4 Project homepage M4 Project wiki ID: 1535 · Rating: 0 · rate: / Reply Quote

TJM Project administrator Project developer Project scientist Send message Joined: 25 Aug 07 Posts: 843 Credit: 267,994,998 RAC: 0	Message 1595 - Posted: 7 May 2010, 18:16:46 UTC Last modified: 7 May 2010, 18:18:14 UTC 07.05.2010 The server is going down for database cleanup, extended backup, software upgrade and maintenance. Later today new ciphertext will be added to the server's queue. M4 Project homepage M4 Project wiki ID: 1595 · Rating: 0 · rate: / Reply Quote

TJM Project administrator Project developer Project scientist Send message Joined: 25 Aug 07 Posts: 843 Credit: 267,994,998 RAC: 0	Message 1598 - Posted: 10 May 2010, 23:26:58 UTC 11.05.2010 The server is back up after a critical database failure. I'm not sure what exactly happened, the mysql daemon just dissapeared and couldn't be restarted because one of the ib_logdata files got damaged. I tried to avoid restoring the DB from backup, instead I used the realtime copy provided by replication slave. Unfortunately the slave is always few minutes behind the main server, so probably there was some (hopefully minor) data loss. Right now everything seems to be running, but if you think that something is wrong with the database, please let me know immediately. M4 Project homepage M4 Project wiki ID: 1598 · Rating: 0 · rate: / Reply Quote