Server outage 14/15th April

I don’t know the full details, but apparently a sever weather incident in Dallas Fort-Worth caused a power outage in the data centre that the weather-watch server is located in. During the switch to UPS a 2500A circuit breaker failed which killed the power supply to thousands of servers. It took 3 hours to get power back on, but when the server came back the databases were corrupt. I’ve just learned a lot more about MySQL in fixing the problem.

Hopefully the server is back up and running properly now, but if you spot anything odd, please let me know ASAP and I’ll try to fix it.

This outage would have affected all weather-watch.com services, i.e. Forum, MML data feed, WxSIM data feed, Wiki, Mantis, etc.

Latest suggestion on the cause of the outage is that the datacentre was still on generator power from last night when a strong line of severe thunderstorms and tornadoes blew through DFW. They were switching back to utility power when the breaker on the transfer switch blew out.