Friday, August 23, 2013

Power Outage

The power went out in SE Lawrence today taking power down for the primary Point of Presence (POP) where data comes into Lawrence.

As planned, the site went on batteries for an hour.  When the hour was up, we put the site on its generator.  During the switch to generator power, the site "blipped" and all of the switches rebooted.

Unfortunately, since I am an idiot and forgot to save the configuration after we upgraded capacity on Wednesday, the switch came back up in its old configuration.  Dan was quick to get the new IPs and routes back into it (remembering to "write memory" when he was done), but it still took the entire network down for 10 min.

My apologies for that mistake.  We've corrected it and have the generator and UPS configured to prevent this type of glitch in the future.