Outage Update 10/24/2008

Posted by Blue Box Group on Fri Oct 24 09:20:00 UTC 2008

Today’s time line: * 9:21am: New Parts On The Way * 3:50pm: Another defect found * 5:09pm: UPS 2 functional * 9:00pm: “Go” for transition to utility power * 9:58pm: Go Time * 10:09pm: We’re back on UPS Power!


UPDATE 1 - 9:21am: We continue to run on generator power this morning after our facility was unable to repair UPS2 last night. Our facility took a fuel delivery yesterday and has enough fuel to run on generator until Monday. Additional fuel deliveries have been scheduled.

After further study of the problems with UPS2 from last evening, MGE has decided to fly in few more repair parts from California. Those should arrive this morning on a commercial flight. Once they’re here, they will be installed and UPS2 will under go the same level of testing UPS1 went under yesterday. Assuming that goes well, the planned transfer will occur tonight after 12am. We will keep you posted throughout the day as we find out more.


UPDATE 2 - 3:50pm: A small defect has been found in the static switch used to transfer the load. This switch is being replaced. Also, today’s refuelling has occurred– there is now enough fuel on site to get us through Tuesday on generator power.


UPDATE 3 - 5:09pm: The repairs to UPS 2 are complete and it has passed load transfer tests. Technicians are now in the process of testing UPS 1 and UPS 2 together to make sure these fail over together correctly under 100% load. The replacement of the static switch defective part will happen after 7:00pm this evening, and this is a very simple replacement. At this time the cut-over back to utility power has been tentatively scheduled for 10:00pm this evening. Again, we’ll post more as we know more.


UPDATE 4 - 9:00pm: The parts for the static switch have arrived and are being installed. We should be good to go for the transition back to utility power in one hour.


UPDATE 5 - 10:00pm: Our staff is in our data center and our facility is waiting on the arrival of a few other people an they will make the cut over within the next few minutes. We will post here when it’s complete. They have done a very great job with research on the cause of the failure which we will pass on in our full write up on Monday.


UPDATE 6 - 10:10pm: The transfer was a complete success and we are now running on UPS backed power. Thank you for your patience throughout this process. As promised, we will have a full documented write up of the entire outage available on Monday along with information on SLA credits.

  • Blue Box Group Tech Support

THE LATEST

THE ARCHIVES

THE FEED