Mystery surrounds outage at Amazon’s Dublin data centre

10 Aug 2011

A dispute has emerged over the events surrounding a service outage at Amazon’s data centre in Dublin last weekend.

In the hours following the incident, Amazon originally blamed a lightning strike and an explosion which knocked out a generator leading to loss of power, which disrupted service for Amazon customers for up to 48 hours in some cases.

Electricity provider ESB Networks has provided a different version of events. A spokesperson told that the problem was due to a fault in one of its substations at Citywest and that power would have been available from an alternate source within a millisecond.

ESB Networks said an outage occurred at 6.15pm on Sunday which lasted less than a second. “In relation to Amazon, it would have experienced less than a second (of downtime) when the centre would have been switched to an alternate supply,” the spokesperson said. The cause was a fault in a substation in Citywest and there was no report of an explosion or fire.

Moreover, the Met Office has no record of a lightning strike in the Dublin area at the time when Amazon claims its centre was taken offline. The wording of Amazon’s initial statement clearly implies cause and effect between the lightning strike and an explosion which prevented its backup generators from providing power which they ordinarily should have done.

Amazon did not return emails seeking comment. spoke to a telecoms firm which has networking equipment at two data centres in Citywest, and it said it was unaffected by any outage in either facility. However, a person who was present at the time said there was a power outage in Citywest which even affected the Luas light-rail system. Generators belonging to the Telecity data centre – which is located next to Amazon’s site – were running for 45 minutes and took care of the outage with no loss of service.

News of the outage travelled around the world and was widely reported on various data centre trade websites. It’s not clear yet whether the news could negatively impact efforts to establish Ireland as a hub for cloud computing activity, as there is a concentration of data centres building up around Dublin, and the Government has backed it to be one of the pillars of economic recovery.

UPDATE: It now appears the source of Amazon’s first statement to customers may have been an initial assessment from ESB staff on site, who speculated that damage to the transformer might be consistent with a lightning strike.

In a further statement issued on Wednesday, 10 August, ESB said it was now ruling out lightning as the cause of the outage. “In this case, the problem was the failure of a 110kV transformer in the Citywest 110kV substation.” The exact cause is still under investigation, the power company added.

Gordon Smith was a contributor to Silicon Republic