Instagram power outage, June 30, 2012

UPDATE 01: At 3:15 pm EDT, it looks like services are back up with only spotty glitches. I’ve got the most recent Amazon Cloud Services report at the end of this post.

Instagram, the popular social photo network, is still mostly down today, hours after violent storms last night in Virginia knocked out its cloud server. The image above is a screenshot of the @Instagram Twitter feed taken on June 30, 2012 at 1:30 PM Eastern Time. According to the feed, it looks like Instagram has been down or spotty for over 14 hours now. According to a recent post on Mashable, Netflix Online is also effected.

More after the jump. >>>

The latest, according to Amazon Web Services Services Heath Dashboard is that Amazon Elastic Compute Cloud (N. Virginia) is still experiencing power issues, but is in the process of coming back online.

“As a result of the power outage, some EBS volumes may have inconsistent data. As we bring volumes back online, any affected volumes will have their status in the “Status Checks” column in the Volume list in the AWS console listed as “Impaired.” If your instances or volumes are not available, please login to the AWS Management Console and perform the following steps: “

I just checked Instagram a few minutes ago. I am able to refresh my feed occasionally, but I get plenty of “Couldn’t refresh feed” errors. Subsequent refreshes show that the service is coming back online more reliably. Instagram says that no images or data were lost.

Due to record-breaking heat on the east coast, power outages in the D.C. area are expected to last for days. A quick look at Weather shows that extreme heat and thunderstorms in the D.C. area is expected to continue throughout the week.

Mashable has more coverage on this story.

UPDATE 01: Although AWS still lists this cloud with “power issues,” it looks like most of the services are back up with onluy spotty glitches for some customers. That’s what they say. Your mileage may vary.

Jun 30, 10:25 AM PDT The majority of affected EC2 instances with no impaired EBS volumes attached have been recovered. Instances with an impaired EBS volume attached may still be unavailable. Creation of EBS recovery volumes is complete for the vast majority of recoverable volumes. For instances in this state follow the steps outlined in our last update. There remain a small subset of EBS volumes that are currently stuck in the affected Availability Zone. ELB load balancers are provisioning successfully, but some may be delayed while we process our backlog.

Jun 30, 11:42 AM PDT We are continuing to work on processing our provisioning backlog for ELB load balancers. We are also continuing to work on restoring IO for the remaining small number of stuck EBS volumes. Customer action is required for EBS volumes that do not have IO currently enabled– if you have not already chosen to Enable Volume IO, outlined in the instructions above, please follow those steps to re-enable IO on your EBS volumes.

=M=

~~~~

Thanks to Stacy Anderson for the tip on this one. =M=