Google Gmail Disruption Caused By Network Load, Code Update
Code changes, data center problems, and heavy usage conspired to make Gmail inaccessible for some last week.
The Gmail outage last week was the product of a Gmail code change that inadvertently increased the load placed on Google's Contacts service, in conjunction with heavy usage of the Contacts service and with a data center network issue.
Those findings were included in a report that Google published on Friday detailing what happened during the service disruption on Thursday morning.
"After an internal investigation and review, the Engineering team has determined that the root cause of the Contacts issue was a high load on the service," the report says.
Between 7:00 AM and 9:50 AM Pacific Time last Thursday, users of Google Apps were unable to access Contacts through the Gmail interface. The report says that during this period, most customers couldn't access Google Talk or add users to their Google Apps accounts.
Until about 8:30 AM that morning, an undisclosed number users could not access Gmail, except through an IMAP client. Google last week characterized those affected as "a small subset of users."
In a statement last week, Google apologized for the inconvenience and said that the slowness and service degradation lasted about an hour. The company said that Gmail was fully operations for everyone by about 8:00 AM on Thursday, and that Contacts functionality was restored shortly after that.
The modest apology appears to correspond with Google's assessment the disruption as relatively minor.
A 100-minute Gmail outage at the beginning of the month was characterized by Google engineering VP and "Site Reliability Czar" Ben Treynor as a "big deal" in his apology.
In early May, a more significant outage brought an apology from someone even higher in the Google hierarchy, SVP of operations Urs Hoelzle. That incident caused a 5% drop in Internet traffic, according to Arbor Networks, and affected 14% of Google users worldwide, according to Hoelzle.
InformationWeek has published an in-depth report on the current state of IT salaries. Download the report here (registration required).
SaaS As Innovation Driver?Software as a service is the clear No. 1 way enterprises consume cloud. InformationWeek's SaaS Innovation Survey reveals three tips to get the most from SaaS: Make it a popularity contest. Have an escape plan. And remember that identity is the new perimeter.