Infrastructure // Networking
12:40 PM
Connect Directly
Repost This

Facebook Malware Crackdown Caused App Outage

Facebook admits own scanning system disabled some third-party apps and vows to prevent problem from occurring again.

10 Facebook Features To Help You Get Ahead
10 Facebook Features To Help You Get Ahead
(click image for larger view)
Facebook revealed details on an incident earlier this week that caused some developer applications to accidentally go offline for several hours. The social network confirmed that on Tuesday, while running automated systems to identify and disable malicious apps, the tools inadvertently disabled some legitimate third-party apps. Facebook did not say how many apps were affected.

"Occasionally we detect an attack that requires us to augment those automated systems," said Eugene Zarakhovsky, software engineer at Facebook, in a blog post. "Specifically, we identify a malicious pattern, find all the apps that match the pattern, and then disable those apps. This normally results in thousands of malicious apps being disabled and improves our automated systems' ability to detect similar attacks in the future."

Facebook said that in this instance, it began with a broad detection pattern that correctly matched thousands of malicious apps. The problem was that it also identified and labeled legitimate apps as malicious. When it detected the error, Facebook said it stopped the process and worked to restore access. This took longer than expected because "of the number of apps and bugs related to the restoration of app metadata."

[ Facebook is only too happy to sort your news feed for you. Take control: read 5 Ways to Customize Facebook News Feed. ]

Developer platforms are a hotbed for malicious activity. In July, Apple said its developer portal was hacked, which put the personal details of 275,000 third-party developers at risk. Google has also struggled to keep its Google Play marketplace safe. Most recently, it came under fire after a study showed that 22% of its apps included adware. Although Facebook's incident did not threaten security or privacy, it was a nuisance for many.

App developers turned to a thread on Hacker News after discovering that their apps had suddenly been disabled. Facebook's developer advocate David Weekly replied to the thread, saying, "We have systems that block spammy apps that are 99.9% of the time really incredibly sophisticated and get a ~0% false positive rate. This is a case of the 0.1%."

To prevent this from happening again, Facebook says it plans to put two measures in place. The first is to "create better tools to detect overly broad patterns and put in place better processes to verify that all apps matched are indeed malicious." The second, Zarakhovsky wrote, will be to address the bugs and bottlenecks that made the recovery process slower than expected.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
8/19/2013 | 11:40:22 PM
re: Facebook Malware Crackdown Caused App Outage
Automated operations in some cases are going to contain their own problems, rather than allowing us to move smoothly into the software-defined data center. The larger the scale, as Amazon learned to its regret on Easter weekend 2011, the greater the mishap when an automated process goes awry. Unplugging one network and replugging the traffic into an alternative and under-capacity network set off "a re-mirroring storm." We need to get smarter about anticipating what can go wrong with automated systems and build in some protection logic. Much of this learning, however, will occur the hard way.
Thomas Claburn
Thomas Claburn,
User Rank: Author
8/19/2013 | 8:13:44 PM
re: Facebook Malware Crackdown Caused App Outage
Signature-based malware detection has been ineffective for a while now. It surprises me Facebook would rely on pattern matching given how easily the same malicious action can be created using different code.
David F. Carr
David F. Carr,
User Rank: Author
8/19/2013 | 12:50:25 PM
re: Facebook Malware Crackdown Caused App Outage
The 0.1% case is always going to happen eventually at this scale.
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.