By backing the Open Compute Project, Facebook can leverage community knowledge to improve its data center infrastructure while lowering data center costs.
6. Hadoop
Hadoop is an open source framework for running distributed applications. Derived from Google technologies and initially developed by Yahoo engineer Doug Cutting, Hadoop has become critical infrastructure for a number of large technology and media companies like Amazon, Facebook, and Yahoo. Facebook claims to run the largest Hadoop cluster in the world, pegged at 30 petabytes in March 2011.
7. The LAMP Stack
Facebook was built with Linux, Apache, MySQL, and PHP, among other technologies. Like Google, Facebook opted for open source technologies to control its destiny and to minimize costs. The open source memory caching system known as Memcached should probably be included here too, even if its function is to augment the performance of Facebook's LAMP systems.
8. Scuba
Facebook has released a lot of its technology as open source. One project it hasn't made available is Scuba, a system for doing real-time, ad-hoc analysis of arbitrary datasets. Scuba was developed because traditional approaches to querying MySQL databases were too slow at scale. When you have as much data as Facebook, performance at scale matters.
9. HipHop For PHP
PHP doesn't perform that well at scale. HipHop for PHP, open sourced in 2010, transforms slower PHP code into optimized C++. Using HipHip for PHP, Facebook reports that its code requires about 50% less CPU usage and that its API infrastructure can service twice as much traffic with about a third less CPU usage.
10. Scribe And Thift
Scribe is an open source framework for collecting log data, originally designed to interface with Facebook's servers. It's built atop Thrift, a system for creating and coordinating software-based services in multiple programming languages. Using Scribe and Thrift, Facebook can log billions of system messages daily using a diverse set of modules written in PHP, Java, Python, or C++ code.
11. Phabricator
Released last summer as open source, Phabricator is a suite of Web applications for creating and managing software projects. It includes tools for managing workflow, bug tracking, and communications management. The Phabricator website offers this gem of a description: "Facebook engineers rave about Phabricator, describing it with glowing terms like 'okay' and 'mandatory.'"
The Enterprise 2.0 Conference brings together industry thought leaders to explore the latest innovations in enterprise social software, analytics, and big data tools and technologies. Learn how your business can harness these tools to improve internal business processes and create operational efficiencies. It happens in Boston, June 18-21. Register today!
