Big Data // Software Platforms
News
3/27/2014
11:06 AM
Connect Directly
LinkedIn
Twitter
Google+
RSS
E-Mail
50%
50%

Facebook Debuts Web-Scale Variant Of MySQL

Facebook, Google, LinkedIn, and Twitter share expertise to launch WebScaleSQL, a super-DBMS built on Oracle's MySQL Community Edition.

In a move that could shake up both the commercial and open source database management system markets, Facebook announced on Thursday that it has worked with fellow Internet giants Google, LinkedIn, and Twitter to develop WebScaleSQL, an open source, web-scale branch on top of Oracle's publically available MySQL Community Edition.

"Our goal in launching WebScaleSQL is to enable the scale-oriented members of the MySQL community to work more closely together in order to prioritize the aspects that are most important to us," wrote Facebook software engineer Steaphan Greene in a draft blog post shared with InformationWeek. Greene explained that the group wants to "help companies leverage the great features already found in MySQL 5.6, while building and adding more features that are specific to deployments in large-scale environments."

[Want more on Facebook's contributions to high-scale hardware? Read Open-Source Hardware: Prepare For Disruption.]

Over the last few months, the collaborators have built a community structure for sharing and reviewing WebScaleSQL contributions, and engineers from all four companies have contributed code and provided feedback to each other "to develop a new, more unified, and more collaborative branch of MySQL," Greene wrote.

MySQL is the most popular open source database management system (DBMS) in the world, accounting for one quarter of all multi-client DBMS deployments, according Evans Data Research. In choosing to build on MySQL, the WebScaleSQL community has opened up an avenue for hundreds of thousands of MySQL developers to grow their deployments to unprecedented scale.

The move also poses a threat to certain NoSQL and NewSQL DBMS upstarts that have gained many of their customers among organizations frustrated by the difficulty of managing MySQL at high scale. According to Greene, the WebScaleSQL community has already developed components, including:

  • Features that make operating at Web scale easier, such as a super read-only feature and the ability to specify sub-second client timeouts.
  • Changes to improve the performance of WebScaleSQL, including buffer pool flushing improvements, optimizations to certain query types, and support for Non-Uniform Memory Architecture (NUMA) interleave policy.
  • An automated framework that will run and publish the results of MySQL's built-in test system for each proposed change.
  • A suite of stress tests and a prototype automated performance testing system.

WebScaleSQL is a set of patches on top of Oracle's publicly available MySQL Community release. The plan is to follow changes to MySQL Community Edition to ensure the latest features are available in WebScaleSQL. Like MySQL, WebScaleSQL is available under GPL licensing.

Conspicuously absent from WebScaleSQL's announcement was any mention of Oracle, which owns MySQL and offers Enterprise and Cluster editions of MySQL aimed at high-scale needs. WebScaleSQL is building on publically available Community Edition software. "As long as the MySQL community releases continue, we are committed to remaining a branch -- and not a fork -- of MySQL," a spokesperson told InformationWeek in an email interview.

WebScaleSQL will welcome other scale-oriented members of the MySQL community to contribute, according to Greene. The group is currently working on:

  • Developing an asynchronous MySQL client that won't have to wait to connect, send, or retrieve while querying. This non-blocking client developed and in production use at Facebook is being code-reviewed by the other WebScaleSQL teams.
  • Adding Facebook production-tested compression and versions of table, user, and compression statistics.
  • Adding Facebook's Logical Read-Ahead mechanism for "up to 10x" speed improvements in full table scans, such as nightly logical backups.

The introduction of WebScaleSQL is not an immediate threat to commercial or open source products in the enterprise market. For starters, Facebook, Google, LinkedIn, and Twitter are all self-supported IT shops, whereas smaller enterprises would want a commercial support provider. It's also unclear where "Web-scale" begins and whether MySQL Enterprise and MySQL Cluster Carrier Grade Edition would be more practical and cost-effective products for current customers looking to scale up. Commercial database vendors including IBM, Microsoft, and Oracle, meanwhile, might learn new tricks on how to scale up their own enterprise-oriented database products.

As for the NoSQL, and particularly the NewSQL, crowd pursuing Web-scale deployments, WebScaleSQL will undoubtedly force them to distinguish their products beyond the baseline promise of offering performance at big data scale.

Improved platforms, standards, bandwidth rates, and data models mean more IT shops are taking a "cloud-first" approach to new services, keeping only select jobs in house. But what's often left out of the calculus is the impact on the end user when you outsource most services. Register for this InformationWeek editorial webinar and learn how to avoid the pitfalls of outsourcing IT. The Performance From The User's Perspective webinar happens March 28. Registration is free.

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Somedude8
50%
50%
Somedude8,
User Rank: Ninja
3/27/2014 | 2:04:08 PM
Go Get 'em Oracle!
I am wondering if Oracle is going to use their pseudo-control of MySQL to put the kabosh on what could easily be seen as a product that competes with their own for-pay products.
David F. Carr
50%
50%
David F. Carr,
User Rank: Author
3/27/2014 | 11:44:11 AM
WebScaleSQL for WordPress-based publications
I'd think this would be very interesting to the operators of large scale web-publications based on WordPress (which uses MySQL as its standard back end DB), including sites like Forbes.com and WordPress.com. The company behind WordPress.com and the main contributor to the open source sw releases, Automattic, isn't on the list of collaborarators -- and isn't in quite the same league as Facebook, Google, LinkedIn, and Twitter -- but could take advantage of their efforts and contribute to them over time. Presumably, they've come up with their own performance hacks over the years for very large scale sites.
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
3/27/2014 | 11:42:19 AM
Kudos to Facebook, Google, LinkedIn, and Twitter!
This is a display of open source power at work, with Facebook, Google, LinkedIn, and Twitter sharing thier MySQL expertise among themselves and now with the rest of the world through WebScaleSQL. It's clearly still early days, but this is going to shake things up just as Facebook's Open Compute efforts are changing the status quo. This will also force NoSQL and, particularly, NewSQL players to further differentiate their products. The focus thus far seems mostly focused on high-scale transactional processing, and a Facebook spokesperson noted: "Our data analytics warehouse is in Hadoop, but our core data store is in MySQL."

I'm reaching out to NoSQL/NewSQL vendors for comments, but feel free to add them here!
In A Fever For Big Data
In A Fever For Big Data
Healthcare orgs are relentlessly accumulating data, and a growing array of tools are becoming available to manage it.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest, Nov. 10, 2014
Just 30% of respondents to our new survey say their companies are very or extremely effective at identifying critical data and analyzing it to make decisions, down from 42% in 2013. What gives?
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 16, 2014.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.