Big Data // Big Data Analytics
News
8/21/2014
09:06 AM
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Candy Crush King Taps Exasol For In-Memory Analysis

Candy Crush publisher King is using Exasol's massively scalable database to strike the right balance between challenging play and a fun gaming experience.

Hadoop Jobs: 9 Ways To Get Hired
Hadoop Jobs: 9 Ways To Get Hired
(Click image for larger view and slideshow.)

Are you one of the millions of users addicted to King Digital Entertainment's Candy Crush? The game saw huge growth when it hit Facebook in 2012, but usage really exploded as the game took off on mobile devices last year. The challenge for King has been dealing with all that data and making sense of the user experience.

The London company stores its raw data on a Cloudera-based Hadoop cluster, so scalability wasn't a problem. "But we were missing exploratory query capabilities," said Andy Done, data platform lead at King. "Hive is great for crunching through huge volumes of data, but when you want to explore and get a feel for your data, it's not responsive enough."

King realized that much of its data was reasonably well structured. Its first attempt to fill the data-analysis gap was to add the InfiniDB database alongside Hadoop. Moving subsets of data to this database opened up plenty of SQL analysis capabilities, but within a matter of months database performance lagged as workloads scaled toward 100 terabytes, according to Done. By this time, King's Hadoop cluster had surpassed 1 petabyte.

[Want more on big SQL analysis? Read Oracle Joins SQL-On-Big-Data Bandwagon.]

In mid-2013, King went back to the market and came across Exasol, a German database management system vendor. Though little known in the US, Exasol has more than 300 customers in Europe, and it's 10-year-old database was among the earliest to embrace technologies such as columnar compression, massively parallel processing, and in-memory analysis capabilities. (Exasol opened an office in San Francisco early this year with plans to build North American sales.)

After a successful proof-of-concept project, King brought Exasol into production about a year ago, and it now moves its hottest, most valuable data into this database, which is scaled to handle nearly 100 terabytes.

The routine applications for Exasol include executive-level reporting and dashboards, but the primary customer-facing analysis goal is analyzing the gaming experience. King studies where players might be breezing through the game and getting bored and where they're getting stuck and having a frustratingly hard experience that might lead them to give up.

"We try to ensure that there's a balance between being challenging and being fun, but one of the things we found was that level 65 in Candy Crush was notoriously difficult," says Done. "We changed the game accordingly to make it slightly less taxing at that level."

King also uses Exasol to study differences between play, customers, and the gaming experience online, on Facebook, and on mobile devices. This gets back to the success of the business and customer habits in playing the game on multiple platforms.

In the year since King brought Exasol into production, a profusion of SQL-on-Hadoop options have emerged. Most of these options would allow King to conduct analysis directly on top of Hadoop, but Done said the company is content to use the parallel connections available between Exasol and Hadoop.

"We've looked at some of the SQL-on-Hadoop offerings, but in our assessment, they're not mature enough to meet our use case as yet," he said. Cloudera Impala was one of the SQL-on-Hadoop candidates that King considered, but it was crossed off the list due to memory and insert-and-merge limitations.

Though in-memory analysis is one of Exasol's attractions, Done said RAM is only 5% of the total storage capacity of King's database deployment. "We get exceptional performance with the hot data that's in RAM, but the rest of the data that's on disk is also readily accessible, and it's not a huge restriction on query speeds."

King is content for now, but as the company has previously discovered, workloads change and grow, so its approach is "to remain as flexible as possible," said Done. "A huge amount of innovation is taking place in this space, and that arms race is hugely beneficial for us, because we'll all benefit from better technology for helping us with our data problems."

In its ninth year, Interop New York (Sept. 29 to Oct. 3) is the premier event for the Northeast IT market. Strongly represented vertical industries include financial services, government, and education. Join more than 5,000 attendees to learn about IT leadership, cloud, collaboration, infrastructure, mobility, risk management and security, and SDN, as well as explore 125 exhibitors' offerings. Register with Discount Code MPIWK to save $200 off Total Access & Conference Passes.

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Angelfuego
50%
50%
Angelfuego,
User Rank: Moderator
8/21/2014 | 7:38:58 PM
Re: SQL-On-Hadoop not ready for primetime?
@Douglas, I am glad they are rethinking about making Level 65 a little less taxing. In my opinion, it was frustrating, nail biting, and aggravating. It was not balanced with fun and it exceeded the adrenaline rush of a challenge.
Angelfuego
50%
50%
Angelfuego,
User Rank: Moderator
8/21/2014 | 7:33:57 PM
Re: Game Data
@Michael Endler, Dunk Hunt! How nostalgic! I used to love that game. I forgot all about it until I read your comment.
Michael Endler
50%
50%
Michael Endler,
User Rank: Author
8/21/2014 | 6:37:29 PM
Game Data
Interesting article, Doug. I've heard something similar from Microsoft, that because Xbox is tied into Microsoft's cloud, they can gather data about gameplay, and potentially create games that adapt in real time to find the right balance between challenge and fun. When I first played video games, Duck Hunt on Nintendo was very popular, so the new directions - some of them data-centric - that interactive entertainment has taken are pretty mind-blowing.
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
8/21/2014 | 10:40:27 AM
SQL-On-Hadoop not ready for primetime?
Moving data from Hadoop to a separate database does present a bit of a burden, says Andy Done of King, but it could not perform required analyses with the SQL-on-Hadoop options available when it chose Exasol last year. Since that time Actian, HP (Vertica), InfiniDB and others have ported their databases to run on top of Hadoop, but Done didn't seem interested in changing out King's current, two-platform approach for an alternative just yet.
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest, Nov. 10, 2014
Just 30% of respondents to our new survey say their companies are very or extremely effective at identifying critical data and analyzing it to make decisions, down from 42% in 2013. What gives?
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 16, 2014.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.