Big Data // Big Data Analytics
11:45 AM
Connect Directly
Repost This

MemSQL Heats Up In-Memory Competition

Adding historical analysis to its bag of tricks, MemSQL takes on real-time contenders from SAP Hana to VoltDB.

MemSQL has made a name for itself as a highly scalable, in-memory transactional database management system, but on Thursday a new release was announced that will add a columnar engine and expanded storage options to better address historical analysis.

MemSQL, the vendor behind the DBMS, says its 3.0 release, which is expected in the second quarter, will blend row-based processing and columnar-analysis capabilities supported by new Flash and disk storage options. The new features complement the RAM-intensive, distributed platform so it can take on deeper historical analysis.

"Many of our customers have been telling us they want to put more information in MemSQL, and the only way to do that without adding more [expensive] machines is to be able to spill over onto Flash or disk storage," Eric Frenkiel, co-founder and CEO of MemSQL, told InformationWeek in a phone interview. "By joining the row store with a columnar engine, we get a seamless combination of live data as well as historical data."

The announcement is just the latest in-memory move emanating from vendors as diverse as SAP with its Hana platform to Teradata with its Intelligent Memory capabilities to MicroStrategy with last week's PRIME announcement to head-on MemSQL competitor VoltDB, which last week announced improvements in analytical query performance to complement its real-time, high-scale transactional capabilities.

[Want more on MemSQL's closest competitor? Read VoltDB Steps Up In-Memory Analytics.]

MemSQL's columnar engine can draw data from RAM, Flash, or disk, depending on the depth of history required, and as with any columnar engine, high compression greatly increases the storage capacity. Online advertising customer CPXi, for example, is beta testing the new feature to trigger ad delivery based on fast analysis of historical clicks and impressions across millions of users.

"The new capabilities in MemSQL v3.0 allow CPXi to shape a real-time response," said Gil Resh, senior VP of product and technology at CPXi, in a statement. "We measure online engagement over time and capture vast amounts of data -- more than 250 billion records with hundreds of different dimensions --to make millisecond decisions about which ads to show, even while the Web page is loading."

CPXi's clickstream data is vast, so even with MemSQL's new feature, it will only store about 90 days' worth of history. To this point, MemSQL is not trying to displace longer-term data stores. Frenkiel points to customers such as Comcast, which uses MemSQL alongside Hadoop, and Zynga, which uses MemSQL in combination with HP Vertica. With the option to retain more historical information, customers will be able to exploit MemSQL's in-memory strengths across deeper -- but still relatively short-term -- histories.

"Zynga is storing weeks' worth of data in MemSQL so they can do recent analysis, and a query that would take 15 minutes in HP Vertica takes three seconds on our platform," Frenkiel said.

Watch this space, as analytical DBMS vendors like HP Vertica are following Teradata route of making use of RAM and Flash as well as disk. But these analytical stores are not designed to run transactional applications, as is MemSQL. SAP Hana is a closer competitor, technology-wise, in that it addresses transactional and analytical uses, but MemSQL, VoltDB, and the other so-called NewSQL vendors seem to appeal to entirely different, Internet-centric customer base.

Too many companies treat digital and mobile strategies as pet projects. Here are four ideas to shake up your company. Also in the Digital Disruption issue of InformationWeek: Six enduring truths about selecting enterprise software. (Free registration required.)

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
D. Henschen
D. Henschen,
User Rank: Author
2/6/2014 | 4:03:55 PM
The poster child versus the real competition
Another competing vendor here might be Aerospike (formerly Citrus Leaf), but that's a NoSQL database that's used mostly in the online advertising world. Eevery vendor with in-memory technology inevitabaly mentions SAP Hana, but I don't see these vendors going after the same sorts of customers.
InformationWeek Elite 100
InformationWeek Elite 100
Our data shows these innovators using digital technology in two key areas: providing better products and cutting costs. Almost half of them expect to introduce a new IT-led product this year, and 46% are using technology to make business processes more efficient.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.