MemSQL Heats Up In-Memory Competition

MemSQL has made a name for itself as a highly scalable, in-memory transactional database management system, but on Thursday a new release was announced that will add a columnar engine and expanded storage options to better address historical analysis.

MemSQL, the vendor behind the DBMS, says its 3.0 release, which is expected in the second quarter, will blend row-based processing and columnar-analysis capabilities supported by new Flash and disk storage options. The new features complement the RAM-intensive, distributed platform so it can take on deeper historical analysis.

"Many of our customers have been telling us they want to put more information in MemSQL, and the only way to do that without adding more [expensive] machines is to be able to spill over onto Flash or disk storage," Eric Frenkiel, co-founder and CEO of MemSQL, told InformationWeek in a phone interview. "By joining the row store with a columnar engine, we get a seamless combination of live data as well as historical data."

The announcement is just the latest in-memory move emanating from vendors as diverse as SAP with its Hana platform to Teradata with its Intelligent Memory capabilities to MicroStrategy with last week's PRIME announcement to head-on MemSQL competitor VoltDB, which last week announced improvements in analytical query performance to complement its real-time, high-scale transactional capabilities.

[Want more on MemSQL's closest competitor? Read VoltDB Steps Up In-Memory Analytics.]

MemSQL's columnar engine can draw data from RAM, Flash, or disk, depending on the depth of history required, and as with any columnar engine, high compression greatly increases the storage capacity. Online advertising customer CPXi, for example, is beta testing the new feature to trigger ad delivery based on fast analysis of historical clicks and impressions across millions of users.

"The new capabilities in MemSQL v3.0 allow CPXi to shape a real-time response," said Gil Resh, senior VP of product and technology at CPXi, in a statement. "We measure online engagement over time and capture vast amounts of data -- more than 250 billion records with hundreds of different dimensions --to make millisecond decisions about which ads to show, even while the Web page is loading."

CPXi's clickstream data is vast, so even with MemSQL's new feature, it will only store about 90 days' worth of history. To this point, MemSQL is not trying to displace longer-term data stores. Frenkiel points to customers such as Comcast, which uses MemSQL alongside Hadoop, and Zynga, which uses MemSQL in combination with HP Vertica. With the option to retain more historical information, customers will be able to exploit MemSQL's in-memory strengths across deeper -- but still relatively short-term -- histories.

"Zynga is storing weeks' worth of data in MemSQL so they can do recent analysis, and a query that would take 15 minutes in HP Vertica takes three seconds on our platform," Frenkiel said.

Watch this space, as analytical DBMS vendors like HP Vertica are following Teradata route of making use of RAM and Flash as well as disk. But these analytical stores are not designed to run transactional applications, as is MemSQL. SAP Hana is a closer competitor, technology-wise, in that it addresses transactional and analytical uses, but MemSQL, VoltDB, and the other so-called NewSQL vendors seem to appeal to entirely different, Internet-centric customer base.

Too many companies treat digital and mobile strategies as pet projects. Here are four ideas to shake up your company. Also in the Digital Disruption issue of InformationWeek: Six enduring truths about selecting enterprise software. (Free registration required.)