Big Data // Big Data Analytics
News
2/27/2014
09:26 AM
Connect Directly
LinkedIn
Twitter
Google+
RSS
E-Mail
50%
50%

DataStax Brings In-Memory To NoSQL

Apache Cassandra vendor DataStax joins Microsoft, Oracle, and others adding in-memory features to their database management systems.

16 Top Big Data Analytics Platforms
16 Top Big Data Analytics Platforms
(Click image for larger view and slideshow.)

Web and mobile applications are getting bigger and people are as impatient as ever. These are two factors hastening the use of in-memory technology, and DataStax on Wednesday became the latest database management system (DBMS) vendor to add in-memory processing capabilities.

DataStax Enterprise is a highly scalable DBMS based on open source Apache Cassandra. Its strengths are flexible NoSQL data modeling, multi-data-center support, and linear scalability on clustered commodity hardware. Customers like eBay, Netflix, and others typically run globally distributed deployments at massive scale.

With the DataStax Enterprise 4.0 release announced on Wednesday, the vendor is adding an in-memory option whereby developers can move new or existing database tables into memory to ensure ultra-fast performance. The move comes in response to growing numbers of DataStax customers who have been deploying in-memory products such as Memcached or Redis alongside Cassandra in order to handle low-latency processing needs.

[Want more on recent in-memory moves? Read VoltDB Steps Up In-Memory Analytics.]

"Instead of having to use two different databases, customers tell us they'd like to have it all under one umbrella," said Robin Schumacher, DataStax's VP of products, in an interview with InformationWeek.

Use cases for the new feature include scenarios in which semi-static data experience frequent overwrites. Examples include sites or apps with top-10 or top-20 lists that are constantly updated, online games with active leader boards, online gambling sites, or online shopping sites with active "like," "want," and "own" listings.

DataStax is following in familiar footsteps, as lots of DBMS vendors are adding in-memory features. Microsoft, for example, has extensively previewed an In-Memory OLTP option (formerly project Hekaton) that will be included in soon-to-be-launched Microsoft SQL Server 2014. And Oracle has announced that it, too, will add an in-memory option for its flagship 12c database. General release of that option isn't expected until early next year.

The NoSQL realm already has in-memory DBMS options such as Aerospike, which is heavily used in online advertising. But Shumacher said DataStax tends to show up in much higher-scale deployments than Aerospike.

In-memory DBMS vendors MemSQL and VoltDB are taking the trend in the other direction, recently adding flash- and disk-based storage options to products that previously did all their processing entirely in memory. The goal here is to add capacity for historical data for long-term analysis. As in the DataStax case, the idea is to covering a broader range of needs with one product.

DataStax's in-memory feature is supported by new management features introduced in OpsCenter 4.1, which was also introduced on Wednesday. This visual monitoring and management console for DataStax Enterprise lets you track and forecast database table size and memory usage over time. Bad things can happen when in-memory tables or DBMSs run out of memory, so OpsCenter lets you set limits and alerts on memory usage. It's also used to specify whether tables are assigned to spinning disks for normal demands, solid state disks for lower latency, or all-in-RAM for fastest-possible retrieval and processing speeds.

Last year Teradata introduced an Intelligent Memory feature that does all this shifting of data to the most appropriate storage speed automatically based on workload demands. You can expect this stort of automation to show up in the next wave of in-memory enhancements.

Engage with Oracle president Mark Hurd, NFL CIO Michelle McKenna-Doyle, General Motors CIO Randy Mott, Box founder Aaron Levie, UPMC CIO Dan Drawbaugh, GE Power CIO Jim Fowler, and other leaders of the Digital Business movement at the InformationWeek Conference and Elite 100 Awards Ceremony, to be held in conjunction with Interop in Las Vegas, March 31 to April 1, 2014. See the full agenda here.

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
MonicaP771
50%
50%
MonicaP771,
User Rank: Apprentice
3/2/2014 | 11:06:50 AM
Re: In-memory DBMS vs. in-memory feature
How fast do you need to go? In the "Impatience Economy" people want everything right now. To create the richest, most personalized and monetizable experience, Apps want to understand context - what's going on right now. With a faster database, Apps can process more data, make better decisions and deliver what the user wants. 

Low latency translates into richer customer experiences, higher throughput with fewer servers and price/performance numbers that enable Apps that could not be imagined and business models that were not possible before. 

This applies to 2 node clusters at e-commerce sites SnapDeal as well as enterprise digital marketing hubs like [X+1] that manage billions of records, each with 5,000 -10,000 attributes and respond with 10 recommendations within 50 milliseconds.
anon7628224144
100%
0%
anon7628224144,
User Rank: Apprentice
2/27/2014 | 5:12:00 PM
Scale comparison between Aerospike and Cassandra

Hi Doug,

 

There seems to be an inaccuracy in the article comparing scale between Aerospike and Cassandra. Scale comes in many ways. One is scaling to handle higher data sizes - DataStax/Cassandra may handle higher DATA VOLUMES that I am not aware of but that would not apply to the in-memory version, definitely. However, if scale is about transaction THROUGHPUT at LOW LATENCY, Aerospike is far and away the market leader.

 

What could be higher scale than real-time platforms like AppNexus, BlueKai -  the third largest DMP that Oracle just acquired -  eXelate, The Trade Desk, Chango and others that are managing terabytes of data at over 1 Million TPS at steady state (>30% writes), with predictable response times 99% in 3-5ms? All these are Aerospike customers.

 

Also, In-Memory has existed in NoSQL for a long time. Aerospike (previously named Citrusleaf) has been in continuous production since early 2010 on various in-memory configurations (in addition to flash/ssd based configurations). I am sure other NoSQL products  have had in-memory also for much longer, like Redis, for example.

 

Srini V. Srinivasan

Founder, VP Engineering and Operations

Aerospike

D. Henschen
50%
50%
D. Henschen,
User Rank: Author
2/27/2014 | 1:12:26 PM
In-memory DBMS vs. in-memory feature
Gotta say, there's a difference between an in-memory database that operates all in memory (SAP Hana, Oracle TimesTen, Aerospike, MemSQL, VoltDB) and a conventional database with added in-memory processing options (DataStax new feature, Microsoft's coming (Hekaton/In-Memory OLTP) feature, and Oracle's announced but far-from-release In-Memory Option for Oracle 12c. With everything, including indexes, you can obviously run faster, but the question for customers is how fast do you need to go? RAM is more expensive than disk, and the point of these added features and options is that you don't have to rip and replace the old database.
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Government Oct. 20, 2014
Energy and weather agencies are busting long-held barriers to analyzing big data. Can the feds now get other government agencies into the movement?
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
A roundup of the top stories and trends on InformationWeek.com
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.