10 In-Memory Database Options Power Speedy Performance
From Altibase to VoltDB and covering options from IBM, Microsoft, Oracle, and SAP, we wrap up leading in-memory databases and add-on options. When you need speed, here are 10 tools to choose.
![](https://eu-images.contentstack.com/v3/assets/blt69509c9116440be8/blt8b22bac9913b356d/64cb56fe95fb5c9ff76666f5/In-memory-intro.jpg?width=700&auto=webp&quality=80&disable=upscale)
Yes, the death of the conventional disk drive has been greatly exaggerated, but Moore's Law has brought down the cost of RAM so dramatically that in-memory technology is getting to be downright pervasive.
With its Hana platform, SAP has been the biggest champion of in-memory technology over the last five years. The latest chapter from SAP is S/4Hana, a rewrite of the company ERP suite launched in early February and designed to take full advantage of Hana's in-memory power. The platform eliminates aggregates, indexes, and even the need for a separate data warehouse, replacing all of that with virtual views of live, transactional data held in memory. The approach shrinks the data footprint by a factor of 10 while speeding both analytical and transactional performance, according to SAP.
[ Want more on this topic? Read In-Memory Databases: Do You Need The Speed? ]
SAP didn't invent in-memory technology and it certainly doesn't have a monopoly on using RAM for processing transactions or analyzing data. Database incumbents IBM, Microsoft, and Oracle have each introduced in-memory options of their own. These are add-on options to conventional databases, not in-memory databases. But IBM, Microsoft, and Oracle tout the fact that applications designed to run on their flagship databases can now take advantage of in-memory performance without changes to those apps. You simply run key tables in memory to enhance transactional or analytical performance.
Big data practitioners are also falling in love with in-memory technology. The hot, open-source Spark and Storm projects run in-memory, but these are purpose-built analytical and streaming platforms, respectively. This collection focuses on databases, so we look at NoSQL and NewSQL options including Aerospike, a DataStax add-on for Cassandra, MemSQL, and VoltDB.
The earliest purveyors of in-memory databases included Altibase, solidDB (recently divested by IBM), and TimesTen (acquired in 2005 and still owned by Oracle). These products emerged for niche applications such as telecom, financial trading, and high-speed e-commerce. Today these products are seeing broader use, branching into analytics, big data, gaming, and Internet-of-Things-style applications.
Read on to get a better sense of the breadth of in-memory databases and add-on features now available. Offering many times faster transactional or analytical performance (and, in many cases, both), these products are changing people's minds about the practicality and affordability of in-memory computing.
Product description: Altibase HDB is a hybrid database offering an in-memory tier, plus disk-based storage, blending the benefits of RAM speed and disk economy. Altibase XDB is the "extreme" in-memory pure-play database, said by the vendor to handle up to 1.5 million transactions per second.
Notable customers: Korea Telecom, Hewlett Packard, Samsung
Company status: Private; founded 1991
Comment: Altibase is a 20-plus-year-old Korean company that is among the pioneers of in-memory technology. Altibase is best known in the Asian market, but it's gaining customers in North America through offices in Fort Lee, N.J., and Palo Alto, Calif. The vendor's hybrid HDB supports transactional and analytic decisions with an ability to blend hot data in RAM and cooler, historical data on disk. The purely in-memory XDB extreme database is aimed at ultra-high-throughput applications seen in telecom, financial, manufacturing, and Internet-of-things applications.
Product description: NoSQL distributed in-memory database offering ACID compliance and tunable consistency.
Notable customers: Appnexus, Kayak, Williams-Sonoma
Company status: Private; founded as Citrusleaf in 2009
Comment: Aerospike brings in-memory speed to high-scale ad networks, gaming companies, and other high-scale businesses that need millisecond response times. Aerospike is pushing into new markets including e-commmerce, travel, and security threat detection, all of which demand ultra-low latency. Aerospike in February appointed a new CEO, Silicon Valley veteran John Dillon, to broaden the company' application and customer base.
Product description: DataStax Enterprises gives the massively scalable, multi-data-center-replicated Cassandra open source NoSQL database an added (commercial) in-memory data-management.
Notable customers: Constant Contact, eBay, Netflix
Company status: Private; founded 2010
Comment: DataStax offers commercial support for Apache Cassandra, an open source NoSQL database with strengths including flexible data modeling, multi-data-center support, and linear scalability on clustered commodity hardware. In February 2014 the vendor added an In-Memory Option architecture to its supported DataStax Enterprise software. The company has also created an integration to the Apache Spark in-memory analytics platform.
The In-Memory Option is aimed at transactional use cases. It lets developers move new or existing database tables into memory to ensure ultra-fast performance. The option offers an alternative to deploying an in-memory caching product, such as Memcached or Redis, alongside Cassandra in order to handle low-latency processing needs.
Product description: IBM's BLU Acceleration option brings compression, in-memory analysis, and vector-processing techniques to DB2.
Notable customers: Coca-Cola Bottling Co., DataProxy Llc., VNS Systemen BV
Company status: Public (NYSE: IBM); founded 1911.
Comment: IBM introduced BLU Acceleration as an option for its flagship DB2 database in 2013. BLU combines compression, in-memory analysis, and vector-processing techniques. As a result, IBM says BLU can crunch 10 terabytes down to 1 terabyte, bring that 1 terabyte into memory and crunch it again down to 10 gigabytes. Using data-skipping technology, BLU can then focus in on the 1 gigabyte that matters to a query without wading through irrelevant data.
Results vary, but IBM says BLU improves query performance by 8X to 25X over DB2 without the option. A mid-2014 update of BLU added Shadow Tables that let users speed transactional reports by as much as 100X, according to IBM.
Product description: NewSQL database combining extreme scalability with ACID compliance, SQL querying, and in-memory performance.
Notable customers: Comcast, Shutterstock, Zynga
Company status: Private; founded 2011
Comment: As its name suggest, MemSQL is distinguished from high-scale NoSQL options by its combination of ACID-compliant transactional performance and SQL compatibility. Adding to its in-memory data tier, MemSQL last year added a compressed columnar store supported by Flash and disk storage options. This combines the advantages of in-memory performance with the ability to economically make high-scale historical data available for analytics. MemSQL's head-to-head competitor is VoltDB, which is also included in this collection.
Product description: Microsoft SQL Server 2014 option typically speeds online transactional processing by 5X to 20X.
Notable customers: Bwin.party, Edgenet
Company status: Public (Nasdaq: MSFT); founded 1975
Comment: Microsoft SQL Server In-Memory OLTP was introduced as part of last year's SQL Server 2014 release. As the name suggests, it speeds online transaction processing. Its use case is getting around disk I/O-throughput bottlenecks and read-write contention without changing applications that run on SQL Server.
Gaming company Bwin.party used In-Memory OLTP to increase the throughput of an online sports-betting app to handle 150,000 bets per second that was previously limited to 12,000 bets per second. The only change required was moving selected database tables to run in memory. The option typically delivers 5X to 20X improvements in performance, according to Microsoft.
Product description: Oracle Database In-Memory is an optional feature of the vendor's flagship database. Oracle TimesTen is an embeddable in-memory database used in niche apps.
Notable customers: Land O Lakes, Mitsubishi, Thales Raytheon Systems, Yahoo
Company status: Public (Nasdaq: ORCL); founded 1977
Comment: Oracle Database In-Memory is an optional, extra-cost feature of the Oracle relational database that promises 100X faster analytical performance and 2X to 4X faster transactional performance without changes to applications that run on the database. Changes aren't needed because Oracle has maintained its longstanding row data approach to ensure support for existing apps. The In-Memory option adds a columnar analytical store cached in RAM. Transactional performance can be improved by eliminating analytical indexes (a DBA change) that are no longer required once customer-selected tables and partitions are placed in memory.
Oracle's first foray into in-memory technology came with its 2005 acquisition of TimesTen, an embedded, in-memory transaction-processing database used since the late 1990s in low-latency applications in telecom, financial trading, and online retail. More recently this database was embedded in Oracle's Exalytics in-memory analytical appliance.
Product description: SAP Hana is an in-memory database now offered as a broader development platform and cloud platform as a service.
Notable customers: ConAgra, Colgate-Palmolive, John Deere
Company status: Public (NYSE: SAP); founded 1972
Comment: SAP didn't invent in-memory technology, but over the last five years it has done more than any other company to focus attention on the benefits and possibilities of the technology given advances in compute capacity and RAM affordability. Hana started out as an in-memory database, but it has matured into a development platform complete with a built-in application server. There's also the Hana Cloud Platform platform-as-a-service offering, which offers multi-tenant database services.
SAP Hana is now the "fastest-growing database platform at scale," according to SAP, with 2,000 startups building on the database and more than 6,000 customers overall. Most recently Hana became the foundation for Business Suite 4 SAP Hana (or S/4Hana), the vendor's latest generation of ERP applications. The app data model is simplified thanks to in-memory advantages including the ability to eliminate separate transactional and analytical environments and associated aggregates, indexes, and other unnecessary copies of data. Granular data can be instantly analyzed in myriad virtual views.
Product description: Teradata Intelligent Memory continuously monitors the temperature of data from frequently queried "hot" data to seldom-accessed "cold" data and stores it on the most appropriate storage option.
Notable customers: American Red Cross, Blue Cross-Blue Shield, eBay, Walmart
Company status: Public (NYSE: TDC); founded 1979
Comment: Teradata was the pioneer of distributed, high-scale computing. Its analytical database has long recognized the difference between hot, frequently accessed data and cold, rarely accessed data and has stored that data on appropriately fast storage media. Today's Teradata Intelligent Memory has a rich set of storage options, from RAM, to SSDs, to fast conventional drives, to high-capacity drives for archival data. Teradata says its approach recognizes that in-memory speeds may only be needed for the 20% of data that gets 80% of the query activity.
Product description: Highly scalable 'NewSQL' in-memory database offering ACID- and SQL-compliance.
Notable customers: Flytxt, Novatel Networks, Shopzilla, Yahoo
Company status: Private; founded in 2009
Comment: VoltDB has high-scale, high-speed transaction processing down pat thanks to its highly distributed, in-memory architecture. The open-source, AGPL-licensed DBMS, dreamed up in part by co-founder and DBMS luminary Dr. Michael Stonebraker, can handle fast-streaming data from the likes of telcos, mobile ad networks, and gaming companies. With analytical upgrades introduced in 2014 and more recent big-data connector upgrades, VoltDB says it offers a resilient, highly available, ACID-compliant alternative to a menagerie of complex open source products used in streaming and Internet-of-Things applications.
-
About the Author(s)
You May Also Like