Take Advantage of In-Memory Analytics - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management
09:44 AM
Connect Directly

Take Advantage of In-Memory Analytics

Fast analysis, better insight and rapid deployment with minimal IT involvement: these are among the benefits of in-memory analytics, but different products are appropriate for different environments. Read our in-depth report on in-memory technologies.

How It Works

As the name suggests, the key difference between conventional BI tools and in-memory products is that the former query data on disk while the latter query data in random access memory (RAM). When a user runs a query against a typical data warehouse, for example, the query normally goes to a database that reads the information from multiple tables stored on a server's hard disk (see "Query Approaches Compared," below).

Click Image To View Larger Version

Query Approaches Compared
With in-memory tools, all information is first loaded into memory. If the in-memory tool is server-based, an administrator may initiate the load process; if it's a desktop analysis tool, the user may initiate the process on his or her workstation. Users then query and interact with data loaded into the machine's memory. Accessing data in memory is literally millions of times faster than accessing data from disk. This is the real, "speed-of-thought" advantage that lives up to all the hyperbole.

In-memory BI may sound like caching, a common approach to speeding query performance, but in-memory products don't suffer from the same limitations. Those caches are typically subsets of data, stored on and retrieved from disk (though some may load into RAM). The key difference is that the cached data is usually predefined and very specific, often to an individual query; but with in-memory tools, the data available for analysis is potentially as vast as an entire data mart. Another approach to solving query performance problems is for database administrators (DBAs) to analyze a query and then create indexes or aggregate tables in the relational database. When a query hits an aggregate table, which contains only a subset of the data, it may scan only a million records rather than the hundreds of millions of rows in a detailed fact table.

Yet one more route to faster performance is to create a MOLAP (Multidimensional Online Analytical Processing) database. But whether it's tuning or MOLAP, these paths to user-tolerable performance are laborious, time consuming and expensive. DBAs are often versed in performance tuning for transaction systems, but the ability to tune analytic queries is a rarer skill set often described as more art than science. What's more, indexes and aggregate tables consume costly disk space. In-memory tools, in contrast, use techniques to store the data in highly compressed formats. Many vendors and practitioners cite a 1-to-10 data-volume ratio when comparing in-memory systems to traditional, on-disk storage.

So while users benefit from lightning-fast queries, in-memory BI is also a big win for BI system administrators. Newell Rubbermaid, for example, says its BWA deployment has eliminated significant administrative time that was formerly required to tune queries. "The queries were so fast [after the in-memory deployment], our users thought some data must have been missing," says Rajeev Kapur, director of business analytics at Newell Rubbermaid. Yet the performance improvement didn't involve analysis of access paths or creation of indexes or aggregate tables as were previously required.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
2 of 3
Comment  | 
Print  | 
More Insights
What Becomes of CFOs During Digital Transformation?
Joao-Pierre S. Ruth, Senior Writer,  2/4/2020
Fighting the Coronavirus with Analytics and GIS
Jessica Davis, Senior Editor, Enterprise Apps,  2/3/2020
IT Careers: 10 Job Skills in High Demand This Year
Cynthia Harvey, Freelance Journalist, InformationWeek,  2/3/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
IT 2020: A Look Ahead
Are you ready for the critical changes that will occur in 2020? We've compiled editor insights from the best of our network (Dark Reading, Data Center Knowledge, InformationWeek, ITPro Today and Network Computing) to deliver to you a look at the trends, technologies, and threats that are emerging in the coming year. Download it today!
Flash Poll