Software // Information Management
News
4/20/2009
09:44 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Take Advantage of In-Memory Analytics

Fast analysis, better insight and rapid deployment with minimal IT involvement: these are among the benefits of in-memory analytics, but different products are appropriate for different environments. Read our in-depth report on in-memory technologies.

How It Works

As the name suggests, the key difference between conventional BI tools and in-memory products is that the former query data on disk while the latter query data in random access memory (RAM). When a user runs a query against a typical data warehouse, for example, the query normally goes to a database that reads the information from multiple tables stored on a server's hard disk (see "Query Approaches Compared," below).

Click Image To View Larger Version

Query Approaches Compared
With in-memory tools, all information is first loaded into memory. If the in-memory tool is server-based, an administrator may initiate the load process; if it's a desktop analysis tool, the user may initiate the process on his or her workstation. Users then query and interact with data loaded into the machine's memory. Accessing data in memory is literally millions of times faster than accessing data from disk. This is the real, "speed-of-thought" advantage that lives up to all the hyperbole.

In-memory BI may sound like caching, a common approach to speeding query performance, but in-memory products don't suffer from the same limitations. Those caches are typically subsets of data, stored on and retrieved from disk (though some may load into RAM). The key difference is that the cached data is usually predefined and very specific, often to an individual query; but with in-memory tools, the data available for analysis is potentially as vast as an entire data mart. Another approach to solving query performance problems is for database administrators (DBAs) to analyze a query and then create indexes or aggregate tables in the relational database. When a query hits an aggregate table, which contains only a subset of the data, it may scan only a million records rather than the hundreds of millions of rows in a detailed fact table.

Yet one more route to faster performance is to create a MOLAP (Multidimensional Online Analytical Processing) database. But whether it's tuning or MOLAP, these paths to user-tolerable performance are laborious, time consuming and expensive. DBAs are often versed in performance tuning for transaction systems, but the ability to tune analytic queries is a rarer skill set often described as more art than science. What's more, indexes and aggregate tables consume costly disk space. In-memory tools, in contrast, use techniques to store the data in highly compressed formats. Many vendors and practitioners cite a 1-to-10 data-volume ratio when comparing in-memory systems to traditional, on-disk storage.

So while users benefit from lightning-fast queries, in-memory BI is also a big win for BI system administrators. Newell Rubbermaid, for example, says its BWA deployment has eliminated significant administrative time that was formerly required to tune queries. "The queries were so fast [after the in-memory deployment], our users thought some data must have been missing," says Rajeev Kapur, director of business analytics at Newell Rubbermaid. Yet the performance improvement didn't involve analysis of access paths or creation of indexes or aggregate tables as were previously required.

Previous
2 of 3
Next
Comment  | 
Print  | 
More Insights
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Must Reads Oct. 21, 2014
InformationWeek's new Must Reads is a compendium of our best recent coverage of digital strategy. Learn why you should learn to embrace DevOps, how to avoid roadblocks for digital projects, what the five steps to API management are, and more.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
A roundup of the top stories and community news at InformationWeek.com.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.