Fast analysis, better insight and rapid deployment with minimal IT involvement: these are among the benefits of in-memory analytics, but different products are appropriate for different environments. Read our in-depth report on in-memory technologies.
How It Works
As the name suggests, the key difference between conventional BI tools and in-memory products
is that the former query data on disk while the latter query data in random access memory
(RAM). When a user runs a query against a typical data warehouse, for example, the query normally goes to a database that reads the information from multiple tables stored on a server's
hard disk (see "Query Approaches Compared," below).
Click Image To View Larger Version
Query Approaches Compared
With in-memory tools, all information
is first loaded into memory. If the in-memory tool is server-based, an administrator may initiate
the load process; if it's a desktop analysis tool, the user may initiate the process on his or her
workstation. Users then query and interact with data loaded into the machine's memory.
Accessing data in memory is literally millions of times faster than accessing data from disk.
This is the real, "speed-of-thought" advantage that lives up to all the hyperbole.
In-memory BI may sound like caching, a common approach to speeding query performance,
but in-memory products don't suffer from the same limitations. Those caches are typically subsets
of data, stored on and retrieved from disk (though some may load into RAM). The key difference
is that the cached data is usually predefined and very specific, often to an individual
query; but with in-memory tools, the data available for analysis is potentially as vast as an
entire data mart.
Another approach to solving query performance problems is for database administrators
(DBAs) to analyze a query and then create indexes or aggregate tables in the relational database.
When a query hits an aggregate table, which contains only a subset of the data, it may
scan only a million records rather than the hundreds of millions of rows in a detailed fact table.
Yet one more route to faster performance is to create a MOLAP (Multidimensional Online
Analytical Processing) database. But whether it's tuning or MOLAP, these paths to user-tolerable
performance are laborious, time consuming and expensive. DBAs are often versed in performance
tuning for transaction systems, but the ability to tune analytic queries is a rarer skill set
often described as more art than science. What's more, indexes and aggregate tables consume
costly disk space. In-memory tools, in contrast, use techniques to store the data in highly compressed
formats. Many vendors and practitioners cite a 1-to-10 data-volume ratio when comparing
in-memory systems to traditional, on-disk storage.
So while users benefit from lightning-fast queries, in-memory BI is also a big win for BI system
administrators. Newell Rubbermaid, for example, says its BWA deployment has eliminated significant administrative time that was formerly required to tune queries. "The queries were so
fast [after the in-memory deployment], our users thought some data must have been missing,"
says Rajeev Kapur, director of business analytics at Newell Rubbermaid. Yet the performance
improvement didn't involve analysis of access paths or creation of indexes or aggregate tables
as were previously required.
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.
InformationWeek Must Reads Oct. 21, 2014InformationWeek's new Must Reads is a compendium of our best recent coverage of digital strategy. Learn why you should learn to embrace DevOps, how to avoid roadblocks for digital projects, what the five steps to API management are, and more.