Software // Information Management
12:13 PM
Doug Henschen
Doug Henschen
Connect Directly
The Analytics Job and Salary Outlook for 2016
Jan 28, 2016
With data science and big data top-of-mind for all types of organizations, hiring analytics profes ...Read More>>

With Hadoop, Big Data Analytics Challenges Old-School Business Intelligence

Datameer and Karmasphere say their Hadoop-based platforms are what's needed for the next era of data analysis. Are you buying it?

12 Hadoop Vendors To Watch In 2012
12 Hadoop Vendors To Watch In 2012
(click image for larger view and for slideshow)
Datameer and Karmasphere, two competing upstart vendors offering reporting, data-visualization, and data-analysis capabilities on top of Hadoop, released new versions of their software on Monday. Both talked up the need for next-generation tools.

It's not that old-school business intelligence software tools are going away, these upstarts grant. But both portray batch-oriented extract-transform-load (ETL) data integration, relational data warehousing, and old-school analytics as too slow, rigid, and expensive to keep up in the big-data era.

Hadoop is the future, these vendor's contend, because it's a massively scalable data-management and analysis environment that can handle variably structure data from many sources--log files, clickstreams, sensor data, social media sources and so on--without the delays inherent in dealing with the static schemas of relational databases.

If companies want to look at recent point-of-sale transactions alongside Web site clickstreams, recent online enrollments, email campaign results, and social media chatter, for example, it would be difficult if not impossible to quickly put all that data into a relational data warehouse and look for correlations.

[ Want more on Apache's big-data platform? Read Why Hadoop Crowd Is Hearing Much About Hortonworks. ]

Datameer and Karmasphere offer integration, data-analysis, and data visualization products that run on Hadoop. They're two of the better-established companies hoping to provide what they describe as next-generation platforms. Founded in 2009 and 2005, respectively, Datameer and Karmasphere are private companies that each have a handful of nameable customers. Datameer points to blue chip Visa, among others, while Karmasphere says Intel and Microsoft are customers.

In his company's work with a global bank on meeting Basel II capital requirements, Datameer CEO Stefan Groschupf reports that more than 250 data sources are required.

"Finding a perfect star schema and building the required ETL for that is impossible," Groschupf told InformationWeek.

In another example, a big retailer wanted to get data that helped it better understand all its interactions with customers, rather than looking at isolated transactions.

"ETL and data warehousing and BI are just fine for the problem of looking at transactions here and there, but there's no chance of bringing it all together to look at the interactions across all of the islands of information," Groschupf says.

Hadoop also scales in linear fashion to solve the data-volume challenge, and it's built on commodity hardware, so it's less expensive, terabyte for terabyte, than conventional relational systems, Datameer and Karmasphere contend.

What about the scarcity of Hadoop talent, the platform's own batch-related delays and the alternative of using "old-school" BI systems to analyse data either inside or moved out of Hadoop? Datameer and Karmasphere have responses to some of these questions, but more on that later.

What's Different?

Datameer's platform for analytics on Hadoop provides modules for data-integration (including connectors to mainframes, databases, social sources like Facebook and Twitter, and more), a spreadsheet-driven data-analysis environment, and a dashboarding and data-visualization environment, the last part being new in version 2.0. The upgrade also introduces two lower-priced editions designed for fewer users and lower data volumes than the enterprise version.

Datameer's key appeal is its spreadsheet-driven data-analysis environment, which the vendor says provides more than 200 analytic functions from simple joins to predictive analytics. Most importantly, it's said to eliminate the need for Hadoop wonks and IT people to support data analysis. The interface is designed to let business analysts--and certainly any SQL-savvy BI or data-warehousing power user--use a point-and-click interface to analyze data within Hadoop. In the background, Datameer's software turns the selections into Hadoop-based MapReduce and data-manipulation jobs without users having to write code.

Datameer Enterprise starts at $100,000 and supports Hadoop deployments with hundreds or even thousands of nodes. Datameer Workgroup is subscription based software that creates a small-scale Hadoop deployment on a single server. It starts at $3,000 per year for up to 10 users, but data volumes are limited to one terabyte. Exceed that ballpark and you'll have to upgrade to the enterprise edition, though the reports and analyses are portable.

1 of 2
Comment  | 
Print  | 
More Insights
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
How to Knock Down Barriers to Effective Risk Management
Risk management today is a hodgepodge of systems, siloed approaches, and poor data collection practices. That isn't how it should be.
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.