Big Data Grows Up: APM Tools Emerging - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Software Platforms
02:50 PM
Connect Directly

Big Data Grows Up: APM Tools Emerging

The next emerging market for big data may be application performance monitoring. Concurrent has released a new tool in this space, and more tools are expected to hit the market in the next six months. The availability of such tools will drive more enterprise adoption of big data.

Big Data: 6 Real-Life Business Cases
Big Data: 6 Real-Life Business Cases
(Click image for larger view and slideshow.)

Big data infrastructure's creation came out of a need for high performance distributed systems, so it may not be surprising that application performance monitoring tools for Hadoop apps haven't been plentiful -- yet.

As big data platforms start to mature, tools will begin to emerge, and that's just what is expected to happen with Hadoop in the next year or so. As they emerge, more enterprises will take a more serious look at implementing Hadoop.

[Looking for more about tools for big data? Read IBM Wants To Bring More IT To Big Data.]

Concurrent is among the few first movers in this space. Last week the company introduced Driven 2.0, an update to its monitoring and management tool for Hadoop that extends the tool to support Spark, too.

"We are focused on enterprises that are moving their processing to Hadoop or Spark and getting serious about monitoring and management," Concurrent CEO Gary Nakamura told InformationWeek in an interview. Nakamura said that this is still an early market, but maturity is happening rapidly with individual customers evolving from a small need to a large need in a matter of months.

"The systems become resource intensive and chaotic," he said. "We provide visibility."

The Current Market

Management tools for big data are following a similar trajectory as they did for previous infrastructures such as client-server, according to Mike Matchett, senior analyst and consultant for the Taneja Group, a market analyst firm.

"Once I put this into production, you've got multiple jobs, multiple workloads, multiple tenants," Matchett told InformationWeek in an interview. "How do you manage this from an IT perspective? This is IT for Agile DevOps, so the bottom line is someone has got to manage this thing in production."

(Image: bofotolux/iStockphoto)

(Image: bofotolux/iStockphoto)

So far not many companies have offered tools in this space.

"The big guys just haven't been in the game here," Matchett said. "They are just starting to look at it." That makes Concurrent pretty early in the market with a strong tool.

"That's not to say someone else won't come out with another tool in six months. I don't see something here that someone else couldn't develop. But it's just a very complex thing to develop this for a distributed environment" as Concurrent has done with Driven, Matchett said.

APM as Market Driver

He sees tools such as Concurrent's Driven 2.0 as enablers that could spur the big data market.

"I've been in the systems management space long enough to know that people are going to be reluctant to put in systems like Hadoop unless they have tools like Driven to manage it," he said. "They are definitely leading the market right now, but the market is definitely going there."

What are some of the use-cases for this product at the leading edge of the big data infrastructure management market? Nakamura told InformationWeek that one of the most common use-cases involves migrating applications to Hadoop from other platforms.

It's complicated: New frameworks, tools, technologies, and compute engines have created new requirements for DevOps, data operations, and teams. Concurrent Driven 2.0 works with Apache Hive, MapReduce, Cascading, Scalding, and now Spark.

The new Spark support is currently in beta. It provides "the comprehensive performance management required to deliver and maintain production Spark data processes," Concurrent said in a prepared statement announcing Driven 2.0.

Matchett told InformationWeek that many of the tools providing support for big data are likely to merge ultimately into a distributed data center management platform. While the market opportunity may not be huge today, "In six months it will be."


Jessica Davis is a Senior Editor at InformationWeek. She covers enterprise IT leadership, careers, artificial intelligence, data and analytics, and enterprise software. She has spent a career covering the intersection of business and technology. Follow her on twitter: ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

How SolarWinds Changed Cybersecurity Leadership's Priorities
Jessica Davis, Senior Editor, Enterprise Apps,  5/26/2021
How CIOs Can Advance Company Sustainability Goals
Lisa Morgan, Freelance Writer,  5/26/2021
IT Skills: Top 10 Programming Languages for 2021
Cynthia Harvey, Freelance Journalist, InformationWeek,  5/21/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll