Big Data Grows Up: APM Tools Emerging - InformationWeek
IoT
IoT
Data Management // Software Platforms
News
10/26/2015
02:50 PM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Big Data Grows Up: APM Tools Emerging

The next emerging market for big data may be application performance monitoring. Concurrent has released a new tool in this space, and more tools are expected to hit the market in the next six months. The availability of such tools will drive more enterprise adoption of big data.

Big Data: 6 Real-Life Business Cases
Big Data: 6 Real-Life Business Cases
(Click image for larger view and slideshow.)

Big data infrastructure's creation came out of a need for high performance distributed systems, so it may not be surprising that application performance monitoring tools for Hadoop apps haven't been plentiful -- yet.

As big data platforms start to mature, tools will begin to emerge, and that's just what is expected to happen with Hadoop in the next year or so. As they emerge, more enterprises will take a more serious look at implementing Hadoop.

[Looking for more about tools for big data? Read IBM Wants To Bring More IT To Big Data.]

Concurrent is among the few first movers in this space. Last week the company introduced Driven 2.0, an update to its monitoring and management tool for Hadoop that extends the tool to support Spark, too.

"We are focused on enterprises that are moving their processing to Hadoop or Spark and getting serious about monitoring and management," Concurrent CEO Gary Nakamura told InformationWeek in an interview. Nakamura said that this is still an early market, but maturity is happening rapidly with individual customers evolving from a small need to a large need in a matter of months.

"The systems become resource intensive and chaotic," he said. "We provide visibility."

The Current Market

Management tools for big data are following a similar trajectory as they did for previous infrastructures such as client-server, according to Mike Matchett, senior analyst and consultant for the Taneja Group, a market analyst firm.

"Once I put this into production, you've got multiple jobs, multiple workloads, multiple tenants," Matchett told InformationWeek in an interview. "How do you manage this from an IT perspective? This is IT for Agile DevOps, so the bottom line is someone has got to manage this thing in production."

(Image: bofotolux/iStockphoto)

(Image: bofotolux/iStockphoto)

So far not many companies have offered tools in this space.

"The big guys just haven't been in the game here," Matchett said. "They are just starting to look at it." That makes Concurrent pretty early in the market with a strong tool.

"That's not to say someone else won't come out with another tool in six months. I don't see something here that someone else couldn't develop. But it's just a very complex thing to develop this for a distributed environment" as Concurrent has done with Driven, Matchett said.

APM as Market Driver

He sees tools such as Concurrent's Driven 2.0 as enablers that could spur the big data market.

"I've been in the systems management space long enough to know that people are going to be reluctant to put in systems like Hadoop unless they have tools like Driven to manage it," he said. "They are definitely leading the market right now, but the market is definitely going there."

What are some of the use-cases for this product at the leading edge of the big data infrastructure management market? Nakamura told InformationWeek that one of the most common use-cases involves migrating applications to Hadoop from other platforms.

It's complicated: New frameworks, tools, technologies, and compute engines have created new requirements for DevOps, data operations, and teams. Concurrent Driven 2.0 works with Apache Hive, MapReduce, Cascading, Scalding, and now Spark.

The new Spark support is currently in beta. It provides "the comprehensive performance management required to deliver and maintain production Spark data processes," Concurrent said in a prepared statement announcing Driven 2.0.

Matchett told InformationWeek that many of the tools providing support for big data are likely to merge ultimately into a distributed data center management platform. While the market opportunity may not be huge today, "In six months it will be."

 

Jessica Davis has spent a career covering the intersection of business and technology at titles including IDG's Infoworld, Ziff Davis Enterprise's eWeek and Channel Insider, and Penton Technology's MSPmentor. She's passionate about the practical use of business intelligence, ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Stackify01
50%
50%
Stackify01,
User Rank: Apprentice
12/1/2015 | 9:09:00 PM
APM data
We have customers of our Stackify APM+ that send us several terrabytes a month. We deal with some serious big data of our own by providing APM services. 
pfretty494
100%
0%
pfretty494,
User Rank: Moderator
11/23/2015 | 12:04:58 PM
No doubt
As much as we have talked about big data in recent years, sometimes its hard to put in perspective how immature the technology still is, especially within a lot of organizations. Fortunately, the tools are there and increasingly more people are beginning to embrace the potential. Peter Fretty, IDG blogger for SAS
Commentary
5 Machine Learning Resolutions for 2019
Lisa Morgan, Freelance Writer,  12/13/2018
News
What Tech Mega Hubs Will Do to East Coast Cities
Kayla Matthews, Technology Writer,  12/11/2018
News
Daimler Financial Services CIO Says: Don't Get Comfortable
Jessica Davis, Senior Editor, Enterprise Apps,  12/5/2018
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Enterprise Software Options: Legacy vs. Cloud
InformationWeek's December Trend Report helps IT leaders rethink their enterprise software systems and consider whether cloud-based options like SaaS may better serve their needs.
Slideshows
Flash Poll