Big Data // Big Data Analytics
Commentary
1/9/2014
02:58 PM
Connect Directly
RSS
E-Mail
100%
0%

IT Analytics: Measuring Up Is Hard To Do

IT groups are struggling to measure their own performance, especially around cloud services, new research says.

Vendor surveys can be a snooze, but they can also offer interesting insights, sometimes unintentionally.

Case in point: The first ever IT Operations Analytics Benchmark Survey, from Continuity Software, has a few things worth noting.

For starters, analytics may be a big buzzword, but it doesn't yet pervade corporate IT. Continuity asked whether companies were using analytics to measure the performance of internal IT, and a majority (56%) do not. Its survey did find that 57% of large companies (those with more than 2,500 employees) use analytic tools to measure performance. But only 29% of companies with fewer than 2,500 employees use such tools.

At their worst, vendor surveys reinforce company talking points or remind investors just how much upside still exists in a market. The Continuity survey does some of that, since the company's product involves helping companies analyze their systems for risk. But Continuity is not a monitoring company. Executives told Information Week they thought the survey was useful for understanding what the market wants, but that no one tool would be likely to fill those needs.

[What's the difference between the three key types of data analytics? Read Big Data Analytics: Predictive Vs. Prescriptive.]

What is surprising is that very few companies appear to measure their cloud services -- only 14% of respondents reported doing this. As for other parts of IT infrastructure, the numbers range from 49% to 75%. Storage and network tied at the top, followed by applications and databases. Clusters were cited by only 49%.

Doron Pinhas, Continuity's CTO, believes the reason many companies aren't monitoring their cloud services is because the technology is still maturing, and it's hard to measure how these services should be performing. For clusters, Pinhas explained, "The rate of change over the last five years has been spectacular, with [the] rise of virtualization across the board." Such rapid change has created a far more complex IT environment that changes rapidly, making it hard to measure.

Measurement is a mantra of management, and the survey found that uptime was the top metric tracked among organizations that are tracking. Here's a breakdown showing the percentage of respondents that track each metric (note that the survey allows for multiple responses):

  • Uptime: 92%
  • Performance/response time: 80%
  • Data loss: 56%
  • Number of open issues: 52%
  • Average time to fix: 51%
  • Security breaches: 49%
  • Mean time between failures: 38%

Less than half of the companies track in real time. When they do, uptime remains the most tracked metric, at 43% of respondents, and security breaches jumps to number two, at 33%.

Respondents said the hardest metrics to meet were performance (29% rated it number one) and uptime (17% rated it number one). Performance was ranked in the top three by 74% of respondents.

Analytics consultants are happy to tell us that many companies are not data-driven. The survey suggests less than half of IT departments are. Of respondents, 48% said that most or all critical decisions are informed by operational analytics, meaning more than half use analytics to inform operational decisions rarely or only some of the time.

The survey, conducted online from companies in Continuity's database, received 90 responses. The majority of respondents were not Continuity customers, according to Eran Livneh, the company's VP of marketing.

Of responding companies, 44% had more than 5,000 employees, 7% had between 2,500 and 5,000, 18% had from 501 to 2,500, and 31% had between 1 and 500 employees. More than half the respondents had more than 500 servers in their datacenters, and of those 25% had more than 2,500 servers. Only 19% of respondents had fewer than 50 servers.

Private clouds are moving rapidly from concept to production. But some fears about expertise and integration still linger. Also in the Private Clouds Step Up issue of InformationWeek: The public cloud and the steam engine have more in common than you might think. (Free registration required.)

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
sushantsaraswat
50%
50%
sushantsaraswat,
User Rank: Strategist
1/15/2014 | 4:42:50 AM
Deploy Data Solutions in Cloud
Big data analytics is going to be mainstream with increased adoption among every industry and forma virtuous cycle with more people wanting access to even bigger data.

However, often the requirements for big data analysis are really not well understood by the developers and business owners, thus creating an undesirable product.

For organizations to not waste precious time and money and manpower over these issues, there is a need to develop expertise and process of creating small scale prototypes quickly and test them to demonstrate its correctness, matching with business goals.

Following up on this, I came across and registered for a webinar on Deploy Big Data solutions Rapidly in Cloud through Harbinger's ABC model (Agile-Big Data-Cloud), it looks a promising one http://j.mp/19xJ6ew
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - September 10, 2014
A high-scale relational database? NoSQL database? Hadoop? Event-processing technology? When it comes to big data, one size doesn't fit all. Here's how to decide.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.