Does IT Really Care About Big Data?

Recent surveys show IT and business unit managers are more worried than eager about big data analytics-- but those surveys are probably misleading, according to at least one expert.

There are rafts of surveys showing the huge market potential of big data analytics and the desire of many companies to get their hands on answers they can't get any other way, according to Daniel Castro, senior analyst at the Information Technology and Innovation Foundation (ITIF), whose focuses are cybersecurity, e-government, and IT in healthcare.

A seminal study on big data by McKinsey and Co., for example, found that analysis of big data sets could enhance the productivity and competitiveness of many companies, save more than $300 billion in healthcare alone by increasing the industry's efficiency, and help retailers increase profit margins by as much as 60%.

Another study, "The Future of Big Data" by the Pew Internet and American Life Project, reiterated the findings. It quoted Microsoft chief strategy officer Craig Mundie and Wal-Mart CIO Rollin Ford predicting that a "data-centered economy" in big data analysis will help both government and corporate organizations avoid big mistakes and waste by pointing out persistent errors in practice or belief.

Many experienced IT managers either ignore those predictions or disregard them because they sound too good to be true. They believe all the potential benefits of big data depend on large, expensive changes in the way corporations collect, buy, store, manage, and analyze data, Castro said.

[ Big data is a big deal for sales and marketing organizations. Learn 5 Ways To Benefit From Big Data. ]

"[Big data] is a very new space. People are moving quickly, some faster than others, and mistakes are being made," Castro said. "These [big data] projects are very complex systems; we have to create increasingly complex systems to manage all of it, and that process is still far from finished."

One study that casts doubt on the success of big data shows tight-budgeted IT departments in midsized companies are more willing to support cloud computing than they are to support big data.

Cloud computing, while still new and unfamiliar to many in IT, has been around a lot longer than big data, giving IT executives more time to get used to the technology and the cost, according to Frank Gillett, VP and principal analyst at analyst company Forrester.

The survey, released Tuesday by market-research firm TheInfoPro, showed that 36% of midsized companies will have more to spend on storage this year than in 2011, compared to 47% last year. That drop--really a slowing of growth rather than an actual reduction--contrasts with large companies, whose networked storage capacity will grow an average of 26% his year, the survey showed.

The primary reason for increasing storage spending was the expansion of server virtualization projects, two-thirds of which rely on fibre-channel storage area networks to store the server images, data, applications, and other bits that used to live on hard drives attached to physical servers.

A subset of respondents are increasing storage capacity in an effort to deliver cloud-like internal data storage and server provisioning, according to Marco Coulter, TheInfoPro research director in charge of storage.

The percentage of companies expanding storage to support internal clouds is uncertain because most respondents said they were focused on optimizing storage for virtual servers--a prerequisite for internal cloud platforms--and few were explicit about any plans for cloud.

Nevertheless, 56% were confident enough about big data to tell TheInfoPro they had no plans for big data projects after next year.

That doesn't mean they won't build big data-based decision systems; it just means both big data and their employers' plans for it are uncertain enough that budget approvers would not commit to big data projects past 2013.