5 Big Wishes For Big Data Deployments

Big data proponents are a vocal bunch, promoting sundry tools and technologies that enable enterprises to mine a steady stream of unstructured data for hard-to-reach insights. But how easy are big data projects to set up? And once implemented, are they worth the cost?

A recent survey by IDG Research Services and Kapow Software shows a fair amount of disillusionment among big data pioneers. But despite the negativity, businesses still see big data projects as a potential boon that's worth pursuing.

The survey of more than 200 IT and business leaders at large organizations shows mixed feelings toward big data. For instance, more than 85% of respondents agree that big data can help businesses make "more informed" data-driven decisions. However, just 23% of these leaders see big data projects as a "success" thus far, while 52% of respondents call the projects "somewhat successful."

So why the chasm between hope and reality? A Kapow Software white paper that examines the survey results offers this explanation: "Big Data projects are taking far too long, costing too much and not delivering on anticipated ROI because it's really difficult to pinpoint and surgically extract critical insights without hiring expensive consultants or data scientists in short demand. The broader issue at hand is a growing mass of data that's difficult to collect from a wide variety of sources."

[ Is your organization ready for the Internet of Things? Read Internet Of Things Wake-Up Call For Enterprises. ]

Obviously, the complexities of big data projects play a major role in this dissatisfaction among business leaders. One of the respondents' thorniest big data problems is the inability to quickly and effectively automate structured and unstructured data. In addition, 60% of respondents say that big data projects typically take a long time -- 18 months or more -- to finish.

Another problem is the dearth of easy-to-use big data tools. Respondents report that business employees who lack special training in big data analysis -- meaning most of them -- must rely on IT departments to glean information from big data sets.

"The lack of simple tools that make it easy to consume and put critical data into the hands of business users is a perceived barrier that keeps Big Data an IT endeavor rather than a business-driven initiative to support decision-making across the organization," the white paper asserts.

Given the implementation challenges and lack of end-user tools, it's no surprise that just 1 in 10 survey respondents say that the big data solutions available today are effective at getting important information to their workers in a timely fashion. (In other words, 90% disagree with that statement.)

All isn't bleak, however. While just 32% percent of companies polled have set up big data initiatives to date, the number of big data projects should double in the next 12 months, the survey says.

There are other glimmers of hope as well. Numerous tech firms are starting to market big data tools for regular business folks, a development that might help alleviate the skills gap, reduce pressure on IT departments, and make big data platforms more accessible to everyone.

BigML, for instance, is a Corvallis, Oregon-based startup with a cloud- based machine learning platform that lets business users create predictive models quickly. And companies like Talend, SnapLogic, which calls its SnapReduce technology "Hadoop for humans," and Tableau Software, maker of the Tableau Desktop drag-and-drop analytics tool, offer big data tools for non-techies as well.

It's little surprise that 85% of survey respondents believe that big data strategies should be "user-centric." Without that focus, the data-driven future we've been hearing so much about will never fulfill come to pass.

What do you think? Are big data projects delivering the goods? Or is their ROI, well, MIA?

E2 is the only event of its kind, bringing together business and technology leaders across IT, marketing, and other lines of business looking for new ways to evolve their enterprise applications strategy and transform their organizations to achieve business value. Join us June 17-19 for three days of 40+ conference sessions and workshops across eight tracks and discover the latest insights in enterprise social software, big data and analytics, mobility, cloud, SaaS and APIs, UI/UX and more. Register for E2 Conference Boston today and save $200 off Full Event Passes, $100 off Conference, or get a FREE Keynote + Expo Pass!