IoT
IoT
Data Management // Big Data Analytics
Commentary
1/9/2015
10:03 AM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
Google+
LinkedIn
Twitter
RSS
100%
0%

5 Analytics, BI, Data Management Trends For 2015

InformationWeek survey shows growth in cloud-based data warehousing, real-time tech, and big-data platforms, but business intelligence standardization is waning.

Companies are moving from business intelligence standardization to experimentation, data-quality concerns are easing a bit, and cloud-based, real-time, and big-data platforms are on the rise.

These are among the findings of our 2015 InformationWeek Analytics, Business Intelligence, and Information Management Survey, published late last month based on responses from 384 business technology professionals. The two most important themes explored in the report are overcoming the complexity of analytics and BI tools and meeting the challenges of big-data analysis. But our annual tracking survey also uncovers interesting changes in respondent attitudes and priorities when it comes to analytics, BI, and information management.

[Want more on analytics and business intelligence? Read Microsoft Previews 6 New Power BI Features.]

Here, then, are five trends to watch out for in 2015:

1. Analytics and BI standardization is waning.
A decade ago, the BI market was consolidating as companies tried to standardize on fewer BI products that could be deployed throughout the organization. Times are changing. Cloud and mobile innovation and big-data exploration favor experimentation with a new generation of tools. Our 2015 survey reveals that only 28% of respondent firms now say they're standardizing, down from 35% in 2014. And 21% say their firms use "many analytics and BI products," up from 16% in 2014.

2. Cloud-based data warehousing is on the rise.
Cloud-based data warehousing services show the biggest increase in adoption of any information-management category, jumping to 34% from just 24% in 2014. The poster child here is Amazon Redshift, the data warehousing service launched in 2013. Last year, Amazon Web Services CTO Werner Vogels described Redshift as the company's fastest-growing web service ever. IBM responded in 2014 with Netezza-based DashDB. Next up will be HP, which announced plans to launch an online version of its Vertica database early this year.

3. Real-time technology is seeing real gains.
Providers of complex event processing (CEP) technology have predicted for more than a decade that the technology is going mainstream. We're still not there, with broad use of CEP still a rarity outside of financial services, security, telco, and intelligence agency circles. But adoption shows an eight-point lift to 35% of our 2015 respondents. Usage is "extensive" among 12% of respondents, while 23% say they use CEP "on a limited basis."

Amazon, once again, provides a mainstream proof point with its Kinesis CEP service, which is powering real-time bidding, buying, gaming, media streaming, and other services. Apache Spark, Apache Storm, Splunk, and other options are bringing real-time analysis to system, service, network, mobile, and application-monitoring scenarios. The cloud may be the enabler this once-exotic technology was waiting for.

4. Hadoop and NoSQL adoption are growing.
No surprise here; NoSQL and Hadoop both show seven-point gains in adoption since our 2014 survey, used by 26% and 22% of our 2015 respondents, respectively. Factors driving interest in NoSQL databases include "faster, more flexible development than achievable with relational databases," cited by 24% of respondents, and "lower software and deployment cost," cited by 21%. Motivations to use Hadoop include the "ability to store and process semistructured, unstructured, and variable data," cited by 31% of respondents, and "lower hardware and storage scaling cost than commercial products," cited by 25% of respondents.

Reality check: Consistent with last year's results, about half of all respondents still say NoSQL and Hadoop are "not a priority for my organization" at this time.

5. Data quality concerns are easing.
Data is messy -- always has been, always will be. Big-data platforms like Hadoop and NoSQL databases accommodate messiness because they don't force you to normalize everything into predefined data models with consistent dimensions of data. Techniques like machine learning and visualization let the data do the talking. Accuracy improves with scale, and trends and exceptions stand out more clearly. It's not that data quality is no longer important; indeed, it's still the No. 1 "barrier to success" cited by both BI and analytics types and information management professionals. But concern is diminishing -- at least where analysis is concerned -- with only 51% putting data quality on their "barriers" list, versus 59% in 2014.

One upshot of this year's report is that many companies are no longer counting on a few favored vendors to provide all the answers (along with hefty consulting bills). Organizations are experimenting. In some cases they're reacting to competitors that are innovating with new data sources and technologies.

The same old data and the same old tools can support the same old business, but how many CEOs are satisfied with the way things are? Not many. How many companies aren't confronting big challenges? Plenty. Don't embrace change for the sake of change, but don't be complacent in 2015.

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
pfretty
50%
50%
pfretty,
User Rank: Ninja
1/16/2015 | 2:10:12 PM
Missing strategy
According to a recent IDG survey one aspect that seems to be missing in a large number of organizations is a holistic data management strategy. This is alarming since data management often excels when there is a clear infrastructure and backbone for success. 

 

Peter Fretty, IDG blogger working on behalf of SAS
arindam.ari
50%
50%
arindam.ari,
User Rank: Apprentice
1/13/2015 | 2:00:21 AM
Analytics and BI standardization is waning
Regarding the first point - the organizations need to get there data in order. Liberate data from applications, maange it and then use best of breed Analytics and BI tools as per use cases - no need to standardize analysis tools. 
Brian.Dean
50%
50%
Brian.Dean,
User Rank: Ninja
1/12/2015 | 8:35:15 PM
Re: Good summary
Every employee of an organization can benefit from data and add value to the company's bottom line by utilizing the data. Communicating data to every part of the organization is a challenge. However, visualization has made a lot of progress to break down the boundaries and enable everyone to work with data. Once communication is sorted out and further progress is made, then I expect these figures to skyrocket.
ricksschultz
50%
50%
ricksschultz,
User Rank: Apprentice
1/12/2015 | 8:06:28 PM
Analytics and Data Blending Challenges (Related Surveys)
Thanks Doug - this is a useful survey and insightful analysis of the results.  At Alteryx, we work directly with thousands of data analysts (#1 audience that utilizes analytics in your survey - at 68%) and see them tackling similar challenges to the ones described in your report- particularly with finding easier to use tools that rapidly prepare their data for visualizations (in Tableau or Qlik, for example) and perform advanced analytics without coding.  

We also see the challenges you highlight related to data quality and combining data from lots of sources, increasingly including Hadoop and NoSQL databases like MongoDB ... and in fact we also just published some survery results of data analysts and their challenges related to Data Blending that showed 93% are typically combining two or more data sources for their analysis and 62% are actually combining five (5) or more! For more detail, their is an InfoGraphic on our Data Blending survey results (and a link to the full report) here: bit.ly/1ASXONY.  

Thanks.

Rick Schultz,

SVP Marketing at Alteryx
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
1/12/2015 | 10:41:29 AM
Good summary
This post is a good summary for the past and provide a kind of trend prediction for the future. Especially I am interested in seeing how real-time technology will perform in 2015. The real-time data have more business value and better ROI compared to just post-analysis result.
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.