Here, then, are five trends to watch out for in 2015:
1. Analytics and BI standardization is waning. A decade ago, the BI market was consolidating as companies tried to standardize on fewer BI products that could be deployed throughout the organization. Times are changing. Cloud and mobile innovation and big-data exploration favor experimentation with a new generation of tools. Our 2015 survey reveals that only 28% of respondent firms now say they're standardizing, down from 35% in 2014. And 21% say their firms use "many analytics and BI products," up from 16% in 2014.
2. Cloud-based data warehousing is on the rise. Cloud-based data warehousing services show the biggest increase in adoption of any information-management category, jumping to 34% from just 24% in 2014. The poster child here is Amazon Redshift, the data warehousing service launched in 2013. Last year, Amazon Web Services CTO Werner Vogels described Redshift as the company's fastest-growing web service ever. IBM responded in 2014 with Netezza-based DashDB. Next up will be HP, which announced plans to launch an online version of its Vertica database early this year.
3. Real-time technology is seeing real gains. Providers of complex event processing (CEP) technology have predicted for more than a decade that the technology is going mainstream. We're still not there, with broad use of CEP still a rarity outside of financial services, security, telco, and intelligence agency circles. But adoption shows an eight-point lift to 35% of our 2015 respondents. Usage is "extensive" among 12% of respondents, while 23% say they use CEP "on a limited basis."
Amazon, once again, provides a mainstream proof point with its Kinesis CEP service, which is powering real-time bidding, buying, gaming, media streaming, and other services. Apache Spark, Apache Storm, Splunk, and other options are bringing real-time analysis to system, service, network, mobile, and application-monitoring scenarios. The cloud may be the enabler this once-exotic technology was waiting for.
4. Hadoop and NoSQL adoption are growing. No surprise here; NoSQL and Hadoop both show seven-point gains in adoption since our 2014 survey, used by 26% and 22% of our 2015 respondents, respectively. Factors driving interest in NoSQL databases include "faster, more flexible development than achievable with relational databases," cited by 24% of respondents, and "lower software and deployment cost," cited by 21%. Motivations to use Hadoop include the "ability to store and process semistructured, unstructured, and variable data," cited by 31% of respondents, and "lower hardware and storage scaling cost than commercial products," cited by 25% of respondents.
Reality check: Consistent with last year's results, about half of all respondents still say NoSQL and Hadoop are "not a priority for my organization" at this time.
5. Data quality concerns are easing. Data is messy -- always has been, always will be. Big-data platforms like Hadoop and NoSQL databases accommodate messiness because they don't force you to normalize everything into predefined data models with consistent dimensions of data. Techniques like machine learning and visualization let the data do the talking. Accuracy improves with scale, and trends and exceptions stand out more clearly. It's not that data quality is no longer important; indeed, it's still the No. 1 "barrier to success" cited by both BI and analytics types and information management professionals. But concern is diminishing -- at least where analysis is concerned -- with only 51% putting data quality on their "barriers" list, versus 59% in 2014.
One upshot of this year's report is that many companies are no longer counting on a few favored vendors to provide all the answers (along with hefty consulting bills). Organizations are experimenting. In some cases they're reacting to competitors that are innovating with new data sources and technologies.
The same old data and the same old tools can support the same old business, but how many CEOs are satisfied with the way things are? Not many. How many companies aren't confronting big challenges? Plenty. Don't embrace change for the sake of change, but don't be complacent in 2015.
Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.
Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio
6 Tools to Protect Big DataMost IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Big Data Brings Big Security ProblemsWhy should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.
Top IT Trends to Watch in Financial ServicesIT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Join us for a roundup of the top stories on InformationWeek.com for the week of September 18, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."