Big Data Meets Trough Of Disillusionment: Gartner - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Software Platforms
08:06 AM
Connect Directly

Big Data Meets Trough Of Disillusionment: Gartner

Gartner says cloud, in-memory databases are headed into Hype Cycle "trough of disillusionment." Productivity expected within two to five years.

Cloud computing and in-memory databases, two darlings of the big data movement, have passed the "peak of inflated expectations" and are headed into the "trough of disillusionment," according to Gartner's Hype Cycle for Big Data, 2013.

Hype Cycles are Gartner's way of communicating the degree of hyperbole versus productivity associated with emerging technologies, with the peak and trough being followed by a "slope of enlightenment" and progressing to "plateau or productivity" as technologies mature. In the case of cloud and in-memory databases, the plateau is two to five years away, according to research discussed at last month's Gartner Symposium.

In the case of cloud, compute capacity and storage in the cloud are thought of by many as cost-saving options, but "organizations are a little disappointed about perceived cost versus reality," Gartner analyst Nick Heudecker told InformationWeek. "We're starting to see some pullback with organizations realizing that they may not be saving that much money."

That could be because moving anything approaching big data scale into the cloud involves storage and data-movement costs that can add up. Heudecker cited the case of the streaming service Spotify, which recently opted to move its cloud deployment on-premises.

[ Looking for the inside edge on big data breakthroughs? See What's The Best Path To Big Data Insight? ]

"Cloud makes sense when you can't predict your load or demand, but once you have a very predictable workload and you can easily estimate what kind of resources you're going to need, the cloud may not be as attractive," Heudecker said.

Spotify's example matches the experience of Rob Rosen, big data field lead at Pentaho, who said he often sees cloud as the place where big data practitioners start. "The cloud is a great place to do prototyping and early architecture work, but once they succeed and start to scale up, they start to see monthly bills that make them say, 'hey, wait a minute; maybe it's time to bring this in house,'" Rosen said.

On the in-memory database front, Heudecker said the technology has captured a lot of attention, but now questions are emerging as people look beyond the touted benefits of fast performance, simplified infrastructure, and potentially smaller data footprints and power requirements. The questions are popping up about support for disaster recovery and high availability, availability of database tooling, and the impact on application development teams.

"Right now in-memory technology is adolescent," said Heudecker. "We forecast that it will be two to five years before it reaches the plateau of productivity -- perhaps longer, depending on the database."

SAP's Hana platform is perhaps the best known in-memory database platform, but others include the Aerospike database, in use for several years mostly in online advertising; IBM's BLU acceleration feature for DB2, introduced earlier this year; and Microsoft SQL Server 2014, an update set for release in the first half of next year that will include an In-Memory Online Transaction Processing option that's now in public beta. Oracle preannounced an in-memory option for Oracle database, but beta and GA release dates have not been announced.

Other technologies plotted on Gartner's Hype Cycle for Big Data include Hadoop SQL interfaces and the Internet of Things now headed for the Peak; social analytics and Hadoop distributions headed into the Trough; and social media monitoring and predictive analytics now on the Plateau. One final observation from Heudecker is that the Big Data Hype Cycle is going to be particularly dynamic.

"There are a large number of technologies on this Hype Cycle, and the majority of them are clustered around the Peak," he said. "That means we're going to see a lot of change over the next two to five years."

Emerging software tools now make analytics feasible -- and cost-effective -- for most companies. Also in the Brave The Big Data Wave issue of InformationWeek: Have doubts about NoSQL consistency? Meet Kyle Kingsbury's Call Me Maybe project. (Free registration required.)

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
D. Henschen
D. Henschen,
User Rank: Author
11/18/2013 | 10:30:23 AM
SAP: Time to emphasize real-world in-memory gains
SAP is the self-proclaimed leader on in-memory technology, but it has to show it's leading out there in the real world. Let's start hearing more concrete customer success stories and less hype about unspecified "never-before-possible applications" unmeasured "dramatically faster performance" and unverifiable "transformation without disruption." I'm talking to customers and hearing lots of good things, but there are realities (like the cost of new servers) and tradeoffs (like rework required to change apps and analytics) that SAP doesn't acknowledge.

Customers need to know they will get to that "plateau of productivity." Hype will deepen the dissolusionment while real-world case examples will help establish realistic expectations.
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Pandemic Responses Make Room for More Data Opportunities
Jessica Davis, Senior Editor, Enterprise Apps,  5/4/2021
10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
Transformation, Disruption, and Gender Diversity in Tech
Joao-Pierre S. Ruth, Senior Writer,  5/6/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll