Big Data In 2014: Top Technologies, Trends

Big data joined the business lexicon in a big way in 2013. Some days it seemed like every press release and technology news item crossing our desk included the phrase. In fact, the wild popularity of the term might have masked important developments -- technical, operational, and other -- involving big data and its associated technologies.

Figure 1:

As we look ahead to 2014, we put out a call on social media asking for 2014 big data predictions. Here are some of the responses we received:

"The hot new data of 2013 was 'exhaust data' powered by the Internet of Things. This will take further hold during 2014, but the hot data of next year will be human data. With knowledge that employee data has been shown to do everything from help manage organizational health to be a leading indicator of quarterly consumer demand, this area will be a hot area for startups. Expect another round of TOS updates from LinkedIn." --Dan Malligner, data science practice lead, Think Big Analytics

"The data-information-insight-decision lifecycle will get shortened due to machine learning-based automated decision systems; increased adoption of Cloud ETL to analyze on-premise and off-premise open data; open-source solutions such as R will replace legacy solutions like SAS; the number of offerings of reporting solutions embedded in cloud with 90-day deployment will increase." --Milind Kelkar, leader, Smart Decisions Lab, Genpact

[Who are next year's rising stars in business intelligence? Read 2014 BI Outlook: Who's Hot, Who's Not.]

"The future of big data in 2014 will be 'where.' Location intelligence is of paramount importance to companies as their customers are increasingly handling the majority of their daily activity on mobile, from online banking to restaurant check-ins and social sharing in real time. Having precise location data helps organizations understand relationships between specific locations so that they can identify growth opportunities, improve information sharing internally and to their customers, and make better strategic business decisions." --James Buckley, SVP, customer data and location intelligence, Pitney Bowes Software

"It's a major problem for businesses that their online and offline data management systems don't talk to one another. For example, when prospective customers start their buying process online but choose to make a phone call for assistance, the online analytics vanish. Businesses will be seeking means of appending specified data points to those interactions so they have the appropriate IDs and device information to retarget that customer on any given channel. The focus will increasingly be on automating how that data is reported, packaged, and sent to other systems." --Eric Holmen, CMO, Invoca

"While Hadoop is still immature, technology advances like YARN are contributing to an enterprise-friendly big data future. Because of YARN, we'll increase opportunities to use new and more optimally efficient engines and expand Hadoop possibilities." --Mike Hoskins, CTO, Actian

"For the first time in 30 years, the data preparation-versus-analytics time ratio will flip as the amount of time we spend getting data ready will be reduced by half. This is great news for analysts since 70-80% of time is spent on data preparation. Traditional large ETL vendors will attempt to enter the business-led data preparation market, but products focused on data scientists will fall short of their promise." --Prakash Nanduri, CEO, Paxata

"Upon realizing that they can harvest the buying signals from the Web, social media and third-party data sources like Lexis Nexis and PIERS to identify the leads most likely to become closed deals, marketers will begin to follow the practices of companies like Google, Facebook, and Netflix and focus on drawing insights from data. With all the pieces now in place -- the cloud, large installed bases of CRM and marketing automation systems, and the exploding data volume -- predictive marketing and applications will democratize the power of buying signals for everyone -- no team of data scientists required." Shashi Upadhyay, CEO, Lattice

"Employers will leverage big data in 2014 to close skills gaps. Advancements in data science and the widespread adoption of recruitment technologies, such as applicant tracking systems (ATS), have created a perfect storm of opportunity for employers to leverage big data in their recruitment efforts." --Ken Lazarus, CEO, Scout

"Smart data -- a more complex, integrated, multi-step, social-network-aware, knowledge-based analysis -- will be replacing [a] big data approach for many problems. Watch for deep network learning, a human-level performing method, to come out of the lab and appear in R & Python packages." --Gregory Piatetsky, KDnuggets

"In 2014, big data will get contextual. More organizations will realize [that] the secret to leveraging big data's full potential is the ability to hone in on the select contextual data that matters most to customers and businesses. We saw many companies get hung up on collecting as much data as possible in 2013 rather than zeroing in on the data that moves the needle. In the upcoming year, companies that are able to define which datasets will have the most impact on crafting a great customer experience will succeed in gaining the competitive edge. This will translate into the realm of sales as well. More companies will recognize that being able to provide their salespeople with information pooled from social channels, purchase history, and CRM means arming them with the big data context needed to engage customers and prospects with highly targeted messages." --Andy MacMillan, SVP and GM of Salesforce Data.com.

"The collection and analysis of big data will be increasingly complemented with a focus on activating that data in real time, at the point of decisioning. Whether in-store, online, over the phone, etc., the ability to extract from massive data stores the exact relevant and actionable details that will enhance a single customer interaction -- and help deliver the desired outcome -- will become a differentiating capability. That will involve not just the data itself, but technological infrastructure as well as business processes built around data to enable its extraction, delivery, and packaging, just in time." --Ari Buchalter, COO, MediaMath

"I predict that gurus who think they are data scientists will give up due to lack of income … Although a lot of people started and will start data-science-as-a-service companies, few will be able to continue in 2014/2015. Therefore, I predict that there will be a lot of acquisitions and mergers among data science companies, as well as people giving up and going back to the corporate world." --Carla Gentry, founder, Analytical-Solution

Ellis Booker is a technology journalist who has covered ecommerce, business strategy and marketing for InformationWeek, InternetWeek, and other publications. He is based in Evanston, Illinois.

IT groups need data analytics software that's visual and accessible. Vendors are getting the message. Also in the State Of Analytics issue of InformationWeek: SAP CEO envisions a younger, greener, cloudier company. (Free registration required.)