Big data and little data
Big data may be a mainstream topic, but we're nowhere near mainstream deployment. According to a recent TDWI survey, only 10 percent of companies have deployed big data technologies.
As I discuss in my book, Successful Business Intelligence: Unlock the Value of BI & Big Data, there is still confusion about what "big data" means. Many consider the term to be synonymous with Hadoop and NoSQL technologies, and, in the early days, they predicted the demise of the data warehouse. Fortunately, even providers of Hadoop solutions, such as Cloudera, are increasingly recognizing that Hadoop is only one part of a big data ecosystem. Analytic appliances such as IBM PureData System for Analytics (formerly Netezza), Pivotal Greenplum, and HP Vertica (to name a few) are also part of the big data market. As are in-memory computing options including SAP Hana, Oracle Exalytics, and Microsoft Hekaton (due in 2014).
[Want more on business intelligence? Download our "2014 Analytics, BI, and Information Management Survey."]
The earliest adopters of these technologies may have large data volumes, but others have small data volumes. These organizations, too, are looking to mine insight from unstructured content such as sensor data, Tweets, Facebook posts, and email messages that are not easy to explore with traditional relational databases. They, too, need new technologies, even if it's not about "big data."
In the year ahead, big data developments in BI will bring:
- Tighter integration between Hadoop technologies and traditional data warehouse storage, with a greater emphasis on the right use cases
- Better support from BI vendors for a wider range of big data sources
- Maturing products from Hadoop-world startups such as Datameer, Platfora, and others
- In-memory advances running transactional applications from SAP, Oracle, and Microsoft, threatening traditional disk-based storage
- Education of BI teams still trying to keep up with vendor innovation
My dad was a disc jockey when I was growing up, and our stereo system was so complicated, with so many knobs and dials, that I didn’t dare go near it. In much the same way, many business people tend to shy away from BI tools. There have been attempts to achieve simplicity in this software category, but with lackluster success.
Expect 2014 to be the year for simpler BI interfaces that don't sacrifice smarts. These products will be driven in part by visual data discovery, but also by search, improved dashboard interfaces, and embedded BI.
In the area of search, the year ahead will bring:
- Oracle Endeca 3.0, released late last year, with cleaner visualizations and faceted navigation of both structured and unstructured content
- Microsoft Q&A, an interface for the upcoming PowerBI release that will let users enter simple search terms to find existing content or to create new data visualizations
- IBM Project Neo, a SaaS-based offering that combines search, smart visualization, and advanced analytics
- New tools form NeutrinoBI and DataRPM, startups using search and natural language processing to bring simplicity to a broader base of users.
It's tough to limit this list to five trends. A top-10 list would have to include advanced analytics, collaboration, social data, the talent shortage, and privacy. We saw a lot of activity in advanced analytics in 2013, including SAP's acquisition of KXEN and broader support for R. The analytic talent shortage has limited adoption of advanced analytic techniques, but vendors are trying to address this gap by embedding the expertise directly in the software.
With the success of Facebook and Twitter IPOs, I would have thought that collaborative BI would be showing strong momentum. It made my top-trend list last year, but this trend has yet to gain traction. The mining of social data, meanwhile, has become part of the savvy marketer’s toolkit. I’ve seen good solutions from BI vendors, particularly Information Builders and IBM, but also from Attivio, which integrates with a number of visual data discovery system vendors.
Privacy, though, just might throw a big wrench in the works where big data and social data is concerned. From NSA phone tapping to the Target data breach, consumers have had a big education on just how much data we're creating, how much it’s being mined, and how easy it is to steal. The greater good in the use of all this data needs to be weighed against the need for privacy. Expect more breaches and a backlash.
Cindi Howson is the founder of BI Scorecard , an independent analyst firm that advises companies on BI tool strategies and offers in-depth business intelligence product reviews.
Small and midsized businesses have a big security problem: They are vulnerable to the same kinds of attacks as large enterprises, but they often don’t have the same resources for fending off, or recovering from, those attacks. Security services can help small and midsized businesses get and stay secure. In this report, 6 Security Services Every Small Business Must Have, Dark Reading examines the reasons MSSPs make sense for SMBs, recommends the six categories of services that every SMB needs, and provides tips for evaluating and hiring providers. (Free registration required.)