Microsoft Goes After 3 Big Data Myths

Big Data's Surprising Uses: From Lady Gaga To CIA

Face it, big data can be scary. There are infrastructure expenses, of course, as well as hiring costs associated with bringing aboard those elusive and expensive data scientists, highly skilled folks who turn massive volumes of unstructured information from a variety of sources into actionable insights.

But perhaps big data isn't as daunting as you think. That's according to Microsoft, which is positioning itself as a big data player with several business intelligence (BI) tools for enterprises. For instance, the company recently released a preview of Data Explorer for Excel 2013, a self-service BI add-in that makes it easier for everyday business workers (not just data scientists) to import data from a variety of sources, including big data platforms like Hadoop.

Microsoft also recently announced the availability of SQL Server 2012 Parallel Data Warehouse (PDW), its massively parallel processing data warehousing appliance built for Hadoop integration.

Which brings us to the big data adoption myths: What are they, and does Microsoft take them down?

The first myth: It's too hard for an organization's IT stack to support big data, which invariably creates infrastructure and scalability woes, wrote Microsoft's Eron Kelly, general manager for SQL Server, in a recent blog post.

An organization may not need racks of servers and other hardware it believes are essential to its big data solution. Rather, an economical solution should offer the "ability to store and process large volumes of data while eliminating any upfront infrastructure cost, as you pay only for the storage and compute capacity that you use," Kelly wrote.

[ Microsoft wants to position Excel as the analytics engine for the masses -- but is that what CIOs want? See: What's Excel's Place In Big Data Age? ]

Not surprisingly, Microsoft's own Windows Azure HDInsight Service fits that description, but Kelly's advice is still useful for enterprises shopping around for big data solutions from other vendors, too.

The second myth, Microsoft said, pertains to the looming data scientist shortage: Enterprises can't find enough qualified big data gurus to pull insights from unstructured information sources, such as social media feeds and machine sensor data.

"While it is true that the industry needs more data scientists, it is equally true that most organizations are equipped with the employees they need today to help them gather the valuable insights from their data that will better their business," writes Kelly.

In other words, big data tools and apps can save the day. Microsoft's argument ties in with the so-called democratization of data movement. Popular tools, such as Excel with the Data Explorer add-in, allow end users to perform BI analysis without having to pester IT for help.

The third myth: Big data is a challenge rather than an opportunity.

OK, this may sound like a lot of marketing hooey, but there's a lot of truth here. Implementing a big data platform is no doubt both a major headache and a major opportunity. But don't let the former dissuade you from pursuing the latter.

"I'm often asked, "Where is the ultimate value in big data and how do I tap into it?" There are two key measures in my mind: 1) time to insight, and, 2) return on accessible data. These measures are, in turn, enabled through a process I call information production," wrote Microsoft technical fellow Dave Campbell.

Information production is the process of converting data from one domain to another, according to Campbell. Good information production tools allow you to gain business insights in less time. Add big data to the mix, and you've got a much wider variety of unstructured and structured data to explore.

These tools "allow you to get from a hunch to validation very quickly," wrote Campbell.