Can Open Source Solve Big Data For SMBs?

8 Big Data Deployments In Detail

(click image for larger view)
Slideshow: 8 Big Data Deployments In Detail

The online age has done wonders to help small companies loom larger in the public eye, but that virtual growth is not simply a marketing phenomenon. Some small and midsize businesses today find themselves contending with enterprise-like amounts of data, but without the equivalent resources to properly manage it.

That's a sentiment expressed by vendors, analysts, and smaller companies alike. "One of the things that's been very clear in the last five years or so is that big data problems and big volumes of data are not an issue relegated to large enterprises," said Jaspersoft CEO Brian Gentile in an interview. Jaspersoft, which makes business intelligence (BI) software, is among a group of open source companies pushing their interoperable "data stack"--their phrase for the spectrum of database on one end to analytics and BI on the other--as a viable alternative to proprietary platforms.

Gentile is not alone in preaching the SMB data explosion. "It is a myth that the analytical requirements in smaller companies are less rigorous than large ones. In fact, it is often the opposite," said Neil Raden, founder and president of Hired Brains, via email interview. "We have clients with fewer than 100 employees who have data warehouses or equivalents in the petabyte range, because data is their business."

That's echoed by Cloudera's vice president of products Charles Zedlewski. "If you look at small and medium-sized businesses, there are a lot of them where their entire business is predicated upon data," said Zedlewski in an interview.

Super-sized data might lead some smaller businesses to sign on with vendors that have traditionally buttered their bread in the enterprise. But the big-ticket platforms--even those that actively seek small or midmarket customers--can still prove to be budget busters for many SMBs. Cost and lack of IT expertise were listed as the top two BI inhibitors for smaller firms in a recent LogiXML survey. As a result, LogiXML chief marketing officer Ken Chow told me in a recent interview, "do nothing and use Excel" is very much a competitor for analytics and BI vendors that target SMBs.

And that doesn't even fully take into account the back-end: All that data has to be stored and maintained somewhere if it's to be mined for analytical gold later. So can open source providers such as Cloudera, Revolution Analytics, and Jaspersoft solve the cost and complexity equation for SMBs?

It's a fairly straightforward sell on cost: Yes, open source software can save money on licensing and maintenance relative to commercial options. "They're typically easier to acquire," said IDC analyst Brian McDonough in an interview. "You don't have the large expenses of an on-premises enterprise license. You can start small, deploy it incrementally, and grow it as you need it." That accessibility has helped Jaspersoft, for example, hit 13 million downloads and 160,000 production deployments worldwide when including its free version in the count. McDonough points out that a key advantage of open source platforms is that SMBs--especially those relatively new to the analytics and BI world--can learn the technology and determine if it even meets their needs before making a significant financial commitment.

As a result, open source software is giving SMBs a new entry point into data management and BI platforms. "This move towards an open stack based on industry standards is really going to level the playing field, especially between SMBs and these larger players that had access to these very expensive solutions," said David Smith, VP of community at Revolution Analytics, in an interview.

But when it comes to IT resources and technical know-how--or lack thereof, in the case of many small companies--open source is not an open-and-shut case. "You're still going to need to be familiar with analytics concepts," said IDC's McDonough. "Things like data integration, information management, how cubes work, and structuring data for analysis." The ability to unlock value from the various platforms, whether individually or as an interoperable stack, depends largely on the abilities of the users. At a minimum, going open source demands a desire to learn the tools inside and out--let's say that's a given--and the time do so. And there's the SMB rub: That time is an ever-dwindling commodity.

10 Tenets Of Enterprise Data Management

(click image for larger view)
Slideshow: 10 Tenets Of Enterprise Data Management

The boundaries of my own knowledge led me to call upon James Phillips, co-founder and SVP of products at Couchbase, for his thoughts on the open source data stack and its utility for SMBs. Phillips gave me a refresher course on the fundamental difference between transactional data and analytical data--while the former might eventually become the latter, the two are (or should be) housed separately for application performance and other reasons. Phillips stresses Couchbase's simplicity and said that on the transactional side of data, his company has legacy technologies such as Oracle and MySQL beat. The analytical side, however, has work to do in terms of ease of use.

"I would agree that some of the other technologies, particularly on the analytics side, are a bit more complex than perhaps some SMBs are comfortable with," Phillips said in an interview. "I think that's going to change over time, certainly, but some of those technologies have focused more on raw analytical power and raw science rather than customization and simplicity. If those markets are going to grow--and I believe they are on their side--then they're going to have to deal with simplicity issues."

There are without doubt SMBs, particularly web services companies and other tech startups, that have the requisite in-house expertise to harness that power today. But the open source data maze could send the broader SMB universe running faster than you can say "NoSQL." It requires a certain amount of knowledge simply to identify the right provider for the right need.

Take Cloudera--the company might be on the forefront of Hadoop-based data management, but it's not for every SMB. In the so-called data stack, Cloudera positions itself as a foundational platform, but it would be a poor choice for a small company not dealing with heavy-duty data volume. "If you don't have your hands on at least a couple of terabytes of data, Hadoop is probably not a good fit for you," Cloudera's Zledewski said. He added that Cloudera is investing in ease of use in a several ways, including working with MicroStrategy to allow for its visualization interfaces to be used on top of Hadoop, as well as streamlining tools for installation and ongoing management. "We try to lower the barrier for what it takes to take advantage of this system," Zedlewski said.

For data-intensive SMBs looking to add IT staff to tackle the terabytes, Smith from Revolution Analytics offers a take that might be welcome news: Current and upcoming crops of university graduates will be increasingly well-versed in open source software, which means young (read: cheap) talent is waiting in the wings. "Being able to find technical people that can work with each of the stages and phases of this data analytics stack--as opposed to having to hire an industry veteran that's been trained for over 20 years in one of these monolithic solutions--is really getting [SMBs] much more advanced talent at a much lower cost."

It may be a matter of time before open source truly emerges as a clear-cut option for smaller businesses grappling with growing data, particularly those making their first foray out of largely manual systems into true data management and analysis. Given the chance to mature and invest in ease of use, the open source players could ultimately become a major force for a wide range of SMBs, particularly those priced out by enterprise-focused vendors. But that could require some patience. Commenting on Cloudera, for example, Raden of Hired Brains said: "This is a different kettle of fish. It doesn't support interactive query, requires real programming skill, and is still pretty raw. Expect that to change, but for now, unless you have very large data requirements, it's not a solution."

In the meantime, the data isn't going to stop growing and flowing. The cost advantages of open source are evident, even if it requires an investment in new in-house skills. "There is a level of technical expertise required, but if you're at a size where you have data that needs to be analyzed through either ad-hoc query or structured reports, then you've outgrown Excel," said IDC's McDonough. "You're at a point as a company where you need to start looking at BI tool options."