Software // Information Management
12:45 PM

NoSQL Everywhere? Not So Fast

Enterprise big data workloads are incredibly diverse and require a smart mix specialized and commodity hardware and software.

Sometimes we need to slow down and sort through the hype, despite pressure to act. A perfect example: NoSQL.

Factors including the speed with which data is piling up and the rate at which the cost of conventional databases is escalating are pushing people who should know better to consider using NoSQL in situations where it's wholly inappropriate. That category includes stores of core customer and product data and related transactions that have to exist in a single source of truth and use standard access techniques.

Now, we get why IT is intrigued. Just 11% of the 760 respondents to our InformationWeek 2012 State of Database Technology Survey are very satisfied with licensing costs and terms for their conventional databases. And the broad "NoSQL" label does indeed reflect the main attraction: Often, these systems are open source or quite inexpensive and can run on commodity hardware.

But remember, there's a reason structured data and relational database management systems have ruled the enterprise roost since the '80s. SQL--along with technologies like distributed caching platforms that have helped cement its dominance by bringing in-memory caching and clustering of data--is the go-to choice for mission-critical, high-performance use cases such as airline reservation systems and financial/securities applications. And the fact is, those jobs aren't going anywhere--especially to NoSQL options like MongoDB on a white-box server--no matter how much the hype machine might suggest otherwise.

However, IBM, Oracle, and Microsoft are seeing an unprecedented challenge to their SQL licensing money machines thanks to enterprise data workloads that are getting more diverse, larger, and more complex. In those cases, IT will trade stringent data consistency for low latency and a favorable cost versus performance balance. This is where NoSQL technologies like those outlined in the chart below shine.

So how can big companies get in on the action? Let's contrast the nature of data suited for NoSQL with the properties of enterprise data that requires the single-source-of-truth systems that we talked about. We'll use three V's: volume, velocity, and variety. (We cover these in more depth in our full report).

Data linked to mission-critical applications--and generally housed in conventional transactional systems--is described by the acronym "ACID."

This data is:

>> Atomic: Transactions are fully committed, meaning they execute completely or are rolled back if anything fails.

>> Consistent: Transactions don't leave the database if there's an inconsistency with the underlying data store.

>> Isolated: Transactions are insular and don't affect one another.

>> Durable: Successful changes are recorded permanently in the database before the transaction is considered complete.

In contrast, Web and social media applications must focus more on massive scalability--often a few orders of magnitude higher than what IT normally handles. That means developers can't be too fastidious about data rigidity. For example, an item on an online auction site can't be locked for long intervals while someone contemplates a bid; it has to be simultaneously made available to thousands of other shoppers. The data properties of such workloads have been dubbed BASE, in which it's OK for data in these environments to be:

>> Basically available: Fairly self-explanatory, and as opposed to being atomic in real time.

>> Soft-state: Meaning it may change or expire, as opposed to being durable.

>> Eventual consistency: As opposed to stringent transactional consistency.

But the nuances of BASE vs. ACID don't tell the whole story when it comes to big data initiatives, because they mainly focus on variety. Use cases where NoSQL is ideal also feature staggering volumes and velocities of data; we're talking, at the low end, a few terabytes of records, with 10,000 or 20,000 concurrent inserts per second, minimum. Once you're at that level, IT needs to focus on platforms that can serve as the infrastructure layer--capable of scaling out, without costing a fortune.

The Trouble With Labels

Our full report on use cases for NoSQL is available free with registration.

This report includes 14 pages of action-oriented analysis and 5 charts:
  • Analysis of the "three V's" of enterprise data
  • Sample big data reference architecture
Get This And All Our Reports

1 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
5/21/2012 | 4:11:04 PM
re: NoSQL Everywhere? Not So Fast
Notwithstanding the comment about Neo4j, the article is informative and contains some numbers like 20K inserts and 1 TB /day that are useful. Wish the article had more details. Wondering if anyone can redo the table to reflect the neo4j comments etc !!
San Mateo
San Mateo,
User Rank: Apprentice
5/12/2012 | 11:53:19 PM
re: NoSQL Everywhere? Not So Fast
Rick Bullotta is right. There is a lot of misinformation in this article. A more thoughtful piece is the recent report by Bloor Research at
User Rank: Apprentice
5/12/2012 | 2:01:38 PM
re: NoSQL Everywhere? Not So Fast
I read with interest Mr. KajeepetaG«÷s article on the need for a more
thoughtful approach to selecting data management technologies, rather
than succumbing to the latest fad/trend. While I agree with much of
what he states, particularly the need for a mixed/multi-database
approach to modern application design, there are a few flawed elements
of the article that I felt merited correction. First of all, graph
databases such as Neo4J, Infinigraph, and AllegroGraph are most
definitely *not* G«£binary large object databasesG«•. In fact, they are
graph databases with a fundamentally different data storage and
retrieval model than all of the other types of databases mentioned,
and in general, are actually quite poor at BLOB data.

In fact, graph databases are a critical piece of a modern application
strategy, in that they allow data models that are often far more
reflective of the G«£real worldG«• than our attempts to fit the worldG«÷s
information neatly into rows, columns, and documents. His example of
product and customer master data as an G«£inappropriateG«• use of NoSQL
demonstrates ignorance of the capabilities of graph databases. Also,
he made a blanket assumption that NoSQL databases do not support ACID
transactions. That is simply untrue, and is one of the
differentiating features of Neo4J, for example. It is very clear from
the article that Mr. Kajeepeta has very little experience with most of
the products he has declared inferior to his beloved relational
databases. I also agreed with his summary statement G«Ű that it isnG«÷t
about NoSQL, itG«÷s about G«£not only SQLG«•. A phrase which, ironically
enough, is attributed to Emil Eifrem, CEO of Neo4J. I should also
note that we leverage both relational and graph database models in our
ThingWorx platform.

While it was a worthy topic for an article, the factual errors
unfortunately undermined the credibility of the overall content.


Rick Bullotta
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
How to Knock Down Barriers to Effective Risk Management
Risk management today is a hodgepodge of systems, siloed approaches, and poor data collection practices. That isn't how it should be.
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.