16 NoSQL, NewSQL Databases To Watch
Traditional relational databases weren't invented with mobile, social, and big data types -- or extreme scale -- in mind. Get expert context on 16 next-era NoSQL and NewSQL choices.
![](https://eu-images.contentstack.com/v3/assets/blt69509c9116440be8/blt8e605f6fef9113e2/64cb57cf903f05d24bbbde87/NoSQL-&-NewSQL.jpg?width=700&auto=webp&quality=80&disable=upscale)
Why are businesses increasingly choosing alternatives to the leading relational database management systems when grappling with new data types and extreme scale?
We put that question to Bryson Koehler, CIO of The Weather Company, which is using a NoSQL database, Riak, as the foundation of a cloud-based weather prediction and forecasting system handling 20 terabytes of data per day. His answer was emphatic:
"We knew that we needed to run globally, and we knew we needed to scale to an astronomical level. I knew that I needed that to be fundamentally baked in to how we built the system. I didn't want it to be an add-on. I didn't want it to be an option. I didn't want it to have to require a team of people to maintain it. Your traditional relational data approaches are incredibly cumbersome, complicated, and don't, in my view, scale globally."
That's as strong an endorsement as you can get for NoSQL in a high-scale cloud deployment. We've heard equally positive comments about the flexible, schema-agnostic data-handling characteristics and ease of development offered by products like MongoDB and Couchbase. These guys get mobile, social, clickstream, and sensor data, proponents argue, and they might note that adding JSON support to a conventional database is like putting lipstick on a pig.
If there's one complaint raised, it's that NoSQL products, well, don't speak SQL, a mature language that supports all sorts of transactional and analytical capabilities. Enter NewSQL, an emerging category populated by vendors like Clustrix, MemSQL, VoltDB, and others that promise the global scalability of NoSQL without giving up SQL.
To date, NewSQL options have been far less popular than the NoSQL options, in part because they are very new, but also because a relational approach and data flexibility aren't easily combined. What's more, NewSQL vendors face more competition from old SQL vendors and complacent customers who give familiar products the first crack at solving new problems. That's an expensive mistake, argue the NewSQL vendors.
This collection offers a closer look at 16 notable NoSQL and NewSQL database management systems that deserve a closer look. Excluded from this portfolio are graph databases, which serve a very specific network-analysis role and do not serve in the same broad transactional role as these NoSQL and NewSQL databases. Also excluded are hugely popular products including Redis and Memcached, which tend to serve as data-caching tiers rather than as durable data stores.
This collection is distinguished from our 16 Top Big Data Analytics Platforms analysis, in that these are not, generally speaking, analytical platforms. Running transactional applications is the name of the game here, although these products can also address operational analytical needs. We're also not calling this a "top" collection because the dust hasn't even begun to settle, particularly in the NewSQL category.
Read on to get a bead on new options for modern data-management needs.
DBMS Type: NoSQL
Description: Flash-optimized, distributed in-memory database offering ACID compliance and tunable consistency
Notable customers: Appnexus, BlueKai (now owned by Oracle), eBay
Company status: Private; founded as Citrusleaf in 2009
Comment: Aerospike offers an in-memory speed edge that has attracted high-scale ad networks and other Web-scale businesses that need millisecond response times. Aerospike is banking on breaking into new categories including gaming, e-commmerce, and security where low latency is everything. We're following to see if Aerospike can become a much broader low-latency platform.
DBMS type: NoSQL
Description: Massively scalable partitioned row store with multi-data-center replication
Notable customers: Constant Contact, eBay, Netflix
Company status: Open source project supported by private DataStax, founded in 2010
Comment: Apache Cassandra's strengths are flexible, NoSQL data modeling, multi-data-center support, and linear scalability on clustered commodity hardware. Commercial support and software distributions are available from DataStax. Cassandra enjoys a reputation as the most scalable of NoSQL databases, but it's also living down a reputation for complexity. Well-funded DataStax is moving to simplify manageability for enterprise use while adding low-latency in-memory features and, more recently, in-demand integrations with Apache Spark. Couchbase is a competitor while HBase, which runs on Hadoop, also looms as a potential threat.
DBMS type: NewSQL
Description: Distributed relational DBMS supporting automatic sharding and replication
Notable customers: AOL, Rakuten, Symantec
Company status: Private; founded in 2006
Comment: Clustrix touts highly distributed, fault-tolerant scalability without giving up SQL or ACID transaction performance. The company bills customer Twoo.com's 21-node deployment (depicted above) as "the world's largest scale-out SQL deployment." All the NewSQL vendors promise SQL compliance with easier operation (than incumbent databases) at high scale. The challenge for Clustrix and competitors like MemSQL, NuoDB, and VoltDB will be differentiating and quickly racking up successful high-scale customer wins.
DBMS type: NoSQL
Description: Document-oriented database with mobile sync and embeddable database option
Notable customers: Concur, LinkedIN, Zynga
Company status: Private; founded in 2009
Comment: Couchbase is working hard to cover all the bases, promising the document-handling advantages of MongoDB and the scalability of Cassandra. The company's latest push is Couchbase Mobile, launched in May, which keeps mobile apps running whether they're connected to networks or not. The system combines the Couchbase Lite device-embeddable database with a Couchbase Sync Gateway that ties into the central Couchbase Server. We'll be watching to see if Couchbase can succeed on all fronts.
DBMS type: NoSQL database service
Description: Fast, scalable key-value service runs on multiple Amazon Availability Zones with S3 backup
Notable customers: AdRoll, Elsevier, SmugMug
Company status: Public since 1997
Comment: Amazon developed Dynamo to scale up its own fast-growing e-commerce business, and its 2007 white papers inspired Cassandra, Riak, and other NoSQL projects in the process. The highly scalable DynamoDB database service arrived in 2012. It has been a big hit, but database services have since proliferated. There are now options to start with popular open source databases (like Couchbase, MongoDB, Riak, and others) in one cloud and move it to another cloud or, if need be, on premises. There are no on-premises or cloud provider alternatives with DynamoDB. If you're a dedicated AWS shop, there's no doubt it's a fast, flexible, and highly scalable option.
DBMS type: NoSQL
Description: Wide-column database built on top of the Hadoop Distributed File System (HDFS)
Notable customers: Bloomberg LP, Facebook, Nielsen
Company status: N/A; HBase is a feature of open source Apache Hadoop
Comment: HBase is the NoSQL database that runs on top of HDFS, so it gives users the unique ability to work directly with the data stored on Hadoop. Features include massive scalability (as used in Facebook's messaging system), consistent reads and writes, automatic and configurable sharding of tables, and automatic failover. As of mid-2014, HBase was in release 0.98.3, still hampered by "major" unresolved issues that stand in the way of broad adoption. There's no doubt HBase has huge potential, but development seems to be lagging behind the pace seen on the wider Hadoop framework. We're following projects including Hoya for YARN support, Phoenix for SQL support, Kiji for application development, and Facebook's Hydrabase project that promise a more mature and usable HBase.
DBMS type: Cloud-based NoSQL DBMS Service
Description: Document-store database-as-a-service based on CouchDB
Notable customers: Expedia, DHL, Samsung
Company status: Cloudant was acquired by IBM in February.
Comment: When IBM acquired Cloudant (for undisclosed terms), pundits said the deal was all about tapping the company's database-as-a-service technology and expertise. An IBM exec also talked up the importance of the underlying open-source CouchDB DBMS (not to be confused with Couchbase). DBMS attributes include scalablity, availability, durability, concurrent reads and writes, and flexible data handling (including JSON, full-text, and geospatial data). But CouchDB seems to have been eclipsed by NoSQL rivals in recent years. Confusing matters, IBM has an important partnership with NoSQL market leader MongoDB. Needless to say we'll be watching IBM's next move on this front. Our bet is we'll see more database-as-a-service options using Cloudant capabilities.
DBMS type: NoSQL
Description: Document-oriented database supporting ACID-compliant transactions and built-in search
Notable customers: DowJones, Citigroup, Boeing
Company status: Private; founded in 2001
Comment: MarkLogic had a long history as a successful XML database long before anybody was talking about NoSQL. MarkLogic is used extensively by large-scale technical, financial, legal, healthcare, and scientific heavyweights who need flexible ways to manage and reuse information. Adapted to broader use as a NoSQL document store, this schema-agnostic, clustered DBMS is highly scalable. The company has had years to build out enterprise-grade features including replication, rollback, automated failover, point-in-time recovery, and backup/restore. MarkLogic has a boatload of high-level (often publishing-focused) customers. Whether the company can win more mainstream adoption going up against the likes of MongoDB remains to be seen.
DBMS type: NewSQL
Description: High-scale, in-memory row store with recently added columnar analytical capabilities
Notable customers: Comcast, CPXi, Shutterstock
Company status: Private; founded in 2011
Comment: As its name suggests, MemSQL is distinguished from high-scale NoSQL options by its combination of in-memory, ACID-compliant transactional performance and SQL compatibility. Adding to its relational interface on an in-memory data tier, MemSQL recently added a compressed columnar store supported by flash- and disk-storage options to support deep historical analysis. The head-to-head competitor is older VoltDB (and other NewSQL types), so the race is on for NewSQL supremacy and customer adoption.
DBMS type: NoSQL
Description: Document-store DBMS
Notable customers: Cisco, eBay, Intuit, MetLife
Company status: Private; founded as 10Gen in 2007/
Comment: Open source MongoDB is the most popular NoSQL database, hands down, with more than 7 million downloads and hundreds of thousands of deployments. Its popularity is owed to the ease of development and flexible handling of the modern data seen in mobile, social, and Web applications. With the product's 2.6 release in May 2014, the vendor eased administration, beefed up security, and improved analytical capabilities. The release also improved scalability, a score on which rivals Cassandra, Couchbase, and Riak assert superiority. Scalability is relative, however, and for every global, petabyte-scale deployment, there are hundreds, if not thousands, of use cases where MongoDB's speed and ease of development and flexible data management seem to be winning over new customers.
DBMS type: NewSQL
Description: Scale-out relational DBMS supporting continuous availability, geo distribution, and cloud deployment
Notable customers: AutoZone, Dropship Commerce, NorthPoint Solutions
Company status: Private; founded in 2010
Comment: NuoDB is another of those NewSQL upstarts promising the combination of scalable, geo-distributable deployments with the familiarity of SQL. It offers expected high-availability, hot-upgrade, data-redundancy, and disaster-recovery capabilities. One of NuoDB's twists is an emphasis on cloud deployment with built-in support for multi-tenancy. It's a promising product, but this is one of the younger NewSQL vendors that has yet to really prove itself with lots of customer wins.
DBMS type: NoSQL
Description: Distributed key-value store with roots in BerkeleyDB
Notable customers: Catalyst IT Services, Passoker, Payback
Company status: Public company as of 1986
Comment: The NoSQL market leaders all say they never see the Oracle NoSQL Database in competitive situations, but then, they wouldn't. Customers for this product are clearly loyal Oracle shops that are making the most of a dominant vendor relationship. You can download the free Community edition of this scalable, distributed key-value store, and you can buy support for both the Community and more-complete Enterprise edition. The database is also bundled with the Oracle Big Data Appliance and integrated with its (Cloudera-based) Hadoop distribution. The Oracle NoSQL community isn't exactly buzzing with announcements and activity. But there's a group of loyalists and a place for Oracle's huge customer base to turn for a flexible, scalable NoSQL alternative.
DBMS type: NoSQL
Description: Distributed key-value store with multi-data-center replication and automated balancing
Notable customers: Best Buy, Ideeli, The Weather Company
Company status: Open source project supported by private Basho; founded in 2007
Comment: Open source Riak is designed for massive scalability, availability, fault tolerance, and operational simplicity. Basho-supported Riak Enterprise and Riak CS bring support plus enterprise-grade features and Amazon Web Services-compatible S3 cloud storage, respectively. The Weather Channel uses Riak in Amazon's cloud and it touts the "baked-in" simplicity, scalability, and always-on availability. Riak's ring node cluster approach (seen above) lets you add or remove virtual nodes at will with Riak redistributing the data accordingly. Basho touts Cassandra scale with greater ease of operation. Basho saw significant management turnover in 2013, but we expect stability to return and Basho to continue push cloud deployment and ease of operation as differentiators.
DBMS type: NewSQL
Description: SQL-on-Hadoop RDBMS supporting transactions and analytics
Notable customers: Harte Hanks
Company status: Private; founded in 2012
Comment: There are plenty of SQL-on-Hadoop options out there, but the unique claim of startup Splice Machine is that it can run transactional applications as well as support analytics on top of Hadoop. Customer Harte Hanks tell us it's running a bunch of apps that were designed to run on conventional databases, including IBM Unica, Cognos BI, and Ab Initio data-integration software. This is a very young company with a very short list of namable customers. But running transactional applications on Hadoop is a unique proposition that stands out in the NewSQL crowd.
DBMS type: NewSQL
Description: Distributed relational DBMS with cloud and appliance deployment options
Notable customers: Oakland Raiders
Company status: Private, founded in 2007
Comment: TransLattice started with a high-scale, distributed variation on PostgreSQL called the TransLattice Elastic Database that deploys on premises, on an appliance, or on multiple clouds. In 2013 the company acquired StormDB, which was also extending PostgreSQL, and used its intellectual property to launch Postgres-XL, a scalable, massively parallel analytical database. The idea is to offer the many Postgres users a familiar way to scale out for big-data OLTP and analysis. The very short list of public reference customers shows that TransLattice, like many NewSQL vendors, needs more market proof points.
DBMS type: NewSQL
Description: ACID- and SQL-compliant in-memory relational database
Notable customers: BooYah, Novatel Networks, QualityHealth
Company status: Private; founded in 2009
Comment: VoltDB has high-scale, high-speed transaction processing down pat thanks to its highly distributed, in-memory architecture. The open source, GNU-licensed DBMS, dreamed up in part by co-founder/DBMS luminary Dr. Michael Stonebraker, can handle fast-streaming data from the likes of telcos, mobile ad networks, and gaming companies. Analytical upgrades introduced in VoltDB 4.0, announced in early 2014, brought increases in query throughput and concurrent-user support as well as deeper SQL capabilities in areas such as time-series analysis. VoltDB's most direct competitors are MemSQL, NuoDB, and Clustrix, but with Oracle and Microsoft wading into in-memory transaction processing this year, the biggest competition may be familiarity with these incumbent database management systems, and customer hopes of leveraging them.
DBMS type: NewSQL
Description: ACID- and SQL-compliant in-memory relational database
Notable customers: BooYah, Novatel Networks, QualityHealth
Company status: Private; founded in 2009
Comment: VoltDB has high-scale, high-speed transaction processing down pat thanks to its highly distributed, in-memory architecture. The open source, GNU-licensed DBMS, dreamed up in part by co-founder/DBMS luminary Dr. Michael Stonebraker, can handle fast-streaming data from the likes of telcos, mobile ad networks, and gaming companies. Analytical upgrades introduced in VoltDB 4.0, announced in early 2014, brought increases in query throughput and concurrent-user support as well as deeper SQL capabilities in areas such as time-series analysis. VoltDB's most direct competitors are MemSQL, NuoDB, and Clustrix, but with Oracle and Microsoft wading into in-memory transaction processing this year, the biggest competition may be familiarity with these incumbent database management systems, and customer hopes of leveraging them.
-
About the Author(s)
You May Also Like