New Database Benchmark Closes Loopholes That Made Tests Unrealistic - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Enterprise Applications
06:50 PM
Connect Directly

New Database Benchmark Closes Loopholes That Made Tests Unrealistic

The new benchmark uses real data and mimics the more complicated transactions that are typical today.

Build a benchmark to compare product performance, and someone's going to find a loophole. But recent work to update the 15-year-old standard for comparing performance of databases should make the results a much more meaningful reflection of today's complex transactions.

The benchmark--TPC-E, from the vendor-sponsored Transaction Processing Performance Council--uses real publicly available data, such as names from the U.S. Census and trading data from the New York Stock Exchange, to arrive at a transactions-per-minute measure. It requires handling several different types of data in a multistep transaction.

TPC-E sought to close loopholes in its predecessor, TPC-C, which specified steps of a transaction to be executed by a system but left implementation to each vendor. That led to shortcuts, such as tests that searched for masses of user names that all began with the same two letters, or tests that retrieved highly similar data from just one section of the database. Those shortcuts yielded results better than any real user could expect.

The old benchmark mimicked a straightforward retail order from a warehouse. TPC-E replicates a stock trade through a brokerage house, with several complex steps along the way.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
10 Cyberattacks on the Rise During the Pandemic
Cynthia Harvey, Freelance Journalist, InformationWeek,  6/24/2020
IT Trade Shows Go Virtual: Your 2020 List of Events
Jessica Davis, Senior Editor, Enterprise Apps,  5/29/2020
Study: Cloud Migration Gaining Momentum
John Edwards, Technology Journalist & Author,  6/22/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
Key to Cloud Success: The Right Management
This IT Trend highlights some of the steps IT teams can take to keep their cloud environments running in a safe, efficient manner.
Flash Poll