Oracle's New Exadata Architecture

A day after Oracle CEO Larry Ellison announced Oracle's first-ever hardware product, the HP Oracle Database Machine, which he touted as the world's fastest database machine, I sat through an OpenWorld session that promised to revolutionize query processing.The fact that Oracle had scheduled three identical mysteriously titled "Oracle's New Database Accelerator" sessions on the last day of its annual user conference had implied big news was in store. Session leader Juan Loaiza, Oracle's senior VP of systems technologies, cleared up the mystery by explaining that the three ambiguously titled sessions and a separately titled technical overview session later in the day had all been designed to go into more technical detail about Ellison's surprise announcement. "That's the trouble with secret projects," Loaiza quipped, "they can't be listed in the session catalog."

The HP Oracle Database Machine is a combination of smart storage software from Oracle and industry-standard hardware from HP. It consists of a grid of Oracle Database servers and a grid of new Oracle Exadata Storage Servers packaged in a single rack that uses an InfiniBand infrastructure, the communications link that has become the de facto interconnect of choice for Crays and other high-performance computers. This allows the database machine to take advantage of its massively parallel architecture to dramatically speed up Oracle data warehouses by offloading data-intensive query processing from Oracle Database Servers and doing the processing closer to the data.

Loaiza lauded the Exadata product's extreme performance and attributed it to a combination of brain and brawn. He said that in addition to having brainy software such as bitmap indexing, join indexing, OLAP cubes, materialized views, result caches, partitioning, etc., to enable very sophisticated data analysis, the product can take advantage of plenty of brawny hardware to perform brute-force scans and joins on databases containing hundreds of terabytes of data.

Explaining that current warehouse deployments often have bottlenecks limiting the movement of data from disks to servers, he said pipes between disks and servers are typically 10 to 100 times too slow for the size of data being transmitted.

Exadata solves this data bandwidth bottleneck in three ways:

• By adding more pipes: using modular storage "cell" building blocks that are organized into a massively parallel grid architecture. This means bandwidth can scale linearly with capacity.

• By making the pipes wider: Exadata' s InfiniBand interconnect transfers data five times faster than conventional storage that uses the Fibre Channel interconnect;

• By shipping less data through the pipes: query processing is moved into storage to dramatically reduce data sent to servers while offloading server CPUs.

Loaiza said that customers have a choice of two different Oracle Exadata Storage Servers. The first is based on 300-GB Serial Attached SCSI (SAS) drives, which provides up to 1 TB of uncompressed user data capacity, and up to 1 GBps of data bandwidth. The second server is based on 1-TB Serial Advanced Technology Attachment (SATA) drives and provides up to 3.3 TB of uncompressed user data capacity, and 750 MBps data bandwidth. When data is stored in compressed format, the amount of user data and the amount of data bandwidth delivered by each cell can be increased by two to three times.

Returning The Needle, Not The Hay Stack In another technical overview session later the same afternoon, product director Ron Weiss and product developer Kodi Umamageswaran went into more detail about how Oracle software provides database intelligence to Exadata storage, which allows storage to be tightly integrated with Oracle database apps. Weiss walked through an example of how Exadata Storage Servers uses "smart scans" to greatly reduce the data that needs to be processed by database hosts. In his example, a telephone company wants to identify customers that spend more than $200 on a single phone call and the information about these premium customers occupies 2 MB in a 1-TB table. With traditional storage, all database intelligence resides in the database hosts. This means that all the blocks from the table need to be read, transferred across the storage network, and copied into memory. This generates a large amount of data transfers which consumes bandwidth and impacts application throughput and response time since many more rows are read into memory than required to complete the requested SQL operation. Weiss explained that database operations are handled much more efficiently with Exadata storage because the Oracle Exadata Storage Server returns a query result set rather than entire tables, which eliminates network bottlenecks and frees up database server CPU cycles. Row filtering, column filtering and some join processing (among other functions) can be performed within the Exadata storage cells. A 10x speed-up in query performance is often achieved compared with traditional data warehouse storage architectures, and 50x improvement or greater is achievable, Weiss said, since Exadata "returns the needle, not the entire hay stack." Umamageswaran closed out the session with some best-practice guidelines for migrating legacy storage to the new Oracle Exadata Storage Server, saying it was important that the database server using the Exadata Storage Servers be running Oracle Enterprise Linux and Oracle 11g 11.1.0.7.

For More Information:

LGR Test Drives HP Oracle Database Machine, by InformationWeek's Charles Babcock

Exadata Website on Oracle.com

Oracle Exadata Technology Portal on Oracle Technical Network (OTN)