EBay has instituted a utility computing model to better manage the growth of its data warehouse that Oliver Ratzesberger, senior director of architecture and operations, refers to as "analytics as a service."

This service lets authorized eBay employees access a virtual slice of the main data warehouse server where they can store and analyze their own data sets--either in isolation or in combination with core data in the enterprise data warehouse. EBay's virtual private data marts have been quite successful--hundreds have been created, with 50 to 100 in operation at any one time. They've eliminated the company's need for new physical data marts that cost an estimated $1 million apiece and require the full-time attention of several skilled employees to provision.

Virtual marts are often used only for a few days or weeks, so system resources are quickly reclaimed. Users typically introduce less than a terabyte of new data, which they often want to analyze in conjunction with the data in the enterprise data warehouse. If these projects were implemented as separate physical data marts, the required core data would probably be extracted to the data mart, swelling its size, requiring a way to keep replicated data up to date, and multiplying cost and complexity in other ways.

EBay's analytics as a service is a way for people to do "agile prototyping," Ratzesberger says. "They can do experiments quickly and succeed or fail quickly and inexpensively." This helps the company move faster to find and exploit opportunities in connection with Web site optimization, fraud detection, and revenue generation.

When an analytic environment is needed for more than 90 days, the data warehouse team explores whether the user's data ought to be incorporated into the enterprise model.

Illustration by Sek Leung

Return to the story:
Scaling The Data Warehouse Continue to the sidebars:
Microsoft And Oracle Are Scaling Out
and
7 Gotchas That Wreck Data Warehouse Scalability