Virtual private data marts eliminate the need for costly new physical data marts.
EBay has instituted a utility computing model to better manage the growth of its data warehouse that Oliver Ratzesberger, senior director of architecture and operations, refers to as "analytics as a service."
This service lets authorized eBay employees access a virtual slice of the main data warehouse server where they can store and analyze their own data sets--either in isolation or in combination with core data in the enterprise data warehouse. EBay's virtual private data marts have been quite successful--hundreds have been created, with 50 to 100 in operation at any one time. They've eliminated the company's need for new physical data marts that cost an estimated $1 million apiece and require the full-time attention of several skilled employees to provision.
Virtual marts are often used only for a few days or weeks, so system resources are quickly reclaimed. Users typically introduce less than a terabyte of new data, which they often want to analyze in conjunction with the data in the enterprise data warehouse. If these projects were implemented as separate physical data marts, the required core data would probably be extracted to the data mart, swelling its size, requiring a way to keep replicated data up to date, and multiplying cost and complexity in other ways.
EBay's analytics as a service is a way for people to do "agile prototyping," Ratzesberger says. "They can do experiments quickly and succeed or fail quickly and inexpensively." This helps the company move faster to find and exploit opportunities in connection with Web site optimization, fraud detection, and revenue generation.
When an analytic environment is needed for more than 90 days, the data warehouse team explores whether the user's data ought to be incorporated into the enterprise model.
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.