Data Warehouse Alternatives Make a Hit in India and Europe
Telco Reliance scales with DW appliances. Web marketer TradeDoubler loads and queries faster, cuts cost with a column-store database.
Demand for ever-larger data warehouses and ever-faster access to the data is a worldwide phenomenon. Take the examples of Reliance Communications of India and Sweden's TradeDoubler, a pan-European digital marketing firm. Both companies have replaced legacy data warehouses with alternative technologies that have one thing in common: lower cost and better performance than conventional technologies could offer. Reliance needed a highly scalable solution for call data records, so it's building a massive store on data warehouse appliances. TradeDoubler needed faster load speeds and analytic performance, but it also wanted to spend less time rebuilding and tuning the database, so it chose a column-store database.
The Need for Scale
Data volumes are mushrooming around the globe, and particularly in the telco sector in India. "For the last few months, telecom in India has been the fastest-growing market in the world, adding about 10 million customers per month," says Raj Joshi, Vice President of Decision Support Systems at Reliance Communications, one of the country's top mobile, land line and long distance providers. "We've been adding as many as 1.5 million customers per month, and we were looking for a solution that would help us optimize storage, efficiency and cost."
After an extensive review in early 2007, Reliance chose and implemented a 60-terabyte Greenplum data warehouse appliance last summer. That deployment was successful, so it now has a 120-terabyte appliance Greenplum coming online. All 180 terabytes of capacity will be dedicated to storing and retrieving call data records (CDRs), an application that was previously supported by an Oracle data warehouse. With nearly one billion new calls made every day and government requirements to retain call records for 13 months, the 50-terabyte conventional warehouse was quickly running out of headroom.
"We chose an appliance for the CDR application because it was the fastest-growing piece of our warehouse," Joshi explains. "Access to CDRs in not very frequent, but they need to go in a big database… we needed fast loading and fast retrieval for large amounts of data."
As is common for many first-time appliance deployments, Reliance is essentially offloading a high-volume, data-mart style application from the conventional data warehouse, which continues to support analysis of subscriber demographics and market trends. "Pre-paid [calling cards] account for almost 85 percent of our business, so analysis of recharges, customer user behavior and payment behavior remains in Oracle," says Joshi. "Greenplum was really new technology for us, so the idea was that once the CDR application is proven, we could expand [use of appliances] into other areas."
Reliance is thus far pleased with the Greenplum deployment in two key respects, says Joshi: "I can't comment on our final costs, but the savings were substantial… As far as performance goes, it's about three to five times faster [than our old warehouse], so the queries that were taking a couple of hours now take 30 minutes."
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.