iBasis, Burlington, Mass. Growth is a good thing for business, but when it's fast-and-furious growth, as it has been for voice-over-IP (VoIP) network service provider iBasis, it can wreak havoc with the best-laid IT plans. A wholesale provider of international calling services, iBasis carried more than 9 billion minutes of voice traffic last year, but with the VoIP market growing like kudzu, the company's call volumes, and therefore data analysis needs, have been growing 40 percent to 50 percent per year.
A middleman that makes international VoIP call connections between major network providers through the Internet, iBasis is in a high-volume, low-margin business that changes by the minute. "We have lots of decisions to make about network capacity and utilization on a rapid-fire basis," says Paul Floyd, senior vice president of R&D, engineering and operations. "We also need to do analysis of our pricing relative to the cost of providing services."
The problem was that the company's three-and-a-half-year-old data warehouse, built on EMC, Oracle and Sun Microsystems infrastructure, couldn't keep up. It was taking between 18 and 24 hours to load and analyze the data and deliver reports on revenue, margins, network traffic and quality. Thus, managers in finance, billing, operations and sales couldn't make decisions reflecting the latest conditions and demands.
How could a comparatively new data warehouse get so bogged down? "I'm sure it was the perfect solution when it was first implemented, but we're now handling between 140 million and 150 million transactions per day, so our growth rate led our old technology to become slower and slower," says Mark Saponar, vice president of information systems.
When iBasis decided to replace its entire warehousing architecture, it looked into the emerging market of data warehousing appliances, devices that combine hardware, database software and storage in a low-cost package optimized for rapid query and retrieval. These appliances, from vendors such as Netezza and Datallegro, were initially targeted at specialized departmental and large data-mart applications, but as computing horsepower, storage capacities and integration flexibility have grown, the devices now sometimes serve as enterprise data warehouses--particularly when limited sets of analyses are confined to well-defined subject areas.
IBasis selected Netezza's Performance Server model NPS8250 appliance late last year, and added Sunopsis DataConductor software as the ETL solution. Sunopsis actually describes DataConductor as "ELT" software because it extracts and loads data directly to data warehouses. With this bulk approach, Sunopsis says, transformations and updates are handled much more quickly than in the conventional, row-by-row approach in an ETL server.
Deployed in just 12 weeks between January and March, the Netezza-Sunopsis combo has improved performance dramatically. "We are now pushing the transactions into the data warehouse in less than an hour," Saponar says. "The benefits are really the combination of speed of transfer of the information [the ETL process] as well as the very fast [Netezza] appliance for retrieval."
Speed of analysis can have a huge impact on profitability. "If I can get a current view of my margin that's up-to-the-second, it can completely change our view of the network and which provider we're using," Saponar says. Speed is also crucial in monitoring call data reports that spot quality problems. Those reports aggregate an enormous amount of data, but what previously took "several hours to several days" is now "a matter of seconds."
Current sources of data include an Oracle 10g database and flat files, but iBasis plans to use DataConductor to natively connect call billing systems and network monitoring equipment directly to the data warehouse.
"Sunopsis offered both a seamless integration with Netezza as well as the ability to work both in real time and in a batch transaction mode," Saponar says.
While data warehouse appliances such as Netezza's are most often used for data marts under 10 TB, "a few users have deployed enterprise data warehouses on appliances," according to TDWI's Philip Russom, an expert and author of the white paper "Defining the Data Warehouse Appliance." Russom says appliance applications are moving into the 20-TB range.
Where iBasis previously had a 6-TB Oracle database with 2.5 TB of queryable data (plus overhead), it now has a 9-TB Netezza appliance with 4.5 TB of queryable data (or about 8 billion records). The current configuration should meet iBasis's needs for three years given current growth projections, says Saponar. But what happens if the company once again blows by its growth projections?
"We looked at the upgrade path with Netezza, and it's very straightforward," he says. "We can upgrade to a 22-TB device within 24 hours, and it's all done in parallel with current processes."
The transition between data warehouses had no impact on the company's BusinessObjects BI platform, according to Saponar, and given the fast-paced deployment, Saponar says he was relieved: "Usually these types of projects are very painful, but in this case it was pretty straightforward."
Ripping and replacing a data warehouse is not a step you take lightly, but by late last year, fast-growing VoIP network service provider iBasis had hit a wall.
"We had two problems," says Paul Floyd, senior vice president of R&D, engineering and operations. "One was handling the scale of the data warehouse," which had far surpassed growth projections. "The second was pushing data into that warehouse," a task that was taking 18 hours or more.
The company decided it needed an entirely new architecture, and Mark Saponar, vice president of information systems, decided to look into data-warehousing appliances, which promise lower cost (relative to conventional technology) and easy deployment, serving as large, high-volume data marts and, increasingly, focused data warehouses.
The company settled on the combination of a 9 -TB Netezza device with Sunopsis's ETL software, DataConductor. Saponar declined to detail the investment, but the technology lived up its reputation for rapid deployment last March when it went into full production in a 12-week deployment.
Administrative savings have been equally compelling. For one thing, the Netezza appliance doesn't require the support of DBAs, which let iBasis redeploy two full-time employees. In addition, Sunopsis's software, which has a drag-and-drop interface for creating definitions, "doesn't require any technical expertise or special development," Saponar says. Training on the new systems totaled three hours on the DataConductor interfaces and one day for the Netezza device.
Although the old data warehouse was administered by three full-time employees--out of 27 IS employees in the United States--"our data-warehouse team now consists of one to two employees that we need once every three months, to do small changes for release verifications," Saponar says. "The majority of our work is in the BusinessObjects environment to pull information, rather than in managing Netezza or Sunopsis."