The kingpin of auto-market data and analysis is using Oracle Exadata and BI tools to give new ammo to its customers, from Acura to VW. Here's a look inside the project.
10 Lessons Learned By Big-Data Pioneers
(click image for larger view and for slideshow)
When Polk finally made the decision to upgrade last summer, you could say the move was overdue. By that point the toughest multidimensional queries--those requesting demographic buyer insight within specific regions and dealer zones--were taking as long as three to five minutes.
The first wave of database migrations from a conventional RAC environment onto an Oracle Exadata Database Machine X2-2 half-rack took about four months, according to Miller. An immediate benefit was storage efficiency, thanks to Exadata's hybrid columnar compression. As an example, the Polk Insight database that previously housed about 2 terabytes compressed down to about 600 gigabytes. Performance instantly improved without any Exadata-specific performance tuning.
"Initially we didn't do anything other than migrate the databases, and all the legacy queries and reports started running dramatically faster," Miller said, on the order of 10 times faster, he added.
Polk chose a capacious half rack with consolidation and room for growth in mind. Since the initial wave last fall, Polk has migrated multiple databases totaling about 22 terabytes out of conventional RAC deployments. On Exadata, compression has brought those stores down to about 13 terabytes. Among the tricks the company has employed is careful sorting of data before loading, a step that helps improve database performance.
"Exadata builds storage indexes automatically, and if you can lump the data that goes together, the index knows exactly where to find the data you're after," Miller said. If you sort data by zip code, for example, the database knows just where to find the data needed for a query by geography and it will run that much faster.
Polk is also making extensive use of materialized views, which effectively store often-requested query results for rapid recall. Exadata's compression has helped reduce the size of materialized views and supported more sophisticated views, said Miller, and that has reduced the number of table joins that have to take place in real time. This, too, improves performance, even when exploring multiple dimensions of data.
"If a customer wanted to look across multiple dealer zones and then start bringing customer demographics and data such as lease versus purchase, that would have taken as long as two to five minutes in the old environment," Miller said. "In Exadata these sorts of queries are running in 10 seconds."
Polk's Exadata migration is still in progress, but to date it has consolidated nine separate databases down to four and eliminated eight out of 22 production database servers. Polk has yet to move the other half of its data still in conventional deployments, but it expects to consolidate on a total of 10 servers.
Polk had some 10,000 custom reports built in Oracle Discoverer before the Exadata migration began. With an upgrade to Oracle Business Intelligence Enterprise Edition (OBIEE), Polk is now delivering user-customizable dashboards that used to require custom development work. This has greatly improved productivity at Polk.
Customers now take advantage of OBIEE threshold and alerting capabilities so they can review dashboards when certain conditions are met, rather than having to review reports and look for notable changes.
OBIEE has ushered in new types of reports. For example, the suite taps Oracle 11g-managed spatial information to map data geographically. Thus, manufacturers can see customer and model counts projected onto maps, with five-year trends and competitive insights.
"We can use shading to indicate market-share gains and losses," said Kelly Garcia, Polk's VP of global application development. "They can instantly see whether they are up or down in a particular region."
There are areas where Polk is still taking a wait-and-see stance where Oracle capabilities are concerned. For example, several Polk customers have equipped or want to equip their field reps with Apple iPads for email access and dealer interaction, but Garcia said Oracle's current iPad release is just a start.
"We're unclear whether we're going to go with that product because you can't rebrand the interface, as we may need to do," he explained. "If a customer really needs a way to get our data on the iPad, we can give it to them, but it's not very interactive and we'd like to see more capabilities."
As was the case with Exadata, Polk seems content to communicate what it wants in an iPad release to its incumbent vendor and to wait a bit--say to a 2.0 version--to get what it wants.
In the new, all-digital issue of Network Computing: Microsoft and Citrix are closing the gap with VMware. Before you roll out the latest edition of vSphere, reconsider your virtualization platform. Download the issue now. (Free registration required.)
The Agile ArchiveWhen it comes to managing data, don’t look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyIT’s tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.
Join InformationWeek’s Lorna Garey and Mike Healey, president of Yeoman Technology Group, an engineering and research firm focused on maximizing technology investments, to discuss the right way to go digital.