Hot Topics in High-Performance Analytics - InformationWeek
IoT
IoT
Software // Information Management
Commentary
11/17/2008
10:03 AM
Curt Monash
Curt Monash
Commentary
50%
50%
RELATED EVENTS
[Dark Reading Crash Course] Finding & Fixing Application Security Vulnerabilitie
Sep 14, 2017
Hear from a top applications security expert as he discusses key practices for scanning and securi ...Read More>>

Hot Topics in High-Performance Analytics

For the past few months, I've collected a lot of data points to the effect that high-performance analytics - i.e., beyond straightforward query - is becoming increasingly important. And I've written about some of these topics, including MapReduce, geospatial analytic capabilities and memory-centric analytics among a few others...

For the past few months, I've collected a lot of data points to the effect that high-performance analytics - i.e., beyond straightforward query - is becoming increasingly important. And I've written about some of them at length. For example:

Ack. I can't decide whether "analytics" should be a singular or plural noun. Thoughts?

Another area that's come up which I haven't blogged about so much is data mining in the database. Data mining accounts for a large part of data warehouse use. The traditional way to do data mining is to extract data from the database and dump it into SAS. But there are problems with this scenario, including:

  • There's a lot of data to move.
  • Therefore it's tempting to only sample the database rather than analyze the whole thing, which could have at least a slight negative effect on model accuracy.
  • The result of the process is often some kind of scoring algorithm, and you may want to execute that real-time rather than in batch mode.

Various interesting fixes have been tried.

  • SAS and Teradata are partnering quite closely to run SAS on Teradata boxes.
  • Database management system vendors are building at least the data scoring part right into the DBMS. SAS rival SPSS - which relies more on just-in-time SQL and less on batch extracts anyway - reports that hooking into Oracle's native scoring produces massive performance gains. (To put that another way - I finally got independent confirmation of what Oracle's Charlie Berger has been telling me for years.)
  • Data preparation can be handled by the general ELT/ETLT (Extract/(Transform)/Load/Transform - i.e., in-database data transformation) strategies of the data warehouse DBMS vendors.
  • Oracle (more than most competitors, although SAS/Teradata are headed that way too) actually does all stages of data mining right in the database.

Vendors who are putting considerable marketing emphasis on parallel analytics include:

I'm sure others would say they belong on the list as well. It's an important area of competitive differentiation.For the past few months, I've collected a lot of data points to the effect that high-performance analytics - i.e., beyond straightforward query - is becoming increasingly important. And I've written about some of these topics, including MapReduce, geospatial analytic capabilities and memory-centric analytics among a few others...

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of IT Report
In today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll