Hard To Find
The reality is that people with both IT and statistics skills--people like Ryan Carr at Catalina Marketing--remain rare birds. Back in 2004, when Carr returned to school to earn a master's in statistics from the University of Chicago, he did it for one big reason: fear of outsourcing. "I figured if I could be one of those people who could translate business questions into answers, that'll be tough to push overseas," he says.
The pace of technology change at Catalina shows why businesses see new opportunity to exploit analytics, and thus why it's such a hot job prospect.
Catalina builds predictive models, so a retailer or consumer goods company can predict by what someone has in a shopping cart what else the person is likely to buy. (All Catalina uses is a loyalty card number and shopping history.) Doing that takes analyzing a database of 400 billion rows of item-level data.
In the first year Carr worked on this effort, he and one other person produced just four models, and the computing time to process all that data took eight to 16 weeks. Today, it takes eight to 20 hours, and five people created 700 models last year. Carr's goal, if he can tackle that 400 billion rows of data with in-memory processing, is to lower that processing time to about 10 minutes.
Speed like that will let companies analyze data to make more decisions and take more actions in real time. And companies will need people who understand that computing and statistical modeling capability. Carr warns, however, that the IT and statistics camps can be rivals at some companies. And it can be hard for IT pros to make the leap without formal statistics training. "For an IT person to migrate over to analytics, there is kind of this paper barrier," he says.
