Next-Wave Business Analytics

The best hedge against an uncertain future is figuring out how to avoid being surprised when the unexpected happens. Better yet, business executives need to be able to quickly take advantage of changing conditions with new products and services. To accomplish these somewhat elusive goals, companies must constantly improve their ability to identify, classify, and intelligently analyze all available information.

A company's enterprise-information assets--particularly customer data--can be vast, but all too often they're squirreled away in application silos. The marketing department has customer demographics; the accounting department oversees purchase histories, payment frequency, and contract terms; and the customer-service department maintains problem reports.

Web sites are adding to the mounds of customer data that companies have to deal with. Web managers can monitor clickstream log files to identify how customers navigate a site, where they came from, how long they were there, what they purchased, and where they headed afterward.

The goal of high-end business analytics is to turn these individually useful but often marginalized data resources into something that lets business managers immediately grasp the dynamic state of their business. This includes the current and projected status of their customers by group and individual needs. Ideally, analytics lets companies combine demographic and behavioral data with sales information to determine how best to leverage the customer relationship.

A company's ultimate goal is to precisely target new and existing goods to those individuals and groups based on the profiles gleaned from the analytic process. Corporate decision makers need to be increasingly attuned to business opportunities that arise whenever a customer, business, or industry factor changes. Exploiting change is the role of business analytics.

"The Holy Grail of analytics is being able to predict with scientific and statistical validity your return on investment--either campaign by campaign or program by program," says Rod Cook, founder and chief technology officer at Sightward Inc., a Bellevue, Wash., Web-analytics software company.

Most companies have Web-log data that's sparse and discrete and a wealth of transaction data, in some cases going back 20 years or more, that's rich and continuous. The nirvana here, Cook says, is to integrate these data sources in a meaningful way so a company can tell what its customers are doing now and have done in the past. Business analysts can take that data, do a little trend analysis, and decide how best to pitch new products to customers, he adds.

The Liberty Mutual Group in Boston is looking at tools that will help the insurance and financial-services company integrate disparate data to better serve its customers. "We're in the throes of looking at analytics models that will let us integrate all of our Web and call-center customer data," says Rebecca Bearden, a Liberty Mutual Web-brand consultant. Bearden envisions eventually being able to integrate Web, call-center, and even lead-generation information. "Integration is one of our biggest issues going forward," she says.

There are costs associated with integrating all this data. The investment in gathering the data and aggregating it in meaningful ways must yield a quantifiable business benefit. "You can gather all kinds of information, but if you don't have context, if you don't advance a business hypothesis and generate a good strategy, it's pretty much worthless information," Cook says.

Reviewing historical and current data over time can optimally yield enough trend information to feed statistical models that let trained users predict events and trends. However, there's a reticence on the part of decision makers to trust "black box" business models used by consultants and in some software tools without understanding the parameters being measured.

Rightfully so, Cook says. Most of the data is extremely diverse and often in a constant state of flux, "so there's usually no single, specific analytic technique appropriate to your data at a particular stage of its evolution," Cook says. The upshot is that predictive models must be appropriate to the task, highly customized to specific business conditions, and targeted to address specific areas of interest or answer particular questions.

The quality of business-intelligence data is a top-of-mind issue at RollingStone.com in Chicago. "A major hurdle for us was coming up with a solid data model," marketing analyst Chris Costello says. "The challenge is to turn our Web-log data into something useful that lets us detect and predict customer behavior." RollingStone uses online analytical processing to do customer segmentation and cluster analysis, but Costello says this is just a starting point.

Rather than just having high-end modeling at one end of the spectrum and static reports at the other, what's needed is analytics and analytic applications that watch for change and initiate actions at both an individual and a group level. Analytics are most useful when the application proactively lets the right people know when relevant business factors change.

"Change is the major concern in business," says Jeffrey Pease, senior director of product marketing at Business Objects SA, a developer of business-intelligence tools. If a company has 200 things to watch, it can't watch them all at once. What's needed are business analytics tied to sensors and thresholds that can alert managers to the slightest nuance of change.

Business managers, for example, want a tool to alert them that platinum customers are buying fewer goods, or that there's a hemorrhage of platinum customers dropping down to gold, silver, or bronze levels. "I'd like to know that information at the group-aggregate level," Pease says. "I'd like to know if my trends are showing danger signs. I'd also like to know at an individual level who those people are."

One way of spotting trends is to be able to measure just the part of the business that's changing. The future level of a lake can be predicted based on how much water is going in and how much is going out. The same analogy applies to business customers.

Analyst firm Meta Group estimates that half of companies perform daily data warehouse updates, 40% have weekly or monthly updates, and 10% have real-time or near-real-time updates. However, several emerging economic factors led Meta Group to anticipate that many of the companies performing weekly or monthly updates are apt to shift en masse to performing daily or continuous updates as a result of evolving market and competitive conditions.

The need to act upon information is a key driver of high-end analytic applications. Folding business intelligence back into the business decision-making process, operational systems, or human interaction is the primary way to make sure that a company can respond appropriately to changes in customer and market conditions. To bring about this organizational dynamic, the analytic results must be available to all of the people within a company. Traditionally, a lot of information gleaned from a company's business-intelligence tools went to upper management, but it didn't percolate quickly down into the trenches where it could be acted upon by the rank and file.

The Bank of Montreal has a business-intelligence environment that's highly customized for different types of workers. A financial-services manager, for instance, doesn't see the same view of data that the CEO does. System users can navigate to their particular area of interest and responsibility, says Jan Mrazek, the bank's senior manager of business-intelligence solutions.

The bank's system consists of an IBM DB2 data warehouse and the Mine business-intelligence application from MicroStrategy Inc. The goal of the implementation is to give front-line managers better tools to assess how the bank is servicing and retaining customers.

Business analytics is moving beyond data warehousing, which a limited number of experts usually use, to include other components, such as publish-and-subscribe technology to distribute market intelligence to the various employees who need it. If certain events happen, the affected parties are notified in a timely fashion. This represents a kind of opting-in capability for specific kinds of information that helps mid-and low-level decision makers more quickly get the data they need to take action.

Improved search and text-mining techniques are aiding the quest for timely information. Predictive modeling applies in this scenario as well. This form of business modeling can help present information based on particular users' past interests and help predict what a manager might want or need to know in the future.

It's all well and good to have a group of statisticians sitting in their ivory cubicles, and it's quite true that companies still need those people to do the data mining today. But if business intelligence is to be more widely used across the enterprise, people must be able to act upon it in a timely fashion and fold the information back into the business process. Critical information about the state of the business must be distributed quickly, efficiently, and appropriately to those people and departments that can affect the company's adaptability.

These are goals that IT departments have avidly pursued but have hitherto never been able to grasp fully. Fortunately, today's advanced analytics tools point to a time when compiling data, monitoring near-real-time business events, and synthesizing that data via data-mining and other advanced techniques will let companies respond almost immediately to perceived or predicted changes in market conditions. Early versions of these tools already let companies to make business forecasts, optimize resources on the fly, and suggest appropriate actions with unprecedented speed, agility, and accuracy.

"Companies should look twice at implementing traditional business-intelligence solutions and look more toward solutions that deliver analytics at the point of a business process," says Doug Laney, Meta Group's VP of application delivery strategies.

A classic example of this is seen in inventory reordering systems based on supply-and-demand forecasts. Market data is fed back into the system to determine where, when, and how much inventory should be reordered. This type of analysis results directly in a modification of the business processes. The trick is to incorporate this intelligence into both tactical and strategic decision-making with managers making real-time decisions.

The latest challenge in business analytics is to capture external data from sources that companies haven't really considered before. If there's a cliché in the making here, it's that data abounds, but knowledge acquisition takes a lot more work. Many alternate sources of data are available via the Web. The number of customer-data sources continues to expand dramatically. The key will be to determine which of these data points are more relevant and to figure out organizational processes that will permit appropriate data to be fed to the analytics engine so a company or department can respond to it in real time and feed it into ongoing projects, sales efforts, and marketing campaigns.

"That doesn't mean that it has to be complex," says Anne Milley, analytics strategies manager at SAS Institute Inc. It doesn't mean that you have to have neural networks under the hood. It does mean that you need to find a model that best fits your specific circumstances.

Neural networks are flexible models that can be applied to predictive analysis and pattern-recognition problems."You want to control for different factors and see what's working for you and what's giving you the most bang for your buck," Milley says. Well-targeted analytics will provide yield indicators and trends that the company can exploit to its advantage.

"Another trend that we're seeing is a kind of cross-disciplinary awareness," Milley says. Companies that have statisticians with different bases of experience or analysts who are able to make analogies more easily than most of us have recognized that there are large data issues in fields such as genomics. Both genetic researchers and companies with terabyte-level customer-relationship management systems share some common data management issues. Each group could learn from the other and share common analytical approaches, Milley says.

Another way business-intelligence tools are evolving is in interactive analytics, in which users are able to slice and dice data and also carry out what-if scenarios. Instead of driving the enterprise by looking in the rearview mirror, you're looking forward to what might happen and can strategize on how to reach that outcome. Interactive analytics is an area where many conventional business-analytical tools fall short.

"Organizations want a single view of the customer, particularly given current economic trends," says Steve Krause, co-founder and chief technology officer of Personify Inc., a business-intelligence vendor in San Mateo, Calif. "This requires a move away from point solutions to more integrated systems. It's a giant problem to access all of the disparate data sources scattered around the organization. The reason it's a giant problem is that many companies wouldn't even know what to do with it once it's in one place."

This was characteristic of the naivete of early data warehouse projects in the 1990s--many of which failed. "We're seeing much more sophistication around the way high-end business analytics are approached today," Krause says. "We don't really just want to bring a lot of data together; we actually want to work backward from the questions we're trying to answer."

There's also the issue of corporate management developing a level of trust in advanced analytical tools. Being comfortable with driving their business based on associations that aren't easily visible to the human eye doesn't come easily to many CEOs and business managers. As well, many VPs and marketing managers approach marketing as more of an art than a science and are somewhat resistant to analytic technologies.

It's been suggested that insurance companies scrambling to aggregate policy data to estimate losses from the Sept. 11 terrorist attacks were unwilling to trust their business-analytics tools to process real data and instead are using actuaries to extrapolate data from previous natural disasters. Rather than waiting for sufficient data to make accurate projections, companies in other industries are instead making knee-jerk reactions, such as Ford Motor Co.'s increase in on-site inventory and the airlines' layoffs of thousands of workers.

The acceptance problem is twofold. The tools and techniques are still complex and difficult to use. Companies require the guys with the lab coats to make these tools hum. The analytic results that derive from them are often barely auditable, especially when employing things such as neural networks. The sheer sophistication of the tools makes it difficult for business managers to understand how the software came to a particular conclusion. As a result, decision makers often feel uncomfortable implementing results from analysis that they can't audit, figure out what the assumptions are, and how the results were derived.

Trust will emerge when people take a couple of these recommendations, implement them, and see a positive impact on the bottom line. The ultimate outcome of high-end analytics will be systems that can process diverse business data, draw conclusions, and alert managers to proposed actions and outcomes.

Hopefully, the impact on business will be companies that are more agile and better informed about all the conditions both within and outside their corporate boundaries.