To better understand what they want from the combination of BI and search, Ventana Research conducted a survey in early 2007 that asked a series of questions about existing and planned deployments of search, text analytics and BI. A total of 322 respondents qualified for and completed the survey, which was sponsored by vendors Business Objects, Cognos, Hyperion and FAST and conducted among Ventana Research community members and subscribers to media cosponsors including Intelligent Enterprise and DM Review.
This executive summary is aimed at helping companies understand what role search will play in accessing BI insights and in delivering new functionality that expands the business value of that information. Based on the results and our knowledge of the BI and search markets, we also outline eight steps every organization should take to assess and prepare for combined solutions aimed at improving access to information and gaining better insight by synthesizing structured and unstructured data.
Why Combine BI and Search?
Though BI was once the province only of technically astute business analysts, many organizations are now deploying it to less-specialized workers in functions such as sales, marketing, the contact center, the supply chain and other aspects of operations. Broad deployment, however, carries with it expectations about ease of use of technology and processes in providing the information needed to make timely decisions. These new users of BI already are accustomed to searching on the Internet and getting results rapidly by using mainstream search technologies from Google, Yahoo and others.
The ability of search engines to deliver useful results in seconds provokes business users to question why they should have to know what database or report holds the information they want and why they should have to use SQL or vendor-specific query languages, as many BI applications require. Taking note of this attitude, many BI vendors have announced they will incorporate search capabilities into their products. The combination is aimed at the need to make it easier to find existing reports in BI repositories and to simplify ad-hoc query. Within the practitioner community, innovative firms are already implementing combinations of BI and search to integrate quantitative BI data with related, qualitative content and documents (a.k.a. unstructured data) to provide a richer context for decision-making.
What Practitioners Want
BI and search are in the early stages of coming together, yet the study reveals there is significant awareness of the trend. In fact, more than half of respondents said their organizations expect to deploy some type of search capability over the next two years. To put this perspective, let's first examine the respondents and the organizations they represent. Most respondents were located in the United States and they represent all major industries and company sizes, from less than $100 million in annual revenue to more than $10 billion. In the area of business intelligence, the majority of respondents have suppliers Microsoft, Oracle, Business Objects or Cognos as their core system and in the area of text mining and analytics, SAS was the most prevalent supplier.
The respondents rely on a diverse set of suppliers overall, but the market penetration of search vendors is minimal. Most respondent organizations use Microsoft-related search technology. Interestingly, Google finished second, ahead of specialized enterprise search suppliers. There has been substantial adoption of Google Enterprise Search and the Google OneBox among respondents. Some organizations also consider the basic desktop and departmental elements of Google as part of their technology adoption.
Vendor diversity underscores the need to integrate information from disparate sources. Respondents said that they want search to encompass various BI systems. That means suppliers will have to make available indexes to their own content while also being able to process other content. The goal of standardizing on one BI vendor quickly is not always practical, nor did most respondents identify this as a priority.
Benchmarking participating organizations using the Ventana Research Maturity Model, we found that 30 percent of these firms are at the top "Innovative" level of maturity in the use and deployment of these technologies, yet only 23 percent have integrated structured and unstructured data via search.
Companies see a strategic element in the integration of search and BI; a majority of respondents indicated they would first deploy the combination to executive management and analysts so they can make better-informed and timelier decisions. Respondents also envision a new interface for ad-hoc query, with easier querying being a top requirement. As well, 83 percent deemed it important to integrate semi-structured data, such as XML and RSS feeds, and unstructured data such as free-form text found in documents.
What do practitioners expect to discover? More than half the respondents expressed a desire to gain a better understanding of customers and to boost revenue, and 54 percent see the integration of customer-related content and documents with BI as critical. Search and text analytics are generic capabilities that can be applied across an organization, but here again respondents said customer insight would determine the business value of combinations of BI and search.
What To Do Next
Ventana Research believes that the largest impediment to adopting integrated BI and search solutions is lack of knowledge, so the solution is education. Having identified current trends, best practices and aspects of organizational maturity through our research, we offer the following recommendations:
1. Find the opportunities. Many organizations adopt technology without understanding the logical steps that will enable their people and processes to use technology effectively. Innovative companies avoid this mistake because they have moved beyond looking at BI reports and semi-structured data, such as images, in isolation. Instead, advanced practitioners have already integrated structured and unstructured data to gain new insights on customer-facing issues and revenue-driving activities. [For example, leading practitioners are bringing heretofore disparate information together in portals, content- and BI- enabled applications and processes, and in emerging hybrid products in order to provide a richer context for decision-making.]
Midsize and large businesses in any industry can adopt this combination to improve decision-making and provide a superior interface for ad-hoc query. Start by assessing the ability of your BI technology providers to add search, and look for areas where you can use BI- or search-vendor-supplied technologies to extend the value of your investments.
2. Develop an integration strategy for semi-structured and unstructured data. Survey respondents tell us that integrating content and data from outside BI systems is a significant challenge. To properly implement a strategy for search and BI, you will need to draw upon your existing content and document management systems. Assess the levels of integration in place or available, from basic crawling and indexing of content to relating semi-structured and unstructured data to your BI systems. Only 23 percent of organizations have integrated BI with unstructured data; this remains a complex step toward fulfilling the promise of business intelligence. Identify your major content and document assets and then determine how these assets can be integrated with other types of information.
3. Analyze the role of search in performance management. The top-two desired benefits of the BI-search combo, this research found, are the abilities to make better-informed decisions and to make decisions faster - key aspects of performance management. Most organizations also expect BI-search integration to provide improvements in customer satisfaction, lowered costs and increasing revenue - again, all key benefits of performance management. Understanding corporate performance and determining how to optimize it requires relevant information. Examine the intersection of BI and search and look for ways to improve the knowledge of your organization; text mining, for example, might uncover and improve your understanding of trends and exceptions.
The number-one capability respondents want to gain through the combination of BI and search is a single point of access to all information in the enterprise. Access to information is important, but it's only an enabler. Remember that information improves our understanding of performance, but it won't tell you how to optimize and align performance with strategic goals.
4. Use demand for customer information to make a business case. Smart companies recognize that maintaining and extending customer relationships is a necessity for revenue growth. More than half of respondents (54 percent) say the structured and unstructured data they most want to integrate is customer information. Marketing and sales processes were the top-ranked performance management processes they expect will benefit. In addition, organizations that are using metrics to measure the benefit of search ranked customer satisfaction as the top area they track.
While providing a single interface to search information was the single most important capability respondents said they want, technological efficiency alone will not meet most organizations' criteria for approving new investment. However, most organizations are willing to invest in technologies that will improve product offerings and service to customers. To make the best case for the combination of search and BI, identify potential improvements in marketing, sales and contact center organization performance.
5. Look beyond executives and analysts to increase value. Most organizations in the early adoption phase of search and BI capabilities target management and business analysts as their primary audience. This repeats the adoption patterns of early BI deployments in the 1990s. These groups have ad-hoc query - and thus search - needs, but the potential user pool is much larger. Once the investment is made, keep in mind that mid-level operations managers and front-line workers can benefit at the next stage of deployment. Organizations should think ahead and determine how to make BI more accessible by providing a search-style interface to all knowledge workers.
6. Focus on integration rather than standardization. While there are cost savings and other benefits to be realized by standardizing (and consolidating) technologies, integration of existing sources is more important. Search-related technology, in this context, should be able to harvest more than data in one or a few BI tools; ideally, it should be able to index all related data in BI systems and enterprise applications as well as in various forms elsewhere in the enterprise. Organizations that are considering standardization should examine their BI vendor's search capability to ensure that it can access the complete variety of sources - or at least be sure it exposes the metadata and indexes required to search the repository.
7. Consider next-generation search capabilities. Organizations that are looking into the role of search with BI also should look ahead. Search can provide a new interface that will support enterprisewide demand for easier ad-hoc query capabilities as well as the sought-after single interface to all information. At the same time, companies should understand that this trend is leading us back to natural-language query. In fact, the fourth-most-desired search capability identified in our research (by 53 percent of organizations) was natural-language processing. Today, natural-language query is transitioning from keyword searching to the use of fragments and sentences to find information. Part of being innovative is anticipating what's next, so consider developments in this area before you settle on more conventional technology.
8. Consider the role of text analytics. Text has been collecting for years inside documents and other forms of content. Residing in repositories of orders, call center records and contracts, as well as in sales force automation and CRM systems, textual data can yield critical insights about your business. But until the recent evolution of text analytics and text mining technologies much of this information has been impossible to harvest. Today these tools can uncover information that could improve products, processes and compliance efforts. [In contrast to conventional search applications, in which you're looking for content related to specific search terms or phrases, text mining looks for patterns across entire bodies of text to uncover latent customer segments, service problems, trends and opportunities.] In fact, analyzing text was the second-most-sought-after capability (cited by 59 percent of respondents) in our survey. Organizations should look beyond crawling, tagging, indexing and search basics and consider how analyzing text could provide insights that might not be surfaced through conventional BI or search.
Making Final Choice
Many organizations have come to realize that the best technology is the one that integrates with legacy technologies and that enhances existing investments. Consider BI and search technology not just in terms of usability and performance but also in terms of integration with current systems. Assess BI-related products not only on their ability to rapidly query internal reports, queries and metrics, but also on their ability to integrate semi-structured and unstructured data. Look not just at the current version of the product but also at features expected in the next release and the vendor's roadmap for supporting advanced requirements in the future. In addition, investigate whether third-party search partners have added value to the BI system. Speak with early adopters to elicit lessons learned and best practices. Seek independent input and advice to be sure that you have considered all potential scenarios and have chosen the best solution for your needs.
About Ventana Research
Ventana Research is the leading Performance Management research and advisory services firm. By providing expert insight and detailed guidance, Ventana Research helps clients operate their companies more efficiently and effectively. These business improvements are delivered through a top-down approach that connects people, process, information and technology. What makes Ventana Research different from other analyst firms is a focus on Performance Management for finance, operations and IT. This focus, plus research as a foundation and reach into a community of over two million corporate executives through extensive media partnerships, allows Ventana Research to deliver a high-value, low-risk method for achieving optimal business performance. To learn how Ventana Research Performance Management workshops, assessments and advisory services can impact your bottom line, visit www.ventanaresearch.com.
