Search Wags The Long Tail

Internet search engines not only help consumers find products, but they help make niche products profitable for merchants, according to research conducted by MIT Sloan School of Management Professor Erik Brynjolfsson, who also is director of MIT's Center for Digital Business.

Because more people are shopping online and search costs are declining, obscure products are garnering a larger share of sales today, says Brynjolfsson. And that's driving the creation of more such products.

Brynjolfsson's research demonstrates that the Long Tail -- the cherished belief sustaining many of today's Web startups that there's more wealth to be had selling the mass of niche products than selling the handful of massively popular ones -- has some legs.

The paper that details these findings, "From Niches to Riches: The Anatomy of the Long Tail," was co-authored by Brynjolfsson, Yu "Jeffrey" Hu, professor at the Krannert School of Purdue University, and Michael D. Smith, professor at the Heinz School of Carnegie Mellon University. It appeared in the Summer 2006 issue of the Sloan Management Review.

"For most of the past century, companies of all types strove to introduce products and services that were blockbuster hits and could capture the mass market," Brynjolfsson and his co-authors explain in their paper. "Bigger was better. But now dozens of markets, from beer to books, music to movies, and software to services of all types are in the early stages of a revolution as the Internet and related technologies vastly expand the variety of products that can be produced, promoted, and purchased. Though based on a simple set of economic and technological drivers, the implications of this are far-reaching, for managers, consumers, and the economy as a whole."

Having analyzed consumer purchase data from a retailer that offers uniform prices across both an Internet site and mail-order catalog, Brynjolfsson, Hu, and Smith conclude that while the top 20% of products generated more than 80% of sales for the catalog, the same products accounted for only about 70% of Internet sales. This, Brynjolfsson says, demonstrates that lower search costs can result in more diverse sales.

Amazon.com represents another example of the Long Tail. It sells nearly 3 million titles, compared with the 40,000 to 100,000 titles stocked at the typical brick-and-mortar bookstore. "One might argue that consumers don't really care about the remaining 2.9 million book titles," the paper says. "However, the data paint a different picture. We analyzed Amazon's sales patterns and found that 30%-40% of sales are in books that wouldn't normally be found in a brick-and-mortar store. Moreover, we found that the consumer surplus created by providing access to these relatively obscure book titles exceeds one billion dollars annually."

Another finding of the research is that while consumers can use search tools to find lower priced products, access to a greater variety of products is up to ten times more valuable than access to low prices.

Sellers may see similar benefits, according to Brynjolfsson, when they use search technology to present consumers with products they might not have seen or bought offline. Sellers also benefit from the lower production costs associated with the use of search tools, which may, for example, help producers of goods find lower-priced contract service providers.

While Brynjolfsson, Hu, and Smith argue that the Long Tail makes it "critically important that retailers provide tools to facilitate the discovery of products through both passive and active search," and are generally bullish about the implications of the Long Tail, they do warn that the "accelerated Darwinism created by improved search" could hurt niche artists if the decline in blockbuster hits results in content companies investing less money in artist development.

"There can also be societal and political implications if consumer tastes become balkanized and common experiences become, well, less common," the paper says. "Will democracy and social cohesion suffer if each voting group reads its own custom news feeds and commentary, while experiencing only carefully tailored movies, music, and videos?"