Why Recommendation Engines Are About To Get Much Better

Expanding data sources, including social media sources, are making recommendation engines much more powerful.

Ellis Booker, Technology Journalist

September 23, 2013

5 Min Read

5 Big Wishes For Big Data Deployments

5 Big Wishes For Big Data Deployments

5 Big Wishes For Big Data Deployments(click image for larger view and for slideshow)

Amazon.com certainly deserves credit for bringing the term "recommendation engine" into the general lexicon. But recommendation engines have expanded well beyond consumer-facing shopping sites like Amazon as programmatic ways of making accurate recommendations.

Recommendation systems are now common in insurance and finance, where they can suggest coverage plans and investment opportunities, and they are also showing up in B2B settings, recommending prospects and strategies to salespeople.

So-called recommender systems analyze a mass of past choices, from one user or many users, to make new suggestions, or take attributes of one item to suggest new items with similar properties. The first approach is called collaborative filtering; the second is known as content-based filtering. The two methods are often combined.

[ Learn how big data is helping companies make better business decisions. See Law Firm Counts On Analytics For Profitability. ]

ACM RecSys, an annual international conference on recommender systems, will hold its seventh meeting Oct. 12-16 in Hong Kong.

"Companies are learning to capture and learn from more data sources," Dan Mallinger, data science lead at Think Big Analytics, wrote InformationWeek in an email. "As big data supports both structured and unstructured data, companies can understand everything from customer transactions, to call center data, to what's selling on secondary auctions."

Recommendation engines come into play "anywhere options proliferate and search alone does not sufficiently limit results," he wrote.

Figuring out what kind of artwork to suggest to online shoppers is the goal of a home-grown recommendation engine being developed at startup Suede Lane

"We're developing a new way for people to develop their taste without needing to know any industry jargon," founder and CEO Yinka Abdu told InformationWeek. These recommendations will, for instance, help shoppers find, buy and enjoy a painting in the impressionist style without needing to know that term.

Abdu also sees an opportunity to use recommendation engines to educate shoppers. "There's a huge swath of people who want to learn about art or design," she said, imagining a future recommendation engine that might point Suede Lane visitors to educational resources once her website detects -- based on explicit or implicit choices -- an interest in, say, cubism. Recommendation engines aren't only making a difference for consumer-facing shopping sites. Three-year-old SalesPrism from Lattice Engines offers recommendations to B2B salespeople.

"By combining public and company data, we mine thousands of different attributes that are predictive of buying behavior," company director of product marketing Rob Bois told InformationWeek in a phone call.

The Lattice tool, which can be integrated into sales-enablement platforms such as Saleforce.com, has more than 75 customers to date, including Dell. The tool helps a salesperson by recommending prospects on their likelihood of buying, and also suggests which product to promote and talking points.

The application pulls information from a variety of sources, such as Lattice's own Data Cloud service, credit bureaus and a company's own sales and customer databases.

Machine learning then helps find relationships across all the data that a salesperson might miss. Similarly, the analysis has revealed that some sacred cows of sales and marketing aren't very predictive when it comes to determining a propensity to buy.

"B2B marketers think [a prospect's] title is very important ... but it turns out to have very little correlation to likelihood to buy," he said. Better markers for buying proclivity are items that indicate growth, such as new locations, stock movement, press attention and even an increase in the number of job listings on social media sites, Bois said. In the third quarter, Lattice will release a new version of its tool that will expand the information from social media feeds, which will be analyzed over time and combined with other data sets.

But Bois doesn't believe ever-increasing volumes of data are the most important change when it comes to refining recommendation models.

Rather, "democratizing the ability to build predictive models," or letting business users experiment with models and look at their predictive output, will be the game-changer, he said. "That's where I think we're going to see the explosion in predictive model use," he said.

Nevertheless, one barrier to deploying an effective recommendation engine is a lack of data, according to Think Big Analytics' Mallinger.

"Recommendation engines require extensive data on user behavior from a large number of users to generate reliable results," he said. "In a common-but-losing scenario, many of our clients attempt to start data collection only after they decide to build a recommendation engine. This approach typically limits early successes and requires more iterations."

Mallinger said successful companies collect data early and leverage the value of historical records.

Learn more about how IT can influence business decisions by attending the Interop conference track on the Business of IT in New York from Sept. 30 to Oct. 4.

The Interop New York Conference and Expo, Sept. 30-Oct. 4, 2013, provides the knowledge and insight to help IT and corporate decision-makers bridge the divide between technology and business value. Through three days of educational conference sessions, two days of workshops, real-world demonstrations on the Expo Floor and live technology implementations in its unique InteropNet program, Interop New York provides the forum for the most powerful innovations and solutions the industry has to offer. Save $200 off Conference & All Access Passes or get a Free Expo Pass -- register with Discount Code MPIWK.

About the Author(s)

Ellis Booker

Technology Journalist

Ellis Booker has held senior editorial posts at a number of A-list IT publications, including UBM's InternetWeek, Mecklermedia's Web Week, and IDG's Computerworld. At Computerworld, he led Internet and electronic commerce coverage in the early days of the web and was responsible for creating its weekly Internet Page. Most recently, he was editor-in-chief of Crain Communication Inc.’s BtoB, the only magazine devoted to covering the intersection of business strategy and business marketing. He ran BtoB, as well as its sister title Media Business, for a decade. He is based in Evanston, Ill.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights