Why Recommendation Engines Still Aren't Accurate - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics
07:00 AM
Lisa Morgan
Lisa Morgan
Connect Directly

Why Recommendation Engines Still Aren't Accurate

Ever noticed that some recommendations from consumer websites tend to be a bit off base? Lisa Morgan takes a look at why those engines get confused.

Recommendation engines are deeply embedded in American culture. Anyone who shops online, subscribes to a streaming media service, conducts on online search, or uses social media is encouraged to do something -- buy this, click on that, listen to this song, watch that movie. Sometimes the recommendations are accurate. Sometimes they're not.

For example, Google's search engine thinks I'm male. Netflix thinks I might enjoy comedies aimed at college-age men. Amazon's recommendations can be strange if not laughable.

"On some level, these algorithms are amazing, but the types of errors that can be made are foolish," said Patrick Wolfe, professor of statistics and honorary professor of computer cience at University College London (UCL) and Chair of IEEE Signal Processing Society's Big Data Special Interest Group. "It probably wouldn't take a lot of data to teach Google you're not male."

Credit: Pixabay
Credit: Pixabay

Context is Everything

Google and Facebook continue to serve up ads for the fringed boots I bought at Macy's two weeks ago. They're also recommending the infant and toddler car seats I bought as a gift in the same time frame. Clearly, the recommendations lack appropriate context.

"One of the reasons I think recommendation engines aren't as accurate as they could be is that much of machine learning is about making predictions -- predicting the weather or whether a document is about politics or not," said Thorsten Joachims, a professor in the Department of Computer Science and in the Department of Information Science at Cornell University. "A recommendation engine has to be smart about the actions it takes."

Pandora serves up more music that I like than dislike, although my genre preferences, Native American flute music and smooth jazz, represent dramatically smaller universes of choices than classical music or rock and roll. Pandora doesn't tell me anything about local musicians or the local music scene. Granted, that would be an extremely difficult undertaking given the number of cities that consider their local music scene an integral part of their culture -- Chicago, Las Vegas, and Ithaca, New York, for example.

Ithaca College Associate Professor Doug Turnbull and Cornell's Joachims are tackling that very problem. With the help of their students, they developed MegsRadio, a music app that recommends local musicians and local music events. The app is specific to Ithaca's local music scene, although others could benefit from the work if they chose to do something similar in another city, Turnbull said.

I tried the app and created a smooth jazz station. The playlist included some of favorite smooth jazz artists and some very talented artists (presumably from Ithaca) with whom I was unfamiliar.

Solving the Problem Pragmatically

Turnbull and Joachims are using counterfactual machine learning to compensate for the fact that they lack a user base large enough to train the algorithms. Counterfactual learning considers the recommendations an algorithm made, users' historical choices, the accuracy of the recommendations made by the algorithm, and what would happen if a different algorithm were applied in the same circumstance.

"Counterfactual learning is saying that if we don't have new users but we do have what the old algorithm predicted and the probability of the thing it predicted and given what the user historically picked in the past, would this new algorithm be better?" said Turnbull. "It's like an alternate parallel universe."

By comparison, popular search engines, shopping sites, and streaming media companies use A/B testing on a small percentage of users to determine whether one algorithm performs better than another. If the new one performs better than the existing one, the new new algorithm is eventually rolled out to all users. Counterfactual learning enables offline research. Users can be brought in later for validation purposes.

"It's really making these systems much more like an agent that thinks about what the effects of its actions will be so that it can learn from past mistakes," said Cornell's Joachims.

What's Your Take on Recommendation Engines?

As more analytics become prescriptive, recommendation engines will find their way into more use cases. Are recommendation engines helping or hindering you? What improvements would you like to see?

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

11 Things IT Professionals Wish They Knew Earlier in Their Careers
Lisa Morgan, Freelance Writer,  4/6/2021
Time to Shift Your Job Search Out of Neutral
Jessica Davis, Senior Editor, Enterprise Apps,  3/31/2021
Does Identity Hinder Hybrid-Cloud and Multi-Cloud Adoption?
Joao-Pierre S. Ruth, Senior Writer,  4/1/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Successful Strategies for Digital Transformation
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll