Home
BYTE Newsletter
Keep up with all the BYTE News and Reviews

Subscribe
Paul McDougall

Paul McDougall

Editor At Large, InformationWeek
Editor At Large, InformationWeek

Bing Flunked My Search Taste Test

Comments | Paul McDougall, InformationWeek | September 07, 2012 09:06 AM


Microsoft SkyDrive Vs. Dropbox, Google: Hands-On
Microsoft SkyDrive Vs. Dropbox, Google: Hands-On
(click image for larger view and for slideshow)
Microsoft this week launched a website that lets users input search queries to see results from Bing and Google side by side, without knowing which is which. Users can then vote for the results they prefer. The exercise is meant to support Microsoft's claim that people looking for information like Bing results over Google by almost two to one. That's not exactly how it turned out for me.

I preferred the Bing results just once in five trials. Google won three times, and I scored one test a draw. Overall, I found that Google's results pointed me more directly to what I wanted, and more frequently offered enhancements, such as maps and deep links.

Let's walk through my results.

I'll be traveling to Canada next week to play ice hockey. Whenever I think hockey, I think of my hometown Toronto Maple Leafs, so "Leafs" was my first query term.

Bing's top result was a link to the team's home page, with deep links to stats, rosters, tickets, and the like. Google also first yielded a link to the home page, but no deep links. Bing's second result was a news story about Leafs' captain Dion Phaneuf getting engaged to "24" star Elisha Cuthbert. Google's second result was a Wikipedia entry explaining what a leaf, as in the non-hockey playing plant variety, is. Google 0, Bing 1.

[ A 2011 study gave Bing, not Google, the edge. Bing Beats Google On Search Effectiveness. ]

Next I searched for my name. Bing yielded an outdated byline photo of me at the top, along with images of four other Paul McDougalls (get me Apple's patent lawyers). Google's first result was a link to PaulMcDougall.com, which has nothing to do with me but promises to help you learn the alphabet "with the help of some charming young creatures and their quirky friends."

Google's second result was my professional bio on InfoWeek.com. Bing's second reference was a Paul McDougall on Storyboard.com, who "lives in a world full of quirky creatures, friendly monsters and oddball ghosts." That is actually all true about me, but wasn't me. Bing's results sample didn't find my bio at all. Google 1, Bing 1.

Back to my hockey trip. I'll be flying to Spokane, Wash., and driving to Invermere, B.C., where the game, an alumni match for a team I played for years ago, is to be held. I input "Spokane to Invermere B.C. driving directions." I figured that was a good test, because although Invermere is a beautiful town in the Kootenay mountains, and is right next to the world-class Panorama ski resort, it's not a household name.

Bing only produced links showing distances between seemingly random points in the western U.S. and Canada. Google's first response was what you want when you need directions--a map. It also gave me the precise distance, 271.4 miles. Google 2, Bing 1.

Next, I'll need a place to stay. So I input "Invermere, B.C. hotels." Bing's first instinct was to display a link to Hotels.com's Invermere page. Google's initial results provided direct links to specific inns and hotels, as well as real-time prices. Much more relevant. Google 3, Bing 1.

For my last test--BingItOn.com gives you five rounds per session--I figured I might need some new equipment for the game. I input the specific make, model, and size of the goal pads I want, "Brian's AirPac 34 inch". Serious hockey players know what that means, but neither Google nor Bing had much of a clue. They both yielded links to outdated eBay auctions, old forum postings, and such. I scored it a tie. Google 3, Bing 1, Draw 1.

So, in five blind tests, Google won 60% of the time, with Bing prevailing just 20% of the time. Now, this is of course subjective. Some people might actually prefer search engines that have no clue who they are. But in most respects I'm a typical searcher. When I need driving directions, I want the results with a map.

So let's put some science behind this. If the chances of a random user like me preferring Bing results over Google were fifty-fifty, then the probability of Bing winning only one in five trials is--using the binomial random variable formula, which yields probabilities for specific outcomes, given an underlying probability--just 15.6%. In other words, it's unlikely Bing was just having an off day.

But there's more. According to Microsoft, Bing was preferred by roughly 66% of users--so it shouldn't be a 50/50 proposition. If two-thirds of users really prefer Bing, the probability of it having only a 20% success rate across a set of five random trials reduces to just 4.4%. Hmmm.

What's Microsoft's point behind the site? The company insists that Google has become so synonymous with search that Bing doesn't get a fair shake. And that's why Bing's share of the market is, according to Comscore, just 15.7%, compared to Google's 66.8% share. All Microsoft is saying is give Bing a chance. You can try the test at BingItOn.com.

Let me know your results.

At this year's InformationWeek 500 Conference, C-level execs will gather to discuss how they're rewriting the old IT rulebook and accelerating business execution. At the St. Regis Monarch Beach, Dana Point, Calif., Sept. 9-11.



Related Reading


More Insights




Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

BYTE encourages readers to engage in spirited, healthy debate, including taking us to task. However, BYTE moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing/SPAM. BYTE further reserves the right to disable the profile of any commenter participating in said activities.

COMMENTS

Tune In to BYTE
Facebook Twitter LinkedIn Newsletter RSS
Whitepapers
whitepaper
In this paper you will learn the five trends shaping the future of enterprise mobility. Learn how the rise of social media as a business application, the lurring between work and home, the emergence of new mobile devices, the demand for tech savvy employees and changing expectations of corporate IT will fundamentally change the workplace.
whitepaper
In a survey of more than 1,700 information workers (iWorkers) in North America, notebooks, desktops, and smartphones were found to be “must-have” devices, while tablets, slates, and netbooks were relegated to “nice-to-have” status, according to a commissioned study conducted by Forrester Consulting on behalf of Dell and Intel.
Sponsored by: Dell
Upcoming Events