Commentary

Thomas Claburn
 

Google Index Reaches 1 Trillion URLs

Three years after Google declared that its index was three times larger than any other search engine and then declined to cite a specific number to support that claim, it was widely believed that Google had tired of index one-upmanship and that it would no longer be measuring its index.

Three years after Google declared that its index was three times larger than any other search engine and then declined to cite a specific number to support that claim, it was widely believed that Google had tired of index one-upmanship and that it would no longer be measuring its index.Well, Google has its yardstick in hand once again.

Two Google engineers on Friday said that Google's index of the Web now contains 1 trillion unique URLs.


More Internet Insights

White Papers

More >>

Reports

More >>

Webcasts

More >>

That's a lot of URLs. However, there's a lot of chaff in there. Consider that "Google" alone returns 2,740,000,000 Google search results, "Yahoo" returns 2,930,000,000, and "eBay" returns 1,080,000,000. Add up the number of results generated by searching for the top 100 keywords and you'd have a significant fraction of 1 trillion.

In 1998, when Google opened for business, it had 26 million URLs. By 2000, it had reached 1 billion. In 2005, Google claimed it had more than 8 billion Web URLs in its index, at least until it took the index count off its home page. In 2008, Google's measure of the Web is 1 trillion Web URLs.

So it appears that Google's index is exploding with new Web pages. From 2000 to 2005, Google's index grew by a factor of 8. From 2005 to 2008, it grew by a factor of 125.

There you have evidence of the information explosion that Google and other companies are trying to fight through the Information Overload Research Group.

Maybe.

Google software engineers Jesse Alpert and Nissan Hajaj admit that they don't really know how many unique Web pages there are. And they acknowledge that Web URLs are essentially infinite because dynamic page generation for things like future calendar months means there's always another page to crawl. "But we're proud to have the most comprehensive index of any search engine," they say in a blog post.

So forget the math. The numbers are too slippery. Really, this is about bragging rights.


Related Reading




Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

InformationWeek encourages readers to engage in spirited, healthy debate, including taking us to task. However, InformationWeek moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing/SPAM. InformationWeek further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
T-Shirt Giveaway T-Shirt Giveaway: Each week we're selecting one great comment from our readers. The author of the comment will receive an InformaitonWeek Community t-shirt. So get posting!
Subscribe to RSS

Resource Links