InformationWeek: The Business Value of Technology

InformationWeek: The Business Value of Technology
InformationWeek - Our New iPad App
News

June 12, 2000

Printer ready
Printer ready

Anatomy Of A Search Engine

By Larry Kahaner

In their simplest form, automated search engines are composed of three pieces: the spider, the index, and the search-engine software.

The spider-or crawler-checks out a home page and the links connected to it. The spider picks up keywords, phrases, and even graphics, and returns those results to the search engine. The spider also ventures out to recheck Web pages on a regular basis-be it weekly, monthly, or every six weeks-to look for changes.

The gathered information is then deposited in the catalog, or index, which contains every page the spider has retrieved. Because of the sheer volume of information, a spider may have gathered information that it hasn't yet had time to categorize.

This lag time can be significant, says Danny Sullivan, editor of Search Engine Watch, an online newsletter. Your Web site report may indicate that your site was probed, but the details may not have shown up yet on a search engine. Have patience.

The last part is the search-engine software. This program uses an algorithm-a code that Webmasters would love to unravel-to determine

where pages are ranked compared with others.

This ranking software is tweaked often by search-engine operators to keep out spam and undesirable sites, and also to respond to the changing needs of the search engine's users.

Return to main story, "Content Matters Most In Search-Engine Placement."

Back to This Week's Issue
Send Us Your Feedback
Top of the Page

Get InformationWeek Daily

Don't miss each day's hottest technology news, sent directly to your inbox, including occasional breaking news alerts.

Sign up for the InformationWeek Daily email newsletter

*Required field

Privacy Statement



This Week's Issue

Technology Whitepapers

Featured Reports







Video