InformationWeek

The spider-or crawler-checks out a home page and the links connected to it. The spider picks up keywords, phrases, and even graphics, and returns those results to the search engine. The spider also ventures out to recheck Web pages on a regular basis-be it weekly, monthly, or every six weeks-to look for changes.

The gathered information is then deposited in the catalog, or index, which contains every page the spider has retrieved. Because of the sheer volume of information, a spider may have gathered information that it hasn't yet had time to categorize.

This lag time can be significant, says Danny Sullivan, editor of Search Engine Watch, an online newsletter. Your Web site report may indicate that your site was probed, but the details may not have shown up yet on a search engine. Have patience.

The last part is the search-engine software. This program uses an algorithm-a code that Webmasters would love to unravel-to determine

where pages are ranked compared with others.

This ranking software is tweaked often by search-engine operators to keep out spam and undesirable sites, and also to respond to the changing needs of the search engine's users.