A Brief Outline of the Google Architecture

The very front end of Google is the spider function which creates a queue of pages visit with content for each. This feeds the indexer which matches these pages to existing pages in the index and either creates a new index entry or updates an existing one with the new content. Originally at least, and probably still true I think, the indexing stage is also where link text is propagated from linking pages to destination pages where it is stored as an augmentation to the target page. This is one of — IMHO — the great ideas inside Google as it essentially reduces a key off-page analysis to one that can instead be conducted entirely on-page.

As an entirely separate process that uses the existing index as input, the PageRank algorithm is run to create a "side file" of PR values indexed by the same unique identifier used to access pages. The architecture makes no demands as to when PR is updated — it can be on any schedule they like.

What normal humans, not to be confused with SEOs :-), call "the search engine" is really the "query engine". This takes the index of content and the PageRank values and performs a computation to rank results as each user query is received and processed. Clearly, there can be no pre-processing of queries — there is just raw data to feed into this engine.

So what happens to a brand new page?

A new page doesn’t actually "exist" as far as Google is concerned until it is spidered nd indexed. Once it is indexed, meaning you can find it at Google using a site: query on your domain, it is my best understanding that all Link Reputation effects are baked into the index and fully affect search queries. Since spidering and indexing is the fastest part of Goolge, this explains the similarly fast positioning changes that can be accomplished through link text alone.

PageRank is another matter and can take any amount of time. There is no requirement in the software architecture that requires any particular schedule. In fact, my research suggests that the way PR is computed and the schedule used to compute it are drastically different today than 3 years ago.

While awaiting PageRank values for a new page, it can be very difficult for that page to rank to any significant searches. Likewise, making significant linking topoloy changes in an existing website, using my Dynamic Linking approach for example, will generally take a quite a while to have a ranking impact because of how long it takes for PR values to be recomputed across your site.

Speak Your Mind