Crawl, index, rank: the three-step machine

Every page that has ever appeared in a Google result went through the same three-step machine. Learn the steps and half of SEO's jargon instantly makes sense.

Step 1 — Crawl (the explorer)

Google runs a program that does nothing but follow links, all day, forever. It lands on a page, reads it, notes every link on it, then follows those links to new pages, and so on — hopping across billions of pages.

This is why links matter so much: a page that nothing links to is a shop on a street with no signposts. It barely exists, as far as the explorer knows.

Step 2 — Index (the filing cabinet)

Everything the crawler collects gets processed and filed in Google's enormous filing cabinet, organised by topic, words used, freshness, and hundreds of other labels.

That last sentence surprises people: a Google search never touches your actual website. It searches Google's copy of it. Which means if your page isn't in the index, it cannot appear in results — no matter how good it is.

You can check what Google has filed for your site by searching:

site:yourwebsite.com

Try it with any site. Every result shown is a page in the index.

Step 3 — Rank (the judgement)

Someone types a search. Google looks in the filing cabinet, finds every page that could answer, and then — in about half a second — sorts them by hundreds of signals into an order. That order is the entire game.

The big signals, in plain words:

Why this model is so useful

Every SEO problem is now diagnosable with three questions, in order:

1. 1Can Google reach the page? (crawl problem — broken links, blocked pages)
2. 2Is the page in the index? (index problem — check with site:)
3. 3Is it ranking poorly? (rank problem — content, quality or reputation)

Beginners jump straight to worrying about #3. Professionals check #1 and #2 first, because a page Google never found can't rank no matter how brilliant it is.