Lesson 2 / Crawl-to-Rank State Map
Published does not mean searchable
If a page has no ranking, do not edit the title first. Translate the problem into not found, not fetchable, not understood, or not worth serving yet, then identify whether it is stuck at discovery, crawling, rendering, indexing, impressions, or post-click fit. Each failure needs different evidence and action.
The previous lesson's asset map tells you which URL deserves attention first. It does not prove that a search system has found, crawled, or indexed that URL. This lesson clears up that URL's actual state.
If you arrived here directly from search, choose any product, collection, or content-page URL; you do not need the prior lesson first. The lesson translates it into not found, not fetchable, not understood, or not yet fit to show before choosing the first evidence.
State chain
Plain-language model
Think of a search engine as an inspector who does not shop
This module is clickable. Choose one inspection action, then compare what a buyer sees with what a search engine can prove. That keeps indexing, impressions, and click problems from turning into title edits or keyword stuffing.
Not found
Symptom: The page exists in the store admin, but the site has no stable entry path and Search Console has no reliable discovery record.
Check first: Start with sitemap, navigation, collections, related articles, product links, and the discovery state in URL Inspection.
This is not a title-copy problem. Search systems may not know the URL exists yet.
Not fetchable
Symptom: You can open the page in a browser, but search systems may be blocked by robots, login, bad status, redirect chains, or slow response.
Check first: Paste the full URL into the Search Console URL Inspection bar, run live test, then check crawl allowed, page fetch, status code, and redirects.
This is not a keyword problem. First prove Google can fetch the current page reliably.
Not understood
Symptom: The storefront screenshot looks complete, but source or rendered HTML misses core products, copy, pagination, filters, or canonical signals.
Check first: Compare page source, the rendered page in URL Inspection, View crawled page, and the latest visible page-content change.
Publishing more articles will not solve this. Make the core content reliably visible first.
Not worth serving yet
Symptom: The page is crawled or even indexed, but Google sees duplication, thin value, canonical conflict, or no meaningful query scene yet.
Check first: Check Indexing Pages, Google-selected canonical, noindex, duplicate URLs, Search Console Pages / Queries, and competing SERP pages.
Only now discuss page value, intent, internal support, title/snippet, and competition. Do not treat every unindexed URL as a technical bug.
Current inspection
Find the address first
What the buyer sees
A buyer may arrive from home, a collection, an article, or an ad and see a collection page with 12 products, knowing the page exists.
What the search inspector can prove
A search engine does not browse like a shopper. It is more like an inspector who follows sitemaps, navigation, internal links, and external links to find URLs.
Do not misread it this way
Changing the title before the page has entry paths is like repainting the sign while the inspector still cannot find the address.
Core artifact
Crawl-to-rank state map: identify state before action
This state map does not answer whether a URL is “good.” It answers which part of the path it is in now. Read each card in order: identify the state, record the one first piece of evidence that state needs, then choose an action. Crawled does not mean indexed, and indexed does not mean served for a query; do not skip the middle layers and jump to a ranking verdict.
Discovery
This stage asks: Search systems first need to know that this URL exists.
Evidence: Sitemap, navigation, collections, article links, external links, URL Inspection.
The page is orphaned and has no stable internal entry point.
Crawling
This stage asks: Search systems request the URL and read status, HTML, links, and base signals.
Evidence: URL Inspection, robots.txt, HTTP status, redirect chain, server logs.
Blocked by robots, login, error statuses such as 404/500, slow response, or redirect chains.
Rendering
This stage asks: If core content depends on JavaScript, the system needs to see the rendered page.
Evidence: URL Inspection live test, rendered HTML, HTML source, whether key content is visible.
Products, body copy, internal links, or title signals only appear after complex JavaScript.
Indexing
This stage asks: The system decides whether the page is worth putting in the index.
Evidence: Pages report, URL Inspection, canonical, noindex, duplicate content, page quality.
The page is duplicate, thin, unclear, canonicalized elsewhere, or noindexed.
Ranking / serving
This stage asks: Only after indexing does the page compete for specific queries.
Evidence: Search Console queries, impressions, average position, competing pages, internal-link support.
Intent mismatch, strong competition, weak title/snippet, or insufficient internal support.
Click / post-click fit
This stage asks: After impressions, judge whether searchers click and whether post-click behavior continues.
Evidence: CTR, title/snippet, page behavior, add-to-cart, inquiries, orders, refunds, support questions.
The SERP promise is unclear, or page fit, product, price, or trust path breaks.
For a newly published collection, write “discovery to confirm,” not “SEO failed.” Let URL Inspection, internal entry paths, and Search Console state supply the facts before moving it to crawl, index, or serving. That keeps review as one falsifiable judgment instead of a string of simultaneous edits.
Ranking judgment
After indexing, four observable factors show where to improve first
Ranking is not one “edit the title” button. Use intent match, content quality, structural clarity, and trust and usability to connect a result-page symptom to evidence and an action; they guide the next judgment but cannot guarantee ranking or indexing.
Intent match
Ask: Does this page complete the job behind the query instead of repeating the query words?
Observable evidence: Compare the query, SERP result type, page job, and first-screen promise together.
Next action: Name the query scene the page serves, then align its structure and promise to that scene.
Content quality
Ask: Can a reader get a complete, distinctive answer that is useful for a decision?
Observable evidence: Check core facts, distinctive explanation, product or case detail, common questions, and duplicate-page signals.
Next action: Fill the missing answer or consolidate duplication; do not substitute word count for useful information.
Structural clarity
Ask: Can both search systems and readers quickly see the topic, hierarchy, and related entry paths?
Observable evidence: Compare heading hierarchy, descriptive anchor text, internal links, rendered HTML, and click depth.
Next action: Give one page job clear headings, sections, and related internal links, then start with a map of 5–10 important pages.
Trust and usability
Ask: Does the page feel trustworthy and usable, including readable copy and a workable mobile experience?
Observable evidence: Check author or source context, key facts, policies, and contact paths, then test mobile readability, scrolling, and the main action.
Next action: Add trustworthy support, fix unreadable layout and mobile blockers, and write the experience issue as a reviewable metric.
Interactive diagnosis
What state is the URL in? Wrong state means wrong fix.
Click different symptoms. This module is not about memorizing the pipeline; it trains you to turn no ranking into a verifiable state: what evidence to inspect, where the blocker likely sits, and which layer to fix next. After clicking, put the diagnosis into the copyable lesson notes below.
Diagnosis
Google does not know this URL
Look first: Sitemap, internal links, navigation, orphan status.
Likely blocker: No entry path, or the page is not submitted in a stable URL set.
Next action: Add internal links and sitemap coverage, then set a review date. Do not edit title first.
Worked URL example
A new collection has no organic traffic after 10 days. First prove which layer is stuck.
An indexable asset is a page with a distinct user job that can be discovered, understood, indexed, and used for a query; in Shopify, it is often a collection, product page, or article. A page loading in a browser only proves that a shopper can visit it—not that a search system has seen, understood, or chosen to show it.
Case: summer travel tumbler collection
Ten days after launch, the team says “SEO is not working.” It still does not know whether the collection was discovered, crawled, fully rendered, indexed, or indexed without entering a meaningful query. Rewriting the title or publishing a similar article now may only hide the real blocker.
Result interpretation: ten days is a signal to inspect, not a failure verdict. Until the state is known, “no traffic” cannot be assigned to keywords, technical setup, or content quality.
| Stage / symptom | Where to look | Do not do first |
|---|---|---|
| Discovery: the new URL has no stable entry path. | Sitemap, navigation, relevant collection, article, or product-page links. | Do not only submit a sitemap or edit the title; add one traceable entry path first. |
| Crawl / render: the browser loads, but the live test, status, or rendered content is wrong. | URL Inspection, robots.txt, HTTP status, redirects, and rendered HTML. | Do not treat a shopper screenshot as proof that search systems saw the content. |
| Indexing: crawled but not indexed, or canonical / noindex / duplicate signals conflict. | Search Console Pages, URL Inspection, canonical, noindex, and duplicate URLs. | Do not publish more similar pages; first decide whether to strengthen, merge, canonicalize, or noindex. |
| Serving / clicks: indexed with no impressions, or impressions with weak clicks. | Search Console Performance queries / impressions / CTR, the SERP, internal links, and first screen. | Do not blame the sitemap or keep changing technical settings; now inspect intent, snippet, and whether the page helps the visitor continue. |
The interactive choices above only practice diagnostic order; they do not read your Search Console. It becomes a reviewable SEO record after the page label or non-sensitive path, state, evidence, next action, owner, and review date are written into the notes below.
Evidence sources
Each source answers a different question. Do not call everything no ranking.
URL Inspection
The indexed version, live test, page fetch, crawl allowed, rendered page, and canonical status for one URL.
A passing live test does not mean the page is already indexed, ranked, or receiving traffic.
Sitemap
Whether important URLs were submitted to search systems.
Does not replace internal links or guarantee indexing.
Internal links / navigation
Whether the page has a site entry and is treated as important.
Does not alone prove the page is high quality.
robots / noindex / canonical
Whether crawling, indexing, or canonical signals block the page.
Does not directly explain post-click conversion issues.
Queries / impressions / CTR
Which search contexts the page enters and whether people click.
Does not alone prove order quality or profit.
GA4 landing page / ecommerce events
When the Google tag and relevant events are correctly implemented, read landing-page, add-to-cart, and purchase evidence after the click.
Does not prove that the URL is indexed, entered a query scene, or has search demand; return to Search Console and URL Inspection for those questions.
Beginner flow
Crawl to Click Flow: inspect each layer from URL discovery to user clicks
This puts URL Inspection, sitemap, canonical, and rendered HTML on one path. In Search Console URL Inspection, paste the full URL; indexed version is Google’s stored view, while live test only checks whether the current page may be crawlable and eligible for indexing. Click a step to see the beginner misread, the action to take, and the evidence to copy. Do not collapse these tools into one vague “Google gave no traffic” problem.
Current step
Discovery entry
Beginner misread
Assuming sitemap submission means Google will prioritize crawling and indexing.
What to do
Confirm the URL is in the sitemap and give it stable internal links from home, collection, article, or product pages. A sitemap cannot replace internal links.
Evidence to copy
Copy: sitemap submitted/discovered state, entry page URL, anchor text, click depth, and orphan-page status.
Saveable practice
URL state lab: read the state clearly before deciding who investigates next
This turns abstract crawl, render, indexing, and serving ideas into a page-state record. Selections only update the exercise. They do not read your Search Console, visit a real URL, submit a sitemap, request indexing, or change a page. A real investigation still needs the appropriate owner to read back the same page in the current system.
Page-state record
First say which layer the page is in, then choose the report to read
Do not enter a full domain, query parameters, account, credential, order, customer, or query export. Use a page label or non-sensitive path with an evidence summary, acceptance metric, owner, review date, next action, important-page map, and candidate pages.
Crawl-to-serving state practice
This is a practice flow, not a live Google crawler or account screenshot. The current scenario highlights the layer to read first.
Search systems first need to know that this URL exists.
Search systems request the URL and read status, HTML, links, and base signals.
If core content depends on JavaScript, the system needs to see the rendered page.
The system decides whether the page is worth putting in the index.
Only after indexing does the page compete for specific queries.
Current page scenario
Crawled - currently not indexed
Signal: Google crawled the page but is not currently keeping it in the index. Separate duplicate, canonical, noindex, page-job, and content-value checks first.
Read first: Read the Page indexing report reason first, then use URL Inspection indexed version for Google-selected canonical and latest crawl information.
Boundary: Do not immediately rewrite at scale, publish more near-duplicates, or write a passing live test as indexed status.
Next research: Have the page owner compare page job, duplicate URLs, and primary-version signals before deciding to strengthen, merge, canonicalize, or keep it unindexed.
Search Console evidence landmarks
Search Console > Indexing > Pages
Read status reason, scope, and source first. Example URLs in the report are limited, so an unlisted URL is not verified by absence.
1. First choose page-level or site-level question
Use Page indexing report for status scope and reasons; use URL Inspection for one complete URL.
2. Then separate stored version from current test
Google Index reads the most recently indexed version; Test live URL reads a current fetch test.
3. Open page output when render detail is needed
Use View tested page or View crawled page to compare screenshot, HTML, resources, and JavaScript output.
Current review gate
Start with a page label or non-sensitive path; do not enter a full domain, query parameters, account, or credentials.
Choose the next move, then read the wrong-choice explanation
Backend evidence paths
Turn crawl, understanding, and ranking into reviewable fields
The state map is not done when it is drawn. Each state must point to backend paths and fields, otherwise the team falls back to editing titles whenever ranking is weak. Use this table in your notes: write the backend surface, fields, what it proves, and the conclusion it does not support.
State 1
Not discovered / weak entry path
Search Console -> Sitemaps; URL Inspection; Shopify Online Store -> Navigation; collection, hub, and product-page internal links.
Fields to record
- URL, sitemap submitted / discovered state, and last submitted time.
- entry page URL, anchor text, click depth, and orphan-page status.
- whether it appears in navigation, collections, related products, or related articles.
Can prove: Proves whether search systems and the site structure have a real path to discover the page.
Do not conclude: Misreading an entry-path problem as a title problem, or stopping at sitemap submission.
Next route
Add internal links and sitemap records first; if entry paths are messy, move into Technical SEO advanced crawl budget / URL governance.
State 2
Unstable crawl or render
URL Inspection -> Live test; server logs; page source / rendered HTML; robots.txt; redirect chain; theme template.
Fields to record
- HTTP status, robots allowed / blocked, redirect target, and crawl time.
- whether body copy, products, pagination, filter links, and canonical are visible in source or stable rendered output.
- current page version, latest visible change, and failing URL sample.
Can prove: Proves the page is not only browser-accessible for users, but also readable for search systems.
Do not conclude: Assuming crawl is fine because the page opens in a browser; treating rendering failure as a keyword problem.
Next route
Fix robots, status, redirects, template output, and core-content visibility before moving into technical SEO basics.
State 3
Crawled but not indexed
Search Console -> Indexing -> Pages; URL Inspection; canonical / noindex / duplicate checks; content and collection-page job table.
Fields to record
- index status, Google-selected canonical, user-declared canonical, and noindex state.
- duplicate-page URL, primary URL, page job, and whether it deserves a standalone URL.
- strengthen / merge / canonicalize / noindex / remove decision and review date.
Can prove: Proves whether the issue is page value, duplication, canonical signals, or Google choosing another primary version.
Do not conclude: Treating every unindexed page as a technical failure.
Next route
Decide page survival and primary version first; complex parameters, pagination, and duplicate issues belong in Technical SEO advanced.
State 4
Indexed but weak impressions / clicks
Search Console -> Performance -> Search results; Pages / Queries / Countries / Devices; manual SERP review; GA4 landing page.
Fields to record
- URL, query, impressions, clicks, CTR, average position, country, and device.
- SERP page type, title link, snippet promise, competing pages, and current ranking URL.
- landing page engagement, add_to_cart, purchase, and support question.
Can prove: Proves whether the page enters the right query scenes, whether searchers click, and whether post-click fit holds.
Do not conclude: Collapsing low impressions, low CTR, and low conversion into one ranking problem.
Next route
Low impressions go to keyword basics; low CTR to title/snippet and page promise; low conversion to CRO / PDP / pricing paths.
Site stage
New sites and established sites need different first checks
“New / established site” only chooses the first evidence source; it is not an excuse for a URL. A new site first proves that core pages have stable entry paths and crawlable versions; an established site first looks for conflicts among similar URLs, historical versions, or internal-link structure. At either age, the current URL state and evidence matter more than the site’s age for the next step.
New site
Focus: Stable discovery, clear structure, base page quality, Search Console setup.
Wrong move: Expecting mature-site ranking speed.
First check: Whether core pages have entries in navigation, collections, sitemap, and links.
Established site
Focus: Duplicate content, old structure, low-value pages, canonical signals, internal competition.
Wrong move: Solving every traffic issue by adding more pages.
First check: Which pages should be strengthened, merged, redirected, noindexed, or removed.
Debug example
The same no ranking complaint can mean four different problems
Treat each row as a diagnostic starting point, not a final cause. Perform that row’s first check; only move to the next layer when evidence rules it out. This keeps “no ranking” from turning content, technical setup, internal links, and titles into simultaneous guesses.
A new product page has no traffic after two days
First check discovery, sitemap, and collection links. No traffic after two days does not mean the page failed.
The page says Crawled - currently not indexed
Check duplication, thin value, and canonical conflict before publishing more similar pages.
Indexed but no impressions
Return to keyword and page job: which search scene should this page enter, and does the site support it with links?
Impressions but low CTR
Now inspect title link, snippet, first-screen promise, and result-page competition instead of blaming sitemap.
Quick check
A page has no ranking. What comes first?
This check blocks a bad reflex: editing the title whenever ranking is absent. Choose the action you would take, read the feedback, then decide whether to move into keyword basics.
Choose one action. This check prevents the reflex of editing title whenever ranking is absent.
Stop / Go
Go rules for search-state diagnosis
Completion here is not fixing every URL. It is leaving one core URL with a state, evidence, owner, and review date. Only then will the next lesson’s keyword and page work avoid hiding a technical or indexing problem.
Pause
Editing title before confirming index state.Submitting sitemap without internal links.Assuming crawled means indexed.Blaming technical SEO for indexed pages with no impressions.No review date or responsible lead.
Go
- State is identified first.
- Every core URL has an evidence source.
- Action matches the failed stage.
- Review date and responsible lead are written.
- Post-click issues are routed to page, product, pricing, or trust review.
Copyable lesson notes
Turn this lesson into copyable URL state diagnosis notes
The notes below update with the URL symptom and quick check you selected. They preserve the current state and first proof. Copy them into keyword basics so keyword, page, and content work does not fix the wrong layer.
Search-state judgment: published does not mean searchable. First decide whether the problem is not found, not fetchable, not understood, or not worth serving yet, then locate discovery, crawling, rendering, indexing, impressions, or post-click fit. Search inspector current check: Find the address first; first proof: First check whether the URL appears in the sitemap, navigation, collections, related articles, or product links. Current URL symptom: Google does not know this URL. First evidence: Sitemap, internal links, navigation, orphan status. Likely blocker: No entry path, or the page is not submitted in a stable URL set. Next action: Add internal links and sitemap coverage, then set a review date. Do not edit title first. Crawl to Click Flow current step: Discovery entry; tool: Sitemaps + internal entry paths; evidence: Copy: sitemap submitted/discovered state, entry page URL, anchor text, click depth, and orphan-page status. URL state lab: Crawled - currently not indexed; evidence surface: Search Console > Indexing > Pages; next research: Have the page owner compare page job, duplicate URLs, and primary-version signals before deciding to strengthen, merge, canonicalize, or keep it unindexed. Quick check: not selected yet. Complete the check before moving to keyword basics. Page label/non-sensitive path: Not filled Evidence summary: Not filled Acceptance metric: Not filled Request indexing request date (fill only after the fix): Not requested Owner: Not filled; review date: Not filled; next action: Not filled Important-page map (5–10 lines): Not filled Candidate pages (at least 3 orphaned, duplicate, or low-value lines): Not filled Record readiness: fields still incomplete; fill the record before review Backend evidence fields: URL Inspection, Sitemaps, Indexing Pages report, Live test, server logs, page source / rendered HTML, robots.txt, canonical / noindex, Search Console Pages / Queries, and GA4 landing page.