ChatGPT, Perplexity and other answer engines fetch pages very differently from your browser. Paste a URL — Lekta fetches it exactly the way they do and grades it A+ to F, with the spec or measurement behind every finding.
5–30 s per audit · 75 s hard limit · no account needed
Specimen reportexample.com/pricing
CPartial · 3 of 4 layers
50 / 80 measured · 20 unmeasured
2 failed3 warnings8.2s
Recency could not be measured, so the grade is capped and marked partial. Unknown is not failure.
Access0.42 · weight 25%
Indexability0.21 · weight 25%
Answerability0.71 · weight 30%
Recencynot measured · weight 20%
Two fetches, one extractor
Indexability · measured 2026-08-16
Every audit fetches the page twice — a plain GET that runs no JavaScript, then a full Chromium render — and passes both through the same extractor. Two fetches, one extractor: that is what makes the two numbers below comparable.
GET · no JavaScript412 chars
What GPTBot, ClaudeBot and PerplexityBot receive.
Chromium render8,930 chars
What your browser receives, through the same extractor.
Ratio0.046
of this page exists in the HTML an AI crawler receives. Major AI crawlers do not execute JavaScript, so the rest is invisible to them.
Four layers, weighted
Score composition
The grade is a weighted composite of four layers: Access, Indexability, Answerability and Recency. Each layer carries the weight shown beside it, and a layer that cannot be measured is withheld from the composite rather than guessed.
01
AccessWhether answer-engine crawlers are allowed to fetch the page at all.
weight 25%
02
IndexabilityHow much content exists in the HTML a crawler actually receives.
weight 25%
03
AnswerabilityWhether the page states answers an engine can lift and cite.
weight 30%
04RecencyWhether the page states when it was last verified. Often unmeasurable — and unknown is not failure, so the grade is capped and marked partial instead.weight 20%
Which AI bots your robots.txt lets in
7 of ~17 product tokens shown
Allowed4of 7 shownSearch and user-triggered tokens can fetch this page.
Blocked3training tokensGPTBot, ClaudeBot and CCBot are disallowed at line 4.
Coverage~17product tokensFull audits decide every token the engine knows.
AI bot access decisions from robots.txt — RFC 9309 §2.2
A rule earns its place in the registry only when it can name its basis. Every finding cites where it comes from: a spec section, vendor documentation, or a dated measurement from the Lekta crawler.
SPECIETF standards · W3C specsStandards text, cited to the exact section.
VENDOROfficial crawler docsVendor documentation, dated at the read.
MEASUREMENTLekta crawler runsOur own measurements, each with the date it was taken.
Rule registryProvisional rules are labelled as such until a dated measurement replaces them.Last registry read ·
Lekta — Can AI read your page? Machine-readability audit