Process
How a scan works
One public URL, no JavaScript, no model asked about your brand. The report is the document a non-JS client received.
Two views of the same URL
A browser runs JavaScript and paints a layout. Lumet does not. It reads the HTML a non-JS crawler actually received.
Acme builds widgets
Get started →
Product
Widgets
From $12
Blog
Shipping notes
May 2026
Docs
Quick start
3 min
1<!doctype html>
2<html>
3<head>
4<title>Acme builds widgets</title>
5<meta name="description">
6<script src="/app.js"></script>
7<style>…</style>
8</head>
9<body>
10<h1>Acme builds widgets</h1>
11<p>A short paragraph that shipped in the first HTML response.</p>
12<script type="application/ld+json">
13{@type:Organization}</script>
14</body>
15</html>
One pass, in order
The User-Agent is Lumet/1.0 (+https://lumet.io/process), so the other site can see what hit it.
- 01
Fetch
HTTP(S), 12s timeout, 1.5 MB cap. Redirects are checked hop by hop. Private and loopback addresses are refused.
- 02
robots.txt
Allow and Disallow for * and the AI crawlers below. A missing file is treated as allow.
- 03
Strip JS
script, style, template, iframe, object, and embed are removed. Remaining body text is counted.
- 04
JSON-LD
application/ld+json is parsed. @type values are collected. Invalid JSON is skipped, not repaired.
- 05
Metadata
title, description, canonical, Open Graph, Twitter tags, meta robots, headings. Counts are real. No 0–100 score.
- 06
Sitemap
Sitemap: lines in robots.txt first (up to three), else /sitemap.xml. Lumet does not download every loc.
How much text is enough
After the strip, Lumet counts words. This is the same rule the report uses.
Empty shell
Under 15 words
Invisible. A crawler sees almost nothing.
Thin
15–39 words, no H1
Partly invisible. A little chrome copy leaked into the first response.
Readable
40+ words, or 15+ with an H1
Visible on this check, if robots and the document itself are not blocking.
Crawlers on this path
Longest matching rule wins. Allow ties beat Disallow. This is a practical parser, not a claim of bit-identical Googlebot behavior.
# robots.txt, this path User-agent: * Allow: / User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: /
How the verdict is decided
Missing schema, Open Graph, or a sitemap stay on the report as warnings. They do not automatically downgrade the verdict.
Visible
Readable HTML, not blocked by robots or noindex, and a title is present.
Partly invisible
A named AI crawler is disallowed, or the static text is thin without an H1.
Invisible
The fetch failed, * hides the path, meta robots says noindex, there is no title, or the page is an empty shell.
Out of scope
If a check cannot be done reliably, Lumet omits it rather than faking a widget.
- omit
No JavaScript
No headless browser, screenshots, or visual diff.
- omit
No model queries
Lumet does not ask ChatGPT, Claude, or Perplexity whether your brand is mentioned.
- omit
No site crawl
One URL per scan. Each finished report gets a shareable /scan/[id] link.