Check what a website really serves before you call it empty

👀 The mistake

On 9 September 2026 we concluded that a company-data site had "no useful data". The conclusion came from reading one page, the home page. The same site served a ranking of the top thousand entries with sources and a catalogue of 56,138 profiles. The home page did not mention either number in a way a quick read would catch. The verdict was wrong, and it was said with confidence.

The error is general. A home page is a story the site tells. Whether the site serves what it claims is a question for the pages behind it.

🔬 What a surface check does

We replaced the habit with a command. For a domain, it reads the sitemap and the robots file, visits the addresses they list and records, for each page, the status and the values defined in a contract. A contract lists named values and the expression that reads them from the visible text, with scripts and markup removed. A value can be a printed number or a count of matches, such as how many links of a kind are on the page.

The result is written to a file for that domain, and the command exits with an error code when any value breaks its rule. Severity has three levels: info is reported, warn and error fail the run.

Today the repository holds contracts for 7 surfaces, and the tests for the checker number 21.

⚖️ The second rule: 200 is not data

A page that answers with status 200 can still be empty. We once saw every section of a dashboard answer 200 with zeros, because the underlying snapshot was empty. Acceptance is by content, for example a count greater than zero, not by status code. The same value on two pages must have one definition. If two pages show different labels for the same quantity with no link between them, that is a defect to report.

🛠️ Practice: write your first contract

1. Pick one site and list five facts it claims: number of products, number of rankings, last update date.
2. For each fact, find the page that should show it and write a regular expression that reads the number from the visible text.
3. Add a rule: the value must exist and must be above a threshold you choose.
4. Write a small script that fetches the sitemap and the robots file, visits each page once with a polite delay, and applies the contract.
5. Save the result with the run time. Never overwrite it silently. Keep the last run for comparison.
6. Decide which failures are warnings and which are errors, and make the script return the right exit code.

✅ Check yourself

• Break a rule on purpose, for example by pointing to a non-existent page. The script exits with an error and the saved file shows which value failed.
• The result file lists every address it visited and the status of each.
• Two pages that show the same quantity give the same number.
• A teammate can reproduce your verdict by running the command, not by trusting your description.

🚧 When this does NOT work

• Pages rendered by script. If the numbers appear only after JavaScript runs, a plain fetch will not see them. Use a rendering step or a data endpoint.
• Login walls. A check cannot judge what it cannot reach. Say so in the report.
• Changing layouts. A regular expression breaks when the wording changes. Review contracts when pages change.
• Heavy sites. A crawl of tens of thousands of pages needs a budget and a delay. Sample, and say that you sampled.
• Meaning. The check proves that a value exists. It does not prove that it is true. For truth you need the source.

📊 Evidence

➡️ Next

Read how we label sources and dates in Sources and verification, and how to keep claims tied to evidence in AI judging.

Sources

• Sources, dates and what “verified” means: how we label evidence.
• AI judging: why “it says it is done” is not acceptance: acceptance by checks.
• Why AI agents make things up, and how to check an answer: checking answers.
• Connections, backlinks and the knowledge graph: how pages link.