A citation gate: every number in published text must be found in the evidence

🔢 The promise

Readers trust a number because it looks measured. If the number came from memory, from a model's guess or from an older run, the page is wrong and looks right. A publication gate answers one question: for each number in this text, can we point to the place in the evidence where it appears, character for character?

🐛 The bug

On 25 August 2026 our verifier judged two true claims "insufficient evidence" while every real number in them was present. The cause was the pattern that finds numbers. The original pattern had two faults.

• No left boundary. Digits inside an identifier counted as numbers: sha256 produced 256, and a date such as 2026-07-26 produced -07 and -26.
• Greedy commas. A comma was accepted inside a number even when it trailed it, so a phrase like "sha256, markers" produced the token 256, that exists in no snapshot.

The checker then looked for these invented tokens in the evidence, did not find them, and rejected true statements.

🧪 The fix, run live

We ship a corrected pattern in a small module with a single definition of what counts as a citation. It refuses to start a number inside a word or a dotted run, and it takes a comma as part of a number only when digits follow. The module also keeps the old pattern for one purpose: to predict which false citations the old verifier would invent for a text.

On 3 November 2026, 21:53 UTC we ran both on one sentence: "Snapshot sha256, markers 2026-07-26 and 1,234 items, 12.5% up."

The corrected pattern no longer invents 256, from the hash and no longer produces minus-prefixed fragments of the date, which exist in no evidence file. It still splits the date into 2026, 07 and 26. That is harmless: each part occurs in the evidence if the date does. It is also a reminder that the gate checks presence, not meaning.

🧰 How the gate works

1. Extract the numbers and quoted phrases from the text. The module recognises four quote styles.
2. For each number, build the locale variants, for example with a comma or a space for thousands.
3. Look for the literal in the evidence files, byte for byte.
4. Report numbers not found, and quotes not found. Any such item blocks publication.
5. Report separately the numbers that the old pattern would have invented, so you can see a risky sentence before it fails.

🛠️ Practice: add a gate to your publishing

1. Choose where your evidence lives, for example a folder of exported results with dates.
2. Write a number finder with a left boundary and a rule that a comma belongs to a number only before digits.
3. Test it on identifiers, dates, version numbers, hashes and percentages.
4. Write the check: every number must occur literally in an evidence file. Fail loudly with the list of misses.
5. Add a place for numbers that are definitions, not measurements, such as "ten minutes" in a title, and keep that list short and reviewed.
6. Run the gate in the publishing step. A failure blocks the release.

✅ Check yourself

• Put a hash and a date into a sentence. The gate does not report the hash fragment or minus-prefixed date parts that exist nowhere.
• Change one digit of a real number. The gate fails and names it.
• Publish a text with a number that has a comma in it. The gate finds the evidence if it is written with the same comma or its locale variant.
• A new teammate can explain what the gate does not prove.

🚧 When this does NOT work

• Present is not true. A number can appear in the evidence and be the wrong number for the claim. The gate checks presence, not meaning.
• Rounded or derived numbers. A percentage computed from two measurements is not in the files. Store derived numbers with their inputs.
• Words for numbers. "Three" is not found by a digit pattern. Prefer digits for measurements.
• Evidence that is itself wrong. The gate trusts its folder.
• Overlong exception lists. If half the numbers are exceptions, the gate means nothing.

📊 Evidence

➡️ Next

Learn how we mark dates and sources in Sources and verification, and why a claim of success is not proof in AI judging.

Sources

• Sources, dates and what “verified” means: how we label evidence.
• AI judging: why “it says it is done” is not acceptance: checking claims.
• Why AI agents make things up, and how to check an answer: checking answers.
• How this wiki is edited and published: editorial workflow.