Freeze what is being judged
Judge a specific version of an artefact against named criteria. Keep the input version or SHA, the scope and the observation time. If the source changes during review, the verdict applies to the old version until the new one is checked. A persuasive explanation is not a substitute for the artefact.
Use four explicit outcomes
PASS means the criterion is demonstrated in the stated scope. FAIL means evidence contradicts it. INSUFFICIENT EVIDENCE means the required observation is missing. NOT APPLICABLE needs a reason. A critical failure or missing mandatory evidence blocks acceptance; an average score or majority vote does not cancel that block.
Verify before consulting
Start with schemas, tests, computations, link checks and receipts. Consult an AI reviewer only where an ambiguity remains. Give independent reviewers the same primary evidence, not a chain of summaries from previous reviewers. Ask for a concise rationale and evidence links, not private internal reasoning. Calibration requires labelled cases, including convincing but wrong answers.
What exists today
AGENTSJUDJE is the public editorial spelling used here for the project’s AGENTJUDGE review method. It is not a claim of certified automated arbitration. In the dated publishing exercise, locked criteria, source hashes, a checkpoint and an authenticated one-use handoff were exercised in a scoped release. The evaluation workshop helps adapt that method to a customer workflow.
Sources and boundaries
Adapted from the project operator wiki and recorded engineering evidence, observed 15 October 2026. Practical recommendations are distinguished from scoped implementation results.
Read the related evidence wiki →