Answers with sources: practice on one small wiki entry

A good source-linked answer helps another person check the reasoning. It names the document revision, places evidence next to each factual statement and leaves gaps visible. This exercise uses fictional information and requires no connection to a business system.

Prepare a small source

Use this synthetic entry, labelled version 1:

The Demo Catalog publishes a weekly index on Monday at 09:00 UTC. Knowledge Operations owns the index. The entry does not specify a recovery deadline or the location of historical exports.

Save the entry as a local file. Record its SHA-256 and an observation date. A hash identifies the bytes used; it does not establish that the text is true, licensed or current. For a real article, also retain its URL, author or publisher where known, and the relevant permissions.

Ask a question with a boundary

Ask: “When is the index published, who owns it, and how long does recovery take?” Request a compact answer with supporting passages and a separate list of unknowns. Keep the result local. The assistant does not need credentials, private records or a tool that sends a message.

The supported part is the Monday schedule and the owner. The recovery deadline is absent. A fluent answer that promises recovery in an hour fails the exercise, even if that sounds reasonable. A response that quietly omits the recovery question also needs correction: the reader should see what the source cannot answer.

Make a review table

Add the source URL or local path and revision hash to the saved result. A page title alone is a weak pointer because different pages can share it. A search snippet is useful for finding a source, but the reviewer should inspect the relevant passage in the source itself.

Try a changed revision

Create a second local file labelled version 2 and change the schedule to Tuesday. Keep version 1. Ask for a new answer against version 2, preserving its new hash. The old Monday answer may still be an accurate account of version 1; it is stale if presented as the current schedule without a revision label.

When a version-aware API returns a stale-revision response, obtain the new source and review the result again. Do not replace the hash beneath an already accepted answer. This makes the exercise about traceable evidence rather than memorizing one sentence.

Review meaning separately from structure

A validator can check that a file exists, its hash matches and a quoted passage occurs in it. Those checks cannot prove that a paraphrase preserves meaning, that the selected passage answers the question or that important caveats were included. A person checks those points.

Record unsupported statements, important omissions and reviewer corrections. Use the same rubric for assisted and manual attempts before comparing time. One clean synthetic answer is evidence about this exercise, not a production accuracy rate or a certification.

Connect the result to a trace

If the actual retrieval is instrumented, retain its run ID and trace ID beside the source pointer. An authorized operator can inspect the protected Phoenix workspace. A trace helps locate a failed retrieval or an incomplete step; it does not turn an unsupported claim into a supported one. Do not put credentials, private source bodies or customer details into trace attributes.

For this exercise, success means a saved answer that identifies its input revision, supports the schedule and owner, marks the deadline unknown and records the review status honestly. It does not imply that a live connector has been tested.

Wiki Connections

• ai pilot evidence and human review
• permissions and data