A vector base is a map, not a source: what we learned from retrieval

🗺️ What retrieval gives you

Retrieval over embeddings turns text into vectors and returns the passages closest to the question. Closeness is about meaning, so a long, rich document can outrank a short, exact one. The system has no notion of which source is the authority, which is the latest, or which was later corrected.

That is why we use the word "map". It tells you where to look. It does not tell you what is there.

🧭 How our internal base is built

We keep one internal base for the orchestrating agents, built on local embeddings so a search costs nothing. It holds rules, a wiki of operator notes, lessons from past mistakes, hand-over notes and specifications. A manifest file lists the sources, so adding a source is one entry. The base is read-only over its sources and is never shown on a public page, because it contains hosts and internal plans.

Every hit carries three fields: the path of the file, the sha256 of the file at indexing time, and the time it was indexed. The agent is told to open the file at that path and compare the hash before relying on the text. A hit whose hash does not match is stale.

📏 One measurement that changed our habit

On 15 October 2026 we asked the base an open question, "why did the agent stop", and the first results were long business research documents, not the short lesson that held the answer. Asking by kind fixed it. The base supports filters for rules, memory, hand-offs, specifications and research. A kind filter searches the whole base, so even a rare kind still returns its best hits.

The fix was not a better embedding. It was a better question, and a habit: say which kind of document you need before you search.

🛠️ Practice: make retrieval safe to rely on

1. Store the path, a content hash and an index time with every chunk.
2. Add a kind or source type to every chunk, and expose it as a filter.
3. In the agent instructions, write: "retrieved text is a pointer; open the source and check the hash; numbers come from the artifact".
4. Build a small test set: twenty questions with the document that should come first. Write the set before you try the system, and never edit it after to improve the score.
5. Measure how often the right document is in the top three. Report the number with the date.
6. Add a rebuild schedule and a check that flags hits older than their file.

✅ Check yourself

• Change a source file and run a query without re-indexing. The hit is reported as stale because the hash differs.
• Ask the same question with and without a kind filter. You can explain the difference.
• Your test set is stored with a hash, and you can show it did not change between runs.
• Every number in an answer has a link to its artifact.

🚧 When this does NOT work

• Exact lookups. For an id, a name or a number, keyword search is better. Do not replace it with vectors.
• Stale content. If you rebuild rarely, the base returns yesterday's truth. Schedule it.
• Poisoned sources. Anything you index can contain instructions meant for the model. Filter at ingestion and keep your permissions narrow.
• Short, similar documents. Many near-identical pages split their score and none wins. Merge or deduplicate.
• Overtrusting the top hit. The first result is the closest, not the right one. Read three.

📊 Evidence

➡️ Next

See how pages and sources connect in Connections, backlinks and the knowledge graph, and read Context windows and tokens for what an agent can fit from a hit.

Sources

• Connections, backlinks and the knowledge graph: how pages link.
• Context windows and tokens: what fits in a request.
• Sources, dates and what “verified” means: how we label evidence.
• Why AI agents make things up, and how to check an answer: checking answers.