Deterministic retrieval vs RAG for engineering decisions

When should an engineering team use deterministic retrieval?

Use deterministic retrieval when authority depends on repository state, lifecycle, path scope or a version that a reviewer must inspect. A deterministic query can return the same record IDs for the same repository revision and task inputs, which makes omissions and changes visible. It can also explain the match: a path pattern, capability declaration, dependency edge or service identity.

Use RAG when the collection is broad, the vocabulary is uncertain or discovery across unstructured material is the real problem. Semantic search is valuable for finding a candidate ADR or locating an unfamiliar term. It should not silently decide that a proposed discussion or superseded record governs a change merely because its wording is similar.

• RAG discovers; scoped records establish authority.
• Lifecycle and precedence outrank text similarity for binding answers.
• Pin the repository revision when a citation must be reproducible.
• Expose uncertainty instead of hiding a weak match behind confidence language.

How can deterministic and semantic retrieval work together?

A practical architecture uses semantic discovery as a front door and deterministic resolution as the final gate. The discovery layer can suggest relevant concepts, source links or candidate records. The resolver then filters by active status, owner, repository, path, capability and precedence, returning the records that are actually in scope. The agent receives concise instructions plus stable links, while a human can inspect the full rationale when the result is surprising.

This hybrid is easier to improve than an all-or-nothing search system. Better semantic recall helps people find candidates; better metadata and scope rules improve authority. Neither layer needs to pretend it replaces the other.

How should retrieval quality be evaluated?

Evaluate retrieval with labelled tasks rather than a single relevance score. Build a small fixture set containing accepted decisions, superseded records, rejected proposals, similar wording and deliberately out-of-scope paths. For each task, record the decisions that should be returned, the decisions that must be excluded and the reason for the match. Then test both recall and authority: missing the accepted data-residency decision is a failure, but returning an obsolete proposal as the binding answer is also a failure.

Measure reproducibility by pinning the repository revision and repeating the query. A deterministic resolver should produce stable IDs and explain any difference when the source or scope changes. Semantic discovery can be measured separately for candidate recall, so the team knows whether a problem belongs to search or authority resolution rather than tuning both layers blindly.

• Accepted, proposed, superseded and rejected records in the fixture set.
• Path and capability cases that test deterministic scope.
• A record of excluded candidates and why they were excluded.
• Separate metrics for discovery recall and authoritative resolution.

What should a team record when retrieval changes?

Record the source revision, resolver version, input scope and returned decision IDs whenever retrieval behaviour changes. If a path rule or lifecycle filter is edited, the evidence should show which tasks gained or lost a result and why. This makes a retrieval change reviewable in the same way as a policy change. It also gives an agent a stable citation that can be reconstructed after the repository moves.

Keep semantic discovery separate from authoritative resolution in the evidence. A vector search may find a useful candidate, but the final answer should document the deterministic filters that made a record applicable or excluded it. When the layers disagree, surface the disagreement for review rather than silently selecting a plausible paragraph. The goal is not to eliminate search; it is to make the boundary between finding and governing visible.

• Repository revision and resolver version.
• Task inputs, returned IDs and exclusion reasons.
• Separate discovery candidates from governing records.
• Reviewable evidence for every scope or lifecycle rule change.

The approaches optimise different things

Retrieval-augmented generation commonly uses semantic representations to find material related to a prompt and place it in model context. It is powerful when wording varies widely and the collection is large or unstructured.

Deterministic retrieval applies explicit parsing, fields, filters and ranking rules. Its value is not that it understands every paraphrase. Its value is that a result can be repeated, tested and explained without another model deciding what is relevant.

Why reproducibility matters for decisions

If an agent cites an accepted decision in a plan, a reviewer should be able to run the same query against the same commit and inspect the same record. Stable behaviour also makes regression tests possible: a known deletion question should continue to retrieve the governing retention decision after the corpus changes.

This is particularly important when an empty result must be treated honestly. A deterministic system can report that no classified decision matched; it does not need to generate a plausible answer to fill the gap.

The trade-off

Deterministic retrieval depends on disciplined records and explicit vocabulary. Poorly titled or untagged decisions may not match a natural-language question. RAG can be more forgiving of loose prose and distant semantic relationships.

A hybrid can be appropriate for exploration, but the boundary should remain visible. Semantic suggestions can help a person find candidates; an authoritative answer should still cite validated records and expose how they were selected.

A decision rule

Choose based on the cost of nondeterminism. For a general documentation assistant, flexible semantic retrieval may be ideal. For a merge gate, compliance explanation or reproducible benchmark, prefer a path whose inputs and outputs can be tested directly.