Chapter 45 Building a Cited RAG System from Scratch

45.1 A search result needs a path to a supported claim

For “Beijing lodging,” Chapter 44 ranked the current policy first. An assistant still has work to do: recover the source, preserve the qualifying lines, fit them into the input, produce an answer and connect its claims to evidence. Merely appending a bibliography after generation would not establish that connection.

Retrieval-augmented generation (RAG) uses retrieved external material when producing an answer. This chapter builds the complete local path, with an important producer boundary: the runnable generator is an exact-source extractive composer, not an LLM. It copies eligible evidence and attaches source addresses under explicit rules. This makes every transition inspectable without an account. The generator interface is replaceable; connecting a neural model and measuring its answer quality is an additional experiment, not something replayed or fabricated here. KA-1’s rejected checkpoints remain rejected.

45.2 Parse the source before shortening it

Example 45.1 — Where 750 comes from. ka4-documents-v1 expands the original fictional KA-0 sources into line-addressable bilingual documents. It leaves the old index cards and labels intact. The entire current English document is:

Table 45.1 — travel-v2, version v2, original source lines.

Line	Exact text
1	Fictional Beijing lodging policy; amounts are per room per night.
2	Effective from 2026-01-01; replaces travel-v1.
3	The lodging limit is 750 yuan.

Line 3 supplies the amount, line 1 the location and unit, and line 2 the effective date. The archived source has the same first line, then Effective from 2025-01-01 through 2025-12-31. and The lodging limit is 600 yuan. The sources are authored CC BY-SA 4.0 material, not downloaded company policies.

Parsing turns stored documents into a consistent text representation. Here it preserves UTF-8 text and splits explicit newline boundaries; it does not perform OCR, infer table columns or repair broken PDFs. Chunking gives smaller pieces stable identities. The parser creates one line chunk per source line, retaining one-based line numbers and zero-based, half-open code-point offsets. End offsets exclude the newline. Chinese and English offsets differ, so a Chinese citation must use the Chinese source text, not copied English offsets.

A number-only chunk looks attractive for search but cannot support the whole reimbursement claim. We therefore distinguish the searchable child index card from the parent document’s exact text. The inherited B card is beijing lodging lodging 750, an authored search summary, not a quotation. It points to travel-v2. A hit on that card expands to the parent lines 1–3 for answer context. Individual line chunks remain available for diagnosing a bad boundary or an overly aggressive compression. This small system uses transparent authored cards; scaling it requires testing how cards or text chunks are produced.

45.3 Metadata is part of the evidence contract

The parent record retains source ID, version, topic, validity interval, authority, license and a book:// source URI. The URI is a local source identifier, not a promise that a browser can fetch it. The actual corpus text and digest make the source recoverable. “Current” is evaluated against the explicit request date, not the computer clock or whichever file was indexed last.

Table 45.2 — A request’s journey.

Step	Concrete input → output	What can be inspected
Parse	Bilingual source → exact lines	Original text and offsets
Chunk	Lines and authored card → addressed pieces	Child identity and parent link
Metadata	Source → version/topic/date/authority/license	Valid interval and origin
Index	Same KA-0 cards → canonical term lists	Tokenizer, length and collection statistics
Retrieve	Question plus explicit date/topic → candidates	Full scores and eligible ranking
Pack	Selected parent lines → KA-2 context	Exact serialization and length
Generate	Context → extractive answer object	Named producer and proposed answer
Cite or abstain	Object → checked answer or no-answer branch	Source recovery, applicability and support

The implementation computes BM25 statistics over the whole supplied collection, then applies eligibility before the top-one cutoff. Filtering does not quietly recompute document frequency on one remaining document. After a hit, both experimental conditions inspect siblings with the same topic, authority and effective start; this makes conflicting parallel policies visible. The baseline in Chapter 46 differs only in whether eligibility is applied before selection.

Supported path

1. Text piecesB card → travel-v2
2. Outside the modelLines 1–3: unit, date, 750
3. An operationPack → copy → check
4. Text pieces750 [travel-v2:1-3]

The card is a search summary; the citation quotes the original.

Lost scope

1. Text piecesKeep line 3 only: 750
2. An operationMissing unit/date → abstain

Correct digits are insufficient evidence for the complete claim.

Figure 45.1 — A citation preserves the source conditions with the number. Original teaching diagram · CC BY-SA 4.0

Figure 45.1 tracks 750 together with the information that qualifies it. The source address survives retrieval, context assembly and answer production. The declared input budget is enforced before the generator receives the payload, rather than after losing the date.

45.4 Inspect the actual context and answer

For the English question Beijing lodging, with explicit date 2026-09-14 and topic travel, B is the only eligible candidate. Its BM25 score remains 1.574906. The unfiltered complete score order is B,A,C,D,E as in Chapter 44. Chinese 北京住宿 has the same card scores. The request date/topic is explicit RAG input in this experiment; it was not smuggled into the earlier plain-query ranking comparison.

The actual English context is Chapter 40’s displayed JSON with the final question replaced by Beijing lodging; all other fields and its full three-line evidence remain unchanged. The Chinese context makes the corresponding replacement with 北京住宿. Thus the complete payload is statically reconstructible, including the historical example. Actual compact lengths are 591 and 348 code points respectively. With 200 reserved in a 1,400-unit window, remaining capacity is $1400-200-591=609$ for English and $1400-200-348=852$ for Chinese. These are not tokenizer measurements.

The composer returns this source-grounded text; it is intentionally extractive:

Fictional Beijing lodging policy; amounts are per room per night. Effective from 2026-01-01; replaces travel-v1. The lodging limit is 750 yuan. [travel-v2:1-3]

Its typed object has status: answered, amount_yuan: 750, null reason and a citation containing version v2, start line 1, end line 3 and the exact quote in Table 45.1. The application’s citation check recovers those lines from the versioned corpus. It verifies the interval, unit-bearing parent span, amount and exact extractive answer text. Source existence alone would not pass this check.

The final object repeats evidence in its answer and citation fields. Compact serialization measures 502 code points in English and 346 in Chinese, exceeding the illustrative 200-unit reserve. This is permitted by the local, input-only packing experiment from Chapter 40; it is not evidence of a working model output limit. A model integration must budget the complete desired output, including its structure and quotes, and explicitly handle truncation.

The claim-support relation here is deliberately narrow: one answer passage makes a policy assertion whose amount, location, unit and date must remain together. The checker requires an exact copy covering the annotated support span, which is all three parent lines in this corpus. It does not understand arbitrary paraphrases or general entailment. A model-written equivalent sentence would need a different, evaluated support checker or human adjudication. Rejecting such a paraphrase is a limitation, not proof that it is false.

45.5 Refuse when the evidence path cannot justify an answer

Example 45.2 — Change one condition. Keep the same question but remove version 2 from the available corpus. Version 1 is real within this fictional archive, yet expired for the supplied date. The filtered search has no eligible lodging source. The system returns:

{"schema_version":"ka-answer-v1","status":"abstained","answer":null,"amount_yuan":null,"citations":[],"reason_code":"insufficient_evidence"}

Now add another policy from the same authority and effective date that says 800. The diagnostic corpus labels this deliberately authored contradiction separately. Both eligible versions enter the context. The composer sees unequal amounts and returns conflicting_evidence, not the larger value or whichever source won a score tie.

Table 45.3 — Static failure contrasts.

Changed condition	Observable failure or safe outcome	Why an answer is not justified
Only expired 600 source remains	No eligible evidence; abstain	Archive truth is not current validity
Query is passport renewal	Unsupported-task/no-evidence branch	The collection does not answer it
Equal-authority current sources say 750 and 800	Conflict abstention	Ranking cannot resolve policy authority
Keep only line 3	Incomplete-source abstention	Amount loses unit and applicability
Reduce total window to 250, keep reserve 200	Context-budget abstention	Complete payload cannot fit in 50 input units
Propose 600 with v2 citation	Schema passes; support gate abstains	Cited source contradicts the amount
Propose 750 but cite v1	Citation gate abstains	Correct digits do not repair a wrong source

A refusal because evidence is insufficient is an abstention; it is distinct from a provider safety refusal or technical error in Chapter 41. Our local program also explicitly declines tasks it cannot compose, including treating an order FAQ as live status. The read tool from Chapter 42 remains the appropriate source for that separate request.

45.6 Retrieval at runtime does not update model parameters

If we replaced the extractive composer with a fixed-parameter language model, changing the retrieved policy would change its input, not its parameters. That is the runtime boundary emphasized in Chapter 2. RAG research can also include training. Lewis and colleagues’ original architecture jointly trained a query encoder and BART generator while keeping the document encoder/index fixed, and marginalized over retrieved latent documents. It is not identical to this book’s explicit parse/chunk/pack/cite pipeline. Modern engineering additions should not all be attributed to that one paper.

RAG can still fail: wrong documents, missing qualifiers, malicious evidence, unfaithful composition or unsupported citations can all survive parts of the pipeline. Citation checks reduce a stated class of failures; they do not prove the corpus is true, complete or current in the real world. The fictional authority and dates are declared assumptions here.

45.7 KA-4: run the complete local path

rag.py combines the shared documents, tokenizer, BM25, KA-2 packer/schema and a replaceable generator callable. rag_eval.py exercises the same function with frozen scenarios, including the failure contrasts. 02-retrieval-rag.ipynb executes it on CPU and shows parsed chunks, index terms, candidate scores, context, proposal, support checks and final answer. Saved traces identify local-extractive-v1, not a commercial model or a replayed generation experiment.

The full result is available in data/part-vii/rag-traces.json. Its source digest detects corpus changes. The inherited freeze ensures that adding this corpus did not overwrite KA-0, the learned representation or KA-1 evaluations. A later model integration must record model/tokenizer versions, decoding settings and actual responses, then face the same evidence and task gates. An adapter’s existence alone is not model acceptance.

45.8 Summary

A cited RAG path connects source text to addressed pieces, retrieval, actual context, answer and support checks. Version, unit and date travel with the number. The runnable local composer makes that path inspectable and explicitly abstains when its evidence contract fails. To improve the system, we now need to distinguish which layer caused each failure.

45.9 Exercises

1. A chunk contains only “The lodging limit is 750 yuan.” What else must be retrieved to support an answer about Beijing on 2026-09-14 in yuan per room per night?
2. Two equally authoritative policies with the same effective date disagree. A reranker scores one 0.9 and the other 0.8. Is that sufficient to resolve the conflict? Explain the appropriate output.
3. A generated answer says 750 and cites an existing archived document saying 600. Which of existence, amount correctness and citation support can pass independently?
4. The source changes tomorrow but the index still contains the old card. Describe two stored artifacts needed to diagnose the result. Would retraining the model be the first repair?

45.10 Reference answers

1. Retrieve the parent lines establishing Beijing, the per-room-per-night unit and effective date. A number without these qualifiers is insufficient even when the correct digits are present. The local checker requires all three lines.
2. No. Relevance scores do not establish authority or resolve contradictory rules. Preserve both sources and abstain with a conflict reason, asking for authoritative clarification outside this automatic answer path.
3. Source existence can pass, and 750 can be correct under the current policy, while support fails because that citation says 600 and is expired. A correct answer with a wrong citation is still a failed cited answer.
4. Preserve the original document/version digest and the indexed card/chunk representation with its build provenance; also inspect the retrieval/context trace. First repair the stale index/source path and rerun the fixed cases. The failure does not by itself implicate model parameters.

45.11 References

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, §§2–3 and 4.5. Explains the original learned retrieval/generation architecture and index replacement; our explicit engineering pipeline is a different construction.
• Gao et al., Enabling Large Language Models to Generate Text with Citations, 2023, §§2–3. Separates answer correctness from citation quality; our exact-source checker is much narrower than its benchmark evaluation.
• JSON Schema Validation 2020-12, §6. Supplies the structural contract reused here, without guaranteeing source support.