Chapter 41 Structured Output: A Machine-Checkable Data Contract

41.1 A reader understands “750 yuan.” What should a program receive?

The evidence from Chapter 40 supports 750 yuan per room per night. A human can read that sentence. A program adding the amount to an expense calculation needs a more definite answer: which part is the number, which unit applies, what source supports it, and what happens if no amount is known?

Searching a sentence for the first digits is a tempting baseline. It fails on “The old 600-yuan limit was replaced by 750.” It also turns “750 is not supported” into 750. The problem is not merely typography. We need a data contract: a representation whose shape a machine can check, plus separate checks for meaning and evidence.

Chapter 29 described how scores become output tokens. An instruction to “return JSON” influences that process; it does not by itself constrain every possible continuation. This chapter separates asking for a shape, enforcing a shape during decoding, and checking the completed result.

41.2 Read a JSON object one field at a time

Example 41.1 — A typed answer with evidence. This authored target object refers to the fictional current policy. Its amount unit is fixed by the contract: yuan per room per night.

{
  "schema_version": "ka-answer-v1",
  "status": "answered",
  "answer": "750 yuan per room per night.",
  "amount_yuan": 750,
  "citations": [{
    "source_id": "travel-v2",
    "version": "v2",
    "line_start": 1,
    "line_end": 3,
    "quote": "Fictional Beijing lodging policy; amounts are per room per night.\nEffective from 2026-01-01; replaces travel-v1.\nThe lodging limit is 750 yuan."
  }],
  "reason_code": null
}

The outer braces contain an object, a collection of named fields. A colon joins a field name to its value; commas separate fields. Names use double quotes. A quoted value such as "answered" is a string, a sequence of characters. The unquoted 750 is a number; "750" is a string containing digits. A parser should not silently treat them as interchangeable.

The square brackets after citations hold an array, an ordered list. This list has one object, whose line numbers identify a range inclusive of both endpoints. null is a special JSON value meaning no value here under our contract. It is not the string "null", not zero, and not an omitted field. JSON also has the Boolean values true and false; they represent two logical states, not the amounts 1 and 0.

A parser turns JSON text into program values. It rejects an unfinished {"amount_yuan": or a trailing comma. Parsing alone still accepts {"amount_yuan":"750"}. Both the question “Is this JSON?” and the question “Is this the object our consumer requires?” need answers.

41.3 A schema specifies shape; the application specifies consequences

A JSON Schema describes allowed values. We use the explicit Draft 2020-12 dialect, rather than assuming every schema system supports the same keywords. The following core fragment can now be read field by field:

{
  "type": "object",
  "properties": {
    "schema_version": {"const": "ka-answer-v1"},
    "status": {"enum": ["answered", "abstained", "refused", "error"]},
    "answer": {"type": ["string", "null"]},
    "amount_yuan": {"type": ["integer", "null"], "minimum": 0},
    "citations": {"type": "array"},
    "reason_code": {"type": ["string", "null"]}
  },
  "required": ["schema_version", "status", "answer", "amount_yuan", "citations", "reason_code"],
  "additionalProperties": false
}

type restricts the kind of value. An integer has no fractional part. minimum rejects negative amounts. enum means choose exactly one listed value; "success" is not "answered". const allows exactly one version label. required means the named field must exist; it does not mean the value cannot be null. additionalProperties: false rejects unexpected fields, making a misspelled ammount_yuan visible instead of quietly ignored.

JSON Schema tests the mathematical value: 750.0 also qualifies as an integer. After validation, our adapter stores such values as Python integers without changing their value. This does not permit converting the string "750", or the Boolean true, into an amount.

This fragment is not the complete validator. The shared schema additionally validates each citation’s fields and positive line numbers. It uses a conditional rule: if status is answered, answer must be nonempty, citations must contain at least one item, and reason must be null. Otherwise, answer and amount must be null, citations empty, and reason a nonempty string. An application then rejects a line range whose end precedes its start. Source existence, exact quotation, date applicability and claim support remain separate checks.

Table 41.1 — Similar-looking responses fail at different gates.

Change to Example 41.1	First outcome	Why
Remove the final closing brace	Parse failure	The text is incomplete
Replace number 750 with string "750"	Schema failure	Wrong type
Remove amount_yuan	Schema failure	Required field missing, even when unknown
Add approved: true	Schema failure	Unexpected property; it would not grant approval anyway
Set citation start to 3 and end to 1	Application failure	Both integers are positive, but the interval is reversed
Set amount and answer to 600; leave v2 citation	Structural acceptance; support failure	The schema cannot know which policy amount is true
Return abstained, null answer/amount, empty citations and a reason	Valid abstention	The contract has an explicit no-answer branch

A version label lets producers and consumers detect incompatible changes. If v2 changes amounts from yuan to cents, an old consumer must not read 75000 as yuan. Publish a new schema and a tested migration; do not infer units from the magnitude. Our v1 remains fixed and rejects an unknown version.

41.4 Where constrained decoding intervenes

Example 41.2 — Restrict a next-token choice. Suppose, at one particular output position, the only allowed next-token candidates are 750 and 600. For this teaching example they are each a single token. The unconstrained next-token probabilities are 0.50 for 750, 0.20 for 600, 0.20 for a quotation mark and 0.10 for a letter. These are authored values, not model measurements.

A grammar for the current numeric field rejects the last two candidates. The remaining mass is $0.50+0.20=0.70$. Renormalizing gives

$$P(750\mid\text{allowed})=\frac{0.50}{0.70}=\frac57,\qquad P(600\mid\text{allowed})=\frac{0.20}{0.70}=\frac27.$$

The denominator is the probability mass of allowed candidates, not the original total 1. This is the same conditioning idea used in earlier probability examples. A grammar-aware decoder restricts continuations while generating; a post-validator inspects an already produced object. Real tokenizers may split these numbers, and real decoders must check whether each token’s characters can continue the grammar, not merely whitelist entire field values.

Both numbers remain allowed because the schema permits nonnegative integers. Constrained decoding can remove a structural error while leaving a factual error possible. No sampling policy can turn that grammar rule into evidence that 750 applies today.

Complete output

1. Outside the modelCheck stop reason
2. An operationParse JSON → validate schema
3. Text pieces600 or 750: valid integers

Refusal or truncation takes an explicit separate branch.

Evidence gate

1. Text piecesClaim: 600 [v2]
2. Outside the modelv2 lines 1–3: 750
3. An operationReject unsupported claim

A valid object still fails the cited-answer task.

Figure 41.1 — Structural gates cannot certify an amount. Original teaching diagram · CC BY-SA 4.0

Figure 41.1 keeps transport completion, parsing, schema validation and evidence support separate. A provider refusal or an interrupted stream may never produce an ordinary answer object. Handle that envelope before parsing. Do not concatenate half an answer into the next request and report success.

41.5 Refusal, abstention and technical failure are different results

In our contract, abstained means available evidence cannot justify an answer. refused represents a provider-level refusal, while error means the attempt failed technically or violated the data contract. They all carry null amounts, but imply different next steps. Missing evidence may call for a source; a timeout calls for bounded recovery; a provider refusal is not repaired by pretending it is an empty answer.

The current vendor example is documented in the dated online implementation notes docs/part-vii-online.md: provider grammar support is a subset, and stop reasons can change the response path. The stable lesson is to validate locally even when a provider offers structured outputs. This book does not call a paid model or measure constrained-decoding accuracy.

A malicious document can also influence a perfectly well-formed object. The caller must not treat a citation’s mere presence as proof, or an approved word as authorization. None of parsing, validation or object construction executes an order operation. We have made output easier to inspect, not given it authority.

41.6 KA-2: one boundary, explicit failure branches

contracts.validate_output parses with duplicate-key and non-JSON-number rejection, validates the full shared schema and constructs an Answer record whose fields cannot be reassigned. provider_boundary handles refusal and incomplete-response envelopes first. It maps invalid responses to an explicit error object; it does not guess a missing amount or repeatedly ask until a convenient answer appears.

The practice Notebook supplies authored malformed, wrong-type, missing-field and unsupported-claim cases to this real validator. They are fault-injection tests, not sampled LLM outputs. The tests include a valid but false 600-yuan object so that passing all format gates can never be reported as passing factual evaluation. The same schema and implementation serve both languages; localized answer text is content data.

41.7 Summary

JSON is a representation; a schema is a checkable shape; constrained decoding restricts the path while text is produced. A valid object still needs application and evidence checks. Explicit version and failure branches make the consumer’s behavior predictable. In Chapter 42, the model will propose a tool call in such an object, and the application will decide whether to execute it.

41.8 Exercises

1. Is "amount_yuan": null equivalent to omitting the field? Give the outcome for an abstention under the full contract.
2. Repeat Example 41.2 with allowed probabilities 0.30 and 0.10. The other candidates total 0.60. Compute the two renormalized probabilities. Does this remove the wrong amount?
3. A response has a valid v2 citation but says 600 yuan. List what has passed and the evidence needed for the next check. Why would changing the schema to require 750 be a poor general solution?
4. A response stops at max_tokens; its prefix resembles a valid answer. Another returns a provider refusal. Describe the two output branches, and explain why neither proves an order operation occurred.

41.9 Reference answers

1. No. Null is an explicit value; absence means the required name is missing. A valid abstention includes amount and answer as null, citations as an empty array, and a nonempty reason. Removing the amount field fails schema validation.
2. The remaining mass is 0.40, giving $0.30/0.40=0.75$ and $0.10/0.40=0.25$. Both amounts are still possible. The constraint changes structural possibilities, not the policy’s truth.
3. Parsing, field types and possibly citation shape have passed. Recover the exact cited lines, verify the version and date, and compare the amount and unit against the claim. Hard-coding 750 would hide the evidence task and reject the correct 600 for a historical question or a future policy change.
4. Truncation becomes error with incomplete_response; the provider refusal becomes refused with provider_refusal. The amount is null in both. These are response outcomes. No execution trace or authorization has been established.

41.10 References

• RFC 8259, §§2–7, 9. Defines the JSON syntax and value distinctions used in the worked object.
• JSON Schema Validation, Draft 2020-12, §§6.1–6.5, and Core, §§10.2–10.3. Defines the type, required, conditional and object-property rules; these do not establish factual truth.
• Anthropic, Structured outputs, verified 2026-09-14. Documents constrained decoding, supported schema subsets and refusal/truncation exceptions; fields and migration caveats are recorded in the online notes.