Chapter 47 Before Agents: Workflows, State, and Memory

The employee asks for the current Beijing lodging limit. Chapter 45’s local RAG path can retrieve the current policy and return 750 yuan per room per night with a citation. Now suppose the assistant is allowed to search again whenever it is dissatisfied. It retrieves the same document, asks itself whether to continue, and searches again. The answer may still be easy; the process has become harder to control.

The missing property is not another model call. We need to know which step is allowed next, what information survives that step, and why the process will stop. Chapter 42 established that a model proposes an operation while an application executes it. Here we apply that division to the whole request. We will first make the route inspectable, then open exactly one decision to the model.

47.1 Four arrangements for the same question

Keep the task and success criterion fixed: answer the current question from an eligible policy, preserve its unit and citation, and stop. A correct amount from an expired source does not pass. Neither does an endless search that happens to print 750 along the way.

In a single call, the application supplies the question and policy to one model call and receives an answer. The model does not fetch the policy merely because its prompt mentions a document. Somebody must put that document into the input. In a chain, the application fixes a sequence: retrieve the policy, construct the input, produce an answer. The retrieval operation need not itself involve a model.

A workflow adds application-defined branches. If the source is missing, stop with an abstention; if it is present, verify its date before answering. A workflow can contain loops, but their entry conditions and limits are specified. An agent can instead select its next action from the current observation: search a different source, use a tool, or finish. The useful distinction here is how much choice is delegated, not whether a loop exists or whether there were two model calls. These are working definitions for analyzing this book’s system, not a claim that every library uses identical labels.

Single call

1. Outside the modelQuestion + supplied policy
2. An operationOne model call
3. Text piecesAnswer → stop

The application supplies everything before the call.

Chain

1. An operationRetrieve
2. An operationCompose
3. Application recordStop

The next operation is fixed in advance.

Workflow

1. An operationRetrieve → verify
2. Application recordEnough evidence?
3. Text piecesAnswer or abstain → stop

Branches and guards are written by the application.

Open decision loop

1. Application recordObserve current state
2. An operationModel chooses tool or finish
3. Outside the modelTool result → observe again

Without a host limit, the return arrow need not terminate.

KA-5 exact graph

1. Application recordSTART → RETRIEVED
2. Application recordValid → VERIFIED → DECIDED
3. Text piecesuse_evidence → COMPOSED → ANSWERED

Invalid evidence → ABSTAINED; abstain → ABSTAINED; invalid choice, timeout or exhausted budget → STOPPED. Terminal states have no outgoing transition.

Figure 47.1 — Who chooses the next step? Original teaching diagram · CC BY-SA 4.0

Figure 47.1 keeps the lodging request constant while moving responsibility for the next step. Its open loop includes a return arrow from a tool result to observation. That arrow is useful when a result changes the plan. It is also where a repeated failure can consume resources indefinitely. An application still needs to control that loop even when a model chooses actions.

The orchestrator is the application code that calls a step, records its result and chooses the permitted next step. In this project it is a small Python function, not another model. The plan is mostly its fixed graph. Tools supply observations; state records progress; terminal states close the request. ReAct provides a research example of interleaving language and environment actions, but its pattern alone does not supply our permission or stopping guarantees. The reference at the end identifies that narrower connection.

47.2 Write down what is allowed to happen

Example 47.1 — Follow the current policy to a terminal state. The input is Beijing lodging, the date is 2026-09-14, and the topic is travel. The inherited corpus retains both policies. travel-v1 permits 600 yuan during 2025; travel-v2 permits 750 yuan from 2026-01-01 and replaces v1. Both amounts mean yuan per room per night. Corpus ordering and ranking stay as in Chapters 44–46.

The application retrieves eligible evidence and checks the complete supporting span before asking the model anything. It then offers the ordered candidates use_evidence, abstain. The model sees the question, the selected evidence, the two candidates and a short decision instruction. It receives no cancellation tool and no credential. Its response must equal one candidate exactly. There is no third option to search again.

Table 47.1 — The successful KA-5 request, with each observation explained.

Recorded state	What just happened	What permits the next step?
RETRIEVED	The selected source is v2, lines 1–3	A source is present; presence alone is insufficient
VERIFIED	Date, authority, complete span and extractive claim pass	The host allows one bounded decision
DECIDED	The model returns use_evidence	The value belongs to the two-candidate set
COMPOSED	The local composer prepares 750 and the complete quotation	A final source-support check is required
ANSWERED	The checked answer is returned	Nothing: this is terminal

The three source lines matter together: line 1 supplies the unit, line 2 supplies the effective date and replacement, and line 3 supplies 750. Keeping only the last line would discard why the number applies. The composer reuses Chapter 45’s exact-source contract; it does not ask another model to improvise the answer. A candidate is prepared during verification so the source check can inspect the intended claim, then released only after the decision and final gate.

There are five recorded transitions on this path. The configured cap is eight transitions, including any recorded stop transition. The count starts at zero before START leaves its initial state. After five transitions, three remain unused; reaching a terminal state does not require spending them. The separate model-call cap is one. A failed decision consumes that call too, so asking again cannot quietly become a second decision point.

We can now name the compact rule behind the table. Let $s_t$ be the recorded state after step $t$, and $o_t$ the observation available for the next step. The application applies a transition rule:

$$s_{t+1}=T(s_t,o_t).$$

Here $T$ is the written graph plus its guards, not a learned parameter matrix. For example, VERIFIED plus the observation use_evidence permits DECIDED; an unrecognized value leads to STOPPED. The subscript counts steps, not seconds. A guard is the condition that must hold before taking an edge. A terminal state has no outgoing edge. The graph in the fixture lists all edges so a reader can check these claims without running Python.

The local Qwen3-1.7B measurement actually returned use_evidence for both language versions. The saved host traces therefore reached ANSWERED with 750. Those two observations demonstrate integration with a model; they do not establish that this model judges evidence reliably. Core tests also inject fixed decision strings to exercise branches predictably. Their results are software behavior, not additional model samples.

47.3 A refusal to continue can be the correct result

Change only the evidence to an empty corpus. The application records RETRIEVED, then ABSTAINED, with insufficient_evidence. It makes zero model calls. Having no source is already an application-visible reason to stop; a model vote would add cost without creating evidence.

With valid evidence but the decision abstain, the route is RETRIEVED → VERIFIED → DECIDED → ABSTAINED. With the decision cancel_order, it is RETRIEVED → VERIFIED → STOPPED. The latter string is invalid decision data. It never becomes an executable command. A timeout at the decision point also stops the request. These distinct records tell a maintainer whether to inspect missing evidence, model behavior or an invalid response.

Reduce the transition cap from eight to two. Retrieval and verification consume both transitions, so the application stops before calling the model. This budget stop is recorded in the returned state and reason even when no additional event slot remains. Therefore an event list must be read with its final state, not treated as the entire result. Chapter 49 will add durable event storage and call budgets for recovery across process failures.

Our graph has no cycle, so its structure already prevents an endless tool loop. The cap remains useful if a later edit accidentally adds a route. In a workflow with retries, each retry must also reduce a finite budget or move toward a deadline. “The model will know when to stop” is not an application invariant: an invariant is a condition the host preserves for every permitted execution, such as model calls never exceeding one.

47.4 Five things people call memory

After answering, we could save “750” for tomorrow. That creates a new question: what exactly was saved, and who will consult it? Chapter 2 distinguished stored information from information supplied to a model. Chapter 11’s recurrent state compressed earlier sequence input into numbers. Neither automatically implements a durable task record.

Table 47.2 — Lifetime and responsibility in this request.

Item	Within this request	Across requests	What would make it usable?
Context	The question and selected v2 quotation supplied to the model	Usually rebuilt; a transcript may be stored separately	The application selects and supplies it again
Memory	A retrieved note may be inserted into context	An application-maintained note or preference can persist	Scope, provenance, permission, expiry and retrieval rules
RNN hidden state	The numerical sequence summary from Chapter 11	Only if an implementation explicitly carries it forward	A model architecture and reset policy; it is not our order ledger
Agent/workflow state	Current stage, evidence IDs, choice and remaining budget	Can survive a failure if explicitly persisted	A schema, checkpoint and recovery rule
Cache	Reuses an earlier retrieval or computation result	May outlive a request until invalidated	A key covering every input that can change the result

A cached result has a different purpose from a remembered preference. The cache avoids repeating a computation; a memory may deliberately influence a new request. Both are stored data, and both can be stale. A note saying “the employee previously used 600” is evidence about a previous interaction, not authority for today’s policy.

Suppose a cache key contains only the text Beijing lodging. It can return the 2025 answer in 2026 even if the new retrieval code is correct. A suitable key for this small computation includes the question, question date, topic, language, corpus version and processing version. If access differs by tenant or user, the key must also include the relevant authorization scope, or the result must be re-authorized. Chapter 53 will demonstrate the tenant failure directly.

Storing a memory can also store an error or an attack. A summary that turns “the document falsely claims permission” into “permission granted” changes the meaning while shortening the text. Our KA-5 default therefore persists no cross-request preference memory and makes no claim about trustworthy long-term recollection. Adding memory is a data-management choice with a purpose and deletion rule, not a free reliability upgrade.

47.5 Practice and the boundary of KA-5

After Primer-PY and Primer-HTTP, the shared implementation is code/knowledge-assistant/workflow.py. Its state graph is data/knowledge-assistant/ka5-workflow-v1.json; bilingual decision strings live in ka5-content-v1.json. The Part VIII Notebook follows the same artifacts. A minimal local call supplies a deterministic test double:

result = run("Beijing lodging", choose=lambda _: "use_evidence")
print(result["state"], result["answer"]["amount_yuan"])

ANSWERED 750

The function in choose stands in for the model only in this software test. The separate model-run.json preserves actual model requests, output, runtime version, digest, seed and CPU observation. Reading the complete path above requires neither this program nor a model installation.

KA-5 is intentionally narrow. It selects between using verified evidence and abstaining; it cannot plan arbitrary travel, browse new sources or cancel an order. This limits flexibility but makes stopping and failure attribution inspectable. If the fixed workflow already meets the task, a broader agent loop must justify its extra decisions against that baseline.

47.6 Summary

A single call, chain, workflow and open agent loop arrange control differently. The orchestrator owns the allowed transitions and limits even where it delegates a choice. Our example reaches 750 through five transitions and one bounded model call, or stops with a reason that identifies the failed boundary. Context, memory, recurrent state, workflow state and cache differ in purpose and lifetime. None grants execution authority simply by storing text.

The source verification step is now explicit enough to repeat. The next chapter asks how to package that practice without copying a large prompt or silently changing its rules.

47.7 Exercises

1. Keep valid v2 evidence but return use_evidence; search_again from the decision adapter. Trace the states, count model calls and explain whether a search occurs.
2. A developer changes the missing-evidence branch to “retrieve again” but leaves every counter unchanged. Give a concrete nonterminating execution and two independent ways to prevent it.
3. A cache includes question, language and corpus version but omits question date. Use the unchanged v1/v2 corpus to construct two requests that must not share an answer.
4. The assistant saves “mira approved cancellation” in a conversation summary. Identify which kind of stored data this is, and the additional evidence required before a write could be allowed.

47.8 Reference answers

1. The route is RETRIEVED → VERIFIED → STOPPED. One model call has been used. The entire returned string is compared with the two allowed values; matching a prefix is insufficient. No search tool is invoked.
2. An empty corpus repeatedly produces no evidence, takes the retry edge and returns to the identical state with unchanged counters. One repair is to remove that edge and abstain. Another is to decrement a finite retry budget on every attempt and stop at zero; a deadline adds a separate time bound. A prompt asking the model to stop does not enforce either rule.
3. Ask the same question for 2025-09-14 and 2026-09-14. Both queries use the same corpus bytes, but the eligible policies yield 600 and 750 respectively. Including corpus version cannot replace including the date used to interpret it.
4. It is stored conversation memory, which could later be inserted into context. It is not an approval record. A write needs a trusted actor, permitted scope, exact action and target, applicable object version, valid approval and current authorization. Chapter 49 implements those checks rather than reading permission from a summary.

47.9 References

• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, §2 and §3.1. The paper illustrates an agent’s observation/action loop; our fixed host guards and budgets are separate engineering mechanisms.
• Agent Skills client implementation: activation. This describes how reusable instructions enter an agent’s context and prepares the next chapter’s distinction between stored resources and loaded instructions.