Chapter 54 From Notebook to Usable Service and Trustworthy UX

Mira asks for the Beijing lodging limit on 2026-09-14. The function from Chapter 46 returns 750 yuan per room per night and the supporting policy. In a Notebook, the author sees when the cell starts, can inspect its variables and knows which account is fictional. A person opening an application sees none of that. If the page stays quiet, should Mira wait or ask again? If she presses Stop, can an answer still appear? If the amount arrives before its source, how should she judge it?

The computational answer is only one part of the task. We need to preserve the user’s question, expose meaningful progress, enforce access and make the outcome understandable. The service developed here continues the existing knowledge assistant. Its default answer producer is the checked extractive path: it quotes the small fictional corpus. The Mini GPT candidates from Chapter 33 did not pass selection, and Chapter 38’s format repair did not make them reliable policy assistants. Putting them behind HTTP would not change that evidence.

54.1 Give each part of the request a responsibility

Start with the simplest arrangement: a page calls the answer function and displays its returned string. This works for one successful request, but does not specify who may ask, which date applies or what happens if the connection fails. We can expose those decisions without building a large distributed system.

The user interface (UI) collects the question and applicable date, displays progress and presents sources. Business logic identifies the caller, validates the request, grants access, limits work and owns its final state. Retrieval selects material eligible for the question date and supplies the complete supporting span. The answer/model layer turns that material into a candidate answer; the host checks its structure and evidence before release. These are responsibilities, not four mandatory machines.

In the CPU path, the last layer calls an extractive adapter and the existing source-verification Skill. It performs zero neural model calls. The model boundary remains useful: a future model adapter must obey the same input, output and stopping contract, and must pass a new evaluation. Chapter 47’s separately measured Qwen decision calls remain evidence about that narrower experiment. They are not additional samples produced by this service.

From question to checked answer

1. Application recordUI: question + 2026 date
2. An operationBusiness: identify caller; accept job
3. Outside the modelRetrieval: v2, lines 1–3
4. An operationAnswer adapter + Skill: validate 750

Question → authorized job → dated evidence → checked result → UI. Progress can arrive earlier; the amount cannot.

When stopping races with completion

1. Application recordStop received: cancelling
2. An operationSignal process group; wait for exit
3. Application recordOne terminal state: cancelled or answered

Stop wins → no answer or cache write. Completion wins first → cancellation returns 409; the view can still be cleared.

Figure 54.1 — A request has a data path and a separate stopping boundary. Original book diagram · CC BY-SA 4.0.

Figure 54.1 separates the data path from the authority to stop or publish a result. The UI supplies a question, never its own trusted tenant identity. A source provides policy text, never permission to read another account. The host, not a sentence in the answer, decides whether a result becomes visible.

Example 54.1 — Follow one real local request. The client sends the following JSON to POST /api/requests, with the fictional North credential in the Authorization header. The credential is a public demo placeholder, not a production secret. The corpus retains v1 and v2; ranking retains Chapter 44’s candidate order. The amount’s unit is yuan per room per night.

{"question":"Beijing lodging","on_date":"2026-09-14","topic":"travel","locale":"en"}

The server first validates the caller and all four fields. It allocates an opaque request identifier and responds with HTTP 202. That means accepted for processing, not answered. The client can read the job through GET /api/requests/{id}. A successful GET can return a still-running job: HTTP’s success describes the retrieval of the job record, while the record’s state describes the work. These are two different questions.

Table 54.1 — Observed stages of the successful loopback request

Stage	What actually happened	What the user may conclude
Accepted	Caller and input passed admission; a job ID was assigned	The service received this request
Running	A bounded worker began; no stored response matched	Work is in progress
Retrieval and composition	Date filtering chose v2, lines 1–3; the adapter prepared 750	A candidate exists, not yet a final answer
Source check	The reviewed Skill checked the exact span, date, authority and claim	The candidate satisfies this narrow evidence contract
Answered	The host committed the terminal result	The UI can show the answer and source together

The returned source is readable without opening another tool:

Fictional Beijing lodging policy; amounts are per room per night.
Effective from 2026-01-01; replaces travel-v1.
The lodging limit is 750 yuan.

This is original fixture text, cited as travel-v2, lines 1–3. Line 1 supplies the unit, line 2 supplies applicability, and line 3 supplies the amount. Displaying only “750” would discard two necessary conditions. For a question dated 2025-12-31, the same corpus yields v1 and 600. The UI therefore shows the date as an input the user can inspect, rather than silently deciding that every question means today.

54.2 A progress stream is not a stream of verified facts

Waiting for a complete response is simple but can leave a long operation looking broken. Streaming lets a connection deliver successive records while work continues. Our endpoint uses the event-stream framing checked in the HTML standard: an ID and a JSON data line, followed by a blank line. For example, one emitted frame is structurally:

id: 2
data: {"id":2,"stage":"running","elapsed_ms":5.057}

The elapsed value here is from the first saved local run, in milliseconds since job admission. The client reads complete event frames, not arbitrary network chunks: one chunk can end halfway through a JSON string, and another can contain several events. The small fetch reader retains incomplete text until the blank-line boundary. This implementation supports the subset it emits; it is not a general EventSource library.

For this short extractive answer, streaming “6”, then “00”, then a correction to “750” would create a needless trust problem. The service streams progress and publishes the checked answer as one final result. If a future model streams unverified text, the application must decide how visibly to label it, what uses to prevent and what a later failed check does to text already read. A later warning cannot make the user unread a number.

Disconnecting the event connection does not automatically cancel the job. A reconnect with Last-Event-ID can request later stored events without repeating the answer computation. If the final status is uncertain, the client retries the read at most once; it does not silently create a second job. These are application choices made explicit by the interface, not guarantees supplied by the word “streaming.”

54.3 Put a bound on waiting and on repeated work

Two requests can be active at once. A third arriving while both slots are occupied receives 503 with a retry hint instead of joining an unbounded queue. Concurrency means work overlaps in time; it does not promise that the CPU executes every instruction simultaneously or that throughput—the number of completed requests per second—doubles. The two-slot limit bounds active answer processes; it does not make the teaching HTTP server resistant to hostile connection floods.

Example 54.2 — One deadline, including the retry. Use assigned teaching times, not the measured service timings. A request has 1,000 ms from arrival to its deadline. Queueing consumes 120 ms, retrieval 80 ms, and a failed dependency attempt 300 ms. Waiting before a retry consumes another 100 ms. The remaining time is $1000-120-80-300-100=400$ ms. A second attempt cannot receive a fresh 1,000 ms merely because it has a new attempt number.

Let $D$ be the deadline on a monotonic clock, and $t$ the current reading on that same clock. A monotonic clock measures elapsed time without being reset by a wall-clock correction. Remaining time is:

$$R=\max(0,D-t).$$

Here $R$, $D$ and $t$ use the same time unit. If arrival is clock reading 0 ms, then $D=1000$ and the second attempt begins at $t=600$, giving $R=400$ ms. If the wait were 550 ms instead of 100, the attempted start would be at 1,050 ms, so $R=0$ and the retry must not begin. The maximum prevents a negative timeout from accidentally acquiring a different meaning in a library.

The actual service has a three-second deadline, a deliberately inserted 50 ms demonstration delay and at most two dependency attempts, separated by 50 ms. Those settings are assigned; observed durations include real Python startup and source-helper execution. They are not a model-speed benchmark. The HTTP and subprocess operations also have bounded waits. A deadline is enforced at explicit checkpoints and by stopping the worker process; process cleanup adds bounded scheduling overhead, so it is not a hard real-time guarantee.

A timeout answers “did this observer finish waiting?” It does not universally answer “did the operation happen?” Repeating a read of a fictional order is acceptable here. Repeating an approved cancellation needs Chapter 49’s durable intent, receipt and recovery checks. This service exposes no order-write route; its Stop button cancels an answer job, not an order.

54.4 Cancellation has to win a race, not just hide a panel

Suppose Mira presses Stop while the answer process is running. Setting a flag in the browser alone leaves the process using resources. Setting a server flag alone also does not terminate arbitrary Python code. The host must signal the active process group, wait for cleanup and prevent later publication.

Example 54.3 — Two possible winners. Before completion, the job is nonterminal. A cancellation request changes it to cancelling; the UI says that stopping is underway. After the worker exits, the host records cancelled. Completion and cancellation share one lock, a mechanism that lets only one thread change the protected state at a time. If cancellation wins, the final result contains no answer and no response-cache entry is written. If completion wins first, cancellation returns 409 with the completed state. The interface explains that the request has already finished.

The actual cancellation test waits until an attempt starts, sends a real HTTP cancellation and then checks the state again after the worker has stopped. It verifies that no late answered event or cache entry appears. This is stronger than asserting that a button changed color. The process-group implementation was executed on macOS CPU; the program explicitly requires POSIX and does not claim Windows support.

The first recording exposed a subtler defect. Its sequence was:

attempt → cancel_requested → worker_stopped → dependency_failure → cancelled

The answer was correctly suppressed, but InterruptedError had been caught by a broader OSError handler and counted as a failed dependency attempt. A new assertion rejected that record. Handling cancellation separately repaired the sequence to:

attempt → cancel_requested → worker_stopped → cancelled

Both the original observation and the regression remain in the evidence pack. A service can behave correctly for the user and still mislead its operator. Chapter 56 will show how an incorrect event classification can distort an alert.

54.5 Help the user decide what to do after the result

An answer with a citation is easier to inspect, but a citation is not a guarantee that the underlying source is authoritative in the real world. The interface names its fictional scope and keeps the source’s date and unit visible. It does not attach an invented confidence percentage to a deterministic quotation.

Table 54.2 — Different outcomes need different explanations

Actual request or action	Observed outcome	Useful next step
Current lodging question	750 with v2’s complete span	Read the source and applicable date
volcano against the travel collection	Abstained: no supporting material	Refine the question or consult a policy owner
overnight accommodation ceiling	Abstained although relevant policy exists	Preserve this vocabulary-coverage failure for evaluation
North account reads its A-104	Processing, from the fictional order snapshot	Distinguish a snapshot from a live merchant record
North account requests unavailable B-205	Refused, with no order changes	Use an authorized route; do not retry as another identity
A dependency remains unavailable	Error, with no fabricated policy answer	Wait or prepare a note for a person

The two abstentions share a user-facing statement about insufficient support, but imply different maintenance work. The first question is outside the collection; the second exposes a known lexical gap. Logs and regression cases preserve that distinction. A polite refusal does not turn missed relevant evidence into a successful answer-quality result.

The local handoff prepares a visible note containing the question and date. A person can copy it to a policy owner; the application neither sends it nor claims that somebody has taken responsibility. Undo restores a cleared question and displayed result. It reverses that interface change only. An order already cancelled elsewhere would require a separate, possibly impossible, compensating business action.

Accessibility helps make these distinctions usable. The page supplies visible, associated labels; native controls can be reached with the keyboard; focus remains visible; a concise status region announces progress without moving focus on each event. Source text remains selectable and readable at a narrow width. These choices follow relevant W3C guidance. Browser checks of labels, focus and layout are scoped evidence, not a full accessibility certification or a substitute for testing with assistive-technology users.

54.6 Test the seams as well as the function

The HTTP Primer is the practice prerequisite: before running this service, be able to distinguish client and server, read a JSON body and status code, keep a credential out of logs, and explain why a timeout does not prove nonexecution. It already includes a runnable loopback scaffold. Here the additional bridge is a job: a server-owned record that lasts beyond the initial HTTP response. An event ID orders records within that job; it is not a user’s identity.

Table 54.3 — Evidence at different software layers

Layer	Observable check	What remains unproved
Pure policy	Date and cache-expiry boundaries; remaining deadline	Network behavior
Component	RAG plus the actual reviewed Skill subprocess	UI interpretation and user comprehension
Integration	Real HTTP creation, event read, cancellation, ownership and errors	Reliability under a production traffic distribution
UI	Submit, readable sources, keyboard focus, clear/undo, handoff and feedback	Full assistive-technology usability
Probabilistic evaluation	Retained model samples and held-out criteria from earlier chapters	Not replaced by any of the software checks above

The core entry is code/knowledge-assistant/service.py; the record runner and tests are under code/part-ix/. The two language interfaces use service-content-v1.json, the same data and the same execution logic. From a CPU environment prepared as described in the project README:

python code/knowledge-assistant/service.py --port 0
python -m unittest discover -s code/part-ix -p 'test_*.py' -v

Port 0 asks the operating system for an available local port; use the URL printed by this process. No account, GPU, paid API or external model service is needed. The script binds only to 127.0.0.1. Python’s HTTP server is a teaching choice, not a production deployment recommendation. The dated companion notes record tested versions and commands.

KA-9 now has a bounded CPU demonstration, actual request/cancellation/error records, an accessible interface and checks at the software boundaries. Its jobs and feedback are in memory; a restart ends that session. Order recovery remains the separate durable SQLite mechanism from Chapter 49. Keeping those lifetimes explicit prevents the existence of a log file from being mistaken for durable task recovery.

54.7 Summary

A usable service turns a calculation into an accountable request. The host owns identity, admission, deadlines and terminal state; retrieval supplies dated evidence; the answer layer proposes content; the UI helps a person inspect the outcome and choose a next step. Streaming progress is useful before an answer is verified. Cancellation, retry and undo each need their own effect boundary. Software tests can establish those boundaries on exercised paths without establishing a model’s general answer quality.

The request is now inspectable. The next problem is identifying exactly which versions produced it, and recovering when a change breaks the contract.

54.8 Exercises

1. In Example 54.2, retain the 1,000 ms budget but change queueing to 200 ms and backoff to 250 ms. Retrieval and the first attempt are unchanged. How much time may the second attempt use? Would an 800 ms per-attempt timeout be acceptable without another bound?
2. A stream prints 600, then fails source verification and displays “Please verify.” Explain why the final warning does not fully repair the user experience. Propose a concrete alternative for this policy task.
3. South knows North’s request ID and supplies its own valid credential. Which check must fail? Why are either a long random ID or successful authentication insufficient?
4. A cancellation returns before worker cleanup. Write an observable test that distinguishes “stopping requested” from “stopped,” including what must never happen afterward.
5. The service’s HTTP tests all pass, but it abstains on the synonym in Table 54.2. Is it ready to claim reliable policy-question understanding? Explain which evidence and repair are still needed.

54.9 Reference answers

1. Remaining time is $1000-200-80-300-250=170$ ms. An 800 ms timeout alone would overrun the request budget. Bound the attempt by the smaller of its normal timeout and 170 ms, and account for cleanup separately.
2. The user may have read or used 600 before the warning. A warning cannot retract that exposure. For this short task, stream progress only, check the complete dated source, and display the final amount and citation together. If verification fails, return an explicit abstention without the unverified number.
3. Ownership authorization must compare the authenticated tenant/user with the job owner and refuse the read. A random ID reduces guessing but does not grant permission. Authentication identifies South; it does not make South the owner of North’s job.
4. Start a sufficiently long controlled request, wait for an attempt, send cancellation, and observe cancelling until cleanup finishes. Then require exactly one cancelled terminal result, no later answer and no cache insertion. Also test the opposite race: an already finished request stays finished and cancellation reports that fact.
5. No. HTTP behavior says the transport and state controls work in these cases; it does not establish vocabulary coverage. Keep the failed synonym, investigate tokenization/retrieval, compare a proposed repair on the same evaluation set and retain source/date checks. Earlier negative model-selection evidence also remains relevant.

54.10 References

• RFC 9110, HTTP Semantics, §§9.2.1–9.2.2 and 15.3.3. Supports the distinctions between safe repetition, accepted processing and completed work.
• Python 3.12 HTTP server documentation. Describes the selected handler/threading stack and its explicit production limitation; tested runtime details are kept in the companion records.
• Python threading: Event objects. Explains signaling between threads; the process cleanup and terminal-state policy are implemented by this project.
• WHATWG HTML: server-sent events, §§9.2.4–9.2.6. Specifies event IDs, UTF-8 and event-frame boundaries.
• WCAG 2.2, SC 2.4.7; Understanding Status Messages; Understanding Labels or Instructions. Support the focused accessibility checks and distinguish visible instructions from programmatic names.
• OWASP LLM09:2025 Misinformation. Its prevention guidance motivates visible source checking, scope communication and human oversight without claiming that a citation guarantees truth.