Econometrics bridge: Sequential decisions, state transitions, dynamic optimization, observation, and independent verification
Estimated time: 135 min
Lab: Open browser lab
Code: Python - R

From sequential decisions to agentic systems

A sequential decision rule should feel familiar after Module 12. A policy chooses an action from a current state; the environment returns an observation, and the state changes. In an agentic application, the action may be a document lookup, a calculator request, a database change, a model call, or an escalation to a person. From Module 11, retrieval supplies evidence, not automatic authority; from Module 13, a language model can be treated as a policy, but its training objective is separate from the business system’s goal.

A useful abstraction is

\[ a_t = \pi(s_t,g,\mathcal T),\qquad o_{t+1}=\operatorname{Execute}(a_t),\qquad s_{t+1}=\operatorname{Update}(s_t,a_t,o_{t+1}), \]

where \(g\) is the task, \(\mathcal T\) denotes available tools, and the controller decides whether to continue, recover, stop, or seek authorization. The policy \(\pi\) may be a deterministic program, an LLM, another learned model, or a hybrid; the mathematics does not require it to be an LLM.

The critical distinction is between selecting, authorizing, executing, and verifying an operation. A generated plan is not an executed action; a message saying “tool succeeded” is not authoritative evidence that an external effect occurred.

What a language model adds—and what it does not

Compare three implementations of the same business request:

System Who chooses the next step? What it adds What it does not guarantee
Fixed workflow Programmed transitions Predictable execution for well-specified tasks Flexibility with ambiguous requests
LLM-directed agent LLM conditioned on state and tool observations Flexible interpretation, planning, and tool selection Correct tool arguments, authorization, or effects
Hybrid controller LLM proposes; rules and tools check and execute Flexible interpretation with explicit boundaries Perfect source quality or complete verification

A simple task such as revenue-growth calculation does not need an LLM once its meaning and data definitions are fixed. An LLM could interpret an ambiguous instruction or select a source, but arithmetic and business-rule checks can be performed deterministically. The lab uses a deterministic controller, not a disguised LLM or an autonomous model. The decision points where a model could be substituted are shown explicitly.

Reasoning, search, and observable traces

Inference-time computation includes candidate generation, search, decomposition, external calculation, and verification. These activities can be combined, but they are different mechanisms. A longer textual explanation alone is not evidence of correct reasoning or execution.

A trace useful for evaluation should record: the current state, the selected action, the actual tool request, the returned observation, a verification result when relevant, and the resulting next state. It is an operational trace, not a transcript of hidden model reasoning.

In a ReAct-style system, the next choice can depend on the latest tool observation:

\[ \text{state}_t\rightarrow\text{action}_t\rightarrow \text{observation}_{t+1}\rightarrow\text{state}_{t+1}. \]

For example, if a lookup returns “missing,” the controller can try an authorized fallback or abstain. A scripted series of calls that cannot branch on observations is a workflow trace, not a demonstration of adaptive decision-making.

Working state, memory, and evidence

These objects may contain overlapping information but play different roles:

Object Main purpose Example
Working state Tracks the current goal, observations, pending action, and status LOOKUP, VERIFY, BLOCKED
Memory Retains information across steps or sessions Prior source selections or summaries
Evidence Supports a specific claim or execution decision A retrieved 2026-sales record with a version

A cached number can be useful memory but stale evidence. Reusing it requires freshness checks appropriate to the task. Storing a value does not make it authoritative, and retrieving a document does not make its instructions binding.

Decision rights and authorization

Decision-right allocation determines who may decide what. Typical boundaries include:

Decision Appropriate controller in the lab
Identify the requested metric Fixed task specification (an LLM could propose it)
Choose a fallback after failed reading Deterministic error-handling rule
Calculate a percentage Calculator function
Verify the result Independent numerical invariant
Propose an inventory adjustment Workflow controller
Approve the simulated write Explicit user approval
Perform the write Authorized tool with an expected-version check
Verify the external result Read back the resulting state

The authorization checkpoint is independent of the numerical result. A mathematically correct proposal does not grant permission to change inventory. In an operational system, approvals also need authenticated identity and an auditable decision record; the browser toggle is only a teaching substitute for such infrastructure.

Tool calls, uncertainty, and recovery

A tool has at least three distinct notions of success:

  1. The controller requested an operation.
  2. The interface returned a status.
  3. The underlying state actually changed (or a read returned trustworthy evidence).

These can disagree. If a write request times out, it may have committed before the timeout. Retrying blindly can duplicate an action. The appropriate response depends on operation semantics, idempotency, authoritative status checks, and authorization rules.

In the lab, a simulated inventory write can return outcome unknown after timeout. The controller performs an authoritative readback before deciding whether the write completed. It does not automatically issue a second write. The simulated read-only operations may safely switch to a fallback data source. A malformed calculator result is caught by an independently computed invariant and corrected by a bounded retry.

For example, revenue growth is calculated as

\[ g=100\frac{S_{2026}-S_{2025}}{S_{2025}}, \]

and checked using the independently stated constraint

\[ S_{2025}(1+g/100)=S_{2026}. \]

Passing the check establishes internal arithmetic consistency with the retrieved figures; it does not by itself certify that the figures are true. Evidence quality and calculation correctness are separate claims.

System-level evaluation

A language model may minimize token-prediction or post-training loss while the deployed system must satisfy a different objective, such as verified task completion under time, cost, and authorization constraints. Do not infer system reliability from model-only benchmarks.

Useful evaluation dimensions include task success, source support, numerical correctness, unauthorized writes, ambiguous outcomes, tool calls, recovery/abstention rate, latency and cost, state consistency, and comparison with a simpler deterministic baseline. Model-directed and deterministic systems should be compared under equivalent tool access and task definitions. A deliberately incorrect “direct answer” is not a valid experimental baseline.

A multi-agent architecture may assign planning, execution, and checking to different actors, but role separation does not necessarily require multiple models. More agents can increase coordination cost and correlated failure. Independent verification is a property to test, not a title to give one of the agents.

Econometrics-to-AI translation: the five questions

Module 0 question Module 14 translation
What is the familiar statistical object? Sequential decision rules, Markov state updating, dynamic optimization, and validation of computed outcomes
What became learnable? Potentially the decision policy or specialized routing/checking models; deterministic rules need no training
What objective is optimized? Model-level prediction or policy objectives, plus separately specified system-level success, cost, and risk criteria
What generates the evidence? User instructions, tool observations, retrieved records, external state changes, and checks
Where does the analogy break? A correct forecast or plan is not an authorized, completed action; correlated tool failures make trace rows dependent

The course ends where its two mathematical spines meet. Learned representations help interpret open-ended requests; probabilistic and sequential models describe uncertain observations and decisions; tools carry out operations; and controls, verification, and authorized people determine which consequences may follow.

Interactive browser lab

The browser lab is an offline workflow and decision-right simulator. It contains no LLM, network call, real database write, or access to external business systems.

Three experiments share the same local records: sales in 2025 and 2026, cost in 2026, current inventory, and target inventory.

  1. Execute a trace: Select growth, gross margin, or inventory excess. Advance one state at a time and inspect actual lookups, arithmetic, and independently checked results. Change the 2026-sales record and verify that the answer recalculates rather than remaining hard-coded.
  2. Inject a failure: Try a missing primary record, stale version, read timeout, faulty calculator, or unavailable evidence. Observe recovery, an actual failed check, or justified abstention.
  3. Authorize a write: Select inventory adjustment and compare approval denied with approval granted. Simulate a write timeout after commitment and verify the actual inventory via readback instead of retrying blindly.

For each scenario, predict the next state before advancing. The trace identifies the actor (controller, tool, verifier, or approval gate) and reports the number of actual tool invocations and checks.

Python and R lab

Run the same finite-state controller without external dependencies. Verify the arithmetic for three read-only tasks, observe an updated figure changing the output, inject lookup/calculation failures, and test both denied and approved inventory adjustments. Assertions should check actual state transitions and tool-call counts rather than simply printing “verified.”

Inspect the difference between the controller’s planned action and the tool’s returned observation. Then change a data value or injected failure and explain why the subsequent path changes. These experiments demonstrate control logic, not autonomous LLM planning.

Practice:

  1. For one task, identify where an LLM could usefully interpret an instruction and where deterministic execution is sufficient.
  2. Explain why tool returned success, result is mathematically consistent, and external state actually changed are three distinct claims.
  3. Modify the 2026-sales value from 150 to 300. Predict the growth result and identify the observations and checks required to justify it.
  4. Inject a missing or stale record. Explain when a fallback is defensible and when the system must abstain.
  5. Compare an approval-denied inventory proposal with an approved, verified inventory write. Which state transitions may occur in each case?
  6. Describe a timeout scenario in which retrying a consequential action would be unsafe.
  7. Propose an evaluation that compares a deterministic workflow, a single LLM-directed controller, and a hybrid controller with matched access and costs.

Learner output

Produce a compact agentic-system design and evaluation specification containing: a goal and success criterion; a state-transition diagram; tool contracts and possible failures; decision rights across LLM/rules/human; evidence and verification rules; authorization conditions; at least one failure/recovery test; and a cost/reliability comparison baseline. Attach one actual execution trace and explain what it proves and what it cannot prove.

Sources and Further Reading

Core sources

Further reading

These references motivate the agent/tool/evaluation concepts. The local business-workflow lab is a constructed teaching example, not an empirical result from those papers.