Econometrics bridge: Bootstrap, simulation, stochastic processes
Estimated time: 90 min
Lab: Open browser lab
Code: Python · R

Why this should feel familiar

A Markov chain is a random process whose next-state distribution depends on the current state. MCMC uses such a process as a sampling machine: instead of drawing independent bootstrap samples, it creates dependent draws whose long-run distribution is the distribution we want.

Mathematical core

For states \(s \in \{1, \ldots, K\}\), a first-order Markov chain satisfies

\[ P(S_{t+1}=j\mid S_t=i,S_{t-1},\ldots)=P(S_{t+1}=j\mid S_t=i)=P_{ij}. \]

A stationary distribution \(\pi\) satisfies

\[ \pi^\top P=\pi^\top,\qquad \sum_i \pi_i=1. \]

MCMC designs a transition kernel P so that a target density \(p(\theta \mid y)\) is stationary. In Metropolis-Hastings, a proposal \(\theta' \sim q(\theta' \mid \theta)\) is accepted with probability

\[ \alpha=\min\left(1,\frac{p(\theta'|y)q(\theta|\theta')}{p(\theta|y)q(\theta'|\theta)}\right). \]

The state of the chain can literally be a coefficient vector \(\beta\). The states do not converge to one OLS number; their empirical distribution converges toward the posterior distribution under suitable conditions.

Plain-English translation

Unlike bootstrap draws, adjacent MCMC draws are generally dependent. “Stationary” here refers to an invariant probability distribution of the stochastic process, not to covariance stationarity of an AR process, although the vocabulary reflects a related idea of distributional invariance.

What changes when we move from econometrics to AI?

The recurring shift is from a small, analyst-specified representation toward a larger learned representation. The underlying statistical questions do not disappear: What is the sample? What is conditioned on? What is estimated? What objective is optimized? Which observations were used to choose hyperparameters? How will uncertainty and distribution shift be handled?

AI terminology becomes much easier when these questions are kept explicit. Whenever you encounter a new architecture, identify four objects: inputs, parameterized transformation, training objective, and evaluation design.

Interactive browser lab

Change the two transition probabilities, simulate a path, and compare empirical state frequencies with the stationary distribution.

Before changing a control, predict the direction of the effect. Then manipulate one control at a time. The goal is not merely to produce a pretty visualization; it is to connect a parameter change to a mathematical statement.

Python and R lab

The Python and R examples implement the same core object as the browser lab. The Python code intentionally favors NumPy and explicit matrix operations. The R code favors base R and explicit loops/matrix algebra. Once the mechanics are clear, the same ideas can be implemented with PyTorch, TensorFlow, JAX, torch for R, or other frameworks.

Check your understanding

  1. Identify the random variables, observed variables, and estimated parameters in this module.
  2. State the objective function or equilibrium condition.
  3. Name the closest concept you already knew from econometrics and one important difference.
  4. Predict one failure mode caused by poor data, poor optimization, or poor evaluation.

Learner output

Explain why a chain can keep moving forever even after it has reached stationarity.