Econometrics bridge: Conditional probability, likelihood, latent variables, state-space models, simulation
Estimated time: 120 min
Lab: Open browser lab
Code: Python - R
From econometric probability models to graphical models
If you have used conditional expectations, likelihoods, or a regression such as
\[ Y_i=\beta_0+\beta_1X_i+\epsilon_i, \]you already know the idea that a statistical model specifies which variables depend on which others. You may also know latent-variable models, time-series recursions, or state-space models. This module makes the conditional structure explicit, then uses it to reason about sequences and to construct sampling algorithms.
Three distinct operations recur:
| Operation | Question | Example |
|---|---|---|
| Specification | Which dependencies does the model assume? | Choose the edges of a Bayesian network |
| Learning | What model parameters or structure are estimated from data? | Estimate a conditional probability table |
| Inference | What can we infer about an unknown given observations? | Calculate \(P(A=1\mid B,C)\) |
| Sampling | How can we approximate difficult distributions or integrals? | Run an MCMC chain for a posterior |
Do not call every inference or sampling computation learning. In the browser lab, the graphical parameters are specified in advance; the exercise performs inference using those parameters.
Bayesian networks: factorization and conditional independence
A Bayesian network is a directed acyclic graph (DAG) together with conditional distributions for its nodes. Its joint distribution factorizes as
\[ p(x_1,\ldots,x_d)=\prod_{i=1}^{d}p(x_i\mid\operatorname{pa}(x_i)), \]where \(\operatorname{pa}(x_i)\) denotes the parents of node \(X_i\).
Start with a fork: \(A\to B\) and \(A\to C\). The implied factorization is
\[ p(a,b,c)=p(a)p(b\mid a)p(c\mid a). \]Thus \(B\perp C\mid A\), even though \(B\) and \(C\) can be associated before conditioning on \(A\). This is a probabilistic conditional-independence claim, not automatically a causal claim. A causal interpretation requires additional assumptions about the graph and the data-generating process.
Take
\[ \begin{aligned} P(A=1)&=0.4,\\ P(B=1\mid A=1)&=0.8, &P(B=1\mid A=0)&=0.2,\\ P(C=1\mid A=1)&=0.7, &P(C=1\mid A=0)&=0.3. \end{aligned} \]Then observing \(B=C=1\) gives
\[ P(A=1\mid B=1,C=1) =\frac{0.4(0.8)(0.7)}{0.4(0.8)(0.7)+0.6(0.2)(0.3)} \approx 0.8615. \]This is exact inference, not the estimation of new parameters. The lab lets you change the observations and inspect both the numerator and the normalizing denominator.
Markov blankets: what is sufficient for local inference?
Extend the graph to five nodes:
\[ A\to B,\quad A\to C,\quad B\to D,\quad C\to D,\quad D\to E. \]The Markov blanket of node \(B\) comprises its parents, children, and the other parents of its children. Here,
\[ \operatorname{MB}(B)=\{A,C,D\}. \]Once \(A,C,D\) are observed, the additional value of \(E\) does not alter the conditional distribution of \(B\):
\[ B\perp E\mid A,C,D. \]This local result can be seen algebraically. The terms relevant to a conditional update of \(B\) are
\[ p(b\mid a,c,d,e)\ \propto\ p(b\mid a)\,p(d\mid b,c). \]The factor for \(E\) cancels when comparing possible values of \(B\). Gibbs sampling uses this idea to update individual variables using only the factors involving that variable (or equivalently, its blanket).
Do not confuse a Markov blanket with a Markov chain. The former identifies a conditioning set in a graphical model; the latter makes an assumption about how the next state depends on previous states.
Markov chains: transitions, stationarity, and convergence
A first-order Markov chain satisfies
\[ P(S_{t+1}=j\mid S_t=i,S_{t-1},\ldots) =P(S_{t+1}=j\mid S_t=i). \]For a two-state chain with \(a=P(0\to1)\) and \(b=P(1\to0)\), the row-stochastic transition matrix is
\[ P=\begin{pmatrix}1-a&a\\b&1-b\end{pmatrix}. \]A stationary row distribution \(\pi\) obeys \(\pi P=\pi\). When \(a+b>0\), one stationary distribution is
\[ \pi=\left(\frac{b}{a+b},\frac{a}{a+b}\right). \]If \(0
This meaning of stationarity is different from covariance stationarity in econometric time-series analysis. The browser lab evolves an initial distribution using \(p_{t+1}=p_tP\), verifies invariance numerically, and demonstrates the alternating case.
Hidden states and temporal probabilistic models
A hidden Markov model (HMM) adds observations to a Markov state:
\[ S_{t-1}\to S_t\to S_{t+1},\qquad S_t\to Y_t. \]The observations are conditionally independent across time given their states under the standard HMM assumptions. A dynamic Bayesian network generalizes the temporal graph to richer variables and dependencies. A linear Gaussian state-space model is a familiar special case within this broader probabilistic modeling tradition.
Three related tasks have different information sets:
\[ \underbrace{p(S_t\mid Y_{1:t})}_{\text{filtering}},\qquad \underbrace{p(S_{t+k}\mid Y_{1:t})}_{\text{prediction}},\qquad \underbrace{p(S_t\mid Y_{1:T})}_{\text{smoothing}},\quad k>0,\ T>t. \]An HMM is not merely an autoregression written differently: it explicitly models an unobserved state and an observation mechanism, which may support filtering and smoothing as well as prediction.
MCMC: constructing a chain to perform Bayesian computation
A Bayesian econometrician may want a posterior expectation
\[ E[g(\theta)\mid y]=\int g(\theta)p(\theta\mid y)\,d\theta, \]but the integral may be difficult to calculate. Markov chain Monte Carlo (MCMC) constructs a chain whose stationary distribution is the target posterior; appropriately converged draws can approximate that expectation.
For a Metropolis–Hastings proposal \(\theta'\sim q(\theta'\mid\theta)\), accept the proposed state with probability
\[ \alpha(\theta,\theta')= \min\!\left(1,\frac{p(\theta'\mid y)q(\theta\mid\theta')} {p(\theta\mid y)q(\theta'\mid\theta)}\right). \]For a symmetric random-walk proposal, the proposal terms cancel. The chain may remain at its old state when a proposal is rejected. Neither its iterations nor its draws correspond to new economic observations: MCMC iteration is algorithmic time, not economic time. MCMC is a sampling method, not a search for the posterior mode.
Even an invariant chain can be unhelpful in a finite run if it has not reached its target distribution or mixes slowly. High acceptance need not mean efficient exploration; very small moves are frequently accepted but strongly correlated.
A regression posterior we can verify exactly
The Python and R labs simulate
\[ y_i=\beta x_i+\epsilon_i,\qquad \epsilon_i\sim N(0,1), \qquad \beta\sim N(0,100). \]With this known observation variance and prior, the conditional posterior is Gaussian:
\[ \begin{aligned} v_n&=\left(\frac{1}{100}+\sum_i x_i^2\right)^{-1},\\ \mu_n&=v_n\sum_i x_i y_i,\\ \beta\mid x,y&\sim N(\mu_n,v_n). \end{aligned} \]You can therefore compare MCMC means and standard deviations with a known target, as well as compare proposal scales by acceptance rate and lag-1 autocorrelation. A seeded run is reproducible within each language, but R and Python use different random-number generators and will not produce identical draws.
Econometrics-to-AI translation
Apply the same five questions introduced in Module 0:
| Question | This module’s answer |
|---|---|
| 1. Familiar statistical object? | Conditional/joint probability, latent state, likelihood, Bayesian posterior, stochastic recursion |
| 2. What became learnable? | Conditional distributions or graph structure when fitted; the examples mostly hold them fixed |
| 3. What objective is optimized? | A graph may be estimated by likelihood or other criteria; exact inference conditions on evidence; MCMC samples rather than optimizes |
| 4. What generates the evidence? | Observed data for fitting; conditional evidence for inference; artificial proposals/draws for MCMC |
| 5. Where does the analogy break? | Graph edges are not automatically causal; graph blankets are not temporal chains; MCMC iterations are not economic periods |
This module belongs mainly to the probabilistic structure spine. Its sampling and computational ideas also support more flexible learning systems, and later modules connect Markov states to decisions and reinforcement learning.
Interactive browser lab
Work through four experiments without network access:
- Graphical inference: vary \(B,C\), predict whether the posterior of \(A\) rises or falls, then reveal the normalized calculation.
- Markov blanket: condition on \(A,C,D\) in the five-node network; change \(E\) and check that the posterior of \(B\) does not change.
- Markov chain: choose transition rates and initial state, then compare \(p_t\) with \(\pi\). Try the alternating-chain preset to see stationarity without convergence.
- MCMC: vary proposal scale for a normal target, then compare acceptance, approximate posterior moments, and lag-1 dependence.
Predict the outcome before pressing Run experiments. Explain the distinction among a conditional distribution, a model transition, and a deliberately constructed sampling transition.
Python and R lab
Run the offline scripts and inspect their explicit numerical checks. Both use standard libraries only:
- Compute the fork posterior from joint probabilities and test the five-node Markov-blanket independence statement.
- Verify the two-state stationary equation, evolve an initial distribution, and inspect the periodic counterexample.
- Derive the exact Gaussian regression posterior and compare it against random-walk Metropolis–Hastings draws for several proposal scales.
The Python and R simulations implement the same statistical experiment, but their specific simulated observations and draws differ.
Practice:
- Write the five-node joint factorization and identify \(\operatorname{MB}(B)\). Which factor(s) cancel in \(p(B\mid A,C,D,E)\)?
- Explain why \(P(A\mid B,C)\) in the fork example is inference rather than parameter learning.
- Compute the stationary distribution of \(P\) for \(a=0.2,b=0.35\), then verify \(\pi P=\pi\).
- Why does a chain with \(a=b=1\) have a stationary distribution but fail to converge from state 0?
- Distinguish filtering, prediction, and smoothing in a hidden-state model.
- Explain why a high MCMC acceptance rate does not necessarily imply good mixing. Compare simulated mean and posterior standard deviation against the Gaussian formula.
- Apply the five-question translation rule to one concept in this module. State clearly where the econometric analogy fails.
Learner output
Submit a concise explanation and numerical evidence showing (i) why a Markov blanket supports local conditional inference, (ii) the difference between stationarity and convergence, and (iii) why MCMC draws are computational samples rather than a real-world time series. Include a small table comparing exact Gaussian posterior quantities with at least two proposal-scale runs.
Sources and further reading
Sources informing this lesson
- Russell, Stuart, and Peter Norvig. Artificial Intelligence: A Modern Approach, 4th edition. Probabilistic reasoning, graphical models, probabilistic reasoning over time, and approximate inference. Book website and contents.
- Koller, Daphne, and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques (2009). Graphical factorization, conditional independence, and inference.
- Gelman, Andrew, et al. Bayesian Data Analysis, 3rd edition (2013). Posterior computation and Markov chain Monte Carlo.
Optional further reading
- Durbin, James, and Siem Jan Koopman. Time Series Analysis by State Space Methods, 2nd edition (2012). A deeper econometrics-oriented treatment of state-space models, filtering, and smoothing.
The readings provide context; the module’s small numerical experiments are self-contained.