Econometrics bridge: AR models, state-space recursion
Estimated time: 90 min
Lab: Open browser lab
Code: Python · R
Why this should feel familiar
An RNN processes a sequence recursively. Its hidden state summarizes earlier information and is updated using the current input. This is close in spirit to a nonlinear state equation whose transition parameters are learned.
Mathematical core
A basic recurrent unit is
\[ h_t=\tanh(W_xx_t+W_hh_{t-1}+b), \qquad \hat y_t=W_yh_t+c. \]Compare this with AR(1):
\[ y_t=\alpha+\phi y_{t-1}+\epsilon_t. \]The RNN state can be multidimensional and nonlinear, and \(h_t\) need not equal the observed outcome. Training unfolds the recursion through time and applies backpropagation through time.
Repeated multiplication by recurrent Jacobians can make gradients shrink or explode, motivating gated architectures such as LSTMs and GRUs.
Plain-English translation
An AR coefficient directly measures persistence in the observed series under the specified model. RNN hidden-state weights are distributed computational parameters and usually do not admit the same direct structural interpretation.
What changes when we move from econometrics to AI?
The recurring shift is from a small, analyst-specified representation toward a larger learned representation. The underlying statistical questions do not disappear: What is the sample? What is conditioned on? What is estimated? What objective is optimized? Which observations were used to choose hyperparameters? How will uncertainty and distribution shift be handled?
AI terminology becomes much easier when these questions are kept explicit. Whenever you encounter a new architecture, identify four objects: inputs, parameterized transformation, training objective, and evaluation design.
Interactive browser lab
Enter a numeric sequence and change the input/recurrent weights. Watch the hidden state remember, forget, or saturate.
Before changing a control, predict the direction of the effect. Then manipulate one control at a time. The goal is not merely to produce a pretty visualization; it is to connect a parameter change to a mathematical statement.
Python and R lab
The Python and R examples implement the same core object as the browser lab. The Python code intentionally favors NumPy and explicit matrix operations. The R code favors base R and explicit loops/matrix algebra. Once the mechanics are clear, the same ideas can be implemented with PyTorch, TensorFlow, JAX, torch for R, or other frameworks.
Check your understanding
- Identify the random variables, observed variables, and estimated parameters in this module.
- State the objective function or equilibrium condition.
- Name the closest concept you already knew from econometrics and one important difference.
- Predict one failure mode caused by poor data, poor optimization, or poor evaluation.
Learner output
Describe why a high recurrent weight can create persistence but also create optimization problems.