State-Space Models

A state-space model (SSM) is a probabilistic framework that explicitly separates an unobserved internal state (the "latent" or "hidden" state) from noisy observations. This gives two advantages: (1) you model dynamics in the hidden, typically lower-dimensional state, and (2) you perform principled inference on that state using the observation stream. Formally, in discrete time, an SSM is specified by a state (transition) model and an observation (emission) model. Two canonical families are:

• •Linear Gaussian state-space model (LGSSM), often called the Kalman model:

where $x_t\in\mathbb{R}^n$ is the latent state, $y_t\in\mathbb{R}^m$ is the observation, $A$ and $C$ are matrices, and $Q,R$ are covariance matrices. Example: an AR(1) process from "Time Series Foundations" can be written as a one-dimensional LGSSM with $A=\phi$, $C=1$, and suitable $Q,R$; e.g., for AR(1) with $\phi=0.9$, process noise variance $\sigma_w^2=1$, and observation noise variance $\sigma_v^2=2$, set $A=0.9$, $Q=1$, $C=1$, $R=2$.

• •Hidden Markov model (HMM), discrete state case:

where $z_t$ takes values in a finite set. Example: a two-state HMM with transition matrix

and emission probabilities $p(y_t|z_t)$ (e.g., categorical or Gaussian) is a simple discrete SSM.

Why build SSMs? From a modeling perspective, many time-series phenomena are naturally explained by a latent quantity evolving over time (hidden economy, object position, unobserved volatility). From an algorithmic perspective, the Markov structure (in the latent space) allows recursive, linear-time inference via dynamic programming: filtering (online) and smoothing (offline). In "Markov Chains" we learned how the memoryless property simplifies analysis; SSMs leverage that same idea: the state at time $t$ depends on the past only through $x_{t-1}$. In "Bayesian Inference" we learned updating beliefs via priors and likelihoods; the Kalman filter and forward-backward algorithm are recursive Bayesian updates tailored to sequential data.

Key inference questions in SSMs:

• •Filtering: compute $p(x_t\mid y_{1:t})$ (belief about current state given past observations). This is online.
• •Prediction: compute $p(x_{t+1}\mid y_{1:t})$ or $p(y_{t+k}\mid y_{1:t})$.
• •Smoothing: compute $p(x_t\mid y_{1:T})$ for $t<T$; this uses future information and is an offline pass that reduces uncertainty.
• •Parameter learning: estimate $A,C,Q,R$ (or transition and emission probabilities in HMMs) from data, typically via Expectation-Maximization (EM) where the E-step uses smoothing to compute sufficient statistics.

Concrete small numeric example to anchor the notation: consider a 1D LGSSM with $A=0.9$, $Q=1$, $C=1$, $R=2$, initial prior $x_0\sim\mathcal{N}(0,1)$. You observe $y_1=1.5$. The filtering problem asks for $p(x_1\mid y_1)$; the Kalman filter will give this in closed form (derived in the next section) and will produce a Gaussian with explicit mean and variance computed from the matrices above.

Intuition: the transition matrix $A$ controls how much we trust the past state to predict the next; the process noise $Q$ injects uncertainty (how much the state can change unpredictably); the observation matrix $C$ and noise $R$ govern how informative each measurement is. When $R$ is large relative to $Q$, measurements are noisy and the filter relies more on predictions; when $Q$ is large, the filter trusts new observations more.

The Kalman filter is the closed-form optimal Bayesian filter for linear-Gaussian SSMs. It computes for each $t$ the Gaussian posterior $p(x_t\mid y_{1:t})=\mathcal{N}(x_{t|t}, P_{t|t})$ by alternating a prediction step and an update (correction) step.

Equations (matrix form).

Prediction (time update):

Update (measurement update):

Derivation idea (sketch): the posterior is proportional to prior times likelihood. With Gaussians, multiply two exponentials gives another Gaussian. Completing the square yields the posterior mean and covariance above; the Kalman gain is the coefficient that balances prior uncertainty with observation uncertainty. This is a direct application of "Bayesian Inference" where the prior is the predictive Gaussian and the likelihood is Gaussian centered at $C x_t$.

Numeric 1D worked mini-example (showing every arithmetic step). Keep everything scalar so matrix operations reduce to multiplication.

Model parameters:

• •$A=0.9$, $Q=1$, $C=1$, $R=2$.
• •Prior at time 0: $x_{0|0}=0$, $P_{0|0}=1$.
• •Observation: $y_1=1.5$.

Step 1 — Predict to time 1:

• •$x_{1|0} = A x_{0|0} = 0.9\times 0 = 0$.
• •$P_{1|0} = A P_{0|0} A^\top + Q = 0.9^2\times 1 + 1 = 0.81 + 1 = 1.81$.

Step 2 — Compute innovation covariance and Kalman gain:

• •$S_1 = C P_{1|0} C^\top + R = 1\times 1.81 \times 1 + 2 = 3.81$.
• •$K_1 = P_{1|0} C^\top / S_1 = 1.81 / 3.81 \approx 0.475$. (This is a scalar; we used $P/ S$.)

Step 3 — Update state mean and covariance with observation $y_1=1.5$:

• •Innovation: $y_1 - C x_{1|0} = 1.5 - 0 = 1.5$.
• •$x_{1|1} = x_{1|0} + K_1\cdot 1.5 = 0 + 0.475\times 1.5 \approx 0.7125$.
• •$P_{1|1} = (1 - K_1 C) P_{1|0} = (1 - 0.475)\times 1.81 = 0.525\times 1.81 \approx 0.95025$.

Interpretation: the posterior mean $\approx0.71$ is between the prediction (0) and the observation (1.5), weighted by the Kalman gain which depends on the relative uncertainties. The posterior variance decreased from $1.81$ (predictive) to $0.95$ (updated): adding the observation reduced uncertainty.

Multiple-step operation and relation to "Time Series Foundations": if you write an AR(1) time series $y_t=\phi y_{t-1}+\epsilon_t$, an equivalent LGSSM can represent the hidden state as the AR(1) latent variable; the Kalman filter then provides the optimal linear minimum MSE estimator for missing values or one-step-ahead forecasts. The Kalman filter is thus the dynamic generalization of the Gaussian conjugate update we saw in "Bayesian Inference".

Important practical points:

• •Numerical stability: the covariance update $P_{t|t}=(I-K_t C)P_{t|t-1}$ can lose symmetry/positive-definiteness due to rounding; implement stabilized forms such as Joseph form

• •Initialization matters: poor initial $P_{0|0}$ or wrong $Q,R$ will bias early estimates; often use an uninformative large covariance to reflect prior uncertainty.

The Kalman filter cost per time step is O(n^3) dominated by matrix inverses, but for moderate state dimension it's efficient and exact.

Filtering answers "what is the current state given the past?" Smoothing answers "given the entire dataset, what was the state at time t?" Smoothing reduces posterior variance by incorporating future observations. Two complementary algorithms handle the main SSM families.

1) Rauch–Tung–Striebel (RTS) smoother — linear Gaussian case

After running the Kalman filter forward and storing $x_{t|t}$, $P_{t|t}$ and the predictive $x_{t+1|t}$, $P_{t+1|t}$ for $t=0\ldots T-1$, the RTS smoother performs a backward pass to compute $p(x_t\mid y_{1:T})=\mathcal{N}(x_{t|T}, P_{t|T})$ via

Derivation sketch: the smoothed estimates follow from the conditional Gaussian identity for jointly Gaussian vectors $(x_t,x_{t+1})$ conditioned on all observations: one expresses $p(x_t\mid y_{1:T}) =\int p(x_t\mid x_{t+1}, y_{1:t}) p(x_{t+1}\mid y_{1:T}) dx_{t+1}$ and uses linear-Gaussian conditional formulas. The matrix $J_t$ is the smoothing gain, analogous to $K_t$ but applied backward.

Numeric RTS example (2-step) to illustrate arithmetic. Use the previous 1D example with parameters $A=0.9,Q=1,C=1,R=2$ and two observations $y_1=1.5, y_2=0.5$. From Section 2 we computed $x_{1|1}\approx0.7125$, $P_{1|1}\approx0.95025$, and $x_{2|1}=A x_{1|1}=0.9\times0.7125\approx0.64125$, $P_{2|1}=A P_{1|1}A^\top + Q = 0.9^2\times0.95025 + 1 \approx 0.7697 + 1 = 1.7697$. After running Kalman update at time 2, suppose we compute $x_{2|2}\approx0.55$ and $P_{2|2}\approx0.9$ (numbers illustrative). Then backward step for $t=1$:

• •$J_1 = P_{1|1} A^\top / P_{2|1} = 0.95025\times 0.9 / 1.7697 \approx 0.4837$.
• •$x_{1|2} = x_{1|1} + J_1 (x_{2|2} - x_{2|1}) = 0.7125 + 0.4837 (0.55 - 0.64125) \approx 0.7125 - 0.0445 \approx 0.6680$.
• •$P_{1|2} = P_{1|1} + J_1 (P_{2|2} - P_{2|1}) J_1 \approx 0.95025 + 0.4837 (0.9 - 1.7697) 0.4837 \approx 0.95025 - 0.201 \approx 0.7492$.

The smoothed mean moved toward values that explain future data; variance decreased from $0.95025$ to $0.7492$.

2) Forward–Backward algorithm — discrete HMM case

For finite-state HMMs, smoothing is performed by the forward–backward algorithm (a dynamic-programming factorization). Define the forward variable $\alpha_t(i) = p(y_{1:t}, z_t=i)$ and backward variable $\beta_t(i) = p(y_{t+1:T} \mid z_t=i)$. Then the smoothed state posterior is

Recursions:

Concrete discrete example: two-state HMM with transitions

Emissions: $p(y_t=H\mid z=1)=0.7$, $p(y_t=H\mid z=2)=0.3$ (H = "High"). Observations: [H, L] where L denotes low with complementary probabilities. Compute $\alpha$ forward and $\beta$ backward and then $\gamma_t$; make sure to renormalize at each step to avoid underflow. Include scaling factors $c_t = \sum_i \alpha_t(i)$ when implementing for long sequences.

Relationship between the two: both RTS and forward–backward perform the same logical operation (compute posterior over states given all data) but are specialized to linear-Gaussian continuous states and discrete finite states respectively. The backward gains $J_t$ and the backward variables $\beta_t$ play analogous roles.

Smoothing is essential for parameter learning (EM): the E-step requires expected sufficient statistics like $\mathbb{E}[x_t x_t^\top]$ or $\mathbb{E}[\mathbf{1}(z_t=i, z_{t+1}=j)]$ which are computed from smoothed marginals and pairwise smoothed distributions (for HMMs, $\xi_t(i,j)=p(z_t=i,z_{t+1}=j\mid y_{1:T})$).

State-space methods are a crossroads connecting "Time Series Foundations", "Markov Chains", and "Bayesian Inference" to many applied domains. Here are concrete applications, practical considerations, and downstream connections that rely on understanding SSMs.

Major application areas with concrete examples:

• •Tracking and navigation (robotics, aerospace): an aircraft's position and velocity are latent states; radar returns are noisy observations. An LGSSM with constant-velocity dynamics ($A$ a block with 1s and timestep multipliers) and small process noise models motion; the Kalman filter provides real-time position estimates. Example: for a 2D constant-velocity model with state $x=[x,\dot x, y,\dot y]^\top$, $A$ and $Q$ are chosen from discrete-time kinematic equations and process acceleration variance. Kalman filter runs at sensor rate to fuse GPS and IMU.

• •Econometrics (state-space ARMA and unobserved components): many macro time series are modeled as combinations of trend and seasonal latent states. The Kalman smoother recovers the trend component; parameter estimation via EM or maximum likelihood (via Kalman-filtered log-likelihood) yields interpretable model parameters (trend variance, cycle frequency). In "Time Series Foundations" we learned ARMA; any ARMA model can be written in state-space form (companion form) and handled by the Kalman filter for missing data or likelihood evaluation.

• •Signal processing and communications: Kalman filters provide optimal linear equalizers and adaptive filters; smoothing is used in offline deconvolution.

• •Finance: stochastic volatility models are often written as nonlinear/non-Gaussian SSMs; particle filters or specialized smoothers are used for inference. The EM algorithm using smoothing can estimate latent volatility parameters.

• •Machine learning and probabilistic modeling: HMMs are the building block of speech recognition (Baum–Welch algorithm = forward–backward EM), bioinformatics (gene prediction), and sequence labeling. Linear-Gaussian SSMs are used in Gaussian process state-space models and as differentiable building blocks in deep learning architectures (e.g., KalmanNet).

Downstream algorithms that require SSM understanding:

• •Expectation–Maximization (EM) for parameter estimation: E-step uses smoothing to compute expectations; M-step updates $A,Q,C,R$ or HMM transition/emission matrices. Concrete: for LGSSM, the expected sufficient statistics are $\sum_t \mathbb{E}[x_t x_t^\top]$, $\sum_t \mathbb{E}[x_t x_{t-1}^\top]$, and these are computed by smoothing outputs.

• •Particle filters and sequential Monte Carlo: when dynamics or observations are nonlinear/non-Gaussian, the Kalman filter is not optimal; particle filters approximate the filtering distribution by weighted particles. The theoretical underpinning (sequential importance sampling and resampling) builds on the same Markov/Bayesian filtering logic.

• •Control theory (LQG — linear-quadratic-Gaussian): the Kalman filter provides the optimal state estimator that, combined with an LQR regulator, gives the separation principle: estimation and control decouple. A practical example: autopilot design separates state estimation (Kalman) and control (LQR), each designed using SSM parameters.

Practicalities and numerical issues:

• •Underflow in forward–backward: use log-space or per-step scaling (store $c_t=\sum_i \alpha_t(i)$ and normalize). Example: in a long HMM, raw $\alpha_t$ values shrink exponentially; the scaled $\hat\alpha_t=\alpha_t/c_t$ avoids underflow.

• •Observability and identifiability: not every SSM parameter is recoverable from data. Observability (a linear algebra condition on $(A,C)$) ensures the state can be inferred from outputs. Example: if $C=[0\;0\;\dots\;0]$, the state is unobservable no matter how many measurements you collect.

• •Model mismatch: wrong noise covariance assumptions ($Q,R$) bias the Kalman gain. In practice, you may treat $Q$ as a tuning parameter or estimate it via EM.

• •Complexity: Kalman filter is O(n^3) per time step; HMM forward–backward is O(S^2 T) where S is number of discrete states; particle filters are O(N T) with sample size N.

Concrete numeric tie-in: the log-likelihood of observations under the LGSSM can be computed incrementally from the Kalman filter via

where $m$ is the observation dimension. In practice this is used in maximum-likelihood parameter estimation; substitute numeric $S_t$ from the filter each step.

Summary: mastering Kalman filtering and forward–backward smoothing gives you powerful, computationally efficient tools to estimate hidden dynamics and to build higher-level algorithms for learning and control. In the next section (worked examples) we will step through concrete instances to cement arithmetic and intuition.