Stochastic Processes

A stochastic process is a collection of random variables indexed by time: $\{X_t\}_{t\in T}$, where $T$ is typically $\{0,1,2,\dots\}$ (discrete time) or an interval $[0,\infty)$ (continuous time). Intuitively, a stochastic process describes the evolution of a random system observed at different times. Two canonical continuous-time families are Poisson processes (jump/counting processes) and Brownian/Wiener processes (continuous-path Gaussian noise).

Why study these two? Poisson processes capture "events" that occur at random times (phone calls, earthquakes), while Brownian motion captures the accumulation of tiny, independent random disturbances (particle diffusion, financial returns at fine time scales). They also form building blocks for more complicated continuous-time models (jump-diffusions, renewal processes, SDEs) used across queueing, finance, physics and engineering.

Connection to prerequisites

• •In Markov Chains, we learned about memoryless dynamics (the Markov property) and transition matrices. Many continuous-time processes inherit a memoryless property (e.g., exponential interarrival times in Poisson processes) and are continuous-time Markov processes.
• •In Common Distributions, we learned the Poisson, exponential and normal laws; these are the marginal/increment distributions for Poisson and Wiener processes.
• •In Integrals, we learned Riemann sums and limits; in continuous-time stochastic calculus we use mean-square limits of Riemann-type sums to define stochastic integrals.

Poisson process — definition and intuition

A counting process $N(t)$ is a Poisson process with rate $\lambda>0$ if:

1) $N(0)=0$.

2) It has independent increments: for $0\le s<t$, $N(t)-N(s)$ is independent of the past up to time $s$ (this is a continuous-time analog of the Markov property learned in Markov Chains).

3) It has stationary increments: distribution depends only on $t-s$.

4) $P(N(h)=1)=\lambda h+o(h)$, $P(N(h)\ge2)=o(h)$ as $h\downarrow0$ (no multiple jumps in infinitesimal time).

The exact distribution: for $t>0$,

Concrete numeric example: with $\lambda=3$ events/hour and $t=2$ hours, $N(2)\sim\mathrm{Poisson}(6)$ so

Interarrival times $T_1,T_2,\dots$ are iid exponential$(\lambda)$; e.g., with $\lambda=3$, $P(T_1>1) = e^{-3\cdot1}\approx 0.0498$ (probability the first event takes more than 1 hour).

Brownian motion / Wiener process — definition and intuition

A standard Brownian motion (or Wiener process) $\{W_t\}_{t\ge0}$ satisfies:

1) $W_0=0$.

2) Independent increments.

3) Stationary Gaussian increments: $W_t-W_s\sim N(0, t-s)$ for $t>s$.

4) Almost surely continuous paths.

This process is the central continuous-time Gaussian model and arises as a scaling limit of random walks (Donsker's invariance principle). Numeric example: $W_2-W_1\sim N(0,1)$, so $P(|W_2-W_1|>1.96)\approx0.05$.

Key qualitative facts:

• •Scalability: $aW_{t/a^2}$ is also Brownian motion (diffusive scaling).
• •Paths are a.s. nowhere differentiable, but of finite quadratic variation: the sum of squared increments over a partition of $[0,t]$ tends to $t$.

Continuous-time stochastic models

Two canonical classes: pure jump (Poisson) and continuous diffusion (Brownian). Real-world models often combine both (jump-diffusions). The mathematical machinery to manipulate SDEs driven by Brownian motion is Ito calculus, which modifies the ordinary chain rule to account for the quadratic variation of Brownian paths.

This section sets the stage: the next sections derive core formulas for Poisson processes and Brownian/Ito calculus and show worked examples.

Distribution and derivation (binomial limit)

A standard construction shows the Poisson law as the limit of Binomial$(n,p_n)$ with $p_n=\lambda/n$ and $n\to\infty$: for fixed $k$,

Concrete numeric check: take $\lambda=2$, $n=1000$, $p_n=0.002$. The probability of $k=0$ is approximately $(1-0.002)^{1000}\approx e^{-2}\approx0.1353$.

Interarrival times and memoryless property

From the Poisson process with rate $\lambda$ the waiting time until the first event $T_1$ satisfies

so $T_1\sim\mathrm{Exp}(\lambda)$. Exponential distributions are memoryless: $P(T_1>t+s\mid T_1>t)=P(T_1>s)$. In Markov Chains, we saw discrete memoryless geometric waiting times; exponential is the continuous analogue.

Order statistics representation

Given $N(t)=n$, the $n$ arrival times conditional on $N(t)=n$ are distributed as the order statistics of $n$ iid Uniform$(0,t)$ variables. Example: with $\lambda=3$, $t=2$ and conditioning on $N(2)=2$, the two arrival times have joint density equal to $2!/2^2$ on $0< u_1<u_2<2$; marginally each arrival is likely near the center.

Superposition and thinning

• •Superposition: if $N_1,N_2$ are independent Poisson processes with rates $\lambda_1,\lambda_2$, then $N=N_1+N_2$ is Poisson$(\lambda_1+\lambda_2)$.

Numeric example: merging two independent streams at rates 2 and 5 per hour yields a Poisson rate 7 per hour.

• •Thinning: each arrival of a Poisson$(\lambda)$ process is kept independently with probability $p$ to produce a sub-process; the kept events form Poisson$(p\lambda)$ and the discarded ones Poisson$((1-p)\lambda)$, independent.

Numeric example: thinning with $p=0.3$ a Poisson process with $\lambda=10$ gives a kept process of rate $3$.

Moment generating and PGF

The probability generating function (PGF) for $N(t)$ is

Numeric example: with $\lambda=4$, $t=0.5$, $G_N(0.5)=\exp(4\cdot0.5(0.5-1))=\exp(2(-0.5))=e^{-1}\approx0.3679$.

A simple applied calculation — probability of at least k events

Question: rate $\lambda=2$ per hour, time $t=3$ hours. What is $P(N(3)\ge3)$?

Solution: $N(3)\sim\mathrm{Poisson}(6)$, so

Generator viewpoint (continuous-time Markov chains)

For a pure birth Poisson process (counting upward by ones), its forward generator acting on bounded functions $f:\mathbb{Z}_{\ge0}\to\mathbb{R}$ is

This mirrors the discrete Markov Chains generator learned earlier, now with rate $\lambda$ for jumps. For example, choose $f(n)=n$. Then $(\mathcal{L}f)(n)=\lambda$ and solves the ODE $dE[N_t]/dt=E[(\mathcal{L}f)(N_t)]=\lambda$, consistent with $E[N_t]=\lambda t$.

Takeaway from this section: Poisson processes give a clean, tractable model for random discrete events; many useful transformations (conditioning, thinning, superposition) are exact and have simple probabilistic proofs that rely on independent and stationary increments and the exponential memoryless property. All formulas above had concrete numeric instantiations to make computation immediate.

Brownian motion (Wiener process) recap and basic computations

Recall $W_t$ is standard Brownian motion with independent stationary Gaussian increments: $W_t-W_s\sim N(0,t-s)$. Key moment: $E[W_t]=0$, $\mathrm{Var}(W_t)=t$. Concrete numeric example: for $t=4$, $W_4\sim N(0,4)$ so $P(|W_4|>2) = P(|N(0,1)|>1)\approx0.3173$ because $2/\sqrt{4}=1$.

Quadratic variation — the source of Ito's extra term

Take a partition $\Pi_n=\{0=t_0<t_1<\dots<t_n=t\}$ with mesh $\max(t_{i+1}-t_i)\to0$. Define the quadratic variation along the partition:

Because increments are independent with variance $t_{i+1}-t_i$, we have

Also $\mathrm{Var}(Q(\Pi_n))\to0$ as mesh shrinks, so $Q(\Pi_n)\to t$ in probability and almost surely along appropriate subsequences. Concrete numeric check: take uniform partition into 100 intervals on $[0,1]$; each increment has variance $0.01$, expected sum of squares is 1.

This nonzero quadratic variation (unlike smooth paths where it is 0) causes Ito calculus to acquire an extra term relative to ordinary calculus.

Ito integral — definition sketch

Let $\{\phi(t)\}$ be a predictable process (non-anticipating, i.e., depends only on the past). For simple processes that are piecewise constant on partitions, define

The Ito integral is the mean-square limit as the mesh goes to zero:

with convergence in L^2. Example: if $\phi(s)=1$ constant, then the integral is $W_t$ itself: $\int_0^t 1\,dW_s=W_t$.

Isometry and computations

The Ito isometry gives

Numeric example: if $\phi(s)=2$ constant on $[0,1]$, then $E[(\int_0^1 2\,dW_s)^2]=E[\int_0^1 4\,ds]=4$. Indeed $\int_0^1 2\,dW_s\sim N(0,4)$.

Ito's formula (stochastic chain rule)

If $X_t$ solves an SDE

and $f(t,x)$ is $C^{1,2}$ (once differentiable in $t$, twice in $x$), then

Note the $\tfrac12 b^2 \partial_{xx}f$ term coming from quadratic variation. Concrete numeric application: let $f(x)=x^2$ and $X_t=W_t$ (so $a=0,b=1$). Then Ito's formula yields

Take expectation to get $dE[W_t^2]=dt$, so $E[W_t^2]=t$, matching the variance property. Numeric check: at $t=3$, $E[W_3^2]=3$.

Proof sketch of Ito's formula for $f(x)$ (time-homogeneous case)

Use Taylor expansion on increments:

For $\Delta X_t = a\Delta t + b\Delta W_t$, the linear term gives $f'(X_t)(a\Delta t + b\Delta W_t)$; the quadratic term yields $\tfrac12 f''(X_t)b^2(\Delta W_t)^2$. But $(\Delta W_t)^2\approx \Delta t$ (quadratic variation), so the second-order term contributes $\tfrac12 b^2 f''(X_t)\Delta t$. Higher-order terms vanish in the limit because $\Delta W_t = O(\sqrt{\Delta t})$.

Martingales and exponential martingales

A useful family: for constant $\theta$, the process

is a martingale. Numeric example: with $\theta=1$ and $t=2$, $E[M_2]=1$ and $M_2=\exp(W_2 - 1)$.

SDE example and solution technique

Consider the linear SDE (Ornstein-Uhlenbeck variant) for constants $\theta,\sigma$:

The integrating factor solution (variation of constants) yields

Numeric example: with $\theta=1,\sigma=2,x_0=1,t=1$, the expectation is $E[X_1]= e^{-1}\approx0.3679$ and variance

Takeaway: Ito calculus alters the ordinary calculus chain rule by a quadratic-variation term. The Ito integral is a mean-square limit defined for non-anticipating integrands, and Ito's formula is the workhorse for manipulating functions of SDE solutions.

Black–Scholes and quantitative finance

One of the clearest applications is option pricing. Model a stock price by the geometric SDE

Ito's formula applied to $\log S_t$ gives

so the explicit solution is

Concrete numeric example: take $s_0=100$, $\mu=0.05$, $\sigma=0.2$, $t=1$ year. Then

Black–Scholes uses risk-neutral pricing ($\mu$ replaced by risk-free rate $r$) and properties of lognormal distributions to price European options analytically.

Queueing, telecommunications and reliability

Poisson processes are the standard model for arrival processes in queues (e.g., M/M/1 queue). Key performance measures — waiting times and queue lengths — are derived from Poisson/exponential properties. Example numerical calculation: with arrival rate $\lambda=5$/hr and service rate $\mu=6$/hr, utilization $\rho=\lambda/\mu\approx0.833$; the stationary average number in system for M/M/1 is $\rho/(1-\rho)\approx5$ customers.

Physics and diffusion

Brownian motion models particle diffusion: the heat equation is the forward equation (Fokker–Planck) for the probability density of Brownian motion. The diffusion constant ties $\mathrm{Var}(W_t)$ to physical diffusivity.

Stochastic control, filtering and estimation

Ito calculus enables stochastic optimal control (Hamilton–Jacobi–Bellman PDEs) and stochastic filtering (Kalman–Bucy filter for linear Gaussian SDEs). For example, the linear SDE + Gaussian noise assumptions produce closed-form filters because all conditional distributions remain Gaussian.

Statistics for stochastic processes

Parameter estimation for rates $\lambda$ in Poisson models or drift/diffusion coefficients in SDEs uses likelihoods based on increments and Girsanov transformations. For example, by observing a Poisson process on $[0,T]$ with $N(T)=n$, the MLE for $\lambda$ is $\hat{\lambda}=n/T$.

Machine learning and stochastic optimisation

Stochastic gradient methods can be viewed as discrete-time stochastic processes; diffusion limits lead to SDE approximations describing algorithm behaviour and escape probabilities from basins of attraction.

Hybrid models and jump-diffusions

Real applications often combine Poisson jumps and Brownian diffusion: e.g., financial returns may have continuous Gaussian noise plus occasional large jumps modeled by a compound Poisson process. SDEs with jumps require an extended Ito formula incorporating jump terms.

Practical modeling checklist

• •Decide whether events are discrete (Poisson) or continuous/noisy (Brownian) or both.
• •Check stationarity/independent increments assumptions; these yield exact tractability.
• •Use Ito's formula to convert between SDEs and deterministic PDEs (Fokker–Planck, backward Kolmogorov).

Downstream methods enabled

• •SDE solution techniques and explicit formulas (Black–Scholes, Ornstein–Uhlenbeck).
• •Stochastic stability, large deviations, and exit-time problems.
• •Nonlinear filtering and stochastic control.

Concrete final illustration: pricing expectation under geometric Brownian motion. Using the $S_t$ formula above with $s_0=100,\mu=0.05,\sigma=0.2,t=1$, the distribution of $S_1$ is lognormal, and the probability $P(S_1>110)=P\left(\sigma W_1 > \log(1.1) - (\mu-\tfrac12\sigma^2)\right)$. Numeric compute: $\log(1.1)\approx0.09531$, $(\mu-0.5\sigma^2)=0.05-0.02=0.03$, so threshold for $W_1$ is $(0.09531-0.03)/0.2\approx0.32755$. Thus $P(S_1>110)=P(W_1>0.32755)\approx0.3716$.

This section shows how Poisson processes, Brownian motion and Ito calculus are not abstract curiosities but precise tools that produce explicit models, closed-form calculations, and pathwise constructions for a wide range of applications.