Dynamic Pricing

Definition and motivation

Dynamic pricing under demand uncertainty is the sequential decision problem where, at each decision epoch t = 1,2,...,T, a seller chooses a price p_t ∈ P (continuous or discrete) for a product facing random demand D_t(p_t, x_t; θ) that depends on price, observed context x_t (features), and unknown demand parameters θ. The seller observes a sales outcome y_t (e.g., quantity sold or binary purchase) and receives reward (profit) r_t = π(p_t, y_t) (e.g., revenue minus costs). The goal is to maximize cumulative expected reward E[Σ_{t=1}^T r_t] or equivalently minimize regret relative to an oracle pricing policy that knows θ*.

Why this matters

In practice, firms cannot wait to identify a demand model perfectly before setting prices: every price both earns revenue and provides information. This is the quintessential learn-and-earn exploration-exploitation tradeoff. In online marketplaces, ride-hailing, and airline revenue management, prices must be updated rapidly with streaming data and contexts (time-of-day, user features). The framework we develop combines Bayesian Forecasting (we reference that prerequisite below), Profit Maximization, and Bayesian Decision Theory to make coherent, optimal-seeming online pricing decisions.

Core intuition

There are two ingredients: (1) a model P(y | p, x, θ) for demand given price and context; (2) a Bayesian posterior π_t(θ) summarizing belief about θ at time t. Thompson sampling (TS) — a Bayesian algorithm from the multi-armed bandit literature — converts beliefs into actions by: sample θ̃_t ∼ π_t(θ), then choose p_t = argmax_{p ∈ P} E[y | p, x_t, θ̃_t]·p (or the profit objective). TS thus implements posterior-randomized decision rules that approximate the Bayes-optimal policy while being computationally tractable.

Formalization (toy deterministic-demand example)

Suppose a linear demand curve Q(p) = a − b p + ε with ε ∼ N(0, σ^2). The seller's instantaneous expected revenue at price p, given parameters θ = (a,b), is

The price maximizing revenue for known θ solves the first-order condition (Profit Maximization prerequisite):

If a and b are unknown, TS samples θ̃ from the posterior and then uses p^(θ̃). For example, if posterior mean is (â,b̂)=(10,1) then p^=10/2=5 and expected revenue is R(5) = 5(10 − 1·5) = 25. This concrete numeric link between posterior sample and pricing is the heart of TS.

Relation to prerequisites

• •In Bayesian Forecasting, we learned to compute posterior predictive distributions and structural model inference; TS uses the posterior over θ and the posterior predictive for demand to choose actions.
• •In Profit Maximization, we used first-order conditions to derive optimal price given parameters; TS plugs in a sampled θ to perform that maximization.
• •In Bayesian Decision Theory, we learned Bayes risk and posterior expected loss minimization; TS is a heuristic that samples from the posterior and chooses a greedy action, approximating posterior sampling of the Bayes-optimal policy while inducing exploration.

Algorithmic skeleton

For a context-free (or time-invariant) parametric demand model P(y | p, θ), Thompson sampling proceeds:

1. 1)Initialize prior π_0(θ).
2. 2)For each t = 1,...,T:

a. Sample θ̃_t ∼ π_{t-1}(θ).

b. Compute p_t = argmax_{p ∈ P} E[r | p, θ̃_t] (profit-maximizing price under θ̃_t).

c. Offer p_t, observe y_t (e.g., units sold), and reward r_t.

d. Update posterior π_t(θ) ∝ P(y_t | p_t, θ) π_{t-1}(θ).

This converts posterior uncertainty into randomized pricing that automatically explores: when posterior uncertainty is large, samples vary, so prices vary; when posterior concentrates, prices stabilize.

Worked mini-example: linear-Gaussian demand with conjugate prior

Model: observed (possibly fractional) sales y_t = a − b p_t + ε_t, ε_t ∼ N(0, σ^2). Unknown parameters θ = (a,b) ∈ R^2. Take Gaussian prior θ ∼ N(μ_0, Σ_0). The likelihood for a single observation is Gaussian, so posterior is Gaussian. Stack data: define X_t as t×2 matrix where each row is [1, −p_i] for i ≤ t (note sign convention to keep coefficients positive), and y_{1:t} the observed sales.

Posterior update (standard Bayesian linear regression):

Numeric illustration: suppose σ^2=1, prior μ_0=(8,0.8), Σ_0=diag(4,0.25) (large uncertainty in intercept, modest in slope). After observing one datapoint at p_1=5 with y_1=6 (sold 6 units):

X_1=[1, −5], y_1=6.

Compute

Then invert to find Σ_1 (compute numerically), and μ_1 using the formula. One can then sample θ̃_2 ∼ N(μ_1, Σ_1) and compute p_2 = ã/(2 b̃) where θ̃_2=(ã,b̃). If ã=9.2 and b̃=1.05 (a possible draw), p_2≈9.2/(2·1.05)≈4.38.

Why conjugacy matters

Conjugate Gaussian updates are algebraically simple and fast; they let us sample θ̃_t instantly and compute closed-form optimal prices p^*(θ̃_t). This is important in real-time systems. In nonconjugate models (e.g., logistic purchase probability), one can use approximate posterior sampling (MCMC, Laplace, variational) or embedding in particle filters.

Regret intuition and theoretical guarantees (sketch)

Define Bayes regret after T steps as BR(T)=E[Σ_{t=1}^T (r^(x_t,θ) − r(p_t, y_t))] where expectation is over θ* drawn from the prior and all randomness. For parametric linear-Gaussian models under boundedness assumptions, Thompson sampling achieves regret scaling like O(d√T log T) for d-dimensional parameter θ (results from linear bandit literature). In the pricing context, continuous action spaces and possibly non-linear revenue curves complicate bounds; one common approach is to show that the revenue function is Lipschitz in θ and p and then couple posterior concentration with discretization of P to get regret bounds. These theoretical tools rely on Bayesian Forecasting-type concentration results for posteriors and on Profit Maximization derivatives to bound loss from a sampled parameter.

Contextual extension: models where demand depends on context x_t

Often demand varies with observable covariates x_t (user features, time of day, inventory state). The demand model is P(y | p, x, θ). In a linear contextual pricing model you might write

or equivalently include interactions with price:

where θ = (β, φ) and the revenue objective is R(p; θ, x_t) = p⋅E[y_t | p, x_t, θ]. The action set of prices can be continuous; the maximization step for sampled θ̃ is a small optimization problem possibly convex (or closed-form for simple models).

Thompson sampling in contextual bandits

TS generalizes straightforwardly: sample θ̃_t ∼ π_{t-1}(θ), then pick p_t = argmax_{p∈P} E[r | p, x_t, θ̃_t]. Posterior updates incorporate the context as design matrix rows depend on x_t and p_t. The linear-Gaussian conjugate update becomes:

Let φ(p,x) be the feature vector (e.g., φ = [x; p x]) so y_t = φ(p_t,x_t)^T θ + ε_t. Stack Φ_t rows φ(p_i,x_i) for i≤t. Then

Numeric example (2-dim context)

Let x_t ∈ R be a single context (e.g., time-of-day indicator), model y = β_0 + β_1 x + φ p + ε (no interaction for simplicity). Then φ(p,x)=[1, x, p]^T and θ=(β_0, β_1, φ). Suppose Σ_0=I·9, μ_0=0, σ^2=4. At t=1, x_1=1, p_1=5, observe y_1=7. Then Φ_1=[1,1,5], compute Σ_1^{-1}=I/9 + (1/4) Φ_1^T Φ_1 = diag(1/9,1/9,1/9) + (1/4)[1;1;5][1,1,5]. Numeric operations produce Σ_1 and μ_1, enabling sampling and pricing.

Practical issues and extensions

1) Discrete vs continuous price sets. If P is continuous but the revenue maximization is non-convex, one practical solution is to optimize p by gradient methods on the sampled revenue function R(p; θ̃_t, x_t) or to discretize P finely and treat it as a multi-armed bandit. Example: discretize P into K=100 grid points and run TS over K arms where arm k has parametric likelihood induced by offering that price. Discretization creates approximation error bounded by the Lipschitz constant of revenue.

2) Binary purchase models and generalized linear models (GLMs). If customers either purchase or not, model the purchase probability as logistic: Pr(y=1 | p,x,θ) = σ(φ(p,x)^T θ). Conjugacy is lost; approximate posteriors via Laplace approximation (Gaussian around MAP) or variational inference. TS then samples θ̃ from the approximate posterior. Numeric example: φ=[1, p], θ prior N(0,10I), observe a purchase at p=3, update via Laplace.

3) Inventory and constrained pricing (connection to Profit Maximization). When inventory I_t is limited, the decision must account for remaining stock: the reward is revenue but actions affect future feasibility. One can embed TS into approximate dynamic programming: sample θ̃_t, compute approximate dynamic program (e.g., myopic price that accounts for remaining inventory through a shadow value λ estimated by dual ascent), and act greedily under θ̃_t. This is a practical and Bayesian-aware heuristic for learn-and-earn when capacity constraints are active.

4) Heteroskedastic noise and nonstationarity. In many e-commerce settings the noise variance or θ itself drifts. Two practical remedies: (a) use discounting/forgetting in the posterior (e.g., treat prior update with weight α<1) or (b) structure θ evolution with a state-space model and apply particle or Kalman filters (this directly leverages Bayesian Forecasting skills). Numeric example: a random-walk prior θ_{t+1}=θ_t+η_t with η_t∼N(0,Σ_η) inserted into the conjugate update yields a Kalman filter.

Theoretical note: regret scaling and model misspecification

When the model is correctly specified, TS for linear contextual bandits has regret bounds O(d√T log T). However, pricing models are often misspecified (wrong functional form, unobserved confounders). In misspecified settings, Bayesian posteriors can concentrate on pseudo-true parameters; TS may then concentrate on suboptimal prices. Robust approaches include: robust priors, model averaging (also in Bayesian Forecasting), and explicit exploration bonuses (upper-confidence bound methods) calibrated to misspecification magnitude.

Real-world use cases

1) Online retail and e-commerce: personalized prices (coupons, dynamic discounts) where x_t contains user features, device, and historical elasticity signals. A pipeline uses a Bayesian contextual demand model updated daily with streaming logs and Thompson sampling to select A/B test-like prices at the user session level.

2) Ride-hailing and surge pricing: supply-demand imbalance interacts with price to control demand and reallocate supply. Context includes location and current fleet state; demand is stochastic and partially observed. TS can be applied when demand response is parametrically modeled, but constraints from policy (caps) and fairness require additional constraints in the optimization step.

3) Revenue management (airlines, hotels): fare classes and inventory controls combined with TS-type learning for unobserved price sensitivities; here the state includes remaining seats (connects to the inventory extension above).

Industry implementation pattern

• •Model selection and priors: practitioners often use hierarchical priors across SKUs (Bayesian Forecasting and Bayesian model averaging) to borrow strength and stabilize early decisions. For example, a hierarchical normal prior for intercepts a_i ∼ N(μ_a, τ_a^2) across products i, updated with pooled data.

• •Computational considerations: closed-form conjugate models are favored in real-time systems. For nonconjugate models, use fast Laplace approximations or sequential Monte Carlo with a moderate number of particles (e.g., 100–1000) to represent π_t(θ).

• •Business constraints: include minimum advertised prices, competitor reactions, regulatory fairness constraints. These constraints alter the argmax step: instead of pure revenue maximization, solve a constrained optimization incorporating fairness/legality.

Connections to research and further reading

• •Theoretical papers on Thompson sampling include Agrawal and Goyal (2013) for multi-armed bandits and Russo and Van Roy for information-theoretic analyses. For linear contextual bandits and regret bounds see Abbasi-Yadkori et al. (2011) and Dani et al. (2008). These results underpin regret guarantees referenced earlier.

• •For dynamic pricing with inventory and learning, see Besbes and Zeevi (2009, 2015) for nonstochastic and stochastic demand settings, and Ferreira et al. (2016) for structural approaches.

What this enables

Mastering TS-based dynamic pricing enables data-driven, principled pricing in settings where demand is unknown and decisions must be sequential. It connects Bayesian Forecasting (posterior computation and model selection), Profit Maximization (analytic pricing rules given parameters), and Bayesian Decision Theory (posterior-based decisions to manage exploration-exploitation). Practically, it opens pathways to personalized pricing, real-time experimentation at scale, and integration with inventory and supply-side constraints.