Jacobian

Why you need a new object beyond the gradient

For a scalar function g: ℝⁿ → ℝ, the gradient ∇g(x) summarizes first-order change: it’s the vector that best predicts how g changes for a small input step h.

But many important maps are vector-valued:

• •A coordinate transform f(u, v) = (x, y)
• •A physics map from state to state
• •A neural network layer that maps an input vector to an output vector

When the output is a vector, the “rate of change” can’t be captured by a single vector. Each output component depends on each input component. The Jacobian packages all those partial derivatives into one matrix.

Definition (matrix of partial derivatives)

Let f: ℝⁿ → ℝᵐ be written in components:

f(x) = (f₁(x), f₂(x), …, f_m(x))

where x = (x₁, x₂, …, x_n).

The Jacobian matrix of f at x is

J_f(x) = Df(x) = [ ∂f_i / ∂x_j ]

It is an m×n matrix whose (i, j) entry is:

(J_f(x))_{ij} = ∂f_i(x) / ∂x_j.

So:

• •Rows correspond to output components f_i
• •Columns correspond to input variables x_j

Relationship to the gradient (a comforting special case)

If m = 1 (scalar output), then J_f is 1×n:

J_f(x) = [ ∂f/∂x₁ ∂f/∂x₂ … ∂f/∂x_n ]

This is exactly the gradient as a row vector (convention). Meanwhile ∇f(x) is usually a column vector. Transpose connects them:

J_f(x) = (∇f(x))ᵀ.

So the Jacobian generalizes the gradient.

A geometric preview: local linear map

The most important intuition is not “array of partials”, but “best linear approximation.” Near a point a, f behaves like:

f(a + h) ≈ f(a) + J_f(a) h

for small h.

That expression is the multivariable analogue of the 1D approximation:

f(a + h) ≈ f(a) + f′(a) h.

Here, f′(a) is a number; in many dimensions, the derivative becomes a matrix.

Square case and determinant

If m = n, then J_f(x) is n×n and you can take its determinant:

det(J_f(x))

This scalar has a deep meaning:

• •sign(det) tells whether the map locally preserves or flips orientation
• •|det| tells how much the map locally scales n-dimensional volume

That volume-scaling is exactly what appears in change-of-variables for integrals.

Why linearization matters

Most nonlinear functions are hard to analyze globally. But if you zoom in enough, smooth functions look linear. This is the core strategy behind:

• •Newton’s method and other root-finding algorithms
• •error propagation and sensitivity analysis
• •optimization and gradient-based learning (locally linear steps)

The Jacobian is the device that turns “zooming in” into a concrete computation.

The best linear approximation statement

Let f: ℝⁿ → ℝᵐ be differentiable at a. Then there exists a linear map L(h) such that

f(a + h) = f(a) + L(h) + r(h)

where the remainder satisfies

‖r(h)‖ / ‖h‖ → 0 as ‖h‖ → 0.

That linear map L is the derivative Df(a). When we choose coordinates, L is represented by the Jacobian matrix J_f(a), and we write:

f(a + h) ≈ f(a) + J_f(a) h.

Interpreting columns and rows

Write eⱼ for the j-th standard basis vector in ℝⁿ (a 1 in position j, else 0). Then:

J_f(a) eⱼ = column j of J_f(a).

But eⱼ corresponds to “nudge only x_j”. So:

column j ≈ how the output vector changes when you increase x_j a tiny bit.

Equivalently, each row i is:

row i = [ ∂f_i/∂x₁ … ∂f_i/∂x_n ]

which is the gradient of the i-th output component (as a row). So:

• •columns = sensitivity directions in input space
• •rows = gradients of each output component

Chain rule in Jacobian form (the practical payoff)

If f: ℝⁿ → ℝᵐ and g: ℝᵐ → ℝᵏ, then the composition g ∘ f: ℝⁿ → ℝᵏ has Jacobian:

J_{g∘f}(x) = J_g(f(x)) · J_f(x).

This is the multivariable chain rule, and it looks exactly like matrix multiplication.

It’s worth pausing to connect this to “linear approximation of a composition”:

• •f turns a small input step h into an approximate output step J_f h
• •g then turns that step into J_g (J_f h)
• •overall: (J_g J_f) h

That is why the chain rule becomes matrix multiplication.

Directional derivatives via the Jacobian

For a direction u ∈ ℝⁿ, the first-order change in f at a in direction u is:

Df(a) u = J_f(a) u.

This is the vector-valued directional derivative.

In the scalar-output case (m = 1), this reduces to:

J_f(a) u = (∇f(a))ᵀ u = ∇f(a) · u

which is the familiar directional derivative formula.

Small-error propagation (a common use)

Suppose your input x has a small perturbation δx (measurement noise). Then the induced output perturbation is approximately:

δy ≈ J_f(x) δx.

So the Jacobian acts like a “local gain matrix.” This is the mathematical foundation for sensitivity analysis and for linearizing nonlinear systems in control and estimation.

Why determinants show up in integrals

Integration measures “total accumulation.” In multiple dimensions, it’s accumulation over area (2D) or volume (3D and beyond). If you change coordinates, a small patch in the new coordinates may correspond to a differently sized patch in the old coordinates.

So you need a conversion factor between tiny volume elements:

(d volume in x-space) = (scale factor) · (d volume in u-space)

That scale factor is the absolute value of the Jacobian determinant.

Local volume scaling intuition

Assume f: ℝⁿ → ℝⁿ is differentiable and J_f(a) is invertible.

Near a, f behaves like:

f(a + h) ≈ f(a) + J_f(a) h.

So near a, f is approximately the linear map h ↦ J_f(a) h. For a linear map A, the determinant det(A) gives the oriented volume scaling:

• •A maps a tiny n-dimensional parallelepiped to another
• •the volume scales by |det(A)|
• •orientation flips if det(A) < 0

Therefore, for the nonlinear map f, the local volume scaling near a is approximately |det(J_f(a))|.

The change-of-variables formula (multivariable substitution)

Let f: U ⊂ ℝⁿ → V ⊂ ℝⁿ be a bijective differentiable map with differentiable inverse (a diffeomorphism), and let g be integrable on V. Then:

∫∫…∫_V g(x) dx = ∫∫…∫_U g(f(u)) · |det(J_f(u))| du.

Here:

• •u are the new coordinates
• •x = f(u) are the old coordinates
• •dx and du represent n-dimensional volume elements

The key idea is:

dx = |det(J_f(u))| du.

2D special case: area scaling

In 2D, if f(u, v) = (x(u, v), y(u, v)), then

J_f(u, v) = [ ∂x/∂u ∂x/∂v

∂y/∂u ∂y/∂v ]

and the area element transforms as:

dx dy = |det(J_f(u, v))| du dv.

Common coordinate transforms

Polar coordinates

x = r cos θ

y = r sin θ

Compute J_f(r, θ):

∂x/∂r = cos θ ∂x/∂θ = −r sin θ

∂y/∂r = sin θ ∂y/∂θ = r cos θ

So

J_f = [ cos θ −r sin θ

sin θ r cos θ ]

and

det(J_f) = (cos θ)(r cos θ) − (−r sin θ)(sin θ)

= r cos²θ + r sin²θ

= r.

Thus:

dx dy = r dr dθ.

That single factor r is exactly the Jacobian determinant’s magnitude.

General lesson

When you see an “extra factor” like r in polar coordinates (or r² sin φ in spherical coordinates), it is not arbitrary—it is the local volume-scaling |det(J_f)|.

Orientation vs absolute value

The determinant can be negative. Integrals measure (unsigned) volume/area, so the change-of-variables formula uses:

|det(J_f)|.

If you’re doing differential geometry or oriented integrals, the sign can matter; for standard multivariable calculus integrals over regions, absolute value is the rule.

Why Jacobians show up constantly in ML

Many ML models are compositions of vector-valued functions:

x → f₁(x) → f₂(f₁(x)) → … → y

Training relies on derivatives of a loss with respect to parameters, and those derivatives are built from Jacobians (and their transposes) via the chain rule.

Even when you mostly hear “gradients,” under the hood:

• •a gradient is a Jacobian of a scalar-output function
• •backprop is repeated application of the Jacobian chain rule

Jacobian vs gradient vs Hessian (positioning)

You already know gradients. The Jacobian sits between gradient and Hessian in complexity:

A helpful mental model:

• •gradient: “slope vector”
• •Jacobian: “slope matrix”
• •Hessian: “curvature matrix”

Jacobian-transpose trick (common in least squares)

Suppose you have residuals r(x) ∈ ℝᵐ and a scalar loss:

L(x) = ½ ‖r(x)‖².

Then the gradient of L can be written using the Jacobian of r:

Let J = J_r(x) (an m×n matrix). Then:

L(x) = ½ ∑_{i=1}^m r_i(x)²

Differentiate component-wise. For j-th component:

∂L/∂x_j = ½ ∑_{i=1}^m 2 r_i(x) · ∂r_i/∂x_j

= ∑_{i=1}^m r_i(x) · J_{ij}

In matrix form:

∇L(x) = J_r(x)ᵀ r(x).

This identity appears in Gauss–Newton, Levenberg–Marquardt, and many optimization routines.

Jacobian in dynamics and stability

For a dynamical system:

x_{t+1} = f(x_t)

the Jacobian J_f at a fixed point x⋆ characterizes local stability:

x_{t+1} − x⋆ ≈ J_f(x⋆) ( x_t − x⋆ )

Eigenvalues of J_f(x⋆) determine whether perturbations shrink or grow.

Bridge to Matrix Calculus

Matrix calculus generalizes these ideas when variables and outputs are vectors/matrices and you want systematic rules.

Key bridge concepts you’ll use next:

• •organizing derivatives consistently (shapes and conventions)
• •Jacobians of common vector operations
• •combining Jacobians with chain rule for complex compositions

This node sets the foundation: once “derivative = linear map = Jacobian matrix” feels natural, matrix calculus becomes mostly careful bookkeeping plus chain rule.