Multivariable Calculus

Why you should care (before definitions)

Single-variable calculus answers: “If I nudge x, how does f(x) change?”

But many quantities depend on multiple inputs:

• •f(x, y) = height of terrain at (x, y)
• •f(x, y, z) = temperature at a point in space
• •L(w₁, …, wₙ) = loss of a model with parameters w₁…wₙ

Multivariable calculus generalizes rate of change and local approximation to these settings.

Multivariable functions

A multivariable function takes several coordinates as input and returns a value.

• •Two variables: f(x, y)
• •n variables: f(x₁, x₂, …, xₙ)

A convenient way to package inputs is as a vector:

• •x = (x₁, …, xₙ)
• •Then f(x) means the same as f(x₁, …, xₙ)

Geometric intuition

For f(x, y):

• •The input (x, y) lives in the plane.
• •The output f(x, y) is a height.
• •So you can imagine a surface z = f(x, y) floating over the xy-plane.

For f(x₁, …, xₙ) with large n, you can’t visualize the surface, but the local rules (derivatives, linear approximations) still work.

The big three ideas in this node

1. 1)Partial derivatives: change with respect to one coordinate while holding others fixed.
2. 2)Gradient ∇f: the vector of all partial derivatives.
3. 3)Linearization: use ∇f to approximate small changes in f using a dot product.

These three together are the backbone of optimization, physics, and machine learning.

Why partial derivatives exist

If f depends on x and y, you might ask two different questions:

• •“What happens if I increase x but keep y the same?”
• •“What happens if I increase y but keep x the same?”

Those are different nudges, so they generally produce different rates of change.

Definition (using the limit)

For f(x, y), the partial derivative with respect to x is

∂f/∂x(x, y) = limₕ→0 [f(x + h, y) − f(x, y)] / h

Similarly,

∂f/∂y(x, y) = limₕ→0 [f(x, y + h) − f(x, y)] / h

The key phrase: hold the other variables fixed.

“Treat the others as constants” rule

In practice, to compute ∂f/∂x:

• •Differentiate as if y (and other variables) are constants.

This works because the limit definition is literally examining motion along the x-direction line.

Geometric picture: slices

For f(x, y), fix y = y₀. Then you get a single-variable function

g(x) = f(x, y₀)

Then

∂f/∂x(x₀, y₀) = g′(x₀)

So a partial derivative is the slope of a cross-section of the surface.

Notation you’ll see

Second partials and mixed partials

Once you have ∂f/∂x (a new function of x and y), you can differentiate again.

Example structure:

• •First partial: fₓ(x, y) = ∂f/∂x
• •Second partial: fₓₓ(x, y) = ∂/∂x (fₓ)
• •Mixed partial: fₓᵧ(x, y) = ∂/∂y (fₓ)

For “nice” functions (continuous second partials), the order of mixed partials doesn’t matter:

fₓᵧ = fᵧₓ

This is often called Clairaut’s theorem (or Schwarz’s theorem). You don’t need the full theorem proof here—just remember: if the function is smooth enough, mixed partials match.

Mini-derivation: partial derivative of a polynomial

Let f(x, y) = x²y + 3y.

Compute ∂f/∂x:

• •Treat y as constant.

∂/∂x (x²y) = y · ∂/∂x (x²) = y · 2x = 2xy

∂/∂x (3y) = 0

So

∂f/∂x = 2xy

Compute ∂f/∂y:

∂/∂y (x²y) = x²

∂/∂y (3y) = 3

So

∂f/∂y = x² + 3

Core intuition checkpoint

A partial derivative is not “the derivative” of f.

It’s a directional rate of change along a coordinate axis.

Later, the gradient will combine these coordinate rates into one object that can predict changes in any direction.

Why the gradient matters

Partial derivatives answer axis-aligned questions: “change x” or “change y.”

But often you change multiple coordinates at once:

• •Walk northeast: x and y both increase.
• •Update model parameters: all weights move together.

You want a single object that:

1. 1)Collects all partial derivatives.
2. 2)Predicts the change in f for a small step in any direction.

That object is the gradient.

Definition

For f(x₁, …, xₙ), define

∇f(x) = (∂f/∂x₁, ∂f/∂x₂, …, ∂f/∂xₙ)

For two variables:

∇f(x, y) = (fₓ(x, y), fᵧ(x, y))

Remember the lesson’s vector convention: gradients are vectors, so we’ll treat them as vectors even though we often write them as coordinate tuples.

Why ∇f predicts change: linearization

Suppose you are at a point x and take a small step h.

You want to approximate

f(x + h) − f(x)

In single-variable calculus:

f(x + h) ≈ f(x) + f′(x)h

Multivariable calculus generalizes this via the dot product:

f(x + h) ≈ f(x) + ∇f(x) · h

This is the first-order Taylor approximation (also called the linearization).

Show the structure in 2D

Let h = (h₁, h₂). Then

∇f(x, y) · h = (fₓ, fᵧ) · (h₁, h₂)

Compute the dot product:

∇f(x, y) · h

= fₓ(x, y)h₁ + fᵧ(x, y)h₂

So the approximation is:

f(x + h₁, y + h₂) ≈ f(x, y) + fₓ(x, y)h₁ + fᵧ(x, y)h₂

This is incredibly practical: it tells you how much each coordinate’s small change contributes to the total change.

Directional derivatives: change along a direction

If you move specifically along a direction u (a unit vector, ‖u‖ = 1) by a small distance t, your step is h = tu.

Plug into linearization:

f(x + tu) − f(x) ≈ ∇f(x) · (tu) = t(∇f(x) · u)

So the instantaneous rate of change in direction u is:

D_u f(x) = ∇f(x) · u

This formula explains two famous facts:

1) Steepest ascent direction

Because ∇f(x) · u = ‖∇f(x)‖‖u‖cosθ and ‖u‖ = 1,

D_u f(x) = ‖∇f(x)‖cosθ

This is maximized when cosθ = 1 ⇒ θ = 0, i.e., u points in the same direction as ∇f.

So ∇f points toward the direction of steepest increase.

2) Maximum slope value

The maximum directional derivative equals ‖∇f(x)‖.

Level sets and perpendicularity (2D intuition)

A level set is the set of points where f(x, y) is constant, like f(x, y) = c.

These are contour lines on a map.

Moving along a level set doesn’t change f, so the directional derivative tangent to the level set is 0.

If t is a tangent direction to the level set at a point, then

0 = D_t f = ∇f · t

A dot product of zero means perpendicular, so:

∇f is perpendicular to the level set.

This is why gradients are drawn as arrows crossing contour lines at right angles.

Summary table: partial derivatives vs gradient

This sets up the next nodes: Gradients and optimization methods that use them.

Why these tools show up everywhere

Multivariable calculus becomes essential when:

• •A system depends on multiple inputs.
• •You need to understand sensitivity: “which input matters most?”
• •You want to optimize: “how do I change inputs to increase/decrease output?”

Sensitivity analysis

Suppose f(x, y) measures cost, with x = labor hours and y = material used.

• •∂f/∂x tells you marginal cost per extra labor hour (holding materials fixed).
• •∂f/∂y tells you marginal cost per extra unit material (holding labor fixed).

The gradient combines this into “cost increases fastest if you increase inputs in the gradient direction.”

Optimization preview (without full algorithms)

If you want to minimize f, a common idea is to step opposite the gradient:

x_{new} = x_{old} − α∇f(x_{old})

where α > 0 is a step size.

You don’t need to master gradient descent here, but notice the logic from linearization:

• •For small h, change in f is ≈ ∇f · h.
• •To make f decrease, you want ∇f · h < 0.
• •Choosing h = −α∇f makes ∇f · h = −α‖∇f‖² ≤ 0.

That’s a clean “why” grounded in the dot product.

Connection to joint distributions (probability)

In probability, you often work with functions of multiple random variables:

• •joint density p(x, y)
• •log-likelihood ℓ(θ₁, …, θₙ)

Partial derivatives and gradients measure how sensitive the log-likelihood is to each parameter. This is foundational for estimation and learning.

See: Joint Distributions

Connection to multiple integrals

Derivatives tell you local change; integrals accumulate quantities over regions.

Once you can describe functions f(x, y, z) and their rates of change, the next step is computing totals:

• •area under a surface (double integrals)
• •mass of a 3D object with density ρ(x, y, z) (triple integrals)

See: Multiple Integrals

A practical mental model to keep

• •Partial derivatives: “If I freeze everything except one coordinate, what’s the slope?”
• •Gradient: “If I move a tiny bit in any direction, how does f change?”
• •Linearization: “Near this point, f behaves like a plane (or hyperplane).”

If you can do those three reliably, you’re ready for deeper gradient-based methods and higher-dimensional modeling.