Key Points for multi-variate Calculus:

1 (Calculus Mantra) When you do calculus, you are, ALWAYS,approximating a small change in a differentiable function by evaluatingTHE (unique) appropriate LINEAR function at the magnitude of thechange. That is, for sufficiently small dx , f (x + dx)− f (x)≈5f (x)dx .

2 Understanding the connection between1 The derivative of a function at a point,5f (x), 5f (x) ∈ Rn

2 The derivative of a function, 5f (·), 5f (·) : Rn→ Rn

3 The differential function, dy =5f (x)dx , Lf x (·) : Rn→ R3 Reisz Representation Theorem:

1 isomorphism between linear functions from Rn to R and vectors in Rn

2 What linear function approximates f (x + dx)− f (x)?3 Answer: Lf x (dx) =5f (x)dx4 gradient f ′(x) is to differential f ′(x)dx as ααα is to ααα · x .

() September 9, 2015 1 / 17

An illustration of the Calculus Mantra: f : R2→ R1

x1

x1

x2

x2

dx1

dx2

0

0 0

0

0

x∗

x∗

x∗ + dx†

x∗ + dx†

f(x∗)

f(x∗)

f(x∗+dx†)

f(x∗+dx†)

f(x∗)+▽f(x∗)·dx†

f(x∗)+▽f(x∗)·dx†

dfx∗(dx†)

Graph of f , with tangent plane at x∗

Graph of f ; rotated

Differential of f at x∗, eval at dx†

Figure 2. Linear Approximation of the change in a nonlinear multivariate function

() September 9, 2015 2 / 17

Directional Derivatives

() September 9, 2015 3 / 17

Directional derivatives and differentiability

1 Defn of a directional derivative: fh(x) = lim|k |→∞

(f (x0+h/k)−f (x0)

)||h||/k

2 Defn of a differentiable function: f : Rn→ R is differentiable at x0 if5f (x0) exists and if for all h ∈ Rn,

lim|k |→∞

(f (x0 + h/k)− f (x0)

)− 5 f (x0) ·h/k

||h||/k= 0

or equivalently

lim|k |→∞

(f (x0 + h/k)− f (x0)

)||h||/k

= 5 f (x0) · h||h||

3 A sufficient condition for differentiability of a function is that each of itspartial derivatives is a continuous function.

4 Relationship between directional derivatives and total derivatives.

() September 9, 2015 4 / 17

Differentiability as a vector space spanning condition

Theorem: a function f : Rn→ R is differentiable at x0 iff5f (x0) exists and forevery h ∈ Rn with ||h||= 1, the pair (h, fh(x0)) can be written as a linearcombination of the set of vectors {

(ei , fi(x0)

): i = 1, ...n}.

Theorem: a function f : Rn→ R is differentiable at x0 iff5f (x0) exists and∀h ∈ R with ||h||= 1, the pair (h, fh(x0)) belongs to the n-dimensional vectorsubspace of Rn+1 spanned by {

(ei , fi(x0)

): i = 1, ...n}, i.e., f is diffable at x0 iff

∀h ∈ Rn with ||h||= 1,h1

h2...

hn

fh(x0)

= h1

10...0

f1(x0)

+ h2

01...0

f2(x0)

+ · · · + hn

00...1

fn(x0)

() September 9, 2015 5 / 17

If f : Rn→ R is diffable then the n fi ’s “generate” all the fh’s

Because {(h, fh(x0)) : h ∈ Rn} belongs to an n-dim vector subspace of Rn+1.

which subspace?the subspace obtained by translating the tangent plane to the origin

00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1

Figure 1. Flat board against the graph

1

() September 9, 2015 6 / 17

A real-world pinwheel

() September 9, 2015 7 / 17

A nondiffable pinwheel whose directional derivatives all exist

−1

0

1

−1

0

1−4

−2

0

2

4

xy

f

−1 −0.5 0 0.5 1−0.8

−0.6

−0.4

−0.2

0

0.2

{(x,y):y = x/3}

f

Cross−section of f along y=x/3

f(x,y) = −γ(x2+y2) +

xy

sgn(x−y)√

|x2−y2|if |x| 6= |y|

0 otherwise

1

() September 9, 2015 8 / 17

The nondifferentiable pinwheel, zoomed

10.8

0.60.4

0.20

-0.2-0.4

x-0.6

-0.8-1-1

-0.8

-0.6

-0.4

y

-0.2

0

0.2

0.4

0.6

0.8

-4

-3

-2

-1

0

1

2

3

1

f

() September 9, 2015 9 / 17

A sufficient (but not necessary) condition for differentiability

Definition: f : Rn→ R is continuously differentiable at x0 if all partial derivativesof f exist and are continuous in a neighborhood of x0 ∈ Rn.

Theorem: f is differentiable if it is continuously differentiable

() September 9, 2015 10 / 17

A diffable but not continuously diffable function

2

−0.1 0 0.1 −0.1 0 0.1

f� (·)

f(·)

xx

Figure 2. A differentiable function that is not continuously differentiable

() September 9, 2015 11 / 17

That nondiffable pinwheel: partial derivative not continuous

−1

−0.5

0

0.5

1

−1

−0.5

0

0.5

1−4

−3

−2

−1

0

1

2

3

4

x

Plot of f1

y

f 1

() September 9, 2015 12 / 17

Even if f is diffable, fii ’s provide no information about fhh’s

E.g., f (x ,y) = 1.5xy−0.25(y2 + x2)slopes: {(h, fh(x0)) : h ∈ Rn} are contained in a vector space.

therefore, fi(0,0) = 0, i = 1,2 imply fh(0,0) = 0.curvatures: {(h, fhh(x0)) : h ∈ Rn} are not arcontained in a vector space.

therefore fii(0,0) < 0, i = 1,2 does not imply fhh(0,0) < 0

-0.50

x

0.5-0.5

0

0.2

0.1

0

-0.1

-0.2

0.5

y

f

() September 9, 2015 13 / 17

Cross-Sections of f

() September 9, 2015 14 / 17

Plot of fhh around unit circle

h degrees0 45 90 135 180 225 270 315 360

2nd p

artial

of X-s

ection

in dir

ection

h

-1.5

-1

-0.5

0

0.5

() September 9, 2015 15 / 17

Total derivative: some notational issues:

The total derivative of f : Rn→ R with respect to xi (general notation):

df (x)

dxi= f1(x)

∂x1

∂xi+ ...+ fi(x) + ...+ fn(x)

∂xn

∂xi(1)

implying xi may depend on other variables in addition to x1. But if it’s writtendf (x)

dxi= f1(x)

dx1

dxi+ ...+ fi(x) + ...+ fn(x)

dxn

dxi(2)

it’s implied that xi depends only on x1. Some people “multiply both sides by dxi ”

df (x) = f1(x)dx1

dxidxi + ...+ fi(x)dxi + ...+ fn(x)

dxn

dxidxi (3)

The dxi ’s in (2) and (3) have totally different interpretations

1df (x)

dxi: slope of f , univariate fcn of xi (num & denom inseparable).

2 df (x) is (bad) notation for the differential evaluated at dx (i.e.,Lx (dx)).3 dxi in (2) is part of a symbol; dxi in (3) is a small finite number

() September 9, 2015 16 / 17

Total derivative, directional derivative and differential

Consider the profit function: π : R2→ R. function π(p,q(p)) = pq(p),the total derivative of π(p̄,q(p̄)):

∆π when you increase p by one unit.

the directional derivative of π(p̄,q(p̄)): in the direction (dp,q′(p)dp)

∆π when you move one unit from (p̄,q(p̄)) in direction (dp,q′(p)dp).

but if you increase p by one unit, you don’t move one unit of length in thedirection (dp,q′(p)dp); in fact, you move ||(1,q′(p)))| | units of length inthis direction!

hence total derivative exceeds corresponding directional derivative

the total derivative of π(p̄,q(p̄)) is the differential of π at (p̄,q(p̄)),evaluated at the magnitude of the change, i.e., at (1,q′(p)).

alternatively, the total derivative is the directional derivative in the direction(1,q′(p)), multiplied by the length of the change, i.e., by ||(1,q′(p)))| |.

() September 9, 2015 17 / 17