Upload
others
View
26
Download
0
Embed Size (px)
Citation preview
Preview Multivariate functions Multivariate differential calculus Integral theory
E 600
Chapter 3: Multivariate Calculus
Simona Helmsmueller
August 21, 2017
Preview Multivariate functions Multivariate differential calculus Integral theory
Goals of this lecture:
• Know when an inverse to a function exists, be able to
graphically and analytically determine whether a function is
convex, concave or quasiconcave.
• Have an intuition about the meaning of derivatives as
measures of small change, be able to calculate the Gradient,
Jacobian and Hessian of a function
• Know all the calculation rules for derivatives (sum, quotient,
product, chain rule)
• Know the difference between partial and total derivatives
• Be able to write down the Taylor approximation of a function
• Know what a function’s differentials tell us about its
continuity and convexity
• Know and be able to apply the calculation rules of integral
theory; know and be able to apply Fubini’s theorem
Following lectures (both in this class and other
courses) will assume these goals have been reached!
Preview Multivariate functions Multivariate differential calculus Integral theory
Contents
Multivariate functions
Invertibility
Convexity, Concavity, and Multivariate Real-Valued Functions
Multivariate differential calculus
Introduction
Revision: single-variable differential calculus
Partial Derivatives and the Gradient
Differentiability of real-valued functions
Differentiability of vector-valued functions
Higher Order Partial Derivatives and the Taylor Approximation Theorems
Characterization of convex functions
Integral theory
Univariate integral calculus
The definite integral and the fundamental theorem of calculus
Multivariate extension
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Surjective functions)
A function f : X → Y is said to be surjective or onto, if for every
y ∈ Y there exists at least one x ∈ X with f (x) = y .
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Injective functions)
A function f : X → Y is said to be injective or one-to-one, if the
following property holds for all x1, x2 ∈ X :
f (x1) = f (x2)⇒ x1 = x2.
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Existence and definition of the inverse function)
Let f : X → Y be onto and one-to-one (i.e. bijective). Then there
exists a unique bijective function f −1 : Y → X with
f −1(f (x)) = x . Conversely, if such a function exists, then f is
bijective.
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Convex Real Valued Function)
Let X ⊆ Rn. A function f : X → R is convex if and only if X is a
convex set and for any two x , y ∈ X and λ ∈ [0, 1] we have
f (λx + (1− λ)y) ≤ λf (x) + (1− λ)f (y)
Moreover, if this statement holds strictly whenever y 6= x and
λ ∈ (0, 1), we say that f is strictly convex.
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Concave Real-Valued Function)
Let X ⊆ Rn. A function f : X → R is concave if and only if −f is
convex. Similarly f is strictly concave if and only if −f is strictly
convex.
Preview Multivariate functions Multivariate differential calculus Integral theory
Preview Multivariate functions Multivariate differential calculus Integral theory
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Multivariate Convexity)
Let X be a convex subset of Rn. A real-valued function
f : X → Rn is (strictly) convex if and only if, for every x ∈ X and
every z ∈ {z ∈ Rn − {0} | ‖z‖= 1}, the function g(t) = f (x + tz)
is (strictly) convex on {t ∈ R | x + tz ∈ X}.
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Quasiconvexity, Quasiconcavity)
Let X be a convex subset of Rn. A real-valued function f : X → Ris quasiconvex if and only if for any c ∈ R the set
L−c := {x |x ∈ X , f (x) ≤ c},
also referred to as f lower-level set, is convex. Iff f is such that
L+c := {x |x ∈ X , f (x) ≥ c},
also referred to as f upper-level set, is convex for any c ∈ R, then
f is said to be quasiconcave.
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Quasiconvexity, Quasiconcavity)
Let X be a convex subset of Rn. A real-valued function f : X → Ris quasiconvex if and only if
∀x , y ∈ X ∀λ ∈ [0, 1] f (λx + (1− λ)y) ≤ max{f (x), f (y)}
Iff f is such that
∀x , y ∈ X ∀λ ∈ [0, 1] f (λx + (1− λ)y) ≥ min{f (x), f (y)}
then f is said to be quasiconcave.
Preview Multivariate functions Multivariate differential calculus Integral theory
Contents
Multivariate functions
Invertibility
Convexity, Concavity, and Multivariate Real-Valued Functions
Multivariate differential calculus
Introduction
Revision: single-variable differential calculus
Partial Derivatives and the Gradient
Differentiability of real-valued functions
Differentiability of vector-valued functions
Higher Order Partial Derivatives and the Taylor Approximation Theorems
Characterization of convex functions
Integral theory
Univariate integral calculus
The definite integral and the fundamental theorem of calculus
Multivariate extension
Preview Multivariate functions Multivariate differential calculus Integral theory
“Calculus is the mathematical study of change, in the
same way that geometry is the study of shape and
algebra is the study of operations and their application to
solving equations. It has two major branches, differential
calculus (concerning rates of change and slopes of
curves), and integral calculus (concerning accumulation
of quantities and the areas under and between curves);
these two branches are related to each other by the
fundamental theorem of calculus, [...] [which] states that
differentiation and integration are inverse operations.”
Preview Multivariate functions Multivariate differential calculus Integral theory
TheoremLet f , g : R→ R differentiable at x0 with derivatives f ′(x0), g ′(x0),
and λ, µ ∈ R. Then
1. λf + µg is differentiable in x0 with
(λf + µg)′(x0) = λf ′(x0) + µg ′(x0),
2. fg is differentiable in x0 with (fg)′(x0) = (f ′g + fg ′)(x0)
3. if g(x) 6= 0 in a neighborhood of x0, then f /g is differentiable
with (f
g
)′(x0) =
f ′g − fg ′
g2(x0),
4. if all the following expressions are well-defined, then g ◦ f is
differentiable in x0 with
(g ◦ f )′(x0) = g ′(f (x0))f ′(x0).
Preview Multivariate functions Multivariate differential calculus Integral theory
• Let f be differentiable at x̄, then f is continuous at x̄.
• Let f be differentiable at x̄, then there exists a good
“linear approximation” of f in a neighborhood
• Let a and b be real numbers such that a < b. If f is
continuous and differentiable on (a,b), then:
(i) f ′(x) = 0 for all x ∈ (a,b) iff f is constant on (a,b).
(ii) f ′(x) < 0 for all x ∈ (a,b) iff f is decreasing on (a,b).
(iii) f ′(x) > 0 for all x ∈ (a,b) iff f is increasing on (a,b).
Preview Multivariate functions Multivariate differential calculus Integral theory
Convince yourself of these geometric properties by looking at the
following four functions:
1. f1(x) = x2
2. f2(x) = |x |3. f3(x) = max{k ∈ Z : k ≤ x}4. f4(x) = 1 if x ∈ Q and f4(x) = 0 if x ∈ R−Q
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Partial Derivative)
Let X ⊆ Rn and suppose f : X → R. If x̄ is an interior point of X ,
then, the partial derivative of f with respect to xi at x̄ is defined as
∂f (x̄)
∂xi:= lim
h→0
f (x̄1, ..., x̄i−1, x̄i + h, x̄i+1, ..., x̄n)− f (x̄1, ..., x̄i , ..., x̄n)
h
with h ∈ R, whenever the limit exists. Another common notation
for the partial derivative of f with respect to xi at x̄ is fi (x̄).
Preview Multivariate functions Multivariate differential calculus Integral theory
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Gradient)
Let f : Rn → R a function which is partially differentiable with
respect ot all xi , i = 1, ..., n. Then the row vector
∇f (x̄) :=(f1(x̄) f2(x̄) · · · fn(x̄)
)is called the gradient of f at x̄ .
Preview Multivariate functions Multivariate differential calculus Integral theory
https://www.khanacademy.org/math/multivariable-
calculus/multivariable-derivatives/partial-derivatives/v/partial-
derivatives-and-graphs
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Jacobian matrix)
Let f : Rn → Rm with partially differentiable component functions
f1, ..., fm : Rn → R, and let x̄ ∈ Rn. Then the Jacobian matrix of f
at x̄ is defined as
Jf (x̄) =
∂f1∂x1
(x̄)∂f1∂x2
(x̄) · · · ∂f1∂xn
(x̄)
∂f2∂x1
(x̄)∂f2∂x2
(x̄) · · · ∂f2∂xn
(x̄)
......
. . ....
∂fm∂x1
(x̄)∂fm∂x2
(x̄) · · · ∂fm∂xn
(x̄)
.
Preview Multivariate functions Multivariate differential calculus Integral theory
https://www.khanacademy.org/math/multivariable-
calculus/multivariable-derivatives/partial-derivatives-of-vector-
valued-functions/v/computing-the-partial-derivative-of-a-vector-
valued-function
https://www.khanacademy.org/math/multivariable-
calculus/multivariable-derivatives/partial-derivatives-of-vector-
valued-functions/v/partial-derivative-of-a-parametric-surface-part-1
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Multivariate Derivative)
Let X ⊆ Rn and suppose f : X → R. If x̄ is an interior point of X ,
then f is differentiable at x̄ if and only if there exists a row vector
Df (x̄) such that
lim‖h‖→0
‖f (x̄ + h)− f (x̄)− Df (x̄) · h‖‖h‖
= 0
where h is a vector in Rn. If such a vector Df (x̄) exists, we
interpret it as the derivative of f at x̄ .
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Total Differential, Partial Derivative, and Gradient)
Let X ⊆ Rn and suppose f : X → R. If x̄ is an interior point of X ,
and if f is differentiable at x̄ , then:
(i) all partial derivatives1 of f exist at x̄ and,
(ii) ∀ z in X , ‖z‖= 1: df (x̄ , z) := Df (x̄) · z = ∇f (x̄) · z
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Partial Differentiablility and Differentiability)
Let X ⊆ Rn, suppose f : X → R, and let x̄ be an interior point of
X . If all the partial derivatives of f at x̄ exist and are continuous,
then f is differentiable.
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Multivariate Derivative of Vector-Valued Functions)
Let X ⊆ Rn and suppose f : X → Rm. If x̄ is an interior point of
X , then f is differentiable at x̄ if and only if there exists a matrix
Df (x̄) such that
lim‖h‖→0
‖f (x̄ + h)− f (x̄)− Df (x̄) · h‖‖h‖
= 0
where h is a vector in Rn. If such a matrix Df (x̄) exists, we
interpret it as the derivative of f at x̄ .
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Multivariate Derivative and Jacobian Matrix)
Let X ⊆ Rn, suppose f : X → Rm, and let x̄ be an interior point
of X . Then, f is differentiable at x̄ if and only if each of its
component functions is differentiable at x̄ . Moreover, if f is
differentiable at x̄ , then:
(i) all partial derivatives of the component functions exist at x̄ ,
and
(ii) the derivative of f at x̄ is the matrix of partial derivatives of
the component functions at x̄ :
Preview Multivariate functions Multivariate differential calculus Integral theory
Jf (x̄) := Df (x̄) =
∇f 1(x̄)...
∇f m(x̄)
=
∂f 1
∂x1(x̄) · · · ∂f 1
∂xn(x̄)
... · · ·...
∂f m
∂x1(x̄) · · · ∂f m
∂xn(x̄)
∈Rm×n
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Multivariate Chain Rule)
Let X ⊆ Rn suppose g : X → Y , where Y ⊆ Rm. Further,
suppose f : Y → Z , where Z ⊆ Rp. If x̄ is an interior point of X ,
g(x̄) an interior point of Y , and g and f are differentiable at x̄ and
g(x̄), respectively, then f ◦ g is differentiable at x̄ and:
D[f ◦ g ](x) = Df (g(x̄))Dg(x̄)
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Hessian matrix)
Let X ⊆ Rn be open, suppose f : X → R, and let x̄ be an element
of X . If all second order partial derivatives of f are defined at x̄ ,
then, in the same way as the first order partial derivatives could be
gathered in a vector – the gradient –, all the second order partial
derivatives can be gathered in a matrix. Such a matrix, denoted
Hf(x̄), is called the Hessian of f at x̄ , is square, and should be
thought of as a generalized second order derivative for multivariate
real valued functions.
Hf(x̄) =
∇f1(x̄)
∇f2(x̄)...
∇fn(x̄)
=
f1,1(x̄) f1,2(x̄) · · · f1,n(x̄)
f2,1(x̄) f2,2(x̄) · · · f2,n(x̄)...
.... . .
...
fn,1(x̄) fn,2(x̄) · · · fn,n(x̄)
Preview Multivariate functions Multivariate differential calculus Integral theory
Definition(Function of class C k)
Let X ⊆ Rn be an open set, Y ⊆ R, and suppose f : X → Y . f is
said to be of class C k on X , denoted f ∈ C k(X ,Y )2, if all partial
derivatives of order less or equal to k exist and are continuous on
X .
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Schwarz’s Theorem / Young’s Theorem)
If f ∈ C k(X ), then the order in which the derivatives up to order k
are taken can be permuted.
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( nth Order Univariate Taylor Approximation)
Let X ⊆ R be an open set and consider f ∈ Cn+1(X ). Then f can
be best nth order approximated around x̄ by the nth order Taylor
expansion:
f (x̄ + h) ≈ f (x̄) +n∑
k=1
f (k)(x̄)hk
k!
where h ∈ R is such that x̄ + h ∈ X and f (k)(x) denotes f ’s
derivative of order k at x̄ . The error of approximation, also known
as the remainder of the Taylor approximation, is given by the
following formula:
Rn(h | x̄) := f (x̄ +h)− f (x̄)−n∑
k=1
f (k)(x̄)hk
k!=
f (n+1)(x + λh)
(n + 1)!hn+1
for some λ ∈ (0, 1).
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Second Order Multivariate Taylor Approximation)
Let X ⊆ Rn be an open set, f ∈ C 3(X ). Then f can be best 2nd
order approximated around x̄ by the second order Taylor expansion:
f (x̄ + h) ≈ f (x̄) +∇f (x̄) · h +1
2h′ ·Hf(x̄)h
where h ∈ Rn is such that x̄ + h ∈ X . As ‖h‖ approaches zero, the
remainder approaches zero at a faster rate than h itself.
R2(h | x̄) := f (x̄ + h)− f (x̄)−∇f (x̄) · h +1
2h′ ·Hf(x̄)h =
f (n)(x + λh)
(n + 1)!hn+1
for some λ ∈ (0, 1).
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Multivariate Convexity )
Let X be a convex subset of Rn. A real-valued function f : X → Rthat is also an element of C 2(X ) is convex if and only if, Hf (x) is
positive semidefinite for all x ∈ Int(X ). Further, if Hf (x) is
positive definite for all x ∈ Int(X ), then f is strictly convex.
Preview Multivariate functions Multivariate differential calculus Integral theory
Contents
Multivariate functions
Invertibility
Convexity, Concavity, and Multivariate Real-Valued Functions
Multivariate differential calculus
Introduction
Revision: single-variable differential calculus
Partial Derivatives and the Gradient
Differentiability of real-valued functions
Differentiability of vector-valued functions
Higher Order Partial Derivatives and the Taylor Approximation Theorems
Characterization of convex functions
Integral theory
Univariate integral calculus
The definite integral and the fundamental theorem of calculus
Multivariate extension
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Calculation rules for indefinite integrals)
Let f , g be two integrable functions3 and let a,C be constants,
n ∈ N. Then
•∫
(af (x) + g(x))dx = a
∫f (x)dx +
∫g(x)dx
•∫
xndx =xn+1
n + 1+ C if n 6= −1 and
∫1
xdx = lnx + C
•∫
exdx = ex + C and
∫ef (x)f ′(x)dx = ef (x) + C
•∫
(f (x))nf ′(x)dx =1
n + 1(f (x))n+1 + C if n 6=
−1 and
∫f (x)
f ′(x)dx = lnf (x) + C
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Integration by parts)
Let u, v be two differentiable functions. Then,∫u(x)v ′(x)dx = u(x)v(x)−
∫u′(x)v(x)dx .
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem
Let f : [a, b]→ R be continuous and define F (x) =
∫ x
af (t)dt for
all x ∈ [a, b]. Then, F is differentiable on (a, b) with
F ′(x) = f (x) for all x ∈ (a, b).
Preview Multivariate functions Multivariate differential calculus Integral theory
Theorem( Fubini’s theorem)
Let Ix = [a, b] and Iy = [c, d ] be two intervals in Rn and define
I := Ix × Iy . Let f : I → R continuous. Then∫If (x , y)d(x , y) =
∫Ix
(∫Iy
f (x , y)dy
)dx ,
and all the integrals on the right-hand side are well-defined.
Preview Multivariate functions Multivariate differential calculus Integral theory
TheoremLet A,B ∈ R two closed intervals, f : A→ R, g : B → Rcontinuous functions. Then∫
A×Bf (x)g(y)d(x , y) =
(∫Af (x)dx
)(∫Bg(y)dy
).