Differentiation of Real Vector Functions

Partial Differentiability

Definition: Partial Derivatives of a Real Vector Function

Let $f:\mathcal{D}\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^n$ be a real vector function with component functions $f_1,\cdots ,f_n$ and let $\varphi$ be a coordinate system for ${\mathbb{R}}^m$.

The partial derivative of $f$ at $\mathbf{p}\in \mathcal{D}$ with respect to the $i$-th coordinate ${\varphi }^i$ is the real vector whose entries are the partial derivatives of $f_1,\cdots ,f_n$ with respect to ${\varphi }^i$:

$$\left[\begin{array}{c}\frac{\partial f_1}{\partial {\varphi }^i}(\mathbf{p})\\ ⋮\\ \frac{\partial f_n}{\partial {\varphi }^i}(\mathbf{p})\end{array}\right]$$

NOTATION

$$\frac{\partial f}{\partial {\varphi }^i}(\mathbf{p}){\frac{\partial f}{\partial {\varphi }^i}|}_{\mathbf{p}}\frac{\partial }{\partial {\varphi }^i}f(\mathbf{p})f_{{\varphi }^i}(\mathbf{p})$$

Definition: (Continuous) Partial Differentiability

We say that $f$ is $k$-times (continuously) partially differentiable if all of its $k$-th order partial derivatives exist (and are continuous) on $\mathcal{D}$.

If $f$ is $k$-times continuously partially differentiable, then we also say that $f$ is of class $C^k$ if $f$.

NOTATION

$$f\in C^k$$

Definition: Smoothness

We say that $f$ is smooth or infinitely differentiable if it is $k$-times continuously partially differentiable for every $k\in \mathbb{N}$.

NOTATION

$$f\in C^{\infty }$$

Definition: Diffeomorphism

Let $U\subseteq {\mathbb{R}}^m$ and $V\subseteq {\mathbb{R}}^n$.

A diffeomorphism between $U$ and $V$ is a smooth bijective real vector function $F:U\to V$ with a smooth inverse $F^{-1}:V\to U$:

$$F\in C^{\infty }\text{ and }{F}^{-1}\in C^{\infty }$$

Differentiability

Definition: Differentiability of Real Vector Functions

Let $f:\mathcal{D}\to {\mathbb{R}}^n$ be a real vector function on a subset $\mathcal{D}\subseteq {\mathbb{R}}^m$ and let $\mathbf{p}$ be an accumulation point of $\mathcal{D}$.

We say that $f$ is (totally) differentiable at $\mathbf{p}$ if there exists a linear transformation $L:\mathcal{D}\to {\mathbb{R}}^n$ such that the following limit is zero.

$$\underset{\mathbf{x}\to \mathbf{p}}{\lim }\frac{||f(\mathbf{x})-f(\mathbf{p})-L(\mathbf{x}-\mathbf{p})||}{||\mathbf{x}-\mathbf{p}||}=0$$

In this case, the transformation $L$ is known as the (total) derivative or (total) differential of $f$ at $\mathbf{p}$. Note that $L$ itself depends on $\mathbf{p}$.

If $f$ is differentiable at every $\mathbf{p}$ in some $S\subseteq {\mathbb{R}}^m$, then we say that $f$ is (totally) differentiable on $S$. If $S=\mathcal{D}$, we can also just say that $f$ is (totally) differentiable.

NOTATION

We usually denote $L$ as $\mathrm{d}f$. To make it clear that $\mathrm{d}f$ depends on $\mathbf{p}$, we can write $\mathrm{d}{f}_{\mathbf{p}}$.

Theorem: Uniqueness of the Derivative

If $f$ is differentiable at $\mathbf{p}$, then its derivative $\mathrm{d}{f}_{\mathbf{p}}$ is unique.

PROOF

TODO

Theorem: Continuous Differentiability

If there exists an open neighborhood of $\mathbf{p}$ on which $f$ is continuously partially differentiable with respect to Cartesian coordinates, then $f$ is totally differentiable at $\mathbf{p}$.

PROOF

TODO

Definition: Continuous Differentiability

In this case, we say that $f$ is continuously differentiable at $\mathbf{p}$.

Theorem: Differentiability via Component Functions

A real vector function $f:\mathcal{D}\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^n$ is differentiable at $\mathbf{p}\in \mathcal{D}$ if and only if its component functions are differentiable there.

PROOF

TODO

Definition: Jacobian Matrix

Let $f:\mathcal{D}\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^n$ be a real vector function with component functions $f_1,\cdots ,f_n$ and let $\mathbf{p}\in \mathcal{D}$.

If $f$ is differentiable at $\mathbf{p}$, then its Jacobian matrix is the $n\times m$-matrix whose rows are the gradients of $f_1,\cdots ,f_n$ at $\mathbf{p}$:

$$\left[\begin{array}{ccc}-& (\nabla f_1(\mathbf{p}){)}^{\mathsf{T}}& -\\ -& ⋮& -\\ -& (\nabla f_n(\mathbf{p}){)}^{\mathsf{T}}& -\end{array}\right]$$

NOTATION

$$Df(\mathbf{p})J_f(\mathbf{p}){\mathbf{J}}_f(\mathbf{p})$$

Theorem: Total Derivative and the Jacobian

Let $f:\mathcal{D}\to {\mathbb{R}}^n$ be a real vector function on an open subset $\mathcal{D}\subseteq {\mathbb{R}}^m$ and let $\mathbf{p}$ be an accumulation point of $\mathcal{D}$.

If $f$ is differentiable at $\mathbf{p}$, then it is partially differentiable with respect to Cartesian coordinates there and the matrix representation of its derivative $\mathrm{d}{f}_{\mathbf{p}}$ with respect to the standard bases of ${\mathbb{R}}^m$ and ${\mathbb{R}}^n$ is the Jacobian matrix $Df(\mathbf{p})$.

PROOF

TODO

Theorem: Differentiability implies Continuity

If a real vector function is differentiable at some point, then it is also continuous there.

PROOF

TODO

Theorem: Linearity of Differentiation

Let $f,g:\mathcal{D}\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^n$ be real vector function.

If $f$ and $g$ are both differentiable at $\mathbf{p}\in \mathcal{D}$, then for all $\lambda ,\mu \in \mathbb{R}$, the

$$\mathrm{d}(\lambda f+\mu g{)}_{\mathbf{p}}=\lambda \mathrm{d}{f}_{\mathbf{p}}+\mu \mathrm{d}{g}_{\mathbf{p}}$$

PROOF

TODO

Theorem: Chain Rule

Let $g:\mathcal{D}\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^n$ be a real vector function on an open subset $\mathcal{D}$ such that the image $g(\mathcal{D})$ is also open subset and let $f:g(\mathcal{D})\subseteq {\mathbb{R}}^n\to {\mathbb{R}}^p$.

If $g$ is differentiable at $\mathbf{p}\in \mathcal{D}$ and $f$ is differentiable at $g(\mathbf{p})$, then the composition $f\circ g:\mathcal{D}\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^p$ is also differentiable at $\mathbf{p}$ with

$$\mathrm{d}(f\circ g{)}_{\mathbf{p}}=\mathrm{d}{f}_{\mathbf{g}(\mathbf{a})}\circ \mathrm{d}{g}_{\mathbf{p}}$$

PROOF

TODO