Skip to content

Partial Differentiability (Real Scalar Fields)#

Definition: Partial Differentiability

Let \(f: \mathcal{D} \subseteq \mathbb{R}^n \to \mathbb{R}\) be a real scalar field and let \(\boldsymbol{p}\) and let be an interior point of \(\mathcal{D}\).

We say that \(f\) is partially differentiable at \(\boldsymbol{p}\) w.r.t. the \(k\)-th variable (\(k \in \{1, \dotsc, n\}\)) if \(f\) is directionally differentiable at \(\boldsymbol{p}\) along the \(k\)-th standard basis vector \(\boldsymbol{e}_k\).

Definition: Partial Derivative

The corresponding directional derivative is known as \(f\)'s partial derivative at \(\boldsymbol{p}\) w.r.t. \(k\)-th variable:

\[\lim_{h \to 0} \frac{f(\boldsymbol{p} + h\boldsymbol{e}_k) - f(\boldsymbol{p})}{h}\]

Notation

In general, the partial derivative of \(f\) at \(\boldsymbol{p}\) w.r.t. the \(k\)-th variable is denoted as follows:

\[\partial_k f(\boldsymbol{p})\]

If labels (for example \(x_1, \dotsc, x_n\)) are introduced for the components of \(\boldsymbol{p}\), we also use the following notations:

\[\partial_{x_k}f(x_1, \dotsc, x_n) \qquad \partial_{x_k}f(x_1, \dotsc, x_n) \qquad \frac{\partial f}{\partial x_k}(x_1, \dotsc, x_n)\]

The labels \(x, y\) and \(x, y, z\) are very common for \(\mathbb{R}^2\) and \(\mathbb{R}^3\), respectively.

We also use the term partial derivative for each real scalar field which maps each \(\boldsymbol{p}\) to \(f\)'s respective partial derivative.

Example: \(f(x, y) = x^2 y^3 + x\)

Consider the real scalar field \(f: \mathbb{R}^2 \to \mathbb{R}\) defined as follows:

\[f\left(x, y\right) = x^2 y^3 + x\]

It is partially differentiable on \(\mathbb{R}^2\):

\[\frac{\partial f}{\partial x} \left(x, y)\right) = 2x y^3 + 1 \qquad \frac{\partial f}{\partial y} \left(x, y\right) = 3 x^2 y^2\]
Example: \(f(\boldsymbol{x}) = \boldsymbol{a}^{\mathsf{T}}\boldsymbol{x}\)

Consider the real scalar ield \(f: \mathbb{R}^n \to \mathbb{R}\) defined as

\[f(\boldsymbol{x}) = \boldsymbol{a}^{\mathsf{T}}\boldsymbol{x}\]

for some fixed real vector \(\boldsymbol{a} = \begin{bmatrix} a_1 & \cdots & a_n \end{bmatrix}^{\mathsf{T}}\in \mathbb{R}^n\).

It is partially differentiable on \(\mathbb{R}^n\):

\[\partial_k f(\boldsymbol{x})= a_k\]
Example: \(f(\boldsymbol{x}) = \boldsymbol{x}^{\mathsf{T}} \boldsymbol{A} \boldsymbol{x}\)

Consider the real scalar field \(f: \mathbb{R}^n \to \mathbb{R}\) defined as

\[f(\boldsymbol{x}) = \boldsymbol{x}^{\mathsf{T}} \boldsymbol{A} \boldsymbol{x}\]

for some real matrix \(\boldsymbol{A} \in \mathbb{R}^{n \times n}\).

For its directional derivative along the \(k\)-th standard basis vector \(\boldsymbol{e}_k\), we have:

\[\begin{aligned}\frac{\partial f}{\partial \boldsymbol{e}_k} (\boldsymbol{x}) & = \lim_{h \to 0} \frac{f(\boldsymbol{x} + h\boldsymbol{e}_k) - f(\boldsymbol{x})}{h} \\ & = \lim_{h \to 0} \frac{(\boldsymbol{x} + h\boldsymbol{e}_k)^{\mathsf{T}}\boldsymbol{A}(\boldsymbol{x} + h\boldsymbol{e}_k) - \boldsymbol{x}^{\mathsf{T}} \boldsymbol{A} \boldsymbol{x}}{h} \\ & = \lim_{h \to 0}\frac{(\boldsymbol{x}^{\mathsf{T}}+ h\boldsymbol{e}_k^{\mathsf{T}})(\boldsymbol{A}\boldsymbol{x} + h\boldsymbol{A}\boldsymbol{e}_k) - \boldsymbol{x}^{\mathsf{T}} \boldsymbol{A} \boldsymbol{x}}{h} \\ & = \lim_{h \to 0} \frac{ \boldsymbol{x}^{\mathsf{T}} \boldsymbol{A}\boldsymbol{x} + h\boldsymbol{x}^{\mathsf{T}}\boldsymbol{A}\boldsymbol{e}_k + h\boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}\boldsymbol{x} + h^2 \boldsymbol{e}_k^{\mathsf{T}} \boldsymbol{A} \boldsymbol{e}_k - \boldsymbol{x}^{\mathsf{T}} \boldsymbol{A} \boldsymbol{x}}{h} \\ & = \lim_{h \to 0} \frac{h\boldsymbol{x}^{\mathsf{T}}\boldsymbol{A}\boldsymbol{e}_k + h\boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}\boldsymbol{x} + h^2 \boldsymbol{e}_k^{\mathsf{T}} \boldsymbol{A} \boldsymbol{e}_k}{h} \\ & = \lim_{h \to 0} \boldsymbol{x}^{\mathsf{T}}\boldsymbol{A}\boldsymbol{e}_k + \boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}\boldsymbol{x} + h \boldsymbol{e}_k^{\mathsf{T}} \boldsymbol{A} \boldsymbol{e}_k \\ & = \boldsymbol{x}^{\mathsf{T}}\boldsymbol{A}\boldsymbol{e}_k + \boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}\boldsymbol{x} \\ & = (\boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}^{\mathsf{T}}\boldsymbol{x})^{\mathsf{T}} + \boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}\boldsymbol{x} \\ & = \boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}^{\mathsf{T}}\boldsymbol{x} + \boldsymbol{e}_k^{\mathsf{T}}\boldsymbol{A}\boldsymbol{x} \\ & = \boldsymbol{e}_k^{\mathsf{T}}(\boldsymbol{A}^{\mathsf{T}} + \boldsymbol{A})\boldsymbol{x}\end{aligned}\]

Therefore, \(f\) is partially differentiable:

\[\partial_k f(\boldsymbol{x}) = \boldsymbol{e}_k^{\mathsf{T}}(\boldsymbol{A}^{\mathsf{T}} + \boldsymbol{A})\boldsymbol{x}\]

Definition: Continuous Partial Differentiability

We say that \(f\) is continuously partially differentiable at \(\boldsymbol{p}\) w.r.t. the \(k\)-th variable if its respective partial derivative there is continuous. If \(k\) is not specified, then we assume it holds for all \(k \in \{1, \dotsc, n\}\) and similarly for \(\boldsymbol{p}\).

Theorem: Chain Rule for Partial Derivatives

Let \(f: \mathcal{D}_f \subseteq \mathbb{R} \to \mathbb{R}\) be a real function, let \(g: \mathcal{D}_g \subseteq \mathbb{R}^n \to \mathbb{R}\) be a real scalar field and let \(\boldsymbol{p}\) be an interior point of \(\mathcal{D}_g\) such that \(g(\boldsymbol{p})\) is an interior point of \(\mathcal{D}_f\).

If \(g\) is partially differentiable at \(\boldsymbol{p}\) w.r.t. the \(k\)-th variable and \(f\) is differentiable at \(g(\boldsymbol{p})\), then the composition \(f\circ g\) is partially differentiable at \(\boldsymbol{p}\) w.r.t. the \(k\)-th variable:

\[\partial_k (f\circ g)(\boldsymbol{p}) = f'(g(\boldsymbol{p})) \partial_k g(\boldsymbol{p})\]
Example

Let \(f: \mathbb{R}_{\gt 0} \to \mathbb{R}\) be a real function which is differentiable on \(\mathbb{R}_{\gt 0}\) and consider the real scalar field \(f(||\boldsymbol{x}||)\).

We have:

\[f(||\boldsymbol{x}||) = f\left(\sqrt{\sum_{i=1}^n x_i^2}\right)\]

For the partial derivative of \(\sqrt{\sum_{i=1}^n x_i^2}\) w.r.t. to the \(k\)-th variable, we have the following:

\[\begin{aligned}\partial_k \sqrt{\sum_{i=1}^n x_i^2} = \frac{2 x_k}{2\sqrt{\sum_{i=1}^n x_i^2}} = \frac{x_k}{||\boldsymbol{x}||}\end{aligned}\]

Since \(f\) is differentiable on \(\mathbb{R}_{\gt 0}\), we know that \(f(||\boldsymbol{x}||)\) is partially differentiable on \(\mathbb{R}^n \setminus \{\boldsymbol{0}\}\) and for all \(k \in \{1, \dotsc, n\}\) its partial derivatives are the following:

\[\begin{aligned}\partial_k f(||\boldsymbol{x}||) = f'(||\boldsymbol{x}||) \frac{x_k}{||\boldsymbol{x}||}\end{aligned}\]
Proof

TODO

Higher-Order Partial Differentiability#

Schwarz's Theorem: Symmetry of Higher-Order Partial Derivatives

Let \(f: \mathcal{D} \subseteq \mathbb{R}^n \to \mathbb{R}\) be a real scalar field and let \(i_1, \dotsc, i_m \in \{1, \dotsc, n\}\) be a sequence of indices.

If \(f\) is \(m\)-times partially differentiable on an open neighborhood of \(\boldsymbol{p} \in \operatorname{int} \mathcal{D}\) and \(m\)-times continuously partially differentiable at \(\boldsymbol{p}\), then

\[\partial_{i_1} \cdots \partial_{i_m} f(\boldsymbol{p}) = \partial_{i_{\sigma(1)}} \cdots \partial_{i_{\sigma(m)}} f(\boldsymbol{p})\]

for all permutations \(\sigma: \{1,\dotsc,m\} \to \{1,\dotsc,m\}\).

Proof

TODO