Real Symmetric Matrices#

The Spectral Theorem

Every real symmetric matrix is diagonalizable and eigenvectors which belong to different eigenvalues are always orthogonal.

Proof

TODO

Theorem: Spectral Norm of Real Symmetric Matrices

If a real square matrix is symmetric, then its spectral norm is the maximum of the absolute values of its eigenvalues:

\[||A||_2 = \max \{|\lambda|: \lambda \text{ is an eigenvalue of } A\}\]

Proof

TODO

Theorem: Definiteness Criteria

A real symmetric matrix \(M \in \mathbb{R}^{n \times n}\) is

positive definite if and only if all of its eigenvalues are positive;
positive semi-definite if and only if all of its eigenvalues are non-negative;
negative definite if and only if all of its eigenvalues are negative;
negative semi-definite if and only if all of its eigenvalues are non-positive;
indefinite if and only if it has both positive and negative eigenvalues.

Proof

TODO

Theorem: Symmetry and Bounds

Let \(M \in \mathbb{R}^{n \times n}\) be a real symmetric matrix with eigenvalues \(\lambda_n \ge \cdots \ge \lambda_1\).

For all \(\boldsymbol{x} \in \mathbb{R}^n\), we have:

\[\lambda_1 ||\boldsymbol{x}||^2 \le \boldsymbol{x}^{\mathsf{T}}M\boldsymbol{x} \le \lambda_n ||\boldsymbol{x}||^2\]

Proof

By the spectral theorem, we know that \(M\) can be diagonalized into

\[M = Q\Lambda Q^{\mathsf{T}},\]

where \(Q\) is orthogonal. Therefore:

\[\boldsymbol{x}^{\mathsf{T}}M\boldsymbol{x} = \boldsymbol{x}^{\mathsf{T}}(Q\Lambda Q^{\mathsf{T}})\boldsymbol{x} = (\boldsymbol{x}^{\mathsf{T}}Q)\Lambda (Q^{\mathsf{T}}\boldsymbol{x})\]

Let \(\boldsymbol{y} = Q^{\mathsf{T}}\boldsymbol{x}\). We get:

\[\boldsymbol{x}^{\mathsf{T}}M\boldsymbol{x} = \boldsymbol{y}^{\mathsf{T}} \Lambda \boldsymbol{y}\]

Since \(\Lambda\) is diagonal with \(\Lambda = \operatorname{diag}(\lambda_1, \dotsc, \lambda_n)\), we get:

\[\boldsymbol{y}^{\mathsf{T}} \Lambda \boldsymbol{y} = \boldsymbol{y}^{\mathsf{T}} (\Lambda \boldsymbol{y}) = \boldsymbol{y}^{\mathsf{T}} \begin{bmatrix}\lambda_1 y_1 \\ \vdots \\ \lambda_n y_n\end{bmatrix} = \sum_{i = 1}^n \lambda_i y_i^2\]

For the bounds in terms of \(\boldsymbol{y}\):

\[\sum_{i = 1}^n \lambda_i y_i^2 \ge \sum_{i = 1}^n \lambda_1 y_i^2 = \lambda_1 \sum_{i = 1}^n y_i^2 = \lambda_1 ||\boldsymbol{y}||^2\]

\[\sum_{i = 1}^n \lambda_i y_i^2 \le \sum_{i = 1}^n \lambda_n y_i^2 = \lambda_n \sum_{i = 1}^n y_i^2 = \lambda_n ||\boldsymbol{y}||^2\]

Therefore:

\[\lambda_1 ||\boldsymbol{y}||^2 \le \boldsymbol{y}^{\mathsf{T}} \Lambda \boldsymbol{y} \le \lambda_n ||\boldsymbol{y}||^2\]

Express \(||\boldsymbol{y}||^2\) in terms of \(\boldsymbol{x}\):

\[||\boldsymbol{y}||^2 = \boldsymbol{y}^{\mathsf{T}}\boldsymbol{y} = (Q^{\mathsf{T}}\boldsymbol{x})^{\mathsf{T}}(Q^{\mathsf{T}}\boldsymbol{x}) = \boldsymbol{x}^{\mathsf{T}}(QQ^{\mathsf{T}}) \boldsymbol{x} = \boldsymbol{x}^{\mathsf{T}}\boldsymbol{x} = ||\boldsymbol{x}||^2\]

We thus get:

\[\lambda_1 ||\boldsymbol{x}||^2 \le \boldsymbol{y}^{\mathsf{T}} \Lambda \boldsymbol{y} \le \lambda_n ||\boldsymbol{x}||^2\]

Since \(\boldsymbol{x}^{\mathsf{T}}M\boldsymbol{x} = \boldsymbol{y}^{\mathsf{T}} \Lambda \boldsymbol{y}\), we get:

\[\lambda_1 ||\boldsymbol{x}||^2 \le \boldsymbol{x}^{\mathsf{T}}M\boldsymbol{x} \le \lambda_n ||\boldsymbol{x}||^2\]