Linear Algebra · Lesson 02

The spectral theorem

Every symmetric matrix can be diagonalised, and better than that, by a rotation or reflection of the axes: it has an orthonormal basis of eigenvectors. In that basis it does nothing but stretch each axis by its eigenvalue. Conversely, only symmetric matrices can be described that way.

Foundations8 min readRevision 1

Read freely. Sign in when you want to save your place.

Sign in to save progress

A = Q Λ Q^{T}

Q^{T} Q = I

A = A^{T}

Λ = diag (λ_{1}, \dots, λ_{n})

Chapter navigation

Intuition

Every symmetric matrix can be diagonalised, and better than that, by a rotation or reflection of the axes: it has an orthonormal basis of eigenvectors. In that basis it does nothing but stretch each axis by its eigenvalue. Conversely, only symmetric matrices can be described that way.

A well-made spring mattress, pressed anywhere, pushes back along a few perpendicular directions, each with its own stiffness. Knowing those directions and stiffnesses is knowing the mattress completely.

Orthogonal diagonalisation

A matrix is orthogonally diagonalisable when $A=QDQ^{T}$ with orthogonal and diagonal: the columns of are an orthonormal basis of eigenvectors and the diagonal of holds their eigenvalues, in the same order. Since this is a diagonalisation. The spectral theorem says exactly which real matrices allow it: the symmetric ones.

T

Q

D

Q

D

Q^{-1}=Q^{T}

A^{T}=A\iff A=QDQ^{T},\quad Q^{T}Q=I

Using it

To find $Q$ : find the eigenvalues; for each, an orthonormal basis of its eigenspace, by Gram–Schmidt if the eigenvalue is repeated; put them together. Eigenspaces of different eigenvalues are already orthogonal.
For a symmetric matrix every eigenvalue has $\dim E_\mu=a_\mu$ , since the theorem supplies $n$ orthonormal eigenvectors.
The spectral decomposition $A=\lambda_1\mathbf{q}_1\mathbf{q}_1^{T}+\cdots+\lambda_n\mathbf{q}_n\mathbf{q}_n^{T}$ writes as a sum of projections onto the lines of its eigenvectors, each scaled by its eigenvalue.
The proof below uses the fact taken on trust in the last lesson: a symmetric matrix has a real eigenvalue.

The spectral theorem

A^{T}=A\iff A=QDQ^{T}\ \text{with }Q\ \text{orthogonal}

If $A=QDQ^{T}$ , transposing gives $QD^{T}Q^{T}=QDQ^{T}$ , so $A$ is symmetric. Conversely, go by the size $n$ . Take a real eigenvalue and a unit eigenvector $\mathbf{q}_1$ , and extend it to an orthonormal basis, the columns of $Q_1$ . In that basis the matrix $Q_1^{T}AQ_1$ is symmetric, and its first column is $\lambda_1\mathbf{e}_1$ , so its first row is $\lambda_1\mathbf{e}_1^{T}$ too: what remains is a symmetric matrix of size $n-1$ , which the same argument diagonalises. Putting the pieces together gives $Q$ and $D$ for $A$ .

Proof steps

(QDQ^{T})^{T}=QD^{T}Q^{T}=QDQ^{T}

One direction: $D^{T}=D$ , so an orthogonally diagonalised matrix is symmetric.

A\mathbf{q}_1=\lambda_1\mathbf{q}_1,\quad\|\mathbf{q}_1\|=1

The other, by induction on $n$ : a symmetric matrix has a real eigenvalue; take a unit eigenvector for it.

Q_1=\begin{pmatrix}\mathbf{q}_1&\mathbf{q}_2&\cdots&\mathbf{q}_n\end{pmatrix}

Extend $\mathbf{q}_1$ to an orthonormal basis of $\mathbb{R}^n$ with Gram–Schmidt: $Q_1$ is orthogonal.

Q_1^{T}AQ_1=\begin{pmatrix}\lambda_1&\mathbf{0}^{T}\\\mathbf{0}&A_1\end{pmatrix}

Its first column is $\lambda_1\mathbf{e}_1$ , because $A\mathbf{q}_1=\lambda_1\mathbf{q}_1$ ; it is symmetric, so its first row is too, and $A_1$ is symmetric of size $n-1$ .

A_1=Q_2D_2Q_2^{T}\implies A=QDQ^{T}

By induction $A_1$ is orthogonally diagonalisable, and $Q=Q_1\begin{pmatrix}1&\mathbf{0}^{T}\\\mathbf{0}&Q_2\end{pmatrix}$ with $D=\operatorname{diag}(\lambda_1,D_2)$ works for $A$ .

Applications

Practice

Orthonormal Eigenvectors as Columns

Normalise one eigenvector for each eigenvalue and set them down as the columns of $Q$ , in the order the eigenvalues appear on the diagonal of $D$ .

\begin{pmatrix}2&1\\1&2\end{pmatrix}=Q\begin{pmatrix}3&0\\0&1\end{pmatrix}Q^{T},\quad Q=\tfrac{1}{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix}

Try it

$A=\begin{pmatrix}2&2\\2&5\end{pmatrix}$ has the eigenvector $(2,-1)$ for $1$ and $(1,2)$ for $6$ . Which $Q$ gives $A=QDQ^{T}$ with $D=\begin{pmatrix}1&0\\0&6\end{pmatrix}$ ?