Linear Algebra, Spectral Theorem

Given a Hermitian matrix A, A is always diagonalizable. It is also the case that all eigenvalues of A are real, and that all eigenvectors are mutually orthogonal. This is given by the "Spectral Theorem":

The Spectral Theorem

Given any n × n Hermitian matrix A, there exists an n × n unitary matrix U, and an n × n diagonal matrix of real values Λ such that A = U Λ {U}^{H}

The columns of U = ( \begin{matrix}{\overset{→}{u}}_{1} & {\overset{→}{u}}_{2} & … & {\overset{→}{u}}_{n}\end{matrix} ) are the eigenvectors of U, and the diagonal entries of Λ = ( \begin{matrix}{λ}_{1} & 0 & ⋯ & 0 \\ 0 & {λ}_{2} & ⋯ & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & ⋯ & {λ}_{n}\end{matrix} ) are the corresponding eigenvalues.

In essence A can be decomposed into a "spectrum" of rank 1 projections: A = {∑}_{i = 1}^{n} {λ}_{i} ( {\overset{→}{u}}_{i} {\overset{→}{u}}_{i}^{H} )

The spectral theorem can in fact be proven without the need for the characteristic polynomial of A, or any of the derivative theorems.

Proof of the Spectral Theorem

The proof will proceed by using induction on n.

Base Case n = 1

When A = ( \begin{matrix}{a}_{1 , 1}\end{matrix} ), it must be the case that {a}_{1 , 1} is real, or else A is not Hermitian. The spectral decomposition is then simply A = ( \begin{matrix}1\end{matrix} ) ( \begin{matrix}{a}_{1 , 1}\end{matrix} ) {( \begin{matrix}1\end{matrix} )}^{H}

Inductive Case n ≥ 2

Let {\overset{→}{e}}_{i} denote the {i}^{\text{th}} standard basis vector of {ℂ}^{n}. Let {𝟎}_{k × m} denote a k × m matrix of 0s.

Let {\overset{→}{u}}_{1} be a unit length vector that maximizes {\overset{→}{u}}_{1}^{H} A {\overset{→}{u}}_{1} (recall that {\overset{→}{u}}_{1}^{H} A {\overset{→}{u}}_{1} is always real), and let {λ}_{1} = {\overset{→}{u}}_{1}^{H} A {\overset{→}{u}}_{1}. Let {U}_{1} be a unitary matrix where the first column is {\overset{→}{u}}_{1}: {U}_{1} {\overset{→}{e}}_{1} = {\overset{→}{u}}_{1}.

A = {U}_{1} {A}_{1} {U}_{1}^{H} where {A}_{1} = {U}_{1}^{H} A {U}_{1}. It will now be shown that {A}_{1} has the form {A}_{1} = ( \begin{matrix}{λ}_{1} & {𝟎}_{1 × ( n − 1 )} \\ {𝟎}_{( n − 1 ) × 1} & {A}_{\text{reduced}}\end{matrix} )

{\overset{→}{e}}_{1}^{H} {A}_{1} {\overset{→}{e}}_{1} = {\overset{→}{u}}_{1}^{H} A {\overset{→}{u}}_{1} = {λ}_{1} so that the (1,1) entry of {A}_{1} is {λ}_{1}. It will now be shown that the first row and column of {A}_{1} is filled with 0s except for the first entry. For an arbitary i = 2 , 3 , … , n, consider the parameterized unit vector {\overset{→}{u}}_{t} ( t ) = ( cos ⁡ t ) {\overset{→}{e}}_{1} + ( sin ⁡ t ) {\overset{→}{e}}_{i}.

{\overset{→}{u}}_{t} ( t {)}^{H} {A}_{1} {\overset{→}{u}}_{t} ( t ) = ( ( cos ⁡ t ) {\overset{→}{e}}_{1} + ( sin ⁡ t ) {\overset{→}{e}}_{i} {)}^{H} {A}_{1} ( ( cos ⁡ t ) {\overset{→}{e}}_{1} + ( sin ⁡ t ) {\overset{→}{e}}_{i} )

= {λ}_{1} {cos}^{2} t + ( {a}_{( i , 1 )} + {a}_{( 1 , i )} ) cos ⁡ t sin ⁡ t + {a}_{( i , i )} {sin}^{2} t

= {λ}_{1} {cos}^{2} t + 2 ℜ ( {a}_{( i , 1 )} ) cos ⁡ t sin ⁡ t + {a}_{( i , i )} {sin}^{2} t

where {a}_{( i , j )} denotes the ( i , j ) entry of {A}_{1} (ℜ and ℑ denote the real and imaginary components respectively).

\frac{d}{d t} ( {\overset{→}{u}}_{t} ( t {)}^{H} {A}_{1} {\overset{→}{u}}_{t} ( t ) ) {|}_{t = 0} = 2 ℜ ( {a}_{( i , 1 )} ). Unit vector \overset{→}{u} = {\overset{→}{u}}_{1} maximizes {\overset{→}{u}}^{H} A \overset{→}{u} which implies that \overset{→}{u} = {\overset{→}{u}}_{t} ( 0 ) = {\overset{→}{e}}_{1} is the unit vector that maximizes {\overset{→}{u}}^{H} {A}_{1} \overset{→}{u}. Therefore \frac{d}{d t} ( {\overset{→}{u}}_{t} ( t {)}^{H} {A}_{1} {\overset{→}{u}}_{t} ( t ) ) {|}_{t = 0} = 0 which gives ℜ ( {a}_{( i , 1 )} ) = 0.

Now consider the parameterized unit vector {\overset{→}{u}}_{t} ( t ) = ( cos ⁡ t ) {\overset{→}{e}}_{1} + i ( sin ⁡ t ) {\overset{→}{e}}_{i}.

{\overset{→}{u}}_{t} ( t {)}^{H} {A}_{1} {\overset{→}{u}}_{t} ( t ) = ( ( cos ⁡ t ) {\overset{→}{e}}_{1} + i ( sin ⁡ t ) {\overset{→}{e}}_{i} {)}^{H} {A}_{1} ( ( cos ⁡ t ) {\overset{→}{e}}_{1} + i ( sin ⁡ t ) {\overset{→}{e}}_{i} )

= {λ}_{1} {cos}^{2} t + ( − i {a}_{( i , 1 )} + i {a}_{( 1 , i )} ) cos ⁡ t sin ⁡ t + {a}_{( i , i )} {sin}^{2} t

= {λ}_{1} {cos}^{2} t + 2 ℑ ( {a}_{( i , 1 )} ) cos ⁡ t sin ⁡ t + {a}_{( i , i )} {sin}^{2} t

\frac{d}{d t} ( {\overset{→}{u}}_{t} ( t {)}^{H} {A}_{1} {\overset{→}{u}}_{t} ( t ) ) {|}_{t = 0} = 2 ℑ ( {a}_{( i , 1 )} ) so \frac{d}{d t} ( {\overset{→}{u}}_{t} ( t {)}^{H} {A}_{1} {\overset{→}{u}}_{t} ( t ) ) {|}_{t = 0} = 0 \ ⟹ \ ℑ ( {a}_{( i , 1 )} ) = 0. Therefore {a}_{( i , 1 )} = {a}_{( 1 , i )} = 0. It has been proven that the first row and column of {A}_{1} is filled with 0s except for the first entry.

{A}_{1} has the form {A}_{1} = ( \begin{matrix}{λ}_{1} & {𝟎}_{1 × ( n − 1 )} \\ {𝟎}_{( n − 1 ) × 1} & {A}_{\text{reduced}}\end{matrix} ), and by an inductive argument, {A}_{\text{reduced}} has the spectral decomposition {U}_{r} {Λ}_{r} {U}_{r}^{H}.

Therefore A = U Λ {U}^{H} where U = {U}_{1} ( \begin{matrix}1 & {𝟎}_{1 × ( n − 1 )} \\ {𝟎}_{( n − 1 ) × 1} & {U}_{r}\end{matrix} ) and Λ = ( \begin{matrix}{λ}_{1} & {𝟎}_{1 × ( n − 1 )} \\ {𝟎}_{( n − 1 ) × 1} & {Λ}_{r}\end{matrix} )

Where this page came from

This page was imported from Wikibooks. From “Linear Algebra” on Wikibooks, by its contributors, under CC BY-SA 4.0. Changed here: set as a page; navigation and edit links left out; each image under its own licence, credited in its caption.

Nobody has written it yet — it is the source material at a new address, which is why search engines are asked to skip it and why no one earns from it. It is up for grabs: take it on, and it is yours to rewrite and to earn from.