Linear Algebra, Projection Onto a Subspace

This subsection, like the others in this section, is optional. It also requires material from the optional earlier subsection on Combining Subspaces.

The prior subsections
project a vector onto a line by decomposing it into two parts: the
part in the line {\text{proj}}_{[ \overset{→}{s} \ ]} ( \overset{→}{v} \ )
and the rest \overset{→}{v} − {\text{proj}}_{[ \overset{→}{s} \ ]} ( \overset{→}{v} \ ).
To generalize projection to arbitrary subspaces, we follow this idea.

Definition 3.1

For any direct sum V = M ⊕ N and any \overset{→}{v} ∈ V, the projection of \overset{→}{v} onto M along N is

{\text{proj}}_{M , N} ( \overset{→}{v} \ ) = \overset{→}{m}

where \overset{→}{v} = \overset{→}{m} + \overset{→}{n} with
\overset{→}{m} ∈ M , \ \overset{→}{n} ∈ N.

This definition doesn't involve a sense of
"orthogonal" so we can apply it to spaces other than subspaces of
an {ℝ}^{n}.
(Definitions of orthogonality for other spaces are perfectly possible, but we
haven't seen any in this book.)

Example 3.2

The space {ℳ}_{2 \ × \ 2} of {}2 \ × \ 2 matrices is the
direct sum of these two.

M = \{ ( \begin{matrix}a & b \\ 0 & 0\end{matrix} ) \ | \ a , b ∈ ℝ \} \ N = \{ ( \begin{matrix}0 & 0 \\ c & d\end{matrix} ) \ | \ c , d ∈ ℝ \}

To project

A = ( \begin{matrix}3 & 1 \\ 0 & 4\end{matrix} )

onto M along N, we first fix bases for the two subspaces.

{B}_{M} = ⟨ ( \begin{matrix}1 & 0 \\ 0 & 0\end{matrix} ) , ( \begin{matrix}0 & 1 \\ 0 & 0\end{matrix} ) ⟩ \ {B}_{N} = ⟨ ( \begin{matrix}0 & 0 \\ 1 & 0\end{matrix} ) , ( \begin{matrix}0 & 0 \\ 0 & 1\end{matrix} ) ⟩

The concatenation of these

B = {B}_{M} \ {}^{⌢} \ {B}_{N} = ⟨ ( \begin{matrix}1 & 0 \\ 0 & 0\end{matrix} ) , ( \begin{matrix}0 & 1 \\ 0 & 0\end{matrix} ) , ( \begin{matrix}0 & 0 \\ 1 & 0\end{matrix} ) , ( \begin{matrix}0 & 0 \\ 0 & 1\end{matrix} ) ⟩

is a basis for the entire space, because the space is the direct sum,
so we can use it to represent A.

( \begin{matrix}3 & 1 \\ 0 & 4\end{matrix} ) = 3 ⋅ ( \begin{matrix}1 & 0 \\ 0 & 0\end{matrix} ) + 1 ⋅ ( \begin{matrix}0 & 1 \\ 0 & 0\end{matrix} ) + 0 ⋅ ( \begin{matrix}0 & 0 \\ 1 & 0\end{matrix} ) + 4 ⋅ ( \begin{matrix}0 & 0 \\ 0 & 1\end{matrix} )

Now the projection of A onto M along N is found by keeping the
M part of this sum and dropping the N part.

{\text{proj}}_{M , N} ( ( \begin{matrix}3 & 1 \\ 0 & 4\end{matrix} ) ) = 3 ⋅ ( \begin{matrix}1 & 0 \\ 0 & 0\end{matrix} ) + 1 ⋅ ( \begin{matrix}0 & 1 \\ 0 & 0\end{matrix} ) = ( \begin{matrix}3 & 1 \\ 0 & 0\end{matrix} )

Example 3.3

Both subscripts on {\text{proj}}_{M , N} ( \overset{→}{v} \ ) are significant.
The first subscript M matters because the result of
the projection is an \overset{→}{m} ∈ M, and changing this subspace would
change the possible results.
For an example showing that the second subscript matters,
fix this plane subspace of {ℝ}^{3} and its basis

M = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ y − 2 z = 0 \} \ {B}_{M} = ⟨ ( \begin{matrix}1 \\ 0 \\ 0\end{matrix} ) , ( \begin{matrix}0 \\ 2 \\ 1\end{matrix} ) ⟩

and compare the projections along two different subspaces.

N = \{ k ( \begin{matrix}0 \\ 0 \\ 1\end{matrix} ) \ | \ k ∈ ℝ \} \ \overset{̂}{N} = \{ k ( \begin{matrix}0 \\ 1 \\ − 2\end{matrix} ) \ | \ k ∈ ℝ \}

(Verification that {ℝ}^{3} = M ⊕ N and
{ℝ}^{3} = M ⊕ \overset{̂}{N} is routine.)
We will check that these projections are different by checking that
they have different effects on this vector.

\overset{→}{v} = ( \begin{matrix}2 \\ 2 \\ 5\end{matrix} )

For the first one we find a basis for N

{B}_{N} = ⟨ ( \begin{matrix}0 \\ 0 \\ 1\end{matrix} ) ⟩

and represent \overset{→}{v} with respect to the concatenation {B}_{M} \ {}^{⌢} \ {B}_{N}.

( \begin{matrix}2 \\ 2 \\ 5\end{matrix} ) = 2 ⋅ ( \begin{matrix}1 \\ 0 \\ 0\end{matrix} ) + 1 ⋅ ( \begin{matrix}0 \\ 2 \\ 1\end{matrix} ) + 4 ⋅ ( \begin{matrix}0 \\ 0 \\ 1\end{matrix} )

The projection of \overset{→}{v} onto M along N is found by
keeping the M part and dropping the N part.

{\text{proj}}_{M , N} ( \overset{→}{v} \ ) = 2 ⋅ ( \begin{matrix}1 \\ 0 \\ 0\end{matrix} ) + 1 ⋅ ( \begin{matrix}0 \\ 2 \\ 1\end{matrix} ) = ( \begin{matrix}2 \\ 2 \\ 1\end{matrix} )

For the other subspace \overset{̂}{N}, this basis is natural.

{B}_{\overset{̂}{N}} = ⟨ ( \begin{matrix}0 \\ 1 \\ − 2\end{matrix} ) ⟩

Representing \overset{→}{v} with respect to the concatenation

( \begin{matrix}2 \\ 2 \\ 5\end{matrix} ) = 2 ⋅ ( \begin{matrix}1 \\ 0 \\ 0\end{matrix} ) + ( 9 / 5 ) ⋅ ( \begin{matrix}0 \\ 2 \\ 1\end{matrix} ) − ( 8 / 5 ) ⋅ ( \begin{matrix}0 \\ 1 \\ − 2\end{matrix} )

and then keeping only the M part gives this.

{\text{proj}}_{M , \overset{̂}{N}} ( \overset{→}{v} \ ) = 2 ⋅ ( \begin{matrix}1 \\ 0 \\ 0\end{matrix} ) + ( 9 / 5 ) ⋅ ( \begin{matrix}0 \\ 2 \\ 1\end{matrix} ) = ( \begin{matrix}2 \\ 18 / 5 \\ 9 / 5\end{matrix} )

Therefore projection along different subspaces may yield different results.

These pictures compare the two maps.
Both show that the projection is indeed "onto" the plane and
"along" the line.

Notice that the projection along N is not orthogonal— there are members of the plane M that are not orthogonal to the dotted line. But the projection along \overset{̂}{N} is orthogonal.

A natural question is: what is the relationship between the projection
operation defined above, and the operation of orthogonal projection
onto a line?
The second picture above suggests the answer— orthogonal projection onto
a line is a special case of the projection defined above; it is just
projection along a subspace perpendicular to the line.

In addition to pointing out that projection along a subspace is a
generalization, this scheme shows how to define orthogonal projection onto
any subspace of {ℝ}^{n}, of any dimension.

Definition 3.4

The orthogonal complementof a subspace M of {ℝ}^{n} is

{M}^{⊥} = \{ \overset{→}{v} ∈ {ℝ}^{n} \ | \ \overset{→}{v} \text{ is perpendicular to all vectors in } M \}

(read "M perp"). The orthogonal projection {\text{proj}}_{M} ( \overset{→}{v} \ ) of a vector is its projection onto M along {M}^{⊥}.

Example 3.5

In {ℝ}^{3}, to find the orthogonal complement of the plane

P = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ 3 x + 2 y − z = 0 \}

we start with a basis for P.

B = ⟨ ( \begin{matrix}1 \\ 0 \\ 3\end{matrix} ) , ( \begin{matrix}0 \\ 1 \\ 2\end{matrix} ) ⟩

Any \overset{→}{v} perpendicular to every vector in B
is perpendicular to every vector in the span of B
(the proof of this assertion is Problem 10).
Therefore, the subspace {P}^{⊥} consists of the vectors that satisfy
these two conditions.

( \begin{matrix}1 \\ 0 \\ 3\end{matrix} ) ⋅ ( \begin{matrix}{v}_{1} \\ {v}_{2} \\ {v}_{3}\end{matrix} ) = 0 \ ( \begin{matrix}0 \\ 1 \\ 2\end{matrix} ) ⋅ ( \begin{matrix}{v}_{1} \\ {v}_{2} \\ {v}_{3}\end{matrix} ) = 0

We can express those conditions more compactly as a linear system.

{P}^{⊥} = \{ ( \begin{matrix}{v}_{1} \\ {v}_{2} \\ {v}_{3}\end{matrix} ) \ | \ ( \begin{matrix}1 & 0 & 3 \\ 0 & 1 & 2\end{matrix} ) ( \begin{matrix}{v}_{1} \\ {v}_{2} \\ {v}_{3}\end{matrix} ) = ( \begin{matrix}0 \\ 0\end{matrix} ) \}

We are thus left with finding the nullspace of the map represented
by the matrix, that is, with
calculating the solution set of a homogeneous linear system.

{P}^{⊥} = \{ ( \begin{matrix}{v}_{1} \\ {v}_{2} \\ {v}_{3}\end{matrix} ) \ | \ \ \begin{matrix}{v}_{1} & & & + & 3 {v}_{3} & = & 0 \\ & & {v}_{2} & + & 2 {v}_{3} & = & 0\end{matrix} \ \} = \{ k ( \begin{matrix}− 3 \\ − 2 \\ 1\end{matrix} ) \ | \ k ∈ ℝ \}

Example 3.6

Where M is the x y-plane subspace of {ℝ}^{3},
what is {M}^{⊥}?
A common first reaction is that {M}^{⊥} is the y z-plane, but
that's not right.
Some vectors from the y z-plane are not perpendicular to every
vector in the x y-plane.

Instead {M}^{⊥} is the z-axis, since proceeding as in the
prior example and taking the natural basis for the x y-plane gives this.

{M}^{⊥} = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ ( \begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0\end{matrix} ) ( \begin{matrix}x \\ y \\ z\end{matrix} ) = ( \begin{matrix}0 \\ 0\end{matrix} ) \ \} = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ x = 0 \text{ and } y = 0 \}

The two examples that we've seen since Definition 3.4
illustrate the first sentence in that definition.
The next result justifies the second sentence.

Lemma 3.7

Let M be a subspace of {ℝ}^{n}. The orthogonal complement of M is also a subspace. The space is the direct sum of the two {ℝ}^{n} = M ⊕ {M}^{⊥}. And, for any \overset{→}{v} ∈ {ℝ}^{n}, the vector \overset{→}{v} − {\text{proj}}_{M} ( \overset{→}{v} \ ) is perpendicular to every vector in M.

Proof

First, the orthogonal complement {M}^{⊥} is a subspace of {ℝ}^{n} because,
as noted in the prior two examples, it is a nullspace.

Next, we can
start with any basis {B}_{M} = ⟨ {\overset{→}{μ}}_{1} , … , {\overset{→}{μ}}_{k} ⟩
for M and expand it to a basis

for the entire space.
Apply the Gram-Schmidt process to get an orthogonal basis
K = ⟨ {\overset{→}{κ}}_{1} , … , {\overset{→}{κ}}_{n} ⟩ for {ℝ}^{n}.
This K is the concatenation of two bases
⟨ {\overset{→}{κ}}_{1} , … , {\overset{→}{κ}}_{k} ⟩
(with the same number of members as {B}_{M}) and
⟨ {\overset{→}{κ}}_{k + 1} , … , {\overset{→}{κ}}_{n} ⟩.
The first is a basis for M, so if we show that the second is a
basis for {M}^{⊥} then we will have that
the entire space is the direct sum of the two subspaces.

Problem 9 from the prior subsection
proves this about any orthogonal basis: each vector \overset{→}{v}
in the space is the sum of its orthogonal projections
onto the lines spanned by the basis vectors.

\overset{→}{v} = {\text{proj}}_{[ {\overset{→}{κ}}_{1} ]} ( \overset{→}{v} \ ) + … + {\text{proj}}_{[ {\overset{→}{κ}}_{n} ]} ( \overset{→}{v} \ ) \ \ ( ∗ )

To check this, represent the vector
\overset{→}{v} = {r}_{1} {\overset{→}{κ}}_{1} + … + {r}_{n} {\overset{→}{κ}}_{n},
apply {\overset{→}{κ}}_{i} to both sides
\overset{→}{v} ⋅ {\overset{→}{κ}}_{i} = ( {r}_{1} {\overset{→}{κ}}_{1} + … + {r}_{n} {\overset{→}{κ}}_{n} ) ⋅ {\overset{→}{κ}}_{i} = {r}_{1} ⋅ 0 + … + {r}_{i} ⋅ ( {\overset{→}{κ}}_{i} ⋅ {\overset{→}{κ}}_{i} ) + … + {r}_{n} ⋅ 0,
and solve to get
{r}_{i} = ( \overset{→}{v} ⋅ {\overset{→}{κ}}_{i} ) / ( {\overset{→}{κ}}_{i} ⋅ {\overset{→}{κ}}_{i} ),
as desired.

Since obviously any member of the span of
⟨ {\overset{→}{κ}}_{k + 1} , … , {\overset{→}{κ}}_{n} ⟩
is orthogonal to any vector in M, to show that this is a basis for {M}^{⊥}
we need only show the other
containment— that any \overset{→}{w} ∈ {M}^{⊥} is in
the span of this basis.
The prior paragraph does this.
On projections onto basis vectors from M, any \overset{→}{w} ∈ {M}^{⊥} gives
{\text{proj}}_{[ {\overset{→}{κ}}_{1} ]} ( \overset{→}{w} \ ) = \overset{→}{0} , … , \ {\text{proj}}_{[ {\overset{→}{κ}}_{k} ]} ( \overset{→}{w} \ ) = \overset{→}{0}
and therefore (∗) gives that \overset{→}{w} is a linear combination of
{\overset{→}{κ}}_{k + 1} , … , {\overset{→}{κ}}_{n}.
Thus this is a basis for {M}^{⊥} and
{ℝ}^{n} is the direct sum of the two.

The final sentence is proved in much the same way. Write \overset{→}{v} = {\text{proj}}_{[ {\overset{→}{κ}}_{1} ]} ( \overset{→}{v} \ ) + … + {\text{proj}}_{[ {\overset{→}{κ}}_{n} ]} ( \overset{→}{v} \ ). Then {\text{proj}}_{M} ( \overset{→}{v} \ ) is gotten by keeping only the M part and dropping the {M}^{⊥} part {\text{proj}}_{M} ( \overset{→}{v} \ ) = {\text{proj}}_{[ {\overset{→}{κ}}_{k + 1} ]} ( \overset{→}{v} \ ) + … + {\text{proj}}_{[ {\overset{→}{κ}}_{k} ]} ( \overset{→}{v} \ ). Therefore \overset{→}{v} − {\text{proj}}_{M} ( \overset{→}{v} \ ) consists of a linear combination of elements of {M}^{⊥} and so is perpendicular to every vector in M.

We can find the orthogonal projection onto a subspace by
following the steps of the proof, but the next result gives a
convienent formula.

Theorem 3.8

Let \overset{→}{v} be a vector in {ℝ}^{n} and let M be a subspace of {ℝ}^{n} with basis ⟨ {\overset{→}{β}}_{1} , … , {\overset{→}{β}}_{k} ⟩. If A is the matrix whose columns are the \overset{→}{β}'s then {\text{proj}}_{M} ( \overset{→}{v} \ ) = {c}_{1} {\overset{→}{β}}_{1} + … + {c}_{k} {\overset{→}{β}}_{k} where the coefficients {c}_{i} are the entries of the vector ( {A}^{t r a n s} A {)}^{− 1} {A}^{t r a n s} ⋅ \overset{→}{v}. That is, {\text{proj}}_{M} ( \overset{→}{v} \ ) = A ( {A}^{t r a n s} A {)}^{− 1} {A}^{t r a n s} ⋅ \overset{→}{v}.

Proof

The vector {\text{proj}}_{M} ( \overset{→}{v} ) is a member of M and so it is a
linear combination of basis vectors
{c}_{1} ⋅ {\overset{→}{β}}_{1} + … + {c}_{k} ⋅ {\overset{→}{β}}_{k}.
Since A's columns are the \overset{→}{β}'s, that can be expressed
as: there is a \overset{→}{c} ∈ {ℝ}^{k} such that
{\text{proj}}_{M} ( \overset{→}{v} \ ) = A \overset{→}{c}
(this is expressed compactly with matrix
multiplication as in
Example 3.5 and 3.6).
Because \overset{→}{v} − {\text{proj}}_{M} ( \overset{→}{v} \ ) is perpendicular to each member
of the basis, we have this (again, expressed compactly).

\overset{→}{0} = {A}^{t r a n s} ( \overset{→}{v} − A \overset{→}{c} ) = {A}^{t r a n s} \overset{→}{v} − {A}^{t r a n s} A \overset{→}{c}

Solving for \overset{→}{c} (showing that {A}^{t r a n s} A is invertible is an
exercise)

\overset{→}{c} = ( {A}^{t r a n s} A {)}^{− 1} {A}^{t r a n s} ⋅ \overset{→}{v}

gives the formula for the projection matrix as {\text{proj}}_{M} ( \overset{→}{v} \ ) = A ⋅ \overset{→}{c}.

Example 3.9

To orthogonally project this vector onto this subspace

\overset{→}{v} = ( \begin{matrix}1 \\ − 1 \\ 1\end{matrix} ) \ P = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ x + z = 0 \}

first make a matrix whose columns are a basis for the subspace

A = ( \begin{matrix}0 & 1 \\ 1 & 0 \\ 0 & − 1\end{matrix} )

and then compute.

\begin{matrix}A ( {A}^{t r a n s} A {)}^{− 1} {A}^{t r a n s} & = ( \begin{matrix}0 & 1 \\ 1 & 0 \\ 0 & − 1\end{matrix} ) ( \begin{matrix}0 & 1 \\ 1 / 2 & 0\end{matrix} ) ( \begin{matrix}1 & 0 & − 1 \\ 0 & 1 & 0\end{matrix} ) \\ & = ( \begin{matrix}1 / 2 & 0 & − 1 / 2 \\ 0 & 1 & 0 \\ − 1 / 2 & 0 & 1 / 2\end{matrix} )\end{matrix}

With the matrix, calculating the orthogonal projection of any vector onto P
is easy.

{\text{proj}}_{P} ( \overset{→}{v} ) = ( \begin{matrix}1 / 2 & 0 & − 1 / 2 \\ 0 & 1 & 0 \\ − 1 / 2 & 0 & 1 / 2\end{matrix} ) ( \begin{matrix}1 \\ − 1 \\ 1\end{matrix} ) = ( \begin{matrix}0 \\ − 1 \\ 0\end{matrix} )

Exercises

This exercise is recommended for all readers.

Problem 1

Project the vectors onto M along N.

• ( \begin{matrix}3 \\ − 2\end{matrix} ) , \ M = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ x + y = 0 \} , \ N = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ − x − 2 y = 0 \}
• ( \begin{matrix}1 \\ 2\end{matrix} ) , \ M = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ x − y = 0 \} , \ N = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ 2 x + y = 0 \}
• ( \begin{matrix}3 \\ 0 \\ 1\end{matrix} ) , \ M = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ x + y = 0 \} , \ N = \{ c ⋅ ( \begin{matrix}1 \\ 0 \\ 1\end{matrix} ) \ | \ c ∈ ℝ \}

This exercise is recommended for all readers.

Problem 2

Find {M}^{⊥}.

• M = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ x + y = 0 \}
• M = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ − 2 x + 3 y = 0 \}
• M = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ x − y = 0 \}
• M = \{ \overset{→}{0} \ \}
• M = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ x = 0 \}
• M = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ − x + 3 y + z = 0 \}
• M = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ x = 0 \text{ and } y + z = 0 \}

Problem 3

This subsection shows how to project orthogonally in two ways,
the method of Example 3.2
and 3.3,
and the method of Theorem 3.8.
To compare them,
consider the plane P specified by {}3 x + 2 y − z = 0 in {ℝ}^{3}.

• Find a basis for P.
• Find {P}^{⊥} and a basis for {P}^{⊥}.
• Represent this vector with respect to the concatenation

of the two bases from the prior item.

\overset{→}{v} = ( \begin{matrix}1 \\ 1 \\ 2\end{matrix} )

• Find the orthogonal projection of \overset{→}{v} onto P
  by keeping only the P part from the prior item.
• Check that against the result from applying

Theorem 3.8.

This exercise is recommended for all readers.

Problem 4

We have three ways to find the orthogonal projection of a
vector onto a line, the Definition 1.1 way from
the first subsection of this section, the
Example 3.2 and 3.3
way of representing
the vector with respect to a basis for the space and then keeping the
M part, and the way of Theorem 3.8.
For these cases, do all three ways.

• \overset{→}{v} = ( \begin{matrix}1 \\ − 3\end{matrix} ) , \ M = \{ ( \begin{matrix}x \\ y\end{matrix} ) \ | \ x + y = 0 \}
• \overset{→}{v} = ( \begin{matrix}0 \\ 1 \\ 2\end{matrix} ) , \ M = \{ ( \begin{matrix}x \\ y \\ z\end{matrix} ) \ | \ x + z = 0 \text{ and } y = 0 \}

Problem 5

Check that the operation of Definition 3.1 is well-defined. That is, in Example 3.2 and 3.3, doesn't the answer depend on the choice of bases?

Problem 6

What is the orthogonal projection onto the trivial subspace?

Problem 7

What is the projection of \overset{→}{v} onto M along N
if \overset{→}{v} ∈ M?

Problem 8

Show that if M ⊆ {ℝ}^{n} is a subspace with orthonormal basis
⟨ {\overset{→}{κ}}_{1} , … , {\overset{→}{κ}}_{n} ⟩ then
the orthogonal projection of \overset{→}{v} onto M is this.

( \overset{→}{v} ⋅ {\overset{→}{κ}}_{1} ) ⋅ {\overset{→}{κ}}_{1} + … + ( \overset{→}{v} ⋅ {\overset{→}{κ}}_{n} ) ⋅ {\overset{→}{κ}}_{n}

This exercise is recommended for all readers.

Problem 9

Prove that the map p : V → V is the projection onto M
along N if and only if the map
\text{id} − p is the projection onto N along M.
(Recall the definition of the difference of two
maps: ( \text{id} − p ) \ ( \overset{→}{v} ) = \text{id} ( \overset{→}{v} ) − p ( \overset{→}{v} ) = \overset{→}{v} − p ( \overset{→}{v} ).)

This exercise is recommended for all readers.

Problem 10

Show that if a vector is perpendicular to every vector in a set then
it is perpendicular to every vector in the span of that set.

Problem 11

True or false: the intersection of a subspace and its orthogonal
complement is trivial.

Problem 12

Show that the dimensions of orthogonal complements add to the dimension of the entire space.

This exercise is recommended for all readers.

Problem 13

Suppose that {\overset{→}{v}}_{1} , {\overset{→}{v}}_{2} ∈ {ℝ}^{n} are such that for all
complements M , N ⊆ {ℝ}^{n}, the projections of {\overset{→}{v}}_{1}
and {\overset{→}{v}}_{2} onto M along N are equal.
Must {\overset{→}{v}}_{1} equal {\overset{→}{v}}_{2}?
(If so, what if we relax the condition to: all
orthogonal projections of the two are equal?)

This exercise is recommended for all readers.

Problem 14

Let M , N be subspaces of {ℝ}^{n}.
The perp operator acts on subspaces; we can ask how it interacts
with other such operations.

• Show that two perps cancel: ( {M}^{⊥} {)}^{⊥} = M.

Prove that M ⊆ N implies
that {N}^{⊥} ⊆ {M}^{⊥}.

• Show that ( M + N {)}^{⊥} = {M}^{⊥} ∩ {N}^{⊥}.

This exercise is recommended for all readers.

Problem 15

The material in this subsection allows us to express a geometric
relationship that we have not yet seen between the rangespace and the
nullspace of a linear map.

• Represent f : {ℝ}^{3} → ℝ given by

( \begin{matrix}{v}_{1} \\ {v}_{2} \\ {v}_{3}\end{matrix} ) ↦ 1 {v}_{1} + 2 {v}_{2} + 3 {v}_{3}

with respect to the standard bases and show that

( \begin{matrix}1 \\ 2 \\ 3\end{matrix} )

is a member of the perp of the nullspace.
Prove that 𝒩 ( f {)}^{⊥} is equal to the span of this
vector.

• Generalize that to apply to any f : {ℝ}^{n} → ℝ.
• Represent f : {ℝ}^{3} → {ℝ}^{2}

( \begin{matrix}{v}_{1} \\ {v}_{2} \\ {v}_{3}\end{matrix} ) ↦ ( \begin{matrix}1 {v}_{1} + 2 {v}_{2} + 3 {v}_{3} \\ 4 {v}_{1} + 5 {v}_{2} + 6 {v}_{3}\end{matrix} )

with respect to the standard bases and show that

( \begin{matrix}1 \\ 2 \\ 3\end{matrix} ) , \ ( \begin{matrix}4 \\ 5 \\ 6\end{matrix} )

are both members of the perp of the nullspace.
Prove that 𝒩 ( f {)}^{⊥} is the span of these two.
(Hint.
See the third item of Problem 14.)

• Generalize that to apply to any f : {ℝ}^{n} → {ℝ}^{m}.

This, and related results, is called the Fundamental Theorem of Linear Algebra
in (Strang 1993).

Problem 16

Define a projection to be a linear transformation
t : V → V with the property that
repeating the projection does nothing more than does the projection
alone: ( t ∘ t ) \ ( \overset{→}{v} ) = t ( \overset{→}{v} )
for all \overset{→}{v} ∈ V.

• Show that orthogonal projection onto a line has that property.
• Show that projection along a subspace has that property.
• Show that for any such t

there is a basis B = ⟨ {\overset{→}{β}}_{1} , … , {\overset{→}{β}}_{n} ⟩ for V such that

t ( {\overset{→}{β}}_{i} ) = \{ \begin{matrix}{\overset{→}{β}}_{i} & i = 1 , 2 , … , \ r \\ \overset{→}{0} & i = r + 1 , r + 2 , … , \ n\end{matrix}

where r is the rank of t.

• Conclude that every projection is a projection along a subspace.
• Also conclude that every projection has a representation

{R e p}_{B , B} ( t ) = ( \begin{matrix}I & Z \\ Z & Z\end{matrix} )

in block partial-identity form.

Problem 17

A square matrix is symmetric if each i , j entry equals the
j , i entry (i.e., if the matrix equals its transpose).
Show that the projection matrix A ( {A}^{t r a n s} A {)}^{− 1} {A}^{t r a n s}
is symmetric. (Strang 1980)
Hint. Find properties of transposes by looking in the index
under "transpose".

Solutions

References

• Strang, Gilbert (1993), "The Fundamental Theorem of Linear Algebra", American Mathematical Monthly, American Mathematical Society: 848–855 {{citation}}: Unknown parameter |month= ignored (help).
• Strang, Gilbert (1980), Linear Algebra and its Applications (2nd ed.), Hartcourt Brace Javanovich

Where this page came from

This page was imported from Wikibooks. From “Linear Algebra” on Wikibooks, by its contributors, under CC BY-SA 4.0. Changed here: set as a page; navigation and edit links left out; each image under its own licence, credited in its caption.

Nobody has written it yet — it is the source material at a new address, which is why search engines are asked to skip it and why no one earns from it. It is up for grabs: take it on, and it is yours to rewrite and to earn from.