Dropping a perpendicular from a point to a line finds the point of the line that is closest. The same construction works for any subspace: split a vector into the part inside the subspace and the part at right angles to it, and the first part is the best approximation the subspace can offer.

At noon the shadow of a ball falls straight down onto the floor. The shadow is where the ball would be if it were forced to lie on the floor with as little movement as possible.

The projection of $\mathbf{b}=(1,3)$ onto the line through $\mathbf{u}=(2,1)$ is $\mathbf{p}=\frac{\mathbf{b}\cdot\mathbf{u}}{\mathbf{u}\cdot\mathbf{u}}\mathbf{u}=\frac55(2,1)=(2,1)$. The error $\mathbf{b}-\mathbf{p}=(-1,2)$, dashed, is perpendicular to the line, and $\mathbf{p}$ is the point of the line closest to $\mathbf{b}$.

Let $W$ have an orthogonal basis $\mathbf{u}_1,\ldots,\mathbf{u}_k$. The orthogonal projection of $\mathbf{x}$ onto is the vector below. Then is orthogonal to , since its dot product with each is . So splits into a part in and a part in , and the split is unique.

• Onto a line through $\mathbf{u}$: $\dfrac{\mathbf{x}\cdot\mathbf{u}}{\mathbf{u}\cdot\mathbf{u}}\mathbf{u}$.
• The formula needs an orthogonal basis of $W$; for any other basis the sum is not the projection.

Let $\mathbf{p}=\operatorname{proj}_W\mathbf{x}$ and take any $\mathbf{w}$ in $W$. Going from $\mathbf{w}$ to through splits into , orthogonal to , and , inside . Pythagoras adds their squared lengths, so is at least as long as , and strictly longer unless .

Go from $\mathbf{w}$ to $\mathbf{x}$ through $\mathbf{p}$.

The first part is orthogonal to $W$ by construction, and the second lies in $W$.

Pythagoras, because the two parts are orthogonal.

The extra term is never negative, and it is zero only when $\mathbf{w}=\mathbf{p}$.

The projection of $\mathbf{x}$ onto the line through $\mathbf{u}$ is a multiple of $\mathbf{u}$, chosen so that what is left over is orthogonal to $\mathbf{u}$.

With an orthogonal basis of $W$, project onto each basis vector separately and add. The parts do not interfere, because the basis vectors are orthogonal.

• $\operatorname{proj}_W\mathbf{x}=\sum\frac{\mathbf{x}\cdot\mathbf{u}_i}{\mathbf{u}_i\cdot\mathbf{u}_i}\mathbf{u}_i$ for an orthogonal basis of .

Great work! You now know how to:

• project onto a line and onto a subspace with an orthogonal basis;
• split a vector into parts in W and orthogonal to W;
• find the closest point of a subspace and the distance to it;
• use the projection matrix $QQ^{T}$.