In linear algebra and functional analysis, a projection is a linear transformation P from a vector space to itself such that P² = P. Projections map the whole vector space to a subspace and leave the points in that subspace unchanged.^[1] Though abstract, this definition of "projection" formalizes and generalizes the idea of graphical projection. Image File history File links No higher resolution available. ... Linear algebra is the branch of mathematics concerned with the study of vectors, vector spaces (also called linear spaces), linear maps (also called linear transformations), and systems of linear equations. ... Functional analysis is the branch of mathematics, and specifically of analysis, concerned with the study of spaces of functions. ... In mathematics, a linear transformation (also called linear map or linear operator) is a function between two vector spaces that preserves the operations of vector addition and scalar multiplication. ... In mathematics, a vector space (or linear space) is a collection of objects (called vectors) that, informally speaking, may be scaled and added. ... Graphical projection in the visual sciences is an imaging procedure the protocols of which preclude the necessity of mathematical calculation. ...

Simple example

For example, the function which maps the point (x, y, z) in three-dimensional space to the point (x, y, 0) is a projection onto the x-y plane. This function is represented by the matrix In mathematics, a matrix (plural matrices) is a rectangular table of numbers or, more generally, a table consisting of abstract quantities that can be added and multiplied. ...

Indeed, the action of this matrix on an arbitrary vector is

and

therefore P = P², proving that P is indeed a projection.

Properties

Assume the underlying vector space is finite dimensional (therefore issues such as continuity of a projection need not be considered).

As stated in the introduction, a projection P is a linear transformation that is idempotent, meaning that P² = P. This definition has immediate implications. First, there is a subspace U of the domain for which the projection acts as the identity; every vector x in this subspace has Px = x. This subspace is exactly the range of the projection. Image File history File links No higher resolution available. ... In mathematics, an idempotent element is an element which, intuitively, leaves something unchanged. ... The concept of a linear subspace (or vector subspace) is important in linear algebra and related fields of mathematics. ... In mathematics, the range of a function is the set of all output values produced by that function. ...

There is a complementary subspace V of the domain that is always zeroed out by the projection; every vector x in this subspace has Px = 0. This subspace is the null space of the projection. In abstract algebra, the direct sum is a construction which combines several modules into a new, larger module. ... It has been suggested that this article or section be merged into kernel (mathematics). ...

The projection is said to be along V onto U. The subspaces U and V determine the projection uniquely.

The subspaces U and V are complementary, i.e. the underlying vector space is the direct sum U ⊕ V. This means that any vector x in the domain can uniquely be written as x = u + v with u in U and v in V. The vector u in this decomposition is given by u = Px, where P is the projection along V onto U. The vector v is given by v = (I − P) x. The operator I − P is the projection along U onto V; it is called the complementary projection.^[2] Decomposition of a vector space into direct sums is not unique in general. Therefore, given a subspace V, in general there are many projections whose range (or kernel) is V. In abstract algebra, the direct sum is a construction which combines several vector spaces (or groups, or abelian groups, or modules) into a new, bigger one. ...

Only 0 and 1 can be an eigenvalue of a projection. The eigenspace corresponding to the eigenvalue 0 is the null space V, and the eigenspace corresponding to 1 is the range U. In mathematics, a number is called an eigenvalue of a matrix if there exists a nonzero vector such that the matrix times the vector is equal to the same vector multiplied by the eigenvalue. ...

Orthogonal projections

If the underlying vector space is endowed with an inner product, orthogonality and its attendant notions (such as the self-adjointness of a linear operator) become available. An orthogonal projection is a projection for which the range U and the null space V are orthogonal subspaces. A projection is orthogonal if and only if it is self-adjoint, which means that, in the context of real vector spaces, the associated matrix is symmetric: P = P^T (for the complex case, the matrix is hermitian: P = P^*). Indeed, if x is a vector in the domain of the projection, then Px ∈ U and x − Px ∈ V, and In mathematics, orthogonal is synonymous with perpendicular when used as a simple adjective that is not part of any longer phrase with a standard definition. ... In mathematics, orthogonal is synonymous with perpendicular when used as a simple adjective that is not part of any longer phrase with a standard definition. ... On a finite-dimensional inner product space, a self-adjoint operator is one that is its own adjoint, or, equivalently, one whose matrix is Hermitian, where a Hermitian matrix is one which is equal to its own conjugate transpose. ... In linear algebra, a symmetric matrix is a matrix that is its own transpose. ... A Hermitian matrix (or self-adjoint matrix) is a square matrix with complex entries which is equal to its own conjugate transpose — that is, the element in the ith row and jth column is equal to the complex conjugate of the element in the jth row and ith column, for...

so Px and x − Px are orthogonal for all x if and only if P − P² = 0.^[3]

The simplest case is where the projection is an orthogonal projection onto a line. If u is a unit vector on the line, then the projection is given by

This operator leaves u invariant, and it annihilates all vectors orthogonal to u, proving that it is indeed the orthogonal projection onto the line containing u.^[4]

This formula can be generalized to orthogonal projections on a subspace of arbitrary dimension. Let u₁, …, u_k be an orthonormal basis of the subspace U, and let A denote the n-by-k matrix whose columns are u₁, …, u_k. Then the projection is given by In mathematics, an orthonormal basis of an inner product space V(i. ...

^[5]

The matrix A^T is the partial isometry that vanishes on the orthogonal complement of U and A is the isometry that embedds U into the underlying vector space. The range of P_A is therefore the final space of A. It is also clear that A^TA is the identity operator on U. In functional analysis a partial isometry is a linear map W between Hilbert spaces H, K such that there is a closed vector subspace H1 of H such that W restricted to H1 is an isometric map and W restricted to the orthogonal complement of H1 is zero. ...

The orthonormality condition can also be dropped. If u₁, …, u_k is a (not necessarily orthonormal) basis, and A is the matrix with these vectors as columns, then the projection is

^[6]

The matrix A^T still embeds U into the underlying vector space but is no longer an isometry in general. The matrix (A^TA)⁻¹ is a "normalizing factor" that recovers the norm. For example, the rank-1 operator uu^T is not a projection if ||u|| ≠ 1. After dividing by u^Tu = ||u||², we obtain the projection u(u^Tu)⁻¹u^T onto the subspace spanned by u.

All these formulas also hold for complex inner product spaces, provided that the conjugate transpose is used instead of the transpose. In mathematics, the conjugate transpose, Hermitian transpose, or adjoint matrix of an m-by-n matrix A with complex entries is the n-by-m matrix A* obtained from A by taking the transpose and then taking the complex conjugate of each entry. ...

Oblique projections

The term oblique projections is sometimes used to refer to non-orthogonal projections. These projections are also used to represent spatial figures in two-dimensional drawings (see oblique projection), though not as frequently as orthogonal projections. This article needs to be cleaned up to conform to a higher standard of quality. ...

Oblique projections are defined by their range and null space. A formula for the matrix representing the projection with a given range and null space can be found as follows. Let the vectors u₁, …, u_k form a basis for the range of the projection, and assemble these vectors in the n-by-k matrix A. The range and the null space are complementary spaces, so the null space has dimension n − k. It follows that the orthogonal complement of the null space has dimension k. Let v₁, …, v_k form a basis for the orthogonal complement of the null space of the projection, and assemble these vectors in the matrix B. Then the projection is defined by In the mathematical fields of linear algebra and functional analysis, the orthogonal complement of a subspace W of an inner product space V is the set of all vectors in V that are orthogonal to every vector in W, i. ...

This expression generalizes the formula for orthogonal projections given above.^[7]

Projections on normed vector spaces

When the underlying vector space X is a (not necessarily finite-dimensional) normed vector space, analytic questions, irrelevant in the finite-dimensional case, need to be considered. Assume now X is a Banach space. In mathematics, with 2- or 3-dimensional vectors with real-valued entries, the idea of the length of a vector is intuitive and can easily be extended to any real vector space Rn. ... In mathematics, Banach spaces, named after Stefan Banach who studied them, are one of the central objects of study in functional analysis. ...

Much of the algebraic notions discussed above survive the passage to this context. A given direct sum decomposition of X into complementary subspaces still specifies a projection, and vice versa. If X is the direct sum X = U ⊕ V, then the operator defined by P(u + v) = u is still a projection with range U and kernel V. It is also clear that P² = P. Conversely, if P is projection on X, i.e. P² = P, then it is easily verified that (I − P)² = (I − P). In other words, (I − P) is also a projection. The relation I = P + (I − P) implies X is the direct sum Ran(P) ⊕ Ran(I − P).

However, in contrast to the finite-dimensional case, projections need not be continuous in general. If a subspace U of X is not closed in the norm topology, then projection onto U is not continuous. In other words, the range of a continuous projection P must be a closed subspace. Furthermore, the kernel of a continuous projection (in fact, a continuous linear operator in general) is closed. Thus a continuous projection P gives a decomposition of X into two complementary closed subspaces: X = Ran(P) ⊕ Ker(P) = Ran(P) ⊕ Ran(I − P). In mathematics, the operator norm is a norm defined on the space of bounded operators between two Banach spaces. ...

The converse holds also, with an additional assumption. Suppose U is a closed subspace of X. If there exists a closed subspace V such that X = U ⊕ V, then the projection P with range U and kernel V is continuous. This follows from the closed graph theorem. Suppose x_n → x and Px_n → y. One needs to show Px = y. Since U is closed and {Px_n} ⊂ U, y lies in U, i.e. Py = y. Also, x_n − Px_n = (I − P)x_n → x − y. Because V is closed and {(I − P)x_n} ⊂ V, we have x − y ∈ V, i.e. P(x − y) = Px − Py = Px − y = 0, which proves the claim. In mathematics, the closed graph theorem is a basic result in functional analysis which characterizes continuous linear operators between Banach spaces in terms of the operator graph. ...

The above argument makes use of the assumption that both U and V are closed. In general, given a closed subspace U, there need not exist a complementary closed subspace V, although for Hilbert spaces this can always be done by taking the orthogonal complement. For Banach spaces, a one-dimensional subspace always has a closed complementary subspace. This is an immediate consequence of Hahn–Banach theorem. Let U be the linear span of u. By Hahn–Banach, there exists a bounded linear functional φ such that φ(u) = 1. The operator P(x) = φ(x)u satisfies P² = P, i.e. it is a projection. Boundedness of φ implies continuity of P and therefore Ker(P) = Ran(I − P) is a closed complementary subspace of U. The mathematical concept of a Hilbert space (named after the German mathematician David Hilbert) generalizes the notion of Euclidean space in a way that extends methods of vector algebra from the plane and three-dimensional space to spaces of functions. ... In the mathematical fields of linear algebra and functional analysis, the orthogonal complement of a subspace W of an inner product space V is the set of all vectors in V that are orthogonal to every vector in W, i. ... In mathematics, the Hahn–Banach theorem is a central tool in functional analysis. ...

Applications and further considerations

Projections (orthogonal and otherwise) play a major role in algorithms for certain linear algebra problems: In mathematics, computing, linguistics, and related disciplines, an algorithm is a finite set of well-defined instructions for accomplishing some task which, given an initial state, will terminate in a defined end-state. ...

• QR decomposition (see Householder transformation and Gram-Schmidt decomposition);
• Singular value decomposition
• Reduction to Hessenberg form (the first step in many eigenvalue algorithms).

As stated above, projections are a special case of idempotents. Analytically, orthogonal projections are non-commutative generalizations of characteristic functions. Idempotents are used in classifying, for instance, semisimple algebras, while measure theory begins with considering characteristic functions of measurable sets. Therefore, as one can imagine, projections are very often encountered in the context operator algebras. In particular, a von Neumann algebra is generated by its complete lattice of projections. In linear algebra, the QR decomposition of a matrix is a decomposition of the matrix into an orthogonal and a triangular matrix. ... In mathematics, a Householder transformation in 3-dimensional space is the reflection of a vector in a plane. ... In mathematics and numerical analysis, the Gram-Schmidt process of linear algebra is a method of orthogonalizing a set of vectors in an inner product space, most commonly the Euclidean space Rn. ... The introduction to this article provides insufficient context for those unfamiliar with the subject matter. ... In linear algebra, a Hessenberg matrix is one that is almost triangular. ... It has been suggested that this article or section be merged with Symbolic computation of matrix eigenvalues. ... Some mathematicians use the phrase characteristic function synonymously with indicator function. ... In mathematics, the term semisimple is used in a number of related ways, within different subjects. ... In functional analysis, an operator algebra is an algebra of continuous linear operators on a topological vector space (such as a Banach space), which is typically required to be closed in a specified operator topology. ... A von Neumann algebra is a *-algebra of bounded operators on a Hilbert space which is closed in the weak operator topology, or equivalently, in the strong operator topology (under pointwise convergence) and contains the identity operator. ... The name lattice is suggested by the form of the Hasse diagram depicting it. ...

See also

• Orthogonalization
• Invariant subspace

In linear algebra, orthogonalization means the following: we start with vectors v1,...,vk in an inner product space, most commonly the Euclidean space Rn which are linearly independent and we want to find mutually orthogonal vectors u1,...,uk which generate the same subspace as the vectors v1,...,vk. ... In mathematics, an invariant subspace of a linear mapping over some vector space V is a subspace W of V such that T(W) is contained in W. An invariant subspace of T is said to be T invariant. ...

Notes

1. ^ Meyer, pp 386+387
2. ^ Meyer, pp 383–388
3. ^ Meyer, p. 433
4. ^ Meyer, p. 431
5. ^ Meyer, equation (5.13.4)
6. ^ Meyer, equation (5.13.3)
7. ^ Meyer, equation (7.10.39)

References

• N. Dunford and J.T. Schwartz, Linear Operators, Part I: General Theory, Interscience, 1958.
• Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, Society for Industrial and Applied Mathematics, 2000. ISBN 978-0-89871-454-8.