World Library  
Flag as Inappropriate
Email this Article

Matrix decomposition

Article Id: WHEBN0000253873
Reproduction Date:

Title: Matrix decomposition  
Author: World Heritage Encyclopedia
Language: English
Subject: QR decomposition, Singular value decomposition, Generalized singular value decomposition, Numerical linear algebra, Gaussian elimination
Collection: Matrix Decompositions, Matrix Theory
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Matrix decomposition

In the mathematical discipline of linear algebra, a matrix decomposition or matrix factorization is a factorization of a matrix into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems.

Contents

  • Example 1
  • Decompositions related to solving systems of linear equations 2
    • LU decomposition 2.1
    • LU reduction 2.2
    • Block LU decomposition 2.3
    • Rank factorization 2.4
    • Cholesky decomposition 2.5
    • QR decomposition 2.6
    • RRQR factorization 2.7
    • Interpolative decomposition 2.8
  • Decompositions based on eigenvalues and related concepts 3
    • Eigendecomposition 3.1
    • Jordan decomposition 3.2
    • Schur decomposition 3.3
    • QZ decomposition 3.4
    • Takagi's factorization 3.5
    • Singular value decomposition 3.6
  • Other decompositions 4
    • Polar decomposition 4.1
    • Algebraic polar decomposition 4.2
    • Sinkhorn normal form 4.3
    • Sectoral decomposition 4.4
  • Generalizations 5
  • See also 6
  • Notes 7
  • References 8
  • External links 9

Example

In numerical analysis, different decompositions are used to implement efficient matrix algorithms.

For instance, when solving a system of linear equations Ax=b, the matrix A can be decomposed via the LU decomposition. The LU decomposition factorizes a matrix into a lower triangular matrix L and an upper triangular matrix U. The systems L(Ux)=b and Ux=L^{-1}b require fewer additions and multiplications to solve, compared with the original system Ax=b, though one might require significantly more digits in inexact arithmetic such as floating point.

Similarly, the QR decomposition expresses A as QR with Q an orthogonal matrix and R an upper triangular matrix. The system Q(Rx) = b is solved by Rx = QTb = c, and the system Rx = c is solved by 'back substitution'. The number of additions and multiplications required is about twice that of using the LU solver, but no more digits are required in inexact arithmetic because the QR decomposition is numerically stable.

Decompositions related to solving systems of linear equations

LU decomposition

LU reduction

Block LU decomposition

Rank factorization

Cholesky decomposition

  • Applicable to: square, symmetric, positive definite matrix A
  • Decomposition: A=U^TU, where U is upper triangular with positive diagonal entries
  • Comment: the Cholesky decomposition is unique
  • Comment: the Cholesky decomposition is also applicable for complex hermitian positive definite matrices
  • Comment: An alternative is the LDL decomposition, which can avoid extracting square roots.

QR decomposition

  • Applicable to: m-by-n matrix A
  • Decomposition: A=QR where Q is an orthogonal matrix of size m-by-m, and R is an upper triangular matrix of size m-by-n
  • Comment: The QR decomposition provides an alternative way of solving the system of equations Ax=b without inverting the matrix A. The fact that Q is orthogonal means that Q^TQ=I, so that Ax=b is equivalent to Rx=Q^Tb, which is easier to solve since R is triangular.

RRQR factorization

Interpolative decomposition

Decompositions based on eigenvalues and related concepts

Eigendecomposition

  • Also called spectral decomposition
  • Applicable to: square matrix A with distinct eigenvectors (not necessarily distinct eigenvalues).
  • Decomposition: A=VDV^{-1}, where D is a diagonal matrix formed from the eigenvalues of A, and the columns of V are the corresponding eigenvectors of A.
  • Existence: An n-by-n matrix A always has n eigenvalues, which can be ordered (in more than one way) to form an n-by-n diagonal matrix D and a corresponding matrix of nonzero columns V that satisfies the eigenvalue equation AV=VD. If the n eigenvectors (not necessarily eigenvalues) are distinct (that is, none is equal to any of the others), then V is invertible, implying the decomposition A=VDV^{-1}.[2]
  • Comment: One can always normalize the eigenvectors to have length one (see definition of the eigenvalue equation). If A is real-symmetric, V is always invertible and can be made to have normalized columns. Then the equation VV^T=I holds, because each eigenvector is orthogonal to the other. Therefore, the decomposition (which always exists if A is real-symmetric) reads as: A=VDV^T
  • Comment: The condition of having n distinct eigenvalues is sufficient but not necessary. The necessary and sufficient condition is for each eigenvalue to have geometric multiplicity equal to its algebraic multiplicity.
  • Comment: The eigendecomposition is useful for understanding the solution of a system of linear ordinary differential equations or linear difference equations. For example, the difference equation x_{t+1}=Ax_t starting from the initial condition x_0=c is solved by x_t = A^tc, which is equivalent to x_t = VD^tV^{-1}c, where V and D are the matrices formed from the eigenvectors and eigenvalues of A. Since D is diagonal, raising it to power D^t, just involves raising each element on the diagonal to the power t. This is much easier to do and to understand than raising A to power t, since A is usually not diagonal.

Jordan decomposition

The Jordan normal form and the Jordan–Chevalley decomposition

  • Applicable to: square matrix A
  • Comment: the Jordan normal form generalizes the eigendecomposition to cases where there are repeated eigenvalues and cannot be diagonalized, the Jordan–Chevalley decomposition does this without choosing a basis.

Schur decomposition

  • Applicable to: square matrix A
  • Comment: there are two versions of this decomposition: the complex Schur decomposition and the real Schur decomposition. A complex matrix always has a complex Schur decomposition.
  • Decomposition (complex version): A=UTU^H, where U is a unitary matrix, U^H is the conjugate transpose of U, and T is an upper triangular matrix called the complex Schur form which has the eigenvalues of A along its diagonal.
  • Decomposition (real version): A=VSV^T, where A, V, S and V^T are matrices that contain real numbers only. In this case, V is an orthogonal matrix, V^T is the transpose of V, and S is a block upper triangular matrix called the real Schur form. The blocks on the diagonal of S are of size 1×1 (in which case they represent real eigenvalues) or 2×2 (in which case they are derived from complex conjugate eigenvalue pairs).

QZ decomposition

  • Also called: generalized Schur decomposition
  • Applicable to: square matrices A and B
  • Comment: there are two versions of this decomposition: complex and real.
  • Decomposition (complex version): A=QSZ^H and B=QTZ^H where Q and Z are unitary matrices, the H superscript represents conjugate transpose, and S and T are upper triangular matrices.
  • Comment: in the complex QZ decomposition, the ratios of the diagonal elements of S to the corresponding diagonal elements of T, \lambda_i = S_{ii}/T_{ii}, are the generalized eigenvalues that solve the generalized eigenvalue problem Av=\lambda Bv (where \lambda is an unknown scalar and v is an unknown nonzero vector).
  • Decomposition (real version): A=QSZ^T and B=QTZ^T where A, B, Q, Z, S, and T are matrices containing real numbers only. In this case Q and Z are orthogonal matrices, the T superscript represents transposition, and S and T are block upper triangular matrices. The blocks on the diagonal of S and T are of size 1×1 or 2×2.

Takagi's factorization

  • Applicable to: square, complex, symmetric matrix A.
  • Decomposition: A=VDV^T, where D is a real nonnegative diagonal matrix, and V is unitary. V^T denotes the matrix transpose of V.
  • Comment: The diagonal elements of D are the nonnegative square roots of the eigenvalues of AA^H.
  • Comment: V may be complex even if A is real.
  • Comment: This is not a special case of the eigendecomposition (see above).

Singular value decomposition

  • Applicable to: m-by-n matrix A.
  • Decomposition: A=UDV^H, where D is a nonnegative diagonal matrix, and U and V are unitary matrices, and V^H denotes the conjugate transpose of V (or simply the transpose, if V contains real numbers only).
  • Comment: The diagonal elements of D are called the singular values of A.
  • Comment: Like the eigendecomposition above, the singular value decomposition involves finding basis directions along which matrix multiplication is equivalent to scalar multiplication, but it has greater generality since the matrix under consideration need not be square.

Other decompositions

Polar decomposition

Algebraic polar decomposition

  • Applicable to: square, complex, non-singular matrix A.[3]
  • Decomposition: A=QS, where Q is a complex orthogonal matrix and S is complex symmetric matrix.
  • Comment: The existence of this decomposition is equivalent to AA^{T} being similar to A^{T}A.[4]

Sinkhorn normal form

  • Applicable to: square real matrix A with strictly positive elements.
  • Decomposition: A=D_{1}SD_{2}, where S is doubly stochastic and D1 and D2 are real diagonal matrices with strictly positive elements.

Sectoral decomposition[5]

  • Applicable to: square, complex matrix A with numerical range contained in the sector S_{\alpha}=\{r e^{i \theta} \in \mathbb{C} \mid r> 0, |\theta| \le \alpha \}.
  • Decomposition: A=CZC^{H}, where C is an invertible complex matrix and Z = diag(e^{i\theta_1},\ldots,e^{i\theta_n}) with all |\theta_{j}| \le \alpha .[5][6]

Generalizations

There exist analogues of the SVD, QR, LU and Cholesky factorizations for quasimatrices and cmatrices or continuous matrices.[7] A ‘quasimatrix’ is, like a matrix, a rectangular scheme whose elements are indexed, but one discrete index is replaced by a continuous index. Likewise, a ‘cmatrix’, is continuous in both indices. As an example of a cmatrix, one can think of the kernel of an integral operator.

These factorizations are based on early work by Fredholm (1903), Hilbert (1904) and Schmidt (1907). For an account, and a translation to English of the seminal papers, see Stewart (2011).

See also

Notes

  1. ^ Simon & Blume 1994 Chapter 7.
  2. ^ Meyer 2000, p. 514
  3. ^ Choudhury & Horn 1987, pp. 219–225
  4. ^ Horn & merino 1995, pp. 43–92
  5. ^ a b Zhang, Fuzhen (30 June 2014). "A matrix decomposition and its applications". Linear and Multilinear Algebra: 1–10.  
  6. ^ Drury, S.W. (November 2013). "Fischer determinantal inequalities and Highamʼs Conjecture". Linear Algebra and its Applications 439 (10): 3129–3133.  
  7. ^ Townsend & Trefethen 2015

References

  • Choudhury, Dipa; Horn, Roger A. (April 1987). "A Complex Orthogonal-Symmetric Analog of the Polar Decomposition". SIAM Journal on Algebraic Discrete Methods 8 (2).  
  •  
  •  
  • Horn, Roger A.; Merino, Dennis I. (January 1995). "Contragredient equivalence: A canonical form and some applications". Linear Algebra and its Applications 214.  
  • Meyer, C. D. (2000), Matrix Analysis and Applied Linear Algebra,  
  •  
  • Simon, C.; Blume, L. (1994). Mathematics for Economists. Norton.  
  • Stewart, G. W. (2011), Fredholm, Hilbert, Schmidt: three fundamental papers on integral equations (PDF), retrieved 2015-01-06 
  • Townsend, A.; Trefethen, L. N. (2015), "Continuous analogues of matrix factorizations",  

External links

  • Online Matrix Calculator
  • Springer Encyclopaedia of Mathematics » Matrix factorization
  • GraphLab GraphLab collaborative filtering library, large scale parallel implementation of matrix decomposition methods (in C++) for multicore.
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 



Copyright © World Library Foundation. All rights reserved. eBooks from World eBook Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.