Unit 10 of 11
Singular Value Decomposition
- 5 lessons
- Videos 47 to 51
Every real matrix, square or rectangular, has a singular value decomposition. This unit reads its geometry, computes it, and uses it for pseudoinverses and low-rank approximation.
The decomposition finds orthonormal input directions that the matrix sends to mutually orthogonal outputs. The nonzero outputs have lengths equal to the positive singular values, and the remaining input directions span the kernel. This shows the rank, the kernel and the largest stretch of the map. The pseudoinverse picks the least-squares solution of smallest length, truncating the decomposition gives a best approximation of each lower rank in both the spectral and Frobenius norms, and centering data before keeping its leading directions gives principal component analysis.
Every lesson will be one video. The videos are coming soon, and each lesson page already has the definitions and an example for every key term.
Geometry and Computation of the SVD
Singular Value Decomposition and Its Geometry
What the singular value decomposition says about a real matrix of any shape, how singular values pair right and left singular vectors, and how they give the rank, the kernel and the largest stretch.
Video 47
Computing an SVD in Full, Economy and Compact Forms
How to compute a singular value decomposition from a symmetric eigenvalue problem, complete orthonormal bases beyond the positive singular values, tell full, economy and compact SVDs apart, and check an answer by reconstruction.
Video 48
Pseudoinverses and Minimum-Norm Solutions
Pseudoinverses and Minimum-Norm Solutions
How the singular value decomposition defines the pseudoinverse, why it gives the least-squares solution of minimum norm, and how the null-space component separates choosing a solution from recovering lost information.
Video 49
Low-Rank Approximation and Principal Components
Outer Products and Low-Rank Approximation
How an outer product builds a rank-one matrix, how the SVD writes a matrix as a sum of rank-one layers, and how truncating that sum gives a best rank-k approximation in the Frobenius and spectral norms.
Video 50
Principal Component Analysis with Centering and Variance
How principal component analysis uses feature means, centering and the sample covariance matrix to find directions of largest projected variance, and how retained variance, reconstruction error and units enter.
Video 51