SciPy - Singular Value Decomposition (SVD)

---

---

Singular Value Decomposition in SciPy

SciPy's Singular Value Decomposition (SVD) is a computational method provided by the scipy.linalg module for decomposing a matrix into three components namely, two orthogonal matrices and a diagonal matrix of singular values. It is a cornerstone in numerical linear algebra with applications in data analysis, signal processing and machine learning.

The Singular Value Decomposition(SVD) is defined for a matrix A of size m x n as follows −

A = UVT

Where −

• U is an m x n orthogonal matrix.
• is an mn diagonal matrix with non-negative real numbers on the diagonal i.e., singular values.
• V^T is an n x n orthogonal matrix.

Syntax

In Scipy the Singular Value Decomposition (SVD) is computed with the help of the svd() function of the linalg module available in scipy library. Below is the syntax of the scipy.linalg.svd() function −

scipy.linalg.svd(a, full_matrices=True, compute_uv=True, overwrite_a=False, check_finite=True, lapack_driver='gesdd')

Where −

• a: Input matrix to be decomposed.
• full_matrices(default : True): If True then computes U and V^T as full size matrices.
• compute_uv(default : True): If false then only computes the singular values without U and V^T.

This functions returns upto three outputs depending on the compute_uv flag as follows −

• u((ndarray, shape (M, M) or (M, K))): The left singular vectors. The shape depends on full_matrices.
• s(ndarray, shape (K,)): The singular values which are always sorted in descending order.
• Vt(ndarray, shape (N, N) or (K, N)): The transpose of the right singular vectors. hape depends on full_matrices.

Applications of SVD

Below are the applications of the Singular Value Decomposition(SVD) −

• Dimensionality Reduction: In machine learning SVD is used to reduce the dimensions of large datasets like in PCA, Principal Component Analysis.
• Noise Reduction: SVD can help in identifying and removing noise in the data.
• Data Compression: SVD is used in applications like image compression.
• Matrix Inversion: SVD can be used to find the pseudoinverse of a matrix.

Example of Basic Full SVD

Following is the example which uses the function scipy.linalg.svd() to compute a full SVD using SciPywith full_matrices=True the default value −

import numpy as np
from scipy.linalg import svd

# Input matrix
A = np.array([[1, 2], [3, 4], [5, 6]])

# Compute Full SVD
U, s, Vt = svd(A, full_matrices=True)

# Diagonalize the singular values for full SVD reconstruction
Sigma = np.zeros((A.shape[0], A.shape[1]))
np.fill_diagonal(Sigma, s)

# Print results
print("Matrix A:")
print(A)

print("\nU (left singular vectors):")
print(U)

print("\nSigma (diagonal matrix of singular values):")
print(Sigma)

print("\nVt (right singular vectors):")
print(Vt)

# Verify reconstruction
A_reconstructed = np.dot(U, np.dot(Sigma, Vt))
print("\nReconstructed A:")
print(A_reconstructed)

Here is the output of the basic full SVD −

Matrix A:
[[1 2]
 [3 4]
 [5 6]]

U (left singular vectors):
[[-0.2298477   0.88346102  0.40824829]
 [-0.52474482  0.24078249 -0.81649658]
 [-0.81964194 -0.40189603  0.40824829]]

Sigma (diagonal matrix of singular values):
[[9.52551809 0.        ]
 [0.         0.51430058]
 [0.         0.        ]]

Vt (right singular vectors):
[[-0.61962948 -0.78489445]
 [-0.78489445  0.61962948]]

Reconstructed A:
[[1. 2.]
 [3. 4.]
 [5. 6.]]

Example of Reduced SVD

Reduced Singular Value Decomposition (Reduced SVD) is a simplified version of full SVD that only computes the essential components corresponding to the rank of the matrix A. It avoids computing unnecessary singular vectors by making it computationally efficient especially for large matrices. Following is the example which use reduced - size U and V^T matrices to save memory for large matrices −

import numpy as np
from scipy.linalg import svd

# Input matrix
A = np.array([[1, 2], [3, 4], [5, 6]])

# Perform reduced SVD
U, s, Vt = svd(A, full_matrices=False)

print("Reduced U:")
print(U)

print("\nReduced Vt:")
print(Vt)

# Notice that U and Vt are smaller compared to full SVD

Here is the output of the basic full SVD −

Reduced U:
[[-0.2298477   0.88346102]
 [-0.52474482  0.24078249]
 [-0.81964194 -0.40189603]]

Reduced Vt:
[[-0.61962948 -0.78489445]
 [-0.78489445  0.61962948]]

Reconstructing the Original Matrix

Here is the example of reconstructing the original matrix A from its SVD components involves multiplying the matrices U, and V^T together −

import numpy as np
from scipy.linalg import svd

# Example matrix
A = np.array([[1, 2], [3, 4], [5, 6]])

# Compute SVD
U, s, Vt = svd(A)

# Rebuild the Sigma matrix (m x n)
Sigma = np.zeros((A.shape[0], A.shape[1]))
np.fill_diagonal(Sigma, s)

# Reconstruct the original matrix
A_reconstructed = np.dot(U, np.dot(Sigma, Vt))

# Output
print("Original Matrix A:")
print(A)
print("\nReconstructed Matrix A from SVD components:")
print(A_reconstructed)

Following is the example of reconstructing the original matrix from the svd components −

Reduced U:
[[-0.2298477   0.88346102]
 [-0.52474482  0.24078249]
 [-0.81964194 -0.40189603]]

Reduced Vt:
[[-0.61962948 -0.78489445]
 [-0.78489445  0.61962948]]
PS D:\Tutorialspoint> python sample.py
Reduced U:
[[-0.2298477   0.88346102]
 [-0.52474482  0.24078249]
 [-0.81964194 -0.40189603]]

Reduced Vt:
[[-0.61962948 -0.78489445]
 [-0.78489445  0.61962948]]