SciPy - Condition Numbers

In SciPy the Condition Number quantifies the sensitivity of a matrix to small changes in its input or its elements. It is a key metric for assessing the numerical stability of linear systems, eigenvalue problems and matrix computations. The condition number of a matrix A is defined as follows −

κ(A) = ‖ A ‖ ċ ‖ A-1 ‖

where A is a matrix norm e.g., 2-norm, Frobenius norm and A^-1 is the norm of the inverse matrix. A higher condition number indicates that the matrix is ill-conditioned which means small changes in the input may cause large errors in the output. For singular matrices, the condition number is infinite because the inverse does not exist.

Key Concepts

Following are the key concepts of the Conditional Numbers −

• Condition number: A value that indicates the sensitivity of the matrix. A large condition number e.g., > 10⁶ indicates that the matrix is ill-conditioned and small errors in the matrix or input could lead to large errors in the computed solution.
• Norm: The condition number depends on the norm used. Common norms include the 2-norm (spectral norm), 1-norm and Frobenius norm.

Interpretation of Condition Numbers

Here are the Interpretations of the Condition Numbers in Scipy −

• Small Condition Number (close to 1): The matrix is well-conditioned which means that small changes in the input data result in small changes in the output. The system of equations is numerically stable.
• Large Condition Number: The matrix is ill-conditioned. Small changes or errors in the input can lead to large errors in the solution. This typically occurs when the matrix is close to singular which means it does not have a full rank or has nearly zero singular values.

Example 1

The condition number of a matrix is the ratio of the largest singular value to the smallest singular value. As we don't have a predefined function to implement the condition number, we can use the scipy.linalg.svd() function to implement the computation of the condition number manually. Here is the example of implementing the condition number −

import numpy as np
from scipy.linalg import svd

# Example matrix (well-conditioned)
A_well = np.array([[2, 1], [1, 3]])

# Compute Singular Value Decomposition (SVD)
U, s, Vh = svd(A_well)

# Compute the condition number: max singular value / min singular value
condition_number = s[0] / s[-1]

print(f"Condition number (well-conditioned matrix): {condition_number}")

# Example matrix (ill-conditioned)
A_ill = np.array([[1, 2], [2, 4]])

# Compute Singular Value Decomposition (SVD)
U, s, Vh = svd(A_ill)

# Compute the condition number
condition_number_ill = s[0] / s[-1]

print(f"Condition number (ill-conditioned matrix): {condition_number_ill}")

Here is the output of the function scipy.linalg.svd() which is used to implement the condition number −

Condition number (well-conditioned matrix): 2.6180339887498953
Condition number (ill-conditioned matrix): 2.5175887275607884e+16

Example 2

Here is the example of the well-conditioned and ill-conditioned Matrices together implemented by using the function scipy.linalg.svd() −

import numpy as np
from scipy.linalg import svd

# Well-conditioned matrix
A_well = np.array([[2, 1], [1, 3]])

# Ill-conditioned matrix
A_ill = np.array([[1, 2], [2, 4]])

# Compute singular values and condition numbers for both matrices

def compute_condition_number(A):
    # Compute singular values using SVD
    U, S, Vt = svd(A)
    # Condition number is the ratio of the largest to the smallest singular value
    return S[0] / S[-1]

# Condition numbers
condition_number_well = compute_condition_number(A_well)
condition_number_ill = compute_condition_number(A_ill)

print(f"Singular values (well-conditioned matrix): {svd(A_well)[1]}")
print(f"Condition number (well-conditioned matrix): {condition_number_well}")

print(f"Singular values (ill-conditioned matrix): {svd(A_ill)[1]}")
print(f"Condition number (ill-conditioned matrix): {condition_number_ill}")

Here is the output of the function scipy.linalg.svd() which is used to implement the well-conditioned and ill-conditioned matrices −

Singular values (well-conditioned matrix): [3.61803399 1.38196601]
Condition number (well-conditioned matrix): 2.6180339887498953
Singular values (ill-conditioned matrix): [5.00000000e+00 1.98602732e-16]
Condition number (ill-conditioned matrix): 2.5175887275607884e+16

Condition Number and Solving Linear Systems

When solving a system of linear equations Ax=b, the condition number (A) of the matrix A gives an estimate of how much the solution x will change in response to small changes in the matrix A or the vector b.

• If the condition number is low, then solving the system is stable and small changes in A or b will only cause small changes in x.
• If the condition number is high then the solution can be highly sensitive to errors. Small changes in A or b could result in large changes in x.