SciPy - L2 Norm(Spectral Norm)

---

---

Spectral Norm in SciPy (L2)

The L2 Norm (also known as the spectral norm) in SciPy is a matrix norm that corresponds to the largest singular value of the matrix. It is widely used in various fields such as numerical optimization, machine learning, and stability analysis due to its significance in measuring the largest scaling factor that the matrix can apply to any vector.

The L2 norm of a matrix A quantifies the maximum amount by which the matrix stretches a vector. In essence, it provides a measure of how much the matrix "expands" vectors in its domain. The norm is equal to the largest singular value of A, which can be found by computing the singular value decomposition (SVD) of the matrix.

If a matrix A has dimensions m x n, the mathematical formula for the L2 Norm (Spectral Norm) ||A||₂ is:

$\mathrm{|A|_2 = σ^{max}}$

Where −

• σ_max denotes the largest singular value of matrix A.
• The spectral norm is the largest eigenvalue of the matrix A^TA where A^T is the transpose of A.

Properties of the L2 Norm (Spectral Norm)

The L2 norm has a number of useful mathematical properties that make it important in many applications, especially in numerical analysis. These are some key properties −

Non-Negativity

The L2 norm is always non-negative and it is zero if and only if the matrix is a zero matrix i.e., all elements of A are zero.

A2  0

Homogeneity (Scaling)

The L2 norm is homogeneous with respect to scalar multiplication. That is, scaling a matrix by a constant factor scales its norm by the absolute value of the constant.

cA2 = |c|  A2

where c is a scalar, and A is a matrix. This means that multiplying a matrix by a constant factor scales its L2 norm by the absolute value of that constant.

Subadditivity (Triangle Inequality)

The L2 norm satisfies the triangle inequality which means the norm of the sum of two matrices is less than or equal to the sum of their norms.

A + B2  A2 + B2

for matrices A and B of the same size. This property ensures that the L2 norm behaves like a distance metric.

Invariance under Orthogonal/Unitary Transformations

The L2 norm remains unchanged when the matrix is multiplied by an orthogonal (real) or unitary (complex) matrix.

UA2 = A2 and AV2 = A2

where U and V are orthogonal or unitary matrices. This property is significant in many numerical methods such as Singular Value Decomposition (SVD) where transformations do not alter the L2 norm.

Example 1

The following example demonstrates how to compute the L2 Norm (Spectral Norm) using SciPy's scipy.linalg.norm() function by passing the argument ord=2. We can calculate the L2 Norm for a small 2D matrix −

import numpy as np
from scipy.linalg import norm

# Define the matrix
A = np.array([[1, 2],
              [3, 4]])

# Compute the L2 norm (spectral norm)
l2_norm = norm(A, ord=2)

print("Matrix A:")
print(A)
print("L2 Norm (Spectral Norm) of A:", l2_norm)

Following is the output for the 2D matrix's L2 Norm −

Matrix A:
[[1 2]
 [3 4]]
L2 Norm (Spectral Norm) of A: 5.464985704219043

Example 2

In this example we calculate the L2 Norm or Spectral Norm for a complex matrix using the scipy.linalg.norm() function −

import numpy as np
from scipy.linalg import norm

# Define the complex matrix
A = np.array([[1 + 1j, 2 - 1j],
              [-1j, 3 + 2j]])

# Compute the L2 norm (spectral norm)
l2_norm = norm(A, ord=2)

print("Complex Matrix A:")
print(A)
print("L2 Norm (Spectral Norm) of A:", l2_norm)

Heres the result of the L2 Norm (Spectral Norm) for a complex matrix −

Complex Matrix A:
[[ 1.+1.j  2.-1.j]
 [-0.-1.j  3.+2.j]]
L2 Norm (Spectral Norm) of A: 4.25045561972017

Example 3

Here in this example let's calculate the L2 Norm (Spectral Norm) for a zero matrix −

import numpy as np
from scipy.linalg import norm

# Define the zero matrix
A = np.zeros((2, 2))

# Compute the L2 norm (spectral norm)
l2_norm = norm(A, ord=2)

print("Zero Matrix A:")
print(A)
print("L2 Norm (Spectral Norm) of A:", l2_norm)

Here is the output for the L2 Norm (Spectral Norm) of a zero matrix −

Zero Matrix A:
[[0. 0.]
 [0. 0.]]
L2 Norm (Spectral Norm) of A: 0.0