Define matrix A

You can compute singular values manually by starting from the eigenvalues of $A^T A$. In practice, engineers delegate this task to numerical libraries—but understanding the underlying derivation helps interpret results correctly.

I verify that singular values are non-negative and typically ordered in descending order. If their ordering is incorrect, low-rank approximations will also be wrong.

In the previous article, we introduced the concept of Singular Value Decomposition (SVD) and discussed why it plays such a pivotal role in data science and machine learning. This article dives deeper into how singular values are computed—equipping you with mastery of this core technique and laying a solid foundation for subsequent applications.

1. Theoretical Foundation

Singular Value Decomposition (SVD) is a matrix factorization method that decomposes any matrix $A$ into the product of three matrices. Specifically, if $A$ is an $m \times n$ matrix, it can be expressed as:

When computing singular values, consider: transpose–product relationships, eigenvalues, square-root mapping, singular vectors, dimensional changes, and numerical stability.

$$A = U \Sigma V^T$$

where:

• $U$ is an $m \times m$ orthogonal matrix;
• $\Sigma$ is an $m \times n$ diagonal matrix containing the singular values of $A$;
• $V^T$ is the transpose of $V$, and $V$ is an $n \times n$ orthogonal matrix.

Singular values represent a kind of “signature” of matrix $A$, reflecting its structural properties and information content. To extract them from $\Sigma$, follow these steps:

1. Compute $A^T A$ and $A A^T$.
2. Obtain the eigenvalues of both matrices.
3. Take the non-negative square roots of those eigenvalues to yield the singular values.

2. Step-by-Step Computation of Singular Values

To compute singular values, proceed as follows:

Step 1: Compute $A^T A$

Given a matrix $A$, first compute its transpose $A^T$, then multiply it by $A$ to obtain $A^T A$. The result is an $n \times n$ matrix.

Step 2: Compute Eigenvalues

Next, compute the eigenvalues $\lambda_i$ of $A^T A$. If $A$ is $m \times n$, then $A^T A$ is $n \times n$, and its eigenvalues can be found by solving the characteristic polynomial.

Step 3: Compute Singular Values

Each singular value $s_i$ is the non-negative square root of the corresponding eigenvalue:

$$s_i = \sqrt{\lambda_i}$$

Apply this to all eigenvalues $\lambda_i$ to obtain the full set of singular values.

3. Worked Example

Let’s demonstrate singular value computation using a concrete matrix. Consider:

“The Computation of Singular Values in SVD” can be read through four lenses: scenario, concept, action, and outcome. First align these four elements, then revisit parameters, code, or workflows in the main text.

$$A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{pmatrix}$$

Detailed Computation Steps

1. Compute $A^T A$:

   $$A^T = \begin{pmatrix} 1 & 3 & 5 \\ 2 & 4 & 6 \end{pmatrix}$$

   So,

   $$A^T A = \begin{pmatrix} 1 & 3 & 5 \\ 2 & 4 & 6 \end{pmatrix} \begin{pmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{pmatrix} = \begin{pmatrix} 35 & 44 \\ 44 & 56 \end{pmatrix}$$

2. Compute eigenvalues: Solve

   $$\det(A^T A - \lambda I) = 0,$$

   where $I$ is the $2 \times 2$ identity matrix. Solving yields eigenvalues:

   $$\lambda_1 = 91,\quad \lambda_2 = 0.$$

3. Compute singular values:

   $$s_1 = \sqrt{\lambda_1} = \sqrt{91} \approx 9.54,$$ $$s_2 = \sqrt{\lambda_2} = \sqrt{0} = 0.$$

Thus, the singular values of $A$ are approximately $9.54$ and $0$.

4. Python Implementation

Using NumPy, we can implement this computation concisely:

import numpy as np

# Define matrix A
A = np.array([[1, 2], [3, 4], [5, 6]])

# Compute SVD
U, s, VT = np.linalg.svd(A)

# Output singular values
print("Singular values:", s)

Running this code yields:

Singular values: [9.52551809 0.51449576]

After studying “The Computation of Singular Values in SVD”, try applying it to your own scenario—pay close attention to whether inputs, processing steps, and outputs align coherently.

To apply “The Computation of Singular Values in SVD” to your own task, begin by narrowing the scope—focus on validating just one critical decision point.

Summary

This article thoroughly explains how singular values are computed—including theoretical foundations, a step-by-step worked example, and a practical Python implementation. This knowledge is essential for deeply understanding SVD and its applications. In the next article, we’ll explore real-world use cases of SVD—for instance, in recommendation systems and image processing.

Stay tuned for our ongoing tutorial series—and deepen your grasp of linear algebra’s vital role in AI!