Define the matrix

Eigenvalues quantify how much a matrix scales vectors along certain special directions. They serve as a crucial entry point for understanding dimensionality reduction, system stability, and the behavior of deep learning models.

I’ll remember: eigenvectors retain their direction under transformation; eigenvalues are merely scaling factors. If the direction changes, it’s not an eigendirection.

In the previous section, we discussed systems of linear equations—including both homogeneous and non-homogeneous cases. Next, we turn our focus to eigenvalues and their computation: foundational knowledge required to understand eigenvectors. Eigenvalues and eigenvectors play vital roles across machine learning, computer vision, quantum mechanics, and many other fields. Mastering them is thus not only a core component of linear algebra study but also a fundamental skill for AI research.

Definition of Eigenvalues

In mathematics, given a linear transformation represented by a matrix, eigenvalues capture intrinsic properties of that matrix. Specifically, for a square matrix $A$, a nonzero vector $\mathbf{v}$ is called an eigenvector of $A$ if there exists a scalar $\lambda$ such that:

To compute eigenvalues: first construct the characteristic equation, then compute its determinant, solve the resulting polynomial, and finally verify eigenvectors and handle repeated roots.

$$A\mathbf{v} = \lambda \mathbf{v}$$

In this equation, $\lambda$ is called the eigenvalue, and $\mathbf{v}$ is the corresponding eigenvector. Intuitively, an eigenvector retains its direction when transformed by $A$, while the eigenvalue indicates the factor by which its length is stretched or compressed along that direction.

Geometric Interpretation of Eigenvalues

Geometrically, eigenvalues and eigenvectors represent “inherent properties” of a linear transformation. For instance, consider a point in the 2D plane as a vector. When we apply a matrix $A$ to transform this point, most points change both direction and magnitude. However, points lying along certain special directions—namely, eigenvectors—remain aligned with their original direction after transformation; only their lengths change. The ratio of the new length to the original length is precisely the eigenvalue associated with that eigendirection.

Computing Eigenvalues

Computing eigenvalues involves solving the characteristic polynomial, detailed below:

After finishing Definition and Computation of Eigenvalues, treat the flowchart in this card as a checklist: Is the problem clearly defined? Are operations concrete and actionable? Can the evaluation criteria be reused?

1. Characteristic Polynomial: First, compute the characteristic polynomial of matrix $A$, obtained via the determinant:

$$\text{det}(A - \lambda I) = 0$$

Here, $I$ denotes the identity matrix of the same size as $A$, and $\lambda$ represents the unknown eigenvalue. Solutions $\lambda$ to this equation are the eigenvalues of $A$.

2. Computation Steps:
   • Compute $A - \lambda I$, then evaluate its determinant.
   • Set the determinant equal to zero to obtain a polynomial in $\lambda$.
   • Solve for the roots of this polynomial—the roots are the eigenvalues.

Example

Suppose we have the matrix:

$$A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$$

The steps to compute its eigenvalues are as follows:

1. Compute $A - \lambda I$:

$$A - \lambda I = \begin{pmatrix} 2 - \lambda & 1 \\ 1 & 2 - \lambda \end{pmatrix}$$

2. Compute the determinant:

$$\text{det}(A - \lambda I) = (2 - \lambda)(2 - \lambda) - 1 \cdot 1 = (2 - \lambda)^2 - 1$$

3. Set the determinant to zero:

$$(2 - \lambda)^2 - 1 = 0$$

4. Solve the equation:

$$(2 - \lambda)^2 = 1 \\ 2 - \lambda = \pm 1 \\ \lambda_1 = 3,\quad \lambda_2 = 1$$

Thus, the eigenvalues of matrix $A$ are $\lambda_1 = 3$ and $\lambda_2 = 1$.

Python Code Example

Below is a Python example using the numpy library to compute eigenvalues:

import numpy as np

# Define the matrix
A = np.array([[2, 1],
              [1, 2]])

# Compute eigenvalues and eigenvectors
eigenvalues, eigenvectors = np.linalg.eig(A)

print("Eigenvalues:", eigenvalues)

When executed, this code outputs an array of eigenvalues—e.g., [3. 1.].

If Definition and Computation of Eigenvalues hasn’t yet fully clicked, revisit the four actions on this card to walk through the material again.

When reviewing Definition and Computation of Eigenvalues, avoid tackling large projects all at once. Instead, start with a simple worked example to confirm whether the core logic is clear.

Summary

In this section, we introduced the definition and geometric meaning of eigenvalues, and walked step-by-step through computing them via the characteristic polynomial. These concepts form the essential foundation for our next discussion: eigenvectors—their definition, interpretation, and computation. To internalize these ideas effectively, hands-on examples and code implementation are invaluable; readers are strongly encouraged to practice repeatedly. In the next section, we will delve into eigenvectors in depth.