16. Eigenvectors: Definition and Intuition

An eigenvector is not a fixed-length arrow—it is a direction. As long as the direction remains unchanged, scalar multiples of the same vector still represent the same eigenvector.

I will substitute back into $A\mathbf{v} = \lambda \mathbf{v}$ to verify. Solving only for candidate values without substituting them back to check risks misidentifying ordinary vectors as eigenvectors.

In the previous article, we discussed the definition and computation of eigenvalues—a foundation essential for understanding eigenvectors. Eigenvectors are a central concept in linear algebra and play critical roles across many applications, especially in artificial intelligence, machine learning, and data analysis.

Definition of Eigenvectors

An eigenvector is a special type of vector that—when transformed by a linear map—changes only in magnitude (and possibly sign), but not in direction. Specifically, for a given square matrix $A$, if there exists a nonzero vector $\mathbf{v}$ and a scalar $\lambda$ satisfying:

After reading “Eigenvectors — Definition”, try working through a small concrete example step-by-step, then assess which steps you can now perform independently.

By this point, you can organize “Eigenvectors — Definition” into a reflection table: first clarify the main conceptual thread, then validate it using a small, self-contained task.

$$A \mathbf{v} = \lambda \mathbf{v},$$

then $\mathbf{v}$ is called an eigenvector of matrix $A$, and $\lambda$ is the corresponding eigenvalue.

Example Illustration

Consider the simple $2\times2$ matrix:

$$A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}.$$

We wish to find the eigenvectors of $A$. First, we compute its eigenvalues $\lambda$ by solving the characteristic equation:

$$\det(A - \lambda I) = 0,$$

where $I$ is the $2\times2$ identity matrix.

Computing $A - \lambda I$, we obtain:

$$A - \lambda I = \begin{pmatrix} 2 - \lambda & 1 \\ 1 & 2 - \lambda \end{pmatrix}.$$

Its determinant is:

$$\det(A - \lambda I) = (2 - \lambda)^2 - 1 = (\lambda - 1)(\lambda - 3).$$

Thus, the eigenvalues are $\lambda_1 = 1$ and $\lambda_2 = 3$.

Next, we compute the eigenvectors associated with each eigenvalue.

Computing Eigenvectors

For each eigenvalue $\lambda$, we solve $(A - \lambda I)\mathbf{v} = \mathbf{0}$ to find the corresponding eigenvectors.

To determine whether a vector is an eigenvector:

1. Confirm it is nonzero;
2. Compute the result of applying the matrix $A$ to it;
3. Check whether the output is a scalar multiple of the original vector.

Eigenvalue $\lambda_1 = 1$:
We solve:

$$A \mathbf{v} = 1 \cdot \mathbf{v} \quad \Longleftrightarrow \quad (A - I)\mathbf{v} = \mathbf{0}.$$

Here,

$$A - I = \begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}.$$

Solving $(A - I)\mathbf{v} = \mathbf{0}$ yields an eigenvector:

$$\begin{pmatrix} 1 \\ -1 \end{pmatrix}.$$

We often prefer normalized eigenvectors—i.e., those of unit length. The above eigenvector normalizes to:

$$\mathbf{v}_1 = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ -1 \end{pmatrix}.$$

Eigenvalue $\lambda_2 = 3$:
Similarly, solve:

$$(A - 3I)\mathbf{v} = \mathbf{0}.$$

Here,

$$A - 3I = \begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix}.$$

Solving gives an eigenvector:

$$\begin{pmatrix} 1 \\ 1 \end{pmatrix}.$$

Normalizing:

$$\mathbf{v}_2 = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ 1 \end{pmatrix}.$$

Summary

Eigenvectors are vectors whose direction remains invariant under a given linear transformation. They provide deep insight into the geometric and algebraic structure of linear maps—and serve as foundational tools in dimensionality reduction, image processing, recommendation systems, and beyond.

“Eigenvectors — Definition” is best read alongside its diagrams. Begin by clarifying the core question and verification criteria; then proceed to the conceptual explanation and worked steps—this helps connect ideas into a coherent mental model.

In the next article, we’ll explore the relationship between eigenvalues and eigenvectors—namely, eigen-decomposition, a concept tightly linked to today’s topic. Stay tuned!