Define matrix A

Eigen decomposition breaks down a complex matrix into two intuitive components: direction (encoded in eigenvectors) and scaling (encoded in eigenvalues). When applicable, many matrix operations become significantly more transparent.

We’ll first verify whether a given matrix is diagonalizable. Not all matrices admit a full eigen decomposition.

In the previous article, we introduced the fundamental definitions of eigenvalues and eigenvectors. This article delves deeper into eigen decomposition—a powerful technique that expresses a matrix as a product of its intrinsic building blocks: eigenvectors and eigenvalues.

Review: Eigenvalues and Eigenvectors

Let’s begin by recalling several core concepts. For a square matrix ( A ), an eigenvalue ( \lambda ) and its corresponding eigenvector ( \mathbf{v} ) satisfy:

$$A \mathbf{v} = \lambda \mathbf{v}$$

Here, applying ( A ) to ( \mathbf{v} ) results purely in scaling—no rotation or shearing—by the scalar factor ( \lambda ).

What Is Eigen Decomposition?

Eigen decomposition expresses a square matrix ( A ) as a product involving its eigenvectors and eigenvalues. Specifically, if ( A ) has ( n ) linearly independent eigenvectors, it can be written as:

$$A = V \Lambda V^{-1}$$

where

• ( V ) is the matrix whose columns are the eigenvectors of ( A ), and
• ( \Lambda ) is a diagonal matrix whose entries are the corresponding eigenvalues of ( A ).

This representation is called the eigen decomposition of ( A ).

Conditions for Eigen Decomposition

Eigen decomposition is only possible when ( A ) is diagonalizable. A necessary and sufficient condition is that ( A ) possesses ( n ) linearly independent eigenvectors (i.e., a full basis of eigenvectors in ( \mathbb{R}^n )).

Step-by-Step Eigen Decomposition

Let’s walk through eigen decomposition using a concrete example. Consider the matrix:

$$A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$$

Step 1: Compute the Eigenvalues

Eigenvalues are found by solving the characteristic equation:

$$\det(A - \lambda I) = 0$$

where ( I ) is the identity matrix. Compute:

$$A - \lambda I = \begin{pmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{pmatrix}$$

Its determinant is:

$$\det(A - \lambda I) = (4 - \lambda)(3 - \lambda) - 2 \cdot 1 = \lambda^2 - 7\lambda + 10$$

Solve the quadratic:

$$\lambda^2 - 7\lambda + 10 = 0$$

Using the quadratic formula yields:

$$\lambda_1 = 5, \quad \lambda_2 = 2$$

Step 2: Compute the Eigenvectors

For each eigenvalue ( \lambda_i ), solve the homogeneous system ( (A - \lambda_i I)\mathbf{v} = \mathbf{0} ).

1. For ( \lambda_1 = 5 ):

$$A - 5I = \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix}$$

The row-reduced system gives:

$$- x + y = 0 \quad \Rightarrow \quad y = x$$

So one eigenvector is:

$$\mathbf{v}_1 = k \begin{pmatrix} 1 \\ 1 \end{pmatrix}, \quad k \neq 0$$

2. For ( \lambda_2 = 2 ):

$$A - 2I = \begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix}$$

The system reduces to:

$$2x + y = 0 \quad \Rightarrow \quad y = -2x$$

So a corresponding eigenvector is:

$$\mathbf{v}_2 = k \begin{pmatrix} 1 \\ -2 \end{pmatrix}, \quad k \neq 0$$

Step 3: Assemble the Eigen Decomposition

We now have:

When interpreting eigen decomposition, first assess: Is the matrix diagonalizable? Are eigenvalues distinct (or at least sufficiently many)? Are eigenvectors linearly independent? And finally—does the reconstructed product match the original matrix?

• Eigenvalues: ( \lambda_1 = 5 ), ( \lambda_2 = 2 )
• Eigenvectors (choosing ( k = 1 )):

$$\mathbf{v}_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}, \quad \mathbf{v}_2 = \begin{pmatrix} 1 \\ -2 \end{pmatrix}$$

Construct:

$$V = \begin{pmatrix} 1 & 1 \\ 1 & -2 \end{pmatrix}, \quad \Lambda = \begin{pmatrix} 5 & 0 \\ 0 & 2 \end{pmatrix}$$

You can verify that ( A = V \Lambda V^{-1} ) holds.

Code Example

Using Python and NumPy, eigen decomposition is straightforward:

Before reading “Eigen Decomposition of Eigenvalues and Eigenvectors”, preview the visual path from problem → computation → result shown above. After reading, revisit this diagram to confirm you can reproduce each step.

import numpy as np

# Define matrix A
A = np.array([[4, 1],
              [2, 3]])

# Compute eigenvalues and eigenvectors
eigenvalues, eigenvectors = np.linalg.eig(A)

print("Eigenvalues:", eigenvalues)
print("Eigenvectors:\n", eigenvectors)

# Reconstruct A
V = eigenvectors
Lambda = np.diag(eigenvalues)
A_reconstructed = V @ Lambda @ np.linalg.inv(V)

print("Reconstructed A:\n", A_reconstructed)

Running this code yields a matrix numerically identical to the original ( A ), confirming correctness of the decomposition.

When reviewing “Eigen Decomposition of Eigenvalues and Eigenvectors”, consolidate key concepts, procedural steps, and observable outcomes on a single page for efficient revision.

When practicing “Eigen Decomposition of Eigenvalues and Eigenvectors”, explicitly list input conditions, computational actions, and expected outputs side-by-side—this supports accurate self-checking and future review.

Summary

Eigen decomposition is a foundational tool in linear algebra, widely used in machine learning and data analysis. It reveals how a matrix acts geometrically—by stretching space along specific directions (eigenvectors), scaled by corresponding magnitudes (eigenvalues). This insight is essential for understanding data structure, performing dimensionality reduction (e.g., Principal Component Analysis), and solving systems of differential equations.

In the next article, we’ll explore inner products and orthogonality, focusing on the definition, properties, and geometric interpretation of inner products. Stay tuned!