Example vectors

An orthogonal basis functions like a set of mutually independent coordinate rulers. When representing vectors using such a basis, projections and reconstructions become transparent and intuitive.

I will compute pairwise inner products. Orthogonality is not merely about appearing perpendicular—it is defined rigorously by an inner product equaling exactly zero.

Orthogonality is a fundamental concept in linear algebra—especially critical when working with high-dimensional data. A solid understanding of orthogonal vectors and orthogonal bases is essential for effectively applying AI and machine learning algorithms. This tutorial dives deeply into the definitions and properties of orthogonal vectors and orthogonal bases, illustrated with concrete examples.

Orthogonal Vectors

Let’s first recall the definition of the inner product. In the previous section, we introduced its key properties. Two vectors in an inner product space are said to be orthogonal if their inner product equals zero. Mathematically, given vectors $\mathbf{u}$ and $\mathbf{v}$, they are orthogonal if and only if:

When learning orthogonal vectors and orthogonal bases, begin by reviewing: inner products, vector normalization (unit length), linear independence of basis vectors, projection computation, and coordinate representation.

$$\langle \mathbf{u}, \mathbf{v} \rangle = 0$$

For example, in $\mathbb{R}^3$, vectors $\mathbf{u} = (1, 0, 0)$ and $\mathbf{v} = (0, 1, 0)$ are orthogonal because their inner product is:

$$\langle \mathbf{u}, \mathbf{v} \rangle = 1 \cdot 0 + 0 \cdot 1 + 0 \cdot 0 = 0$$

Example: Application of Orthogonal Vectors

Suppose we have two vectors in the 2D plane: $\mathbf{a} = (2, 3)$ and $\mathbf{b} = (-3, 2)$. We can verify whether they are orthogonal:

$$\langle \mathbf{a}, \mathbf{b} \rangle = 2 \cdot (-3) + 3 \cdot 2 = -6 + 6 = 0$$

Since their inner product is zero, these two vectors are indeed orthogonal.

Orthogonal Bases

In a vector space, a basis is a set of linearly independent vectors such that any vector in the space can be expressed uniquely as a linear combination of them. A basis is called an orthogonal basis if every pair of distinct vectors in the set is orthogonal.

After finishing Orthogonal Vectors and Orthogonal Bases, take one minute to reflect: Are the core concepts clearly distinguished? Can you reproduce the practice steps? Can you restate the conclusions in your own words?

For instance, in three-dimensional space, the standard basis vectors $\mathbf{e}_1 = (1, 0, 0)$, $\mathbf{e}_2 = (0, 1, 0)$, and $\mathbf{e}_3 = (0, 0, 1)$ form an orthogonal basis—because each pair is orthogonal and each vector has unit length.

Properties of Orthogonal Bases

A key property of orthogonal bases is normalization (or unit-length scaling). If we normalize each vector in an orthogonal basis—i.e., scale it to have length 1—we obtain an orthonormal basis, where every basis vector satisfies:

$$\|\mathbf{e}_i\| = 1, \quad \forall i$$

Example: Computing an Orthogonal Basis in Python

We can construct an orthogonal basis from a set of linearly independent vectors using the Gram–Schmidt process. Below is a simple implementation in Python:

import numpy as np

def gram_schmidt(vectors):
    orthogonal_vectors = []
    
    for v in vectors:
        for av in orthogonal_vectors:
            v -= np.dot(v, av) / np.dot(av, av) * av
        orthogonal_vectors.append(v)
    
    return np.array(orthogonal_vectors)

# Example vectors
vectors = np.array([[1, 1, 0], [1, 0, 1], [0, 1, 1]])

# Compute orthogonal basis
orthogonal_basis = gram_schmidt(vectors)

print("Original vectors:\n", vectors)
print("Orthogonal basis:\n", orthogonal_basis)

In this example, we define a set of input vectors and apply the Gram–Schmidt process to produce an orthogonal basis. The output is a set of mutually orthogonal vectors.

If Orthogonal Vectors and Orthogonal Bases hasn’t fully clicked yet, revisit this card and walk through its four actionable steps.

When reviewing Orthogonal Vectors and Orthogonal Bases, avoid tackling large projects upfront. Instead, start with a single, simple example to confirm whether the core logic is clear.

Summary

In this tutorial, we explored the significance and foundational concepts of orthogonal vectors and orthogonal bases. Orthogonality plays a vital role across many domains—including signal processing, dimensionality reduction, and machine learning. Identifying or constructing orthogonal bases simplifies computations and improves numerical stability. In the next installment, we’ll examine the practical importance of inner product spaces—with real-world applications and case studies. Stay tuned!