Set random seed for reproducibility

The Law of Large Numbers tells us that, given a sufficiently large number of repetitions, the sample average will gradually converge toward the theoretical expectation—but it does not guarantee stability in short-term outcomes.

I distinguish between long-term patterns and short-term fluctuations. With small samples, avoid mistaking random variation for genuine underlying patterns.

In the previous article, we explored key concepts in probability theory: expected value, variance, covariance, and correlation. The next topic—central to probability theory—is the Law of Large Numbers, a foundational theorem with both theoretical significance and rich practical implications.

Concept of the Law of Large Numbers

The Law of Large Numbers states that, when conducting a large number of independent and identically distributed (i.i.d.) random experiments, the sample mean converges toward the population’s expected value. Put simply: as the sample size increases, the average of the observed data increasingly approximates the true expected value.

When learning the Law of Large Numbers, first clarify these core elements: independent repeated trials, sample mean, expected value, sample size, and the meaning of “convergence.”

Formal Definition

Let $X_1, X_2, \ldots, X_n$ be a sequence of independent and identically distributed random variables, each with finite expected value $E(X)$. According to the Law of Large Numbers, as the sample size $n \to \infty$, the sample mean $\bar{X}_n$ converges almost surely to $E(X)$. Mathematically:

$$\bar{X}_n = \frac{1}{n} \sum_{i=1}^{n} X_i \quad \text{and} \quad \lim_{n \to \infty} \bar{X}_n = E(X) \quad \text{almost surely.}$$

Example: Rolling a Fair Die

Consider a simple experiment: rolling a fair six-sided die. Each outcome $X_i \in \{1,2,3,4,5,6\}$ occurs with equal probability $\frac{1}{6}$. The expected value is:

$$E(X) = \frac{1+2+3+4+5+6}{6} = 3.5$$

By the Law of Large Numbers, if we roll the die many times and compute the running sample mean $\bar{X}_n$, this average will approach 3.5 as the number of rolls grows.

Practical Code Example

We can simulate this process using Python:

import numpy as np
import matplotlib.pyplot as plt

# Set random seed for reproducibility
np.random.seed(42)

# Number of die rolls
n = 1000
# Simulate n independent die rolls
dice_rolls = np.random.randint(1, 7, size=n)

# Compute cumulative sample means
sample_means = np.cumsum(dice_rolls) / np.arange(1, n + 1)

# Plot convergence
plt.figure(figsize=(10, 5))
plt.plot(sample_means, label='Sample Mean')
plt.axhline(3.5, color='red', linestyle='--', label='Expected Value (3.5)')
plt.xlabel('Number of Trials')
plt.ylabel('Sample Mean')
plt.title('Convergence of Sample Mean to Expected Value')
plt.legend()
plt.grid()
plt.show()

In this code, we simulate 1000 die rolls and compute the cumulative sample mean after each trial. The resulting plot visually demonstrates how the sample mean progressively approaches 3.5 as the number of trials increases.

After studying “Illustrating the Law of Large Numbers”, try adapting it to a scenario of your own—pay special attention to whether inputs, processing steps, and outputs align coherently.

To apply “Illustrating the Law of Large Numbers” to your own task, start by narrowing the scope: test just one critical decision point first.

Key Takeaways

• The Law of Large Numbers describes how, under repeated i.i.d. sampling, the sample mean converges to the population’s expected value.
• In practice, this principle helps us interpret data stability and generalizability—and remains one of the cornerstones of foundational statistics.

Before reading “Illustrating the Law of Large Numbers”, use the accompanying diagram to confirm the main thread; afterward, revisit it to identify which steps you can execute directly—and which require supplemental background material.

In the next article, we’ll focus on the Central Limit Theorem, exploring its relationship with the Law of Large Numbers and its real-world applications—deepening our understanding of probability theory. Stay tuned!