17. Bayesian Updating: Priors and Posteriors

Bayesian updating is not a one-time formula—it is an ongoing process of assimilating evidence. Today’s posterior can serve as tomorrow’s prior.

I record which evidence was used in each update. If the source or reliability of the evidence is unclear, even a mathematically precise posterior remains untrustworthy.

In the previous article, we explored the foundational understanding of Bayes’ Theorem, which provides a framework for adjusting our belief about the probability of an event after observing new evidence. This article delves deeper into the concept of Bayesian updating, and how to reason using prior probabilities and posterior probabilities. Finally, we illustrate these ideas through concrete examples.

1. Bayesian Updating and Core Concepts

Bayesian updating refers to the process of revising our beliefs—represented by a prior probability—in light of newly observed data, resulting in a posterior probability. Its essence lies in how we integrate new information into our existing knowledge.

When performing Bayesian updating, first write down the prior probability, likelihood of the evidence, marginal probability, and posterior result—and then explain what changed in your belief due to the new evidence.

• Prior Probability: Your initial belief about an event before seeing new data.
• Posterior Probability: Your updated belief about the event after incorporating the new data.

Bayes’ Formula

The core of Bayesian updating is Bayes’ Theorem, expressed mathematically as:

$$P(H|E) = \frac{P(E|H) \cdot P(H)}{P(E)}$$

Here,

• $P(H|E)$ is the posterior probability: the probability that hypothesis $H$ is true, given evidence $E$.
• $P(E|H)$ is the likelihood: the probability of observing evidence $E$, assuming hypothesis $H$ is true.
• $P(H)$ is the prior probability: your belief in $H$ before observing $E$.
• $P(E)$ is the marginal likelihood: the total probability of observing $E$ across all possible hypotheses (i.e., the weighted average of $P(E|H)$ and $P(E|\neg H)$).

2. Case Study: COVID-19 Testing

Let’s ground Bayesian updating in a concrete scenario: a diagnostic test for SARS-CoV-2.

Before reading “Bayesian Updating, Priors, and Posteriors”, align the questions, keywords, actions, and acceptance criteria shown in the diagram—this makes reading the main text significantly more efficient. After reading, try re-explaining the concepts using your own project as an example.

Assumptions

• Prior probability of disease: The baseline probability that a randomly selected person is infected before testing is $P(H) = 0.01$ (i.e., a 1% prevalence in the population).
• Test accuracy:
  • True positive rate (sensitivity): If infected, the test returns positive with probability $P(E|H) = 0.9$.
  • False positive rate: If not infected, the test still returns positive with probability $P(E|\neg H) = 0.05$.

Computing the Posterior Probability

We want to compute $P(H|E)$: the probability that a person is truly infected given a positive test result.

1. Compute $P(E)$ — the overall probability of a positive test result:

   $$P(E) = P(E|H) \cdot P(H) + P(E|\neg H) \cdot P(\neg H)$$

   where $P(\neg H) = 1 - P(H) = 0.99$. Substituting values:

   $$P(E) = 0.9 \cdot 0.01 + 0.05 \cdot 0.99 = 0.009 + 0.0495 = 0.0585$$

2. Compute the posterior:

   $$P(H|E) = \frac{P(E|H) \cdot P(H)}{P(E)} = \frac{0.9 \cdot 0.01}{0.0585} \approx 0.1538$$

Thus, despite a positive test result, the actual probability that the person is infected is only ~15.38%. This highlights a crucial insight: test results alone cannot definitively confirm infection—especially when the prior probability (baseline prevalence) is low.

When reviewing “Bayesian Updating, Priors, and Posteriors”, place key concepts, procedural steps, and observable outcomes on the same page for efficient revision.

When practicing “Bayesian Updating, Priors, and Posteriors”, write input conditions, computational steps, and resulting outputs together—making future review straightforward.

3. Summary

Through Bayesian updating, we see how to begin with a prior probability, incorporate new evidence (e.g., a test result), apply Bayes’ Theorem, and arrive at a posterior probability. In practice, grasping the relationship between priors and posteriors is essential for sound inference.

The next article will explore real-world case studies, demonstrating how Bayesian reasoning supports practical data analysis. Stay tuned!