Set seed for reproducibility

Conditional probability is not merely ordinary probability with an extra symbol—it redefines the scope of discussion given that a particular event has occurred.

First ask: Does condition B alter the sample space? If it does, the denominator can no longer be the original total count.

Building upon our understanding of events and sample spaces, we now delve into two pivotal concepts in probability theory: conditional probability and independence. These ideas play indispensable roles in machine learning and data science—mastering them is therefore essential for deeper study and practical application.

Conditional Probability

Definition

When learning conditional probability and independence, always begin by asking whether the new condition changes how we assess events. If the condition alters the probability, the events cannot simply be treated as mutually unaffected.

Conditional probability refers to the probability of one event occurring given that another event has already occurred. Mathematically, for events $A$ and $B$, the conditional probability $P(A|B)$ denotes the probability that event $A$ occurs under the condition that event $B$ has occurred.

Formula

The formula for computing conditional probability is:

$$P(A|B) = \frac{P(A \cap B)}{P(B)}$$

where $P(A \cap B)$ is the joint probability that both events $A$ and $B$ occur, and $P(B)$ is the probability that event $B$ occurs.

Example

Consider a simple example: drawing a single card from a standard 52-card deck. Suppose we want to compute the conditional probability of drawing a red card (event $A$), given that the drawn card is a heart (event $B$).

1. Event $A$: Drawing a red card.
2. Event $B$: Drawing a heart.

In a standard deck, red cards consist of hearts (13 cards) and diamonds (13 cards). Since all hearts are red, $P(A \cap B) = P(\text{drawing a heart}) = \frac{13}{52}$, and $P(B) = \frac{13}{52}$. Therefore:

$$P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{13/52}{13/52} = 1$$

This result indicates that, given the card drawn is a heart, the probability it is red is 1 (i.e., certain).

Code Example

We can simulate this scenario using Python:

import numpy as np

# Set seed for reproducibility
np.random.seed(42)

# Define suits
cards = ['♠', '♣', '♦', '♥']
total_cards = 52
red_cards = ['♦', '♥']

# Draw one card
drawn_card = np.random.choice(cards, p=[1/4]*4)
is_red = 1 if drawn_card in red_cards else 0

print(f"Drawn card: {drawn_card}, Is red: {bool(is_red)}")

Independence

Definition

The article “Foundational Concepts in Probability: Conditional Probability and Independence” can be read through four lenses: scenario, concept, action, and outcome. First align these four elements; then revisit the parameters, code, or workflows described in the main text.

Two events are independent if the occurrence of one provides no information about the occurrence of the other. Formally, events $A$ and $B$ are independent if and only if:

$$P(A \cap B) = P(A) \cdot P(B)$$

Example

Suppose we have two completely independent experiments:

1. Flipping a fair coin: event $A$ = “getting heads”.
2. Rolling a fair six-sided die: event $B$ = “rolling a 4”.

We know:

• Probability of heads: $P(A) = \frac{1}{2}$.
• Probability of rolling a 4: $P(B) = \frac{1}{6}$.

Then the joint probability $P(A \cap B)$—i.e., getting heads and rolling a 4—is:

$$P(A \cap B) = P(A) \cdot P(B) = \frac{1}{2} \cdot \frac{1}{6} = \frac{1}{12}$$

Code Example

Here’s a simple Python simulation illustrating independence:

import numpy as np

# Sampling functions for coin flip and die roll
def flip_coin():
    return np.random.choice(['Heads', 'Tails'], p=[0.5, 0.5])

def roll_die():
    return np.random.randint(1, 7)  # integers 1 through 6

# Run many trials
trials = 10000
independent_events = [(flip_coin(), roll_die()) for _ in range(trials)]

# Count occurrences of (Heads, 4)
count = sum(1 for outcome in independent_events if outcome[0] == 'Heads' and outcome[1] == 4)
probability = count / trials

print(f"After {trials} trials, the empirical probability of 'Heads' and '4' occurring together is approximately: {probability:.4f}")

If you haven’t fully internalized “Foundational Concepts in Probability: Conditional Probability and Independence”, revisit this card and walk through its four key actions step-by-step.

When reviewing “Foundational Concepts in Probability: Conditional Probability and Independence”, avoid jumping straight into large-scale projects. Instead, first verify conceptual clarity using a single, straightforward example.

Summary

After practicing the definitions, formulas, and computations related to conditional probability and independence, you should now possess a deeper, more intuitive grasp of these core probabilistic concepts. They are not only theoretically fundamental but also frequently encountered in real-world applications across data science and artificial intelligence. Next, we will explore random variables and their distributions in greater depth.