Conditional Probability, introduction to probability lesson

A bag holds 20 equally likely tokens, 12 red and 8 blue, and four of the red tokens and two of the blue ones are striped. If you are told only that the drawn token is striped, what is the probability that it is red? The news does not change the token, but it rules out the 14 tokens without stripes, so the question is now asked inside a smaller group. This lesson makes that idea precise, shows why the words after “given” decide the denominator, and explains why the condition must have positive probability.

Key terms

Conditional probability

Definition Let $A$ and $B$ be events in the same probability model, with $P(B) > 0$ . The conditional probability of $A$ given $B$ is

P(A \mid B) = \frac{P(A \cap B)}{P(B)}.

The vertical bar is read “given.” It is not a division sign and not an absolute value. When the sample space $\Omega$ is finite and all of its outcomes are equally likely, the common factor $1/\lvert \Omega \rvert$ cancels and the definition becomes a ratio of counts,

P(A \mid B) = \frac{\lvert A \cap B \rvert}{\lvert B \rvert}.

In plain words Keep only the outcomes in $B$ , then ask what share of their total probability also lies in $A$ . With equally likely outcomes, that share is the fraction of the outcomes in $B$ that also lie in $A$ .

Example Let $R$ be the event that the drawn token is red and $S$ the event that it is striped. The complete table of counts for the bag is

Color	Striped $S$	Plain $S^{c}$	Total
Red $R$	4	8	12
Blue $R^{c}$	2	6	8
Total	6	14	20

The intersection $R \cap S$ holds 4 tokens, so $P(R \cap S) = 4/20 = 1/5$ , and $P(S) = 6/20$ . Then

\begin{aligned} P(R \mid S) &= \frac{4/20}{6/20} \\[0.4em] &= \frac{4}{6} = \frac{2}{3}. \end{aligned}

Both numbers count the same 4 tokens. The probability $P(R \cap S)$ measures “red and striped” against all 20 tokens, while $P(R \mid S)$ measures “red, given striped” against the 6 striped tokens. Because $P(B) \le 1$ , the conditional probability $P(A \mid B)$ is never smaller than $P(A \cap B)$ .

Conditioning event

Definition In $P(A \mid B)$ , the event $B$ written after the bar is the conditioning event. It stands for the information that the outcome of the experiment lies in $B$ . The experiment and its outcome stay the same, and $A$ and $B$ are events in the same sample space. Conditioning describes what is known about the outcome. It says nothing about what would happen if the outcome were changed.

In plain words The conditioning event is the fact you have been told. It does not redraw the token or alter it. It only narrows down where the outcome can be.

Example Before any information about stripes, $P(R) = 12/20 = 3/5$ . Learning that the token is striped raises this to $P(R \mid S) = 2/3$ , and learning that it is plain lowers it to

P(R \mid S^{c}) = \frac{8}{14} = \frac{4}{7}.

Both changes come from the make-up of the striped and plain groups in this bag. They do not mean that painting stripes on a blue token would turn it red. A claim about what an intervention would do needs a model of that intervention, and a conditional probability alone does not supply one.

Reference group

Definition Given an event $B$ with $P(B) > 0$ , the reference group is the set of outcomes in $B$ , treated as the new whole. Every event $E$ inside $B$ gets the conditional probability $P(E)/P(B)$ , and every event disjoint from $B$ gets $0$ . So the outcomes of $B$ keep their relative weights and are rescaled to total $1$ . As a function of the event $A$ , the rule $A \mapsto P(A \mid B)$ satisfies the probability axioms. In particular $P(B \mid B) = 1$ , and the complement rule holds inside the group,

P(A^{c} \mid B) = 1 - P(A \mid B).

In plain words The condition becomes the new sample space. Probabilities inside it are multiplied by the same factor $1/P(B)$ so that they add to $1$ , and the usual rules hold inside it. Counting outcomes is justified when the group has finitely many outcomes, all equally likely. Otherwise divide probabilities, not counts.

Example Given that the token is plain, the reference group is the 14 plain tokens. Inside it $P(R \mid S^{c}) = 8/14$ and $P(R^{c} \mid S^{c}) = 6/14$ , which add to $1$ . Now take a die weighted so that a 6 has probability $1/2$ and each other face has probability $1/10$ , and let $E$ be the event that the roll is even. Then $P(E) = 1/10 + 1/10 + 1/2 = 7/10$ , and

P(\{6\} \mid E) = \frac{1/2}{7/10} = \frac{5}{7},

not the $1/3$ that counting the three even faces would suggest.

Reversed condition

Definition The reversed condition of $P(A \mid B)$ is $P(B \mid A)$ , with the roles of the two events swapped. When $P(A) > 0$ and $P(B) > 0$ , the two share the numerator $P(A \cap B)$ , but the first divides by $P(B)$ and the second by $P(A)$ ,

\begin{gathered} P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \\[0.4em] P(B \mid A) = \frac{P(A \cap B)}{P(A)}. \end{gathered}

They are equal exactly when $P(A) = P(B)$ or $P(A \cap B) = 0$ . Outside these cases, reversing the condition changes the answer.

In plain words Find the words after “given” first, since they name the group you divide by. Swapping the condition swaps the group, even when the favorable outcomes stay the same.

Example In the bag, $P(R \mid S) = 4/6 = 2/3$ . The reversed question asks how many of the 12 red tokens are striped,

P(S \mid R) = \frac{4}{12} = \frac{1}{3}.

The same 4 tokens are counted, first out of 6 and then out of 12. The same care applies to everyday statements. The probability of a fault given an alarm need not be close to the probability of an alarm given a fault, because the two divide by different groups.

Positive-probability requirement

Definition The ratio $P(A \cap B)/P(B)$ defines $P(A \mid B)$ only when $P(B) > 0$ . If $P(B) = 0$ , then $P(A \cap B) = 0$ as well, the ratio is $0/0$ , and the formula assigns no value. The conditional probability is then undefined by this definition, which is different from being $0$ . In a finite model where every outcome has positive probability, $P(B) = 0$ happens only for $B = \varnothing$ . In continuous models, events of probability $0$ can occur, and conditioning on them needs a separate construction such as conditional densities.

In plain words A group with no probability cannot be rescaled to total $1$ . Check that the condition has positive probability before you divide.

Example Ask for the probability that the token is red given that it is green. The bag has no green tokens, so the event $G$ of drawing a green token is empty and $P(G) = 0$ . The expression $P(R \cap G)/P(G)$ is $0/0$ , so $P(R \mid G)$ is undefined, and writing $0$ or $1$ as the answer would be wrong. By contrast, every event $B$ with $P(B) > 0$ has $P(B \mid B) = 1$ .

Check yourself

A second bag holds 18 equally likely tokens. Of the 10 square tokens, 3 are dotted and 7 are plain, and of the 8 circular tokens, 5 are dotted and 3 are plain. Let $Q$ be the event that the token is square and $D$ the event that it is dotted. Name each reference group, then find $P(Q \mid D)$ and $P(D \mid Q)$ .

Show the answer

For $P(Q \mid D)$ the reference group is the $3 + 5 = 8$ dotted tokens, and 3 of them are square, so $P(Q \mid D) = 3/8$ . For $P(D \mid Q)$ the reference group is the 10 square tokens, and 3 of them are dotted, so $P(D \mid Q) = 3/10$ .

A classmate answers $3/18 = 1/6$ for the probability that a token in the second bag is square given that it is dotted. Which probability did the classmate compute, and how does the definition repair the answer?

Show the answer

The fraction $3/18$ is $P(Q \cap D)$ , the probability that the token is square and dotted, measured against all 18 tokens. The condition “dotted” makes the 8 dotted tokens the reference group, so the definition divides by $P(D) = 8/18$ ,

P(Q \mid D) = \frac{3/18}{8/18} = \frac{3}{8}.

In the second bag, find the probability that a token is circular given that it is plain, and check it with the plain square tokens. Then explain why the probability that a token is dotted given that it is triangular has no value here.

Show the answer

The plain group has $7 + 3 = 10$ tokens, and 3 of them are circular, so $P(Q^{c} \mid D^{c}) = 3/10$ . The other 7 plain tokens are square, giving $P(Q \mid D^{c}) = 7/10$ , and the two answers add to $1$ because square and circular split the plain group. The value $3/10$ matches $P(D \mid Q)$ from question 1 only by coincidence, since it counts different tokens in a different group.

No token is triangular, so that conditioning event has probability $0$ and the ratio would be $0/0$ . The conditional probability is undefined, not $0$ .

Conditional Probability

Key terms

Conditional probability

Conditioning event

Reference group

Reversed condition

Positive-probability requirement

Check yourself

How this lesson connects

Builds on

Used later in