Conditional Probability Mass Functions, introduction to probability lesson

A parcel is selected from a constructed shipping model, and you learn that it took the express route. The question of how likely an inspection is now concerns express parcels only, so the overall inspection rate may no longer be the right answer. This lesson shows how to turn the matching row of a joint table into a complete distribution given that information, how to compute a mean from it, and how to keep conditional, marginal and joint probabilities apart.

Key terms

Conditional probability mass function

Definition Let $X$ and $Y$ be discrete random variables with joint PMF $p_{X,Y}$ , and let $x$ be a value with $p_X(x) > 0$ . The conditional PMF of $Y$ given $X = x$ is

p_{Y \mid X}(y \mid x) = \frac{p_{X,Y}(x, y)}{p_X(x)}

for every real $y$ , which is the conditional probability $P(Y = y \mid X = x)$ . For fixed $x$ it is a PMF in $y$ . Its masses are nonnegative and add to $1$ , since the numerators add to $p_X(x)$ . It is positive exactly at the values $y$ with $p_{X,Y}(x, y) > 0$ , which form the conditional support. If $p_X(x) = 0$ , the conditional PMF given $X = x$ is not defined. The conditional PMF of $X$ given $Y = y$ is defined in the same way when $p_Y(y) > 0$ .

In plain words Learning that $X = x$ keeps only the outcomes in row $x$ of the joint table. The conditional PMF spreads all of the probability over that row, in the same proportions the row already had.

Example In the parcel model, $X = 1$ for express and $X = 0$ for standard routing, and $Y = 1$ for an extra inspection and $Y = 0$ for none. The joint table with its margins is

	$Y = 0$	$Y = 1$	$p_X(x)$
$X = 0$	$0.3$	$0.2$	$0.5$
$X = 1$	$0.1$	$0.4$	$0.5$
$p_Y(y)$	$0.4$	$0.6$	$1$

Given $X = 1$ , the conditional masses are $p_{Y \mid X}(0 \mid 1) = 0.1 / 0.5 = 0.2$ and $p_{Y \mid X}(1 \mid 1) = 0.4 / 0.5 = 0.8$ , which add to $1$ . Among express parcels the inspection probability is $0.8$ , compared with $0.6$ over all parcels.

Renormalizing a row or column

Definition To find the conditional PMF of $Y$ given $X = x$ from a joint table, take row $x$ , check that its total $p_X(x)$ is positive, and divide every entry of the row by that total. To find the conditional PMF of $X$ given $Y = y$ , do the same with column $y$ and its total $p_Y(y)$ . Dividing by one common positive number keeps every ratio between entries of the row or column, keeps zero entries at zero and makes the new total $1$ .

In plain words The selected row or column becomes the whole world. Its entries keep their relative sizes and are rescaled so that they add to one.

Example In the parcel table the row $X = 0$ has entries $0.3$ and $0.2$ with total $0.5$ , so given standard routing, $Y$ equals $0$ or $1$ with probabilities $0.6$ and $0.4$ . The row $X = 1$ has entries $0.1$ and $0.4$ in the ratio $1$ to $4$ , and they become $0.2$ and $0.8$ . So the inspection probability is $0.4$ for standard parcels, $0.8$ for express parcels and $0.6$ overall. The overall value lies between the two conditional values because it is a weighted average of them, but it equals neither. A zero entry stays zero. Toss a fair coin twice, and let $X$ be the number of heads on the first toss and $Y$ the total number of heads. The row $X = 1$ has entries $0$ , $\tfrac14$ and $\tfrac14$ for $Y = 0, 1, 2$ , so given $X = 1$ the total is $1$ or $2$ with probability $\tfrac12$ each, and the conditional support is $\{1, 2\}$ .

Reversed conditioning

Definition Let $p_X(x) > 0$ and $p_Y(y) > 0$ . The probabilities $P(Y = y \mid X = x)$ and $P(X = x \mid Y = y)$ have the same numerator $p_{X,Y}(x, y)$ , but the first divides it by the row total $p_X(x)$ and the second by the column total $p_Y(y)$ . They are linked by Bayes’ theorem,

\begin{aligned} &P(X = x \mid Y = y) \\[0.4em] &\quad = \frac{p_{X,Y}(x, y)}{p_Y(y)} \\[0.4em] &\quad = \frac{p_{Y \mid X}(y \mid x)\, p_X(x)}{p_Y(y)}. \end{aligned}

When $p_{X,Y}(x, y) > 0$ , the two conditional probabilities are equal exactly when $p_X(x) = p_Y(y)$ .

In plain words Read what comes after the bar first. It names the group you are working inside, and that group decides the denominator. Swapping the two sides of the bar asks a different question.

Example In the parcel table, inspection among express parcels uses the row $X = 1$ , which gives $0.4 / 0.5$ . Express routing among inspected parcels uses the column $Y = 1$ , which gives $0.4 / 0.6$ . So

\begin{gathered} P(Y = 1 \mid X = 1) = 0.8, \\[0.4em] P(X = 1 \mid Y = 1) = \tfrac{2}{3}. \end{gathered}

The same cell $0.4$ is divided by two different totals. Renormalizing the whole column $Y = 1$ gives the conditional PMF of $X$ given $Y = 1$ , with mass $\tfrac13$ at $0$ and $\tfrac23$ at $1$ . Bayes’ theorem reaches the reversed value from the conditional probability $0.8$ and the two margins, since $0.8 \cdot 0.5 / 0.6 = \tfrac23$ .

Conditional mean from a conditional PMF

Definition Let $p_X(x) > 0$ . The conditional mean of $Y$ given $X = x$ is the mean of the conditional PMF,

\begin{aligned} &E[Y \mid X = x] \\[0.4em] &\quad = \sum_{y} y \, p_{Y \mid X}(y \mid x), \end{aligned}

where the sum runs over the conditional support. It exists when $\sum_{y} \lvert y \rvert \, p_{Y \mid X}(y \mid x)$ is finite, which always holds when $Y$ has finitely many possible values. It is a number attached to the value $x$ , and it need not be a possible value of $Y$ .

In plain words Find the right conditional distribution first, then average $Y$ with its weights. Averaging with the raw joint masses of the row skips the division by the row total and in general gives a wrong answer.

Example In a constructed timing model, a task belongs to group A, coded $X = 0$ , or to group B, coded $X = 1$ , and it takes $T = 2$ or $T = 6$ minutes.

	$T = 2$	$T = 6$	$p_X(x)$
$X = 0$ , group A	$\tfrac{9}{16}$	$\tfrac{3}{16}$	$\tfrac34$
$X = 1$ , group B	$\tfrac{1}{16}$	$\tfrac{3}{16}$	$\tfrac14$

Dividing each row by its total gives the masses $\tfrac34$ and $\tfrac14$ at the times $2$ and $6$ in group A, and $\tfrac14$ and $\tfrac34$ in group B. Hence $E[T \mid X = 0] = 2 \cdot \tfrac34 + 6 \cdot \tfrac14 = 3$ minutes and $E[T \mid X = 1] = 2 \cdot \tfrac14 + 6 \cdot \tfrac34 = 5$ minutes. Neither mean is a possible duration. Weighting the times by the raw row entries of group A would give $2 \cdot \tfrac{9}{16} + 6 \cdot \tfrac{3}{16} = \tfrac94$ , which is the conditional mean $3$ multiplied by the row total $\tfrac34$ and not the conditional mean itself.

Joint probability from conditional and marginal

Definition For every $x$ with $p_X(x) > 0$ and every real $y$ ,

\begin{aligned} &p_{X,Y}(x, y) \\[0.4em] &\quad = p_X(x)\, p_{Y \mid X}(y \mid x), \end{aligned}

and $p_{X,Y}(x, y) = 0$ for every $y$ when $p_X(x) = 0$ . Adding over $x$ gives the law of total probability for PMFs,

\begin{aligned} &p_Y(y) \\[0.4em] &\quad = \sum_{x} p_X(x)\, p_{Y \mid X}(y \mid x), \end{aligned}

where the sum runs over the values $x$ with $p_X(x) > 0$ . So $p_Y(y)$ is a weighted average of the conditional probabilities $p_{Y \mid X}(y \mid x)$ with weights $p_X(x)$ . A plain average of the conditional probabilities agrees with it when the weights are equal, and in general it does not.

In plain words Conditioning divides a joint mass by a group total. Multiplying a within-group probability by the probability of the group undoes that step and recovers the joint mass, and adding over the groups recovers the overall probability.

Example Change the parcel model so that $P(X = 0) = 0.75$ and $P(X = 1) = 0.25$ , while the inspection probabilities stay $0.4$ for standard and $0.8$ for express parcels. The joint masses with inspection are $0.75 \cdot 0.4 = 0.3$ and $0.25 \cdot 0.8 = 0.2$ , and those without inspection are $0.75 \cdot 0.6 = 0.45$ and $0.25 \cdot 0.2 = 0.05$ , so the four cells add to $1$ . The overall inspection probability is

P(Y = 1) = 0.3 + 0.2 = 0.5,

not the plain average $0.6$ of $0.4$ and $0.8$ . In the original model the two routes were equally likely, which is why the plain average happened to give the right value there. The timing table above was built the same way, since $\tfrac34 \cdot \tfrac34 = \tfrac{9}{16}$ and $\tfrac14 \cdot \tfrac14 = \tfrac{1}{16}$ .

Check yourself

A joint table has the row $(0.1, 0.2)$ for $X = 0$ and the row $(0.3, 0.4)$ for $X = 1$ , with columns for $Y = 0$ and $Y = 1$ . Find $P(Y = 1 \mid X = 1)$ and $P(X = 1 \mid Y = 1)$ , and then the full conditional PMF of $Y$ given $X = 0$ .

Show the answer

The row $X = 1$ has total $0.7$ , so $P(Y = 1 \mid X = 1) = 0.4 / 0.7 = \tfrac47$ . The reversed question uses the column $Y = 1$ , whose total is $0.2 + 0.4 = 0.6$ , so $P(X = 1 \mid Y = 1) = 0.4 / 0.6 = \tfrac23$ . The row $X = 0$ has total $0.3$ , so given $X = 0$ the variable $Y$ equals $0$ with probability $0.1 / 0.3 = \tfrac13$ and $1$ with probability $0.2 / 0.3 = \tfrac23$ . These two masses add to $1$ .

A joint PMF has the following table, and $X$ takes no values other than $0$ and $1$ .

	$Y = 0$	$Y = 1$	$Y = 2$
$X = 0$	$\tfrac18$	$\tfrac18$	$0$
$X = 1$	$\tfrac18$	$\tfrac14$	$\tfrac38$

Find $E[Y \mid X = 0]$ and $E[Y \mid X = 1]$ , and the conditional PMF of $X$ given $Y = 2$ . Can $E[Y \mid X = 2]$ be computed?

Show the answer

The row $X = 0$ has total $\tfrac14$ , so given $X = 0$ the masses at $Y = 0, 1, 2$ are $\tfrac12$ , $\tfrac12$ and $0$ . Only the value $1$ contributes to the mean, so

E[Y \mid X = 0] = 1 \cdot \tfrac12 = \tfrac12.

The row $X = 1$ has total $\tfrac34$ , so the conditional masses are $\tfrac16$ , $\tfrac13$ and $\tfrac12$ , and

\begin{aligned} E[Y \mid X = 1] &= 1 \cdot \tfrac13 + 2 \cdot \tfrac12 \\[0.4em] &= \tfrac43. \end{aligned}

The column $Y = 2$ has total $\tfrac38 > 0$ and entries $0$ and $\tfrac38$ , so given $Y = 2$ the variable $X$ equals $1$ with probability $1$ . The conditional support is $\{1\}$ . The value $X = 2$ has probability $p_X(2) = 0$ , so there is no row to renormalize. The conditional PMF of $Y$ given $X = 2$ is not defined, and neither is $E[Y \mid X = 2]$ , because the formula would divide by zero.

In a constructed model, 20 percent of parts come from line 1, coded $X = 1$ , and the rest from line 0, coded $X = 0$ . A check flags a part, coded $Y = 1$ , for 90 percent of the parts from line 1 and 10 percent of the parts from line 0. Build the joint table, and find $P(Y = 1)$ and $P(X = 1 \mid Y = 1)$ . A classmate answers $0.9$ for the last probability. What went wrong?

Show the answer

Multiplying each line’s probability by the conditional probabilities gives $p_{X,Y}(1, 1) = 0.2 \cdot 0.9 = 0.18$ , $p_{X,Y}(1, 0) = 0.2 \cdot 0.1 = 0.02$ , $p_{X,Y}(0, 1) = 0.8 \cdot 0.1 = 0.08$ and $p_{X,Y}(0, 0) = 0.8 \cdot 0.9 = 0.72$ , which add to $1$ . The column $Y = 1$ gives $P(Y = 1) = 0.18 + 0.08 = 0.26$ , and

\begin{aligned} &P(X = 1 \mid Y = 1) \\[0.4em] &\quad = \frac{0.18}{0.26} = \frac{9}{13} \\[0.4em] &\quad \approx 0.692. \end{aligned}

The value $0.9$ is $P(Y = 1 \mid X = 1)$ , the flag rate within line 1, which divides the cell $0.18$ by the row total $0.2$ instead of the column total $0.26$ . Here reversing the condition changes the denominator and with it the answer. A plain average of the two flag rates, $0.5$ , would also be wrong for $P(Y = 1)$ , because the lines have weights $0.2$ and $0.8$ .

Conditional Probability Mass Functions

Key terms

Conditional probability mass function

Renormalizing a row or column

Reversed conditioning

Conditional mean from a conditional PMF

Joint probability from conditional and marginal

Check yourself

How this lesson connects

Builds on

Used later in