Joint Probability Mass Functions, introduction to probability lesson

A constructed shipping model records two things about each parcel, whether it travels by the standard or the express route and whether it receives an extra inspection. Knowing how often parcels go express and how often they are inspected does not in general tell you how often both happen to the same parcel. This lesson builds a table that gives a probability to every pair of recorded values, shows how to check that such a table is valid, and answers questions about both quantities by adding the right cells.

Key terms

Random vector

Definition Let $X$ and $Y$ be random variables defined on the same sample space $\Omega$ . The random vector $(X, Y)$ assigns to each outcome $\omega$ in $\Omega$ the ordered pair $(X(\omega), Y(\omega))$ , so a single run of the experiment produces a value of each variable. It is a discrete random vector when $X$ and $Y$ are both discrete, and then with probability $1$ its value is one of finitely or countably many pairs. In the same way, $n$ random variables on one sample space form a random vector $(X_1, \dots, X_n)$ . The order of the coordinates is part of the definition.

In plain words One trial produces several recorded numbers, and the random vector keeps them together in a fixed order, so you always know which number belongs to which variable.

Example In the parcel model, let $X = 0$ for the standard route and $X = 1$ for express, and let $Y = 0$ when the parcel gets no extra inspection and $Y = 1$ when it does. The random vector $(X, Y)$ takes one of the four pairs $(0, 0)$ , $(0, 1)$ , $(1, 0)$ and $(1, 1)$ . The pair $(0, 1)$ means the standard route with inspection, while $(1, 0)$ means express without inspection. The numbers $0$ and $1$ are codes for categories, so every question about a parcel has to be translated into a statement about these codes.

Joint probability mass function

Definition Let $X$ and $Y$ be discrete random variables on the same sample space. Their joint probability mass function, or joint PMF, is the function defined for all real numbers $x$ and $y$ by

\begin{aligned} &p_{X,Y}(x, y) \\[0.4em] &\quad = P(X = x, Y = y), \end{aligned}

where the comma means “and”, so the right side is the probability of the event $\{X = x\} \cap \{Y = y\}$ . Every joint PMF has three properties. Each mass $p_{X,Y}(x, y)$ is nonnegative, only finitely or countably many pairs have positive mass, and the positive masses add to one,

\sum_{x} \sum_{y} p_{X,Y}(x, y) = 1,

where the sums run over the pairs with positive mass. Conversely, any function of two variables with these three properties is the joint PMF of some discrete random vector.

In plain words For any two numbers you name, the joint PMF reports how likely it is that $X$ equals the first and $Y$ equals the second in the same trial. For a finite list of pairs, nonnegative masses and a total of one are the two checks to make.

Example The parcel model assigns the masses $p_{X,Y}(0, 0) = 0.3$ , $p_{X,Y}(0, 1) = 0.2$ , $p_{X,Y}(1, 0) = 0.1$ and $p_{X,Y}(1, 1) = 0.4$ . All four are nonnegative, and $0.3 + 0.2 + 0.1 + 0.4 = 1$ . Every other pair of numbers, such as $(2, 0)$ or $(0.5, 1)$ , has joint mass $0$ . The mass $p_{X,Y}(0, 1) = 0.2$ is the probability of standard routing together with inspection, which is a different event from express routing without inspection, whose mass is $0.1$ .

Joint probability table

Definition When $X$ and $Y$ have finitely many possible values, their joint PMF can be written as a joint probability table with one row for each possible value $x$ of $X$ and one column for each possible value $y$ of $Y$ . The cell in row $x$ and column $y$ holds $p_{X,Y}(x, y)$ , the probability that the row condition and the column condition hold in the same trial. Every outcome produces exactly one pair, so the cells stand for disjoint events, and together these events have probability $1$ . A table whose rows and columns list all possible values is therefore a valid joint PMF exactly when no entry is negative and the entries add to $1$ .

In plain words The table is the joint PMF laid out as a grid. Say which variable labels the rows, check the entries, and read each cell as a statement about both variables at once.

Example With $X$ on the rows and $Y$ on the columns, the parcel model is

	$Y = 0$	$Y = 1$
$X = 0$	$0.3$	$0.2$
$X = 1$	$0.1$	$0.4$

A second proposal with rows $(0.2, 0.3)$ and $(0.2, 0.3)$ also passes both checks, so it is a valid joint PMF as well. It describes a different model, in which express routing with inspection has probability $0.3$ instead of $0.4$ . A third proposal with rows $(0.3, 0.2)$ and $(0.1, 0.5)$ has total $1.1$ , so it is not a joint PMF on these four pairs. Dividing every entry by $1.1$ would give a valid table, but of a different model, and it would hide the mistake that produced the extra $0.1$ . A table with an entry such as $-0.1$ fails even when its total is $1$ .

Joint support

Definition The joint support of a discrete random vector $(X, Y)$ is the set of pairs $(x, y)$ with $p_{X,Y}(x, y) > 0$ . If a pair lies in the joint support, then $x$ is a possible value of $X$ and $y$ is a possible value of $Y$ , because $P(X = x) \ge P(X = x, Y = y) > 0$ , and likewise for $Y$ . The converse fails in general. A possible value of $X$ and a possible value of $Y$ can form a pair of joint mass $0$ , so the joint support can be a proper subset of the set of all such pairs.

In plain words The joint support lists the pairs that can actually happen together. Two values that are each possible on their own can still be impossible in combination, and the table shows this with a zero cell.

Example Toss a fair coin twice. Let $X$ be the number of heads on the first toss and $Y$ the total number of heads. The four equally likely outcomes HH, HT, TH and TT give the pairs $(1, 2)$ , $(1, 1)$ , $(0, 1)$ and $(0, 0)$ .

	$Y = 0$	$Y = 1$	$Y = 2$
$X = 0$	$\tfrac14$	$\tfrac14$	$0$
$X = 1$	$0$	$\tfrac14$	$\tfrac14$

The joint support has four pairs, although $X$ has two possible values and $Y$ has three. The pair $(1, 0)$ is impossible, since a head on the first toss means at least one head in total, and $(0, 2)$ is impossible, since a tail on the first toss allows at most one head. Adding cells gives $P(X = 1) = \tfrac12$ and $P(Y = 0) = \tfrac14$ , yet $P(X = 1, Y = 0) = 0$ and not the product $\tfrac18$ . Multiplying separate probabilities needs independence, and these two variables are not independent.

Event region

Definition Let $(X, Y)$ be a discrete random vector and let $B$ be a set of pairs of real numbers. The event $\{(X, Y) \in B\}$ has probability

\begin{aligned} &P\big((X, Y) \in B\big) \\[0.4em] &\quad = \sum_{(x, y) \in B} p_{X,Y}(x, y), \end{aligned}

where only pairs of the joint support contribute. The events $\{X = x, Y = y\}$ for different pairs are disjoint, so their probabilities add. The set $B$ is the event region, and in a table it is a set of cells. If $B$ contains no pair of the joint support, every term of the sum is $0$ , and so is the probability.

In plain words Translate the question into a set of cells, mark them, and add their entries. Marking the region first keeps you from counting a cell twice or leaving one out. Two different regions can have the same probability without being the same event.

Example In the parcel table, matching codes means the region $\{(0, 0), (1, 1)\}$ , so $P(X = Y) = 0.3 + 0.4 = 0.7$ . Express routing or inspection, including both, means every cell except $(0, 0)$ , so its probability is $0.2 + 0.1 + 0.4 = 0.7$ , which is also $1 - 0.3$ . The two answers agree, but the events differ, since the cell $(0, 1)$ lies in the second region and not in the first. The region for both conditions is the single cell $(1, 1)$ , with probability $0.4$ . The region for exactly one condition is $\{(0, 1), (1, 0)\}$ , with probability $0.3$ , and the region for neither is the cell $(0, 0)$ , with probability $0.3$ . These three regions split the table, and $0.4 + 0.3 + 0.3 = 1$ . The union also agrees with inclusion-exclusion. The bottom row gives $P(X = 1) = 0.5$ and the right column gives $P(Y = 1) = 0.6$ , and $0.5 + 0.6 - 0.4 = 0.7$ .

Function of a random pair

Definition Let $(X, Y)$ be a discrete random vector and let $g$ be a real-valued function defined at every pair that $(X, Y)$ can take. Then $W = g(X, Y)$ is a discrete random variable on the same sample space, and its PMF is

p_W(w) = \sum_{g(x, y) = w} p_{X,Y}(x, y),

where the sum runs over the pairs of the joint support with $g(x, y) = w$ . Each cell passes its whole mass to the value it produces, and cells that produce the same value are added. The values of $W$ need not be equally likely, and a value produced by more cells is not automatically more likely.

In plain words Write the new value in each cell, collect the cells by value, and add their probabilities. The answer depends on the masses of the cells, not on how many cells there are.

Example In the parcel model let $W = X + Y$ , the number of the two conditions, express routing and inspection, that hold for the parcel. The value $0$ comes only from $(0, 0)$ , the value $1$ from $(0, 1)$ and $(1, 0)$ , and the value $2$ only from $(1, 1)$ .

$w$	$0$	$1$	$2$
$p_W(w)$	$0.3$	$0.3$	$0.4$

The masses add to $1$ . The value $1$ collects two cells, yet the value $2$ is more likely. Another function is the maximum $M = \max(X, Y)$ , which equals $1$ exactly on the region for express routing or inspection, so $P(M = 1) = 0.7$ and $P(M = 0) = 0.3$ .

Check yourself

A joint table for two codes $X$ and $Y$ , each $0$ or $1$ , has the row $(0.1, 0.2)$ for $X = 0$ and the row $(0.3, 0.4)$ for $X = 1$ , with columns for $Y = 0$ and $Y = 1$ . Find the probability that the codes differ and the PMF of $S = X + Y$ . Is a sum of $1$ more likely than a sum of $2$ ?

Show the answer

The table is valid, since its entries are nonnegative and add to $1$ . The codes differ on the cells $(0, 1)$ and $(1, 0)$ , so $P(X \ne Y) = 0.2 + 0.3 = 0.5$ . The sum is $0$ only at $(0, 0)$ , it is $1$ at $(0, 1)$ and $(1, 0)$ , and it is $2$ only at $(1, 1)$ . So $p_S(0) = 0.1$ , $p_S(1) = 0.5$ and $p_S(2) = 0.4$ , which add to $1$ . A sum of $1$ is more likely, because $0.5 > 0.4$ . The comparison rests on the added masses, not on the fact that two cells produce the value $1$ .

Let $p(x, y) = c(x + y + 1)$ for $x \in \{0, 1\}$ and $y \in \{0, 1, 2\}$ , and $p(x, y) = 0$ for every other pair. Find the constant $c$ that makes $p$ a joint PMF, and then find $P(X < Y)$ .

Show the answer

The values of $x + y + 1$ are $1$ , $2$ and $3$ in the row $x = 0$ and $2$ , $3$ and $4$ in the row $x = 1$ , with total $15$ . Normalization gives $15c = 1$ , so $c = \tfrac{1}{15}$ , and all six masses are positive. The event $X < Y$ is the region $\{(0, 1), (0, 2), (1, 2)\}$ , so

\begin{aligned} P(X < Y) &= \frac{2 + 3 + 4}{15} \\[0.4em] &= \frac{9}{15} = \frac{3}{5}. \end{aligned}

Two tickets are drawn without replacement from a box with tickets labeled $1$ , $2$ and $3$ , and all six ordered pairs of different labels are equally likely. Let $X$ be the first label and $Y$ the second. Describe the joint support, find $P(X = 2, Y = 2)$ and $P(X < Y)$ , and explain what is wrong with computing the first probability as $P(X = 2)\,P(Y = 2)$ .

Show the answer

The joint support is the set of the six pairs $(x, y)$ with $x$ and $y$ in $\{1, 2, 3\}$ and $x \ne y$ , each with mass $\tfrac16$ . The pairs $(1, 1)$ , $(2, 2)$ and $(3, 3)$ have mass $0$ , so $P(X = 2, Y = 2) = 0$ . The event $X < Y$ is the region $\{(1, 2), (1, 3), (2, 3)\}$ , so $P(X < Y) = \tfrac36 = \tfrac12$ .

Adding cells gives $P(X = 2) = \tfrac26 = \tfrac13$ , and likewise $P(Y = 2) = \tfrac13$ from the cells $(1, 2)$ and $(3, 2)$ . Using their product $\tfrac19$ as the joint mass would assume that the two draws are independent, and they are not. Drawing ticket $2$ first removes it from the box, so the second label cannot also be $2$ . The joint table gives the correct answer $0$ , and the product of the separate probabilities does not.

Joint Probability Mass Functions

Key terms

Random vector

Joint probability mass function

Joint probability table

Joint support

Event region

Function of a random pair

Check yourself

How this lesson connects

Builds on

Used later in