Probability Mass Functions, introduction to probability lesson

The rule of a random variable tells you which number to record, but the rule alone does not say how likely each number is. Suppose a small editing task needs $0$ , $1$ or $3$ corrections. Before you can answer a question such as how likely at least one correction is, you need the probability that belongs to each of those values. This lesson introduces the function that holds that information, shows how to check that a proposed table of probabilities is valid, and turns any question about the variable into a sum of the right masses.

Key terms

Probability mass function

Definition Let $X$ be a discrete random variable. Its probability mass function, or PMF, is the function $p_X$ defined for every real number $x$ by

p_X(x) = P(X = x).

It is positive exactly at the values in the support of $X$ and equal to $0$ at every other real number. The input $x$ stands for a value of $X$ and can be any real number, and the output $p_X(x)$ is a probability.

In plain words For any number you name, the PMF reports how likely the random variable is to equal exactly that number. Values and masses are different kinds of numbers, which is why a PMF table keeps them in separate rows.

Example In a constructed teaching model, let $X$ be the number of corrections an editing task needs, with the following PMF.

$x$	$0$	$1$	$3$
$p_X(x)$	$\tfrac{1}{5}$	$\tfrac{1}{2}$	$\tfrac{3}{10}$

So $p_X(1) = \tfrac{1}{2}$ , and the value $3$ is a number of corrections whose probability is $\tfrac{3}{10}$ . The value $2$ is not in the support, so $p_X(2) = 0$ , and likewise $p_X(-1) = 0$ and $p_X(2.5) = 0$ . The table assigns probability to three values and zero to every other real number.

Nonnegative masses

Definition A function $p$ on the real numbers is the PMF of some discrete random variable exactly when three conditions hold. Every mass is nonnegative, so $p(x) \ge 0$ for all real $x$ . The set of values with $p(x) > 0$ is finite or countably infinite. The masses on that set add to one,

\sum_{x \,:\, p(x) > 0} p(x) = 1.

The last condition is called normalization. Together the conditions give $p(x) \le 1$ for every $x$ , since one nonnegative term cannot exceed a sum of nonnegative terms that equals $1$ .

In plain words No probability can be negative, and the masses must account for all of the probability, no more and no less. The nonnegativity check and the total check each catch mistakes that the other one misses.

Example The correction table passes both checks. All three masses are positive, and written in tenths they add to one,

\tfrac{2}{10} + \tfrac{5}{10} + \tfrac{3}{10} = 1.

A proposed table with masses $0.4$ , $0.7$ and $-0.1$ sums to $1$ but has a negative mass, so it is not a PMF. A table with masses $0.2$ , $0.5$ and $0.2$ has no negative entry but sums to $0.9$ . If its three values are claimed to be all the possible values, $0.1$ of the probability is unaccounted for and the table is not a PMF. A table that openly lists only some of the values may total less than $1$ , but it is then a partial report, not a complete distribution.

Normalizing constant

Definition Sometimes masses are known only up to a constant factor, $p(x) = c\,g(x)$ for $x$ in a stated finite or countably infinite set $S$ and $p(x) = 0$ otherwise. If $g(x) \ge 0$ on $S$ and the sum of $g$ over $S$ is finite and positive, normalization forces

c = \frac{1}{\sum_{x \in S} g(x)},

and this value is the normalizing constant. It is positive, so every mass $c\,g(x)$ is then nonnegative. When the proposed formula can take negative values, solving the normalization equation is not enough, and every resulting mass still has to be checked.

In plain words Write the masses with an unknown scale, add them, and choose the scale that makes the total $1$ . Then confirm that no mass came out negative.

Example A variable takes the values $-1$ , $0$ and $2$ with masses $c$ , $2c$ and $3c$ . Normalization gives $c + 2c + 3c = 6c = 1$ , so $c = \tfrac{1}{6}$ , and the masses are $\tfrac{1}{6}$ , $\tfrac{1}{3}$ and $\tfrac{1}{2}$ , all nonnegative with total $1$ . The same method works on a countably infinite support. For masses $c\left(\tfrac{1}{2}\right)^{k}$ at $k = 0, 1, 2, \dots$ the geometric series gives

\sum_{k=0}^{\infty} c\left(\tfrac{1}{2}\right)^{k} = \frac{c}{1 - \tfrac{1}{2}} = 2c,

so $c = \tfrac{1}{2}$ and the masses are $\tfrac{1}{2}$ , $\tfrac{1}{4}$ , $\tfrac{1}{8}$ and so on. Now try $p(x) = c(x - 1)$ for $x = 0, 1, 2, 3$ . Normalization gives $c(-1 + 0 + 1 + 2) = 2c = 1$ , so $c = \tfrac{1}{2}$ , but then $p(0) = -\tfrac{1}{2}$ . Since normalization allows no other value of $c$ , this formula defines no PMF.

Event probability as a sum of masses

Definition Let $X$ be a discrete random variable with PMF $p_X$ and support $S$ . For every set $B$ of real numbers,

P(X \in B) = \sum_{x \in B \cap S} p_X(x).

The set $B \cap S$ is finite or countably infinite, and the events $\{X = x\}$ for its elements are mutually exclusive, so their probabilities add. The values in $B$ outside the support contribute $0$ together, because $X$ lies in $S$ with probability $1$ . When $B$ contains no support value the sum is empty and the probability is $0$ .

In plain words Mark the support values that satisfy the condition, then add their masses. Words such as at least, at most, fewer than and greater than decide whether an endpoint belongs in the sum.

Example For the correction count, at least one correction means $X$ is $1$ or $3$ , so

\begin{aligned} P(X \ge 1) &= p_X(1) + p_X(3) \\[0.4em] &= \tfrac{1}{2} + \tfrac{3}{10} = \tfrac{4}{5}. \end{aligned}

The complement agrees, since $1 - p_X(0) = 1 - \tfrac{1}{5} = \tfrac{4}{5}$ . Fewer than three corrections includes $0$ and $1$ but not $3$ , so $P(X < 3) = \tfrac{1}{5} + \tfrac{1}{2} = \tfrac{7}{10}$ . At most three includes all three support values, so $P(X \le 3) = 1$ . Now let $K$ have the countably infinite PMF found above, with masses $(\tfrac{1}{2})^{k+1}$ at $k = 0, 1, 2, \dots$ . The complement of $\{K \ge 3\}$ is $\{K < 3\}$ , which contains the support values $0$ , $1$ and $2$ . That gives $P(K \ge 3) = 1 - (\tfrac{1}{2} + \tfrac{1}{4} + \tfrac{1}{8}) = \tfrac{1}{8}$ .

Finite uniform distribution

Definition Let $S$ be a set of $n \ge 1$ distinct real numbers. A random variable $X$ has the uniform distribution on $S$ if $p_X(x) = \tfrac{1}{n}$ for every $x$ in $S$ and $p_X(x) = 0$ otherwise. Then for every set $B$ of real numbers

P(X \in B) = \frac{\lvert B \cap S \rvert}{n},

where $\lvert B \cap S \rvert$ is the number of elements of $S$ that lie in $B$ . The word uniform refers to equal masses on $S$ . The values need not be evenly spaced, and numbers that lie between elements of $S$ but are not in $S$ receive no probability.

In plain words Every listed value is equally likely, so the probability of an event is the fraction of the listed values it contains. When the masses are unequal, that fraction need not equal the probability.

Example Four tickets labeled $2$ , $5$ , $9$ and $12$ are in a box, and one is drawn with each ticket equally likely. Its label $X$ is uniform on $\{2, 5, 9, 12\}$ , so each value has mass $\tfrac{1}{4}$ and $P(X > 4) = \tfrac{3}{4}$ . Although $7$ lies between $2$ and $12$ , the probability $P(X = 7)$ is $0$ . Even spacing does not make a distribution uniform either. The values $0$ , $1$ and $2$ are evenly spaced, but a variable with masses $0.5$ , $0.3$ and $0.2$ on them is not uniform.

Probability bar chart

Definition A probability bar chart of a discrete distribution places a vertical bar or spike at each support value $x$ on a horizontal axis of values, with height $p_X(x)$ . Values outside the support get no bar. The heights are probabilities and add to $1$ over the whole support, and the widths of the bars carry no meaning.

In plain words Read a PMF chart by the heights of its bars, not by their areas. A gap in the chart means zero probability at that value.

Example The chart for the correction count has bars at $0$ , $1$ and $3$ with heights $0.2$ , $0.5$ and $0.3$ . The tallest bar is at $1$ , the most likely value, and there is no bar at $2$ because $p_X(2) = 0$ . Drawing the bars wider or narrower changes nothing about the distribution. Later in the course, density curves are read by area. A PMF bar chart is read by height.

Check yourself

A ticket is drawn from four tickets labeled $2$ , $5$ , $9$ and $12$ , each equally likely, and $X$ is its label. Find $P(X = 7)$ , $P(X \le 9)$ and $P(4 < X < 10)$ .

Show the answer

Each of the four labels has mass $\tfrac{1}{4}$ , and $7$ is not a label, so $P(X = 7) = 0$ . The labels with $X \le 9$ are $2$ , $5$ and $9$ , so $P(X \le 9) = \tfrac{3}{4}$ . The labels strictly between $4$ and $10$ are $5$ and $9$ , so $P(4 < X < 10) = \tfrac{2}{4} = \tfrac{1}{2}$ . Treating the labels as if they filled the interval from $2$ to $12$ and computing $\tfrac{9 - 2}{12 - 2} = 0.7$ for $P(X \le 9)$ would be wrong, because the distribution puts mass only on the four labels.

A score $X$ takes the values $0$ , $2$ , $4$ and $6$ with masses $k$ , $2k$ , $2k$ and $k$ . Find $k$ . Then find $P(X \le 2)$ , $P(2 < X \le 6)$ and the probability that $X$ is odd.

Show the answer

The masses add to $6k$ , so $k = \tfrac{1}{6}$ and the masses are $\tfrac{1}{6}$ , $\tfrac{1}{3}$ , $\tfrac{1}{3}$ and $\tfrac{1}{6}$ , all nonnegative. The event $X \le 2$ includes $0$ and $2$ , so

P(X \le 2) = \tfrac{1}{6} + \tfrac{1}{3} = \tfrac{1}{2}.

The event $2 < X \le 6$ excludes $2$ and includes $4$ and $6$ , so $P(2 < X \le 6) = \tfrac{1}{3} + \tfrac{1}{6} = \tfrac{1}{2}$ . No support value is odd, so the probability of an odd score is $0$ . As a check, the first two events are disjoint and together cover the support, and their probabilities add to $1$ .

For which constant $c$ is $p(x) = c(3 - x)$ for $x = 0, 1, 2, 3, 4$ , with $p(x) = 0$ for every other $x$ , a PMF?

Show the answer

For no value of $c$ . At $x = 0, 1, 2, 3, 4$ the factor $3 - x$ equals $3$ , $2$ , $1$ , $0$ and $-1$ , which add to $5$ . Normalization therefore requires

\sum_{x=0}^{4} c(3 - x) = 5c = 1,

so $c = \tfrac{1}{5}$ is the only candidate. With that value $p(4) = -\tfrac{1}{5}$ , which is negative, so the formula fails nonnegativity. If the formula were used only for $x = 0, 1, 2, 3$ , normalization would give $6c = 1$ and the masses $\tfrac{1}{2}$ , $\tfrac{1}{3}$ , $\tfrac{1}{6}$ and $0$ , a valid PMF whose support is $\{0, 1, 2\}$ .

Probability Mass Functions

Key terms

Probability mass function

Nonnegative masses

Normalizing constant

Event probability as a sum of masses

Finite uniform distribution

Probability bar chart

Check yourself

How this lesson connects

Builds on

Used later in