Discrete Random Variables and Probability Distributions, statistics lesson

An order can pass through several processing routes, but for planning only one number matters, how many follow-up messages the order will need. Before the order is handled that number is uncertain, yet a probability model can say how likely each value is. This lesson defines a random variable as a numerical function of the outcomes and shows how a probability mass function and a cumulative distribution function describe its distribution. It also shows why the events $X < 1$ and $X \le 1$ can have different probabilities.

Key terms

Random variable

Definition A random variable $X$ is a function that assigns a real number $X(\omega)$ to each outcome $\omega$ of a sample space $\Omega$ . For a set $B$ of real numbers, the event $\{X \in B\}$ consists of the outcomes $\omega$ with $X(\omega)$ in $B$ , and $P(X \in B)$ is its probability. In particular $\{X = x\}$ is the set of outcomes that $X$ maps to the number $x$ . A capital letter names the random variable and a lowercase letter names one of its possible values.

In plain words The randomness lies in which outcome occurs. The random variable is a fixed rule that reads one number off that outcome, and several outcomes can give the same number.

Example Five processing routes $a$ , $b$ , $c$ , $d$ and $e$ have probabilities $0.2$ , $0.2$ , $0.3$ , $0.1$ and $0.2$ . Let $X$ be the number of follow-up messages, with $X(a) = 0$ , $X(b) = X(c) = 1$ and $X(d) = X(e) = 2$ . The event $\{X = 1\}$ is $\{b, c\}$ , so $P(X = 1) = 0.2 + 0.3 = 0.5$ . The variable keeps only the count and sets the other details of the route aside.

Discrete random variable

Definition A random variable $X$ is discrete if there is a finite or countably infinite set $C$ of real numbers with $P(X \in C) = 1$ . Countably infinite means that the values can be listed in a sequence $x_1, x_2, x_3, \dots$ that never ends.

In plain words A discrete random variable takes separate values that can be listed, such as $0, 1, 2$ and so on, and its probability sits on those individual values. Counts are the typical case. A quantity measured on a continuous scale, such as a waiting time, is usually modeled in a different way, with a density.

Example The follow-up count takes only the values $0$ , $1$ and $2$ , so it is discrete with finitely many values. Let $W$ be the number of tosses of a fair coin up to and including the first head. Its values are $1, 2, 3, \dots$ with $P(W = w) = (1/2)^w$ , and these probabilities add to $1$ , so $W$ is discrete with countably infinitely many values.

Support

Definition The support of a discrete random variable $X$ is the set of values $x$ with $P(X = x) > 0$ . The probabilities of the values in the support add to $1$ , and every value outside the support has probability $0$ .

In plain words The support lists the values that the variable takes with positive chance. A value written in a table with probability $0$ is not part of it.

Example The follow-up count has support $\{0, 1, 2\}$ . If a table also listed the value $3$ with probability $0$ , the support would still be $\{0, 1, 2\}$ . The value $1.5$ is outside the support, so $P(X = 1.5) = 0$ , even though it lies between two support values.

Probability distribution

Definition The probability distribution of a random variable $X$ is the rule that gives the probability $P(X \in B)$ for every set $B$ of real numbers, that is, how the total probability $1$ is spread over the possible values. For a discrete random variable the distribution is determined by the probabilities of the individual support values, through

P(X \in B) = \sum_{x \in B} P(X = x),

where the sum runs over the support values that lie in $B$ . Different random variables can have the same distribution.

In plain words The distribution says which values $X$ can take and how likely every set of values is. It is a model for a value that has not been observed yet, not a table of frequencies in observed data.

Example The follow-up count has the distribution with probabilities $0.2$ , $0.5$ and $0.3$ at the values $0$ , $1$ and $2$ . Suppose 10 observed orders needed $0$ , $1$ and $2$ follow-ups $1$ , $6$ and $3$ times. The relative frequencies $0.1$ , $0.6$ and $0.3$ summarize those data, and they can be compared with the model, but they are not the model. For a second example, toss a fair coin once and let $H$ count heads and $T$ count tails. These are different random variables, since $H + T = 1$ means they never take the same value. Yet each takes the values $0$ and $1$ with probability $0.5$ , so they have the same distribution.

Probability mass function

Definition The probability mass function, or PMF, of a discrete random variable $X$ is the function

p_X(x) = P(X = x),

defined for every real number $x$ . A function $p$ is the PMF of some discrete random variable exactly when $p(x) \ge 0$ for every $x$ , $p(x) > 0$ for at most countably many $x$ , and those positive values add to $1$ . Nonnegativity and the total of $1$ are separate requirements. A table with a negative entry can still total $1$ , and a table of nonnegative entries can fail to total $1$ .

In plain words The PMF puts a mass of probability on each possible value. To find the probability of any event about $X$ , list the values that satisfy it and add their masses.

Example For the follow-up count, $p_X(0) = 0.2$ , $p_X(1) = 0.5$ , $p_X(2) = 0.3$ and $p_X(x) = 0$ for every other $x$ . The masses are nonnegative and $0.2 + 0.5 + 0.3 = 1$ . The event $X > 0$ contains the values $1$ and $2$ , so it has probability $0.5 + 0.3 = 0.8$ . The event $X < 1$ contains only the value $0$ , so it has probability $0.2$ . The masses $0.3$ , $0.8$ and $-0.1$ total $1$ but do not form a PMF, and neither do the masses $0.2$ , $0.5$ and $0.2$ , whose total is $0.9$ .

Cumulative distribution function

Definition The cumulative distribution function, or CDF, of a random variable $X$ is

F_X(x) = P(X \le x)

for every real number $x$ . It is nondecreasing, it approaches $0$ as $x \to -\infty$ and $1$ as $x \to \infty$ , and it is right-continuous, so its value at a point includes the probability at that point. For a discrete random variable the jump of $F_X$ at each value $x$ equals $p_X(x)$ , and when the support values are spaced apart, as with counts, $F_X$ is a step function that is flat between them. For real numbers $a < b$ ,

\begin{aligned} &P(a < X \le b) \\[0.4em] &\quad = F_X(b) - F_X(a), \end{aligned}

and for every real $x$ the strict event has probability $P(X < x) = F_X(x) - p_X(x)$ .

In plain words The CDF gives, for every cutoff $x$ , the probability that $X$ is at most $x$ . Moving the cutoff to the right can only add values, so the CDF never decreases. With discrete values the inequality sign matters, because a single value can carry positive probability.

Example For the follow-up count,

F_X(x) = \begin{cases} 0, & x < 0, \\ 0.2, & 0 \le x < 1, \\ 0.7, & 1 \le x < 2, \\ 1, & x \ge 2. \end{cases}

So $P(X \le 1) = F_X(1) = 0.7$ , while $P(X < 1) = 0.7 - 0.5 = 0.2$ . The difference $F_X(2) - F_X(0.5) = 1 - 0.2 = 0.8$ is the probability of the values $1$ and $2$ . The event $1 \le X \le 2$ also has probability $0.8$ , but $F_X(2) - F_X(1) = 0.3$ , because that difference is $P(1 < X \le 2)$ and leaves out the mass at $1$ .

Check yourself

A random variable $X$ takes only the values $0$ , $1$ and $2$ , with $p_X(0) = 0.1$ and $p_X(1) = 0.6$ . Find $p_X(2)$ and $P(X \ge 1)$ .

Show the answer

The masses must add to $1$ , so $p_X(2) = 1 - 0.1 - 0.6 = 0.3$ . The event $X \ge 1$ contains the values $1$ and $2$ , so $P(X \ge 1) = 0.6 + 0.3 = 0.9$ . This is not $F_X(1) = 0.7$ , which is the probability of the different event $X \le 1$ .

For the random variable of question 1, find $P(X < 2)$ , $P(X \le 2)$ and $F_X(1.5)$ , and write out its CDF.

Show the answer

The strict event $X < 2$ contains $0$ and $1$ , so its probability is $0.1 + 0.6 = 0.7$ . The inclusive event $X \le 2$ contains every support value, so its probability is $1$ . No support value lies between $1$ and $1.5$ , so $F_X(1.5) = F_X(1) = 0.7$ . The CDF is

F_X(x) = \begin{cases} 0, & x < 0, \\ 0.1, & 0 \le x < 1, \\ 0.7, & 1 \le x < 2, \\ 1, & x \ge 2. \end{cases}

Someone proposes the following function as the CDF of a count. Can it be one?

G(x) = \begin{cases} 0, & x < 0, \\ 0.6, & 0 \le x < 1, \\ 0.4, & 1 \le x < 2, \\ 1, & x \ge 2. \end{cases}

Show the answer

No. The function falls from $0.6$ to $0.4$ at $x = 1$ , but a CDF never decreases, because the event $X \le 0$ is contained in the event $X \le 1$ . Reading the jump at $1$ as a probability mass would give $P(X = 1) = 0.4 - 0.6 = -0.2$ , and a probability cannot be negative.

Discrete Random Variables and Probability Distributions

Key terms

Random variable

Discrete random variable

Support

Probability distribution

Probability mass function

Cumulative distribution function

Check yourself

How this lesson connects

Builds on

Used later in