Cumulative Distribution Functions, introduction to probability lesson

Many questions about a random variable ask how much probability lies at or below a threshold, such as at most three errors or a score change of no more than one point. A probability mass function answers each of them only after you add the right masses. The cumulative distribution function does that addition once for every threshold on the real line, and its definition includes the threshold itself. This lesson builds it for discrete random variables, reads the masses back from its jumps, and finds tail and interval probabilities with every endpoint accounted for.

Key terms

Cumulative distribution function

Definition The cumulative distribution function, or CDF, of a random variable $X$ is the function $F$ defined for every real number $t$ by

F(t) = P(X \le t).

Every CDF is nondecreasing, is right-continuous, and satisfies $F(t) \to 0$ as $t \to -\infty$ and $F(t) \to 1$ as $t \to \infty$ . If $X$ is discrete with PMF $p_X$ and support $S$ , then $F(t)$ is the sum of $p_X(x)$ over the support values $x \le t$ . The inequality includes equality, so the mass at $t$ itself counts.

In plain words The value $F(t)$ is all the probability at or below the threshold $t$ . The threshold can be any real number, including numbers the random variable never takes. Raising the threshold can only add outcomes, never remove them, which is why $F$ cannot decrease.

Example In a constructed model, a score change $X$ takes the values $-2$ , $1$ and $4$ with probabilities $0.2$ , $0.5$ and $0.3$ . Moving the threshold from left to right and adding each mass as it is reached gives

F(t) = \begin{cases} 0, & t < -2, \\ 0.2, & -2 \le t < 1, \\ 0.7, & 1 \le t < 4, \\ 1, & t \ge 4. \end{cases}

So $F(0) = 0.2$ even though $0$ is not a possible value, and $F(1) = 0.2 + 0.5 = 0.7$ . This $F$ reaches $0$ and $1$ because the support is finite. When the support has no upper bound, $F(t) < 1$ at every threshold, and $F$ only approaches $1$ . For the number $T$ of independent fair coin tosses up to and including the first head, the CDF $F_T$ of $T$ has $F_T(n) = 1 - (\tfrac{1}{2})^{n}$ at each positive integer $n$ , which is less than $1$ for every $n$ and tends to $1$ .

Step function

Definition Let $X$ be discrete with finite support $x_1 < x_2 < \dots < x_m$ . Its CDF is a step function. It equals $0$ for $t < x_1$ , it equals $1$ for $t \ge x_m$ , and on each interval $x_k \le t < x_{k+1}$ with $1 \le k < m$ it is constant and equal to

\sum_{j=1}^{k} p_X(x_j),

the total mass of the first $k$ support values. More generally, for any discrete $X$ and any $a < b$ , the difference $F(b) - F(a)$ is the total mass of the support values in $(a, b]$ , so $F$ is constant on every interval that contains no support value.

In plain words Between consecutive support values nothing new is added, so the graph stays flat. At each support value it steps up. A flat stretch means that no mass lies between its ends, not that probability is spread evenly along it.

Example For the score change $X$ the graph is flat at height $0.2$ from $-2$ up to just below $1$ . That does not mean $X$ is spread over the values between $-2$ and $1$ . The values $-1$ and $0$ have probability $0$ , and indeed $F(-1) = F(0) = 0.2$ . A careful graph draws filled dots at the attained heights $(-2, 0.2)$ , $(1, 0.7)$ and $(4, 1)$ , and open dots at the right ends of the lower pieces, at $(-2, 0)$ , $(1, 0.2)$ and $(4, 0.7)$ .

Jump size

Definition For every real number $a$ , the limit of $F(t)$ as $t$ increases to $a$ exists. It is written $F(a^-)$ and equals $P(X < a)$ . The jump of $F$ at $a$ is

F(a) - F(a^-) = P(X = a).

So $F$ jumps exactly at the values with positive probability, and the size of the jump is the mass at that value. For a discrete random variable this recovers the whole PMF from the CDF.

In plain words The rise at each step is the probability of that one value. The height of the CDF itself is accumulated probability, which also contains every earlier mass.

Example For the score change, the jump at $1$ is $F(1) - F(1^-) = 0.7 - 0.2 = 0.5$ , which is $P(X = 1)$ . The jump at $-2$ is $0.2 - 0 = 0.2$ , and the jump at $4$ is $1 - 0.7 = 0.3$ . At $0$ there is no jump, since $F(0) - F(0^-) = 0.2 - 0.2 = 0$ , so $P(X = 0) = 0$ . Reading the height $F(1) = 0.7$ as $P(X = 1)$ is a common error, because that height also includes the mass $0.2$ at $-2$ .

Right-continuity

Definition A CDF is right-continuous at every real number $a$ ,

\lim_{t \to a^+} F(t) = F(a).

As $t$ decreases to $a$ , the events $\{X \le t\}$ shrink to $\{X \le a\}$ , and probability is continuous along a decreasing sequence of events. The limit from the left, $F(a^-) = P(X < a)$ , differs from $F(a)$ exactly when $P(X = a) > 0$ . So right-continuity allows jumps, and at a jump the CDF takes the upper value.

In plain words At a jump the CDF already includes the mass at that point, because the definition uses less than or equal to. On a graph this is the filled dot at the top of the step.

Example For the score change, $F(t) = 0.7$ for $1 \le t < 4$ , so the limit from the right at $1$ is $0.7 = F(1)$ . From the left, $F(t) = 0.2$ for $-2 \le t < 1$ , so $F(1^-) = 0.2$ . Setting $F(1) = 0.2$ would leave out $P(X = 1)$ , which $P(X \le 1)$ includes, and the resulting function would not be right-continuous at $1$ .

Tail probability

Definition The upper tail probabilities of $X$ at a threshold $t$ are

\begin{gathered} P(X > t) = 1 - F(t), \\[0.4em] P(X \ge t) = 1 - F(t^-). \end{gathered}

Both follow from the complement rule, since $\{X > t\}$ is the complement of $\{X \le t\}$ and $\{X \ge t\}$ is the complement of $\{X < t\}$ . The two tails differ by $P(X = t)$ . The lower tails are $P(X \le t) = F(t)$ and $P(X < t) = F(t^-)$ .

In plain words Whether the threshold value is included decides whether you subtract the CDF at $t$ or the level just below $t$ .

Example For the score change, $P(X > 1) = 1 - F(1) = 1 - 0.7 = 0.3$ , which matches the table, since $4$ is the only value above $1$ . Changing one word gives $P(X \ge 1) = 1 - F(1^-) = 1 - 0.2 = 0.8$ , which is $0.5 + 0.3$ . Using $1 - F(1)$ for $P(X \ge 1)$ would remove the mass at $1$ as well and give the wrong answer $0.3$ .

Interval probability

Definition For real numbers $a < b$ ,

\begin{aligned} &P(a < X \le b) \\[0.4em] &\quad = F(b) - F(a), \end{aligned}

because the event $\{X \le b\}$ is the disjoint union of $\{X \le a\}$ and $\{a < X \le b\}$ . The other endpoint choices use limits from the left.

Event	Probability
$a < X \le b$	$F(b) - F(a)$
$a \le X \le b$	$F(b) - F(a^-)$
$a < X < b$	$F(b^-) - F(a)$
$a \le X < b$	$F(b^-) - F(a^-)$

In plain words Take all the probability up to the right end and remove all the probability up to the left end. Whether an endpoint is in or out decides whether to use the CDF at that point or the level just below it.

Example For the score change, $P(-1 < X \le 3) = F(3) - F(-1) = 0.7 - 0.2 = 0.5$ . The only support value in $(-1, 3]$ is $1$ , so the answer checks, and neither endpoint needed to be a possible value. With both endpoints at support values,

\begin{aligned} &P(-2 \le X < 4) \\[0.4em] &\quad = F(4^-) - F\big((-2)^-\big) \\[0.4em] &\quad = 0.7 - 0 = 0.7, \end{aligned}

which matches $0.2 + 0.5$ from the included values $-2$ and $1$ , with $4$ left out. Interval probabilities come from heights of the CDF and their differences, not from areas under its graph.

Check yourself

A random variable $Y$ takes the values $-1$ , $0$ and $2$ with probabilities $\tfrac{1}{4}$ , $\tfrac{1}{2}$ and $\tfrac{1}{4}$ . Write its CDF $F_Y$ for every real $t$ , and find $F_Y(-0.5)$ and $F_Y(1)$ .

Show the answer

Adding the masses as the threshold passes each support value gives

F_Y(t) = \begin{cases} 0, & t < -1, \\ \tfrac{1}{4}, & -1 \le t < 0, \\ \tfrac{3}{4}, & 0 \le t < 2, \\ 1, & t \ge 2. \end{cases}

So $F_Y(-0.5) = \tfrac{1}{4}$ , since only the support value $-1$ lies at or below $-0.5$ , and $F_Y(1) = \tfrac{3}{4}$ , since the support values $-1$ and $0$ lie at or below $1$ .

For the same $Y$ , use the CDF to find $P(Y \ge 0)$ and $P(0 < Y \le 2)$ . Why do the two answers differ by $\tfrac{1}{2}$ ?

Show the answer

The event $Y \ge 0$ includes $0$ , so

\begin{aligned} P(Y \ge 0) &= 1 - F_Y(0^-) \\[0.4em] &= 1 - \tfrac{1}{4} = \tfrac{3}{4}, \end{aligned}

which matches the masses at $0$ and $2$ . The event $0 < Y \le 2$ excludes $0$ , so $P(0 < Y \le 2) = F_Y(2) - F_Y(0) = 1 - \tfrac{3}{4} = \tfrac{1}{4}$ , the mass at $2$ alone. The first event is the second one together with the value $0$ and the values above $2$ , and $P(Y > 2) = 0$ . So the difference is $P(Y = 0) = \tfrac{1}{2}$ , the jump $F_Y(0) - F_Y(0^-)$ .

Consider the function

G(t) = \begin{cases} 0, & t < 0, \\ \tfrac{1}{2}, & 0 \le t \le 1, \\ 1, & t > 1. \end{cases}

It is nondecreasing, tends to $0$ on the left and reaches $1$ on the right. Is it a CDF? If not, change only the endpoint conventions to make it one, and describe the distribution you get.

Show the answer

It is not a CDF, because it is not right-continuous at $1$ . The value is $G(1) = \tfrac{1}{2}$ , but $G(t) = 1$ for every $t > 1$ , so the limit from the right at $1$ is $1$ . A CDF with these values would need $P(X \le 1) = \tfrac{1}{2}$ while $P(X \le t) = 1$ for all $t > 1$ , which contradicts the continuity of probability along the shrinking events $\{X \le t\}$ .

Moving the endpoint so that the middle piece is $0 \le t < 1$ and the last piece is $t \ge 1$ gives a right-continuous step function. It is the CDF of a random variable that equals $0$ or $1$ , each with probability $\tfrac{1}{2}$ , since its jumps at $0$ and $1$ both have size $\tfrac{1}{2}$ .

Cumulative Distribution Functions

Key terms

Cumulative distribution function

Step function

Jump size

Right-continuity

Tail probability

Interval probability

Check yourself

How this lesson connects

Builds on

Used later in