Point Probabilities and Mixed Distributions, introduction to probability lesson

In a constructed model of a service desk, $20$ percent of requests are handled at once and the rest wait a time spread evenly over the first minute. Waiting exactly zero then has probability $0.2$ , while waiting exactly half a minute has probability $0$ , even though it can happen. A density alone cannot describe the first fact, and the point probabilities alone add up to only $0.2$ , so they cannot describe how the remaining $0.8$ is spread over the minute. This lesson puts both kinds of probability into one distribution, reads point probabilities from the jumps of its CDF, and separates events of probability zero from impossible events.

Key terms

Point mass

Definition A random variable $X$ has a point mass, also called an atom, at a real number $a$ if

P(X = a) > 0,

and the number $P(X = a)$ is the mass at $a$ . For each positive integer $n$ , fewer than $n$ values can have mass greater than $1/n$ , since the masses of distinct values add to at most $1$ . So a distribution has at most countably many atoms. The distribution that puts probability $1$ on the single value $a$ , the law of the constant random variable equal to $a$ , is also called the point mass at $a$ .

In plain words An atom is a single value that holds a positive share of probability by itself. A discrete distribution is made entirely of atoms. A distribution with a density has none, because a density spreads probability over intervals and gives each single value probability $0$ . A density height is not a point mass, so an atom is drawn as a separate labeled spike or dot, not as part of a density curve.

Example For the waiting time $T$ in minutes at the service desk, $P(T = 0) = 0.2$ , so $T$ has an atom at $0$ with mass $0.2$ . It has no other atom. Every time $t$ with $0 < t < 1$ can occur, but $P(T = t) = 0$ , as the card on probability-zero events shows. A fair die roll, by contrast, has six atoms of mass $\tfrac16$ each and no continuous part.

Mixed distribution

Definition A random variable $X$ has a mixed distribution, in the sense used in this course, if it has atoms $a_1, a_2, \dots$ with masses $m_1, m_2, \dots$ and there is a function $h \ge 0$ such that for all real numbers $c < d$

\begin{aligned} &P(c < X \le d) \\[0.4em] &\quad = \sum_{c < a_i \le d} m_i + \int_c^d h(x)\,dx, \end{aligned}

where the total atom mass $M = \sum_i m_i$ satisfies $0 < M < 1$ and the total area under $h$ is $1 - M$ . Equivalently, with probability $M$ the value of $X$ comes from the discrete distribution with masses $m_i / M$ , and with probability $1 - M$ it comes from the continuous distribution with density $h / (1 - M)$ . A mixed $X$ has no density at all, because a variable with a density gives every single value probability $0$ . The function $h$ describes only the continuous part.

In plain words Part of the probability sits on single values and the rest is spread out. The two parts are kept apart in the bookkeeping, the atoms by their masses and the spread part by the area under $h$ , and together they add to $1$ . The continuous part carries only its own weight $1 - M$ , not a full unit of probability.

Example The waiting time $T$ has the atom $0$ with mass $M = 0.2$ , and $h(t) = 0.8$ for $0 < t < 1$ with $h(t) = 0$ elsewhere. The area under $h$ is $0.8$ , and $0.2 + 0.8 = 1$ . Given $T > 0$ , the waiting time is uniform on $(0, 1)$ . Its CDF $F_T(t) = P(T \le t)$ has three pieces.

Threshold	$F_T(t)$
$t < 0$	$0$
$0 \le t < 1$	$0.2 + 0.8t$
$t \ge 1$	$1$

So $P(T \le 0.5) = 0.2 + 0.4 = 0.6$ , where the atom contributes $0.2$ and the continuous part contributes $0.8 \cdot 0.5 = 0.4$ . Among requests that wait at all, the share that waits at most half a minute is

\begin{aligned} &P(T \le 0.5 \mid T > 0) \\[0.4em] &\quad = \frac{0.4}{0.8} = 0.5, \end{aligned}

which uses $P(T > 0) = 0.8 > 0$ . This answer and $0.6$ refer to different groups of requests. If instead $40$ percent were handled at once and the positive waits kept the same uniform law, then $P(T \le 0.5) = 0.4 + 0.6 \cdot 0.5 = 0.7$ .

Left limit of a CDF

Definition Let $F$ be the CDF of a random variable $X$ and let $a$ be a real number. The left limit of $F$ at $a$ is

F(a^-) = \lim_{t \to a^-} F(t),

the limit as $t$ increases to $a$ through values $t < a$ . The superscript means from below and does not make $a$ negative. The limit exists because $F$ is nondecreasing and bounded, and it equals $P(X < a)$ , because the events $\{X \le a - 1/n\}$ increase to $\{X < a\}$ . So

\begin{gathered} P(X < a) = F(a^-), \\[0.4em] P(X \ge a) = 1 - F(a^-), \end{gathered}

and for $a \le b$ the closed interval has $P(a \le X \le b) = F(b) - F(a^-)$ .

In plain words The left limit is the level the CDF approaches just before reaching $a$ . It counts all the probability strictly below $a$ and leaves out any mass at $a$ itself, while $F(a)$ includes that mass. Whether an endpoint belongs to an event decides which of the two to use.

Example For the waiting time, $F_T(t) = 0$ for every $t < 0$ , so $F_T(0^-) = 0$ , while $F_T(0) = 0.2$ . Hence

\begin{aligned} &P(0 < T \le 0.5) \\[0.4em] &\quad = F_T(0.5) - F_T(0) \\[0.4em] &\quad = 0.6 - 0.2 = 0.4, \end{aligned}

but $P(0 \le T \le 0.5) = F_T(0.5) - F_T(0^-) = 0.6$ . The two events differ only by the value $0$ , and the answers differ by its mass $0.2$ . At $0.5$ the left limit is $0.2 + 0.8 \cdot 0.5 = 0.6$ , which equals $F_T(0.5)$ , so including or excluding the endpoint $0.5$ changes nothing.

CDF jump

Definition The jump of the CDF $F$ of a random variable $X$ at a real number $a$ is

F(a) - F(a^-) = P(X = a),

since $P(X \le a) - P(X < a) = P(X = a)$ . So $F$ is continuous at $a$ exactly when $P(X = a) = 0$ , the atoms of $X$ are exactly the points where $F$ jumps, and each jump size is the mass of its atom. Because $F$ is right-continuous, $F(a)$ is the upper level at a jump. The CDF of a mixed distribution rises in two ways, by jumps at its atoms and continuously as area under $h$ accumulates.

In plain words A vertical step in the graph of a CDF marks a single value that holds probability, and the height of the step is that probability. A steep but unbroken rise is not a jump. It means a large amount of probability on a short stretch, as with a high density, and each single value there still has probability $0$ .

Example The waiting-time CDF jumps by $0.2$ at $0$ and has no other jump. At $1$ it reaches $1$ continuously, since $F_T(1^-) = 0.2 + 0.8 = 1 = F_T(1)$ . An atom can also sit at the top of a range. In a constructed model, a load $X$ in kilograms is uniform on $(0, 10)$ , and a scale that cannot show more than $8$ displays $Y = \min(X, 8)$ . Then $F_Y(y) = 0$ for $y < 0$ , $F_Y(y) = y/10$ for $0 \le y < 8$ , and $F_Y(y) = 1$ for $y \ge 8$ . The jump at $8$ is

\begin{aligned} &F_Y(8) - F_Y(8^-) \\[0.4em] &\quad = 1 - 0.8 = 0.2, \end{aligned}

which is $P(Y = 8) = P(X \ge 8)$ . The displayed reading has an atom at $8$ and a continuous part of weight $0.8$ .

Probability-zero event

Definition An event $A$ with $P(A) = 0$ is a probability-zero event, also called a null event. The empty event $\varnothing$ has probability $0$ , but a probability-zero event need not be empty. If the CDF $F$ of $X$ is continuous at $a$ , then $\{X = a\}$ has probability $0$ even when $a$ is a value that $X$ can take, because

\begin{aligned} &P(X = a) \\[0.4em] &\quad \le F(a) - F(a - \varepsilon) \end{aligned}

for every $\varepsilon > 0$ , and the right side tends to $F(a) - F(a^-) = 0$ as $\varepsilon$ decreases to $0$ . A countable union of probability-zero events again has probability $0$ . The formula $P(A \mid B) = P(A \cap B)/P(B)$ requires $P(B) > 0$ , so it cannot condition on a probability-zero event.

In plain words Probability zero means the model gives the event no share of probability, not that the event cannot happen. An impossible event always has probability $0$ , but the converse is false in general, and continuous models give many counterexamples. These statements concern an idealized exact quantity. A measurement recorded to a fixed number of digits has a discrete set of recorded values, and each of them can have positive probability.

Example Every waiting time between $0$ and $1$ minute can occur. For each $\varepsilon$ with $0 < \varepsilon \le 0.5$ , the interval $(0.5 - \varepsilon, 0.5]$ contains $0.5$ and has probability $0.8\varepsilon$ . So $P(T = 0.5) \le 0.8\varepsilon$ for every such $\varepsilon$ , which forces $P(T = 0.5) = 0$ . The event that $T$ is a positive rational number is the countable union of the events $\{T = r\}$ over positive rational $r$ , each of probability $0$ , so it also has probability $0$ . If waiting times are recorded to the nearest tenth of a minute with halves rounded up, the recorded value $0.5$ means $0.45 \le T < 0.55$ , an event of probability $0.8 \cdot 0.1 = 0.08$ .

Almost surely

Definition An event $A$ happens almost surely if $P(A) = 1$ , which is the same as $P(A^{c}) = 0$ . Random variables $X$ and $Y$ are equal almost surely if $P(X = Y) = 1$ . An almost sure event can fail on a nonempty set of outcomes, as long as that set has probability $0$ . When the sample space $\Omega$ is finite or countable and every outcome has positive probability, the only probability-zero event is $\varnothing$ , and almost surely means surely.

In plain words Almost surely means with probability one under the model. It is weaker than logical certainty, since exceptions may exist as long as together they carry no probability.

Example Let $U \sim \operatorname{Uniform}(0, 1)$ . Its CDF is continuous, so $P(U = 1/2) = 0$ and $P(U \ne 1/2) = 1 - 0 = 1$ . So $U \ne 1/2$ almost surely, although $1/2$ is a value $U$ can take. For the waiting time, $T \ne 0.5$ almost surely, while $T > 0$ is not almost sure, since $P(T > 0) = 0.8$ . A variable that equals $T$ except that the value $0.5$ is replaced by $0.6$ is equal to $T$ almost surely and has the same distribution.

Check yourself

In a constructed model, $30$ percent of tasks finish immediately, so the completion time $T$ in minutes has an atom at $0$ with mass $0.3$ . The other $70$ percent have a completion time uniform on $(2, 4)$ . Find $P(T \le 3)$ , $P(0 < T < 2)$ and $P(T = 3)$ .

Show the answer

By $3$ minutes all immediate tasks have finished, and so has the share $(3 - 2)/(4 - 2) = \tfrac12$ of the others. So

\begin{aligned} P(T \le 3) &= 0.3 + 0.7 \cdot \tfrac12 \\[0.4em] &= 0.65. \end{aligned}

The open interval $(0, 2)$ excludes the atom at $0$ and lies below the continuous part, so $P(0 < T < 2) = 0$ . The value $3$ lies inside $(2, 4)$ and is not an atom, so $P(T = 3) = 0$ .

For the same $T$ , write the CDF $F_T$ as a table. Use it to find $P(0 < T \le 3)$ and $P(0 \le T \le 3)$ , and find $P(T \le 3 \mid T > 0)$ .

Show the answer

The CDF is $0$ below $0$ , jumps to $0.3$ at $0$ , stays flat until $2$ , rises with slope $0.7/2 = 0.35$ until $4$ , and then stays at $1$ .

Threshold	$F_T(t)$
$t < 0$	$0$
$0 \le t < 2$	$0.3$
$2 \le t < 4$	$0.3 + 0.35(t - 2)$
$t \ge 4$	$1$

So $P(0 < T \le 3) = F_T(3) - F_T(0) = 0.65 - 0.3 = 0.35$ , while $P(0 \le T \le 3) = F_T(3) - F_T(0^-) = 0.65 - 0 = 0.65$ . The two answers differ by the jump $0.3$ at $0$ . Since $P(T > 0) = 0.7 > 0$ , the conditional probability is defined, and

\begin{aligned} &P(T \le 3 \mid T > 0) \\[0.4em] &\quad = \frac{0.35}{0.7} = 0.5. \end{aligned}

Among tasks that do not finish immediately, half finish by $3$ minutes, the midpoint of $(2, 4)$ .

A student says that every random variable has $P(X \le a) = P(X < a)$ , because a single value has probability zero. Give a counterexample from question 1, and find every $a$ for which the equation does hold for that $T$ . Then decide whether $P(T = 3) = 0$ means that a completion time of exactly $3$ minutes is impossible in the model.

Show the answer

The equation fails at $a = 0$ , since $P(T \le 0) = F_T(0) = 0.3$ but $P(T < 0) = F_T(0^-) = 0$ . In general the difference $P(X \le a) - P(X < a)$ is $P(X = a)$ , the jump of the CDF at $a$ . For this $T$ the only atom is $0$ , so the equation holds for every $a \ne 0$ and fails only at $a = 0$ .

The event $\{T = 3\}$ has probability $0$ , but it is not impossible. The value $3$ lies inside the interval where the continuous part has positive density, and every interval $(3 - \varepsilon, 3]$ with $0 < \varepsilon \le 1$ has probability $0.35\varepsilon > 0$ . What is true is that $T \ne 3$ almost surely.

Point Probabilities and Mixed Distributions

Key terms

Point mass

Mixed distribution

Left limit of a CDF

CDF jump

Probability-zero event

Almost surely

Check yourself

How this lesson connects

Builds on