Continuous Distributions, Density and Cumulative Probability, statistics lesson

A rider waiting for a shuttle wants the chance that the wait lasts more than two minutes. A waiting time can be any real number in a range, so its probabilities cannot be listed value by value the way the probabilities of a count can. Continuous models put probability on intervals instead, as area under a density curve. This lesson reads probabilities as areas and from a cumulative distribution function, and it keeps two models apart, a uniform model for an arrival somewhere in a fixed window and an exponential model for waiting on a constant-rate process.

Key terms

Continuous random variable

Definition A random variable $X$ is continuous, in the sense used in this course, if it has a probability density function $f$ , so that for all numbers $a \le b$ the probability $P(a \le X \le b)$ equals the area under the graph of $f$ between $a$ and $b$ . Its support is the set of values the model allows, such as all nonnegative times for a waiting time or the interval $[0, 4]$ for an arrival time within four minutes. For every single value $a$ ,

P(X = a) = 0,

because the region above one point has no width. So including or excluding the endpoints of an interval does not change its probability, and $P(a \le X \le b) = P(a < X < b)$ .

In plain words Probability $0$ does not mean impossible here. Some value always occurs, even though each individual value had probability $0$ beforehand. Real measurements are recorded to some precision, so a recorded value stands for a short interval of true values, and that interval can have positive probability.

Example Suppose an arrival time $U$ in minutes is equally likely to fall anywhere between $0$ and $4$ , so the probability of an interval inside $[0, 4]$ is its length divided by $4$ . Then $P(1 \le U \le 3) = \tfrac24 = \tfrac12$ , while $P(U = 1) = 0$ . A time recorded as 1 minute to the nearest second stands for true times within half a second of 1 minute, an interval of width $\tfrac{1}{60}$ minute, so the recorded value has probability

\frac{1/60}{4} = \frac{1}{240} \approx 0.0042.

Probability density function

Definition A probability density function, or density, is a function $f$ with $f(x) \ge 0$ for every $x$ and total area $1$ under its graph. A random variable $X$ has density $f$ when, for all $a \le b$ , $P(a \le X \le b)$ is the area under $f$ between $a$ and $b$ . In calculus notation this area is $\int_a^b f(x)\,dx$ , but areas of simple shapes, a supplied CDF or a checked numerical tool are enough to find it. If $X$ is measured in some unit, $f(x)$ is measured in the reciprocal unit, such as per minute, so that height times width is a pure number.

In plain words The height of a density is not a probability. It says how densely probability is packed near a value. For a short interval of width $h$ starting at a point $x$ where $f$ is continuous, the probability is close to $f(x) \cdot h$ . Only areas are probabilities, so a density may be larger than $1$ .

Example A uniform model on a window half a minute wide has height $2$ per minute, since $0.5 \text{ minute} \times 2 \text{ per minute} = 1$ . The height exceeds $1$ , yet every probability is an area of at most $1$ . The exponential waiting time with rate $0.5$ per minute, met later on this page, has density $0.5e^{-0.5t}$ per minute for $t \ge 0$ . Its height at 2 minutes is $0.5e^{-1} \approx 0.184$ per minute, and the probability of a wait between 2 and 2.1 minutes is close to $0.184 \times 0.1 \approx 0.0184$ . The exact value is $e^{-1} - e^{-1.05} \approx 0.0179$ .

Probability as area

Definition For a continuous random variable $X$ with density $f$ , the probability of any interval is the area under $f$ over that interval. The cumulative distribution function, or CDF, is

F(t) = P(X \le t),

the area under $f$ to the left of $t$ . It never decreases, it approaches $0$ far to the left and $1$ far to the right, and for $a < b$

\begin{aligned} &P(a < X \le b) \\[0.4em] &\quad = F(b) - F(a), \end{aligned}

with the same value whether each endpoint is included or not. For $0 < c < 1$ , a $c$ quantile is a value $q$ with $F(q) = c$ , a cutoff with probability $c$ to its left. It is unique when $F$ is strictly increasing on the support.

In plain words The CDF accumulates area from the left. Its values are probabilities, so they always lie between $0$ and $1$ , unlike density heights. The probability of an interval is the area accumulated up to its right end minus the area accumulated up to its left end. A quantile answers the reverse question, which cutoff leaves a given probability to its left.

Example Let $X$ have density $f(x) = x/2$ for $0 \le x \le 2$ and $0$ elsewhere. Its graph is a triangle with base $2$ and height $1$ , so the total area is $\tfrac12 \cdot 2 \cdot 1 = 1$ . For $0 \le t \le 2$ , the area to the left of $t$ is a smaller triangle with base $t$ and height $t/2$ , so $F(t) = t^2/4$ . Then $F(1) = \tfrac14$ and

P(1 < X \le 2) = 1 - \tfrac14 = \tfrac34.

The median solves $q^2/4 = \tfrac12$ , so $q = \sqrt{2} \approx 1.414$ . The density reaches $1$ at $x = 2$ , but $P(X = 2) = 0$ .

Uniform distribution

Definition For numbers $a < b$ , a random variable $U$ has the uniform distribution on $[a, b]$ if its density is $1/(b - a)$ for $a \le u \le b$ and $0$ elsewhere. For $a \le c \le d \le b$ ,

P(c \le U \le d) = \frac{d - c}{b - a},

and the CDF is

F(t) = \begin{cases} 0, & t < a, \\ \frac{t - a}{b - a}, & a \le t \le b, \\ 1, & t > b. \end{cases}

By symmetry the mean is $(a + b)/2$ .

In plain words Every stretch of the same length inside $[a, b]$ gets the same probability, and an interval’s probability is its share of the full width. The model suits a time that is equally likely to fall anywhere in a known window, and it gives no probability to values outside the window.

Example Let $U$ be uniform on $[0, 4]$ minutes. The density is $\tfrac14$ per minute, and $P(1 \le U \le 3) = (3 - 1)/4 = \tfrac12$ . The CDF gives $F(3) = \tfrac34$ , so 3 minutes is the $0.75$ quantile, and the mean and median are both 2 minutes.

Exponential distribution

Definition The survival probability of a waiting time $T$ at time $t$ is

\begin{aligned} S(t) &= P(T > t) \\[0.4em] &= 1 - F(t), \end{aligned}

the probability that the wait lasts longer than $t$ . For a rate $r > 0$ , the waiting time $T$ has the exponential distribution with rate $r$ if

P(T > t) = e^{-rt} \quad \text{for } t \ge 0.

Then $F(t) = 1 - e^{-rt}$ for $t \ge 0$ and $F(t) = 0$ for $t < 0$ . The density is $re^{-rt}$ for $t \ge 0$ and $0$ for $t < 0$ , the mean is $E(T) = 1/r$ and the variance is $\operatorname{Var}(T) = 1/r^2$ . If events follow a homogeneous Poisson process with rate $r$ , the wait from time $0$ until the first event has this distribution. For $t > 0$ , the wait exceeds $t$ exactly when the $\operatorname{Poisson}(rt)$ count in $[0, t]$ is $0$ , which has probability $e^{-rt}$ .

In plain words Survival probability just means the chance that the wait is still going at time $t$ , and it need not involve life or death. A higher rate means shorter typical waits. The usual reason to choose an exponential model is a constant-rate process behind the events. Without such a reason the exponential shape is an assumption that needs support.

Example With rate $0.5$ per minute, the probability of waiting more than 2 minutes is $e^{-0.5 \cdot 2} = e^{-1} \approx 0.3679$ , so $F(2) \approx 0.6321$ . The density starts at $0.5$ per minute and decreases. The mean wait is 2 minutes, the variance is 4 square minutes and the SD is 2 minutes. The uniform arrival time on $[0, 4]$ minutes has the same mean, but it never exceeds 4 minutes, while the exponential wait exceeds 4 minutes with probability $e^{-2} \approx 0.1353$ .

Memorylessness

Definition A waiting time $T \ge 0$ with $P(T > s) > 0$ for every $s \ge 0$ is memoryless if, for all $s, t \ge 0$ ,

\begin{aligned} &P(T > s + t \mid T > s) \\[0.4em] &\quad = P(T > t). \end{aligned}

An exponential waiting time with rate $r$ is memoryless, since

\begin{aligned} &P(T > s + t \mid T > s) \\[0.4em] &\quad = \frac{e^{-r(s + t)}}{e^{-rs}} = e^{-rt}. \end{aligned}

Among waiting times with a density, the exponential distributions are the only memoryless ones.

In plain words Under a memoryless model, a wait that has already lasted $s$ minutes has the same chance of lasting $t$ more minutes as a fresh wait. Time already spent does not bring the end any closer. This is a property of the exponential model, not of waiting in general. For a bus that runs on a schedule, an aging part or a task that moves toward completion, the chance of ending soon can change as time passes.

Example With rate $0.5$ per minute,

\begin{aligned} &P(T > 5 \mid T > 3) \\[0.4em] &\quad = \frac{e^{-2.5}}{e^{-1.5}} = e^{-1} \\[0.4em] &\quad \approx 0.3679, \end{aligned}

the same as the probability $P(T > 2)$ for a fresh wait. The uniform model on $[0, 4]$ minutes is not memoryless. After 3 minutes without an arrival the remaining wait is at most 1 minute, while a fresh wait can last up to 4.

Check yourself

A train arrives at a time $U$ that is uniform on $[0, 10]$ minutes. Find the density, $P(2 \le U \le 5)$ , $F(6)$ and $P(U = 5)$ .

Show the answer

The density is $\tfrac{1}{10} = 0.1$ per minute on $[0, 10]$ and $0$ elsewhere. The interval from 2 to 5 has width 3, so $P(2 \le U \le 5) = \tfrac{3}{10}$ . The CDF gives $F(6) = \tfrac{6}{10} = 0.6$ . A single time has no width, so $P(U = 5) = 0$ .

A wait $T$ is exponential with rate $0.5$ per minute. Find $P(2 < T \le 4)$ . The density at 2 minutes is about $0.184$ per minute. Is that the probability of waiting exactly 2 minutes?

Show the answer

Use the CDF $F(t) = 1 - e^{-0.5t}$ . Then

\begin{aligned} &P(2 < T \le 4) \\[0.4em] &\quad = F(4) - F(2) \\[0.4em] &\quad = e^{-1} - e^{-2} \\[0.4em] &\quad \approx 0.2325. \end{aligned}

This removes the tail beyond 4 minutes from the tail beyond 2 minutes. The value $0.184$ is a density height in units of per minute, not a probability. The probability of waiting exactly 2 minutes is $0$ , because a single point has no width. The height only says that a short interval of width $h$ starting at 2 minutes has probability close to $0.184h$ .

A rider has already waited 2 minutes. Under the uniform model on $[0, 4]$ minutes, find the probability of waiting more than 1 more minute, and compare it with the probability that a fresh wait exceeds 1 minute. Then do the same under the exponential model with rate $0.5$ per minute.

Show the answer

Under the uniform model,

\begin{aligned} P(U > 3 \mid U > 2) &= \frac{1/4}{2/4} \\[0.4em] &= \tfrac12, \end{aligned}

while a fresh wait exceeds 1 minute with probability $P(U > 1) = \tfrac34$ . The elapsed time changed the answer, so this model is not memoryless.

Under the exponential model,

\begin{aligned} &P(T > 3 \mid T > 2) \\[0.4em] &\quad = \frac{e^{-1.5}}{e^{-1}} = e^{-0.5} \\[0.4em] &\quad \approx 0.6065, \end{aligned}

which equals $P(T > 1) = e^{-0.5}$ . The elapsed 2 minutes make no difference, as memorylessness says.

Continuous Distributions, Density and Cumulative Probability

Key terms

Continuous random variable

Probability density function

Probability as area

Uniform distribution

Exponential distribution

Memorylessness

Check yourself

How this lesson connects

Builds on

Used later in