Normal Distributions and Quantiles, statistics lesson

A filling machine is described by a constructed model in which the fill amount is normal with mean 100 units and standard deviation 10 units. Under that model, what fraction of fills exceeds 120 units, and which cutoff has only 5 percent of fills above it? Both questions are answered by areas under one standard curve. This lesson computes those areas, reverses them into cutoffs called quantiles, and keeps in view that every answer describes the model, which a bell-shaped sketch does not establish.

Key terms

Normal distribution

Definition A continuous random variable $X$ has the normal distribution with mean $\mu$ and standard deviation $\sigma > 0$ , written $X \sim N(\mu, \sigma^2)$ , if its density is

f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-(x - \mu)^2/(2\sigma^2)}

for every real $x$ . The second parameter in $N(\mu, \sigma^2)$ is the variance, so $N(100, 10^2)$ has standard deviation $10$ , not $100$ . The density is positive on the whole real line, symmetric about $\mu$ and highest at $\mu$ , so the mean, the median and the mode all equal $\mu$ . Probabilities are areas under the density, and the total area is $1$ .

In plain words Two numbers fix the whole curve. The mean sets its center and the standard deviation its spread, and a larger $\sigma$ gives a wider and lower curve with the same total area. Calling a variable normal is a modeling assumption. A histogram that looks roughly like a bell makes the model worth trying, but it does not show that the model holds, especially in the tails, where data are scarce.

Example For the constructed fill model $X \sim N(100, 10^2)$ , in units, the density is highest at $x = 100$ , with height $1/(10\sqrt{2\pi}) \approx 0.0399$ per unit. That height is not a probability. Since $X$ is continuous, $P(X = 100) = 0$ , and probabilities come from areas such as $P(90 \le X \le 120)$ . The model also gives positive probability to negative fills, which no container can have. Here that probability is below $10^{-23}$ and does not matter, but for a quantity whose mean lies only one or two standard deviations above zero the same issue can matter a great deal.

Standard normal distribution

Definition The standard normal distribution is $N(0, 1)$ , the normal distribution with mean $0$ and standard deviation $1$ . Its CDF is written $\Phi(z) = P(Z \le z)$ for $Z \sim N(0, 1)$ . If $X \sim N(\mu, \sigma^2)$ with $\sigma > 0$ , then

Z = \frac{X - \mu}{\sigma} \sim N(0, 1).

The z-score of a value $x$ is $z = (x - \mu)/\sigma$ . Because $\sigma > 0$ , the event $X \le x$ is the same event as $Z \le z$ , so $P(X \le x) = \Phi(z)$ . Symmetry of the density gives $\Phi(-z) = 1 - \Phi(z)$ for every real $z$ .

In plain words Standardizing measures a value in standard deviations from the mean. It changes the units on the axis but not which outcomes belong to an event, so the event keeps its probability, and one table of $\Phi$ serves every normal model. Standardizing any variable with a positive standard deviation gives mean $0$ and standard deviation $1$ , but it does not make a variable normal. Reading a probability from $\Phi$ relies on the assumption that $X$ itself is normal.

Example In the fill model the value $120$ has z-score $(120 - 100)/10 = 2$ , so the event $X > 120$ is the event $Z > 2$ . The value $90$ has z-score $-1$ . A table or a checked numerical function gives $\Phi(2) \approx 0.97725$ and $\Phi(-1) \approx 0.15866$ . Dividing by the variance $100$ instead of the standard deviation would give the wrong z-score $0.2$ for the value $120$ .

Tail probability

Definition For a random variable $X$ and a cutoff $c$ , the upper-tail probability is $P(X > c)$ and the lower-tail probability is $P(X < c)$ . If $X \sim N(\mu, \sigma^2)$ and $z = (c - \mu)/\sigma$ , then

\begin{gathered} P(X < c) = \Phi(z), \\[0.4em] P(X > c) = 1 - \Phi(z). \end{gathered}

In a continuous model a single point has probability $0$ , so $<$ and $\le$ give the same answer. For cutoffs $a < b$ with z-scores $z_a$ and $z_b$ , the probability between them is $P(a \le X \le b) = \Phi(z_b) - \Phi(z_a)$ .

In plain words A tail probability is the area beyond a cutoff on one side. Sketch the curve and shade the event before subtracting anything, which guards against using the wrong side or removing a tail twice. Under a stated model the result is a model probability. The proportion of real items beyond the cutoff is a different quantity, and the model predicts it only as well as the model fits.

Example In the fill model the upper tail beyond $120$ units is

\begin{aligned} P(X > 120) &= 1 - \Phi(2) \\[0.4em] &\approx 0.02275, \end{aligned}

about $2.3$ percent. Suppose the tolerance runs from $80$ to $120$ units. The value $80$ has z-score $-2$ , and by symmetry the lower tail below $80$ has the same area, so the model puts about $2 \times 0.02275 = 0.0455$ of fills outside the tolerance, or $4.55$ percent. The interval from $90$ to $120$ has probability

\begin{aligned} &P(90 \le X \le 120) \\[0.4em] &\quad = \Phi(2) - \Phi(-1) \\[0.4em] &\quad \approx 0.97725 - 0.15866 \\[0.4em] &\quad = 0.81859 . \end{aligned}

Normal quantile

Definition Let $X \sim N(\mu, \sigma^2)$ and $0 < q < 1$ . The $q$ quantile of $X$ , also called its $100q$ th percentile, is the unique value $x_q$ with $P(X \le x_q) = q$ . For the standard normal it is $z_q = \Phi^{-1}(q)$ , and for $X$

x_q = \mu + \sigma z_q .

Here $\Phi^{-1}$ is the inverse function of $\Phi$ , often called the inverse CDF. It takes a probability back to a cutoff and is not the reciprocal $1/\Phi$ .

In plain words A tail probability starts from a cutoff and finds an area. A quantile starts from an area to the left and finds the cutoff. To find a cutoff with a given area above it, first convert that area to the area below.

Example For the fill model the 95th percentile uses $z_{0.95} \approx 1.64485$ , so

\begin{aligned} x_{0.95} &\approx 100 + 10(1.64485) \\[0.4em] &\approx 116.45 . \end{aligned}

The model puts 95 percent of fills below about $116.45$ units and 5 percent above. The central interval that holds 95 percent of the model’s probability leaves 2.5 percent in each tail, so it uses $z_{0.975} \approx 1.95996$ and runs from about $80.40$ to $119.60$ units. Its upper endpoint is the 97.5th percentile, not the 95th.

Empirical rule

Definition If $X \sim N(\mu, \sigma^2)$ and $k > 0$ , the probability that $X$ lies within $k$ standard deviations of the mean is

\begin{aligned} &P(\lvert X - \mu \rvert \le k\sigma) \\[0.4em] &\quad = \Phi(k) - \Phi(-k), \end{aligned}

which does not depend on $\mu$ or $\sigma$ . For $k = 1$ , $2$ and $3$ it is about $0.6827$ , $0.9545$ and $0.9973$ . The empirical rule, or 68-95-99.7 rule, rounds these to 68, 95 and 99.7 percent.

In plain words The rule is a quick summary of areas under a normal curve, and its percentages are rounded. Applied to data, with $\bar{x}$ and $s$ in place of $\mu$ and $\sigma$ , it says what a normal model would predict. It is not a guarantee about any finite data set. With 20 observations, for example, the proportion within one standard deviation is a whole number of observations divided by 20, so it can never equal $0.6827$ . Distributions that are not normal can give quite different percentages.

Example In the fill model about 68 percent of the probability lies between $90$ and $110$ units, about 95 percent between $80$ and $120$ , and about 99.7 percent between $70$ and $130$ . For contrast, an exponential distribution with mean $1$ also has standard deviation $1$ , and its probabilities within one, two and three standard deviations of the mean are $1 - e^{-2} \approx 0.8647$ , $1 - e^{-3} \approx 0.9502$ and $1 - e^{-4} \approx 0.9817$ . Its probability beyond three standard deviations is $e^{-4} \approx 0.0183$ , nearly seven times the normal value $0.0027$ , so the rule does not describe this skewed distribution.

Check yourself

Under the fill model $X \sim N(100, 10^2)$ , find the probability that a fill is below $85$ units.

Show the answer

The z-score of $85$ is $(85 - 100)/10 = -1.5$ , so the event $X < 85$ is the event $Z < -1.5$ . By symmetry

\begin{aligned} P(X < 85) &= \Phi(-1.5) \\[0.4em] &= 1 - \Phi(1.5) \\[0.4em] &\approx 1 - 0.93319 \\[0.4em] &= 0.06681, \end{aligned}

about 6.7 percent of fills under the model.

Under the same model, find the cutoff with 10 percent of fills above it. A classmate answers $100 + 10\,\Phi^{-1}(0.10)$ . What went wrong?

Show the answer

Ten percent above the cutoff means 90 percent below it, so the cutoff is the $0.90$ quantile. With $z_{0.90} \approx 1.28155$ ,

\begin{aligned} x_{0.90} &\approx 100 + 10(1.28155) \\[0.4em] &\approx 112.82 \end{aligned}

units. The classmate used the area above as if it were the area below. Since $z_{0.10} \approx -1.28155$ , that formula gives about $87.18$ units, a cutoff with 10 percent of fills below it and 90 percent above.

Waiting times at a clinic cannot be negative, and they have mean 4 minutes and standard deviation 5 minutes. Someone proposes the model $N(4, 5^2)$ to compute the chance of very short and very long waits. Is the model plausible?

Show the answer

No. The value $0$ has z-score $(0 - 4)/5 = -0.8$ , so the model gives $P(X < 0) = \Phi(-0.8) \approx 0.2119$ . About 21 percent of its probability falls on impossible negative waits. A stronger argument needs no table. If a distribution of nonnegative values were symmetric about its mean $4$ , the probability above $8$ would equal the probability below $0$ , which is zero. Every value would then lie between $0$ and $8$ , no value could be more than $4$ from the mean, and the standard deviation would be at most $4$ . A standard deviation of $5$ therefore rules out any symmetric distribution, normal or not. Tail probabilities and quantiles from $N(4, 5^2)$ should not be trusted here, even if a histogram of the waits looked roughly like a bell in its middle.

Normal Distributions and Quantiles

Key terms

Normal distribution

Standard normal distribution

Tail probability

Normal quantile

Empirical rule

Check yourself

How this lesson connects

Builds on

Used later in