Normal Probabilities and Quantiles, introduction to probability lesson

A normal model answers two kinds of questions. Given a threshold, what is the probability of landing below it, above it or between two thresholds? Given a probability, which threshold has exactly that probability below it? This lesson computes interval and tail probabilities with the standard normal CDF and its symmetry, then reverses the calculation to find quantiles and central intervals, with every threshold returned to the original units.

Key terms

Z-score

Definition For a random variable $X$ with mean $\mu$ and standard deviation $\sigma > 0$ , the z-score of a value $x$ is

z = \frac{x - \mu}{\sigma}.

It has no units. If $X \sim N(\mu, \sigma^{2})$ and $Z = (X - \mu)/\sigma$ , then $Z \sim N(0, 1)$ , and because $\sigma > 0$ the events $\{X \le x\}$ and $\{Z \le z\}$ are the same event. Hence $P(X \le x) = \Phi(z)$ , and the event $a < X < b$ is the event that $Z$ lies between the z-scores of $a$ and $b$ .

In plain words A z-score says how many standard deviations a value lies from the mean, positive above the mean and negative below it. It divides by the standard deviation, not the variance. A z-score can be computed for any variable with a positive standard deviation, but reading a probability from $\Phi$ relies on the assumption that $X$ is normal.

Example A constructed measurement $X$ has a normal model with mean $50$ and standard deviation $4$ , so $X \sim N(50, 16)$ . The values $46$ , $48$ , $54$ and $56$ have z-scores $-1$ , $-0.5$ , $1$ and $1.5$ , and the event $46 < X < 54$ is the event $-1 < Z < 1$ . Dividing by the variance $16$ instead would give the wrong z-score $0.25$ for the value $54$ .

Standard normal CDF

Definition The standard normal CDF is $\Phi(z) = P(Z \le z)$ for $Z \sim N(0, 1)$ and every real $z$ . It is continuous and strictly increasing, with $0 < \Phi(z) < 1$ , and $\Phi(z)$ tends to $0$ as $z \to -\infty$ and to $1$ as $z \to \infty$ . For $X \sim N(\mu, \sigma^{2})$ and $a < b$ ,

\begin{aligned} &P(a < X < b) \\[0.4em] &\quad = \Phi\Bigl(\frac{b - \mu}{\sigma}\Bigr) \\[0.4em] &\qquad - \Phi\Bigl(\frac{a - \mu}{\sigma}\Bigr). \end{aligned}

Since $P(X = a) = P(X = b) = 0$ , the same value holds with $\le$ at either endpoint.

In plain words Every value of $\Phi$ is an area to the left. The area to the left of the upper z-score contains the wanted interval and the unwanted lower tail, and subtracting the area to the left of the lower z-score removes that tail. Endpoints make no difference in a continuous model. In a discrete count, an endpoint that has positive probability does change the answer.

Example In the model $N(50, 16)$ ,

\begin{aligned} &P(46 < X < 54) \\[0.4em] &\quad = \Phi(1) - \Phi(-1) \\[0.4em] &\quad \approx 0.8413447 - 0.1586553 \\[0.4em] &\quad \approx 0.682689, \end{aligned}

about $68.27$ percent. Values of $\Phi$ from a table or software are numerical approximations, so keep extra digits until the end and round only the final answer.

Symmetry of the standard normal CDF

Definition The standard normal density is symmetric about $0$ , so $P(Z \le -z) = P(Z \ge z)$ and

\Phi(-z) = 1 - \Phi(z)

for every real $z$ . In particular $\Phi(0) = 0.5$ . For $X \sim N(\mu, \sigma^{2})$ and every $d \ge 0$ this gives $P(X \le \mu - d) = P(X \ge \mu + d)$ . Symmetry relates events that are mirror images about the mean. Two intervals of the same length need not have the same probability.

In plain words A table that lists $\Phi$ only for positive $z$ is enough, since the value at $-z$ is the leftover area $1 - \Phi(z)$ . Keep the sign of every z-score until the end. Dropping a minus sign replaces a threshold below the mean by its mirror image above the mean and changes the event.

Example In the model $N(50, 16)$ , the thresholds $42$ and $58$ lie two standard deviations below and above the mean, so $P(X < 42) = \Phi(-2) = 1 - \Phi(2)$ , which equals $P(X > 58)$ , about $0.022750$ . The interval from $48$ to $56$ has the same length as the interval from $46$ to $54$ , but it is not centered at the mean. Its z-scores are $-0.5$ and $1.5$ , and

\begin{aligned} &P(48 < X < 56) \\[0.4em] &\quad = \Phi(1.5) - \bigl(1 - \Phi(0.5)\bigr) \\[0.4em] &\quad \approx 0.933193 - 0.308538 \\[0.4em] &\quad \approx 0.624655, \end{aligned}

less than the $0.682689$ of the centered interval. Among all intervals of length $8$ , the one centered at the mean has the largest probability.

Upper-tail probability

Definition For $X \sim N(\mu, \sigma^{2})$ and a threshold $c$ with z-score $z = (c - \mu)/\sigma$ , the upper-tail probability is

\begin{aligned} P(X > c) &= 1 - \Phi(z) \\[0.4em] &= \Phi(-z). \end{aligned}

It equals $P(X \ge c)$ , lies strictly between $0$ and $1$ , and decreases as the threshold $c$ increases.

In plain words The CDF gives the area to the left, and the upper tail is its complement. Reading $\Phi(z)$ itself as the answer to an “above” question gives the probability on the other side of the threshold, a common slip that shows up as an answer close to $1$ for a threshold well above the mean.

Example In the model $N(50, 16)$ , the threshold $58$ has z-score $2$ , so $P(X > 58) = 1 - \Phi(2) \approx 0.022750$ , about $2.3$ percent. The value $\Phi(2) \approx 0.977250$ is $P(X \le 58)$ , the probability of the other side.

Normal quantile and inverse CDF

Definition For $0 < p < 1$ , the $p$ -quantile of $X \sim N(\mu, \sigma^{2})$ is the number $q_p$ with $P(X \le q_p) = p$ . It exists and is unique because the normal CDF is continuous and strictly increasing on the whole real line and takes every value strictly between $0$ and $1$ . The function $p \mapsto q_p$ is the inverse CDF, and the $0.9$ -quantile is also called the $90$ th percentile. For the standard normal the inverse CDF is written $z_p = \Phi^{-1}(p)$ , and undoing standardization gives

q_p = \mu + \sigma z_p.

Symmetry gives $z_{1 - p} = -z_p$ . No real number has cumulative probability exactly $0$ or $1$ , so $q_p$ is defined only for $0 < p < 1$ .

In plain words A quantile reverses the usual direction. Instead of starting from a threshold and finding the probability below it, start from the probability and find the threshold. First find the standard normal z-score with the required area to its left, then multiply by $\sigma$ and add $\mu$ to return to the original units. Translate the wording before any lookup. A threshold exceeded with probability $0.1$ has $0.9$ below it, so it is the $0.9$ -quantile, while a threshold exceeded with probability $0.9$ is the $0.1$ -quantile.

Example In the model $N(50, 16)$ , software gives $z_{0.9} \approx 1.281552$ , so the $90$ th percentile is

\begin{aligned} q_{0.9} &= 50 + 4(1.281552) \\[0.4em] &\approx 55.1262. \end{aligned}

Substituting back, $(55.1262 - 50)/4 \approx 1.2816$ , and $\Phi(1.2816) \approx 0.9$ . The $10$ th percentile is $50 - 4(1.281552) \approx 44.8738$ , the mirror image of the $90$ th percentile about the mean.

Central probability interval

Definition For $X \sim N(\mu, \sigma^{2})$ and $0 < c < 1$ , the central interval with probability $c$ runs from $q_{(1 - c)/2}$ to $q_{(1 + c)/2}$ . It leaves probability $(1 - c)/2$ in each tail, and it equals

\mu \pm \sigma z_{(1 + c)/2}.

Among all intervals with probability $c$ under this model, it is the shortest. It is a statement about a single future value of $X$ under a model whose parameters are given. It is not a confidence interval for an unknown mean, which is built from data and needs further assumptions.

In plain words Split the leftover probability $1 - c$ equally between the two tails, then find the two quantiles. For $95$ percent each tail gets $0.025$ , so the endpoints are the $0.025$ -quantile and the $0.975$ -quantile.

Example In the model $N(50, 16)$ , the central interval with probability $0.95$ uses $z_{0.975} \approx 1.959964$ , so it is

\begin{aligned} &50 \pm 4(1.959964) \\[0.4em] &\quad \approx [42.1601,\ 57.8399]. \end{aligned}

It is slightly narrower than $\mu \pm 2\sigma = [42, 58]$ , whose probability is about $0.9545$ rather than $0.95$ .

Check yourself

A constructed model says $Y$ is normal with mean $80$ and standard deviation $5$ . Find $P(Y > 90)$ .

Show the answer

The model is $N(80, 25)$ , and the threshold $90$ has z-score $(90 - 80)/5 = 2$ . So

\begin{aligned} P(Y > 90) &= 1 - \Phi(2) \\[0.4em] &\approx 0.022750, \end{aligned}

about $2.3$ percent.

In the same model, find the threshold that $Y$ exceeds with probability $0.05$ , and check that it fits with the first answer.

Show the answer

Probability $0.05$ above means probability $0.95$ below, so the threshold is the $0.95$ -quantile. With $z_{0.95} \approx 1.644854$ ,

\begin{aligned} q_{0.95} &= 80 + 5(1.644854) \\[0.4em] &\approx 88.2243. \end{aligned}

It lies below $90$ , as it must, because its upper tail $0.05$ is larger than the upper tail $0.0228$ above $90$ . Using $z_{0.05} \approx -1.644854$ by mistake would give about $71.7757$ , a threshold that $Y$ exceeds with probability $0.95$ .

In the same model, find $P(70 < Y < 85)$ . Why does the shortcut $2\Phi(k) - 1$ not apply here?

Show the answer

The z-scores are $(70 - 80)/5 = -2$ and $(85 - 80)/5 = 1$ , so

\begin{aligned} &P(70 < Y < 85) \\[0.4em] &\quad = \Phi(1) - \bigl(1 - \Phi(2)\bigr) \\[0.4em] &\quad \approx 0.841345 - 0.022750 \\[0.4em] &\quad \approx 0.818595. \end{aligned}

The shortcut $2\Phi(k) - 1$ gives the probability of an interval centered at the mean, from $\mu - k\sigma$ to $\mu + k\sigma$ . This interval reaches two standard deviations below the mean but only one above, so neither $k = 1$ , which gives about $0.682689$ , nor $k = 2$ , which gives about $0.954500$ , is correct.

Normal Probabilities and Quantiles

Key terms

Z-score

Standard normal CDF

Symmetry of the standard normal CDF

Upper-tail probability

Normal quantile and inverse CDF

Central probability interval

Check yourself

How this lesson connects

Builds on

Used later in

Where this leads in Statistics