The Normal Distribution

Definition Let $\mu$ be a real number and $\sigma > 0$. A random variable $X$ has the normal distribution with mean $\mu$ and variance $\sigma^{2}$, written $X \sim N(\mu, \sigma^{2})$, if it has the density

for every real $x$. Then $E[X] = \mu$, $\operatorname{Var}(X) = \sigma^{2}$ and $\operatorname{SD}(X) = \sigma$. In this course the second entry of $N(\mu, \sigma^{2})$ is always the variance. The support is the whole real line, since every interval of positive length has positive probability. The density is symmetric about $\mu$ and highest there, so the mean, the median and the unique mode all equal $\mu$.

Definition The normal density with parameters $\mu$ and $\sigma > 0$ is the function $f$ of the previous card. It is positive for every real $x$ and symmetric about $\mu$, so $f(\mu + d) = f(\mu - d)$ for every real $d$. It increases up to $x = \mu$, where it takes its largest value $1/(\sigma\sqrt{2\pi})$, and decreases after that, with inflection points at $\mu - \sigma$ and $\mu + \sigma$. Its total area is $1$, which rests on the Gaussian integral

a calculus result used here without proof. Probabilities are areas under $f$, so $P(a \le X \le b)$ is the integral of $f$ from $a$ to $b$ for $a \le b$, and $P(X = x) = 0$ for every real $x$.

Definition The standard normal distribution is $N(0, 1)$, with mean $0$ and standard deviation $1$. A variable with this law is usually called $Z$. Its CDF is

The function $\Phi$ is continuous and strictly increasing, with $\Phi(0) = 0.5$ by symmetry. It cannot be written with elementary functions, so its values come from tables or numerical software and are reported as approximations.

Definition For a random variable $X$ with mean $\mu$ and standard deviation $\sigma > 0$, standardization is the change of variable

which has mean $0$ and standard deviation $1$. If $X \sim N(\mu, \sigma^{2})$, then $Z \sim N(0, 1)$, and conversely $\mu + \sigma Z \sim N(\mu, \sigma^{2})$ whenever $Z \sim N(0, 1)$. So for a normal $X$ and any $a < b$,

Standardization does not change the shape of a distribution, so the standardized variable is normal exactly when $X$ is normal.

Definition For $X \sim N(\mu, \sigma^{2})$ and any $k > 0$,

For $k = 1, 2, 3$ this gives approximately $0.6827$, $0.9545$ and $0.9973$, often rounded to $68$, $95$ and $99.7$ percent. These values hold for every normal distribution, whatever $\mu$ and $\sigma$. They are not general facts about other distributions. For any distribution with finite positive variance, Chebyshev’s inequality guarantees only that at least $1 - 1/k^{2}$ of the probability lies within $k$ standard deviations of the mean, for example at least $0.75$ for $k = 2$.

Definition For $X \sim N(\mu, \sigma^{2})$ and a threshold $c$ with standardized value $z = (c - \mu)/\sigma$, the upper and lower tail probabilities are

For $k \ge 0$, the probability of lying more than $k$ standard deviations from the mean on either side is $P(\lvert X - \mu \rvert > k\sigma) = 2\bigl(1 - \Phi(k)\bigr)$, twice one tail. Every tail probability is positive, however far the threshold lies from $\mu$.