Choosing a Continuous Probability Model

Definition A distribution family is a set of distributions described by one formula with parameters, where each allowed choice of parameter values gives one member. The continuous families compared in this lesson are the uniform laws $\operatorname{Uniform}(a, b)$ with $a < b$, the exponential laws $\operatorname{Exp}(\lambda)$ with $\lambda > 0$, and the normal laws $N(\mu, \sigma^2)$ with real $\mu$ and $\sigma > 0$. Choosing a family is an assumption about how the quantity arises, and choosing the parameter values then picks one member. Each family rests on its own assumption.

• A uniform law on $[a, b]$ gives every subinterval $(c, d)$ of $[a, b]$ the probability $(d - c)/(b - a)$, so subintervals of equal length are equally likely.
• An exponential law is memoryless. A continuous random variable $T$ with values in $[0, \infty)$ and $P(T > t) > 0$ for every $t \ge 0$ satisfies $P(T > t + s \mid T > t) = P(T > s)$ for all $s, t \ge 0$ exactly when $T \sim \operatorname{Exp}(\lambda)$ for some $\lambda > 0$.
• A normal law is symmetric about its mean, gives positive probability to every interval of positive length and has very thin tails. It is a common model for a signed measurement error that is roughly symmetric, although symmetry alone does not make a distribution normal.

Definition A parameter convention states what each parameter of a family means, which values it may take and in which units it is measured. This course uses the following conventions.

• The uniform law $\operatorname{Uniform}(a, b)$ is written with its endpoints $a < b$. Its density is $1/(b - a)$ between $a$ and $b$ and $0$ outside $[a, b]$, its mean is $(a + b)/2$ and its variance is $(b - a)^2/12$.
• The exponential law $\operatorname{Exp}(\lambda)$ is written with its rate $\lambda > 0$, measured per unit of time. Its density is $f(t) = \lambda e^{-\lambda t}$ for $t \ge 0$ and $0$ for $t < 0$, its survival probability is $P(T > t) = e^{-\lambda t}$ for $t \ge 0$, its mean is $1/\lambda$ and its variance is $1/\lambda^2$.
• The normal law $N(\mu, \sigma^2)$ is written with its mean $\mu$ and its variance $\sigma^2 > 0$, so its standard deviation is $\sigma$.

Other books write the exponential law with its mean $\theta = 1/\lambda$, the scale parameter, or put the standard deviation of a normal law in the second place. If a random variable is multiplied by a constant $c > 0$, as in a change of units, then

where $U \sim \operatorname{Uniform}(a, b)$, $T \sim \operatorname{Exp}(\lambda)$ and $X \sim N(\mu, \sigma^2)$. An event that describes the same physical outcome in the new units has the same probability.

Definition The support of a distribution is the set of values $x$ such that every interval $(x - \varepsilon, x + \varepsilon)$ with $\varepsilon > 0$ has positive probability. It is the smallest closed set with probability $1$. The supports of these three families are $[a, b]$ for $\operatorname{Uniform}(a, b)$, the half-line $[0, \infty)$ for $\operatorname{Exp}(\lambda)$ and the whole real line for $N(\mu, \sigma^2)$. Under a continuous model every single value has probability $0$, including the values in the support, so the support is a statement about intervals rather than single points. A model that gives positive probability to an interval of values the quantity cannot take, or probability $0$ to an interval the quantity reaches with positive probability, is not an exact description. At best it is an approximation whose error has to be judged.

Definition The tail behavior of a distribution is the way its tail probabilities $P(X > x)$ and $P(X < -x)$ shrink as $x$ grows. One way to compare tails is the conditional probability $P(X > x + s \mid X > x)$ of going a further distance $s > 0$ past a level $x$ already passed, defined when $P(X > x) > 0$. The three families behave differently.

• For $X \sim \operatorname{Uniform}(a, b)$ the tail probabilities are exactly $0$ above $b$ and below $a$. For $a \le x < b$ the conditional probability equals $1 - s/(b - x)$ when $x + s < b$ and $0$ otherwise, so it falls as $x$ approaches $b$.

• For $T \sim \operatorname{Exp}(\lambda)$ the conditional probability does not depend on the level. For every $x \ge 0$ and $s > 0$,

  $$\begin{aligned} &P(T > x + s \mid T > x) \\[0.4em] &\quad = \frac{e^{-\lambda(x + s)}}{e^{-\lambda x}} = e^{-\lambda s}. \end{aligned}$$

  This is the memoryless property.

• For $X \sim N(\mu, \sigma^2)$ the conditional probability tends to $0$ as $x \to \infty$, for every fixed $s > 0$, and the left tail behaves the same way by symmetry. Far out, a normal tail shrinks faster than any exponential tail.

Models that agree near the center can differ greatly in the tails, so tail behavior matters most when the question is about rare or extreme values.

Definition Model adequacy is the judgment that a model’s answers are accurate enough for a stated purpose, especially for the events that purpose depends on. It is separate from mathematical correctness, which asks only whether the calculations follow from the stated model. A careful choice states the quantity and its units, the values it can take, the mechanism and the assumptions behind the family, and the parameters with their convention. It then checks the consequences that matter, such as the support, the tails and the probabilities a decision uses, against what is known. When data are available, the checks look at bounds, asymmetry, extreme values, changes over time and dependence between observations, not only at the average.