Random Variables and Probability Distributions

Definition A random variable $X$ is a function that assigns a real number $X(\omega)$ to each outcome $\omega$ in the sample space $\Omega$. Its distribution gives the probabilities of the events it defines, such as $\{X \le t\}$ or $\{a \le X \le b\}$. The value that occurs in a particular trial is a realization of $X$, often written in lowercase as $x$. The variable $X$ is discrete if, with probability $1$, its values lie in a finite or countable set, and its support is then the set of values $x$ with $P(X = x) > 0$. It is absolutely continuous if the probabilities of intervals are areas under a density, as defined below. Some variables are neither, such as a mix of a point mass and a density, but the two kinds are enough for this lesson.

Definition The probability mass function, or PMF, of a discrete random variable $X$ is $p_X(x) = P(X = x)$. It satisfies $p_X(x) \ge 0$ for every $x$ and

with the sum over the support, an infinite series when the support is infinite. For a set of values $A$, the probability $P(X \in A)$ is the sum of $p_X(x)$ over the values $x$ in $A$. In summation notation,

with one term for each value of the index $i$ from $1$ to $n$. The index is only a placeholder, so $\sum_{i=1}^{n} a_i$ and $\sum_{j=1}^{n} a_j$ are the same number.

Definition The cumulative distribution function, or CDF, of a random variable $X$ is

for every real number $t$. It never decreases, it tends to $0$ as $t \to -\infty$ and to $1$ as $t \to \infty$, and for $a < b$

At each value $x$ the jump of $F$ equals $P(X = x)$. So for a discrete variable with finitely many values, $F$ is a step function that rises by $p_X(x)$ at each support value $x$ and already takes the higher value at $x$ itself.

Definition For a nonnegative, piecewise continuous function $f$ and numbers $a \le b$, the definite integral

is the area of the region under the graph of $f$ and above the horizontal axis between $x = a$ and $x = b$. When $f$ has a constant value $c$ on $[a, b]$ the region is a rectangle with area $c\,(b - a)$. For other shapes the area is the limit of the total area of narrow rectangles under the graph as their widths shrink to $0$. A probability density function, or PDF, is such a function $f$ with total area $1$ under its whole graph. A random variable $X$ has density $f$, and is absolutely continuous, if for all $a \le b$

Then $P(X = a) = 0$ for every single value $a$, so including or excluding an endpoint does not change the probability of an interval, and $F(t)$ is the area under $f$ to the left of $t$. If $X$ carries a unit, $f$ carries the reciprocal unit, so that height times width is a pure number.

Definition A Bernoulli random variable takes the value $1$ with probability $p$ and the value $0$ with probability $1 - p$, where $0 \le p \le 1$. The value $1$ is called a success, whatever the event coded $1$ means. Let $K$ count the successes in a fixed number $n$ of Bernoulli trials that are independent and share the same success probability $p$. Then $K$ has the binomial distribution with parameters $n$ and $p$, and for $k = 0, 1, \dots, n$

where the binomial coefficient $\binom{n}{k} = \frac{n!}{k!\,(n - k)!}$ counts the ways to choose which $k$ of the $n$ trials are successes, and the convention $0^0 = 1$ applies when $p$ is $0$ or $1$. A categorical distribution extends the Bernoulli model to one of $m$ named categories, with probabilities that are nonnegative and sum to $1$.

Definition For a real number $\mu$ and a number $\sigma > 0$, a random variable $X$ has the normal distribution with mean $\mu$ and standard deviation $\sigma$, written $X \sim N(\mu, \sigma^2)$, if it has the density

for every real $x$. The number $z$ is the signed distance from $x$ to $\mu$ in units of $\sigma$. The density is symmetric about $\mu$, highest at $x = \mu$ with height $1/(\sigma\sqrt{2\pi})$, and positive on the whole real line. The CDF has no formula in elementary functions, so interval probabilities come from software or tables. About $0.6827$ of the probability lies within one $\sigma$ of $\mu$, about $0.9545$ within two and about $0.9973$ within three. The parameters $\mu$ and $\sigma$ are the mean and standard deviation of the distribution, quantities defined through expectation in the next lesson.