Transformations and Inverse-CDF Sampling

Definition Let $X$ be a random variable with a known distribution, let $g$ be a function defined at every value $X$ can take, and let $Y = g(X)$. The CDF method finds the CDF of $Y$ at each real $y$ from

where $A_y = \{x : g(x) \le y\}$ is the set of all inputs whose output is at most $y$, the preimage of $(-\infty, y]$. First find the set of values $Y$ can take. Below all of them $F_Y(y) = 0$, and at or above all of them $F_Y(y) = 1$. For the remaining $y$, solve the inequality $g(x) \le y$ for $x$ and compute $P(X \in A_y)$ from the distribution of $X$. If the resulting $F_Y$ is continuous everywhere and differentiable at all but finitely many points, then $f_Y = F_Y'$ is a density of $Y$, with any values at the exceptional points.

Definition Let $X$ take its values in an interval $I$, let $g$ be continuous and strictly monotone on $I$, and let $Y = g(X)$. If $g$ is strictly increasing, then for every $x_0$ in $I$ the events $\{g(X) \le g(x_0)\}$ and $\{X \le x_0\}$ are equal, so

If $g$ is strictly decreasing, the inequality reverses. The event $\{g(X) \le g(x_0)\}$ equals $\{X \ge x_0\}$, so

which equals $1 - F_X(x_0)$ when $X$ is continuous. Each $y$ in the image $g(I)$ is $g(x_0)$ for exactly one $x_0$ in $I$. For $y$ outside $g(I)$, $F_Y(y)$ is $0$ when $y$ lies below $g(I)$ and $1$ when $y$ lies above it.

Definition A function $g$ is many-to-one on the values of $X$ when some output is produced by two or more different inputs that $X$ can take. Then an output event can come from inputs on several branches of the inverse, and $F_Y(y)$ must include the probability of every input $x$ with $g(x) \le y$, on every branch. For $Y = X^2$, $F_Y(y) = 0$ when $y < 0$, and for $y \ge 0$

So for a continuous $X$ and $y \ge 0$,

Definition A density $f$ is unbounded when it takes arbitrarily large values, that is, for every number $M$ there is a point $y$ with $f(y) > M$. This is allowed. A density only has to be nonnegative with total integral $1$, so its graph may rise without bound near a point as long as the area under it stays finite. A random variable with an unbounded density is still continuous, so every single value has probability $0$.

Definition Let $U \sim \operatorname{Uniform}(0, 1)$, so $P(U \le u) = u$ for $0 \le u \le 1$. Let $F$ be a CDF that is continuous and strictly increasing on an interval $(a, b)$, where $a$ may be $-\infty$ and $b$ may be $\infty$, with $F(x) \to 0$ as $x \to a$ and $F(x) \to 1$ as $x \to b$. Then $F$ maps $(a, b)$ one to one onto $(0, 1)$, and since $U$ lies in $(0, 1)$ with probability $1$, the random variable $X = F^{-1}(U)$ is defined with probability $1$. It has CDF $F$. For $x$ in $(a, b)$ the events $\{F^{-1}(U) \le x\}$ and $\{U \le F(x)\}$ agree whenever $U$ lies in $(0, 1)$, so

and for $x \le a$ or $x \ge b$ both sides are $0$ or both are $1$. Generating $X$ this way is inverse-CDF sampling. The next card removes the conditions of continuity and strict increase. The reverse statement, that $F(X)$ is $\operatorname{Uniform}(0, 1)$ when $X$ has CDF $F$, holds whenever $F$ is continuous, but it fails for discrete distributions.

Definition For any CDF $F$ and any $u$ with $0 < u < 1$, the generalized quantile is

The set is nonempty and bounded below, because $F$ tends to $1$ on the right and to $0$ on the left, and right-continuity of $F$ makes the infimum attained, so $F(Q(u)) \ge u$. For every real $x$ and every $u$ in $(0, 1)$,

Hence $X = Q(U)$ has CDF $F$ when $U \sim \operatorname{Uniform}(0, 1)$, with no continuity or strict increase required. When $F$ is continuous and strictly increasing where $0 < F < 1$, the generalized quantile is the ordinary inverse $F^{-1}$ of the previous card.