Derivatives and the Density-CDF Connection, introduction to probability lesson

Suppose a random variable $X$ between $0$ and $1$ is described by its cumulative distribution function, $F(x) = x^2$ for $0 \le x \le 1$ , and no density is given. The CDF says how much probability has accumulated up to each threshold. How densely is probability packed near the threshold $0.4$ ? That is a question about how fast the accumulated probability grows there, and rates of change are measured by derivatives. This lesson defines the derivative as a limit of difference quotients, links it to integration through the fundamental theorem of calculus, and shows when differentiating a CDF recovers a density and when it cannot.

Key terms

Difference quotient

Definition Let $F$ be a function defined at $x$ and at $x + h$ , where $h \ne 0$ . The difference quotient of $F$ at $x$ with step $h$ is

\frac{F(x + h) - F(x)}{h}.

It is the slope of the secant line through the points $(x, F(x))$ and $(x + h, F(x + h))$ on the graph of $F$ . When $F$ is the CDF of a random variable $X$ and $h > 0$ , the numerator is $P(x < X \le x + h)$ , so the quotient is the average probability per unit of $x$ over the interval $(x, x + h]$ .

In plain words Move the threshold by a small step, see how much the function changes, and divide that change by the step, counting a step to the left as negative. For a CDF, the result is the probability of the short interval between the two thresholds per unit of width.

Example Let $X$ have the CDF

F(x) = \begin{cases} 0, & x < 0, \\ x^2, & 0 \le x \le 1, \\ 1, & x > 1. \end{cases}

When $x$ and $x + h$ both lie in $[0, 1]$ , the squared terms cancel,

\begin{aligned} &\frac{F(x + h) - F(x)}{h} \\[0.4em] &\quad = \frac{(x + h)^2 - x^2}{h} \\[0.4em] &\quad = \frac{2xh + h^2}{h} = 2x + h. \end{aligned}

At $x = 0.4$ the quotient is $0.8 + h$ . For $h = 0.1$ the numerator is $P(0.4 < X \le 0.5) = 0.25 - 0.16 = 0.09$ , and dividing by the width $0.1$ gives $0.9$ .

Step $h$	Difference quotient at $0.4$
$0.1$	$0.9$
$0.01$	$0.81$
$0.001$	$0.801$
$-0.001$	$0.799$
$-0.01$	$0.79$
$-0.1$	$0.7$

Derivative

Definition Let $F$ be defined on an open interval containing $x$ . If the difference quotients approach a limit as $h \to 0$ through nonzero values of both signs, then $F$ is differentiable at $x$ , and the limit

\begin{aligned} &F'(x) \\[0.4em] &\quad = \lim_{h \to 0} \frac{F(x + h) - F(x)}{h} \end{aligned}

is the derivative of $F$ at $x$ . The line through $(x, F(x))$ with slope $F'(x)$ is the tangent line, and $F'(x)$ is the tangent slope. If the limit does not exist, $F$ has no derivative and no tangent slope at $x$ . The derivative is measured in units of $F$ per unit of $x$ .

In plain words The derivative is the slope that the secant lines settle on as the second point slides toward the first from either side. It measures how fast $F$ changes at the point itself, not over a stretch.

Example For the CDF above, the quotients at $0.4$ equal $0.8 + h$ , which tends to $0.8$ from both sides, so $F'(0.4) = 0.8$ . The tangent line at $(0.4, 0.16)$ has slope $0.8$ , and if $x$ is measured in hours the derivative is $0.8$ per hour. The same computation gives $F'(x) = 2x$ for every $x$ with $0 < x < 1$ . At $x = 1$ there is no derivative. For $h < 0$ the quotient is $2 + h$ , which tends to $2$ , while for $h > 0$ it is $(1 - 1)/h = 0$ . The graph has a corner there, with slope $2$ on the left and slope $0$ on the right.

Power rule for derivatives

Definition For every positive integer $n$ , the function $x^n$ is differentiable at every real $x$ , and

\frac{d}{dx}\, x^n = n x^{n-1},

where $x^0$ means $1$ , so the derivative of $x$ is $1$ . A constant function has derivative $0$ everywhere. Derivatives respect constant multiples and sums, so $(cF)'(x) = cF'(x)$ and $(F + G)'(x) = F'(x) + G'(x)$ at every $x$ where $F$ and $G$ are differentiable. Together these rules differentiate every polynomial.

In plain words Bring the exponent down as a factor and lower the exponent by one. A constant does not change, so its rate of change is zero.

Example For $n = 3$ , expanding $(x + h)^3 = x^3 + 3x^2h + 3xh^2 + h^3$ gives

\begin{aligned} &\frac{(x + h)^3 - x^3}{h} \\[0.4em] &\quad = 3x^2 + 3xh + h^2, \end{aligned}

which tends to $3x^2$ as $h \to 0$ . So the derivative of $x^3$ is $3x^2$ , and adding a constant does not change it, since $x^3 + 2$ also has derivative $3x^2$ . With the multiple and sum rules, the polynomial $3x^2 - 2x^3$ has derivative $6x - 6x^2$ .

Fundamental theorem of calculus

Definition Let $f$ be continuous on $[a, b]$ with $a < b$ . The theorem has two parts.

The first part says that the accumulated area $G(x) = \int_a^x f(t)\,dt$ , defined for $a \le x \le b$ , is differentiable at every $x$ with $a < x < b$ , and $G'(x) = f(x)$ . If $f$ is only integrable on $[a, b]$ , for example bounded with finitely many jumps, the conclusion $G'(x) = f(x)$ still holds at each $x$ with $a < x < b$ where $f$ is continuous.

The second part says that if $G$ is continuous on $[a, b]$ and $G'(x) = f(x)$ for every $x$ with $a < x < b$ , then

\int_a^b f(x)\,dx = G(b) - G(a).

In plain words Differentiating accumulated area gives back the height of the curve wherever the curve is continuous. In the other direction, to integrate a continuous function over an interval it is enough to know any function whose rate of change equals the integrand throughout that interval, and to subtract its values at the two limits.

Example For $f(t) = 2t$ on $[0, 1]$ , the accumulated area is $\int_0^x 2t\,dt = x^2$ , and its derivative $2x$ is the height $f(x)$ , as the first part says. For the second part, take the continuous integrand $6x - 6x^2$ on $[0, 1]$ . The power rule shows that $G(x) = 3x^2 - 2x^3$ has derivative $6x - 6x^2$ , so

\begin{aligned} &\int_0^{1/2} (6x - 6x^2)\,dx \\[0.4em] &\quad = G\left(\tfrac12\right) - G(0) \\[0.4em] &\quad = \left(\tfrac34 - \tfrac14\right) - 0 = \tfrac12. \end{aligned}

Antiderivative

Definition Let $f$ be a function on an open interval $I$ . A function $G$ is an antiderivative of $f$ on $I$ if $G'(x) = f(x)$ for every $x$ in $I$ . If $G$ is one antiderivative of $f$ on $I$ , then the antiderivatives of $f$ on $I$ are exactly the functions $G + C$ with $C$ a constant. In the second part of the fundamental theorem the constant cancels, because

\begin{aligned} &\bigl(G(b) + C\bigr) - \bigl(G(a) + C\bigr) \\[0.4em] &\quad = G(b) - G(a). \end{aligned}

In plain words An antiderivative undoes differentiation. Functions with the same rate of change on an interval differ only by a vertical shift, and the shift drops out when you subtract values at two limits.

Example On the whole real line, $x^3$ and $x^3 + 2$ are both antiderivatives of $3x^2$ . Either one gives

\int_0^{1/2} 3x^2\,dx = \tfrac18,

since $\bigl(\tfrac18 + 2\bigr) - (0 + 2) = \tfrac18$ . They are not equally good as CDFs, though. For a density $f$ that is continuous on $[c, d]$ and $0$ outside it, the CDF on $[c, d]$ is $\int_c^x f(t)\,dt$ , the antiderivative that starts at $0$ at $c$ . For the density $3x^2$ on $[0, 1]$ that is $x^3$ , while $x^3 + 2$ starts at $2$ and is no CDF at all.

Density as the derivative of the CDF

Definition Let $X$ have density $f$ and CDF $F$ , so that $F(x) = \int_{-\infty}^{x} f(t)\,dt$ for every real $x$ . By the fundamental theorem, $F'(x) = f(x)$ at every $x$ where $f$ is continuous. In the other direction, suppose the CDF $F$ of a random variable $X$ is continuous everywhere, and that outside a finite set of points $F$ is differentiable with a continuous derivative. Then setting $f(x) = F'(x)$ outside that set, and choosing any nonnegative values at its points, gives a density of $X$ . Changing a density at finitely many points changes no integral, so all such choices give the same probabilities. If $F$ jumps at $c$ , then $P(X = c) > 0$ , and $X$ has no density at all, because a density gives every single value probability $0$ . Continuity alone is not enough either. The Cantor function is a continuous CDF that rises from $0$ to $1$ on $[0, 1]$ , yet it is constant on intervals of total length $1$ . Its derivative is $0$ on all of those intervals, the remaining points have total length $0$ , and no density produces this CDF.

In plain words When a variable has a density, the slope of its CDF at each point where the CDF is smooth is the density there, the local rate at which probability accumulates. A density value is not the probability of a point. Near a point $x$ where $f$ is continuous, a short interval of width $h$ has probability close to $f(x)h$ . Differentiation cannot see probability that sits on a single value, because that probability shows up as a jump, not as a slope.

Example For the CDF $F(x) = x^2$ on $[0, 1]$ , the derivative is $2x$ for $0 < x < 1$ and $0$ for $x < 0$ and $x > 1$ . At $0$ both one-sided slopes are $0$ , so $F'(0) = 0$ , and at the corner $x = 1$ there is no derivative. So $f(x) = 2x$ for $0 \le x \le 1$ and $f(x) = 0$ otherwise is a density of $X$ , and setting $f(1) = 0$ instead would change no probability. The value $F'(0.4) = 0.8$ is not $P(X = 0.4)$ , which is $0$ . It says that

\begin{aligned} &P(0.4 < X \le 0.41) \\[0.4em] &\quad = 0.41^2 - 0.4^2 = 0.0081 \end{aligned}

is close to $0.8 \cdot 0.01 = 0.008$ .

Check yourself

A random variable $X$ has CDF $F(x) = 0$ for $x < 0$ , $F(x) = x^3$ for $0 \le x \le 1$ and $F(x) = 1$ for $x > 1$ . Find a density of $X$ . Where does $F$ fail to have a derivative, and does that matter?

Show the answer

By the power rule, $F'(x) = 3x^2$ for $0 < x < 1$ , and $F'(x) = 0$ for $x < 0$ and for $x > 1$ , where $F$ is constant. At $x = 0$ both one-sided slopes are $0$ , so $F'(0) = 0$ . At $x = 1$ the quotients from the left are $3 + 3h + h^2$ , which tend to $3$ , while those from the right are $0$ , so $F$ has a corner and no derivative there. A density is

f(x) = \begin{cases} 3x^2, & 0 \le x \le 1, \\ 0, & \text{otherwise}, \end{cases}

and its total area is $1^3 - 0^3 = 1$ . The missing derivative at the single point $1$ does not matter, because any nonnegative value of $f(1)$ gives the same integrals and so the same probabilities.

For the same $X$ , find $P(\tfrac14 < X < \tfrac12)$ in two ways, from the CDF and by integrating the density.

Show the answer

From the CDF, $F(\tfrac12) - F(\tfrac14) = \tfrac18 - \tfrac{1}{64} = \tfrac{7}{64}$ . Leaving out the right endpoint changes nothing, since a single point has probability $0$ . By integration, $x^3$ is an antiderivative of $3x^2$ , so

\begin{aligned} \int_{1/4}^{1/2} 3x^2\,dx &= \left(\tfrac12\right)^3 - \left(\tfrac14\right)^3 \\[0.4em] &= \tfrac{8}{64} - \tfrac{1}{64} = \tfrac{7}{64}. \end{aligned}

The two routes agree, as the fundamental theorem says they must.

A random variable $Y$ has CDF $G(y) = 0$ for $y < 0$ , $G(y) = (1 + y)/2$ for $0 \le y < 1$ and $G(y) = 1$ for $y \ge 1$ . A classmate differentiates and proposes the density $g(y) = \tfrac12$ for $0 < y < 1$ and $g(y) = 0$ otherwise. What goes wrong?

Show the answer

The total area under $g$ is $\tfrac12 \cdot 1 = \tfrac12$ , not $1$ , so $g$ is not a density. The CDF jumps at $0$ , from $0$ just to the left to $G(0) = \tfrac12$ , so $P(Y = 0) = \tfrac12$ . Differentiating sees only the slope $\tfrac12$ on $(0, 1)$ and misses the probability that sits on the single value $0$ . Since every density gives each single value probability $0$ , the variable $Y$ has no density at all. Its distribution combines a point mass at $0$ with probability spread over $(0, 1)$ .

Derivatives and the Density-CDF Connection

Key terms

Difference quotient

Derivative

Power rule for derivatives

Fundamental theorem of calculus

Antiderivative

Density as the derivative of the CDF

Check yourself

How this lesson connects

Builds on

Used later in