Integrals and Probability as Area, introduction to probability lesson

A computer job finishes at some time between $0$ and $1$ hour after it starts, and in a constructed model a stretch of later times is more likely than an equally long stretch of earlier times. What is the probability that it finishes in the second half of the hour? A discrete model would list a few possible times and give each one a probability, but here every time in the interval is allowed. So the model spreads probability along the time axis with a curve, and the probability of an interval is the area under the curve over that interval. This lesson builds that area from rectangles, defines the definite integral, and evaluates the integrals of powers that simple densities need.

Key terms

Probability density

Definition A probability density is a function $f$ defined for every real number $x$ with two properties. It is nonnegative, $f(x) \ge 0$ for every $x$ , and the total area between its graph and the horizontal axis is $1$ . The definite integral below makes this area precise, and the total area is written

\int_{-\infty}^{\infty} f(x)\,dx = 1.

When $f$ is $0$ outside an interval $[c, d]$ , the total area is the area over $[c, d]$ . If the values of $x$ carry a unit, such as hours, then $f$ carries the reciprocal unit, per hour, so that a height times a width is a pure number.

In plain words A density shows how probability is spread along a scale. Its height is probability per unit of width, not a probability. Near a point where the curve is continuous, the height times a short width is approximately the probability of that short interval, so a tall curve over a very narrow interval can still hold very little probability, and a height above $1$ breaks no rule.

Example In the constructed job model, the finishing time $X$ in hours has density

f(x) = \begin{cases} 2x, & 0 \le x \le 1, \\ 0, & \text{otherwise}. \end{cases}

The graph is a triangle with base $1$ hour and height $2$ per hour, so its area is $\tfrac12 \cdot 1 \cdot 2 = 1$ , and the hours cancel. The height $2$ at $x = 1$ is a rate, not a probability of $2$ . The clause ” $0$ otherwise” is part of the model. The formula $2x$ on its own is negative for $x < 0$ and encloses more and more area as $x$ grows, so it is not a density.

Probability as area

Definition A random variable $X$ has density $f$ when, for all real numbers $a \le b$ ,

\begin{aligned} &P(a \le X \le b) \\[0.4em] &\quad = \int_a^b f(x)\,dx, \end{aligned}

the area between the graph of $f$ and the horizontal axis over $[a, b]$ . Taking $a = b$ gives an interval of width $0$ and area $0$ , so $P(X = a) = 0$ for every $a$ . Hence including or excluding an endpoint does not change the probability, and $P(a < X < b)$ , $P(a < X \le b)$ and $P(a \le X < b)$ equal the same area.

In plain words In a density model, probability lives in areas over intervals. To find the probability of an event about $X$ , shade the part of the horizontal axis that the event describes and measure the area under the curve above it.

Example For the job model, the probability of finishing in the second half of the hour is the area over $[\tfrac12, 1]$ . That region is a trapezoid with width $\tfrac12$ and parallel sides $f(\tfrac12) = 1$ and $f(1) = 2$ , so

\begin{aligned} P\left(\tfrac12 \le X \le 1\right) &= \tfrac12 \cdot \frac{1 + 2}{2} \\[0.4em] &= \tfrac34. \end{aligned}

The first half of the hour is a triangle with base $\tfrac12$ and height $1$ , so it has probability $\tfrac14$ . The two halves have the same width but different probabilities, because the curve is higher over the second half.

Riemann sum

Definition Let $f$ be a function on an interval $[a, b]$ with $a < b$ . Split $[a, b]$ into $n$ pieces with endpoints $a = x_0 < x_1 < \dots < x_n = b$ , choose a point $x_i^{*}$ in each piece $[x_{i-1}, x_i]$ , and write $\Delta x_i = x_i - x_{i-1}$ for its width. The number

\sum_{i=1}^{n} f(x_i^{*})\,\Delta x_i

is a Riemann sum for $f$ on $[a, b]$ . With $n$ equal widths $\Delta x = (b - a)/n$ , choosing every left endpoint gives the left sum $L_n$ , and choosing every right endpoint gives the right sum $R_n$ . When $f$ is nonnegative, each term is the area of a rectangle. When $f$ is also nondecreasing on $[a, b]$ , every rectangle of a left sum lies under the graph and every rectangle of a right sum covers the region beneath the graph on its piece, so the left sum is at most the area and the right sum is at least the area.

In plain words Replace the curved region by a row of rectangles, each as wide as its piece and as tall as the curve at the chosen point. The rectangles may miss some area or include some extra, so a Riemann sum estimates an area. For a density, a Riemann sum is only an estimate of a probability, and it can even exceed $1$ .

Example For the density $2x$ on $[0, 1]$ , split into four pieces of width $\tfrac14$ . The left endpoints give the heights $0$ , $\tfrac12$ , $1$ and $\tfrac32$ , which add to $3$ , and the right endpoints give $\tfrac12$ , $1$ , $\tfrac32$ and $2$ , which add to $5$ . So

\begin{aligned} L_4 &= \tfrac14 \cdot 3 = \tfrac34, \\[0.4em] R_4 &= \tfrac14 \cdot 5 = \tfrac54. \end{aligned}

The true area $1$ lies between these bounds. The value $\tfrac54$ is not a probability of the model. It only shows that the right rectangles overshoot. With $n$ equal pieces the same computation gives $L_n = 1 - \tfrac1n$ and $R_n = 1 + \tfrac1n$ , for example $0.99$ and $1.01$ when $n = 100$ .

Definite integral

Definition Let $f$ be a function on $[a, b]$ with $a < b$ . Suppose the Riemann sums of $f$ approach a single number $I$ as the widest piece shrinks to width $0$ , whatever points $x_i^{*}$ are chosen. Then $f$ is integrable on $[a, b]$ , and $I$ is its definite integral,

\int_a^b f(x)\,dx = I.

The limit exists, for example, when $f$ is continuous on $[a, b]$ , and also when $f$ is bounded with finitely many jumps. The numbers $a$ and $b$ are the lower and upper limits of integration, $f(x)$ is the integrand, and $dx$ names the variable that runs from $a$ to $b$ . The letter used for that variable does not matter, so $\int_a^b f(t)\,dt$ is the same number. By convention the integral from $a$ to $a$ is $0$ . For a nonnegative $f$ the integral is the area under the graph over $[a, b]$ . When $f$ is integrable on $[a, c]$ and $a < b < c$ , integrals over adjacent intervals add,

\begin{aligned} \int_a^c f(x)\,dx &= \int_a^b f(x)\,dx \\[0.4em] &\quad + \int_b^c f(x)\,dx. \end{aligned}

In plain words The integral is the value the rectangle sums settle on as the rectangles get narrower. The limits of integration say where to start and stop collecting area, and the integrand says how tall the curve is along the way. The elongated S of the integral sign is a reminder that it comes from a sum.

Example The density $2x$ is continuous on $[0, 1]$ , so its integral exists. Because $2x$ is increasing, every Riemann sum with $n$ equal pieces lies between $L_n = 1 - \tfrac1n$ and $R_n = 1 + \tfrac1n$ , and both bounds approach $1$ . Hence

\int_0^1 2x\,dx = 1,

which agrees with the triangle. The event $X > \tfrac12$ has no upper end, but the density is $0$ above $1$ , so the limits of integration are $\tfrac12$ and $1$ . Continuing the formula $2x$ past $1$ would count area that the model does not have.

Accumulated probability

Definition For a random variable $X$ with density $f$ , the accumulated probability up to a real number $x$ is

\begin{aligned} F(x) &= P(X \le x) \\[0.4em] &= \int_{-\infty}^{x} f(t)\,dt, \end{aligned}

which is the cumulative distribution function of $X$ evaluated at $x$ . When $f$ is $0$ below $c$ , this gives $F(x) = 0$ for $x < c$ , and for $x \ge c$ the integral can start at $c$ . The moving endpoint $x$ is the upper limit, so the variable of integration gets a different letter. For real numbers $a < b$ , subtracting accumulated probabilities gives the probability of the interval between them,

\begin{aligned} &P(a < X \le b) \\[0.4em] &\quad = F(b) - F(a). \end{aligned}

In plain words Sweep a vertical line from left to right and record how much area lies to its left. The probability of an interval is the amount the sweep adds between the two endpoints.

Example For the job model and $0 \le x \le 1$ , the region to the left of $x$ is a triangle with base $x$ and height $2x$ , so

\begin{aligned} F(x) &= \int_0^x 2t\,dt \\[0.4em] &= \tfrac12 \cdot x \cdot 2x = x^2. \end{aligned}

Also $F(x) = 0$ for $x < 0$ and $F(x) = 1$ for $x > 1$ . Halfway through the hour the accumulated probability is $F(\tfrac12) = \tfrac14$ , not $\tfrac12$ . The middle half-hour has probability

\begin{aligned} &P\left(\tfrac14 < X \le \tfrac34\right) \\[0.4em] &\quad = F\left(\tfrac34\right) - F\left(\tfrac14\right) \\[0.4em] &\quad = \tfrac{9}{16} - \tfrac{1}{16} = \tfrac12, \end{aligned}

and the second half-hour has probability $1 - \tfrac14 = \tfrac34$ , which answers the opening question. The first, middle and last half-hours have the same width but probabilities $\tfrac14$ , $\tfrac12$ and $\tfrac34$ .

Power rule for integrals

Definition For every nonnegative integer $k$ and real numbers $a \le b$ ,

\int_a^b x^k\,dx = \frac{b^{k+1} - a^{k+1}}{k+1},

where $x^0$ means $1$ , so the case $k = 0$ gives $b - a$ . For integrable functions $f$ and $g$ and a constant $c$ , integrals respect constant multiples and sums,

\begin{gathered} \int_a^b c f(x)\,dx = c \int_a^b f(x)\,dx, \\[0.4em] \int_a^b \bigl(f(x) + g(x)\bigr)\,dx \\[0.4em] = \int_a^b f(x)\,dx + \int_a^b g(x)\,dx. \end{gathered}

Together these rules integrate every polynomial over every interval $[a, b]$ .

In plain words To integrate a whole-number power of $x$ , raise the power by one, divide by the new power, and subtract the value at the lower limit from the value at the upper limit. A constant factor scales every strip of area, so it scales the total.

Example The constructed density $f(x) = 6x - 6x^2$ for $0 \le x \le 1$ , and $0$ otherwise, is nonnegative because $6x(1 - x) \ge 0$ on $[0, 1]$ . Its total area is

\begin{aligned} &\int_0^1 (6x - 6x^2)\,dx \\[0.4em] &\quad = 6 \cdot \tfrac12 - 6 \cdot \tfrac13 \\[0.4em] &\quad = 3 - 2 = 1, \end{aligned}

so it is a valid density, even though its peak height is $f(\tfrac12) = \tfrac32$ . The probability of a value below $\tfrac14$ is

\begin{aligned} &\int_0^{1/4} (6x - 6x^2)\,dx \\[0.4em] &\quad = 3\left(\tfrac14\right)^2 - 2\left(\tfrac14\right)^3 \\[0.4em] &\quad = \tfrac{3}{16} - \tfrac{1}{32} = \tfrac{5}{32}. \end{aligned}

Check yourself

A quantity $X$ has density $f(x) = 3x^2$ for $0 \le x \le 1$ and $f(x) = 0$ otherwise. Check that $f$ is a valid density, find the accumulated probability $F(x)$ for $0 \le x \le 1$ , and find $P(X > \tfrac12)$ .

Show the answer

The function is nonnegative because $x^2 \ge 0$ , and by the power rule its total area is

\int_0^1 3x^2\,dx = 3 \cdot \frac{1^3 - 0^3}{3} = 1,

so it is a valid density. For $0 \le x \le 1$ the accumulated probability is $F(x) = \int_0^x 3t^2\,dt = x^3$ . By the complement rule,

\begin{aligned} P\left(X > \tfrac12\right) &= 1 - F\left(\tfrac12\right) \\[0.4em] &= 1 - \tfrac18 = \tfrac78. \end{aligned}

For the same $X$ , find $P(\tfrac12 < X < \tfrac34)$ and check that the answer is reasonable.

Show the answer

Subtract accumulated probabilities,

\begin{aligned} &P\left(\tfrac12 < X < \tfrac34\right) \\[0.4em] &\quad = F\left(\tfrac34\right) - F\left(\tfrac12\right) \\[0.4em] &\quad = \tfrac{27}{64} - \tfrac{8}{64} = \tfrac{19}{64}. \end{aligned}

The endpoints have probability $0$ , so it does not matter that they are excluded. The answer, about $0.297$ , lies between $0$ and $\tfrac78$ , as it must, because the event is part of the event $X > \tfrac12$ . A negative result would point to a reversed subtraction.

A classmate computes the right Riemann sum with four equal pieces for the density $3x^2$ on $[0, 1]$ and gets $\tfrac{45}{32}$ , which is more than $1$ . The classmate also notices that the density reaches the height $3$ at $x = 1$ . Does either fact show that $f$ is not a valid density?

Show the answer

Neither does. The right heights are $\tfrac{3}{16}$ , $\tfrac34$ , $\tfrac{27}{16}$ and $3$ , which add to $\tfrac{45}{8}$ , so $R_4 = \tfrac14 \cdot \tfrac{45}{8} = \tfrac{45}{32}$ is computed correctly. Because $3x^2$ is increasing on $[0, 1]$ , each right rectangle uses the tallest height on its piece, so $R_4$ overestimates the area. The left sum is $L_4 = \tfrac14(0 + \tfrac{3}{16} + \tfrac34 + \tfrac{27}{16}) = \tfrac{21}{32}$ , an underestimate, and the exact area $1$ lies between the two.

The height $3$ is probability per unit of $x$ . Just below $x = 1$ , a short interval of width $0.01$ has probability close to $3 \cdot 0.01 = 0.03$ , and indeed $P(0.99 < X \le 1) = 1 - 0.99^3 \approx 0.0297$ .

Integrals and Probability as Area

Key terms

Probability density

Probability as area

Riemann sum

Definite integral

Accumulated probability

Power rule for integrals

Check yourself

How this lesson connects

Builds on

Used later in