Integration in Probability Theory, calculus lesson

How long will a light bulb last? The answer could be any positive number, so asking for the chance of exactly $1000$ hours makes little sense. The useful question is the chance that it lasts between $900$ and $1100$ hours. For quantities like this, probability becomes area under a curve, and integrals do the work.

Key terms

Continuous random variable

Definition A random variable $X$ is continuous if there is a nonnegative function $f$ , integrable on every bounded interval and called its density, such that for all real numbers $a \le b$

P(a \le X \le b) = \int_a^b f(x) \, dx.

It follows that every single value $a$ has probability

P(X = a) = \int_a^a f(x) \, dx = 0,

so it does not matter whether the endpoints are included.

In plain words A continuous random variable can take any value in an interval, like a time, a length or a weight. Single values have probability zero, and only ranges of values carry probability.

Example If a bus arrives at a random time between $0$ and $10$ minutes from now, with every moment equally likely, the waiting time $X$ has density $f(x) = \frac{1}{10}$ on $[0, 10]$ and $0$ elsewhere. Then

P(2 \le X \le 5) = \int_2^5 \frac{1}{10} \, dx = 0.3.

Probability density

Definition A probability density function is a function $f$ with $f(x) \ge 0$ for all $x$ , integrable on every bounded interval, such that

\int_{-\infty}^{\infty} f(x) \, dx = 1.

The value $f(x)$ is not itself a probability. When $f$ is continuous at $x$ and $\Delta x > 0$ is small,

P(x \le X \le x + \Delta x) \approx f(x) \, \Delta x.

In plain words The density shows where the values tend to fall. Probabilities are areas under its graph, and the total area is one.

Example The lifetime $X$ in years of a part has density $f(x) = 2e^{-2x}$ for $x \ge 0$ and $f(x) = 0$ for $x < 0$ . The chance it fails within a year is

P(0 \le X \le 1) = \int_0^1 2e^{-2x} \, dx = 1 - e^{-2} \approx 0.865.

Normalization

Definition Let $g(x) \ge 0$ for all $x$ , with $g$ integrable on every bounded interval and

0 < I = \int_{-\infty}^{\infty} g(x) \, dx < \infty.

Then $f = g / I$ is a probability density. Choosing the constant $c$ so that $c g$ has total integral $1$ is called normalization.

In plain words You often know the shape of a density but not its height. Scale it so the total area is exactly one.

Example Find $c$ so that $f(x) = c\,x(2 - x)$ on $[0, 2]$ , and $0$ elsewhere, is a density. Since

\int_0^2 x(2 - x) \, dx = 4 - \frac{8}{3} = \frac{4}{3},

we need $c = \frac{3}{4}$ .

Cumulative distribution function

Definition The cumulative distribution function of $X$ is

F(x) = P(X \le x) = \int_{-\infty}^{x} f(t) \, dt.

It is continuous and nondecreasing, $\lim_{x \to -\infty} F(x) = 0$ , $\lim_{x \to \infty} F(x) = 1$ , and $P(a < X \le b) = F(b) - F(a)$ . By the fundamental theorem of calculus, $F'(x) = f(x)$ at every $x$ where $f$ is continuous.

In plain words The value $F(x)$ is the area under the density to the left of $x$ . It climbs from $0$ to $1$ .

Example For the lifetime density $2e^{-2x}$ above, $F(x) = 0$ for $x < 0$ and $F(x) = 1 - e^{-2x}$ for $x \ge 0$ . So

P(0.5 < X \le 1) = F(1) - F(0.5) = e^{-1} - e^{-2} \approx 0.233.

Expected value

Definition The expected value, or mean, of $X$ with density $f$ is

\mu = E[X] = \int_{-\infty}^{\infty} x f(x) \, dx,

defined only when

\int_{-\infty}^{\infty} \lvert x \rvert f(x) \, dx < \infty.

In plain words The mean is the balance point of the density. If you cut its graph out of cardboard, it would balance on a pin at $\mu$ .

Example For $f(x) = \frac{3}{4} x(2 - x)$ on $[0, 2]$ ,

E[X] = \frac{3}{4} \int_0^2 (2x^2 - x^3) \, dx = \frac{3}{4} \left( \frac{16}{3} - 4 \right) = 1.

That fits the picture, since the graph is symmetric about $x = 1$ .

Variance

Definition Suppose the second moment

E[X^2] = \int_{-\infty}^{\infty} x^2 f(x) \, dx

is finite. Then the mean $\mu$ exists, and the variance of $X$ is

\operatorname{Var}(X) = \int_{-\infty}^{\infty} (x - \mu)^2 f(x) \, dx = E[X^2] - \mu^2.

In plain words Variance measures how far values typically fall from the mean, using squared distances.

Example For the same density

E[X^2] = \frac{3}{4} \int_0^2 (2x^3 - x^4) \, dx = \frac{3}{4} \left( 8 - \frac{32}{5} \right) = \frac{6}{5},

so $\operatorname{Var}(X) = \frac{6}{5} - 1 = \frac{1}{5}$ .

Standard deviation

Definition The standard deviation of $X$ is $\sigma = \sqrt{\operatorname{Var}(X)}$ , defined whenever the variance is.

In plain words Taking the square root brings the spread back to the same units as $X$ , so it can be compared with the values themselves.

Example The density $\frac{3}{4} x(2 - x)$ has $\sigma = \sqrt{1/5} \approx 0.447$ . For the lifetime density $2e^{-2x}$ a similar calculation gives mean $\frac{1}{2}$ and standard deviation $\frac{1}{2}$ year.

Normal distribution

Definition For a real number $\mu$ and $\sigma > 0$ , the normal distribution with mean $\mu$ and standard deviation $\sigma$ has density

f(x) = \frac{1}{\sigma \sqrt{2\pi}} \, e^{-(x - \mu)^2 / (2\sigma^2)}, \qquad -\infty < x < \infty.

That it is normalized follows from the Gaussian integral

\int_{-\infty}^{\infty} e^{-x^2} \, dx = \sqrt{\pi}.

Its antiderivative is not an elementary function, so probabilities are found from tables or numerically.

In plain words This is the bell curve, centered at $\mu$ , wider when $\sigma$ is larger. Many measured quantities are close to normally distributed.

Example Suppose heights are normal with $\mu = 170$ cm and $\sigma = 8$ cm. About $68.3\%$ of values lie within one standard deviation of the mean and about $95.4\%$ within two, so

P(162 \le X \le 178) \approx 0.683, \qquad P(154 \le X \le 186) \approx 0.954.

Existence of moments

Definition For a positive integer $n$ , the $n$ th moment of $X$ is

E[X^n] = \int_{-\infty}^{\infty} x^n f(x) \, dx.

It is said to exist when the integral converges absolutely, that is, when

\int_{-\infty}^{\infty} \lvert x \rvert^n f(x) \, dx < \infty.

If the $n$ th moment exists, then so does every lower moment.

In plain words Mean and variance are improper integrals, and those can diverge. A density with a heavy tail may have no mean or no variance at all.

Example Let $f(x) = \frac{2}{x^3}$ for $x \ge 1$ and $0$ otherwise. It is a density, since $f \ge 0$ and $\int_1^\infty 2x^{-3} \, dx = 1$ . The mean is

\int_1^\infty x \cdot \frac{2}{x^3} \, dx = \int_1^\infty \frac{2}{x^2} \, dx = 2,

but the second moment is

\int_1^\infty x^2 \cdot \frac{2}{x^3} \, dx = \int_1^\infty \frac{2}{x} \, dx = \infty,

so $X$ has no variance.

Check yourself

Find $k$ so that $f(x) = kx^2$ on $[0, 3]$ , and $0$ elsewhere, is a density. Then find $P(X \le 1)$ .

Show the answer

The total area is $\int_0^3 kx^2 \, dx = 9k$ , and setting it equal to $1$ gives $k = \frac{1}{9}$ . Then

P(X \le 1) = \int_0^1 \frac{x^2}{9} \, dx = \frac{1}{27}.

A random variable is uniform on $[2, 6]$ , with density $\frac{1}{4}$ there. Find its mean and variance.

Show the answer

The mean and second moment are

E[X] = \int_2^6 \frac{x}{4} \, dx = 4, \qquad E[X^2] = \int_2^6 \frac{x^2}{4} \, dx = \frac{52}{3},

so $\operatorname{Var}(X) = \frac{52}{3} - 16 = \frac{4}{3}$ .

A waiting time has density $0.5e^{-0.5x}$ for $x \ge 0$ . What is the chance it exceeds $4$ ?

Show the answer

The chance is

P(X > 4) = \int_4^\infty 0.5e^{-0.5x} \, dx = e^{-2} \approx 0.135.

Integration in Probability Theory

Key terms

Continuous random variable

Probability density

Normalization

Cumulative distribution function

Expected value

Variance

Standard deviation

Normal distribution

Existence of moments

Check yourself

How this lesson connects

Builds on