Expected Value, introduction to probability lesson

A service request might need $0$ , $2$ or $5$ hours of work, and staffing has to be planned before the request arrives. The most likely workload is one possible summary, but it ignores how large the other possibilities are. Expected value combines every possible value with its probability into a single weighted average. This lesson defines it, separates it from the mode and from any single outcome, and shows that a valid distribution with infinitely many values need not have a finite mean.

Key terms

Expected value

Definition Let $X$ be a discrete random variable with support $S$ and PMF $p_X$ . When $S$ is finite, the expected value of $X$ is the probability-weighted average of its support values,

E[X] = \sum_{x \in S} x \, p_X(x).

Each value appears once, weighted by its own probability. When $S$ is countably infinite, the same series defines $E[X]$ as a finite number provided $\sum_{x \in S} \lvert x \rvert \, p_X(x) < \infty$ , as explained under absolute integrability below. The expected value is also called the mean and is often written $\mu$ . It carries the units of $X$ , and for finite support it lies between the smallest and the largest support value.

In plain words Multiply each possible value by its probability and add the products. Values with more probability get more weight. An ordinary average of the listed values gives each of them equal weight, which is the right weighting for a uniform distribution on those values but not in general.

Example In a constructed model, the number of hours $X$ needed by one service request is $0$ , $2$ or $5$ , with probabilities $0.5$ , $0.3$ and $0.2$ .

Hours $x$	$0$	$2$	$5$
Probability $p_X(x)$	$0.5$	$0.3$	$0.2$
Contribution $x \, p_X(x)$	$0$	$0.6$	$1.0$

Adding the contributions gives $E[X] = 1.6$ hours. The same number appears if ten equally likely slots hold five zeros, three twos and two fives, since their ordinary average is $16/10$ . The unweighted average $(0 + 2 + 5)/3 = 7/3 \approx 2.33$ answers a different question, because the three values are not equally likely. No single request takes $1.6$ hours in this model, so the mean describes the distribution as a whole, not the workload of any individual request.

Mode

Definition A mode of a discrete random variable $X$ is a value $m$ with the largest probability mass, that is, $p_X(m) \ge p_X(x)$ for every real number $x$ . A distribution can have more than one mode. The mode and the mean are different summaries and need not agree.

In plain words The mode is the most likely value, or one of the most likely values when there is a tie. It ignores how far the other values lie from it, while the mean weighs both their sizes and their probabilities.

Example For the service request, the value $0$ has the largest probability, $0.5$ , so $0$ hours is the only mode, while the mean is $1.6$ hours. The two-hour and five-hour requests do not change the mode, but they pull the mean above $0$ . If two values shared the largest probability, both would be modes.

Conditional mean

Definition Let $A$ be an event with $P(A) > 0$ . The conditional PMF of $X$ given $A$ is $p_{X \mid A}(x) = P(X = x \mid A) = P(\{X = x\} \cap A) / P(A)$ , and the conditional mean of $X$ given $A$ is

E[X \mid A] = \sum_{x} x \, p_{X \mid A}(x),

with the sum over the support of $X$ , provided $\sum_x \lvert x \rvert \, p_{X \mid A}(x) < \infty$ , which always holds when the support is finite. When $A$ is the event $\{X \in C\}$ for a set $C$ of values, the conditional masses are $p_X(x) / P(X \in C)$ for $x$ in $C$ and $0$ for every other $x$ .

In plain words When the new information is a statement about $X$ itself, it can rule out some values. Keep the values that are still possible, divide their masses by the probability that remains so that they add to $1$ again, and average with these new weights.

Example Suppose we learn that the request needs some work, the event $X > 0$ , which has probability $0.3 + 0.2 = 0.5$ . The conditional masses are $0.3/0.5 = 0.6$ at $2$ hours and $0.2/0.5 = 0.4$ at $5$ hours. The contributions are $2 \cdot 0.6 = 1.2$ and $5 \cdot 0.4 = 2$ , so

\begin{aligned} E[X \mid X > 0] &= 1.2 + 2 \\[0.4em] &= 3.2 \text{ hours}. \end{aligned}

It is twice the unconditional mean $1.6$ , which is no contradiction, since it averages over a different group of requests. As a check, half of the probability sits on requests with conditional mean $3.2$ hours and the other half on requests needing $0$ hours, and $0.5 \cdot 3.2 + 0.5 \cdot 0 = 1.6$ .

Net gain

Definition When a payoff $G$ is received and a fixed cost $c$ is paid in every outcome, the net gain is the random variable $N = G - c$ , in the same units as $G$ . If $E[G]$ is finite, then

E[N] = E[G] - c,

because subtracting $c$ from every value lowers the weighted sum by $c$ times the total weight, which is $1$ .

In plain words Decide what is being averaged before averaging. A gross prize and the result after paying to play are different random variables, and only the second says whether you end up ahead. An expected net gain can be negative, zero or positive.

Example A constructed training game awards $8$ tokens with probability $0.2$ and nothing otherwise, and entry costs $1$ token. The net gain is $7$ tokens with probability $0.2$ and $-1$ token with probability $0.8$ , so

\begin{aligned} E[N] &= 7 \cdot 0.2 - 1 \cdot 0.8 \\[0.4em] &= 0.6 \text{ tokens}. \end{aligned}

The expected gross award is $8 \cdot 0.2 = 1.6$ tokens, and $1.6 - 1 = 0.6$ agrees. Averaging the award alone would report $1.6$ and forget the fee. The expected net gain is positive, yet the most likely result, with probability $0.8$ , is a loss of $1$ token.

Infinite expectation

Definition Let $X$ be a discrete random variable whose support contains no negative values. Every term of $\sum_x x \, p_X(x)$ is nonnegative, so the series either converges to a finite number or diverges to $+\infty$ . In the second case we write $E[X] = +\infty$ and say that $X$ has infinite expectation. This can happen only when the support is infinite.

In plain words A valid probability distribution need not have a finite mean. If large values keep enough probability, the contributions $x \, p_X(x)$ add up without bound, even though every single value is a finite number.

Example Let $X$ take the values $n = 1, 2, 3, \dots$ with

P(X = n) = \frac{1}{n(n+1)}.

Since $\frac{1}{n(n+1)} = \frac{1}{n} - \frac{1}{n+1}$ , the first $m$ masses add to $1 - \frac{1}{m+1}$ , which tends to $1$ , so this is a valid PMF. Its expectation is

\begin{aligned} E[X] &= \sum_{n=1}^{\infty} n \cdot \frac{1}{n(n+1)} \\[0.4em] &= \sum_{n=1}^{\infty} \frac{1}{n+1} = +\infty, \end{aligned}

the harmonic series without its first term. The tail probabilities $P(X \ge n) = 1/n$ shrink, but too slowly for a finite mean.

Absolute integrability

Definition A discrete random variable $X$ is absolutely integrable if

E\bigl[\lvert X \rvert\bigr] = \sum_{x} \lvert x \rvert \, p_X(x) < \infty.

Then the series for $E[X]$ converges absolutely, every ordering of its terms gives the same sum, and $E[X]$ is a finite number equal to $E[X^{+}] - E[X^{-}]$ , where $X^{+} = \max(X, 0)$ and $X^{-} = \max(-X, 0)$ . If exactly one of $E[X^{+}]$ and $E[X^{-}]$ is infinite, then $E[X]$ is $+\infty$ when $E[X^{+}]$ is the infinite one and $-\infty$ when $E[X^{-}]$ is. If both are infinite, $E[X]$ is undefined. Every variable with finite support is absolutely integrable.

In plain words Before adding signed contributions, check that their sizes alone add to a finite total. When they do, gains and losses can be added in any order. When the positive and the negative contributions both add to infinity, pairing them off to get a number is not legitimate, since different orders or groupings of the terms give different answers.

Example Let $X$ take the values $n$ and $-n$ , each with probability $\frac{1}{2n(n+1)}$ , for $n = 1, 2, 3, \dots$ . The masses add to $1$ and the distribution is symmetric about $0$ . Yet

\begin{aligned} E[X^{+}] &= \sum_{n=1}^{\infty} \frac{1}{2(n+1)} \\[0.4em] &= +\infty, \end{aligned}

and $E[X^{-}] = +\infty$ for the same reason, so $E[X]$ is undefined. Symmetry alone does not give a mean of $0$ . Adding the terms of $\sum_x x \, p_X(x)$ with the values taken in the order $1, -1, 2, -2, \dots$ gives partial sums that tend to $0$ , while taking two positive terms for each negative one gives partial sums that tend to $\tfrac12 \ln 2 \approx 0.347$ .

Check yourself

A constructed task lasts $1$ , $3$ or $7$ hours with probabilities $0.25$ , $0.5$ and $0.25$ . Find its expected duration and its mode. Can a single task last exactly the expected duration?

Show the answer

The contributions are $1 \cdot 0.25 = 0.25$ , $3 \cdot 0.5 = 1.5$ and $7 \cdot 0.25 = 1.75$ , so the expected duration is $3.5$ hours. As a check, it lies between $1$ and $7$ . The mode is $3$ hours, the value with the largest probability.

In this model every task lasts $1$ , $3$ or $7$ hours, so no single task lasts $3.5$ hours. The mean describes the distribution, not an individual task.

For the same task, a student computes the mean duration of tasks longer than $1$ hour as $3 \cdot 0.5 + 7 \cdot 0.25 = 3.25$ hours. What went wrong, and what is the correct conditional mean?

Show the answer

The event $X > 1$ has probability $0.5 + 0.25 = 0.75$ , and the student used the original masses without dividing by it. Those masses add to $0.75$ , not $1$ , so they are not a distribution. The conditional masses are $0.5/0.75 = \tfrac23$ at $3$ hours and $0.25/0.75 = \tfrac13$ at $7$ hours. The contributions are $3 \cdot \tfrac23 = 2$ and $7 \cdot \tfrac13 = \tfrac73$ , so

\begin{aligned} E[X \mid X > 1] &= 2 + \tfrac73 \\[0.4em] &= \tfrac{13}{3} \\[0.4em] &\approx 4.33 \text{ hours}. \end{aligned}

It is larger than the unconditional mean $3.5$ because the shortest duration has been ruled out.

Let $K$ take the values $1, 2, 3, \dots$ with $P(K = k) = 2^{-k}$ , and let $W = (-2)^{K}$ . Someone groups the terms of $\sum_w w \, p_W(w)$ in pairs and concludes that $E[W] = 0$ . Is that right?

Show the answer

No. The PMF of $K$ is valid, since $\sum_{k \ge 1} 2^{-k} = 1$ , and the value $(-2)^k$ of $W$ has probability $2^{-k}$ . The terms of the series are therefore $(-2)^k \, 2^{-k} = (-1)^k$ , that is, $-1, 1, -1, 1, \dots$ . Grouping them as $(-1 + 1) + (-1 + 1) + \cdots$ gives $0$ , but grouping them as $-1 + (1 - 1) + (1 - 1) + \cdots$ gives $-1$ , and the partial sums themselves alternate between $-1$ and $0$ without settling.

Each positive value $2^k$ , for even $k$ , contributes $2^k \cdot 2^{-k} = 1$ to $E[W^{+}]$ , so $E[W^{+}] = +\infty$ . In the same way each negative value $-2^k$ , for odd $k$ , contributes $1$ to $E[W^{-}]$ , so $E[W^{-}] = +\infty$ . Both parts are infinite, so $W$ is not absolutely integrable and $E[W]$ is undefined.

Expected Value

Key terms

Expected value

Mode

Conditional mean

Net gain

Infinite expectation

Absolute integrability

Check yourself

How this lesson connects

Builds on

Used later in