Bertrand's Box Paradox, math that surprises you lesson

Three boxes each hold two coins. One box has two gold coins, one has a gold coin and a silver coin, and one has two silver coins. You pick a box at random, take out one of its coins without looking, and it is gold. Only two boxes could have produced it, so is the other coin in your box gold with probability one half? This page labels every coin to show that the answer is $2/3$ , explains the same answer with Bayes’ theorem, and describes a different way of producing a gold coin for which one half is correct.

Key terms

Bertrand’s box paradox

Definition Bertrand’s box paradox is a probability puzzle about three boxes that look alike from outside, each holding two coins. Box GG holds two gold coins, box GS holds one gold and one silver coin, and box SS holds two silver coins. The selection procedure has two steps. A box is chosen at random, each with probability $1/3$ , and then one of its two coins is drawn at random, each with probability $1/2$ , without anyone looking inside. Given that the drawn coin is gold, the other coin in the same box is gold with probability $2/3$ , not $1/2$ . The reason is that the chosen box is GG with conditional probability $2/3$ and GS with conditional probability $1/3$ , because box GG gives a gold draw every time it is chosen, while box GS gives one only half the time. The puzzle appears in Joseph Bertrand’s 1889 book on probability. It is a different problem from Bertrand’s chord paradox, which is about choosing a chord of a circle at random.

In plain words A gold coin does more than rule out box SS. It also counts as evidence for box GG, which has two gold coins that could have been drawn, against box GS, which has only one.

Example Give each coin its own name. Box GG holds $g_1$ and $g_2$ , box GS holds $g_3$ and $s_1$ , and box SS holds $s_2$ and $s_3$ . Each coin is drawn with probability

\frac{1}{3} \cdot \frac{1}{2} = \frac{1}{6},

so the six coins are six equally likely first draws. A gold draw leaves three of them, $g_1$ , $g_2$ and $g_3$ , still equally likely. Their partners in the same box are $g_2$ , $g_1$ and $s_1$ , so in two of the three cases the other coin is gold, which gives probability $2/3$ .

Equally likely outcomes

Definition The outcomes of a finite sample space, the set of all possible results of an experiment, are equally likely when each of its $N$ outcomes has probability $1/N$ . Then an event made of $k$ outcomes has probability $k/N$ . If we learn that an event $B$ with $m \ge 1$ outcomes happened, the outcomes of $B$ stay equally likely, each now with probability $1/m$ , so any event $A$ has conditional probability equal to the number of outcomes in both $A$ and $B$ divided by $m$ . Whether outcomes are equally likely depends on the procedure that produces them. Listing the possibilities does not make them equally likely, and neither does the fact that only a few remain after an observation.

In plain words Counting gives probabilities only after the possibilities have been cut into pieces of equal chance. Here the equal pieces are the six coins, not the three boxes and not the two boxes that remain after a gold draw.

Example Before any coin is drawn the three boxes are equally likely, and so are the six coins. After a gold draw boxes GG and GS remain possible, but they are no longer equally likely, since box GG contains two of the three equally likely gold draws. Counting coins gives box GG probability $2/3$ and box GS probability $1/3$ . Treating the two remaining boxes as equal chances would give $1/2$ , which is exactly the mistake the paradox invites.

Bayes’ theorem

Definition Let $H$ and $E$ be events with $P(H) > 0$ and $P(E) > 0$ . Bayes’ theorem states that

\begin{aligned} &P(H \mid E) \\[0.4em] &\quad = \frac{P(E \mid H)\,P(H)}{P(E)}, \end{aligned}

where $P(H \mid E)$ , the conditional probability of $H$ given $E$ , is the probability of $H$ once $E$ is known to have happened. In this use $H$ is a hypothesis and $E$ is evidence. The prior probability $P(H)$ describes the hypothesis before the evidence is taken into account, the likelihood $P(E \mid H)$ says how probable the evidence is if the hypothesis is true, and the posterior probability $P(H \mid E)$ describes the hypothesis after the evidence. If hypotheses $H_1, \dots, H_k$ each have positive probability and exactly one of them is true, the denominator is

\begin{aligned} &P(E) \\[0.4em] &\quad = \sum_{i=1}^{k} P(E \mid H_i)\,P(H_i), \end{aligned}

the law of total probability. The lesson Bayes’ Theorem and Base Rates in the probability course gives more examples.

In plain words Each possible explanation of an observation gets the weight “how probable it was beforehand” times “how probable it makes the observation”. Dividing each weight by the total of all the weights turns them into probabilities that add up to $1$ .

Example Let $E$ be the event that the drawn coin is gold. Each box has probability $1/3$ before the draw, and the likelihoods of a gold draw are

\begin{gathered} P(E \mid \text{GG}) = 1, \\[0.4em] P(E \mid \text{GS}) = \tfrac12, \\[0.4em] P(E \mid \text{SS}) = 0. \end{gathered}

The total is $P(E) = \tfrac13 + \tfrac16 + 0 = \tfrac12$ , so

P(\text{GG} \mid E) = \frac{1 \cdot \tfrac13}{\tfrac12} = \frac{2}{3},

and in the same way box GS has probability $1/3$ and box SS probability $0$ after the gold draw. The same weights appear as counts. Over $600$ runs of the procedure each box is expected to be chosen $200$ times. Every choice of box GG gives gold, half the choices of box GS do, and no choice of box SS does, so the expected number of gold draws is $200 + 100 + 0 = 300$ , and $200$ of them come from box GG. Real runs vary around these expected counts.

Selection procedure

Definition The selection procedure of a probability problem is the complete rule that produces what we observe. It says which choices are random and with what probabilities, and whether anyone looks before choosing. The procedure fixes the likelihood $P(E \mid H)$ of the observation under each hypothesis, so two procedures that can end in the same visible result, such as a gold coin taken from a closed box, can give different conditional probabilities.

In plain words To learn from an observation you need to know how it was produced. The same picture can be evidence for one explanation under one procedure and no evidence at all under another.

Example Change the procedure. Someone chooses between boxes GG and GS with probability $1/2$ each, looks inside, and deliberately takes out a gold coin. Let $E$ now be the event that a gold coin is taken out. It happens with probability $1$ from either box, so

\begin{aligned} P(\text{GG} \mid E) &= \frac{1 \cdot \tfrac12}{1 \cdot \tfrac12 + 1 \cdot \tfrac12} \\[0.4em] &= \frac{1}{2}. \end{aligned}

The other coin is gold exactly when the box is GG, so in this experiment the answer really is $1/2$ . The original procedure gave $2/3$ because a random draw from box GS shows gold only half the time. A deliberate choice of a gold coin removes that difference, so the gold coin no longer favors either box.

Check yourself

Add a fourth box, a second box with one gold and one silver coin, so the boxes are GG, GS, GS and SS. A box is chosen at random, each with probability $1/4$ , and one of its coins is drawn at random. The coin is gold. What is the probability that the other coin is gold?

Show the answer

Each of the eight coins is drawn with probability $\tfrac14 \cdot \tfrac12 = \tfrac18$ . Four of them are gold, the two in box GG and one in each mixed box, and after a gold draw these four stay equally likely. The other coin is gold only for the two draws from box GG, so the probability is

\frac{2}{4} = \frac{1}{2}.

One half is right this time, but not because the remaining boxes are equally likely. Three boxes remain possible, box GG with probability $1/2$ and each mixed box with probability $1/4$ . The answer is one half because box GG now holds half of all the gold coins.

Change the procedure. A box is chosen at random from all three, and a helper who looks inside shows you a gold coin whenever the box holds one, and a silver coin otherwise. You are shown a gold coin. What is the probability that the other coin in that box is gold?

Show the answer

Let $E$ be the event that you are shown a gold coin. Under this procedure it happens with probability $1$ from box GG, with probability $1$ from box GS and with probability $0$ from box SS. With each box at probability $1/3$ beforehand, the total is $P(E) = 2/3$ , and Bayes’ theorem gives

P(\text{GG} \mid E) = \frac{1 \cdot \tfrac13}{\tfrac23} = \frac{1}{2}.

The other coin is gold only in box GG, so the answer is $1/2$ . The helper makes box GS a sure source of a gold coin, just like box GG, so the gold coin no longer counts as evidence for either of them.

Return to the original procedure. After seeing the gold coin, you put it back in the same box, shake the box, and draw again at random from it. The second draw is also gold. What is the probability now that the box is GG?

Show the answer

Let $E$ be the event that both draws are gold. Two draws with replacement give gold both times with probability $1$ from box GG, with probability $\tfrac12 \cdot \tfrac12 = \tfrac14$ from box GS and with probability $0$ from box SS. Bayes’ theorem with each box at probability $1/3$ gives

\begin{aligned} P(\text{GG} \mid E) &= \frac{\tfrac13 \cdot 1}{\tfrac13 \cdot 1 + \tfrac13 \cdot \tfrac14} \\[0.4em] &= \frac{4}{5}. \end{aligned}

The same answer comes in two steps. After the first gold coin the box is GG with probability $2/3$ and GS with probability $1/3$ , and the second gold draw has likelihood $1$ under GG and $1/2$ under GS. The weights $\tfrac23 \cdot 1 = \tfrac23$ and $\tfrac13 \cdot \tfrac12 = \tfrac16$ give box GG the probability $\tfrac23 \big/ \left(\tfrac23 + \tfrac16\right) = \tfrac45$ . Each gold draw doubles the ratio of the probability of box GG to that of box GS, from $1$ before any draw to $2$ after one draw and $4$ after two. A silver draw at any point would rule out box GG at once.

Bertrand's Box Paradox

Video 1, chapter 8

Key terms