Streaks and the Gambler’s Fallacy, introduction to probability lesson

A fair coin has just landed heads five times in a row, and a tail starts to feel overdue. The long-run frequency of heads settles near one half, so it seems natural that the coin should now start correcting the imbalance. This lesson computes the probability for the next toss exactly, shows how an early imbalance fades from the running proportion without any correction, and separates a known model with independent trials from situations where the probability is unknown or the trials are dependent.

Key terms

Streak

Definition In a sequence of trials, a run or streak of length $k$ is a block of $k$ consecutive trials that all have the same outcome. A run is maximal when the trials just before and just after it, where they exist, have a different outcome. A sequence contains a streak of $k$ heads when it has at least one block of $k$ consecutive heads, wherever that block starts.

In plain words A streak is several equal results in a row. It catches the eye, yet for a fair coin tossed independently a sequence with a long streak is exactly as likely as any other specified sequence of the same length.

Example The record HHTHHHHT has the maximal runs HH, T, HHHH and T, so its longest run of heads has length 4. In six fair independent tosses each of the $2^6 = 64$ sequences has probability $1/64$ , so the streak HHHHHH and the alternating sequence HTHTHT are equally likely. The event of exactly three heads is more likely, with probability

\frac{\binom{6}{3}}{64} = \frac{20}{64} = \frac{5}{16},

because it contains 20 different sequences. The comparison is between one sequence and a collection of sequences, not between tidy and messy patterns.

Gambler’s fallacy

Definition Let $X_1, X_2, \dots$ be independent trials, where $X_i = 1$ means success and every trial has the same known success probability $p$ . For every event $H$ determined by the first $n$ trials with $P(H) > 0$ , independence gives $P(\{X_{n+1} = 1\} \cap H) = p \, P(H)$ , and therefore

P(X_{n+1} = 1 \mid H) = p.

The gambler’s fallacy is the mistaken belief that in such a process an outcome becomes less likely, and the opposite outcome more likely, because the first outcome has occurred unusually often in the recent past.

In plain words With a fixed known probability and independent trials, the next trial does not depend on the record. Five heads create no debt that the coin must repay, and they do not make heads more likely either. The argument uses both assumptions, so it says nothing about a process in which the probability is unknown or the trials are dependent.

Example Let $A$ be the event that the first five of six fair independent tosses are heads, and $B$ the event that the sixth toss is a tail. The event $A \cap B$ is the single sequence HHHHHT, so

\begin{aligned} P(B \mid A) &= \frac{P(A \cap B)}{P(A)} \\[0.4em] &= \frac{1/64}{1/32} = \frac{1}{2}. \end{aligned}

Before any toss, six heads in a row has probability $1/64$ . Once five heads are known, the probability of completing six heads is $1/2$ . The two numbers answer different questions, one asked before the record and one asked given the record.

Compensation misconception

Definition The compensation misconception is the belief that the law of large numbers works by correction, so that after an excess of one outcome the future trials must produce an excess of the other outcome until the counts balance. For independent trials with known success probability $p$ , the number $S_m$ of successes among the next $m$ trials has $E[S_m] = mp$ whatever the record. So if the record has $k$ successes in $n$ trials, the excess of the total success count over $p$ times the number of trials, $(k + S_m) - p(n + m)$ , has expected value $k - np$ for every $m$ .

In plain words The future does not pay back the past. The law of large numbers is about proportions, and a proportion can settle near $p$ while the expected excess in the count stays exactly where it was.

Example After five known heads, plan 100 more fair independent tosses. Their expected numbers of heads and tails are both 50, so the expected difference between the total numbers of heads and tails is still 5 at the end. The heads minus tails difference among the new tosses has mean $0$ and variance $100$ , so it has standard deviation $10$ . No excess of tails is expected. The new tosses add fluctuation of their own, and on average they leave the old imbalance in the counts unchanged.

Dilution

Definition Suppose a record has $k$ successes in its first $n$ trials and is followed by $m$ new independent trials, each a success with probability $p$ . Let $S_m$ be the number of successes among the new trials, and let $Q_m = (k + S_m)/(n + m)$ be the overall success proportion. Since $E[S_m] = mp$ ,

\begin{aligned} E[Q_m] &= \frac{k + mp}{n + m} \\[0.4em] &= p + \frac{k - np}{n + m}. \end{aligned}

Dilution is the effect by which the fixed early excess $k - np$ is divided by a growing number of trials, so its contribution to the expected proportion tends to $0$ as $m$ grows. The realized proportion $Q_m$ also converges to $p$ in probability as $m$ grows, by the weak law of large numbers applied to the new trials.

In plain words An early imbalance becomes a smaller and smaller share of a longer record. The expected proportion moves toward $p$ because the denominator grows, not because the trials start favoring the other outcome.

Example After five known heads and then 100 new fair tosses, the expected head proportion is

\frac{5 + 50}{105} = \frac{11}{21} \approx 0.524,

which is still above one half by $1/42 \approx 0.024$ . After 1000 new tosses instead, it is $505/1005 = 101/201 \approx 0.5025$ , only $1/402 \approx 0.0025$ above one half. In both cases the new tosses are expected to split evenly between heads and tails.

Run at a preselected position

Definition A run at a preselected position is the event that a run occupies trials chosen before the data are seen. Write $A_j$ for the event that trials $j, j + 1, \dots, j + k - 1$ are all successes, with $j$ fixed in advance. For independent trials with success probability $p$ this event has probability $P(A_j) = p^k$ . The event that a run of at least $k$ successes occurs somewhere among $N \ge k$ trials is the union of the events $A_j$ for the $N - k + 1$ possible starting positions $j$ . Its probability is at least $p^k$ , and larger when $N > k$ and $0 < p < 1$ . The union bound

P\Bigl(\bigcup_{j} A_j\Bigr) \le \sum_{j} P(A_j)

gives an upper bound. When $N > k$ and $0 < p < 1$ the bound is strictly larger than the probability, because overlapping windows such as $A_1$ and $A_2$ can occur together.

In plain words Asking for five heads in tosses 1 to 5 is a narrow question. Asking for five heads in a row anywhere in a long record gives the pattern many chances to appear, so finding one after searching is much less surprising. Selection matters too. A record shown because it had the longest streak among many records is not a typical record, and its streak should not be judged by the probability of one preselected window.

Example In 100 fair independent tosses, heads in tosses 1 through 5 has probability $1/32 \approx 0.031$ . There are 96 windows of five consecutive tosses, and the sum of their probabilities is $96/32 = 3$ , which tells us nothing, since every probability is at most $1$ . A recurrence that tracks the length of the current run of heads gives the exact probability of at least one run of five or more heads, about $0.81$ .

Learning about an unknown probability

Definition When the success probability is not known, it can be modeled as a random parameter, with trials that are independent given its value. Unless the parameter is certain to take one value, the trials are then dependent, because the record carries information about the parameter. The probability of success on the next trial given the record is found with Bayes’ theorem and can differ from the probability before the record. This is learning about an unknown probability, and it is a different situation from the gambler’s fallacy.

In plain words A streak can be evidence about the device. If the coin might favor heads, a run of heads makes that more plausible and raises the prediction for heads. Dependence through the mechanism, as in drawing without replacement or a machine that stays in a faulty state, can also make the past matter. Each of these cases needs its own model, and none of them follows from how long a streak feels.

Example A coin is chosen from two coins, each with probability $\tfrac12$ . One is fair and the other lands heads with probability $\tfrac34$ , and given the coin the tosses are independent. Before any toss the probability of heads is $\tfrac12 \cdot \tfrac12 + \tfrac12 \cdot \tfrac34 = \tfrac58$ . After five heads, Bayes’ theorem gives the biased coin the probability

\begin{aligned} &\frac{(3/4)^5}{(3/4)^5 + (1/2)^5} \\[0.4em] &\quad = \frac{243}{275} \approx 0.884, \end{aligned}

so the next toss lands heads with probability

\begin{aligned} &\tfrac{243}{275} \cdot \tfrac34 + \tfrac{32}{275} \cdot \tfrac12 \\[0.4em] &\quad = \tfrac{793}{1100} \approx 0.721. \end{aligned}

The prediction moved toward heads, the opposite of what the gambler’s fallacy expects.

Check yourself

Trials are independent, and each is a success with known probability $0.3$ . The first four trials were all successes. Find the probability that trial 5 is a success, the expected number of successes in the next 20 trials, and the expected overall success proportion once those 20 trials are added to the four known ones. Must the next 20 trials contain extra failures?

Show the answer

By independence, the probability that trial 5 is a success is still $0.3$ . The next 20 trials have expected success count $20 \cdot 0.3 = 6$ . Adding the four known successes gives the expected overall proportion

\frac{4 + 6}{24} = \frac{10}{24} = \frac{5}{12} \approx 0.417.

It is above $0.3$ because the known streak is still a large part of a short record. The next 20 trials need no extra failures. Their expected failure count is $20 \cdot 0.7 = 14$ , the same as for any 20 trials of this process, and the early excess fades through dilution as the record grows.

Five fair coins are tossed independently. Find the probability that tosses 1, 2 and 3 are all heads, and the probability of at least one run of three or more heads anywhere in the five tosses. Why is $3 \cdot \tfrac18 = \tfrac38$ wrong for the second probability?

Show the answer

The preselected run has probability $(1/2)^3 = 1/8$ . For a run anywhere, sort the sequences by where their first run of three heads starts. A start at toss 1 leaves tosses 4 and 5 free, which gives 4 sequences. A start at toss 2 needs a tail at toss 1 and leaves toss 5 free, which gives 2. A start at toss 3 needs a tail at toss 2 and leaves toss 1 free, which gives 2. So 8 of the 32 sequences qualify, and the probability is $8/32 = 1/4$ . The sum $3/8$ counts every sequence once per window it fills. HHHHT and THHHH fill two windows and HHHHH fills three, so the window counts add to 12 rather than 8, and the sum is only an upper bound.

A box holds 4 red and 4 black balls, and each draw is equally likely to be any ball still in the box. Three balls are drawn without replacement, and all three are red. Find the probability that the next draw is black. A friend says that expecting black now is the gambler’s fallacy. Is that right, and what changes if every ball is put back and mixed before the next draw?

Show the answer

The box now holds 1 red and 4 black balls, so the next draw is black with probability $4/5$ . This is not the gambler’s fallacy. Draws without replacement are dependent, and the record changed the contents of the box, so the higher probability comes from the actual mechanism. With replacement and mixing, the draws are independent and each is black with probability $1/2$ . Then three reds leave the probability of black at $1/2$ , and expecting black would be the gambler’s fallacy.

Streaks and the Gambler’s Fallacy

Key terms

Streak

Gambler’s fallacy

Compensation misconception

Dilution

Run at a preselected position

Learning about an unknown probability

Check yourself

How this lesson connects

Builds on