Bayes' Theorem and Base Rates, statistics lesson

An inspection that catches 90 percent of defective components and flags 5 percent of sound ones has just flagged a component. How likely is it that a flagged component is defective? The two rates alone cannot answer that, because the answer also depends on how common defects are among the components being inspected, and it equals 90 percent for only one particular base rate. This lesson shows how Bayes’ theorem combines that base rate with the inspection rates. All rates on this page are constructed to illustrate the arithmetic and make no claim about any real test.

Key terms

Bayes’ theorem

Definition Let $A$ and $B$ be events with $P(A) > 0$ and $P(B) > 0$ . Then

P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}.

The proof is that $P(A \mid B)\,P(B)$ and $P(B \mid A)\,P(A)$ both equal $P(A \cap B)$ by the multiplication rule. If also $P(A) < 1$ , the law of total probability over $A$ and $A^c$ gives the denominator

\begin{aligned} P(B) = {} & P(B \mid A)\,P(A) \\[0.4em] {} + {} & P(B \mid A^c)\,P(A^c). \end{aligned}

The theorem does not require $A$ and $B$ to be independent. If they were independent, it would give $P(A \mid B) = P(A)$ , and the evidence would change nothing.

In plain words Bayes’ theorem reverses the direction of a conditional probability. It combines the probability of the evidence given a hypothesis with the probability of the hypothesis before the evidence, and it gives the probability of the hypothesis given the evidence. Take the probability that the hypothesis and the evidence occur together, then divide by the total probability of the evidence, added up over every way the evidence can arise.

Example Let $D$ be the event that a component is defective, and write $+$ for a flag and $-$ for no flag. Suppose $P(D) = 0.1$ , $P(+ \mid D) = 0.9$ and $P(+ \mid D^c) = 0.05$ . True flags contribute $0.9 \times 0.1 = 0.09$ to the probability of a flag and false flags contribute $0.05 \times 0.9 = 0.045$ , so

\begin{aligned} P(D \mid +) &= \frac{0.09}{0.09 + 0.045} \\[0.4em] &= \frac{0.09}{0.135} = \frac{2}{3}. \end{aligned}

Base rate and prior probability

Definition The base rate of a condition is the proportion of a stated population that has it, also called its prevalence. In a Bayes calculation the prior probability of a hypothesis $H$ is $P(H)$ , its probability before the evidence at hand is taken into account. When a unit is chosen at random from a population, with every unit equally likely, the base rate of a condition in that population is the prior probability that the chosen unit has the condition. In this lesson every prior is a base rate in this sense.

In plain words The prior is where the calculation starts. For a component picked at random, the starting point is how common defects are among the components it was picked from. That base rate has to describe the population the component actually comes from, not some other group.

Example In the inspection model the base rate of defects is $0.1$ , so a component chosen at random has prior probability $P(D) = 0.1$ of being defective. If the inspection were used on a line where only $2$ percent of components are defective, and it still flagged $90$ percent of the defective and $5$ percent of the sound components there, the prior would be $0.02$ and a flag would lead to a far lower posterior probability, as the questions at the end show.

Posterior probability

Definition The posterior probability of a hypothesis $H$ after evidence $E$ , where $P(E) > 0$ , is the conditional probability $P(H \mid E)$ in the same model as the prior. Bayes’ theorem computes it from the prior and the probabilities of the evidence under each hypothesis. If $0 < P(E) < 1$ , the posteriors after $E$ and after $E^c$ average back to the prior, weighted by the probabilities of the evidence,

\begin{aligned} P(H) = {} & P(H \mid E)\,P(E) \\[0.4em] {} + {} & P(H \mid E^c)\,P(E^c). \end{aligned}

In plain words The posterior is the updated probability once the evidence is in. It is still a probability and not a verdict. A posterior of $2/3$ after a flag describes flagged components from this population as a group. If the model holds, about two thirds of a large number of them would be defective. It does not settle whether a particular flagged component is one of them.

Example In the inspection model the prior is $0.1$ . A flag raises the probability of a defect to $2/3$ . No flag has probability $P(-) = 1 - 0.135 = 0.865$ , and a component is both defective and unflagged with probability $0.1 \times 0.1 = 0.01$ , so

P(D \mid -) = \frac{0.01}{0.865} = \frac{2}{173},

about $0.0116$ . This is low but not zero. Weighting the two posteriors by $P(+) = 0.135$ and $P(-) = 0.865$ recovers the prior, since $\tfrac23 \times 0.135 + \tfrac{2}{173} \times 0.865 = 0.09 + 0.01 = 0.1$ .

Sensitivity and specificity

Definition Let $D$ be the event that a unit has the condition, with $0 < P(D) < 1$ , and write $+$ and $-$ for a positive and a negative result. The sensitivity of the test is $P(+ \mid D)$ , the probability of a positive result for a unit with the condition. The specificity is $P(- \mid D^c)$ , the probability of a negative result for a unit without it. Their complements are the false-negative rate $P(- \mid D)$ , equal to $1$ minus the sensitivity, and the false-positive rate $P(+ \mid D^c)$ , equal to $1$ minus the specificity. All four rates condition on the true state of the unit.

In plain words Sensitivity looks only at units that have the condition and asks how often the test catches them. Specificity looks only at units without it and asks how often the test correctly stays negative. Neither answers the question raised by a positive result, which conditions on the result instead. Whether a test’s rates carry over from one population to another is an assumption to check, not a fact built into the definitions.

Example In the inspection model the sensitivity is $0.9$ and the specificity is $0.95$ , so the false-negative rate is $0.1$ and the false-positive rate is $0.05$ . Among flagged components the share that are sound is $0.045/0.135 = 1/3$ , not $0.05$ . The false-positive rate divides by all sound components, while that share divides by all flagged components.

Positive predictive value

Definition The positive predictive value of a test in a population is $P(D \mid +)$ , the probability that a unit with a positive result has the condition, defined when $P(+) > 0$ . By Bayes’ theorem it equals the true-positive term $P(+ \mid D)\,P(D)$ divided by $P(+)$ , which is the sum of the true-positive term and the false-positive term $P(+ \mid D^c)\,P(D^c)$ . So it depends on the base rate as well as on the sensitivity and the specificity. When the sensitivity and the false-positive rate are both positive and held fixed, the positive predictive value increases strictly as the base rate increases. The negative predictive value $P(D^c \mid -)$ is the analogous probability for a negative result.

In plain words The positive predictive value asks what share of all positive results come from units that really have the condition. When the condition is rare, the many units without it can supply most of the positive results, even at a small false-positive rate.

Example With sensitivity $0.9$ and false-positive rate $0.05$ , the positive predictive value is $2/3$ at base rate $0.1$ . At base rate $0.01$ the two terms are $0.009$ and $0.0495$ , and the positive predictive value is $0.009/0.0585 = 2/13$ , about $0.154$ . At base rate $0.5$ it is $0.45/0.475 = 18/19$ , about $0.947$ . The inspection is the same in all three populations, and only the mix of defective and sound components differs. At base rate $0.1$ the negative predictive value is $0.855/0.865 = 171/173$ , about $0.988$ .

Natural frequencies

Definition Natural frequencies express a probability model as whole-number counts in an imagined group of fixed size, chosen so that every count matches the model’s proportions exactly. A conditional probability then becomes a ratio of counts, with the conditioning group as the denominator. The imagined counts restate the model. They are not data, and a real random batch of the same size would usually show somewhat different counts.

In plain words Replace the rates by a concrete group, follow it through the branches and count. The denominator of the question you are asking becomes a group you can see.

Example Imagine $1{,}000$ components that follow the inspection model exactly.

	Flagged	Not flagged	Total
Defective	90	10	100
Sound	45	855	900
Total	135	865	1,000

The flagged column holds $135$ components, of which $90$ are defective, so the positive predictive value is $90/135 = 2/3$ , the value Bayes’ theorem gave. The sensitivity $90/100$ uses the defective row as its denominator, while the positive predictive value $90/135$ uses the flagged column. The count $90$ appears in both.

Check yourself

A different constructed inspection has sensitivity $0.8$ and specificity $0.9$ , and it is used where $5$ percent of components are defective. Use $1{,}000$ imagined components to find the positive predictive value.

Show the answer

Of the $1{,}000$ components, $50$ are defective and $950$ are sound. The inspection flags $0.8 \times 50 = 40$ defective components and misses $10$ . The false-positive rate is $1 - 0.9 = 0.1$ , so it flags $0.1 \times 950 = 95$ sound components and leaves $855$ unflagged. There are $40 + 95 = 135$ flags, so the positive predictive value is

\frac{40}{135} = \frac{8}{27},

about $0.296$ . The formula agrees, since $0.8 \times 0.05 = 0.04$ and $0.1 \times 0.95 = 0.095$ give $0.04/0.135 = 8/27$ . Most flags here come from sound components.

Keep sensitivity $0.9$ and false-positive rate $0.05$ , but suppose only $2$ percent of components are defective. Find the probability that a flagged component is defective, explain why it is lower than $2/3$ , and say what it does and does not tell you about one flagged component.

Show the answer

True flags contribute $0.9 \times 0.02 = 0.018$ and false flags $0.05 \times 0.98 = 0.049$ , so $P(+) = 0.067$ and

\begin{aligned} P(D \mid +) &= \frac{0.018}{0.018 + 0.049} \\[0.4em] &= \frac{18}{67} \approx 0.2687. \end{aligned}

In $1{,}000$ imagined components there are $20$ defective ones, of which $18$ are flagged, and $980$ sound ones, of which $49$ are flagged. The denominator is now the $67$ flags, and $49$ of them come from sound components. Lowering the base rate shrank the true-flag group and enlarged the sound group, which feeds the false flags, so the share of defective components among flags fell. The flag still raises the probability of a defect from $0.02$ to about $0.269$ , more than $13$ times the prior, but that is an updated probability and not certainty. If the model holds, about $27$ percent of a large number of flagged components would be defective, and any one flagged component either is defective or is not.

Someone claims that with sensitivity $0.9$ and false-positive rate $0.05$ , a flag means a defect with probability $0.9$ whatever the population. For which base rate is that true? For which base rate is a flagged component equally likely to be defective or sound?

Show the answer

Write $b$ for the base rate, with $0 < b < 1$ . The positive predictive value $P(D \mid +)$ is

\frac{0.9b}{0.9b + 0.05(1 - b)}.

Setting it equal to $0.9$ and dividing by $0.9$ gives $b = 0.9b + 0.05(1 - b)$ , so $0.15b = 0.05$ and $b = 1/3$ . The claim holds only at that one base rate. A flagged component is equally likely to be defective or sound when the true-flag and false-flag terms are equal, $0.9b = 0.05(1 - b)$ , so $0.95b = 0.05$ and $b = 1/19$ , about $0.053$ . Below that base rate most flags are false. The sensitivity conditions on a defect, while the claim is about the probability of a defect given a flag, and that probability depends on the base rate.

Bayes' Theorem and Base Rates

Key terms

Bayes’ theorem

Base rate and prior probability

Posterior probability

Sensitivity and specificity

Positive predictive value

Natural frequencies

Check yourself

How this lesson connects

Builds on

Used later in