Unit 2, lesson 13

Inclusion-Exclusion

How inclusion-exclusion corrects a sum of three event probabilities for overlaps, why pairwise intersections contain the triple intersection, and how disjoint Venn regions give exactly one or exactly two of three events.

  • Video 13
  • Overlaps and Collisions

A constructed collection of 100100 records has 4040 records with flag A, 3535 with flag B and 3030 with flag C. The three counts add up to 105105, more than the number of records, because a record with several flags is counted once for each of them. To find how many records carry at least one flag, the overlaps have to be removed, and in the right amounts. This lesson derives that correction for three events, splits the picture into its disjoint Venn regions, and uses the regions to answer questions such as exactly one flag or exactly two flags.

Key terms

Inclusion-exclusion principle

Definition For any two events AA and BB in a probability model,

P(A∪B)=P(A)+P(B)−P(A∩B),\begin{aligned} P(A \cup B) &= P(A) + P(B) \\[0.4em] &\quad - P(A \cap B), \end{aligned}

as in The Addition Rule. For any three events AA, BB and CC,

P(A∪B∪C)=P(A)+P(B)+P(C)−P(A∩B)−P(A∩C)−P(B∩C)+P(A∩B∩C).\begin{aligned} &P(A \cup B \cup C) \\[0.4em] &\quad = P(A) + P(B) + P(C) \\[0.4em] &\quad - P(A \cap B) - P(A \cap C) \\[0.4em] &\quad - P(B \cap C) \\[0.4em] &\quad + P(A \cap B \cap C). \end{aligned}

The same identities hold for the numbers of elements of finite sets, with ∣A∣\lvert A \rvert in place of P(A)P(A). Neither version needs independence or equally likely outcomes. Equal likelihood matters only when counts are turned into probabilities by dividing by the size ∣Ω∣\lvert \Omega \rvert of a finite sample space Ω\Omega. The reason for the signs is visible one outcome at a time. An outcome in exactly one of the events is counted once by the first three terms and never again. An outcome in exactly two is counted twice and subtracted once. An outcome in all three is counted three times, subtracted three times and added back once. So every outcome of the union is counted exactly once, and outcomes outside the union are never counted. For more events the pattern continues with alternating signs, adding the probabilities of the single events, subtracting those of all pairwise intersections, adding those of all triple intersections and so on. Later cards use the shorthand

S1=P(A)+P(B)+P(C),S2=P(A∩B)+P(A∩C)+P(B∩C),S3=P(A∩B∩C),\begin{aligned} S_1 &= P(A) + P(B) + P(C), \\[0.4em] S_2 &= P(A \cap B) + P(A \cap C) \\[0.4em] &\quad + P(B \cap C), \\[0.4em] S_3 &= P(A \cap B \cap C), \end{aligned}

in which the three-event formula reads P(A∪B∪C)=S1−S2+S3P(A \cup B \cup C) = S_1 - S_2 + S_3.

In plain words Adding the three events counts the overlaps too often. Subtracting the pairwise overlaps repairs the double counts but removes the center of the diagram completely, so the center is added back once.

Example For the records, the pairwise intersections have 1515, 1010 and 1212 records, and 55 records carry all three flags. Then

∣A∪B∪C∣=(40+35+30)−(15+10+12)+5=105−37+5=73.\begin{aligned} &\lvert A \cup B \cup C \rvert \\[0.4em] &\quad = (40 + 35 + 30) \\[0.4em] &\qquad - (15 + 10 + 12) + 5 \\[0.4em] &\quad = 105 - 37 + 5 = 73. \end{aligned}

If one record is chosen with all 100100 records equally likely, the probability of at least one flag is 0.730.73, and by the complement rule the probability of no flag is 0.270.27.

Pairwise intersection

Definition For three events AA, BB and CC, the pairwise intersections are A∩BA \cap B, A∩CA \cap C and B∩CB \cap C. The event A∩BA \cap B contains every outcome that lies in both AA and BB, including the outcomes that also lie in CC. The outcomes in AA and BB but not in CC form the smaller event A∩B∩CcA \cap B \cap C^{c}, with

P(A∩B∩Cc)=P(A∩B)−P(A∩B∩C).\begin{aligned} &P(A \cap B \cap C^{c}) \\[0.4em] &\quad = P(A \cap B) \\[0.4em] &\qquad - P(A \cap B \cap C). \end{aligned}

Inclusion-exclusion uses the full pairwise intersections, not these smaller pieces.

In plain words The event that AA and BB both occur places no condition on CC. An outcome in all three events belongs to every pairwise intersection.

Example Of the 1515 records with flags A and B, 55 also carry flag C, so 1010 records have flags A and B and no flag C. Likewise, 10−5=510 - 5 = 5 records have A and C without B, and 12−5=712 - 5 = 7 have B and C without A. Using these exactly-two counts in place of the pairwise totals gives 105−(10+5+7)+5=88105 - (10 + 5 + 7) + 5 = 88, which is wrong because each of the five triple-flag records is then counted four times instead of once, and 73+3⋅5=8873 + 3 \cdot 5 = 88.

Triple intersection

Definition The triple intersection A∩B∩CA \cap B \cap C is the event that all three events occur. The single and pairwise probabilities do not determine it in general, so it has to be given or found separately. Once they are fixed, the triple probability determines every other piece of the diagram, and it can take exactly those values for which all eight disjoint regions described in the next card have nonnegative probability.

In plain words The triple intersection is the center of the Venn diagram. Subtracting the pairs removes the center once too often, which is why it returns with a plus sign at the end.

Example Keep the single counts 4040, 3535 and 3030 and the pairwise counts 1515, 1010 and 1212, but change the triple count from 55 to 00. The union then has 105−37+0=68105 - 37 + 0 = 68 records instead of 7373. The regions become 1515 with only A, 88 with only B, 88 with only C, then 1515, 1010 and 1212 with exactly the pairs AB, AC and BC, and 3232 records with no flag. All of these are nonnegative, so the changed data describe a possible collection. With these single and pairwise counts, every triple count from 00 to 1010 gives nonnegative regions, and the union then has between 6868 and 7878 records.

Disjoint Venn regions

Definition Three events split the sample space into eight disjoint regions, one for each choice of being inside or outside each event, from A∩B∩CA \cap B \cap C to Ac∩Bc∩CcA^{c} \cap B^{c} \cap C^{c}. Every outcome lies in exactly one region, so the eight region probabilities add up to 11, and the seven regions inside the union add up to P(A∪B∪C)P(A \cup B \cup C). The regions with exactly two events come from the pairwise card, the outside region has probability 1−P(A∪B∪C)1 - P(A \cup B \cup C), and the region with only AA has probability

P(A∩Bc∩Cc)=P(A)−P(A∩B)−P(A∩C)+P(A∩B∩C),\begin{aligned} &P(A \cap B^{c} \cap C^{c}) \\[0.4em] &\quad = P(A) - P(A \cap B) \\[0.4em] &\qquad - P(A \cap C) \\[0.4em] &\qquad + P(A \cap B \cap C), \end{aligned}

with the same pattern for only BB and only CC. Supplied single, pairwise and triple values describe a possible model exactly when all eight regions computed this way are nonnegative.

In plain words Fill the diagram from the inside out. Write the center first, then the three pieces with exactly two events, then the three pieces with one event, and finally the outside.

Example For the records with triple count 55, the regions are the following.

RegionRecords
Only A40−15−10+5=2040 - 15 - 10 + 5 = 20
Only B35−15−12+5=1335 - 15 - 12 + 5 = 13
Only C30−10−12+5=1330 - 10 - 12 + 5 = 13
Only A and B15−5=1015 - 5 = 10
Only A and C10−5=510 - 5 = 5
Only B and C12−5=712 - 5 = 7
All three55
None100−73=27100 - 73 = 27

The seven inside regions add up to 7373, matching inclusion-exclusion, and with the 2727 records outside they account for all 100100 records.

Exactly one of three events

Definition The event that exactly one of AA, BB and CC occurs is the union of the three disjoint regions A∩Bc∩CcA \cap B^{c} \cap C^{c}, Ac∩B∩CcA^{c} \cap B \cap C^{c} and Ac∩Bc∩CA^{c} \cap B^{c} \cap C, so its probability is the sum of their probabilities. In the shorthand of the first card,

P(exactly one)=S1−2S2+3S3.\begin{aligned} &P(\text{exactly one}) \\[0.4em] &\quad = S_1 - 2S_2 + 3S_3. \end{aligned}

To check the weights, follow one outcome. An outcome in exactly one event contributes 11 to the right side. An outcome in exactly two events contributes 2−2⋅1=02 - 2 \cdot 1 = 0, and an outcome in all three contributes 3−2⋅3+3⋅1=03 - 2 \cdot 3 + 3 \cdot 1 = 0.

In plain words Exactly one means that one of the events occurs and the other two do not. Its probability is smaller than that of at least one exactly when the event that two or more of them occur has positive probability.

Example For the records, the three one-flag regions hold 20+13+13=4620 + 13 + 13 = 46 records. The formula agrees, since 105−2⋅37+3⋅5=46105 - 2 \cdot 37 + 3 \cdot 5 = 46. With all records equally likely, the probability of exactly one flag is 0.460.46, compared with 0.730.73 for at least one flag.

Exactly two of three events

Definition The event that exactly two of AA, BB and CC occur is the union of the three disjoint regions A∩B∩CcA \cap B \cap C^{c}, A∩Bc∩CA \cap B^{c} \cap C and Ac∩B∩CA^{c} \cap B \cap C. Each pairwise intersection contains the triple intersection, so S2S_2 includes every outcome of A∩B∩CA \cap B \cap C three times and every outcome in exactly two events once. Therefore

P(exactly two)=S2−3S3,P(at least two)=S2−2S3.\begin{aligned} P(\text{exactly two}) &= S_2 - 3S_3, \\[0.4em] P(\text{at least two}) &= S_2 - 2S_3. \end{aligned}

In plain words Start from the three pairwise totals and take the center out of each of them. Keeping the center once gives at least two instead of exactly two.

Example For the records, the exactly-two regions hold 10+5+7=2210 + 5 + 7 = 22 records, and the formula gives 37−3⋅5=2237 - 3 \cdot 5 = 22, so the probability is 0.220.22. At least two flags covers 22+5=2722 + 5 = 27 records. That is also the number of records with no flag, but only by coincidence in these constructed counts. The events “at least two flags” and “no flag” are disjoint and neither is empty, so they are certainly not the same event.

Check yourself

  1. A number is chosen uniformly from 1,2,…,201, 2, \dots, 20. Let AA, BB and CC be the events that it is divisible by 22, by 33 and by 55. Find the probability that at least one of the events occurs, and list the outcomes where none occurs.
Show the answer

The single events have 1010, 66 and 44 outcomes. Since 22, 33 and 55 are distinct primes, A∩BA \cap B is the set of multiples of 66, namely 66, 1212 and 1818, then A∩CA \cap C is the set of multiples of 1010, namely 1010 and 2020, and B∩CB \cap C contains only 1515. A number in all three events would be a multiple of 3030, and there is none up to 2020. So

∣A∪B∪C∣=(10+6+4)−(3+2+1)+0=20−6+0=14,\begin{aligned} &\lvert A \cup B \cup C \rvert \\[0.4em] &\quad = (10 + 6 + 4) \\[0.4em] &\qquad - (3 + 2 + 1) + 0 \\[0.4em] &\quad = 20 - 6 + 0 = 14, \end{aligned}

and the probability is 14/20=7/1014/20 = 7/10. The six remaining outcomes are 11, 77, 1111, 1313, 1717 and 1919, and the probability of none is 3/103/10.

  1. In the same experiment, find the probability of exactly one of the three events and the probability of exactly two.
Show the answer

With counts in place of probabilities, S1=20S_1 = 20, S2=3+2+1=6S_2 = 3 + 2 + 1 = 6 and S3=0S_3 = 0. Exactly two has 6−3⋅0=66 - 3 \cdot 0 = 6 outcomes, namely 66, 1010, 1212, 1515, 1818 and 2020, so its probability is 6/20=3/106/20 = 3/10. Exactly one has 20−2⋅6+3⋅0=820 - 2 \cdot 6 + 3 \cdot 0 = 8 outcomes, namely 22, 33, 44, 55, 88, 99, 1414 and 1616, so its probability is 8/20=2/58/20 = 2/5. As a check, 8+6+0=148 + 6 + 0 = 14 is the size of the union.

  1. A report on 5050 constructed records gives 2020 records in each of AA, BB and CC, 1212 records in each pairwise intersection and 22 records in the triple intersection. Inclusion-exclusion gives a union of 60−36+2=2660 - 36 + 2 = 26 records, which is less than 5050. Can these data be correct?
Show the answer

No. The region with only A would have

20−12−12+2=−220 - 12 - 12 + 2 = -2

records, and the same holds for only B and only C. A region cannot have a negative number of records, so no collection has these counts. The union formula alone did not reveal the problem, because 2626 looks plausible. Checking every disjoint region does.

How this lesson connects