Indicator Random Variables, introduction to probability lesson

Five positions on a manufactured part are inspected, and a single fault in the machine can spoil several positions at once. How many defects should you expect per part? The full distribution of the count would need the joint behavior of all five positions. Indicators avoid that work. Each yes-or-no question becomes a variable equal to $1$ or $0$ , the count becomes a sum of these variables, and linearity turns its expectation into a sum of probabilities, with no independence assumption.

Key terms

Indicator random variable

Definition Let $A$ be an event in a probability model with sample space $\Omega$ . The indicator of $A$ is the random variable $I_A$ defined on each outcome $\omega$ by

I_A(\omega) = \begin{cases} 1, & \omega \in A, \\ 0, & \omega \notin A. \end{cases}

It takes only the values $0$ and $1$ , and event operations become arithmetic. The complement has indicator $I_{A^{c}} = 1 - I_A$ , for a second event $B$ the intersection has indicator $I_{A \cap B} = I_A I_B$ , and $I_A^2 = I_A$ because $0^2 = 0$ and $1^2 = 1$ . The notation $\mathbf{1}_A$ is also common.

In plain words An indicator records whether the event happened on this particular run of the experiment, with $1$ for yes and $0$ for no. It is a random variable, not a probability. The probability of $A$ is one fixed number, while the value of $I_A$ depends on the outcome.

Example A label is drawn at random from a box in which two fifths of the labels are blue, and $A$ is the event that the drawn label is blue. Then $I_A = 1$ with probability $\frac25$ and $I_A = 0$ with probability $\frac35$ . If $B$ is the event that the label is large, the product $I_A I_B$ equals $1$ exactly when the drawn label is both blue and large, so it is the indicator of $A \cap B$ .

Expectation of an indicator

Definition For every event $A$ ,

E[I_A] = P(A).

The indicator takes the value $1$ with probability $P(A)$ and the value $0$ with probability $1 - P(A)$ , so its expectation is $1 \cdot P(A) + 0 \cdot (1 - P(A))$ . Since $0 \le I_A \le 1$ , this expectation is always finite.

In plain words The indicator $I_A$ puts weight $P(A)$ on the value $1$ and the rest on $0$ , so its weighted average is exactly $P(A)$ . The probability of any event is the expectation of a random variable, its indicator, and that is what lets the rules for expectations work on probabilities.

Example For the blue label, $E[I_A] = 1 \cdot \frac25 + 0 \cdot \frac35 = \frac25$ . Ten repeated draws with replacement need not produce exactly four blue labels. The value $\frac25$ belongs to the model, and it is not a promised frequency in a finite run.

Sum of indicators

Definition Let $A_1, \dots, A_n$ be events in one probability model, and let $N$ be the number of them that occur. Then

N = I_{A_1} + I_{A_2} + \cdots + I_{A_n}

on every outcome, because each event that occurs contributes $1$ to the sum and each event that does not occur contributes $0$ . The identity is exact, holds before any averaging and needs no assumption about how the events are related.

In plain words To count, give each thing that might be counted its own zero-one switch and add the switches. The definition of each indicator decides what is counted, so state its yes-or-no question first and check that the sum matches the intended count on every outcome.

Example Let $A_j$ be the event that position $j$ of the inspected part is defective, for $j = 1, \dots, 5$ . If positions $2$ and $5$ are defective and the others are sound, then $I_{A_2} = I_{A_5} = 1$ , the other three indicators are $0$ , and the sum is $2$ , the number of defective positions. If every position is sound, all five indicators are $0$ and so is the count.

Expected count

Definition If $N = I_{A_1} + \cdots + I_{A_n}$ counts how many of the events $A_1, \dots, A_n$ occur, then linearity and the expectation of an indicator give

E[N] = \sum_{j=1}^{n} P(A_j).

The events may be dependent. Each indicator is bounded, so each term has finite expectation and linearity applies. The formula uses only the individual probabilities and gives the mean of the count, not its distribution.

In plain words The expected number of events that happen is the sum of their probabilities. Whether the events tend to happen together can change how the count is spread out, but not its mean. An expected count need not be a possible value of the count.

Example Suppose the five positions have defect probabilities $0.10$ , $0.20$ , $0.05$ , $0.10$ and $0.15$ . The expected number of defects is $0.10 + 0.20 + 0.05 + 0.10 + 0.15 = 0.60$ , even if a shared fault makes defects occur together. For a second case, draw three tokens at random without replacement from a bag of ten, four of them blue, and let $B_j$ be the event that draw $j$ is blue. Each draw position is equally likely to hold any of the ten tokens, so $P(B_j) = \frac{4}{10}$ for each $j$ , and the expected number of blue tokens is $3 \cdot \frac{4}{10} = \frac{6}{5}$ . The draws are dependent, and the law of total probability confirms the value for the second draw,

\begin{aligned} P(B_2) &= \frac{4}{10} \cdot \frac{3}{9} + \frac{6}{10} \cdot \frac{4}{9} \\[0.4em] &= \frac{36}{90} = \frac{2}{5}. \end{aligned}

Indicator of a union

Definition For events $A$ and $B$ in one probability model,

I_{A \cup B} = I_A + I_B - I_{A \cap B}

on every outcome. Taking expectations gives the addition rule

\begin{aligned} P(A \cup B) &= P(A) + P(B) \\[0.4em] &\quad - P(A \cap B). \end{aligned}

The plain sum $I_A + I_B$ counts how many of the two events occur. It equals $2$ on $A \cap B$ , so it is the indicator of the union on every outcome exactly when $A \cap B = \varnothing$ . In the same way, if $N$ counts how many of $A_1, \dots, A_n$ occur, then the indicator that at least one occurs is never larger than $N$ , so $P(N \ge 1) \le E[N]$ , with equality exactly when $P(N \ge 2) = 0$ .

In plain words Adding two indicators counts an outcome in both events twice, and subtracting the indicator of the overlap brings it back to one. That outcome-by-outcome correction is where the overlap term of the addition rule comes from. A sum of probabilities is an expected count, and it can exceed the probability that at least one of the events occurs.

Example Roll a fair die. Let $A = \{2, 4, 6\}$ be the event of an even result and $B = \{5, 6\}$ the event of a result of at least $5$ . On the outcome $6$ the sum $I_A + I_B$ is $2$ , while $I_{A \cup B}$ is $1$ . The expected number of these two events that occur is $\frac12 + \frac13 = \frac56$ , but

P(A \cup B) = \tfrac12 + \tfrac13 - \tfrac16 = \tfrac23,

which matches the four favorable faces $2$ , $4$ , $5$ and $6$ .

Expected number of distinct categories

Definition Suppose each of $n$ draws falls into one of the categories $1, \dots, k$ . For each category $c$ , let $J_c$ be the indicator that $c$ appears at least once among the draws. The number of distinct categories seen is $D = J_1 + \cdots + J_k$ , so

E[D] = \sum_{c=1}^{k} P(J_c = 1),

where each term is $1$ minus the probability that category $c$ never appears. If the draws are independent and each draw falls into each category with probability $\frac1k$ , a given category is missed by all $n$ draws with probability $(1 - \frac1k)^n$ , and

E[D] = k\Bigl(1 - \bigl(1 - \tfrac{1}{k}\bigr)^{n}\Bigr).

The indicators $J_1, \dots, J_k$ are usually dependent. In the equal-probability model with $k \ge 2$ and $n \ge 1$ , for instance, each of them equals $0$ with positive probability, yet they cannot all equal $0$ at once, because every draw falls into some category. Linearity does not need their independence, and independence of the draws is used only to find each probability $P(J_c = 1)$ .

In plain words To count distinct kinds rather than items, give each kind its own indicator for being seen at least once. A category that appears twice still contributes $1$ . The chance that a category appears usually comes most easily from the complement, the chance that every draw misses it.

Example Two stickers are drawn independently, each in one of four equally likely colors. A given color is missed by both with probability $(\frac34)^2 = \frac{9}{16}$ , so it appears with probability $\frac{7}{16}$ , and $E[D] = 4 \cdot \frac{7}{16} = \frac74$ . A direct check agrees. The two colors match with probability $\frac14$ , giving one distinct color, and differ with probability $\frac34$ , giving two, so $E[D] = 1 \cdot \frac14 + 2 \cdot \frac34 = \frac74$ .

Check yourself

A bag holds five tokens, two blue and three white, and two are drawn at random without replacement. Find the expected number of blue tokens drawn and the probability that at least one blue token is drawn. Why are the two answers different?

Show the answer

Let $B_1$ and $B_2$ be the events that the first and the second draw are blue. Each has probability $\frac25$ , so the expected number of blue tokens is $2 \cdot \frac25 = \frac45$ . Both draws are white with probability $\frac35 \cdot \frac24 = \frac{3}{10}$ , so at least one blue token is drawn with probability $1 - \frac{3}{10} = \frac{7}{10}$ .

The count can equal $2$ , which happens with probability $\frac25 \cdot \frac14 = \frac{1}{10}$ . Such an outcome adds $2$ to the count but only $1$ to the indicator of at least one blue, and indeed $\frac45 - \frac{7}{10} = \frac{1}{10}$ . An expected count equals the probability of at least one occurrence only when the count is $0$ or $1$ with probability one.

The cards numbered $1$ to $n$ , with $n \ge 1$ , are shuffled so that all $n!$ orders are equally likely. A match is a card that lands in the position carrying its own number. Find the expected number of matches.

Show the answer

Let $I_j$ be the indicator of the event that card $j$ lands in position $j$ . Card $j$ is equally likely to be in any of the $n$ positions, so $E[I_j] = \frac1n$ . The number of matches is $I_1 + \cdots + I_n$ , so its expectation is $n \cdot \frac1n = 1$ for every $n$ . For $n \ge 2$ the indicators are dependent. If the first $n - 1$ cards are all matches, the last card must be a match too. Linearity does not need independence, so this dependence leaves the answer unchanged.

A fair die is rolled three times. Find the expected number of distinct faces that appear, and check the answer by finding the distribution of that number.

Show the answer

Let $D$ be the number of distinct faces, written as the sum of six indicators, one for each face being seen at least once. A given face is missed by all three independent rolls with probability $(\frac56)^3 = \frac{125}{216}$ , so it appears with probability $\frac{91}{216}$ . Adding over the six faces gives

E[D] = 6 \cdot \frac{91}{216} = \frac{91}{36},

about $2.53$ . For the check, all three rolls agree with probability $\frac{6}{216} = \frac{1}{36}$ , and all three differ with probability $\frac{6 \cdot 5 \cdot 4}{216} = \frac{20}{36}$ . Exactly two distinct faces appear with the remaining probability $\frac{15}{36}$ . Then $E[D] = 1 \cdot \frac{1}{36} + 2 \cdot \frac{15}{36} + 3 \cdot \frac{20}{36} = \frac{91}{36}$ , as before.

Indicator Random Variables

Key terms

Indicator random variable

Expectation of an indicator

Sum of indicators

Expected count

Indicator of a union

Expected number of distinct categories

Check yourself

How this lesson connects

Builds on

Used later in