Discrete Random Variables, introduction to probability lesson

Two switches are tested, and each one either works or fails. You could record the full result for both switches, or only how many of them work. Both records describe the same experiment, but they answer different questions and keep different amounts of information. This lesson makes the second kind of record precise. A random variable is a fixed rule that turns each outcome into a number, and its distribution says how probability is spread over the numbers that rule can produce.

Key terms

Random variable

Definition Let $\Omega$ be the sample space of an experiment with probability $P$ . A random variable is a function $X$ that assigns a real number $X(\omega)$ to each outcome $\omega$ in $\Omega$ . For a set $B$ of real numbers, the event $\{X \in B\}$ is the set of outcomes

\{\omega \in \Omega : X(\omega) \in B\}

at which $X$ takes a value in $B$ . So expressions such as $P(X = 1)$ or $P(X \ge 1)$ are probabilities of events in $\Omega$ . On a finite or countably infinite sample space in which every set of outcomes is an event, every real-valued function on $\Omega$ is a random variable. When not every set of outcomes is an event, which is typical on larger sample spaces, a technical condition called measurability is added so that these sets are events.

In plain words The rule itself is not random. What is uncertain before the experiment is which outcome will occur, and therefore which number the rule will produce.

Example Inspect switch one and then switch two, writing W for working and F for failed. The sample space has four ordered outcomes. In this constructed model each switch works with probability $\tfrac{1}{2}$ and the two results are independent, so each outcome has probability $\tfrac{1}{2} \cdot \tfrac{1}{2} = \tfrac{1}{4}$ . Let $X$ be the number of working switches.

Outcome	WW	WF	FW	FF
Probability	$\tfrac{1}{4}$	$\tfrac{1}{4}$	$\tfrac{1}{4}$	$\tfrac{1}{4}$
Value of $X$	$2$	$1$	$1$	$0$

The event $\{X = 1\}$ is $\{\mathrm{WF}, \mathrm{FW}\}$ , so $P(X = 1) = \tfrac{1}{4} + \tfrac{1}{4} = \tfrac{1}{2}$ . The event $\{X \ge 1\}$ leaves out only FF, so $P(X \ge 1) = 1 - \tfrac{1}{4} = \tfrac{3}{4}$ .

Realization

Definition A realization of a random variable $X$ is the number $X(\omega)$ for the outcome $\omega$ that actually occurs. A capital letter such as $X$ names the random variable, and a lowercase letter such as $x$ stands for a possible or observed value. The event $\{X = x\}$ is the set of outcomes at which the rule produces the number $x$ .

In plain words The random variable is the rule, and the realization is the number the rule gave this time. A single realization is one number. It does not say how likely that number was.

Example Suppose the test of the two switches produces WF. The realization of $X$ is then $x = 1$ . Another run might produce FF, with realization $0$ . Neither run changes the function $X$ or the model probabilities $P(X = 0) = \tfrac{1}{4}$ , $P(X = 1) = \tfrac{1}{2}$ and $P(X = 2) = \tfrac{1}{4}$ . Notice also that the realization $1$ does not reveal which switch works, because WF and FW give the same value.

Discrete random variable

Definition A random variable $X$ is discrete if there is a finite or countably infinite set $S$ of real numbers with $P(X \in S) = 1$ . A set is countably infinite when it is infinite and its elements can be listed as a sequence $x_1, x_2, x_3, \dots$ in which every element appears. Every random variable on a finite or countably infinite sample space is discrete, because it takes at most countably many values. Whether the values are whole numbers plays no role.

In plain words With probability $1$ , a discrete random variable takes one of a list of values that can be written one after another, even when the list never ends. Fractions and negative numbers are allowed in that list.

Example Suppose each working switch earns half a token, and let $Y = X/2$ . The possible values of $Y$ are $0$ , $\tfrac{1}{2}$ and $1$ . They are not all integers, but there are only three of them, so $Y$ is discrete. By contrast, if a random variable gives probability $0$ to every single real number, it cannot be discrete. For any finite or countably infinite set $S$ , countable additivity would give $P(X \in S)$ as a sum of zeros, which is $0$ and not $1$ . Such variables appear later in the course.

Support

Definition The support of a discrete random variable $X$ is the set of values that have positive probability,

\{x \in \mathbb{R} : P(X = x) > 0\}.

It is finite or countably infinite, $X$ lies in it with probability $1$ , and every real number $x$ outside it has $P(X = x) = 0$ . A countably infinite support is a support with infinitely many values, which can still be listed in a sequence.

In plain words The support lists the values that can actually turn up with positive probability. A countably infinite support cannot be written as a finite list, but its values can still be counted off one at a time.

Example For the two switches the support of $X$ is $\{0, 1, 2\}$ . The value $3$ is outside it, since two switches cannot give three working switches, so $P(X = 3) = 0$ . For a countably infinite support, toss a fair coin repeatedly with independent tosses, and let $T$ be the number of tosses up to and including the first head. On the one sequence with no head at all, set $T = 0$ , so that $T$ has a value at every outcome. The event $\{T = k\}$ means $k - 1$ tails followed by a head, so independence gives

P(T = k) = \left(\tfrac{1}{2}\right)^{k}

for every positive integer $k$ . The event that no head ever appears is contained in the event that the first $n$ tosses are all tails, so its probability is at most $(\tfrac{1}{2})^{n}$ for every $n$ , which forces it to be $0$ . So the value chosen on that sequence affects no probability, and with probability $1$ the value of $T$ is one of $1, 2, 3, \dots$ . The support is the countably infinite set of positive integers. No finite list would do, because $P(T > n) = (\tfrac{1}{2})^{n} > 0$ for every $n$ .

Distribution of a random variable

Definition The distribution of a random variable $X$ is the rule that gives the probability $P(X \in B)$ for each set $B$ of real numbers. For a discrete $X$ it is determined by the support together with the probabilities $P(X = x)$ of its values, because the probability of any statement about $X$ is found by adding those probabilities over the support values that satisfy it. Two discrete random variables have the same distribution when they give every real number the same probability, even if they are different functions on $\Omega$ .

In plain words The distribution is the full pattern of chances before anything is observed. A realization is one number produced by a single run. Having the same distribution does not make two random variables the same.

Example The switch count $X$ has $P(X = 0) = \tfrac{1}{4}$ , $P(X = 1) = \tfrac{1}{2}$ and $P(X = 2) = \tfrac{1}{4}$ . Its three values are not equally likely, because the value $1$ has two outcomes behind it and the others have one each. Now let $A_1 = 1$ if switch one works and $A_1 = 0$ otherwise, and define $A_2$ in the same way for switch two. Then $X = A_1 + A_2$ at every outcome. The variables $A_1$ and $A_2$ have the same distribution, each equal to $1$ with probability $\tfrac{1}{2}$ and $0$ with probability $\tfrac{1}{2}$ . They are still different random variables, since at the outcome WF the value of $A_1$ is $1$ and the value of $A_2$ is $0$ .

Check yourself

Three switches are tested, and each works with probability $\tfrac{1}{2}$ , independently of the others. Let $N$ be the number of working switches. Find the support and the distribution of $N$ , and find $P(N \ge 2)$ .

Show the answer

There are $2^3 = 8$ ordered outcomes such as WWF, and by independence each has probability $(\tfrac{1}{2})^3 = \tfrac{1}{8}$ . The numbers of outcomes with $0$ , $1$ , $2$ and $3$ working switches are $1$ , $3$ , $3$ and $1$ . So the support is $\{0, 1, 2, 3\}$ and the distribution is

$k$	$0$	$1$	$2$	$3$
$P(N = k)$	$\tfrac{1}{8}$	$\tfrac{3}{8}$	$\tfrac{3}{8}$	$\tfrac{1}{8}$

The probabilities add to $\tfrac{8}{8} = 1$ . The event $\{N \ge 2\}$ contains WWF, WFW, FWW and WWW, so $P(N \ge 2) = \tfrac{4}{8} = \tfrac{1}{2}$ .

A classmate says that $N$ has four possible values, so each value has probability $\tfrac{1}{4}$ . Explain the mistake. Then suppose you are told only that the realization of $N$ is $2$ . What can you say about which switch failed?

Show the answer

The probabilities of the values come from the outcomes behind them, not from the number of values. The value $1$ is produced by three of the eight equally likely outcomes, so its probability is $\tfrac{3}{8}$ , while the value $0$ is produced by only FFF and has probability $\tfrac{1}{8}$ .

The realization $2$ is consistent with WWF, WFW and FWW, so the failed switch could be any of the three. In this model each of those outcomes has conditional probability $\tfrac{1/8}{3/8} = \tfrac{1}{3}$ given $N = 2$ . The count records how many switches work and forgets which ones.

With three switches as above, let $A_1 = 1$ if switch one works and $A_1 = 0$ otherwise, and define $A_3$ in the same way for switch three. Do $A_1$ and $A_3$ have the same distribution? Are they the same random variable?

Show the answer

Switch one works in $4$ of the $8$ equally likely outcomes, so $P(A_1 = 1) = \tfrac{1}{2}$ and $P(A_1 = 0) = \tfrac{1}{2}$ . The same count holds for switch three, so the two variables have the same distribution.

They are not the same random variable. At the outcome WWF the value of $A_1$ is $1$ and the value of $A_3$ is $0$ . In fact the two values differ in the $4$ outcomes where switches one and three disagree, so $P(A_1 \ne A_3) = \tfrac{1}{2}$ .

Discrete Random Variables

Key terms

Random variable

Realization

Discrete random variable

Support

Distribution of a random variable

Check yourself

How this lesson connects

Builds on

Used later in