Probability Rules and Events

Definition The sample space $\Omega$ of a chance process is the set of all its possible outcomes, where an outcome is one complete result. An event is a set of outcomes to which the model assigns a probability, and it occurs when the outcome that happens belongs to it. In a finite sample space every set of outcomes is usually taken to be an event. A probability model gives each event $A$ a number $P(A)$ with $0 \le P(A) \le 1$ and $P(\Omega) = 1$, and for any finite or infinite sequence of disjoint events the probability of their union is the sum of their probabilities. For a finite sample space the model is fixed by giving each outcome a probability, with these outcome probabilities nonnegative and adding to $1$. Then $P(A)$ is the sum of the probabilities of the outcomes in $A$. If $\Omega$ has $n$ outcomes, all equally likely, and $A$ contains $k$ of them,

Definition The complement $A^c$ of an event $A$ is the set of outcomes in $\Omega$ that are not in $A$. The events $A$ and $A^c$ are disjoint and together make up $\Omega$, so $P(A) + P(A^c) = P(\Omega) = 1$. This gives the complement rule

Definition The union $A \cup B$ is the event containing every outcome that lies in $A$, in $B$ or in both. It occurs when at least one of the two events occurs. The intersection $A \cap B$ is the event containing the outcomes that lie in both, and it occurs when both events occur. In probability the word “or” means the union, which includes the outcomes where both events happen, unless “exactly one” is said. Since $A \cap B$ lies inside $A$ and $A$ lies inside $A \cup B$, every model has $P(A \cap B) \le P(A) \le P(A \cup B)$.

Definition Events $A$ and $B$ are disjoint, also called mutually exclusive, if they share no outcomes, that is, if $A \cap B = \varnothing$. Events $A_1, \dots, A_k$ are disjoint if every two of them are disjoint. They are exhaustive if their union is the whole sample space, so that at least one of them occurs. Events that are both disjoint and exhaustive split $\Omega$ into pieces of which exactly one occurs, and their probabilities add to $1$.

Definition For any two events $A$ and $B$ in a probability model,

The reason is that $A \cup B$ splits into the disjoint pieces $A$ and $B \cap A^c$, while $B$ splits into the disjoint pieces $B \cap A$ and $B \cap A^c$. It is the two-event case of the inclusion-exclusion principle. When $A$ and $B$ are disjoint, the last term is $0$ and $P(A \cup B) = P(A) + P(B)$. More generally, if $A_1, \dots, A_k$ are disjoint, then $P(A_1 \cup \dots \cup A_k) = P(A_1) + \dots + P(A_k)$.

Definition If a chance process is repeated $n$ times and an event $A$ occurs in $m$ of them, the relative frequency of $A$ in those trials is $m/n$. The long-run interpretation of probability applies when the trials are modeled as independent repetitions under the same conditions, each giving $A$ the probability $P(A)$. In that model, for every margin $\varepsilon > 0$, the probability that $m/n$ differs from $P(A)$ by more than $\varepsilon$ tends to $0$ as $n$ grows, which is the law of large numbers. It does not say that the relative frequency moves steadily toward $P(A)$, and it does not say that a finite run must balance out.