Suppose a delivery forecast says a parcel has an 80 percent chance of arriving by its deadline. That does not promise that eight of the next ten parcels arrive on time, and it does not say this one parcel is 80 percent on time, because the parcel either meets the deadline or misses it. The number describes uncertainty about that single result, under a model and the information behind it. This lesson separates what can happen, what a model says about it and what is actually observed, and it shows where the numbers in a model can come from.
Key terms
Experiment and trial
Definition An experiment is a procedure whose result will be observed, together with a rule that says exactly what is recorded. A trial is one performance of that procedure. When an experiment is repeated, the model also has to say how the trials relate to each other, for example whether a drawn object is put back and mixed in before the next draw.
In plain words An experiment does not need a laboratory. Drawing a label from a bag, or checking whether one parcel arrives by its deadline, is an experiment once you say what you will write down.
Example A constructed teaching bag holds 20 labels numbered 1 to 20. Labels 1 to 12 say desk, labels 13 to 17 say locker and labels 18 to 20 say reception. The experiment is to draw one label and record its number and collection point. Drawing ten times, returning the label and mixing after each draw, is ten trials of this experiment. The same activity can also be described as one trial of a larger experiment that records all ten labels in order. Either description works, but it has to be fixed before anything is calculated, because it decides what counts as one result.
Outcome and event
Definition An outcome is one complete possible result of an experiment, at the level of detail its recording rule fixes. On each trial exactly one outcome occurs. The set of all possible outcomes is the sample space . An event is a set of outcomes, usually described by a condition, and it occurs on a trial when the observed outcome belongs to it. In the finite models of these first lessons every set of outcomes counts as an event, including the empty event , which never occurs, and itself, which always occurs.
In plain words An outcome is the particular result you write down. An event comes from a yes or no question about that result, such as whether the label says desk, and it collects all the outcomes for which the answer is yes.
Example In the bag experiment the number on a label determines its collection point, so each outcome can be named by its number. There are 20 outcomes, and the event “desk” is the set of 12 of them,
This event occurs whenever any one of those twelve labels is drawn. For a parcel, recording only “on time” or “not on time” gives a sample space with two outcomes. Recording the exact arrival time, with one extra outcome for a parcel that never arrives, gives a much larger sample space, and the event “on time” then contains every arrival time up to the deadline. The parcel is the same, but a different recording rule gives different outcomes.
Probability model
Definition A probability model for an experiment consists of a sample space and a rule that assigns to each event a number , read “the probability of ”. These numbers satisfy and , and when two events have no outcome in common, the probability that one or the other occurs is the sum of their probabilities. The general form of these rules, the probability axioms, comes in a later lesson. For a finite sample space it is enough to give each outcome a probability so that these numbers add to 1, and then
The model also includes the assumptions that justify its numbers, such as the contents of a bag, the selection procedure and whether drawn objects are returned. The empty event has probability and the certain event has probability . The reverse statements can fail. In models with continuous quantities, and in a finite model that gives some outcome probability , an event can have probability without being impossible, and an event of probability need not be logically certain. In a finite model in which every outcome has positive probability, probability does mean impossible and probability does mean certain.
In plain words A model lists what can happen, says how likely each part of it is and states the assumptions behind those numbers. A probability is a statement made inside such a model. Whether the model describes the real procedure well is a separate question, and a calculation can be correct while the model is wrong.
Example Suppose each of the 20 labels has the same chance of being drawn. Then each outcome has probability , and adding over the outcomes in each event gives
These add to 1, because the three events share no label and together cover the bag. No label says airport, so the event “airport” is empty and has probability . If a label goes missing, or the draw favors labels near the top of the bag, these assumptions no longer describe the procedure and the model has to be rebuilt.
Theoretical probability
Definition A theoretical probability is a probability calculated from the assumptions of a stated model rather than from observed results. A common case is a finite model in which each of outcomes has the same probability. An event that contains of the outcomes then has
The equal-chance assumption has to come from the procedure, for example a well-mixed bag of labels that feel the same, not from the number of names on a list.
In plain words A theoretical probability answers the question of what the model says. It is only as good as the assumptions behind it, and it has to be worked out again when the situation changes.
Example The bag names three collection points, but that does not give each one probability . With equally likely labels the event desk has probability , because 12 of the 20 labels say desk. Now change the procedure. A label is drawn, observed to be label 1, which says desk, and left out of the bag. If the next draw is equally likely to pick any of the 19 remaining labels, 11 of which say desk, then given this removal the desk probability for that draw is
The method of counting is the same, but the situation is not. The bag has changed and the result of the first draw is known.
Empirical relative frequency
Definition Suppose an experiment is performed times and an event occurs on of those trials. The empirical relative frequency of the event in that record is . It is computed from data, so it describes one finite run and can differ from run to run. If the trials are independent and the event has the same probability on each of them, the law of large numbers, a result studied later in the course, says that for every fixed margin the probability that the relative frequency after trials differs from by more than tends to as grows.
In plain words A probability belongs to the model, and a relative frequency belongs to the data. Over many independent trials, with the same probability on each, the two are likely to be close. A finite run need not match the model, a longer run is not guaranteed to be closer, and later draws do not make up for earlier ones.
Example Return to drawing with replacement, where every draw is from the full, well-mixed bag of 20 labels and the draws are treated as independent, and suppose an invented record of 50 draws shows desk 36 times. The relative frequency is
while the model probability is still . The record does not change the model. By itself it neither proves the model right nor contradicts it, since under the model any desk count from 0 to 50 is possible. Deciding whether such a difference is strong evidence against the model needs tools from later lessons.
Subjective probability
Definition A subjective probability, also called a personal or epistemic probability, is a number between 0 and 1 that expresses a degree of belief that an event occurs, given the information available to the person assigning it. It can describe a one-time event that cannot be repeated. A set of such assessments is coherent when it obeys the same rules as any probability model, so that, for example, the probabilities of an event and of its complement add to 1. The assessments should be revised when relevant evidence arrives.
In plain words Subjective does not mean arbitrary. The number reflects a judgment about one situation, but it still has to fit together with the other probabilities you assign, and it should respond to evidence.
Example A forecaster assigns probability to the event that one particular parcel arrives by its deadline. This is a hypothetical forecast, not a measured rate. Coherence then requires probability for the opposite event, which includes arriving late and never arriving. A forecaster who also gave probability to “late or never delivered” would be incoherent, because . Records of comparable deliveries can inform such a forecast, but a record from quiet mornings may say little about a holiday evening.
Check yourself
- The bag holds 12 desk, 5 locker and 3 reception labels, and each of the 20 labels is equally likely to be drawn. What is the probability that the drawn label does not say desk?
Show the answer
The event “not desk” contains the 5 locker labels and the 3 reception labels, so
As a check, , since “desk” and “not desk” share no label and together cover the bag.
- Ten labels are drawn from the same bag, one at a time, and each is returned and the bag mixed before the next draw, so the model treats the draws as independent. Since the desk probability is , must exactly six of the ten draws say desk?
Show the answer
No. The value is the probability of desk on each single draw. It does not fix the count in a finite batch, and every count from 0 to 10 is possible under the model. A later lesson on the binomial distribution shows that six is the most likely single count, yet its probability is only about , so the model gives a count other than six a probability of about .
- Start again with the full bag. The first label drawn is observed to say locker and is not returned. The second label is drawn so that each remaining label is equally likely. What is the probability that it says desk, and why is no longer the answer?
Show the answer
After the removal 19 labels remain, and all 12 desk labels are still among them, so the probability is
The value describes a draw from the full bag of 20. Here the procedure is different, because the first label is not returned, and the result of the first draw is known. Had the first label said desk, the answer would have been instead.