Three fair coins are tossed independently. Counting the eight equally likely sequences gives probability for exactly two heads. A computer can also repeat this experiment many times and report how often exactly two heads appear, but a finite run usually gives a frequency that differs from . This lesson builds such a simulation one trial at a time, makes its output reproducible, and separates ordinary run-to-run fluctuation from a simulator that follows the wrong model.
Key terms
Simulation
Definition A simulation of a probability model is a procedure, usually a computer program, that generates outcomes of an experiment according to that model, repeats the experiment a chosen number of times and records how often an event of interest occurs. Its result for the event is an empirical relative frequency, the number of repetitions in which the event occurred divided by . A simulation follows the model written into its algorithm. If that algorithm describes a different experiment from the intended one, the frequencies estimate a probability in the wrong model.
In plain words A simulation lets a computer play out a chance experiment over and over and keep score. It explores a stated model. By itself it does not tell you whether that model is the right one, and a finite run does not prove that a probability equals a particular value.
Example The three-toss calculation gives a known target for testing a simulator. Simulate three fair independent tosses many times, record how often exactly two heads appear and compare that frequency with . A tempting shortcut shows why the algorithm matters. A program that picks a head count uniformly from 0, 1, 2 and 3 and calls 2 a success simulates a model in which success has probability . In the three-toss model the head counts 0, 1, 2 and 3 have probabilities , , and , so the shortcut has changed the model, and long runs of it typically give frequencies near rather than .
Simulated trial
Definition A simulated trial is one complete repetition of the modeled experiment, generated from fresh random inputs and ending in a recorded result. For an event of interest, trial records the indicator , equal to if the event occurred in that trial and otherwise. The number of successes in the first trials is the partial sum , and the estimate after trials is
The number counts repetitions of the whole experiment, not the individual random draws made inside each repetition.
In plain words Before writing any code, say in ordinary words what one repetition is and what counts as a success. The estimate is the fraction of repetitions that succeeded.
Example For exactly two heads in three tosses, trial generates three bits , and , with meaning heads and meaning tails. It adds them to get the head count and records exactly when . The next trial starts again with three new bits. A run of 1000 trials uses 3000 bits, but its number of trials is . The algorithm treats the bits as independent fair tosses, both within each trial and across trials. A program that carried a head count over from one trial into the next would be computing something else.
Pseudorandom generator
Definition A pseudorandom generator is a deterministic algorithm that produces a sequence of numbers from an internal state. It is designed so that, for suitable uses, the sequence behaves like independent draws from a stated distribution, such as independent fair bits. Because the algorithm is deterministic, the same starting state always produces the same sequence. The standard random module of Python, for example, is built on a generator called the Mersenne Twister.
In plain words The numbers look random but are computed by a fixed rule. Treating them as fair coin tosses is a modeling assumption about the generator, a reasonable one for simulations like these. General-purpose generators such as the Mersenne Twister are meant for simulation and are not designed for security uses such as making passwords.
Example The simulation on this page uses Python’s random.Random generator. Each call of getrandbits(1) returns one pseudorandom bit, or . One trial makes three such calls and treats the three bits as three independent fair tosses, so a run of 1000 trials makes 3000 calls in a fixed order.
Random seed
Definition A random seed is a value that sets the starting state of a pseudorandom generator. Reproducibility means that the same computation can be run again and gives the same recorded output. For a seeded simulation this requires the same generator algorithm, the same seed, the same sequence of calls to the generator and a compatible software environment. The seed is a starting point. It is not a probability, a measure of quality or an extra trial, and changing it usually gives a different run.
In plain words Recording the seed together with the algorithm lets anyone replay the identical run and check every count. Reproducing a run confirms the computation, not the model behind it.
Example With seed 61012 and three getrandbits(1) calls per trial, the first ten trials give the bit strings 011, 011, 101, 100, 110, 010, 001, 110, 111 and 010, where means heads. The first, second, third, fifth and eighth contain exactly two ones, so and . After 1000 trials the count is , an estimate of , and running the same algorithm again with the same seed in Python 3.12 reproduces all 1000 bit strings. The same algorithm with seed 61013 gives a different run, with 390 successes in 1000 trials.
Running estimate
Definition The running estimate of a simulation is the sequence , where uses the first trials. It is often plotted against , together with a horizontal line at the exact probability when that is known. Each new trial changes the estimate by
so a single step has size at most . Small steps do not make the path move steadily toward the exact value.
In plain words The running estimate is the success fraction updated after every trial. The most a single trial can move it shrinks as trials accumulate, yet it still wanders, and the estimate after more trials can be farther from the target than an earlier one.
Example The run with seed 61012 has the checkpoints below. Values that do not terminate are rounded to four decimal places.
| Trials | Successes | Estimate | Distance from |
|---|---|---|---|
| 10 | 5 | 0.5 | 0.125 |
| 50 | 21 | 0.42 | 0.045 |
| 100 | 41 | 0.41 | 0.035 |
| 500 | 191 | 0.382 | 0.007 |
| 600 | 226 | 0.3767 | 0.0017 |
| 700 | 268 | 0.3829 | 0.0079 |
| 1000 | 368 | 0.368 | 0.007 |
After 600 trials the estimate is within 0.002 of , and after 700 trials it is almost 0.008 away. The 100 trials in between contained successes, a frequency of , which is ordinary fluctuation for a correct simulator. The final estimate after 1000 trials is also farther from than the estimate after 600.
Sampling variability
Definition Sampling variability is the variation of an estimate computed from a finite number of random trials, both from one run to another and around the exact model value. Two correct runs of the same simulator with different seeds generally give different estimates, and neither has to equal the exact probability. Sampling variability is different from model error. For a correct simulator and any fixed positive margin, a deviation larger than that margin becomes unlikely as the number of trials grows, which the law of large numbers makes precise later in the course. A simulator built on the wrong model estimates the probability of the event in that wrong model, which can differ from the intended probability, and running it longer does not remove that difference.
In plain words Honest simulations of the same model usually disagree a little with each other and with the exact answer, and that scatter is expected. It tends to shrink as the runs get longer. A simulation that settles firmly on the wrong value has a different problem, one that more trials do not fix.
Example With the same algorithm, seed 61012 gave 368 successes in 1000 trials and seed 61013 gave 390, so the estimates and lie below and above the exact value . Both runs are faithful outputs of the same model, and the difference between them is sampling variability. The shortcut that picks a head count uniformly is another matter. Its long runs typically give frequencies close to , and that gap of about is model error. Comparing a simulator with the exact answer in a small case that can be listed by hand can expose such an error, while a longer run alone only makes the wrong estimate more stable.
Check yourself
- Design a simulation for the event of at least one head in two independent fair coin tosses. Describe one trial, say how the estimate is computed, and find the exact probability.
Show the answer
One trial generates two fresh independent fair bits, with for heads, and records a success when their sum is at least 1. That accepts HH, HT and TH and rejects only TT. After trials the estimate is , the number of successes divided by the number of two-toss trials. If the bits come from a pseudorandom generator, record the seed and the algorithm so the run can be reproduced. For the exact value, let be the event of at least one head. The four sequences are equally likely with probability each, so
- Suppose a run of that simulator with 40 trials gave 34 successes. This count is hypothetical, not output from an actual run. Does it contradict the exact probability ?
Show the answer
The relative frequency is , which is above , but it does not contradict the model. Under the model every success count from 0 to 40 has positive probability, so a finite run can give a frequency different from , and equality with the exact value is not required. Whether a deviation of this size is strong evidence against the model or against the simulator needs further analysis. A count like this one must also stay labeled as hypothetical, because presenting it as actual output would be false.
- To estimate the probability that two fair dice show a sum of 7, a program draws the sum uniformly from the eleven values 2 through 12 and counts a 7 as a success. Suppose a very long run gives a stable estimate close to . Is the program estimating the right probability, and how should it be fixed?
Show the answer
No. It estimates , the success probability in its own model, where the eleven sums are equally likely. In the standard model of two fair dice rolled independently, the 36 ordered pairs of faces are equally likely, and six of them, , , , , and , have sum 7. The exact probability is
The sums are not equally likely, since a sum of 2 has probability while a sum of 7 has probability . A stable estimate reflects small sampling variability, not a correct model, so more trials would only pin the estimate more tightly near . The fix is to let each trial roll two independent dice, each uniform on the faces 1 to 6, add the faces and record a success when the total is 7. Comparing the program with the 36-pair enumeration catches this kind of error.