Unit 8 of 13
Repeated Sampling and the Discrete Law of Large Numbers
- 6 lessons
- Videos 52 to 57
An average of many independent draws from one distribution is likely to be near its mean. This unit proves a discrete law of large numbers with Chebyshev’s inequality and measures simulation error.
The sample mean is unbiased, and for independent and identically distributed observations its standard error is the standard deviation divided by the square root of the sample size. Markov’s and Chebyshev’s inequalities bound tail probabilities using only a mean or a variance, and they lead to the weak law of large numbers. The chance that an average misses the mean by at least a fixed tolerance tends to zero as the sample grows, with no compensation from future outcomes. The unit ends by measuring Monte Carlo error and comparing an exact sampling distribution with an actual simulation.
Every lesson will be one video. The videos are coming soon, and each lesson page already has the definitions and an example for every key term.
Averages and Probability Bounds
Mean and Variance of the Sample Mean
How the random sample mean differs from a realized average, why it is unbiased for the model mean, how independence makes its variance shrink like one over n, and what standard error and square-root scaling mean.
Video 52
Markov and Chebyshev Inequalities
How Markov's inequality bounds an upper tail of a nonnegative variable by its mean, how Chebyshev's inequality bounds two-sided deviations by the variance, when these bounds are sharp, and why a bound is not an estimate.
Video 53
The Law of Large Numbers and Streaks
Law of Large Numbers for Discrete Variables
How Chebyshev's inequality proves the weak law of large numbers for IID averages and relative frequencies, what convergence in probability means for a fixed error tolerance, and what a sufficient sample size guarantees.
Video 54
Streaks and the Gambler’s Fallacy
Why the gambler’s fallacy fails for independent trials with a known probability, how dilution differs from compensation, how a streak anywhere differs from a run at a preselected position, and when a streak is evidence.
Video 55
Simulation Error
Simulation Error and Number of Trials
How the standard error of a simulation estimate follows from indicator variance, why precision grows with the square root of the trial count, and how plug-in and conservative standard errors, replication and model error differ.
Video 56
A Repeated-Sampling Investigation
How to compare an exact benchmark with a reproducible repeated-sampling simulation, read the empirical distribution and empirical variance of its output, and keep the sample size apart from the replication count.
Video 57