Law of large numbers
Averages of many trials converge to the expected value.
The law of large numbers is a fundamental theorem in probability theory stating that the average of results obtained from a large number of independent random samples converges to the true expected value, if it exists. It guarantees stable long-term averages for random events, such as a casino's earnings over many spins of a roulette wheel, and is essential in fields including statistics, economics, and insurance. The law applies only when a large number of observations are considered; a small number of observations will not necessarily coincide with the expected value, nor will a streak of one value be immediately balanced by others. For example, rolling a single six-sided die many times will cause the average of all rolls to approach 3.5, with greater precision as more rolls are added. Similarly, the proportion of heads from a large number of fair coin flips will converge to one-half, though the absolute difference between the number of heads and tails will tend to grow large, even as the ratio of that difference to the number of flips approaches zero. The law also underlies Monte Carlo methods, which rely on repeated random sampling to obtain numerical results when other approaches are difficult or impossible. However, the law does not hold for all distributions: the average of results from a Cauchy distribution will not converge, as it lacks an expectation, and some Pareto distributions with infinite expectations also fail. Additionally, if trials embed a selection bias, increasing the number of trials does not remove that bias. Historically, the Italian mathematician Gerolamo Cardano noted without proof that empirical statistics improve with more trials, and Jacob Bernoulli later proved a special form of the law for binary random variables, a process that took him over twenty years to develop rigorous mathematics.
- field
- Probability theory
- known_for
- Stating that the sample mean converges to the true mean for large independent and identically distributed samples
- first_proved_by
- Jacob Bernoulli
- named_by
- S. D. Poisson
- key_refiners
- Chebyshev, Markov, Borel, Cantelli, Kolmogorov, Khinchin
Lore & Background
The law of large numbers is a fundamental principle in probability theory stating that the average of results from a large number of independent random samples will converge to the true expected value, provided that value exists. More formally, for a sample of independent and identically distributed values, the sample mean approaches the true mean as the sample size grows. This law ensures stable long-term averages for random events; for instance, while a casino may lose money on a single roulette spin, its earnings over many spins will tend toward a predictable percentage, and any player’s winning streak will eventually be overcome by the game’s parameters. The law applies only when a large number of observations are considered—there is no principle that a small number of observations will match the expected value or that a streak will be immediately balanced. A classic illustration involves rolling a six-sided die: the expected value of a single roll is 3.5, and as the number of rolls increases, the average of all results approaches 3.5, with precision improving as more rolls are added. For a fair coin toss, the proportion of heads after many flips will almost surely converge to 1/2, though the absolute difference between the number of heads and tails will grow large, even as the ratio of that difference to the number of flips approaches zero. The law also underpins Monte Carlo methods, which rely on repeated random sampling for numerical results, with larger repetitions yielding better approximations. However, the law fails for distributions like the Cauchy or certain Pareto distributions (with α<1), where the average does not converge due to heavy tails or infinite expectations. Additionally, if trials contain selection bias, increasing the number of trials does not correct the bias. The Italian mathematician Gerolamo Cardano first noted that empirical statistics improve with more trials, and Jacob Bernoulli later proved a special form of the law for binary random variables, a result published in his *Ars Conjectandi*.
Reader's Guide
The law of large numbers is significant because it provides a mathematical foundation for the stability of long-term averages in random processes. It assures that, for independent and identically distributed random variables with a finite expected value, the sample mean converges to that expected value as the sample size grows. This principle underlies many practical applications, such as the Monte Carlo method, which relies on repeated random sampling to approximate numerical results. The law has two forms: the weak law and the strong law, the latter implying the former. However, the law has limitations: it does not apply to distributions without a finite expectation, such as the Cauchy distribution, nor does it correct selection bias in trials. Its development involved contributions from many mathematicians, including Chebyshev, Markov, Borel, Cantelli, Kolmogorov, and Khinchin.
Did You Know?
- The law of large numbers was first proved for a binary random variable by Jacob Bernoulli, who called it his 'golden theorem.'
- The law does not apply to distributions like the Cauchy distribution, which lacks an expected value.
More in Probability And Stochastic Processes 1-21
Spotted an error? Know more?
This is a living reference — every entry is fact-audited, and reader corrections feed straight into our audit queue. Suggest an edit · See this site's audit record