Regression analysis
Statistical method for estimating relationships between variables.
Regression analysis is a statistical method for estimating the relationship between a dependent variable and one or more independent variables. The most common form is linear regression, which finds the line or hyperplane that best fits the data according to a specific mathematical criterion, such as ordinary least squares, which minimizes the sum of squared differences between the data and the fitted line. This allows estimation of the conditional expectation of the dependent variable for given independent variable values. Less common forms estimate alternative location parameters, such as quantile regression, or use non-linear models. Regression serves two primary purposes: prediction and forecasting, which overlaps with machine learning, and, with careful justification, inferring causal relationships from observational data. The earliest form of regression is attributed to Isaac Newton in 1700, who averaged data, forced a regression line through the average point by summing residuals to zero, and distinguished between inhomogeneous datasets. The method of least squares was published by Legendre in 1805 and Gauss in 1809, who applied it to astronomical orbit determination; Gauss later developed the Gauss–Markov theorem. Francis Galton coined the term "regression" in the 19th century to describe the biological phenomenon of regression toward the mean, where descendants' heights tend toward the average. Udny Yule and Karl Pearson extended this to a general statistical context, assuming a Gaussian joint distribution, a condition later weakened by R.A. Fisher, who assumed only a Gaussian conditional distribution. In the 1950s and 1960s, economists used electromechanical desk calculators, with results sometimes taking up to 24 hours. Modern regression is typically performed with statistical software. A regression model includes unknown parameters, independent variables, a dependent variable, and unobserved error terms.
- field
- Statistics
- known_for
- Estimating relationships between variables; linear regression; method of least squares
- key_contributors
- Isaac Newton, Legendre, Gauss, Francis Galton, Udny Yule, Karl Pearson, R.A. Fisher
Lore & Background
The term 'regression' was coined by Francis Galton in the 19th century to describe the biological phenomenon of regression toward the mean, where heights of descendants of tall ancestors tend to regress down towards a normal average. Galton's work was later extended by Udny Yule and Karl Pearson to a more general statistical context, assuming a Gaussian joint distribution of response and explanatory variables. In the 1950s and 1960s, economists used electromechanical desk calculators to calculate regressions, with results sometimes taking up to 24 hours. Modern regression analysis is typically done with statistical and spreadsheet software on computers and handheld calculators.
Reader's Guide
Regression analysis is a foundational tool in statistics, used to model and analyze relationships between variables. Its most common form, linear regression, finds the line or hyperplane that best fits data according to criteria like ordinary least squares, which minimizes the sum of squared differences. The method is widely applied in prediction and forecasting, overlapping with machine learning, and can be used to infer causal relationships when carefully justified, especially with observational data. Historically, regression evolved from Newton's early averaging methods through the least squares developments by Legendre and Gauss, to Galton's biological concept of regression toward the mean. The work of Yule, Pearson, and Fisher expanded its statistical foundations, relaxing assumptions about data distributions. Regression methods continue to be an active research area, with modern developments including robust regression, nonparametric regression, Bayesian methods, and causal inference. Its significance lies in providing a rigorous framework for understanding how independent variables influence a dependent variable, enabling predictions and causal insights across fields from astronomy to economics.
Did You Know?
- The term 'regression' was coined by Francis Galton to describe the biological phenomenon of regression toward the mean.
- In the 1950s and 1960s, economists used electromechanical desk calculators to calculate regressions, sometimes taking up to 24 hours for one result.
- R.A. Fisher assumed the conditional distribution of the response variable is Gaussian, but the joint distribution need not be.
More in Probability & Statistics 1-24
Spotted an error? Know more?
This is a living reference — every entry is fact-audited, and reader corrections feed straight into our audit queue. Suggest an edit · See this site's audit record