Ordinary least squares
Last updated
Eteri byazrova (GraphMath) · CC BY-SA 4.0
Ordinary least squares (OLS) is a method for selecting unknown parameters in a linear regression model by minimizing the sum of squared differences between observed and predicted values.[1] The OLS estimator is identical to the maximum likelihood estimator when error terms are normally distributed.[2] OLS is sometimes equated with linear regression by certain references.[1][3]
Quick Facts
- Field
- Statistics
- Method type
- Linear least squares method
Facts from the source article.
Key facts and history
Under the Gauss–Markov theorem, OLS is optimal among linear unbiased estimators when errors are homoscedastic and serially uncorrelated.[4][2] The estimator is unbiased under strict exogeneity, meaning its expected value equals the true parameter value.[2] The estimator is uncorrelated with the model residuals.[2] The coefficient of determination R² measures goodness-of-fit by comparing explained variance to total variance.[2]
The projection matrix P = X(XᵀX)⁻¹Xᵀ is also called the hat matrix.[2] The annihilator matrix M creates residuals from the regression.[2]
What the research sources say
A 2022 Feature Column describes OLS as a principled way to pick the line minimizing squared distances between the line and each point.[3] When errors are not independent or have non-constant variance, generalized least squares can be used instead.[3] Akbarzadeh (2014) presents the regression line as a weighted average of lines through any two data points.[5][6]
References
- Wikipedia, "Ordinary least squares" (CC BY-SA 4.0), which cites: "The Origins of Ordinary Least Squares Assumptions", Feature Column, 2022-03-01 (we also read that source) en.wikipedia.org
- Wikipedia, "Ordinary least squares" (CC BY-SA 4.0) en.wikipedia.org
- "The Origins of Ordinary Least Squares Assumptions", Feature Column, 2022-03-01. facts only, no text reused. mathvoices.ams.org
- Wikipedia, "Ordinary least squares" (CC BY-SA 4.0), which cites: "What is a complete list of the usual assumptions for linear regression?", Cross Validated (original not read by us) en.wikipedia.org
- Akbarzadeh, "Line Estimation", 7 May 2014. facts only, no text reused. mlmadesimple.wordpress.com
- Wikipedia, "Ordinary least squares" (CC BY-SA 4.0), which cites: Akbarzadeh, "Line Estimation", 7 May 2014 (we also read that source) en.wikipedia.org
Each numbered sentence above was checked against the source it cites before publication. Sentences citing Wikipedia adapt text from the English Wikipedia article Ordinary least squares (contributors), used under CC BY-SA 4.0; this entry is available under the same licence. Other sources were used for facts only; no text or images are copied from them.
More in Probability & Statistics
Sources
Compiled from Wikipedia and the sources listed below. Text from Wikipedia is available under CC BY-SA 4.0; this entry is adapted from it.
- Wikipedia: Ordinary least squares (CC BY-SA 4.0).
- Word definitions: the Codexery glossary, each quoted from its Wikipedia article.
Spotted an error? Know more?
Reader corrections go straight into our review queue. Suggest an edit · How this site is sourced