Principal component analysis
Last updated
Oliveira et al. 2022 · CC BY-SA 4.0
Principal component analysis (PCA) transforms data to a new coordinate system where the first coordinate captures the greatest variance.[1] The principal components are a sequence of unit vectors, each orthogonal to the previous ones, that minimize average squared perpendicular distance from data points.[2]
Key facts and history
Karl Pearson invented PCA in 1901 as an analogue of the principal axis theorem in mechanics.[3] Harold Hotelling independently developed and named the method in the 1930s.[3] PCA is also called the discrete Karhunen–Loève transform in signal processing and proper orthogonal decomposition in mechanical engineering.[3] In meteorological science it is known as empirical orthogonal functions.[1]
Eigenvectors of the data's covariance matrix are what the principal components turn out to be.[4] Eigendecomposition of the covariance matrix or singular value decomposition of the data matrix are the usual computational methods for obtaining principal components.[4] Canonical correlation analysis (CCA) is another method related to PCA; CCA finds coordinate systems that best describe cross-covariance between two datasets, whereas PCA finds coordinates that best describe variance within a single dataset.[4]
What the research sources say
A 2023 textbook by Holmes states that the transformation P = X W maps data vectors to a new space of uncorrelated variables.[5] A 2019 paper by Forkman, Josse, and Piepho states that dimensionality reduction is appropriate when variables are noisy, and that PCA can concentrate signal into the first few components.[6] A 1990 textbook by Fukunaga and a 2016 paper by Alizadeh state that PCA is a popular primary technique in pattern recognition but is not optimized for class separability.[7] Parametric bootstrap can be used to test the significance of principal components, as noted in that paper, provided the dataset is not excessively large.[6]
References
- Wikipedia, "Principal component analysis" (CC BY-SA 4.0), which cites: Jolliffe, "Principal Component Analysis", Springer-Verlag, 2002 (we also read that source) en.wikipedia.org
- Wikipedia, "Principal component analysis" (CC BY-SA 4.0), which cites: Gewers, "Principal Component Analysis: A Natural Approach to Data Exploration", ACM Comput. Surv., 24 May 2021, p. 70:1–70:34 (original not read by us) en.wikipedia.org
- Wikipedia, "Principal component analysis" (CC BY-SA 4.0), which cites: Pearson, K., "On Lines and Planes of Closest Fit to Systems of Points in Space", Philosophical Magazine, 1901, p. 559–572; Hotelling, "Relations between two sets of variates", Biometrika, 1936, p. 321–377; Stewart, "On the early history of the singular value decomposition", SIAM Review, 1993, p. 551–566 (original not read by us) en.wikipedia.org
- Wikipedia, "Principal component analysis" (CC BY-SA 4.0), which cites: "Origins and levels of monthly and seasonal forecast skill for United States surface air temperatures determined by canonical correlation analysis", Monthly Weather Review, 1987, p. 1825; Hsu, "A spectral algorithm for learning hidden markov models", 2008; Markopoulos, "Efficient L1-Norm Principal-Component Analysis via Bit Flipping", IEEE Transactions on Signal Processing, 15 August 2017, p. 4252–4264; Chachlakis, "L1-norm Tucker Tensor Decomposition", IEEE Access, 22 November 2019, p. 178454–178465; Markopoulos, "Optimal en.wikipedia.org
- Wikipedia, "Principal component analysis" (CC BY-SA 4.0), which cites: Holmes, "Introduction to Scientific Computing and Data Analysis", Springer, 2023, p. 475–490 (we also read that source) en.wikipedia.org
- Wikipedia, "Principal component analysis" (CC BY-SA 4.0), which cites: Forkman J., Josse, J., Piepho, H. P., "Hypothesis tests for principal component analysis when variables are standardized", Journal of Agricultural, Biological, and Environmental Statistics, 2019, p. 289–308 (original not read by us) en.wikipedia.org
- Wikipedia, "Principal component analysis" (CC BY-SA 4.0), which cites: Fukunaga, "Introduction to Statistical Pattern Recognition", Elsevier, 1990; Alizadeh, "Measuring systematic changes in invasive cancer cell shape using Zernike moments", Integrative Biology, 2016, p. 1183–1193 (original not read by us) en.wikipedia.org
Each numbered sentence above was checked against the source it cites before publication. Sentences citing Wikipedia adapt text from the English Wikipedia article Principal component analysis (contributors), used under CC BY-SA 4.0; this entry is available under the same licence. Other sources were used for facts only; no text or images are copied from them.
More in Probability & Statistics
Sources
Compiled from Wikipedia and the sources listed below. Text from Wikipedia is available under CC BY-SA 4.0; this entry is adapted from it.
- Wikipedia: Principal component analysis (CC BY-SA 4.0).
- Word definitions: the Codexery glossary, each quoted from its Wikipedia article.
Spotted an error? Know more?
Reader corrections go straight into our review queue. Suggest an edit · How this site is sourced