Principal Component Analysis
PCA online with eigenvalues, scree plot, loadings, varimax rotation and biplot.
When to use it
Use when many correlated variables describe the same set of subjects and you want to reduce them to a few orthogonal components for visualisation, screening, or downstream regression. Standard step before clustering high-dimensional data and before regression with multicollinearity.
When NOT to use it
Do not use when variables are on very different scales without first standardising (correlation-matrix PCA). Do not interpret PCA as if it returned latent factors; use exploratory factor analysis (EFA) for that. PCA is a variance-decomposition, not a measurement model. Do not use as a black-box feature selector.
What you get
Eigenvalues with cumulative percent variance explained; the scree plot; loadings (eigenvectors) per component, optionally varimax-rotated; component scores per observation; biplot of the first two components; Kaiser criterion (retain components with eigenvalue greater than 1) and Horn parallel analysis when run.
How to interpret the output
Components are linear combinations of the original variables; the first captures the most variance, the second the most subject to orthogonality, and so on. Decide how many to keep using the scree plot 'elbow', Kaiser's eigenvalue-greater-than-1 rule (lenient), or parallel analysis (more conservative; recommended). Loadings of absolute value 0.4 or higher are conventionally considered meaningful for interpretation.
Common pitfalls
- Running PCA on raw variables of different scales without standardising.
- Confusing PCA with EFA (different theoretical models).
- Treating high-loading variables as a 'factor' label without theoretical justification.
- Reporting principal components rotated and unrotated together without saying which.
Try it in StatVeda
The Principal Component Analysis engine runs entirely in the browser. No signup, no install, no data sent to a server. Paste your data, hit Run, copy the output.
Open Principal Component AnalysisReferences
- Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2(11), 559 to 572.
- Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6), 417 to 441.
- Jolliffe, I. T. (2002). Principal Component Analysis, 2nd edition. Springer, New York.