Linear Discriminant Analysis
LDA online with classification, eigenvalues, Wilks lambda and confusion matrix.
When to use it
Use to classify observations into known groups based on multiple continuous predictors and to find the linear combinations of predictors that best separate the groups. Two groups gives Fisher's linear discriminant. Three or more groups gives canonical discriminant axes (the multivariate generalisation). Standard tool in taxonomy, species identification, fraud detection, and any classification setting where group membership is known on a training set.
When NOT to use it
Do not use when within-group covariance matrices are very different across groups; use quadratic discriminant analysis (QDA) or a kernel method instead. Do not use when predictors are mostly categorical; use logistic or multinomial regression. With more predictors than observations, classical LDA is unstable; use regularised LDA. For exploratory dimensionality reduction without group labels, use PCA.
What you get
Per-axis eigenvalues with cumulative percent of separation explained; canonical correlations; Wilks lambda with its F approximation and p as the global test of group separation; standardised and unstandardised discriminant coefficients per axis; group centroids in canonical space; confusion matrix of classification (resubstitution and leave-one-out); per-class classification accuracy.
How to interpret the output
Wilks lambda close to 0 and p well below alpha indicates the predictors separate the groups; lambda close to 1 indicates poor separation. The first canonical axis explains the most between-group variance; subsequent axes explain the residual. Standardised discriminant coefficients indicate which predictors drive the separation. Leave-one-out classification accuracy is a less optimistic estimate than resubstitution; report both.
Common pitfalls
- Reporting resubstitution accuracy without leave-one-out (over-optimistic).
- Using LDA when the equal-covariance assumption fails; check Box's M.
- Confusing LDA with PCA (LDA uses group labels; PCA does not).
- Treating high classification accuracy on the training sample as proof of generalisation.
Try it in StatVeda
The Linear Discriminant Analysis (Fisher / Canonical) engine runs entirely in the browser. No signup, no install, no data sent to a server. Paste your data, hit Run, copy the output.
Open Linear Discriminant Analysis (Fisher / Canonical)References
- Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2), 179 to 188.
- Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning, 2nd edition. Springer, New York. Chapter 4.
- Tabachnick, B. G. and Fidell, L. S. (2013). Using Multivariate Statistics, 6th edition. Pearson, Boston. Chapter 9.