Hierarchy of Classifiers

Andrius Kulikauskas, 2020.04.06: I have set up this page to organize thoughts on understanding Šarūnas Raudys’s hierarchy of classifiers. See this related Discussion on information theory.

Šarūnas Raudys describes the following hierarchy of classifiers:

- the Euclidean distance classifier;
- the standard Fisher linear discriminant function (DF);
- the Fisher linear DF with pseudo-inversion of the covariance matrix;
- regularized linear discriminant analysis;
- the generalized Fisher DF;
- the minimum empirical error classifier;
- the maximum margin classifier.

Andrius Kulikauskas: My thought was that these classifiers might be distinguished by an increasing number of perspectives, from one to seven.

**Readings**

- Paper: Evolution and generalization of a single neurone: I. Single-layer perceptron as seven statistical classifiers
- Paper: Evolution and generalization of a single neurone:: II. Complexity of statistical classifiers and sample size considerations
- Paper: Evolution and generalization of a single neurone. III. Primitive, regularized, standard, robust and minimax regressions
- Book: Šarūnas Raudys. Statistical and Neural Classifiers: An Integrated Approach to Design.