|
Linear discriminant analysis (LDA), is sometimes known as Fisher's linear discriminant, after its inventor, Ronald A. Fisher, who published it in The Use of Multiple Measures in Taxonomic Problems (1936). It is typically used as a feature extraction step before classification. Sir Ronald Fisher Sir Ronald Aylmer Fisher, FRS (February 17, 1890–July 29, 1962) was an extraordinarily talented evolutionary biologist, geneticist and statistician. ...
1936 was a leap year starting on Wednesday (link will take you to calendar). ...
Feature extraction is an area of image processing which involves using algorithms to detect and isolate various desired portions of a digitized image or video stream. ...
Statistical classification is a type of supervised learning algorithm which takes a feature representation of an object or concept and maps it to a classification label. ...
LDA is used for two-class classification, or equivalently, given a vector of observations x, predict the probability of a binary random class variable c. LDA is based on the following observation: if the densities and are both normally distributed, with identical full-rank covariances, but possibly different means, then a sufficient statistic for is given by The normal distribution, also called Gaussian distribution, is an extremely important probability distribution in many fields, especially in physics and engineering. ...
In probability theory and statistics, the covariance between two real_valued random variables X and Y, with expected values and is defined as: where E is the expected value. ...
In statistics, mean has two related meanings: the average in ordinary English, which is more correctly called the arithmetic mean, to distinguish it from geometric mean or harmonic mean. ...
In statistics, one often considers a family of probability distributions for a random variable X (and X is often a vector whose components are scalar-valued random variables, frequently independent) parameterized by a scalar- or vector-valued parameter, which let us call θ. ...
That is, the probability of an input x being in a class c is purely a function of this dot product.
A nice property of this dot product is that, out of all possible one-dimensional projections, this one maximizes the distance between the projected means to the variance of the projected normal distributions. Thus, in some sense, this projection maximizes the signal-to-noise ratio. In mathematics, the dot product (also known as the scalar product and the inner product) is a sesquilinear function (·) : V × V → F, where V is a vector space over the field F, having some further properties. ...
The phrase signal-to-noise ratio, often abbreviated SNR or S/N, is an engineering term for the ratio between the magnitude of a signal (meaningful information) and the magnitude of background noise. ...
In practice, this technique can be used by assuming that the two densities and have different means and shared covariance, and then use the maximum likelihood estimate or the maximum a posteriori estimate of the means and covariance. In statistics, the method of maximum likelihood, pioneered by geneticist and statistician Sir Ronald A. Fisher, is a method of point estimation, that uses as an estimate of an unobservable population parameter the member of the parameter space that maximizes the likelihood function. ...
LDA can be generalized to multiple discriminant analysis, where c becomes a categorical variable with N possible states, instead of only two. Analogously, if the class-conditional densities are normal with shared covariances, the sufficient statistic for are the values of N projections, which are the subspace spanned by the N means, affine projected by the inverse covariance matrix. These projections can be found by solving a generalized eigenvalue problem, where the numerator is the covariance matrix formed by treating the means as the samples, and the denominator is the shared covariance matrix. The level of measurement of a variable in mathematics and statistics describes how much information the numbers associated with the variable contain. ...
In linear algebra, a generalized eigenvector of a matrix A is a nonzero vector v, which has associated with it an eigenvalue λ having algebraic multiplicity k, satisfying Generalized eigenvectors can be used to determine the Jordan form. ...
See also
In decision theory (for example risk management), a decision tree is a graph of decisions and their possible consequences, (including resource costs and risks) used to create a plan to reach a goal. ...
Data mining, also known as knowledge-discovery in databases (KDD), is the practice of automatically searching large stores of data for patterns. ...
A linear classifier is a classifier that uses a linear function of its inputs to base its decision on. ...
The logit (pronounced with a long o and a soft g, SAMPA /loUdZIt/) of a number p between 0 and 1 is Plot of logit in the range 0 to 1, base is e (The base of the logarithm function used here is of little importance in the present article...
Machine learning is an area of artificial intelligence concerned with the development of techniques which allow computers to learn. More specifically, machine learning is a method for creating computer programs by the analysis of data sets. ...
The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. ...
Statistics is the science and practice of developing knowledge through the use of empirical data expressed in quantitative form. ...
References - Pattern Classification (2nd ed.), R.O. Duda, P.E. Hart, D.H. Stork, Wiley Interscience, (2000). ISBN 0471056693
- Fisher, R.A. The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7: 179-188 (1936) pdf file (http://www.library.adelaide.edu.au/digitised/fisher/138.pdf)
Annals of Human Genetics, formerly Annals of Eugenics is a scientific journal. ...
External links - Tutorial about LDA from msstate.edu (http://www.isip.msstate.edu/publications/reports/isip_internal/1998/linear_discrim_analysis/lda_theory.pdf)
- LDA lecture notes from South Africa (http://espresso.ee.sun.ac.za/~schwardt/pr813/lectures/lecture01/node6.html)
|
Feeds |
Contact