Look up likelihood in Wiktionary, the free dictionary.

Likelihood as a solitary term is a shorthand for likelihood function. In non-technical usage, "likelihood" is a synonym for "probability", but throughout this article only the technical definition is used. Informally, if "probability" allows us to predict unknown outcomes based on known parameters, then "likelihood" allows us to determine unknown parameters based on known outcomes. Wikipedia does not have an article with this exact name. ... Wiktionary (from wiki and dictionary) is a multilingual, Web-based project to create a free content dictionary, available in over 150 languages. ... This article or section does not adequately cite its references or sources. ... Probability is the chance that something is likely to happen or be the case. ... A definition is a form of words which states the meaning of a term. ...

In a sense, likelihood works backwards from probability: given B, we use the conditional probability Pr(A|B) to reason about A, and, given A, we use the likelihood function L(B|A) to reason about B. This mode of reasoning is formalized in Bayes' theorem: Bayes theorem (also known as Bayes rule or Bayes law) is a result in probability theory, which relates the conditional and marginal probability distributions of random variables. ...

In statistics, a likelihood function is a conditional probability function considered as a function of its second argument with its first argument held fixed, thus: A graph of a Normal bell curve showing statistics used in educational assessment and comparing various grading methods. ... This article defines some terms which characterize probability distributions of two or more variables. ... Partial plot of a function f. ...

and also any other function proportional to such a function. That is, the likelihood function for B is the equivalence class of functions In mathematics, given a set X and an equivalence relation ~ on X, the equivalence class of an element a in X is the subset of all elements in X which are equivalent to a: [a] = { x ∈ X | x ~ a } The notion of equivalence classes is useful for constructing sets out...

for any constant of proportionality α > 0. Thus the numerical value L(b | A) is immaterial; all that matters are ratios of the form A ratio is a quantity that denotes the proportional amount or magnitude of one quantity relative to another. ...

since these are invariant with respect to the constant of proportionality. This article is about proportionality, the mathematical relation. ...

For more about making inferences via likelihood functions, see also the method of maximum likelihood, and likelihood-ratio testing. Maximum likelihood estimation (MLE) is a popular statistical method used to make inferences about parameters of the underlying probability distribution from a given data set. ... A likelihood-ratio test is a statistical test relying on a test statistic computed by taking the ratio of the maximum value of the likelihood function under the constraint of the null hypothesis to the maximum with that constraint relaxed. ...

Concentrated likelihood

For a likelihood function of more than one parameter, it is sometimes possible to write some parameters as functions of other parameters, thereby reducing the number of independent parameters. (The function is the parameter value which maximises the likelihood given the value of the other parameters.) This procedure is called concentration of the parameters and results in the concentrated likelihood function. The factual accuracy of this article is disputed. ...

For example, consider a regression analysis model with normally distributed errors. The most likely value of the error variance is the variance of the residuals. The residuals depend on all other parameters. Hence the variance parameter can be written as a function of the other parameters. In statistics, regression analysis examines the relation of a dependent variable (response variable) to specified independent variables (predictors). ... The normal distribution, also called Gaussian distribution by scientists (named after Carl Friedrich Gauss due to his rigorous application of the distribution to astronomical data (Havil, 2003)) is a probability distribution of great importance in many fields. ... In statistics and optimization, the concepts of error and residual are easily confused with each other. ... In probability theory and statistics, the variance of a random variable (or somewhat more precisely, of a probability distribution) is a measure of its statistical dispersion, indicating how its possible values are spread around the expected value. ... In statistics and optimization, the concepts of error and residual are easily confused with each other. ...

Historical remarks

Some early thoughts on likelihood were made in a book by Thorvald N. Thiele published in 1889^[1]. The first paper where the full idea of the "likelihood" appears was written by R.A. Fisher in 1922^[2]: "On the mathematical foundations of theoretical statistics". In that paper, Fisher also uses the term "method of maximum likelihood". Fisher argues against inverse probability as a basis for statistical inferences, and instead proposes inferences based on likelihood functions. Thorvald Nicolai Thiele (December 24, 1838 – September 26, 1910) was a Danish astronomer, actuary, and mathematician, most notable for his work in statistics, interpolation, and the three-body problem. ... Year 1889 (MDCCCLXXXIX) was a common year starting on Tuesday (link will display the full calendar) of the Gregorian calendar (or a common year starting on Thursday of the 12-day slower Julian calendar). ... Sir Ronald Fisher Sir Ronald Aylmer Fisher, FRS (February 17, 1890–July 29, 1962) was an extraordinarily talented evolutionary biologist, geneticist and statistician. ... In statistics, the method of maximum likelihood, pioneered by geneticist and statistician Sir Ronald A. Fisher, is a method of point estimation, that uses as an estimate of an unobservable population parameter the member of the parameter space that maximizes the likelihood function. ... In probability theory, inverse probability is an obsolete term for the probability distribution of an unobserved variable. ...

Likelihood function of a parameterized model

Among many applications, we consider here one of broad theoretical and practical importance. Given a parameterized family of probability density functions In mathematics, a probability density function (pdf) serves to represent a probability distribution in terms of integrals. ...

where θ is the parameter (in the case of discrete distributions, the probability density functions are probability "mass" functions) the likelihood function is

where x is the observed outcome of an experiment. In other words, when f(x | θ) is viewed as a function of x with θ fixed, it is a probability density function, and when viewed as a function of θ with x fixed, it is a likelihood function.

Note: This is not the same as the probability that those parameters are the right ones, given the observed sample. Attempting to interpret the likelihood of a hypothesis given observed evidence as the probability of the hypothesis is a common error, with potentially disastrous real-world consequences in medicine, engineering or jurisprudence. See prosecutor's fallacy for an example of this. To meet Wikipedias quality standards, this article or section may require cleanup. ...

Example

For example, if I toss a coin, with a probability p_H of landing heads up ('H'), the probability of getting two heads in two trials ('HH') is p_H². If p_H = 0.5, then the probability of seeing two heads is 0.25. This article does not cite its references or sources. ...

In symbols, we can say the above as

Another way of saying this is to reverse it and say that "the likelihood of p_H = 0.5, given the observation 'HH', is 0.25", i.e.,

.

But this is not the same as saying that the probability of p_H = 0.5, given the observation, is 0.25.

To take an extreme case, on this basis we can say "the likelihood of p_H = 1 given the observation 'HH' is 1". But it is clearly not the case that the probability of p_H = 1 given the observation is 1: the event 'HH' can occur for any p_H > 0 (and often does, in reality, for p_H roughly 0.5).

The likelihood function is not a probability density function – for example, the integral of a likelihood function is not in general 1. In this example, the integral of the likelihood density over the interval [0, 1] in p_H is 1/3, demonstrating again that the likelihood density function cannot be interpreted as a probability density function for p_H. On the other hand, given any particular value of p_H, e.g. p_H = 0.5, the integral of the probability density function over the domain of the random variables is 1. In mathematics, a probability density function (pdf) serves to represent a probability distribution in terms of integrals. ... A random variable is a mathematical function that maps outcomes of random experiments to numbers. ...

See also

• Bayes factor
• Bayesian inference
• conditional probability
• likelihood principle
• likelihood-ratio test
• maximum likelihood
• principle of maximum entropy
• score (statistics)

In statistics, the use of Bayes factors is a Bayesian alternative to classical hypothesis testing. ... Bayesian inference is statistical inference in which evidence or observations are used to update or to newly infer the probability that a hypothesis may be true. ... This article defines some terms which characterize probability distributions of two or more variables. ... In statistics, the likelihood principle is a controversial principle of statistical inference which asserts that all of the information in a sample is contained in the likelihood function. ... A likelihood-ratio test is a statistical test relying on a test statistic computed by taking the ratio of the maximum value of the likelihood function under the constraint of the null hypothesis to the maximum with that constraint relaxed. ... Maximum likelihood estimation (MLE) is a popular statistical method used to make inferences about parameters of the underlying probability distribution from a given data set. ... The principle of maximum entropy is a method for analyzing the available information in order to determine a unique epistemic probability distribution. ... In statistics, the score is the derivative, with respect to some parameter θ, of the logarithm of the likelihood function. ...

Notes

1. ^ Steffen L. Lauritzen, Aspects of T. N. Thiele's Contributions to Statistics (1999).
2. ^ Ronald A. Fisher. "On the mathematical foundations of theoretical statistics". Philosophical Transactions of the Royal Society, A, 222:309-368 (1922). ("Likelihood" is discussed in section 6.)

Sir Ronald Fisher Sir Ronald Aylmer Fisher, FRS (February 17, 1890–July 29, 1962) was an extraordinarily talented evolutionary biologist, geneticist and statistician. ...

References

• A. W. F. Edwards (1972). Likelihood: An account of the statistical concept of likelihood and its application to scientific inference, Cambridge University Press. Reprinted in 1992, expanded edition, Johns Hopkins University Press.