In statistics, the use of Bayes factors is a Bayesian alternative to classical hypothesis testing. An example of statistics used in educational assessment. ... Bayesian refers to probability and statistics -- either methods associated with the Reverend Thomas Bayes (ca. ... One may be faced with the problem of making a definite decision with respect to an uncertain hypothesis which is known only through its observable consequences. ...

Given a model selection problem in which we have to choose between two models M₁ and M₂, on the basis of a data vector x. The Bayes factor K is given by A mathematical model is an abstract model that uses mathematical language to describe the behaviour of a system. ... In statistics, a data point is a single typed measurement. ...

This is similar to a likelihood-ratio test, but instead of maximising the likelihood Bayesians average it over the parameters. Generally, the models M₁ and M₂ will be parametrised by vectors of parameters θ₁ and θ₂; thus K is given by A likelihood-ratio test is a statistical test relying on a test statistic computed by taking the ratio of the maximum value of the likelihood function under the constraint of the null hypothesis to the maximum with that constraint relaxed. ... Graph of a butterfly curve, a parametric equation discovered by Temple H. Fay In mathematics, a parametric equation explicitly relates two or more variables in terms of one or more independent parameters. ... A parameter is a measurement or value on which something else depends. ...

The logarithm of K is sometimes called the weight of evidence given by x for M₁ over M₂, measured in bits, nats, or bans, according to whether the logarithm is taken to base 2, base e, or base 10. This article is about the unit of information. ... A nat is a logarithmic unit of information or entropy, based on natural logarithms and powers of e, rather than the powers of 2 and base 2 logarithms which define the bit. ... A ban is a logarithmic unit which measures information or entropy, based on base 10 logarithms and powers of 10, rather than the powers of 2 and base 2 logarithms which define the bit. ...

A value of K > 1 means that the data indicate that M₁ is more likely than M₂ and vice versa. Note that classical hypothesis testing gives one hypothesis (or model) preferred status (the 'null hypothesis'), and only considers evidence against it. Harold Jeffreys gave a scale for interpretation of K: One may be faced with the problem of making a definite decision with respect to an uncertain hypothesis which is known only through its observable consequences. ... Sir Harold Jeffreys (22 April 1891 - 18 March 1989) was a mathematician, statistician, geophysicist, and astronomer. ...

K	dB	Strength of evidence
< 1:1	< 0	Negative (supports M2)
1:1 to 3:1	0 to 5	Barely worth mentioning
3:1 to 12:1	5 to 11	Positive
12:1 to 150:1	11 to 22	Strong
> 150:1	> 22	Very strong

The second column gives the corresponding weights of evidence in decibans (tenths of a power of 10). According to I. J. Good a change in a weight of evidence of 1 deciban (ie a change in an odds ratio from evens to about 55:45) is about as finely as humans can reasonably perceive their degree of belief in a hypothesis in everyday use. A ban, sometimes called a hartley, is a logarithmic unit which measures information or entropy, based on base 10 logarithms and powers of 10, rather than the powers of 2 and base 2 logarithms which define the bit. ... Irving John (Jack) Good (born 9 December 1916) is a British statistician who worked also as a cryptographer and developer of the Colossus computer at Bletchley Park. ... Bayesianism is the philosophical tenet that the mathematical theory of probability applies to the degree of plausibility of a statement. ...

The use of Bayes factors or classical hypothesis testing takes place in the context of inference rather than decision-making under uncertainty. That is, we merely wish to find out which hypothesis is true, rather than actually making a decision on the basis of this information. Frequentist statistics draws a strong distinction between these two because hypothesis tests are not coherent in the Bayesian sense. Bayesian procedures, including Bayes factors are coherent, so there is no need to draw such a distinction. Inference is then simply regarded as a special case of decision-making under uncertainty in which the resulting action is to report a value. In a decision-making context Bayesian statisticians might use a Bayes factor as part of making a choice, but would also combine it with a prior distribution and a loss function associated with making the wrong choice. In an inference context the loss function would take the form of a score function. Use of a logarithmic score function for example, leads to the expected utility taking the form of the Kullback-Leibler divergence. If the logarithms are to the base 2 this is equivalent to Shannon information. ... Statistical regularity has motivated the development of the relative frequency concept of probability. ... In a thought experiment proposed by the Italian probabilist Bruno de Finetti in order to justify Bayesian probability, an array of wagers is coherent precisely if it does not expose the wagerer to certain loss if his opponent is prudent. ... A prior probability is a marginal probability, interpreted as a description of what is known about a variable in the absence of some evidence. ... In statistics, decision theory and economics, a loss function is a function that maps an event (technically an element of a sample space) onto a real number representing the economic cost or regret associated with the event. ... In [economics]], utility is a measure of the happiness or satisfaction gained consuming good and services. ... In probability theory and information theory, the Kullback-Leibler divergence, or relative entropy, or information divergence is a natural distance measure from a true probability distribution P to arbitrary probability distribution Q. Typically P represents data, observations, or a precise calculated probability distribution. ...

Example

Suppose we have a random variable which produces either a success or a failure. We want to consider a model M₁ where the probability of success is q=½, and another model M₂ where q is completely unknown and we take a prior distribution for q which is uniform on [0,1]. We take a sample of 200, and find 115 success and 85 failures. The likelihood is: A random variable can be thought of as the numeric result of operating a non-deterministic mechanism or performing a non-deterministic experiment to generate a random result. ... A prior probability is a marginal probability, interpreted as a description of what is known about a variable in the absence of some evidence. ... In mathematics, the uniform distributions are simple probability distributions. ...

So we have

but

The ratio is then 1.197..., which is "barely worth mentioning" even if it points very slightly towards M₁.

This is not the same as a classical likelihood ratio test, which would have found the maximum likelihood estimate for q, namely ¹¹⁵⁄₂₀₀=0.575, and from that get a ratio of 0.1045..., and so pointing towards M₂. A frequentist hypothesis test would have produced an even more dramatic result, saying that that M₁ could be rejected at the 5% confidence level, since the probability of getting 115 or more successes from a sample of 200 if q=½ is 0.0200..., and as a two-tailed test of getting a figure as extreme as or more extreme than 115 is 0.0400... Note that 115 is more than two standard deviations away from 100. Maximum likelihood estimation (MLE) is a popular statistical method used to make inferences about parameters of the underlying probability distribution of a given data set. ... Statistical regularity has motivated the development of the relative frequency concept of probability. ...

M₂ is a more complex model than M₁ because it has a free parameter which allows it to model the data more closely. The ability of Bayes factors to take this into account is a reason why Bayesian inference has been put forward as a theoretical justification for and generalisation of Occam's razor, reducing Type I errors. Bayesian inference is a statistical inference in which probabilities are interpreted not as frequencies or proportions or the like, but rather as degrees of belief. ... Occams Razor (also spelled Ockhams Razor), is a principle attributed to the 14th-century English logician and Franciscan friar, William of Ockham. ... In statistical hypothesis testing, a Type I error consists of rejecting a null hypothesis that is true, in other words finding a result to have statistical significance when this has in fact happened by chance. ...

See also

• Bayesian model comparison

The posterior probability of a model given data, P(H|D), is given by Bayes theorem: P(H|D) = P(D|H)P(H)/P(D) The key data_dependent term P(D|H) is a likelihood, and is sometimes called the evidence for model H; evaluating it correctly is the...

External links

• Bayesian critique of classical hypothesis testing
• Why should clinicians care about Bayesian methods?