Bayesian spam filtering (pronounced "Bays-ee-en", IPA pronunciation: ['beɪz.i.ən], after Rev. Thomas Bayes), a form of e-mail filtering, is the process of using a Naive Bayes classifier to identify spam email. Articles with similar titles include the NATO phonetic alphabet, which has also informally been called the “International Phonetic Alphabet”. For information on how to read IPA transcriptions of English words, see IPA chart for English. ... Thomas Bayes (c. ... Email filtering is the processing of e-mail to organize it according to specified criterion. ... A naive Bayes classifier (also known as Idiots Bayes) is a simple probabilistic classifier based on applying Bayes theorem with strong (naive) independence assumptions. ...

Bayesian filtering was proposed by Sahami et al. (1998)^[1] and gained attention in 2002 when it was described in a paper by Paul Graham.^[2] Since then it has become a popular mechanism to distinguish illegitimate spam email from legitimate email (sometimes called ham). Many modern mail programs implement Bayesian spam filtering. Users can also install separate email filtering programs. Server-side email filters, such as SpamAssassin, SpamBayes, Bogofilter and ASSP, make use of Bayesian spam filtering techniques, and the functionality is sometimes embedded within mail server software itself. Paul Graham For Paul Graham the photographer, see Paul Graham (photographer). ... E-mail spam is a subset of spam that involves sending nearly identical messages to numerous recipients by e-mail. ... Electronic mail, abbreviated e-mail or email, is a method of composing, sending, and receiving messages over electronic communication systems. ... Email filtering is the processing of e-mail to organize it according to specified criterion. ... In computer networking, the term server-side refers to operations that are performed by the server in a client-server relationship. ... SpamAssassin is a computer program released freely under the Apache License 2. ... SpamBayes is a Bayesian spam filter written in Python using techniques by Paul Graham improved by Gary Robinson and Tim Peters, among others. ... Bogofilter is a mail filter that classifies e-mail as spam or ham (non-spam) by a statistical analysis of the messages header and content (body). ... The Anti-Spam SMTP Proxy (ASSP) server project is an Open Source platform-independent transparent SMTP proxy server that leverages numerous methodologies and technologies to both rigidly and adaptively identify spam. ... A mail transfer agent or MTA (also called a mail server, or a mail exchange server in the context of the Domain Name System) is a computer program or software agent which transfers electronic mail messages from one computer to another. ...

Bayesian poisoning is a technique used by spammers in an attempt to degrade the effectiveness of spam filters that rely on Bayesian filtering. A spammer practicing Bayesian poisoning will send out emails with large amounts of legitimate text (gathered from legitimate news or literary sources). Bayesian poisoning is a technique used by spammers to degrade the effectiveness of spam filters that rely on bayesian filtering. ...

Mathematical foundation

Bayesian email filters take advantage of Bayes's theorem. Bayes's theorem, in the context of spam, says that the probability that an email is spam, given that it has certain words in it, is equal to the probability of finding those certain words in spam email, times the probability that any email is spam, divided by the probability of finding those words in any email: A mail filter is a piece of software which takes an input of an email message. ... Bayes theorem is a result in probability theory, which gives the conditional probability distribution of a random variable A given B in terms of the conditional probability distribution of variable B given A and the marginal probability distribution of A alone. ...

Process

Particular words have particular probabilities of occurring in spam email and in legitimate email. For instance, most email users will frequently encounter the word Viagra in spam email, but will seldom see it in other email. The filter doesn't know these probabilities in advance, and must first be trained so it can build them up. To train the filter, the user must manually indicate whether a new email is spam or not. For all words in each training email, the filter will adjust the probabilities that each word will appear in spam or legitimate email in its database. For instance, Bayesian spam filters will typically have learned a very high spam probability for the words "Viagra" and "refinance", but a very low spam probability for words seen only in legitimate email, such as the names of friends and family members. Probability is the chance that something is likely to happen or be the case. ... // ...

After training, the word probabilities (also known as likelihood functions) are used to compute the probability that an email with a particular set of words in it belongs to either category. Each word in the email contributes to the email's spam probability. This contribution is called the posterior probability and is computed using Bayes' theorem. Then, the email's spam probability is computed over all words in the email, and if the total exceeds a certain threshold (say 95%), the filter will mark the email as a spam. Email marked as spam can then be automatically moved to a "Junk" email folder, or even deleted outright. Look up likelihood in Wiktionary, the free dictionary. ... The posterior probability can be calculated by Bayes theorem from the prior probability and the likelihood function. ...

Advantages

The advantage of Bayesian spam filtering is that it can be trained on a per-user basis.

The spam that a user receives is often related to the online user's activities. For example, a user may have been subscribed to an online newsletter that the user considers to be spam. This online newsletter is likely to contain words that are common to all newsletters, such as the name of the newsletter and its originating email address. A Bayesian spam filter will eventually assign a higher probability based on the user's specific patterns.

The legitimate e-mails a user receives will tend to be different. For example, in a corporate environment, the company name and the names of clients or customers will be mentioned often. The filter will assign a lower spam probability to emails containing those names.

The word probabilities are unique to each user and can evolve over time with corrective training whenever the filter incorrectly classifies an email. As a result, Bayesian spam filtering accuracy after training is often superior to pre-defined rules.

It can perform particularly well in avoiding false positives, where legitimate email is incorrectly classified as spam. For example, if the email contains the word "Nigeria", which frequently appeared in a long spam campaign, a pre-defined rules filter might reject it outright. A Bayesian filter would mark the word "Nigeria" as a probable spam word, but would take into account other important words that usually indicate legitimate e-mail. For example, the name of a spouse may strongly indicate the e-mail is not spam, which could overcome the use of the word "Nigeria."

Some spam filters combine the results of both Bayesian spam filtering and pre-defined rules resulting in even higher filtering accuracy. Recent spammer tactics include insertion of random innocuous words that are not normally associated with spam, thereby decreasing the email's spam score, making it more likely to slip past a Bayesian spam filter. E-mail spam is a subset of spam that involves sending nearly identical messages to numerous recipients by e-mail. ...

General applications of Bayesian filtering

While Bayesian filtering is used widely to identify spam email, the technique can classify (or "cluster") almost any sort of data. It has uses in science, medicine, and engineering. One example is a general purpose classification program called AutoClass which was originally used to classify stars according to spectral characteristics that were otherwise too subtle to notice. There is recent speculation that even the brain uses Bayesian methods to classify sensory stimuli and decide on behavioural responses (Trends in Neuroscience, 27(12):712-9, 2004) (pdf).

See also

• Bayesian inference
• Bayes's theorem
• Email filtering
• Naive Bayes classifier
• Recursive Bayesian estimation
• Stopping e-mail abuse

Bayesian inference is statistical inference in which evidence or observations are used to update or to newly infer the probability that a hypothesis may be true. ... Bayes theorem is a result in probability theory, which gives the conditional probability distribution of a random variable A given B in terms of the conditional probability distribution of variable B given A and the marginal probability distribution of A alone. ... Email filtering is the processing of electronic mail to organise it according to specified criterion. ... A naive Bayes classifier (also known as Idiots Bayes) is a simple probabilistic classifier based on applying Bayes theorem with strong (naive) independence assumptions. ... It has been suggested that this article or section be merged with Sequential_bayesian_filtering. ... This article or section does not cite its references or sources. ...

External links

• Guide to Bayesian spam filters: part 1, part 2.
• Detailed explanation of Paul Graham's formulas, which lack mathematical rigor

References

1. ^ M. Sahami, S. Dumais, D. Heckerman, E. Horvitz (1998). "A Bayesian approach to filtering junk e-mail". AAAI'98 Workshop on Learning for Text Categorization.
2. ^ Graham, Paul (2002). A Plan for Spam.