1

Spam Filtering Using Bayesian Approach

Presented by: Nitin Kumar

2

References

• What you need to know about Bayesian Spam filtering

http://email.about.com/cs/bayesianfilters/a/bayesian_filter_2.htm

• A plan for Spam by Paul Graham

http://www.paulgraham.com/spam.html

• Genetic Algorithm Tutorial

http://cs.felk.cvut.cz/%7Exobitko/ga/

• Spam filtering using Bayesian Approach

3

Advantages of Bayesian Method

•Bayesian approach is self adapting. It keeps learning from the new spams.

•Bayesian method takes whole message into account.

•Bayesian method is easy to use and very accurate (Claimed Accuracy Percentage is 97).

•Bayesian approach is multi-lingual.

• Reduces the number of false positives.

4

Project Goal

1. Getting a learning set of various Spam and Ham emails.

2. Parsing the individual mails to extract the words of interest.

3. Implementing the Naïve Bayesian Method.

5

Learning Set

• The learning set will consist of:

2500 Ham emails & 4000 Spam emails

•The testing set will consist of:

2700 Ham emails & 1400 Spam emails

Spam Mail archive: http://spamassassin.org/publiccorpus/

6

Parsing Words of Interest To Form Tokens

• Remove the common words (e.g. you, I, for, is, are, etc).

• Strip off the HTML tags and punctuation marks.

• Multiple occurrence of a word in a single mail should not increase its count.

• Token may be formed from a pair of words.

7

Naïve Bayesian Algorithm

•Naïve Bayesian method is used for the learning process. Analyze a mail to calculate its probability of being a Spam using individual characteristic of words in the mail.

•For each word in the mail, Calculate the following:

S(w)=(number of Spam emails containing the word)/(total number of Spam emails)

H(w)=(number of Ham emails containing the word)/(total number of Ham emails)

P(w)=S(w)/(S(w)+H(w))

•P(w) can be interpreted as the probability that a randomly chosen email containing the word w is Spam.

8

Exceptions

Say a word w =“success” appears only once and it is a Spam email. Then the above formula calculates P(w)=1.

This doesn’t mean that all future mails containing this word will be considered as Spam. It will rather depend upon its degree of beliefdegree of belief. The Bayesian method allow us to combine our intuitive background information with this collected data.

Degree of belief f(w)= [(s*x)+(n*p(w))]/(s+n)

s=Assumed strength of the background information.

x= Assumed probability of the background information.

n= no of emails received containing word w.

9

Combining The Probabilities

Each email is represented by a set of probabilities. Combining these individual probabilities gives the overall indicator of spamminess.

Fisher’s Method:

H= Chi_inverse(-2*ln(Product of all(f(w)), 2*n)

S= Chi_inverse(-2*ln(Product of all(1-f(w)), 2*n)

I= [1+H-S]/2

Here, I is the Indicator of Spamminess.

10

Genetic Algorithm

A mail can be divided into three parts:

•Body

•From

•Subject

Genetic Algorithm can be used to get an appropriate weights say α, β and γ for “body” part, “from”part and “subject” part.

IFinal= α*IBody+ β*IFrom+ γ*ISubject

The overall accuracy is a function of α, β and γ. Genetic Algorithm maximize the above function.

11

Result

A mail is declared as SPAM, if the value of IFinal is greater than a threshold value.