AR mining Implementation and comparison of three AR mining algorithms Xuehai Wang, Xiaobo Chen, Shen...

AR mining

Implementation and comparison of three AR mining algorithms

Xuehai Wang, Xiaobo Chen, Shen chen

CSCI6405 class project

AR mining

Outline

• Motivation

• Dataset

• Apriori based hash tree algorithm

• FP-tree algorithm

• Conclusion

• Reference

AR mining

Motivation

• Make the time of generating rules as shot as possible!

• To understand the three algorithms– Apriori algorithm– Apriori with hash tree algorithm– FP-tree algorithm

• Learn how to improve an algorithm

AR mining

Dataset• IBM dataset generator

– Can set item number– Can set minimal support– Can set dataset size

1 2 5 8 9

2 3 4 6 7 12

Tid item

AR mining

Apriori principle

• Apriori principle– A candidate generation-and-test Approach [4]– Given a frequent itemset, its subset must be fre

quent– A set is infrequent, its super set will not be gene

rated and tested

• But there is still some places can be improved– Count the support– I/O scan times

AR mining

Apriori Hash Tree Alg

• Candidate K-itemset size is l• There is n transactions• Average transaction size is m• Calculate support count:

– Original Apriori Alg:

– With hash tree: O( n.log(l).(mk) )

)( mklnO

)log( mklnO

AR mining

Apriori Hash Tree Alg

• Candidate is stored in a hash tree structure

Tid Items

1 1 2

2 1 3 6

3 1 2 3

4 2 4

5 2 3 6

6 5 6

1-itemset candidate hash tree

1(1)2(1)1(2)

3(1)

1(2) 3(1)2(1)

AR mining

Apriori Hash Tree Alg

Tid

Items

1 1 2

2 1 3 6

3 1 2 3

4 2 4

5 2 3 6

6 5 6

2(4)5(1) 6(3)

1(3) 3(3)4(1)

1itemset , Min support = 2

AR mining

Apriori Hash Tree Alg

Tid

Items

1 1 2

2 1 3 6

3 1 2 3

4 2 4

5 2 3 6

6 5 6

2 3(2)2 6(1)

1 3(2)1 2(2)

3 6(2)

1 6(1)

2 itemset, Min support = 2

3 itemset, Min support = 2

1 2 3(1)

AR mining

FP-tree

• Since the mining dataset is always very huge, it’s impossible to read all transactions into computer memory all in once.

• But I/O scan is very time consuming.

• FP-tree algorithm will try to suite all information from the dataset into computer memory, hence only need to scan I/O two times.

AR mining

FP-tree

• FP-tree algorithm and implementation– By Xiaobo Chen

AR mining

FP-tree (Frequent Pattern Tree)

• Mining frequent pattern without candidate generation

• Divide and conquer methodology: decompose mining tasks into smaller ones

AR mining

FP-tree (Merits of FP-tree algorithm)

• Make most use of common shared prefix

• Complete and compact

All information of a transaction is

stored in a path

The size is constrained by the data set consequently, the longest path corresponds to the longest

pattern

The compact ratio: over 100

AR mining

FP-tree (Construction of FP-tree)

• TID freq. Items bought

• 100 {f, c, a, m, p}

• 200 {f, c, a, b, m}

• 300 {f, b}

• 400 {c, p, b}

• 500 {f, c, a, m, p}

min_support = 3Item frequency f 4c 4a 3b 3m 3p 3

f:1

c:1

a:1

m:1

p:1

root

AR mining

FP-tree (construction (Cont’d))TID freq. Items bought100 {f, c, a, m, p}200 {f, c, a, b, m}300 {f, b}400 {c, p, b}500 {f, c, a, m, p}

f:2

c:2

a:2

m:1

p:1

b:1

m:1

root

AR mining

FP-tree construction (Cont’d)• TID freq. Items bought

• 100 {f, c, a, m, p}

• 200 {f, c, a, b, m}

• 300 {f, b}

• 400 {c, p, b}

• 500 {f, c, a, m, p}

min_support = 3Item frequency f 4c 4a 3b 3m 3p 3Header Table

Item frequency head f 4c 4a 3b 3m 3p 3

f:4

c:3

a:3

m:2

p:2

b:1

m:1

b:1

c:1

b:1

p:1

root

AR mining

FP-tree (Mining Frequent Patterns Using the FP-tree)

• General idea (divide-and-conquer)– Recursively grow frequent pattern path using the FP-

tree

• Method – For each item, construct its conditional pattern-base,

and then its conditional FP-tree

– Repeat the process on each newly created conditional FP-tree

– Until the resulting FP-tree is empty, or it contains only one path (single path will generate all the combinations of its sub-paths, each of which is a frequent pattern)

AR mining

FP-tree (Mining Frequent Patterns Using the FP-tree)

Conditional pattern base for p

fcam:2, cb:1

f:4

c:3

a:3

m:2

p:2

c:1

b:1

p:1

p

• Start with last item in order (i.e., p).

• Follow node pointers and traverse only the paths containing p.

• Accumulate all of transformed prefix paths of that item to form a conditional pattern base

root

Constructing a new FP-tree based on this pattern base leads to only one branch c:3Thus we derive only one frequent pattern cont. p. Pattern cp

AR mining

FP-tree (Mining Frequent Patterns Using the FP-tree)

• Move to next least frequent item in order, i.e., m

• Follow node pointers and traverse only the paths containing m.

• Accumulate all of transformed prefix paths of that item to form a conditional pattern base

Conditional pattern base for m

fca:2, fcab:1

f:4

c:3

a:3

m:2

m

m:1

b:1

Constructing a new FP-tree based on this pattern base leads to path fca:3From this we derive frequent patterns fcam, fcm, cam, fm, cm, am

root

AR mining

FP-tree (Conditional Pattern-Bases for the example)

EmptyEmptyf

{(f:3)}|c{(f:3)}c

{(f:3, c:3)}|a{(fc:3)}a

Empty{(fca:1), (f:1), (c:1)}b

{(f:3, c:3, a:3)}|m{(fca:2), (fcab:1)}m

{(c:3)}|p{(fcam:2), (cb:1)}p

Conditional FP-treeConditional pattern-baseItem

AR mining

FP-tree (Why is Frequent pattern Growth fast?)

• Performance studies show that

FP-growth is an order of magnitude faster than

Apriori, and is also faster than tree-projection

• Reasoning:

– No candidate generation, no candidate test

– Use compact data structure

– Eliminate repeated database scan

– Basic operation is counting and FP-tree building

AR mining

FP-tree: Expected result: FP-growth vs. Apriori: Scalability With the Support Threshold

0

10

20

30

40

50

60

70

80

90

100

0 0.5 1 1.5 2 2.5 3

Support threshold(%)

Ru

n t

ime(s

ec.)

D1 FP-grow th runtime

D1 Apriori runtime

AR mining

Conclusion

• FP-tree is faster than other two algorithms.

• Apriori as well as hash tree algorithms are easier to implement.– We can easily combine them with other

methods or tools. (i.e. distributed parallel computing).

• The parameter of dataset is very important too.– Density, size, min support …

AR mining

References

• [1] Jiawei Han and Micheline Kamber: "Data Mining: Concepts and Techniques ", Morgan Kaufmann, 2001

• [2] Jiawei Han, Jian Pei, Yiwen Yin: Mining Frequent Patterns without Candidate Generation, ACM SIGMOD, 2000

• [3] N.Mamoulis, Advanced Database Technologies (Slides)

• [4] Jiawei Han and Micheline Kamber. Data Mining - Concepts and Techniques. MorganKaufmann Publishers, 2001.