Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns...

CS102Data Mining

Data Mining Algorithms

CS102Spring 2020

CS102Data Mining

Data Tools and Techniques

§ Basic Data Manipulation and AnalysisPerforming well-defined computations or asking well-defined questions (“queries”)

§ Data MiningLooking for patterns in data

§ Machine LearningUsing data to build models and make predictions

§ Data VisualizationGraphical depiction of data

§ Data Collection and Preparation

CS102Data Mining

Data MiningLooking for patterns in data

Similar to unsupervised machine learning• Popularity predates popularity of machine learning• “Data mining” often associated with specific data

types and patterns

We will focus on “market-basket” data• Widely applicable (despite the name)

And two types of data mining patterns• Frequent item-sets• Association rules

CS102Data Mining

Other Data and Patterns

Other types of data• Networks/graphs• Streams• Text (“text mining”)

Other patterns• Similar items• Structural patterns in large graphs/networks• Clusters, anomalies

Specific techniquesfor each one

CS102Data Mining

(In)Famous Early Success Stories

Victoria’s Secret

Walmart

Beer & Diapers

CS102Data Mining

Market-Basket Data

Originated with retail data• Each shopper buys “market basket” of groceries• Mine data for patterns in buying habits

General definition• Domain of items• Transaction - one or more items occurring

together• Dataset – set of transactions (usually large)

CS102Data Mining

Market-Basket Examples

Items Transaction

CS102Data Mining

Items Transaction

Groceries

CS102Data Mining

Items Transaction

Groceries Grocery cart

CS102Data Mining

Items Transaction

Groceries Grocery cartOnline goods

CS102Data Mining

Items Transaction

Groceries Grocery cartOnline goods Virtual shopping cart

CS102Data Mining

Items Transaction

University courses

CS102Data Mining

Items Transaction

University courses Student transcript

CS102Data Mining

Items Transaction

University courses Student transcriptUniversity students

CS102Data Mining

Items Transaction

University courses Student transcriptUniversity students Party

CS102Data Mining

Items Transaction

Movies

CS102Data Mining

Items Transaction

Movies Person

CS102Data Mining

Items Transaction

Movies PersonSymptoms

CS102Data Mining

Items Transaction

Movies PersonSymptoms Patient

CS102Data Mining

Items Transaction

Movies PersonSymptoms PatientMenu items

CS102Data Mining

Items Transaction

Movies PersonSymptoms PatientMenu items Restaurant customer

CS102Data Mining

Items Transaction

CS102Data Mining

Items Transaction

Words Document

CS102Data Mining

Frequent Item-Sets – sets of items that occur frequently together in transactions• Groceries bought together• Courses taken by same students• Students going to parties together• Movies watched by same people

Association Rules – When certain items occur together, another item frequently occurs with them• Shoppers who buy phone + charger also buy case• Students who take Databases also take Machine

Learning• Diners who order curry and rice also order bread

CS102Data Mining

Frequent Item-SetsSets of items that occur frequently

together in transactions

ØHow large is a “set”?ØWhat does “frequently” mean?

CS102Data Mining

Frequent Item-SetsSets of items that occur frequently

together in transactions

ØHow large is a “set”?Usually specify a minimum min-set-sizePossibly also a maximum max-set-size

ØWhat does “frequently” mean?Notion of support

CS102Data Mining

Support

Support for a set of items S in a dataset of transactions is the fraction of the transactions containing S:

Specify support-threshold for frequent item-setsOnly return sets wheresupport > support-threshold

# of transactions containing S____________________________________

total # of transactions

CS102Data Mining

Your TurnTransactions:

T1: milk, eggs, juiceT2: milk, juice, cookiesT3: eggs, chipsT4: milk, eggsT5: milk, juice, cookies, chips

What are the frequent item-sets if:• min-set-size = 2 (no max-set-size)• support-threshold = 0.3

Support:# of transactions containing S

____________________________________

CS102Data Mining

Computing Frequent Item-Sets“Apriori” algorithm

Efficiency relies on the following property:

Or the inverse:

If S is a frequent item-set satisfyingsupport-threshold t, then every subset of S is alsoa frequent item-set satisfying support-threshold t.

If S is not a frequent item-set satisfyingsupport-threshold t, then no superset of S can bea frequent item-set satisfying support-threshold t.

CS102Data Mining

Association RulesWhen a set of items S occurs together,

another item i frequently occurs with themS à i

ØHow large is a “set”?ØWhat does “occurs together” mean?ØWhat does “frequently occurs with them” mean?

CS102Data Mining

ØHow large is a “set”?Usually specify a minimum min-set-size for SPossibly also a maximum max-set-size for S

ØWhat does “occurs together” mean?ØWhat does “frequently occurs with them” mean?

CS102Data Mining

ØHow large is a “set”?Usually specify a minimum min-set-size for SPossibly also a maximum max-set-size for S

ØWhat does “occurs together” mean?Notion of support

ØWhat does “frequently occurs with them” mean?Notion of confidence

CS102Data Mining

Support and ConfidenceSupport for association rule S à i in a dataset of transactions is fraction of transactions containing S:

Confidence for association rule S à i in a dataset of transactions is the fraction of transactions containing S that also contain i:

# of transactions containing S and i__________________________________________

# of transactions containing S

CS102Data Mining

Support and Confidence

Specify support-threshold and confidence-threshold for association rules

Only return rules where:support > support-threshold andconfidence > confidence-threshold

CS102Data Mining

Your TurnTransactions:

What are the association rules S à i if:• min-set-size = 1 (no max-set-size)• support-threshold = 0.5• confidence-threshold = 0.5

# of transactions containing SConfidence:

Support:

Reminder: support andconfidence must be >

threshold, not ≥

CS102Data Mining

Computing Association Rules

1. Use frequent item-sets to find left-hand sides S satisfying support threshold

2. Then extend to find right-hand sides S à isatisfying confidence threshold

NOT a property:

If S à i is an association rule satisfyingsupport-threshold t and confidence-threshold c,

and S’Í S, then S’ à i is an an associationrule satisfying support-threshold t and

confidence-threshold c.

Why Not?

CS102Data Mining

Association Rules: LiftAssociation rule S à i might have high confidence because item i appears frequently, not because it’s associated with S.

Confidence for association rule S à i in a dataset of transactions is the fraction of transactions containing S that also contain i :

# of transactions containing S

divided by the overall frequency of i :,

#trans containing S and i #trans containing i______________________________ ÷ _______________________

#trans containing S total #trans

CS102Data Mining

Lift: ExamplesTransactions:

juice à cookies Lift = (2/3) ÷ (2/5) = 10/6 = 1.67

eggs à milk Lift = (2/3) ÷ (4/5) = 10/12 = 0.83

Lift:#trans containing S and i #trans containing i

______________________________ ÷ _______________________

#trans containing S total #trans

Lift = 1: no associationLift > 1: associationLift < 1: anti-association

CS102Data Mining

CS102Spring 2020

Data Mining Algorithms - Stanford University · Data Mining CS102 Data Mining Looking for patterns...

Documents

What is Data Mining? Data Mining Motivation Data Mining Applications Applications of Data Mining in CRM Data Mining Taxonomy Data Mining Techniques

Data Mining Taylor Statistics 202: Data Mining

Text Mining - Data Mining

Applied Data Mining - Lagout Mining/Applied Data Mining [Xu... · The book reviews applied data mining from theoretical basis to ... data mining application arenas, ... research topics

Secure Data Processing - uni-leipzig.de · PRIVACY-PRESERVING DATA MINING 12 Local Data Local Data Local Data Warehouse Data Mining Local Data Mining Local Data Mining Combiner Local

CS590D: Data Mining Chris Clifton - Purdue University · Data Mining: Classification Schemes • General functionality – Descriptive data mining – Predictive data mining ... –

Data Mining LECTURE # 01 Introduction to Data Mining

September 4, 20151 Chapter 1. Introduction Motivation: Why data mining? What is data mining? Data Mining: On what kind of data? Data mining functionality

Web Mining – Data Mining im Internet Mining – Data Mining im Internet Vorlesung SS 2010 ... Web Mining is Data Mining for Data on the World-Wide Web Text Mining: Application of

Data Mining: What is Data Mining?

Data Mining für Business Intelligence Data Mining for

Data Mining Introduction to Data Mining

Data Mining and Applications - antoniomucherino.it · Data Mining and Applications Data Mining Why Data Mining? Introduction to Data Mining Example III - text mining Let us suppose

Massive Data Analytics Data Mining Introductiondisi.unitn.it/.../MassiveDataAnalytics/slides/DataMiningIntro-2in1.pdfMassive Data Analytics Data Mining Introduction ... Data mining,

Introduction to Data Mining - homepages.math.uic.eduhomepages.math.uic.edu/~jyang06/stat486/R/DataMining_JYang2014.pdfFundamentals of Data Mining Typical Data Mining Tasks Data Mining

Web Mining – Data Mining im Internet Mining – Data Mining im Internet Vorlesung SS 2014 ... Web Mining is Data Mining for Data on the World-Wide Web Text Mining: Application of

Gufran Ahmad. Contents What is Data Mining? Data Mining / KDD process Different aspects of Data Mining Why Data Mining? Data Mining in Business Examples

UNIT - I Data Mining. UNIT - I Introduction : Fundamentals of data mining, Data Mining Functionalities, Classification of Data Mining systems, Major issues

Data Mining - The opportunity lies with Data!Data mining

Data Mining: The Next Generation - Dagstuhldrops.dagstuhl.de/opus/volltexte/2005/270/pdf/... · Data Mining: The Next Generation 1 1 Introduction Data Mining has enjoyed great popularity