Upload
laureen-webb
View
219
Download
2
Tags:
Embed Size (px)
Citation preview
Fox MISSpring 2011
Data Mining
Week 9Introduction to Data Mining
Data WarehouseData Warehouse
Customer No. Name Address Membership
Product No. Product Name Price Description
External Source
MySQL
ERD
Data Mining
Competitive Advantage Performance
Good Business Decision Better Understanding
Defining User Communities
• Information user– Generally requires standard reports and
that often includes charts and tables– Wants to scan consistently structured
reports without needing slice or dice to find the desired values
– Static or simple interactive reports• Information consumer
– Requires the ability to dynamically query the database, without becoming an expert at database design or the query tool
– Ad-hoc multidimensional analysis– Many business people cross the line
between information users and information consumers
• Power analyst– Require the full analytical power of the
data mart in order to perform free-form ad hoc analysis
Some Questions Analysts Need to Answers
• Sales analysis:– What are the sales by quarter and geography?– How do sales compare in two different stores in the same
state?
• Profitability analysis:– Which is the most profitable store in the state CA? – Which product lines are the highest revenue producers this
year?– Which products and product lines are the most profitable
this quarter?
• Sale force analysis– Which salesperson is the best revenue producer this year?
Do salesperson X meet his sale target this quarter?
Finding a Pattern from Data• Tenure and sick days by department
– Average tenure for each department: 9.0– Average number of sick days is 7.5 for each
Finding a Pattern: Graphical Representation
Data Analysis Evolutionary Step
Evolutionary Step Business Question Enabling Technologies Characteristics
Data Collection (1960s)
"What was my total revenue in the last five years?"
Computers, tapes, disks Retrospective,static data delivery
Data Access (1980s) "What were unit sales in New England last March?"
Relational databases (RDBMS), Structured Query Language (SQL)
Retrospective, dynamic data delivery at record level
Data Warehousing & Decision Support(1990s)
"What were unit sales in New England last March? Drill down to Boston."
On-line analytic processing (OLAP), multidimensional databases, data warehouses
Retrospective, dynamic data delivery at multiple levels
Data Mining (Emerging Today)
"What’s likely to happen to Boston unit sales next month? Why?"
Advanced algorithms,multiprocessor computers, massive databases
Prospective, proactive information delivery
• The application of specific algorithms for extracting patterns from data
• Data mining tools automatically search data for patterns and relationships
• Data mining tools– Analyze data– Uncover problems or opportunities– Form computer models based on findings– Predict business behavior with models– Require minimal end-user intervention
Data Mining
Data Mining
• Goal– Simplification and automation of the overall
statistical process, from data source(s) to model application
• Data mining is ready for application in the business community because it is supported by three technologies that are now sufficiently mature: – Massive data collection – Powerful multiprocessor computers – Data mining algorithms
Convergence of Three Key Technologies
Data Mining and Knowledge Discovery in the Real World
• Marketing– If customer bought X, he/she is also likely to
buy Y and Z• Investment
– Stock investment• Fraud detection
– Identify financial transactions that might indicate money-laundering activity
A Problem...• You are a marketing manager for a brokerage
company
• Problem: Churn is too high– Turnover (after six month introductory period
ends) is 40%– Customers receive incentives (average cost:
$160) when account is opened– Giving new incentives to everyone who might
leave is very expensive (as well as wasteful)– Bringing back a customer after they leave is both
difficult and costly
… A Solution
• One month before the end of the introductory period is over, predict which customers will leave
• If you want to keep a customer that is predicted to churn, offer them something based on their predicted value
• The ones that are not predicted to churn need no attention
A weather problem
A numeric weather problem
Benefit of Data Mining• New business opportunities by providing these
capabilities: • Automated prediction of trends and behaviors
– Targeted marketing.• Promotional mailings to identify the targets most likely to
maximize return on investment in future mailings. – Forecasting bankruptcy and other forms of default
• Automated discovery of previously unknown patterns. – Data mining tools sweep through databases and
identify previously hidden patterns in one step– Analysis of retail sales data to identify seemingly
unrelated products that are often purchased together
Descriptive Data Mining
• Descriptive Data Mining
– Seeks to describe new patterns in the data and requires human interaction to determine the significance and meaning of these patterns
– Affinity grouping• Which item goes together
– Clustering• Divides data into smaller groups based on similarity
without predefinition of the groups– Customers with similar buying habits
– Visualization• Graphical representation of data
Predictive Data Mining
• Likelihood of a particular outcome
• Mathematical algorithms are used to create models
• Classification
– A new record is assigned to a specific category defined by the model
– New credit applicants as low risk, medium risk, or high risk
• Estimation
– Assign a new record with a predicted value
– Length of time a customer will stay
Defining Data Mining
• The automated extraction of predictive information from (large) databases
• Two key words:– Automated– Predictive
• Data mining lets you be proactive• Prospective rather than Retrospective
How Data Mining Works: Modeling• Modeling is simply the act of building a model in one
situation where you know the answer and then applying it to another situation that you don't.
• Some models are better than others– Accuracy– Understandability
• Models range from “easy to understand” to incomprehensible
• Decision trees• Rule induction• Regression models• Neural Networks
Techniques in Data Ming
• Decision Trees
• Nearest Neighbor Classification
• Neural Networks
• Rule Induction
• K-means Clustering
Distinctions
Distinctions (Continued)