Intro to Data Mining: Extracting Information and Knowledge from Data
Preview:
Citation preview
- Slide 1
- Intro to Data Mining: Extracting Information and Knowledge from
Data
- Slide 2
- Topics Relationships between DSS/BI, database, data management
DSS/BI: transforming data into info to support decision making How
operational data and DSS/BI data differ What a data warehouse is,
how data for it are prepared, and how it is implemented
Multidimensional database Database technology for BI: OLAP, OLTP
Examples of applications in healthcare 2
- Slide 3
- BI: Extraction Of Knowledge From Data
- Slide 4
- DSS/BI Architecture: Learning and Predicting Courtesy: Tim
Graettinger
- Slide 5
- DSS/BI DSS/BI are technologies designed to extract information
from data and to use such information as a basis for decision
making Decision support system (DSS) Arrangement of computerized
tools used to assist managerial decision making within business
Usually requires extensive data massaging to produce information
Used at all levels within organization Often tailored to focus on
specific business areas Provides ad hoc query tools to retrieve
data and to display data in different formats 5
- Slide 6
- DSS/BI Components Data store component Basically a DSS database
Data extraction and data filtering component Used to extract and
validate data taken from operational database and external data
sources End-user query tool Used to create queries that access
database End-user presentation tool Used to organize and present
data 6
- Slide 7
- Main Components Of A DSS/BI
- Slide 8
- DSS/BI: Needs a different type of database A specialized DBMS
tailored to provide fast answers to complex queries. Database
schema Must support complex data representations Must contain
aggregated and summarized data Queries must be able to extract
multidimensional time slices Database size: DBMS must support very
large databases (VLDBs), Wal-Mart data warehouses is measured in
petabyte (1,000 terabyte) Technology: Data warehouse and OLAP
- Slide 9
- Operational vs. DSS/BI Data
- Slide 10
- Operational vs DSS Data
- Slide 11
- What is Data Warehouse? The Data Warehouse is an integrated,
subject- oriented, time-variant, non-volatile database that
provides support for decision making. Usually a read-only database
optimized for data analysis and query processing centralized,
consolidated database periodically updated, never removed Requires
time, money, and considerable managerial effort to create
- Slide 12
- OLAP (Online Analytical Processing) 12 Advanced data analysis
environment that supports decision making, business modeling, and
operations research engine or platform for DSS or Data Warehouse
OLAP systems share four main characteristics: Use multidimensional
data analysis techniques Provide advanced database support Provide
easy-to-use end-user interfaces Support client/server
architecture
- Slide 13
- OLAP vs OLTP Online Transactional Processing (OLTP) emphasize
speed, security, flexibility, reduce redundancy and abnormalities.
Online Analytical Processing (OLAP) multi-dimensional data analysis
advanced database support easy-to-use user interface support
client/server architecture
- Slide 14
- Multidimensional Data Analysis Goal: analyze data from
different dimensions and different levels of aggregation
- Slide 15
- Multidimensional Data Analysis Techniques Data are processed
and viewed as part of a multidimensional structure Particularly
attractive to business decision makers Augmented by following
functions: Advanced data presentation functions Advanced data
aggregation, consolidation and classification functions Advanced
computational functions Advanced data modeling functions 15
- Slide 16
- Multidimensional Data Analysis: Operational vs multidimensional
view
- Slide 17
- Integration OLAP with Spreadsheet
- Slide 18
- Easy-to-Use End-User Interface Many of interface features are
borrowed from previous generations of data analysis tools that are
already familiar to end users Makes OLAP easily accepted and
readily used
- Slide 19
- Client/Server Architecture Provides framework within which new
systems can be designed, developed, and implemented Enables OLAP
system to be divided into several components that define its
architecture OLAP is designed to meet ease-of-use as well as system
flexibility requirements
- Slide 20
- OLAP Architecture Designed to use both operational and data
warehouse data Defined as an advanced data analysis environment
that supports decision making, business modeling, and an operations
research activities In most implementations, data warehouse and
OLAP are interrelated and complementary environments
- Slide 21
- OLAP Architecture: OLAP engine provides ETL (DTS)
functions
- Slide 22
- Relational OLAP Provides OLAP functionality by using relational
databases and familiar relational query tools to store and analyze
multidimensional data Adds following extensions to traditional
RDBMS: Multidimensional data schema support within RDBMS Data
access language and query performance optimized for
multidimensional data
- Slide 23
- Relational OLAP (ROLAP)
- Slide 24
- Multidimensional OLAP (MOLAP) Extends OLAP functionality to
multidimensional database management systems (MDBMSs) MDBMS end
users visualize stored data as a 3D cube-a data cube Data cubes can
grow to n number of dimensions, becoming hypercubes To speed
access, data cubes are held in memory in a cube cache
- Slide 25
- Multidimensional OLAP
- Slide 26
- Relational vs. Multidimensional OLAP
- Slide 27
- Star Schemas Data modeling technique used to map
multidimensional decision support data into relational database
Creates near equivalent of multidimensional database schema from
existing relational database Yield an easily implemented model for
multidimensional data analysis, while still preserving relational
structures on which operational database is built Has four
components: facts, dimensions, attributes, and attribute
hierarchies
- Slide 28
- Facts Numeric measurements (values) that represent specific
business aspect or activity Normally stored in fact table that is
center of star schema Fact table contains facts that are linked
through their dimensions Metrics are facts computed or derived at
run time
- Slide 29
- Dimensions: simple star schema
- Slide 30
- Attributes Used to search, filter, or classify facts Dimensions
provide descriptive characteristics about the facts through their
attributes
- Slide 31
- Attributes: Three-dimensional view of sales
- Slide 32
- Attributes: slice-and-dice view of sales
- Slide 33
- Attribute Hierarchies Provides top-down data organization
Provides capability to perform drill-down and roll-up searches in a
data warehouse
- Slide 34
- Attribute Hierarchies in multidimensional analysis
- Slide 35
- Star Schema Representation Each dimension record is related to
thousands of fact records Facilitates data retrieval functions
- Slide 36
- Slice and Dice
- Slide 37
- Star Schema Representation: order star schema
- Slide 38
- Apply Database Design Procedures: DW design and
implementation
- Slide 39
- Data Warehouse Vendors
- Slide 40
- OLAP Market Size 40
- Slide 41
- OLAP Market Share 41
- Slide 42
- Market Consolidation 42
- Slide 43
- Latest Development Oracle-Hyperion Merger Cognos was bought by
IBM SPSS was bought by IBM 43
- Slide 44
- Application 1: Rehab Outcome Data Warehouse Rehabilitation
Outcome Database Center for Rehabilitation Service (CRS) UPMC More
than fifty community rehabilitation centers contributed to this
database. 547,719 transactions 13 Outcome indicators, 72,541
episodes of treatment, 17,205 patients, 108 therapists, 48
institutions
- Slide 45
- Multi-dimensional database Fact Table P_id D_id A_id T_id no of
patient Demographic D_id gender age N 1 Diagnosis P_id Disease
Status 1 N Area A_id Country State City 1 N Time T_id Year Month
Week N 1 fact dimension attribute
- Slide 46
- Star Schema
- Slide 47
- Slide 48
- Output Example: Hierarchy of a dimension: drill-down and
roll-up
- Slide 49
- Power of a visual presentation
- Slide 50
- Difference in Improvement: Young and Old patients
- Slide 51
- radar display
- Slide 52
- Application 2: Clinical Research Management 52
- Slide 53
- 53
- Slide 54
- 54
- Slide 55
- Application 3: Public Health Combining Data Warehouse (OLAP)
and GIS OLAP: handles large data, fast retrieval multidimensional,
multilevel aggregation, analyses/data mining on huge complex
databases GIS: visualization and spatial analyses Visualization and
Analysis: Charts and Maps + Statistical Analysis. 55
- Slide 56
- SOVAT (Spatial OLAP Viz and Analytical Tool)
- Slide 57
- Linkage of OLAP Cube and spatial data 57 Cube Geography
Dimension
- Slide 58
- Multidimensional database Multidimensional database Functions:
Drill-up/Drill-down, Slice/Dice, Pivot
- Slide 59
- Star Schema
- Slide 60
- Snowflake schema
- Slide 61
- Spatial Drill-Up Spatial Drill-Down Spatial Drill-Out
- Slide 62
- 62 Comparison and Border Analysis: Compare Allegheny Countys
cancer incidence rate against its bordering counties.
- Slide 63
- Slide 64
- Slide 65
- Slide 66
- Slide 67
- Ranking and sorting Massive data 67
- Slide 68
- Slide 69
- Slide 70
- 70
- Slide 71
- Comparing two arbitrarily defined communities: Compare the
incidence/death rate/procedure related to certain cancer or
specific diagnosis between the two metropolitans of Philadelphia
and Pittsburgh
- Slide 72
- Slide 73
- Slide 74
- Slide 75
- Time Series Example: Compare Cancer Incidence of Allegheny
County to Erie County from 1996-2000
- Slide 76
- Slide 77
- Statistical Analysis
- Slide 78
- Red nodes shows toxic industrial places in Allegheny
County
- Slide 79
- Buffer within 2.5 mile from CLEARWATER INC and the affected
municipalities Set the radius here List of affected municipalities
Buffer within 2.5 mile
- Slide 80
- Slide 81
- Authentication for accessing iSOVAT
- Slide 82
- Multidimensional view: cancer incidence in urban & rural
areas
- Slide 83
- Drill-down Washington county