Final Report for Sales Dataset Project

8/12/2019 Final Report for Sales Dataset Project

1/25

OLLEGE OF ARTS AND S IEN ES(SQIT 3033 )

KNOWLEDGE ACQUISITION IN DECISION MAKING

PROJECT REPORT : SALES

GROUP A

PREPARED FOR :

ENCIK IZWAN NIZAL BIN MOHD SHAHARANEE

PREPARED BY:

NURSAHLINA BINTI AHAR 212350KHAIRUN NADHIRAH BINTI JOHARI 212429

NURUL NAJWA BINTI GHAZALI 212418

NURUL SITI AISHAH BINTI ZAKARIA 212413


2/25

C!"#!"

1.0 INTRODUCTION

1.1 Data Background

2.0 PROBLEM STATEMENT

2.1 Tecn!"ue# To Be U#ed

$.0 RESEARC% MET%EDOLO&'

$.1 (no)*edge D!#co+er, !n Data-a#e (DD/

$.1.1 Se*ect!on

$.1.2 Preroce##!ng

.0 RESEARC% SOLUTION

.1 Data M!n!ng Tecn!"ue

.2 Ste In+o*+e !n SAS

.2.1 EMDATA

.2.2 Part!t!on

.2.$ Tran#3or4

.2. Regre##!on

.2.5 Dec!#!on Tree

.2.6 Neura* Net)ork

.2.7 A##e##4ent

5.0 SUMMAR'

RE8ERENCES


3/25

1$0 INTRODUCTION

1$1 D%"% B%&'()*!+

8or our ro9ect: )e u#ed Sa*e# data-a#e tat a*read, g!+en -, our *ecturer. Te data-a#e

!# a*read, 3!*tered and co4*ete data #et. Sa*e# data-a#e 3or to)ard# tree ,ear# o3 12;

co4an!e#. Our data#et doe# not a+e 4!##!ng +a*ue. Te data #et g!+en te target ro3!t

o3 eac co4an,. Sa*e# data #et o3 12; nu4-er# o3 co4an!e# and 15 nu4-er o3

attr!-ute#. A** our attr!-ute# 4ea#ure4ent #ca*e con#!#t# o3 no4!na*: !nter+a*: unar, and

-!nar,.


4/25

2$0 THE USE O, SPESI,IC TECHNI-UE

2$1 D#&./.! T)##

Dec!#!on tree !# a tree#aed #tructure tat rere#ent# #et o3 dec!#!on# or red!ct!on o3

data trend#. It !# #u!ta-*e to de#cr!-e #e"uence o3 !nterre*ated dec!#!on# or red!ct!on o3

3uture data trend# and a# te caa-!*!t, to c*a##!3, ent!t!e# !nto #ec!3!c c*a##e# -a#ed on

3eature o3 ent!t!e#. Eac tree con#!#t# o3 tree t,e# o3 node#= root node: !nterna* node

and ter4!na* node>*ea3. Te to 4o#t node !# te root node and !t rere#ent# a** o3 te

ro)# !n te data#et. Node# )!t c!*d node# are te !nterna* node# )!*e node# )!tout

c!*d node are ca**ed te ter4!na* node or *ea3. A co44on a*gor!t4 3or -u!*d!ng a

dec!#!on tree #e*ect# a #u-#et o3 !n#tance# 3ro4 te tra!n!ng data to con#truct an !n!t!a*

tree. Te re4a!n!ng tra!n!ng !n#tance# are ten u#ed to te#t te accurac, o3 te tree. I3

an, !n#tance !# !ncorrect*, c*a##!3!ed te !n#tance !# added to te current #et o3 tra!n!ngdata and te roce## !# reeated. A 4a!n goa* !# to 4!n!4!?e te nu4-er o3 tree *e+e*#

and tree node#: tere-, 4a@!4!?!ng data genera*!?at!on. Dec!#!on tree# a+e -een

#ucce##3u**, a*!ed to rea* ro-*e4: are ea#, to under#tand: and 4a n!ce*, to a #et o3

roduct!on ru*e#.


5/25

2$2 R#()#//.!

Te Regre##!on node !n Enterr!#e M!ner doe# e!ter *!near or *og!#t!c regre##!on

deend!ng uon te 4ea#ure4ent *e+e* o3 te target +ar!a-*e. L!near regre##!on !# done !3

te target +ar!a-*e !# an !nter+a* +ar!a-*e. In *!near regre##!on te 4ode* red!ct# te

4ean o3 te target +ar!a-*e at te g!+en +a*ue# o3 te !nut +ar!a-*e#. Log!#t!c regre##!on

!# done !3 te target +ar!a-*e !# a d!#crete +ar!a-*e. In *og!#t!c regre##!on te 4ode*

red!ct# te ro-a-!*!t, o3 a art!cu*ar *e+e*#/ o3 te target +ar!a-*e at te g!+en +a*ue#

o3 te !nut +ar!a-*e#. Becau#e te red!ct!on# are ro-a-!*!t!e#: )!c are -ounded -, 0

and 1 and are not *!near !n t!# #ace: te ro-a-!*!t!e# 4u#t -e tran#3or4ed !n order to -e

ade"uate*, 4ode**ed. Te 4o#t co44on tran#3or4at!on 3or a -!nar, target !# te *og!c

tran#3or4at!on. Pro-!t, and co4*e4entar, *og*og tran#3or4at!on# are a*#o a+a!*a-*e !n

te regre##!on node. Tere are tree +ar!a-*e #e*ect!on 4etod# a+a!*a-*e !n te

Regre##!on node o3 Enterr!#e M!ner. 8or)ard 3!r#t #e*ect# te -e#t one+ar!a-*e 4ode*.

Ten !t #e*ect# te -e#t t)o +ar!a-*e# a4ong to#e tat conta!n te 3!r#t #e*ected +ar!a-*e.

T!# roce## cont!nue# unt!* !t reace# te o!nt )ere no add!t!ona* +ar!a-*e# a+e a

+a*ue *e## tan te #ec!3!ed entr, +a*ue. Back)ard #tart# )!t te 3u** 4ode*. Ne@t:

te +ar!a-*e tat !# *ea#t #!gn!3!cant: g!+en te oter +ar!a-*e#: !# re4o+ed 3ro4 te

4ode*. T!# roce## cont!nue# unt!* a** o3 te re4a!n!ng +ar!a-*e# a+e a +a*ue *e##

tan te #ec!3!ed #ta, +a*ue. Ste)!#e !# a 4od!3!cat!on o3 te 3or)ard #e*ect!on

4etod. Te d!33erence !# tat +ar!a-*e# a*read, !n te 4ode* do not nece##ar!*, #ta,

tere. A3ter eac +ar!a-*e !# entered !nto te 4ode*: t!# 4etod *ook# at a** te +ar!a-*e#

a*read, !nc*uded !n te 4ode* and de*ete# an, +ar!a-*e tat !# not #!gn!3!cant at te

#ec!3!ed *e+e*. Te roce## end# )en none o3 te +ar!a-*e# out#!de te 4ode* a# a

+a*ue *e## tan te #ec!3!ed entr, +a*ue and e+er, +ar!a-*e !n te 4ode* !# #!gn!3!cant at

te #ec!3!ed #ta, +a*ue.

2$3 N#*)% N#")'

An art!3!c!a* neura* net)ork !# a net)ork o3 4an, #!4*e roce##or# un!t#/: eac

o##!-*, a+!ng a #4a** a4ount o3 *oca* 4e4or,. Te un!t# are connected -,

co44un!cat!on canne*# connect!on#/ tat u#ua**, carr, nu4er!c a# oo#ed to


6/25

#,4-o*!c/ data encoded -, +ar!ou# 4ean#. Te un!t# oerate on*, on te!r *oca* data and

on te !nut# te, rece!+e +!a te connect!on#. Te re#tr!ct!on to *oca* oerat!on# !# o3ten

re*a@ed dur!ng tra!n!ng. More #ec!3!ca**,: neura* net)ork# are a c*a## o3 3*e@!-*e:

non*!near regre##!on 4ode*#: d!#cr!4!nant 4ode*#: and data reduct!on 4ode*# tat are

!nterconnected !n a non*!near d,na4!c #,#te4. Neura* net)ork# are u#e3u* too*# 3or

!nterrogat!ng !ncrea#!ng +o*u4e# o3 data and 3or *earn!ng 3ro4 e@a4*e# to 3!nd attern#

!n data. B, detect!ng co4*e@ non*!near re*at!on#!# !n data: neura* net)ork# can e*

4ake accurate red!ct!on# a-out rea*)or*d ro-*e4#

3$0 RESEARCH METHODOLOG

Te 4etod tat u#ed !n re#earc 4etodo*og, !# kno)n kno)*edge d!#co+er, (DD/.

(DD !# nontr!+!a* e@tract!on o3 !4*!c!t: re+!ou#*, unkno)n and otent!a* u#e3u*

!n3or4at!on 3ro4 data. T!# 4etod u#ed to d!ge#t>3!nd !dden !n3or4at!on !n data-a#e

and con+ert unkno)n > !dden attern !nto u#e3u*: under#tanda-*e and !n3or4at!+e )a,.

(DD roce## conta!n# 5 #te# tat are #e*ect!on: reroce##!ng: tran#3or4at!on: data

4!n!ng: !nterretat!on and e+a*uat!on.

3$1$ K!#+(# D./) .! D%"%%/# 6KDD7

$.1.1 Se*ect!on

&enera**,: #e*ect!on roce## !# te data need# 3or te data 4!n!ng roce## 4a, -e

o-ta!ned 3ro4 4an, d!33erent and eterogeneou# data #ource#. Data #e*ect!on roce## !#

to ac"u!re te 4o#t aror!ate #!?e tat u#e3u* to te (DD roce##. U#!ng #a4*!ng

4etod !3 data too -!g. Sa4*!ng 4etod a# t)o t,e# tat are ro-a-!*!t, #a4*!ng

rando4*, co#en/ and nonro-a-!*!t, #a4*!ng e@ert 9udg4ent/.

$.1.2 Preroce##!ng


7/25

Preroce##!ng roce## !# to en#ure tat data !# c*ean !g "ua*!t, o3 data/. Te data to

-e u#ed -, te roce## 4a, a+e !ncorrect or 4!##!ng data. Tere 4a, -e ano4a*ou#

data 3ro4 4u*t!*e #ource# !n+o*+!ng d!33erent data t,e# and 4etr!c#. Erroneou# data

4a, -e corrected or re4o+ed: )erea# 4!##!ng data 4u#t -e #u*!ed or red!cted. Data

roce##!ng can -e categor!?ed !nto 3e) 4etod#=

Data c*ean!ng !# a roce## o3 detect!ng and correct!ng or re4o+!ng/ corrut or

!naccurate record# 3ro4 a record #et: ta-*e: or data-a#e. Te uro#e o3 u#!ng data

c*ean!ng !# to !dent!3,!ng !nco4*ete: !ncorrect: !naccurate: !rre*e+ant: etc. art# o3 te

data and ten re*ac!ng: 4od!3,!ng: or de*et!ng t!# d!rt, data. Data c*ean!ng can and*e

!nco4*ete: no!#, and !ncon#!#tent data.

Inco4*ete data !# te 4!##!ng +a*ue aen# due to !4roer data co**ect!on

4etod#. Dur!ng co**ect!on o3 data: tere 4a, aen no recorded +a*ue 3or

certa!n attr!-ute and cau#!ng !nco4*ete o3 data. To o+erco4e t!# ro-*e4: )e

can u#e 4ean+a*ue: e#t!4ate te ro-a-*e +a*ue u#!ng regre##!on: u#!ng con#tant

+a*ue or !gnore te 4!##!ng record.

No!#, data !# rando4 error or +ar!ance !n data. T!# aen# due to corruted

data tran#4!##!on: tecno*og!ca* *!4!tat!on. Dur!ng tran#4!##!on data !nto certa!n

#o3t)are #uc a# SPSS or SAS: )e 4a, ke, !n )rong data !n !t: tu#: t!# )!**

cau#e no!#, data aened. To #o*+e t!# ro-*e4: )e can u#e -!nn!ng 4etod or

out*!er re4o+a* 4etod.

Incon#!#tent data 4ean# te data conta!n# re*!cat!on or o##!-*, redundanc,

data. Metod to o+erco4e t!# ro-*e4 !# re4o+!ng redundant or re*!cate data.


8/25

Data !ntegrat!on !n+o*+e# co4-!n!ng data re#!d!ng !n d!33erent #ource# and ro+!d!ng

u#er# )!t a un!3!ed +!e) o3 te#e data. Data co4e# 3ro4 d!33erent #ource# )!t d!33erent

na4!ng #tandard. T!# )!** cau#e !n !ncon#!#tenc!e# and redundanc!e#. Tere are #e+era*

)a,# to and*e t!# ro-*e4 )!c !# Con#o*!date d!33erent #ource !nto one reo#!tor,

u#!ng 4etadata/: Corre*at!on ana*,#!# 4ea#ure te #trengt o3 re*at!on#! -et)een

d!33erent attr!-ute/.

Data reduct!on !# te tran#3or4at!on o3 nu4er!ca* or a*a-et!ca* d!g!ta* !n3or4at!on

der!+ed e4!r!ca* or e@er!4enta**, !nto a corrected: ordered: and #!4*!3!ed 3or4. Te

-a#!c concet !# te reduct!on o3 4u*t!tud!nou# a4ount# o3 data do)n to te 4ean!ng3u*

art#. T!# !# to !ncrea#e e33!c!enc,: can reduce te uge data #et !nto a #4a**er

rere#entat!on. Se+era* tecn!"ue# can -e u#ed !n data reduct!on #uc a# data cu-e

aggregat!on: d!4en#!on reduct!on: data co4re##!on and d!#cret!?at!on.

$.2.$ Tran#3or4at!on

In te tran#3or4at!on roce##: )!c a*#o kno)n a# data nor4a*!?at!on: !# -a#!ca**, re

#ca*e te data !nto a #u!ta-*e range. T!# roce## !# !4ortant -ecau#e !t can !ncrea#e te

roce##!ng #eed and reduce te 4e4or, a**ocat!on. Tere are #e+era* 4etod# !n

tran#3or4at!on=

Dec!4a* Sca*!ng

M!nMa@

Score

Logar!t4!c Nor4a*!?at!on.


9/25

In out #a*e data. Tere are no 4!##!ng data !# occur# !n t!# ro9ect.

8!gure 1= #a*e# +ar!a-*e

8ro4 te ta-*e a-o+e: our target !# red!ct!ng te c*a##.

$.2. Interretat!on e+a*uat!on roce##

In !nterretat!on and e+a*uat!on roce##: certa!n data 4!n!ng outut !# nonu4an

under#tanda-*e 3or4at and )e need !nterretat!on 3or -etter under#tand!ng. So: )e

con+ert outut !nto an ea#, under#tand 4ed!u4.


10/25

4$0 RESEARCH SOLUTION

4$1$ D%"% M.!.!( T#&!.*# " S# "# P)#

Data M!n!ng !# a-out e@*a!n!ng te a#t and red!ct!ng te 3uture -, 4ean# o3 data ana*,#!#.

Data 4!n!ng !# a 4u*t!d!#c!*!nar, 3!e*d )!c co4-!ne# #tat!#t!c#: 4ac!ne *earn!ng: art!3!c!a*

!nte**!gence and data-a#e tecno*og,. Te +a*ue o3 data 4!n!ng a*!cat!on# !# o3ten e#t!4ated to

-e +er, !g. Man, -u#!ne##e# a+e #tored *arge a4ount# o3 data o+er ,ear# o3 oerat!on: and

data 4!n!ng !# a-*e to e@tract +er, +a*ua-*e kno)*edge 3ro4 t!# data. Te -u#!ne##e# are ten

a-*e to *e+erage te e@tracted kno)*edge !nto 4ore c*!ent#: 4ore #a*e#: and greater ro3!t#. T!#

!# a*#o true !n te eng!neer!ng and 4ed!ca* 3!e*d#. Our ro9ect !# a-out to create our o)n #cenar!o

-a#ed on te data and c*a##!3, te -!g co4an, *!#ted )eter te co4an, ac!e+e te target o3#a*e# or not. In t!# ro9ect: )e u#ed regre##!on: neura* net)ork# and dec!#!on tree u#!ng SAS

Enterr!#e M!ner to #o*+e our data #et#. SAS Enterr!#e M!ner #trea4*!ne# te data 4!n!ng

roce## to create !g*, accurate red!ct!+e and de#cr!t!+e 4ode*# -a#ed on *arge +o*u4e# o3

data 3ro4 acro## te enterr!#e. It o33er# a r!c: ea#,tou#e #et o3 !ntegrated caa-!*!t!e# 3or

creat!ng and #ar!ng !n#!gt# tat can -e u#ed to dr!+e -etter dec!#!on#.


11/25

S"#;/ I!.!( .! SAS E!"#);)./# M.!#)

To -eg!n: #e*ect So*ut!on Ana*,#!# Enterr!#e M!ner. Ten: te Start# Enterr!#e M!ner

)!ndo) )!** oen. A3ter tat: c*!ck 8!*e Ne) Pro9ect to create a ne) ro9ect. Ten )e na4e

our ro9ect a# SALES

DATA=


12/25

CLASS FARIABLE=

Ten: )e c*!ck G+ar!a-*e#H and #et te 4ode* ro*e.


13/25

Te 3unct!on o3 t!# node !# to art!t!on te !nut data #et# o3 #a*e# !nto a tra!n!ng: +a*!dat!on and

te#t 4ode*. Te tra!n!ng data #et !# u#ed 3or re*!4!nar, 4ode* 3!tt!ng. Te +a*!dat!on data #et !#

u#ed to 4on!tor and tune te 3ree 4ode* ara4eter# dur!ng e#t!4at!on and !# a*#o u#ed 3or 4ode*

a##e##4ent. Te te#t data #et !# an add!t!ona* o*dout data #et tat )e can u#e 3or 4ode*

a##e##4ent. R!gt c*!ck on t!# data art!t!on node ten c*!ck oen and )e #et te ercentage o3

eac art!t!on and )e dec!de to #et 70J 3or tra!n!ng: 0J 3or +a*!dat!on and $0J 3or te#t.


14/25

+ar!ance: re4o+e non*!near!t,: !4ro+e add!t!+!t,: and counter nonnor4a*!t,. Tere3ore: 3or

4an, 4ode*#: tran#3or4at!on# o3 te !nut data e!ter deendent or !ndeendent +ar!a-*e#/ can

*ead to a -etter 4ode* 3!t. Te#e tran#3or4at!on# can -e 3unct!on# o3 e!ter a #!ng*e +ar!a-*e or o3

4ore tan one +ar!a-*e. In our ro9ect: )e u#e te Tran#3or4. Far!a-*e# node to 4ake +ar!a-*e#

-etter #u!ted 3or *og!#t!c regre##!on 4ode*# and neura* net)ork#.

8ro4 t!# o-#er+at!on: )e are tran#3or4ed a** te #ke)ne## o3 te +ar!a-*e#.

LogSALES;5/:*ogPRO8IT/:*ogSALES;/ and *ogSALES;$/ !# gett!ng te!r ne) #ke)ne##

and a** o3 te4 are !n te 4a@!4u4 nor4a*!t,.

Te !#togra4 o3 te 4a@!4u4 nor4a*!t, 3or eac +ar!a-*e#.

Te tran#3or4at!on o3 +ar!a-*e 3or #a*e# data ,ear ;$.


15/25

Te tran#3or4at!on o3 +ar!a-*e 3or #a*e# data ,ear ;

Te tran#3or4at!on o3 +ar!a-*e 3or #a*e# data ,ear ;5.


16/25

Ten )e dou-*e c*!ck at regre##!on node and #et our regre##!on ot!on.

Ten )e roceed to 4ode* ot!on.

In ot!on 4ode* )e coo#e *og!#t!c -ecau#e our target +ar!a-*e !# #et to -!nar,.

Se*ect!on 4etod.


17/25

Ten !n #e*ect!on 4etod: #e*ect!on o3 #te)!#e 4etod !# co#en: -, de3au*t: )!t no cand!datee33ect !n te 4ode* and te #,#te4at!ca**, add# e33ect# tat are #!gn!3!cant a##oc!ated )!t te

target. Ste)!#e can re4o+e an, e33ect tat a*read, !n te 4ode*.

Ten )e run te regre##!on node and ana*,#!ng te re#u*t.Regre##!on re#u*t.


18/25

8ro4 t!# re#u*t )e 3ocu# on te 4!#c*a##!3!cat!on rate 3or tra!n!ng and te#t!ng data #et. A# )e

can #ee tat te 4!#c*a##!3!cat!on rate 3or tra!n!ng !# 0.0667 and te#t!ng data #et !# 0.026$.

Te ne@t node )e u#e !n SAS Enterr!#e M!ner 3or our ro9ect !# Dec!#!on Tree Node. Te u#e

o3 dec!#!on tree node !# to 3!t te dec!#!on tree 4ode* to te data.


19/25

Te re#u*t 3ro4 dec!#!on tree node.

%ere )e can #ee tere are t)o nu4-er# o3 -rance# 3ro4 a root node.


20/25

Ten )e )ant to #ee *og )ort o3 te +ar!a-*e.


21/25

Re#u*t o3 te Neura* Net)ork Node.


22/25

M!#c*a##!3!cat!on rate 3or tra!n!ng data #et !# 0 and te#t!ng data #et !# 0.07K;.

A##e##4ent Node


23/25

Te *a#t node )e u#e !n SAS Enterr!#e M!ner !# A##e##4ent node. A##e##4ent node# connect

)!t tree node# )!c are Dec!#!on Tree node: Neura* Net)ork Node and Regre##!on node.

D!rect*, run te a##e##4ent node and get te re#u*t.

8ro4 te re#u*t: )e can #ee tat Dec!#!on Tree !# te -e#t 4ode* 3or no). T!# !# -ecau#e te+a*ue o3 4!#c*a##!3!cat!on rate 3or te#t!ng data #et !# 0.


24/25

5$0 SUMMAR AND DISCUSSION:

Mode* Tra!n!ng= M!#c*a##!3!cat!on

Rate

Te#t!ng= M!#c*a##!3!cat!on

Rate

Dec!#!on Tree 0.0111 0Neura* Net)ork 0 0.07K;

Regre##!on 0.0667 0.026$

Ba#ed on te a-o+e re#u*t: )e can conc*ude tat te tree !# te -e#t 4ode* tan 2 oter# )!t te

te#t 4!#c*a##!3!cat!on rate 0J. Te )or#t 4ode* 3or #a*e# data#et !# neura* net)ork )!t 7.K;J

te#t 4!#c*a##!3!cat!on rate.

A3ter tat: )e !g*!gt at te Dec!#!on Tree too* and c*!ck at te GDra) L!3t CartH.

8ro4 te *!3t cart: te cu4u*at!+e J re#on#e !# 100J troug te 7$t ercent!*e. At te 70t

ercent!*e: te ne@t o-#er+at!on )!t te !ge#t red!cted ro-a-!*!t, !# a not ac!+e: #o te

cu4u*at!+e re#on#e dro# to 100J. Tu#: TREE !# te -e#t 4ode*..


25/25

1. tt=>>#uort.#a#.co4>docu4entat!on>cd*>en>#t#ug>6225;>%TML>de3au*t>+!e)er.t4ug

a data#et##ectK.t4

2. Lecture Note SIT $0$$

Documents

Final Report for Sales Dataset Project