8635_Demo Script Predictive Analysis With HANA Including PAL and R

7/21/2019 8635_Demo Script Predictive Analysis With HANA Including PAL and R

1/51

DOCUMENT CLASSIFICATION: INTERNAL

General Information: Retail Cross Industry

SAP HANA

Authors: Philip Mugglestone

Date Last Updated: January, 2013

DEMONSTRATION

SCENARIO

PREDICTIVE ANALYTICS

WITH SAP HANA

CUSTOMER SEGMENTATION


2/51

2

TABLE OF CONTENTS

TABLE OF CONTENTS ................................................................................................................... 2

1. Demo Script Overview ......................................................................................................... 3

1.1. Demo Description ...................................................................................................................................... 3

1.2.

Protagonists .............................................................................................................................................. 3

1.3.

Business Pain Points................................................................................................................................. 3

1.4.

Key Messages and Value Proposition ...................................................................................................... 4

1.5. Storyflow Summary ................................................................................................................................... 4

2. Technical Requirements ...................................................................................................... 5

2.1.

Prerequisites ............................................................................................................................................. 5

2.2.

System Access Information ....................................................................................................................... 5

2.2.1. System Access ............................................................................................................................. 5

2.2.1.1. SAP Demo Cloud Showroom .................................................................................................... 5

2.2.1.2. Users ......................................................................................................................................... 5

3. Demo Script .......................................................................................................................... 6

3.1. Step-By Step Guide................................................................................................................................... 6

Step 1: Introduction................................................................................................................................ 6

Step 2: Customer Segmentation with SAP HANA PAL ....................................................................... 8

Step 3: Customer Segmentation with R Integration for SAP HANA................................................ 19

Step 4: Recommending the best number of clusters........................................................................ 25

Conclusion ............................................................................................................................................. 30

Step 1: Introduction.............................................................................................................................. 31

Step 2: Single Exponential Smoothing............................................................................................... 33

Step 3: Double Exponential Smoothing.............................................................................................. 39

Step 4: Triple Exponential Smoothing................................................................................................ 45

Step 5: Developer Overview ................................................................................................................. 47

Conclusion 50


3/51

3

1.DEMO SCRIPT OVERVIEW

1.1. DEMO DESCRIPTION

The purpose of this demo is to showcase the capabilities of SAP HANA to support predictive analytics in big data

scenarios.

In the first part of this demo, a business user or data analyst (perhaps supported by a data scientist) can perform

customer segmentation in real-time using SAP HANA. The emphasis is on speed and flexibility just seconds are

needed to segment a large dataset and without resorting to sampling the source data. So models are (arguably) more

accurate as all source data is leveraged and the business user wins time back which they can use to perform further

analyses and simulations. In the second part of the demo, a business user or data analyst can perform time series

analysis in real-time using SAP HANA. The emphasis is on speed and flexibility just seconds are needed to

aggregate over a million sales transactions covering 50 electronic items from the last two years and perform focused

time series analysis. Analysts are able to compare actual sales to predicted (i.e fitted) for all products or focused on a

specific product category.

This demo showcases SAP HANA as a platform and is particularly targeted to the SAP HANA developer audience to

demonstrate how Predictive Analytics with SAP HANA can easily be embedded into customer solutions.

This demonstration is based on retail customer data.

However, it can easily be understood by anyone.

The datasets used are hosted by SAP HANA

Predictive analysis is performed by SAP HANA Predictive Analysis Library (PAL) and R Integration for SAP HANA.

Exponential smoothing can be applied to time series data either to produce smoothed data for presentation or in orderto make forecasts.

Single exponential smoothing is the simplest form and easily applied; double exponential smoothing is better suited to

data that trends, whereas triple exponential smoothing also incorporates seasonal changes.

Fit can be fine-tuned via alpha, beta, and gamma factors (,,). Values closer to 0 tend to produce smoother results;

values nearer to 1 give greater weight to recent changes.

NB: A similar scenario is available that combines SAP Predictive Analysis with SAP HANA. SAP Predictive Analysis

provides richer visualizations along with modeling and workflow. The two approaches are complementary. The primary

objective of this demo is to showcase how the predictive capabilities of SAP HANA can be embedded into custom

solutions by developers using SAP HANA as a platform.


4/51

4

This demo allows customers and partners to:

Make smarter decisions faster with real-time insights

Accelerate key business processes with rapid analysis and reporting

Invent new business models and processes by leveraging innovative solutions

Reduce TCO with less hardware, maintenance, and testing

There are many use cases for cluster analysis

Customer segmentation

Data reduction / problem refinement when faced with large, complex data sets

Segmenting the market and determining target markets

Product positioning

Selecting test markets

Crime pattern analysis

Medical research, social services, psychiatry, education, archaeology, astronomy, taxonomy

Or the opposite of segmentation anomaly detection

1.4. KEY MESSAGES AND VALUE PROPOSITION

- SAP HANA as a platform to develop custom solutions leveraging all kinds of big data

- Accelerate and improve key business processes through rapid and integrated predictive analysis

- Invent new business models and processes by leveraging innovative predictive analysis solutions

- Exploit SAP HANA to reduce TCO with less hardware, maintenance, and testing

1.5. STORYFLOW SUMMARY

The overall story is:Customer Segmentation

Cluster analysis being used to group or cluster various retail customers given data on recent and total spend,income and age, in order that promotional sales strategies may be applied to groups of similar customers.

Once we have grouped or segmented the customers, we would like to know why customers were allocated toeach segment and when new customers are acquired what group sales strategy they should be assigned to.

We start with an initial data analysis to see if there are any obvious segments / groupings.

Then we run the k-means algorithm to perform the customer segmentation i.e. identify clusters.

When new customers are acquired we score (or predict) which group strategy should be applied to them.

Time Analysis Series

Business analyst wishes to better understand existing sales patterns and predict future sales & revenues, anddo this in an interactive ad-hoc manner for any product category.

Different products exhibit different sales patterns over time. Cameras, for example, are trending


5/51

5

2.TECHNICAL REQUIREMENTS

2.1. PREREQUISITES

This demo has been optimized for iPad. For desktops, use of Internet Explorer 9 or Firefox is strongly recommended.

There are no additional pre-requisites.

2.2. SYSTEM ACCESS INFORMATION

2.2.1. S!"#$ A%%#!!

2.2.1.1. SAP D#$& C'&() S*&+,&&$

Showroom =HANA Showroom Edition 2013-1

Click here for information for accessing all SAP Demo Cloud Showrooms

2.2.1.2. U!#,!

Predefined User (Role) Password Component

visual welcome Predictive Analytics


6/51

6

Copyright 2012 SAP AG CUSTOMER

3.DEMO SCRIPT

3.1. STEP-BY STEP GUIDE

3.1.1. D#$& /," 1: C(!"&$#, S#0$#"/"&

What to do/say What you should see

Step 1: Introduction

Using Internet Explorer enter:

http://hana1:8020/palr/default.html

logon with VISUAL/welcome

An overview of the scenario is providedon the right hand side of the homescreen.

We have a number of options available well see these in more detail later.

First well get an understanding for thedataset were working with which fornow is a sample of the full dataset withjust 150 retail customers.


7/51

7



Click on DATA to view the customerdataset

We have data for 150 customers whichincludes information on their:

Lifetime spend (on our productsand services)

New spend (recent spend onour products and services)

Customers annual income

Customer loyalty index (basedon years theyve been acustomer)

Optionally scroll down to show morecustomers.

Also note that the cluster column iscurrently empty.

We are going to segment thesecustomers into a small number of similargroups (clusters) based on thecombination of the 4 key pieces ofinformation outlined above.


8/51

8



Step 2: Customer Segmentation with SAP HANA PAL

Click on HOME to return to the initialscreen

Review the options available:

Source Data: we have just viewed the150 customers dataset (well get to the 1

Million+ shortly)Predictive Library: SAP HANA providestwo options for performing predictiveanalytics and data mining.

PAL is native to SAP HANA andprovides a selection of popularalgorithms for in-memory in-database processing for

optimum performance.

R Integration for SAP HANAprovides the ability to work withover 3500 algorithms availablethrough R

Clusters (K): This is where we selecthow many clusters (or groups orsegments we wish to create)

Recommend K: Well get to this later

NB: Display Data is simply to controlhow many observations (rows) from thesource / result datasets should beshown in the DATA tab. It has no effecton the segmentation process which isperformed on the SAP HANA server.

Press the Perform K-Means Clusteringbutton


9/51

9



The system will take a second toperform the K-Means processing. As

this is a very small dataset it will be veryfast as youd expect.

The results are displayed in theCLUSTERS tab


10/51

10



As a first initial analysis of the clusteringyou can see how many customers havebeen assigned to eachcluster/group/segment.

This provides a quick overview of howbalanced the segmentation is.

It's often a good idea to run an algorithmon a small subset of the data to get anidea how the model might work beforeapplying to the larger data set.

Optionally move the mouse over thebars to see more details.

We now want to view the cluster thathas been assigned to each individualcustomer


11/51

11



Click on DATA to return to thedataset tab

Now we can see that the CLUSTERcolumn has been populated for eachcustomer.

So what? you may be thinking

well weve now seen the process forcustomer segmentation on a tinydataset.

But how will this perform on big data for example similar analysis to segmentmore than 1 million customers?


12/51

12




Change the Source Data value to 1Million+ Customers.



13/51

13



Whilst this runs explain that the K-Means algorithm is a data intensive

process not at all like a simple BIquery.

In fact, K-Means involves multiplepasses of the entiredataset.

It is not generally possible to do really

large data volume data analysis withtraditional systems and the typicalworkaround is to sample (effectivelysubset) the dataset.

However through in-memory computingwith SAP HANA, where the data isstored and processedin SAP HANA

this now becomes possible.

The system needs just a few seconds toperform the K-Means processing!!!

WOW!!!

As previously, the results are displayedin the CLUSTERS tab


14/51

14



Again you can see how many customershave been assigned to each cluster.

Optionally, move the mouse over a barto see more details.

NB: Notice the customer frequencyvalues have increased significantly fromthe previous example now in hundred

on thousands of customers. We reallyare working with over 1 millioncustomers!


15/51

15



Click on SILHOUETTE to show thesilhouette value

The screen shows the averagesilhouette (or level of confidence in thefit) when 3 clusters are created againstthis dataset.

More explanation of silhouette isprovided in the text on the screen.Basically, the close the value is to 1 themore appropriate the number of clusters(K) will be.

We will come back to this later


16/51

16



Click on ARCHITECTURE to showthe architecture diagram

So we have seen how complex datamining processing against largevolumes of detailed data becomespossible thanks to SAP HANA.

And its sofastthat you can make thisan iterative and interactive process forexample try a number of different valuesfor K (number of clusters to create) tosee what provides the best fit

At this point, especially to a developeraudience, you can speak to the overallarchitecture of predictive analytics withSAP HANA and how it forms the centerof a rich ecosystem.

Business focused applications such asSAP Predictive Analysis, BI tools, andcustomer applications can all exploit thepower of SAP HANA for predictiveanalysis.

Data can come from any source (notlimited to SAS transactional data).


17/51

17



To a developer audience, nows a goodtime to explain a little about how easy itis to take advantage of the predictiveanalytics capabilities of SAP HANA

Click on CODE to show the SQLScript

Predictive Analytics processing in SAPHANA is accessed with SAP HANA SQLScript syntax (looks similar to storedprocedures in regular RDBMS).

The procedure is created once duringdevelopment.

In order to execute a K-Means wesimply need to CALL it using the singleline below.

Highlight the pal: lines to show whichPAL algorithms were invoking.


18/51

18



Click on ARGS to show thearguments that were passed to SAPHANA for processing

A number of parameters are available tocontrol how processing is performed.

These are stored in a HANA table andcan be modified immediately prior toprocessing

In this case the application set theGROUP_NUMBER item to 3 in order tocreate 3 clusters.

The max number of iterations the K-Means algorithm performs can also becontrolled via the MAX_ITERATIONoption etc.

Its really as simple as that with SAPHANA!


19/51

19



Step 3: Customer Segmentation with R Integration for SAP HANA

As well as the Predictive AnalysisLibrary of SAP HANA (PAL) we can alsoleverage R processing from a SAPHANA environment.

More the 3500 algorithms are availablewith R but for this demonstration well

stick with K-Means for simplicity.


Change the Source Data value to 150Customers.

Change the Predictive Library to RIntegration for SAP HANA.

Change the Clusters (K) value to 10.



20/51

20



It runs just like previously however thistime R has been used for the heavylifting to determine the clusters.


21/51

21



Click on SILHOUETTE to show thesilhouette screen

The screen shows the averagesilhouette (or level of confidence in thefit) when 5 clusters are created againstthis dataset.

Notice that for 5 clusters the silhouettevalue is lower than for 3.

Identifying the best number of clusters(K value) can be a major challengemore to come on this later!


22/51

22



Click on ARCHITECTURE to show

the architecture diagram

Optionally show the ARCHITECTUREscreen again highlight how RIntegration for SAP HANA fits into theecosystem.


23/51

23



To a developer audience, nows a good

time to explain a little about how easy itis to take advantage of the predictiveanalytics capabilities of SAP HANA




In order to execute a K-Means wesimply need to CALL it using the singleline below.

Highlight the 4 lines between BEGINand END to show the custom R scriptthat was embedded.


24/51

24



Click on ARGS to show thearguments that were passed to SAPHANA for processing

A number of parameters are available tocontrol how processing is performed.These are stored in a HANA table and

can be modified immediately prior toprocessing

In this case we updated item 1 to 5 inorder to create 5 clusters.

Any R capability can be accessed in this

way from SAP HANA.



25/51

25



Step 4: Recommending the best number of clusters

We have already discovered thatidentifying the best number of clusters(K value) can be a major challenge

Wouldnt it be great if the applicationcould find this out for us?

Well the power of in-memory processingin SAP HANA makes it possible!

Lets have SAP HANA performclustering for a range of K values (from2 to 10) in order to determine the bestaverage silhouette value.

Click on HOME to show the homescreen

Change the Predictive Library to SAPHANA Predictive Analysis Library(PAL).

Check the Recommend K box.



26/51

26



SAP HANA now performs the k-means

analysis 9 times.

Processing may take a few seconds

and leverages parallel processing todo this extremely quickly!

27


27/51

27



The SILHOUETTE screen is

displayed

This time the table contains the averagesilhouette value for all 9 K values.

Lets sort the table by silhouette value tosee the largest first.

Click on the SILHOUETTE column

28


28/51

28



Select the Sort Descending option.

29


29/51

29



Now we can see that the first K

displayed is 2 which with the highestaverage silhouette value of 0.6893becomes the recommended K value forthis dataset (based on the 4 customerattributes we used in the K-Meansanalysis).

30


30/51

30



Conclusion


Lets summarize what we did:

Performed K-Means on 150

customers via SAP HANAPredictive Analysis Library(PAL)

The same for 1 Million+customers took mere seconds!!!

Showed how easy it is fordevelopers to leverage thisfunctionality

Demonstrated R Integration forSAP HANA

Showed how SAP HANAenables new scenarios such asrecommending the best K value

NB: for business analysts

consider SAP PredictiveAnalysis which also works withSAP HANA and providesmodeling and visualizationcapabilities (i.e. no coding)

End of the demo.

31


31/51

31


3.1.2 D#$& /," 2: T$# S#,#! A/'!!


Step 1: Introduction

Before you start, make sure you havethis screen

An overview of the scenario is provided

on the right hand side of the homescreen.

We have a number of options available well see these in more detail later.

First well get an understanding for thedataset were working with whichcontains over a million transactions for a

leading electronic goods store.

32


32/51



Click on DATA to view the dataset

We have data for:

Over 1 million transactions

200,000 customers

Around 50 electronic itemsgrouped into 5 productcategories

2 years of history

Optionally scroll down to show moresales transactions.

We are going to perform time seriesanalysis on the purchases by these

customers over the last 2 years - inorder to better understand salespatterns and predict future revenues.

33


33/51



Step 2: Single Exponential Smoothing

Click on HOME to return to the initial

screen

Review the options available:

Predictive Library: SAP HANA providestwo options for performing predictiveanalytics and data mining.

PAL is native to SAP HANA andprovides a selection of popular

algorithms for in-memory in-database processing foroptimum performance.

R Integration for SAP HANAprovides the ability to work withover 3500 algorithms availablethrough R

In this demo we focus on the PAL.

Product Category: ability to filter basedon product category (Cameras, TVs,etc.)

Exponential smoothing method: type ofsmoothing to perform. Single for now

NB: Display Data is simply to controlhow many observations (rows) from thesource / result datasets should beshown in the DATA tab. It has no effecton the segmentation process which isperformed on the SAP HANA server.

Press the Perform Time SeriesAnalysis button

34


34/51



The system will take a second to

perform the time series analysis.

We have not filtered by product categoryso analysis is performed for all of thedata we have over the last 2 years!

This is big data but SAP HANA is able to

process (aggregrate and fit thepredictive model) extremely quickly inreal-time.

And its not necessary to export datainto a separate data mining application the predictive capabilities are native toSAP HANA.

35


35/51



The results are displayed in the

PREDICTIONS tab

We see total revenues (in millions) forthe last two years for all products in theblue bars.

The green line shows the fitted

predicted line based on singleexponential smoothing. We will interpretthat shortly

36


36/51



Optionally you can use the mouse to

highlight information on the chart andsee the actual values.

Sales are rising steadily this is good,but lets now take a look at a specificproduct category.

37


37/51




screen

Change the Product Category value toCameras.


38


38/51



The system needs just a few seconds to

perform the filtering, aggregation, andexponential smoothing analysis!

As previously, the results are displayedin the PREDICTIONS tab

Seems that Cameras follow the overall

trend of steadily rising sales over time.

Notice the green line seems to be a littlelow compared to the chart. This isbecause single exponential smoothinginvolves an inherent time-lag so valuesare typically about one time periodbehind.

NB: It is possible to influence this time-lag with the Start Period slider (not partof the script for this demo).

The analyst is not satisfied with the fit,and would like to consider other

possibilities

39


39/51



Step 3: Double Exponential Smoothing


screen

Change the Exponential SmoothingMethod value to Double.

Double exponential smoothing is usuallybetter suited to data that is trending.

Notice that a Trend Smoothing Factorslider has appeared. This allows theuser to fine tune the smoothing process.This is also referred to as the beta ()factor.

Values closer to 0 tend to producesmoother results; values nearer to 1

give greater weight to recent changes.

We will leave the value at 0.1.


40

Wh t t d / Wh t h ld


40/51



This time the fit looks much better!

Also notice that with double exponentialsmoothing we can predict future timeperiods.

Currently were looking 6 months intothe future.

All looks good for Cameras.

But what about TVs?

41



41/51




Change the Product Category value toTV.


42



42/51



Hmmm the fit is not so good for TVs.

Double exponential smoothing dealswith the trending - but this data appearsto have follow a seasonal cycle.

It seems that sales are lower during thesecond month of each quarter.

Its time to try Triple ExponentialSmoothing

43



43/51




Change the Exponential SmoothingMethod value to Triple.

Triple exponential smoothing is usuallybetter suited to data that is both trendingand exhibits seasonality (or any kind ofperiodicity).

Notice that Seasonality radio buttonshave appeared. As our TV sales seemto follow a distinct quarterly cycle wellleave the Quarterly button pressed.

Press the Perform Time Series

Analysis button

44



44/51


at to do say at you s ou d see

Ah thats better. We now have

seasonality included in the model butthe fit of the line is still a little off...

The business analyst suspects thatolder data from last year may be havingundue influence on the fit - perhaps heshould give greater emphasis to morerecent data?

45



45/51


y y

Step 4: Triple Exponential Smoothing


Notice that a Seasonal SmoothingFactor slider is now present. Thisallows the user to fine tune thesmoothing process. This is also referredto as the gamma () factor.

Values closer to 0 tend to produce

smoother results; values nearer to 1

give greater weight to recent changes.

Change the Seasonal Smoothing

Factor value to 0.4.

At the same time lets project the modelfarther into the future

Change the Forecast Periods value to11.


46



46/51


Ahh yes!!!

A perfect fit!

Not only do we see the predictionsbetter reflect the recent past, but wealso can now see future revenuesprojected until the end of the summer

next year.

We can continue to work interactively inthis way for any product or category andfor any number of periods into thefuture.

SAP HANA provides not only the in-memory processing power to aggregatethe data in real-time but also to performflexible, interactive, predictive analysisat the same time.

47



47/51


Step 5: Developer Overview

Click on ARCHITECTURE to showthe architecture diagram

So we have seen how complex datamining processing against largevolumes of detailed data becomespossible thanks to SAP HANA.

And its sofastthat you can make thisan iterative and interactive process tosee what provides the best fit

At this point, especially to a developeraudience, you can speak to the overallarchitecture of predictive analytics withSAP HANA and how it forms the centerof a rich ecosystem.

Business focused applications such asSAP Predictive Analysis, BI tools, andcustomer applications can all exploit thepower of SAP HANA for predictiveanalysis.

Data can come from any source (notlimited to SAS transactional data).

48



48/51


To a developer audience, nows a good

time to explain a little about how easy itis to take advantage of the predictiveanalytics capabilities of SAP HANA




In order to execute exponentialsmoothing we simply need to CALL itusing the single line below.

Highlight the pal: lines to show whichPAL algorithms were invoking.

49



49/51


Click on ARGS to show the

arguments that were passed to SAPHANA for processing

A number of parameters are available tocontrol how processing is performed.These are stored in a HANA table andcan be modified immediately prior toprocessing

In this case we updated the itemsnamed GAMMA to 0.4 andFORECAST_NUM to 11.


50



50/51


Conclusion


Lets summarize what we did:

Performed filtering, aggregation,and exponential smoothing ontwo years of sales transactionsvia SAP HANA PredictiveAnalysis Library (PAL) inseconds!!!

Single, double, and tripleexponential smoothing ispossible with PAL and suitdifferent data scenarios

encompassing trending andseasonality.

Showed how easy it is fordevelopers to leverage thisfunctionality.

NB: for business analystsconsider SAP PredictiveAnalysis which also works withSAP HANA and providesmodeling and visualizationcapabilities (i.e. no coding)

End of the demo.

51

COPYRIGHT 2412 SAP AG. ALL RIGHTS RESERVED.


51/51


SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and servicesmentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objectsproducts and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. BusinessObjects is an SAP company.Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logosare trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360, and B2B 360Servi ces are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAPcompany.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposesonly. National product specifications may vary.

These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informationalpurposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The onlywarranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any.Nothing herein should be construed as constituting an additional warranty.

Documents

8635_Demo Script Predictive Analysis With HANA Including PAL and R