MATLAB®MATLAB®

Scalable Fast Parallel SVM in Cloud Clusters for Large Datasets Classification

ByGhazanfar Latif (Gabe)gabe@prebinary.com

MATLAB®

Presentation Outline

Part 1: Introduction of Cloud Computing

Part 2: Introduction of Support Vector

Machine

Part 3: Problem Description

Part 4: Distributing SVM on Cloud Cluster Nodes

Part 5: Experimental Results & Conclusion

2

MATLAB®

As an end-consumer, believe it or not

you’ve been using Cloud for long times

3

MATLAB®

Cloud computing characteristicsCloud Computing is not answer for all

But it could simplify our lives……

4

MATLAB®

Your data is replicated3 or 4 times in their data center

High Availability

5

MATLAB®

Adding “servers” is a click away. Running in just minutes, not days

6

MATLAB®

Sensitive Data in the Cloud?are there yet?

Data at Rest

Data in Motion

Encryption

7

MATLAB®

It can even load balance your server traffic

8

MATLAB®

Cloud Computing is relatively new technology

9

MATLAB®

Hosted Server & Applications Access

Employees

Customers Suppliers

10

MATLAB®

Hosting Players

Often Monthly

Your contracts

Cloud Players

Pay As You Go

Pay only what you use

11

MATLAB®

Amazon Cloud Services

12

Amazon EC2

Cloud Servers ranges from 1GHz CPU, 613MB RAM to 110GHz

CPU and 68GB RAM. (6 Regions, 3 Zones)

Amazon S3

Cloud Storage Service where we can upload up to 5000 TB of

Data.

Amazon VPC

Virtual Private Cloud within the Cloud Servers or in between

Cloud Servers and our local machines.

Amazon Cloud Watch/SNS

Resources Utilization Monitoring and sending emails or SMS to

the concerned persons.

MATLAB®

Support Vector Machine

• Support vector machines were originally proposed by Boser, Guyon and Vapnik in 1992 and gained increasing popularity in late 1990s.

• SVM is supervised learning methods that analyze data and recognize patterns, used for classification.

• SVMs are currently among the best performers for a number of classification tasks ranging from text to genomic data.

13

MATLAB®

SVM Applications

• SVMs can be applied to complex data types(e.g. graphs, sequences, relational data) by designing kernel functions for such data.

• Currently, SVM is widely used in object detection & recognition. Text Recognition Speech Recognition Pattern recognition

content-based image retrieval DNA array expression data analysis

Protein classification

Handwriting Recognition

Face Expression Recognition

Email filtering

Web searching

Sorting documents by topic

Words counts

14

MATLAB®

SVM: Basic Idea

• Find the hyper-plane that maximizes the margin

• The perpendicular distance to the closest positive sample or negative sample is called the margin

• Tuning SVMs remains a black art: selecting a specific kernel and parameters is usually done in a try-and-see manner.

15

Which of the linear separators is optimal?

MATLAB®

SVM: Basic Idea (continue)

16

Vectors on the margin are the support vectors, and the total margin is 2/llWll

Class 1Margin

Total Margin

-

+

support vectors

MATLAB®

Problem Statement

• For testing and training of a multidimensional large datasets by using SVM requires a lot of computing resources in terms of memory and computational power.

• It is very expensive to purchase High performance computational hardware for training of large datasets.

• Researchers also face problems due to limited computational resources available at their institutions and they need to wait a lot to get results.

17CS Department, KFUPM (KSA).

MATLAB®

Proposed Solution

• Cloud Computing is emerging today as a commercial infrastructure that eliminates the need for maintaining expensive computing hardware.

• We purposed a technique for running support vector machines in parallel on distributed cloud cluster nodes which reduced memory requirements and computational power.

• Our solution is auto scalable and cost effective in terms of time and computational power expenditures.

18CS Department, KFUPM (KSA).

MATLAB®

Proposed ArchitectureInput Dataset “D”

Equal Dataset Distribution

Cluster Node #2 Cluster Node #3 Cluster Node #nCluster Node #1

D/N

D/ND/N

D/N

Merging Generated Data Vectors

SV-nSV-3SV-2SV-1

Master Cluster Node

SV

NewSV

.…

19CS Department, KFUPM (KSA).

MATLAB®

Algorithm

•

20CS Department, KFUPM (KSA).

MATLAB®

Experiments

• We used 4 nodes of Amazon EC2 HPC Clusters which are locally interconnected via VPC for testing our datasets in the cloud.

• EC2 Cluster Specifications Memory: 23 GB Memory CPU: 33.5 EC2 Compute Units (≈ 43.5 GHz) Network Connectivity: 10 Gigabit Ethernet Platform: 64-bit Operating System: Linux Tools: MATLAB, AWS Scripting in Java

21CS Department, KFUPM (KSA).

MATLAB®

Testing Datasets

• For testing our proposed solution, we used 8 different sized datasets having 2, 4, 8 features:

• To created Testing Datasets we used Cos-Exp, Gaussian, Multi Class Gaussian distribution classes.

• We also tested our proposed solution on online available LIBSVM Classification datasets at www.ntu.edu.tw.

22CS Department, KFUPM (KSA).

Test # Data Size # of Features1 2000 22 5000 23 10000 24 16000 25 24000 26 4000 47 22400 48 59535 8

MATLAB®

Single Node Test Results

23CS Department, KFUPM (KSA).

Test # Data Size FeaturesSingle Node

PT ISV Accuracy

1 2000 2 14.549 804 86.2

2 5000 2 89.35 1916 84.84

3 10000 2 982.68 3620 85.12

4 16000 2 21422.22 5715 84.84

5 24000 2 79195 8407 84.97

6 4000 4 388.5193 1815 90.375

7 22400 4 53052.36 8647 85.96

8 59535 8 83517 25074 96.797

PT Processing TimeISV Identified Support Vectors

MATLAB®

Parallel Cluster Nodes Test Results

24CS Department, KFUPM (KSA).

Test # Data Size Features

Multi Node Parallel Clusters (P1)Node 1 Node 2 Node 3 Node 4

TSVPT ISV PT ISV PT ISV PT ISV

1 2000 2 0.634 251 0.553 228 0.505 241 0.515 228 9482 5000 2 8.269 563 8.407 530 8.649 534 8.648 542 21693 10000 2 31.021 1001 24.772 964 18.939 1039 20.824 1015 40194 16000 2 58.139 1526 61.31 1591 52.27 1577 45.71 1566 62605 24000 2 200.94 2303 123.21 2286 135.26 2272 227.79 2219 9080

6 4000 4 7.737 593 7.786 594 8.224 617 7.913 609 2413

8 22400 4 1054.898 2428 1231.171 2420 910.6977 2363 2246.163 2500 9711

9 59535 8 13931 7979 14037 8773 8606.2 6046 12018 8254 31052

PT Processing TimeISV Identified Support VectorsTSV Total Identified Support Vectors

MATLAB®

Parallel Cluster Nodes Test Results (continue)

25CS Department, KFUPM (KSA).

Test # Data Size Features

Multi Node Parallel Clusters (P2)Merging Results of Multi Node to single Node

TSV PT ISV Accuracy TPT Efficiency Accuracy Effect

1 2000 2 948 4.321 721 85.3 4.955 65.94 1.04%2 5000 2 2169 37.53 1822 84.88 46.179 49 -0.047%3 10000 2 4019 313.1 3494 85.09 344.121 64.88 0.035%4 16000 2 6260 2102.75 5603 84.8 2164.06 89.89 0.047%5 24000 2 9080 4959.9 8259 85.021 5187.69 93.45 -0.06%6 4000 4 2413 214.1918 1610 89.125 222.4164 42.75 1.30%8 22400 4 9711 25815.7 7959 85.92 28061.87 47.1 0.10%

9 59535 8 31052 36007 24467 96.67 50044 46.01 0.131%

TSV Total Identified Support VectorsPT Processing TimeISV Identified Support VectorsTPT Total Processing time for Dataset

MATLAB®

Accuracy Comparison

26CS Department, KFUPM (KSA).

1 2 3 4 5 6 7 875

80

85

90

95

100

M-Accuracy S-Accuracy

Dataset #

Acc

urac

y

MATLAB®

Performance Efficiency

27CS Department, KFUPM (KSA).

1 2 3 4 5 6 7 80%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

34.06

51

35.12

10.11

6.55

57.25

52.9

53.99

M-Time S-Time Percentage

Dataset #

% P

roce

ssin

g Ti

me

MATLAB®

Identified Support Vectors

28CS Department, KFUPM (KSA).

1 2 3 4 5 6 7 80

5000

10000

15000

20000

25000

30000

S-ISV M-ISV

Dataset #

Supp

ort

Vect

ors

MATLAB®

Comparison with Existing Techniques

I. An Intelligent System for Accelerating Parallel SVM Classification Problems on Large Datasets Using GPU.

II. Parallel Support Vector Machines: The Cascade SVM.III. Distributed Parallel Support Vector Machines in Strongly Connected Networks.IV. A Fast Parallel Optimization for Training Support Vector Machine.

29CS Department, KFUPM (KSA).

Type of Infrastructure Efficiency Accuracy Resources Cost

Amazon Cloud Clusters Up to 60% On Average 0.20% Overhead

Hourly basedPay only what you use

GPU Clusters Up to 80% On average 0.55% Overhead

Physical MachinesGPU Maintenance Cost

Local Cascade SVM Method Depending upon the # of iterations

Depending upon the # of iterations

Physical MachinesNetworking Cost

Local Strongly Connected Networks Depending upon the # of iterations

Depending upon the # of iterations

Physical MachinesNetworking Cost

Local Single Node Maximum Time Maximum Efficiency

Normal Physical Machine

MATLAB®

Conclusion

• We prove that our proposed solution is very efficient in terms of training time as compared to the existing techniques and it classifies the datasets correctly with minimal error rate.

• Experiments over a real-world and test databases shows that this algorithm is scalable and robust.

30CS Department, KFUPM (KSA).

MATLAB®

Future Work

• We will extend the performance evaluation results by running similar experiments on other IaaS providers and clouds also on other real large-scale platforms, such as grids and commodity clusters .

31CS Department, KFUPM (KSA).

MATLAB®

References

32CS Department, KFUPM (KSA).

[1] Florian Schatz, Sven Koschnicke, Niklas Paulsen, Christoph Starke, and Manfred Schimmler, “MPI Performance Analysis of Amazon EC2 Cloud Services for High Performance Computing”, A. Abraham et al. (Eds.): ACC 2011, Part I, CCIS 190, pp. 371–381, 2011. Springer-Verlag Berlin Heidelberg 2011.

[2] Simon Ostermann, AlexandruIosup , Nezih Yigitbasi, Radu Prodan, Thomas Fahringer and Dick Eperna, “A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing”, D.R. Avreskyetal. (Eds.) : Cloudcomp 2009 , LNICST 34, pp. 115- 131 , 2010. Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 2010.

[3] Amazon Elastic Compute Cloud (Amazon EC2): http://aws.amazon.com/ec2/ [4] High Performance Computing (HPC) on AWS Clusters: http://aws.amazon.com/hpc-applications/ [5] G. Zanghirati and L. Zanni, “A parallel solver for large quadratic programs in training support vector

machines,” Parallel Comput., vol. 29, pp. 535–551, Nov. 2003.[6] C. Caragea, D. Caragea, and V. Honavar, “Learning support vector machine classifiers from distributed data

sources,” in Proc. 20th Nat. Conf. Artif. Intell. Student Abstract Poster Program, Pittsburgh, PA, 2005, pp. 1602–1603.

[7] A. Navia-Vazquez, D. Gutierrez-Gonzalez, E. Parrado-Hernandez, and J. Navarro-Abellan, “Distributed support vector machines,” IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 1091–1097, Jul. 2006.

[8] Yumao Lu, Vwani Roychowdhury, and Lieven Vandenberghe, “Distributed Parallel Support Vector Machines in Strongly Connected Networks”, IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 19, NO. 7, JULY 2008.

MATLAB®

References

33CS Department, KFUPM (KSA).

[9] C.-C. Chang and C.-J. Lin, LIBSVM: a library for support vector machines, 2001, software and datasets available at http://www.csie.ntu.edu.tw/cjlin/libsvm.

[10] B. Catanzaro, N. Sundaram, and K. Keutzer, “Fast support vector machine training and classification on graphics processors,” in ICML ’08: Proceedings of the 25th international conference on Machine learning. New York, NY, USA: ACM, 2008, pp. 104–111.

[11] S. Herrero-Lopez, J. R. Williams, and A. Sanchez, “Parallel multiclass classification using svms on gpus,” in GPGPU’10: Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units. New York, NY, USA: ACM, 2010, pp. 2–11.

[12] Cao, L., Keerthi, S., Ong, C.-J., Zhang, J., Periyathamby, U., Fu, X. J., & Lee, H. (2006). Parallel sequential minimal optimization for the training of support vector machines. IEEE Transactions on Neural Networks, 17, 1039-1049.

[13] Graf, H. P., Cosatto, E., Bottou, L., Dourdanovic, I., & Vapnik, V. (2005). Parallel support vector machines: The cascade svm. In L. K. Saul, Y. Weiss and L. Bottou (Eds.), Advances in neural information processing systems 17, 521-528. Cambridge, MA: MIT Press.

[14] Wu, G., Chang, E., Chen, Y. K., & Hughes, C. (2006). Incremental approximate matrix factorization for speeding up support vector machines. KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 760-766). New York, NY, USA: ACM Press.

[15] Zanni, L., Serani, T., & Zanghirati, G. (2006). Parallel software for training large scale support vector machines on multiprocessor systems. J. Mach. Learn. Res., 7, 1467-1492.

[16] Qi Li, Raied Salman, Vojislav Kecman, “An Intelligent System for Accelerating Parallel SVM Classification Problems on Large Datasets Using GPU”, 2010 10th International Conference on Intelligent Systems Design and Applications.

MATLAB®MATLAB®

Thanks fromGhazanfar Latif

Questions ?Comments ??

Suggestions ???