Predictive Auto Scaling of In-Memory Data Stores

TEAM MEMBERS:● AKSHAY RAUL● RAHUL IYER● SHWETHA KALYANARAMAN

Motivation

● Caching is a major part of many application infrastructures

● Service needs to be resilient, reliable, and scalable

● Costly to maintain multiple nodes to service clients

● Auto-Scaling to the rescue!

What are In-Memory Data Stores?

● Type of non-relational database

● Relies on memory for storage

● Helps in minimizing response time

Introduction to Redis

● In-Memory data store

● Simple key value store

● Supports sharding and replication

● Lightweight and multi-platform

● Provides a lot of performance metrics

Setup Infrastructure: AWS

● ECS (Docker containers running Redis)

● Easy to scale

● redis-cli to capture Redis’ metrics

● collectd to capture system level metrics

● Aggregated metrics to be sent out to the prediction engine

Load Testing

● Huge set of supported operations

● Easy to program

● Varied workload intensities

Metrics Considered

● CPU/Memory usage from Redis

● Request Latency

● System’s CPU/Memory usage from collectd

Approach

● Based on Utilization factor

● Predicting Future Workload → Predicting the need for Autoscaling

● Using Prediction Algorithms

Prediction Algorithms

● Recurrent Neural Network

● ARIMA MODEL

RNN

● Make use of sequential information

● Output is dependant on previous computations

● RNNs are used in a lot of areas:

○ Speech Recognition

○ Machine Translation

RNN

ARIMA Model● Stands for Autoregressive integrated moving average

● Model is fitted to time series data either to better

understand the data or to predict future points in the

series (forecasting).

● Used in short-term forecasting that requires 40 historic

data points

● It uses autocorrelation to correlate data points which are

at specific time interval apart

ARIMA Model● AR: Autoregression. A model that uses the dependent

relationship between an observation and some number of lagged observations.

● I: Integrated. The use of differencing of raw observations (e.g. subtracting an observation from an observation at the previous time step) in order to make the time series stationary.

● MA: Moving Average. A model that uses the dependency between an observation and a residual error from a moving average model applied to lagged observations.

Algorithm Skeleton

Utilization_Factor=(Load Present+Predicted Load)

If Utilization_Factor > 0.70

Autoscale!!

If Utilization_Factor <0.30

Scale down!!

Benefits of Auto Scaling

● Better fault tolerance

● Better availability

● Better cost management

● Optimization of resource needs

Work Distribution● AKSHAY - Data Cleaning, Data Selection and Training the ARIMA

Model for prediction. Experimenting by varying lag observations, raw observations and moving average component to evaluate the performance of the model

● SHWETHA - Data Pre-Processing and Training of Recurrent Neural Network for prediction of incoming load. Experimenting with different hidden layers and compare analysis with ARIMA model

● RAHUL - Setting up the Redis cluster and will be working on load testing to capture metrics. Also, will manage the addition and removal of nodes as per the prediction.

QUESTIONS?

References● Chung-Horng Lung Ali Yadavar Nikravesh, Samuel A. Ajila. 2015. Towards an Autonomic Auto-Scaling Prediction

System for Cloud Resource Provisioning. IEEE (2015). https://doi.org/10.1109/SEAMS.2015.22● Shu Tao Nikos Anerousis Arijit Khan, Xifeng Yan. 2012. Workload characterization and prediction in the cloud: A

multiple time series approach. (2012). https://ieeexplore.ieee.org/document/6212065● Guo Ma Bin Xie, Guanyi Sun. 2017. Prediction-based autoscaling for container-based PaaS system. IEEE (2017).

https://ieeexplore-ieee-org/document/8239026● Emiliano Casalicchio and Vanessa Perciballi. 2017. Auto-Scaling of Containers: The Impact of Relative and Absolute

Metrics. 2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2017). https://doi.org/10.1109/fas-w.2017.149

● Yao-Tsung Yang Li-Der Chou Chia-Wei Tseng, Ming-Shiun Tsai. 2017. A Rapid Auto-Scaling Mechanism in Cloud Computing Environment. In Int’l Conf. Grid, Cloud, & Cluster Computing (GCC’17) CSREA Press. https://csce.ucmss.com/cr/books/2017/LFS/CSREA2017/GCC6087.pdf

● Zhenhuan Gong, Xiaohui Gu, and John Wilkes. 2010. PRESS: PRedictive Elastic ReSource Scaling for cloud systems. 2010 International Conference on Network and Service Management (2010). https://doi.org/10.1109/cnsm.2010.5691343

● Miguel-Alonso J. & Lozano Lorido-Botran, T. 2014. A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments. Grid Computing (2014). https://doi.org/10.1007/s10723-014-9314-7

● Marty Humphrey Ming Mao. 2011. Auto-Scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows. IEEE (2011). https://ieeexplore.ieee.org/document/7027559

● Alexei Karve Andrzej Kochut Muhammad Wajahat, Anshul Gandhi. 2016. Using machine learning for black-box autoscaling. IEEE (2016). https://ieeexplore.ieee.org/document/7892598

References● Abhishek Dubey Nilabja Roy and Aniruddha Gokhale. 2011. Efficient Autoscaling in the Cloud using Predictive

Models for Workload Forecasting. IEEE (2011). https://ieeexplore.ieee.org/document/6008748● Maria Alejandra Rodriguez and Rajkumar Buyya. 2018. Containers Orchestration with Cost-Efficient Autoscaling

in Cloud Computing Environments. CoRR abs/1812.00300 (2018). arXiv:1812.00300 http://arxiv.org/abs/1812.00300

● Moumita Mitra Manna Sunirmal Khatua and Nandini Mukherjee. 2014. Prediction-based Instant Resource Provisioning for Cloud Applications. IEEE (2014). https://doi.org/10.1109/UCC.2014.92

● Dehai Zhao Faming Gong Weishan Zhang, Bo Li and Qinghua Lu. Workload Prediction for Cloud Cluster using A Recurrent Neural Network. IEEE (2016) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8281183&tag=1

● Hao Zhang, Bogdan Marius Tudor, Gang Chen, and Beng Chin Ooi. 2014. Efficient In-memory Data Management: An Analysis. Proc. VLDB Endow. 7, 10 (June 2014), 833–836. https://doi.org/10.14778/2732951.2732956