Embed Size (px)
DESCRIPTION
Monitoring is an important issue in cloud environments because it assures that acquired cloud slices attend the user's expectations. However, these environments are multi-tenant and dynamic, requiring automation techniques to offload cloud administrators. In a previous work, we proposed FlexACMS: a framework to automate monitoring configuration related to cloud slices using multiple monitoring solutions. In this work, we enhanced FlexACMS to allow dynamic and automatic attribution of monitoring configuration tasks to servers without administrator intervention, which was not available in previous version. FlexACMS also considers the monitoring server load when attributing configuration tasks, which allows load balancing between monitoring servers. The evaluation showed that enhancements reduced FlexACMS response time up to 60% in comparison to previous version. The scalability evaluation of enhanced version demonstrated the feasibility of our approach in large scale cloud environments.
Citation preview
1/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Efficient Configuration of Monitoring Slices for
Cloud Platform Administrators
Marcio Barbosa de Carvalho, Rafael Pereira Esteves, Guilherme da CunhaRodrigues, Clarissa Cassales Marquezan (2), Lisandro Zambenedetti Granville,
Liane Margarida Rockenbach TaroucoInstitute of Informatics – Federal University of Rio Grande do Sul – Brazil
(2) Paluno, University of Duisburg-Essen, Germany
19th IEEE Symposium on Computer and CommunicationsJune 24th, 2014
2/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Outline
1 Introduction
2 FlexACMSNew architecture
3 EvaluationComparativeScalability
4 Conclusions and future work
3/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Introduction
Motivation
Cloud providers grant computational resources (e.g., compute,storage, network) to cloud users in form of cloud slices
Cloud slices must be closely monitored by the cloud providerto avoid wasting expensive physical resources while stillsatisfying the cloud users expectations
A single monitoring solution does not address all cloudmonitoring requirements, which imposes to cloudadministrators the utilization of multiple monitoring solutions
Once a cloud slice is created, a set of monitoring solutionsmust be configured in order to start monitoring the computingresources that form the new cloud slice
4/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Introduction
Monitoring Slices
Monitoring Slices
Monitoring Slices reflect all the monitoring information abouta cloud slice, which is composed by the collected values of themonitored metrics and the configuration of the monitoring solutionsthat are needed to collect these metrics.
Every cloud slice is coupled with a monitoring slice, whose goal isto detect cloud slice malfunctioning
5/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Introduction
Monitoring slices
CPU
Memory
Network
CPU Utilization
Memory Utilization
Network Utilization
Monitoring SlicesCloud Slices
OpenStack
Ceilometer
6/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Introduction
Problem
Problem
However, the lack of integration between some monitoring solutionsand cloud platforms imposes for cloud administrators to manuallyset up monitoring solutions or develop scripts to automate the mon-itoring slice creation
7/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
Approach
In a previous work1, we propose a framework to automaticallycreate monitoring slices when cloud slices are created
Flexible Automated Cloud Monitoring Slices (FlexACMS):
enables self-configurable cloud monitoring strategiesindependent of the monitoring solutions employedcreates monitoring slices automatically using monitoringsolutions that satisfy the cloud administrator’s needsfacilitates the reuse of scripts developed by cloudadministrators to automate the creation of monitoring slicessupports diverse cloud platforms and monitoring solutionscollects information from cloud platforms and triggersappropriated components to build monitoring slices
1CNSM 2013, Poster session
8/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
Approach
Now, we enhanced FlexACMS:
Dynamic and automatic attribution of configuration tasks in aMessage-Queueing fashion: enables the automatic mappingbetween type of metrics to be monitored in a monitoring sliceand the type of monitoring server to consume the request -Enhancement!Load balancing during the configuration of the monitoringslices: enables the monitoring servers to volunteer to receive atask (i.e., setting up the monitoring slice) when they havecapacity for doing so - Enhancement!
9/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
New architecture
FlexACMS Core
Gatherers
Configurators
REST
WebService
...
Cloud Monitoring
... Monitoring
Slices
Cloud Platform
... Cloud
Slices
Requests
Queue
Request
Workers
Changes
Queue
Change
WorkerChange
WorkerConfigurator
Workers
Change
WorkerChange
WorkerChange
Workers
Configurator
Queues
12
3
4
5
7
8
9
6
...
Gatherers:
responsible for collecting information from cloud platforms(e.g., @slice.ip, @slice.identifier, @slice.owner)handle peculiarities of APIs (e.g., Amazon EC2 API)send collected information to FlexACMS Core through RESTWeb service
10/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
New architecture
FlexACMS Core
Gatherers
Configurators
REST
WebService
...
Cloud Monitoring
... Monitoring
Slices
Cloud Platform
... Cloud
Slices
Requests
Queue
Request
Workers
Changes
Queue
Change
WorkerChange
WorkerConfigurator
Workers
Change
WorkerChange
WorkerChange
Workers
Configurator
Queues
12
3
4
5
7
8
9
6
...
FlexACMS Core:
responsible for processing information collected by gatherersbreaks the request into small tasks to be processed in a parallelmanner - Enhancement!detects operations performed in the platform (e.g., cloud slicecreation) (Request Workers)
11/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
New architecture
FlexACMS Core
Gatherers
Configurators
REST
WebService
...
Cloud Monitoring
... Monitoring
Slices
Cloud Platform
... Cloud
Slices
Requests
Queue
Request
Workers
Changes
Queue
Change
WorkerChange
WorkerConfigurator
Workers
Change
WorkerChange
WorkerChange
Workers
Configurator
Queues
12
3
4
5
7
8
9
6
...
FlexACMS Core (Change Workers):
evaluates the rules predefined by cloud administrators totrigger configuration taskseach Configurator has an interest and a set of conditionsputs the configuration task in the Configurator Queuepredefined by the cloud administrator - Enhancement!
12/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
New architecture
Examples of Configurator rules
Configurator Attribute ValueName: nagios host basic Interest New SliceQueue: nagios basic Condition @slice.MaaS =˜ /basic/Name: nagios cpu basic Interest New ResourceQueue: nagios basic Condition @resource.identifier =˜ /CPU/
Condition @slice.MaaS =˜ /basic/Name: nagios host plat Interest New SliceQueue: nagios platinum Condition @slice.MaaS =˜ /platinum/Name: nagios cpu plat Interest New ResourceQueue: nagios platinum Condition @resource.identifier =˜ /CPU/
Condition @slice.MaaS =˜ /platinum/
13/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
New architecture
FlexACMS Core
Gatherers
Configurators
REST
WebService
...
Cloud Monitoring
... Monitoring
Slices
Cloud Platform
... Cloud
Slices
Requests
Queue
Request
Workers
Changes
Queue
Change
WorkerChange
WorkerConfigurator
Workers
Change
WorkerChange
WorkerChange
Workers
Configurator
Queues
12
3
4
5
7
8
9
6
...
Configurator Queues:
represent pools of monitoring servers that share somecapability (e.g., monitoring solutions, monitoring purpose,server capacity) - Enhancement!
14/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
FlexACMS
New architecture
FlexACMS Core
Gatherers
Configurators
REST
WebService
...
Cloud Monitoring
... Monitoring
Slices
Cloud Platform
... Cloud
Slices
Requests
Queue
Request
Workers
Changes
Queue
Change
WorkerChange
WorkerConfigurator
Workers
Change
WorkerChange
WorkerChange
Workers
Configurator
Queues
12
3
4
5
7
8
9
6
...
Configurator Workers:
consume configuration tasks from the Configurator Queue ifthe monitoring server has capacity to perform the task (e.g.,appropriated server load) - Enhancement!execute the Configurator and send its status and output toREST Web service for past analysis and debug
15/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Evaluation
Comparative
Comparative
Goal: evaluate what is the impact of the enhancements in theFlexACMS performance
Cloud platform: OpenStack
Monitoring solutions: Nagios and MRTG
Created cloud slices: from 1 to 10 cloud slices
Monitoring slices: from 1 to 10 monitoring slices, with 2 and52 metrics
Workers: 10 configurator workers
16/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Evaluation
Comparative
Previous vs Enhanced version
1 2 3 4 5 6 7 8 9 10
Created Cloud Slices
Re
sp
on
se
tim
e(s
)
020
40
60
80
100
120
140
27.1 %
30.9 %36.9 %
36.7 %
35.5 %
34.5 %
35.6 %
36.7 %
35.9 %37.4 %
OpenStack
FlexACMS
(a) Previous (2 metrics)
1 2 3 4 5 6 7 8 9 10
Created Cloud Slices
Re
sp
on
se
tim
e(s
)
020
40
60
80
100
120
140
25.8 %
29.1 %28.3 %
27.4 %25.6 %
23.5 %
22.3 %
20.6 %20.5 %
18.2 %
OpenStackFlexACMS
(b) Enhanced (2 metrics)
1 2 3 4 5 6 7 8 9 10
Created Cloud Slices
Re
sp
on
se
tim
e(s
)
020
40
60
80
100
120
140
43.7 %
46.1 %
45.9 %
41.5 %
41.6 %
41.3 %
40.7 %42.9 %
41.3 %
40.7 %
OpenStackFlexACMS
(c) Enhanced (52 metrics)
Figure: Previous vs Enhanced version
17/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Evaluation
Scalability
Scalability
Goal: evaluate the scalability in regards to response timevarying:
number of cloud slices already in place (101,102,103,104)number of new monitoring slices in a burst (10,40,70,100)number of metrics per monitoring slice (5,25,50)
Cloud platform: a gatherer generates requests similar to anactual OpenStack gatherer
Monitoring solutions: Nagios
Workers: 10 configurator workers
18/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Evaluation
Scalability
Scalability0
50
15
02
50
35
0
log(cloud slices already in place)
Re
sp
on
se
tim
e(s
)
New monitoring slices
10 40 70 100
1 2 3 4
(a) 5 metrics permonitoring slice
05
01
50
25
03
50
log(cloud slices already in place)
Re
sp
on
se
tim
e(s
)
New monitoring slices
10 40 70 100
1 2 3 4
(b) 25 metrics permonitoring slice
05
01
50
25
03
50
log(cloud slices already in place)
Re
sp
on
se
tim
e(s
)
New monitoring slices
10 40 70 100
1 2 3 4
(c) 50 metrics permonitoring slice
Figure: Scalability
19/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Conclusions and future work
Conclusions
New FlexACMS features:
Dynamic and automatic attribution of monitoring tasks topools of monitoring serversLoad balancing when attributing monitoring tasks
Performance enhancements:
reduces circa of 10% its influence in the evaluation time(Comparative)FlexACMS time reduced up to 60% (Comparative - 10 newmonitoring slices)
20/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Conclusions and future work
Conclusions
Scalability evaluation:
The number of cloud slices already in place does not affect theresponse timeThe number of metrics per monitoring slice and the number ofmonitoring slices that must be built affect the response time,but they do not affect the response time at same rate ofscenario growth
The automatic configuration of multiple monitoring solutionsfor cloud computing is feasible
FlexACMS can be used to help cloud administrators toachieve their monitoring needs using monitoring solutions thatare not integrated to cloud platforms
21/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Conclusions and future work
Future work
Future work:
Complete the cloud slice life-cycle automation (reconfigurationand destruction of monitoring slices)Improve load balancing strategiesInvestigate FlexACMS feasibility for PaaS and SaaS cloudmodels
22/22
Efficient Configuration of Monitoring Slices for Cloud Platform Administrators
Conclusions and future work
Acknowledgments
Thank you for your attention!
Questions?
