46
Telefónica Digital TELEFÓNICA DIGITAL Barcelona, October 11th 2013

Devopsdays barcelona

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: Devopsdays barcelona

Telefónica Digital

TELEFÓNICA DIGITAL

Barcelona, October 11th 2013

Page 2: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

About: me

last 8 years i have been working as performance test engineer with different tools and enviroments.

2

Page 3: Devopsdays barcelona

Architecture DesignWeb Performance OptimizationWeb Performance Optimization

Page 4: Devopsdays barcelona

No instruments

Users Review

Late or none Performance Testing

No Real User Monitoring

Reactive Performance Tuning

Page 5: Devopsdays barcelona

No tools, no performance dashboard, performance is for sysadmins and operatorsNo tools, no performance dashboard, performance is for sysadmins and operators

Page 6: Devopsdays barcelona

Releases are costly and it may take several months of workReleases are costly and it may take several months of work

Page 7: Devopsdays barcelona

Manual testing of each release after code freezeManual testing of each release after code freeze

Page 8: Devopsdays barcelona

Non functional Requirements are most likely ignoredNon functional Requirements are most likely ignored

Page 9: Devopsdays barcelona

In production there is no monitoring of the traffic and how it affects the businessIn production there is no monitoring of the traffic and how it affects the business

Page 10: Devopsdays barcelona

Users feedback is usually negative and there is no interaction with developers and designersUsers feedback is usually negative and there is no interaction with developers and designers

Page 11: Devopsdays barcelona

Application’s performance affects directly to market’s performanceApplication’s performance affects directly to market’s performance

Page 12: Devopsdays barcelona

• Continuous Integration– Functional testing– Automation– Monitoring

Page 13: Devopsdays barcelona

Continuous Integration for functional testing is working already in nightly buildsContinuous Integration for functional testing is working already in nightly builds

Page 14: Devopsdays barcelona

Automation reduces time to market for the applicationsAutomation reduces time to market for the applications

Page 15: Devopsdays barcelona

Monitoring the real user behaviour and not just healthcheck of serversMonitoring the real user behaviour and not just healthcheck of servers

Page 16: Devopsdays barcelona

Error and risks managementError and risks management

Page 17: Devopsdays barcelona

Tuning and bugfixingTuning and bugfixing

Page 18: Devopsdays barcelona

Listening to user feedbackListening to user feedback

Page 19: Devopsdays barcelona

The FutureThe Future

• Continuous Performance Integration– Performance tests integrated in Jenkins– Automation of the trend reports– Real User Monitoring Real time feedback

Page 20: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

SCRUM and PERFORMANCE

20

Page 21: Devopsdays barcelona

Pruebas de RendimientoPruebas de RendimientoProactive Performance testing for each release. Load tests will discover the flaws and bottlenecks, the application or the systemProactive Performance testing for each release. Load tests will discover the flaws and bottlenecks, the application or the systemmay have in production environmentmay have in production environment

TestingTestingTestingTesting

Page 22: Devopsdays barcelona

Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es Conocer el escenario productivo para tomar una buena decisión sobre cómo orientar las pruebas de rendimiento es La hipótesis de partida con más éxito.La hipótesis de partida con más éxito.High availability of the application and the system is the goal of a ready for service status. The application and the systems must be stable , High availability of the application and the system is the goal of a ready for service status. The application and the systems must be stable , efficient and dimensioned according to the usage.efficient and dimensioned according to the usage.

AvailabilityAvailabilityAvailabilityAvailability

Page 23: Devopsdays barcelona

Not only the response time is important, an intelligent use of the resources is needed to grow in the future. Efficiency, understood as Not only the response time is important, an intelligent use of the resources is needed to grow in the future. Efficiency, understood as capacity to dispose of the system resources to achive an objective, in our case response time and uptimecapacity to dispose of the system resources to achive an objective, in our case response time and uptime

VelocityVelocityVelocityVelocity

Page 24: Devopsdays barcelona

Being able to grow depending on the necessities of the market, users and new technologies is one of the questions to which a performance Being able to grow depending on the necessities of the market, users and new technologies is one of the questions to which a performance engineerfor will have to answerengineerfor will have to answer

ScalabilityScalabilityScalabilityScalability

Page 25: Devopsdays barcelona

A performance test is easy. It is easy to design non realistic scenarios. It is easy to collect irrelevant data. Even with a good scenario and A performance test is easy. It is easy to design non realistic scenarios. It is easy to collect irrelevant data. Even with a good scenario and Appropiate data, it is easy to use and incorrect statistic method to analysis the results. Appropiate data, it is easy to use and incorrect statistic method to analysis the results. - Alberto Savoia- Alberto Savoia

ScenariosScenariosScenariosScenarios

Page 26: Devopsdays barcelona

One of the most important parts of a good performance test design is to have an appropiate load test environment, as similar as possible to One of the most important parts of a good performance test design is to have an appropiate load test environment, as similar as possible to Production at all levels, networking, systems and application architecture.Production at all levels, networking, systems and application architecture.

PreProductionPreProductionPreProductionPreProduction

Page 27: Devopsdays barcelona

Otro título

EscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenariosEscenarios

To know the production environment is key to take good decisions about how to design a performance test plan. Designing a plan according To know the production environment is key to take good decisions about how to design a performance test plan. Designing a plan according To real traffic and usage of the platform is key in creating validation criteriaTo real traffic and usage of the platform is key in creating validation criteria

MonitoringMonitoringMonitoringMonitoring

Page 28: Devopsdays barcelona

DevelopersDevelopers, , DBA's,DBA's, QA's, DevOps, product owners ... All the team is part of performanceQA's, DevOps, product owners ... All the team is part of performance

Performance TeamsPerformance TeamsPerformance TeamsPerformance Teams

Page 29: Devopsdays barcelona

Otro título

There are many tools available in the market for load testing and monitoring. An effort in evaluating these tools will benefit at long term the There are many tools available in the market for load testing and monitoring. An effort in evaluating these tools will benefit at long term the Execution of the tests. However, the most important part is how the reports are generated and who is going to interpret them. Execution of the tests. However, the most important part is how the reports are generated and who is going to interpret them.

ToolsToolsToolsTools

Page 30: Devopsdays barcelona

Otro título

Mas puntos

Not only unique users or session times are important. How the users work with the application and the psicology of the them are key to Not only unique users or session times are important. How the users work with the application and the psicology of the them are key to Understand the results and how it affects to business. Understand the results and how it affects to business.

Real User MonitoringReal User MonitoringReal User MonitoringReal User Monitoring

Page 31: Devopsdays barcelona

Keep it simple, use cache wisely, invest in testing and monitoring, create a culture of performance in all the organizationKeep it simple, use cache wisely, invest in testing and monitoring, create a culture of performance in all the organization

Best PracticesBest PracticesBest PracticesBest Practices

Page 32: Devopsdays barcelona

Tuning

Techonology develops at high speed. To bring out the best of our product, business and techonology need to evolve by the hand. Investing in Techonology develops at high speed. To bring out the best of our product, business and techonology need to evolve by the hand. Investing in Performance research is crucial to keep up with other internet competitors. Performance research is crucial to keep up with other internet competitors.

InnovationInnovationInnovationInnovation

Page 33: Devopsdays barcelona
Page 34: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

Understand the Project Vision and ContextProject Vision

Project Context

Understand the system

Understand the Project Environment

Understand the Performance Build Schedule

35

Page 35: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

Improved way of working

Improve performance unit testing by pairing performance testers with developers.

Assess and configure new hardware by pairing performance testers with administrators.

Evaluate algorithm efficiency.

Monitor resource usage trends.

Measure response times.

Collect data for scalability and capacity planning.

36

Page 36: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

Configure the Test Environment

Set up isolated networking environment

Procure hardware as similar as possible to production or at least keeping ration amongst all elements

Coordinate bank of IP’s for IP spoofing

Monitoring tools and operating systems like production

Load generation tools or develop your own

37

Page 37: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

Identify and Coordinate Tasks

Work item execution method

Specifically what data will be collected

Specifically how that data will be collected

Who will assist, how, and when

Sequence of work items by priority

38

Page 38: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

Execute Task(s)

Keys to Conducting a Performance-Testing Task

•Analyze results immediately and revise the plan accordingly.

•Work closely with the team or sub-team that is most relevant to the task.

•Communicate frequently and openly across the team.

•Record results and significant findings.

•Record other data needed to repeat the test later.

•Revisit performance-testing priorities after no more than two days.

39

Page 39: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

Analyze Results and Report

 pause periodically to consolidate results

conduct trend analysis

create stakeholder reports,

pair with developers, architects, and administrators to analyze results

40

Page 40: Devopsdays barcelona

But … what are you actually doing day by day?But … what are you actually doing day by day?

Page 41: Devopsdays barcelona

Telefónica DigitalProduct Development & Innovation

Case of STUDY

42

Page 42: Devopsdays barcelona

• HTML5 trends using Yslow and Firebug

Page 43: Devopsdays barcelona

Branches comparison

Page 44: Devopsdays barcelona
Page 45: Devopsdays barcelona
Page 46: Devopsdays barcelona