codecentric AG

Monitoring Performance from Development through Production

AGILE DEVELOPMENT OF HIGH PERFORMANCE APPLICATIONS

codecentric AG

- Fabian Lange- Head of Competence Center Performance

- Java since its beginning- Agile since its establishment- Performance since waiting got boring

ME AND CODECENTRIC

- codecentric AG- Specialized in

- Perfomance Services- Agile Software Factory

- Always looking for new talent

codecentric AG

TABLE OF CONTENTS

- Chapter One"The curse of non-functional Requirements”

- Chapter Two“Ensuring Great Performance”

- Chapter Three“A Real World Example”

- Chapter Four“The DevOps Revolution"

- Epilogue

codecentric AG

THE CURSE OF NON-FUNCTIONAL REQUIREMENTS

Chapter One

codecentric AG

- Who measures performance …- in production?

- before production?

- during development?

- Who does development …- the waterfall way?

- the agile way?

LET‘S DO A POLL

codecentric AG

- We finally can test functional requirements!

- Many modern practices- TDD

- ATDD

- BDD

TESTING REQUIREMENTS

codecentric AG

- Executable Specifications makes functional testing a breeze!

REQUIREMENTS ARE THE TEST

*** Settings ***Resource ${RESOURCES}/BDD.txtTest Template Branch Manager Change Should not affect Employee *** Keyword ***Branch Manager Change Should not affect Employee [Arguments] ${periodClosed} ${periodOpenAndModified} ${importDay} ${oldManagerValidUntil} ${newManagerValidFrom} Given initialized criteria for bonus commercial And a branch B with branch manager M_OLD and employee E1 And evaluation for E1 for period ${periodClosed} which is closed And evaluation for E1 for period ${periodOpenAndModified} which is open and modified When M_NEW becomes new manager of branch B And import service is called on ${importDay} Then the new branch manager of branch B is M_NEW valid from ${newManagerValidFrom} And branch manager M_OLD manages employee E until ${oldManagerValidUntil} And branch manager M_NEW manages employee E from ${newManagerValidFrom} And Evaluations for E1 still have the same content | *Test* | *Closed Period* | *Open Period* | *Run Import On* | *Old Manager Stops* | *New Manager Starts* || 1 | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 | 11.11.2009 | 30.11.2009 | 1.12.2009 || 2 | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 | 1.11.2009 | 31.10.2009 | 1.11.2009 || 3 | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 | 1.12.2009 | 30.11.2009 | 1.12.2009 |

codecentric AG

PERFORMANCE IS NON-FUNCTIONAL!

- All Non Functional Requirements are not very agile

- They cannot be added later on

- So you need to know about them!

- They form the Definition of Done

codecentric AG

TESTING NON-FUNCTIONAL IS HARD!

- There are no absolute measures

- No production infrastructure

codecentric AG

HOW DO YOU MEASURE PERFORMANCE?

- Relevant Measures are hard to find- Response Time

- For users- System Load

- For planning- Traffic

- For money

- Realistic Measures are hard to obtain- 2 seconds?

- Load avg < 2.8 ?

- Less than 2TB per month?

codecentric AG

HOW DO YOU TEST PERFORMANCE?

- Who ...... has a process for performance tests?

... does loadtests?

... plans for scalability?

... uses a profiler?

... uses a server monitor?

... uses an application monitor?

codecentric AG

„PRODUCTION IS FASTER“

codecentric AG

ENSURING GREAT PERFORMANCE

Chapter Two:

codecentric AG

A TYPICAL PERFORMANCE ANALYSIS PROCESS

- Tom, the boss calls:“We loose customers because of bad performance”

- Lynn from QA does a load test:“Application is slow as a snail”

- Task force is set up- John tries to learn performance tools- Sarah does a microbenchmark and gains 5 ms

Application still slow

Everybody unhappy

codecentric AG

HOW ABOUT…

- Developers care about performance- Good tools are understood and used- Performance is tested regularly- Anomalies are taken care of

Application is running smoothly

Everybody is happy

codecentric AG

CARE ABOUT PERFORMANCE

- Caring is fundamentally important

- Development teams need to extend their scope

- In Scrum teams need to be able to do all the work to get done

codecentric AG

GOOD TOOLS

- Tool paralysis does not help

- Choose 1 or 2 good tools and learn them

- Tools should work everywhere

flickr.c

om/p

hotos/p

mto

rrone/2

381346935

codecentric AG

CONTINUOUSLY TEST PERFORMANCE

- Find a good balance- Automated Checks

- Manual Tests

- Functional tests already provide data

- How about a load test every iteration?

codecentric AG

INVESTIGATE SUSPICIOUS DATA

- Because you care- And you have the tools- And you have the data

- You should investigate findings

“When you have eliminated the impossible,whatever remains, however improbable,must be the truth”

- Sherlock Holmesflickr.com/photos/cayusa/2159980025

codecentric AG

A REAL WORLD EXAMPLEChapter Three

codecentric AG

TEAM CARES ABOUT PERFORMANCE

- Definition of Done includes a lot- spec, design, unit test, code, acceptance tests, documentation,

usability review, code review, stability tests, compatibility tests, interoperability test, load tests, security tests, performance tests…

- Get it right from the beginning- Do not pile up technical debt

- To go well you have to go slow

codecentric AG

– 20% Slowdown due to debt

– Sprint 1– 15 Points delivered

– Sprint 2– 12 Points delivered

– Sprint 3– 9 Points delivered

– Sprint 4– 7 Points delivered

TECHNICAL DEBT / UNDONE WORK

11.04.2023 22

Wrong Definition of Done

– Sprint 1– 12 Points delivered

– Sprint 2– 12 Points delivered

– Sprint 3– 12 Points delivered

– Sprint 4– 12 Points delivered

Better Definition of Done

codecentric AG

WHAT MAKES A GREAT TOOL

- Zero configuration a must for agile

- Very low overhead for clean results

- Single tool for all use cases

- Free!

codecentric AG

APPDYNAMICS LITE DEMO

- Webcasts in our bloghttp://blog.codecentric.de/en/2010/08/easy-performance-analysis-with-appdynamics-lite/

codecentric AG

AUTOMATED WORK

- Monitoring is not only great for production

- Runs in Continuous Integration environments

- Uses automated tests to run

- Provides Trends during iterations

codecentric AG

- Continuous Integration Tools show where to look- Usually already providing information about where to look- Sometimes providing information about how to fix

EASY INVESTIGATION

11.04.2023 26

– JUnit Report

– [Run a debugger]

– Fix it

JUnit

– Business Transaction Overview

– Call Graph

– [Run a profiler]

– Fix it

AppDynamics

codecentric AG

A JUNIT REPORT

codecentric AG

DAILY HEALTH CHECK

codecentric AG

SPOT ON ANSWER

codecentric AG

FIND ANANOMALIES

codecentric AG

INVESTIGATE WITH FULL CALL STACK VISIBILITY

codecentric AG

ALL THE DATA YOU NEED

codecentric AG

MANUAL WORK

- Manual load and scalability tests once an iteration

- Requires close to production configuration

codecentric AG

BASIC LOAD TESTING

codecentric AG

THE DEVOPS REVOLUTIONChapter Four

codecentric AG

DEVOPS THINKS DIFFERENT

- Agile process provides high quality

- Test environments are slow, and often not real

- Done features go to production every day

- Use real users for testing

- Planned rollbacks integral part

com

mon

s.w

ikim

ed

ia.o

rg/w

iki/Fi

le:B

eta

-bad

ge.s

vg

codecentric AG

DEVOPS AND PERFORMANCE

- In the cloud, the only real test is production

- Avoid premature optimization

- Requires great tools

flickr.com/photos/design-dog/1249337589

codecentric AG

HIGH PERFORMANCE APPS GO LIVE EVERY DAY

Epilogue

codecentric AG

- Non-Functional Requirements are known and taken care of

- Performance is monitored in development

- Anomalies are taken care of

- Pre-Release sanity check is performed

- Up- and Downgrade is planned

- New version can go into production

- Productive software is examined around the clock

SUMMARY

flic

kr.c

om

/photo

s/re

dux/4

740529728