Embed Size (px)
Citation preview
11
COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
LIFECYCLE BY DESIGN
Enabling Collaboration across Development, Test and Production Teams
Wolfgang Gottesheim
22
Application Lifecycle Management…?Application Lifecycle Management…?
33
Application Lifecycle ManagementApplication Lifecycle Management
Application lifecycle management (ALM) is the product lifecycle management (governance, development, and maintenance) of application software. It encompasses requirements management, software architecture, computer programming, software testing, software maintenance, change management, project management, and release management.
Source: http://en.wikipedia.org/wiki/Application_lifecycle_managementaccessed Dec 16, 2013
44
The Application LifecycleThe Application Lifecycle
Source: http://www.teamquest.com/solutions/itil/application-management/
55
Project deliverablesProject deliverables
…and what you get…and what you get
What you set out to build…What you set out to build…
66
What we tried to do…What we tried to do…
• Update our Customer Self-Service Portal
77
Our (Business) RequirementsOur (Business) Requirements
• “Provide a modern user experience”
– aka “Get the Community from Web 1.0 to Web 2.0”
– New features: A JAXified editors, Popups when someone else is editing the same page, notifications about new comments, auto refresh in the background,…
88
What went wrong?
Our mistakes:
– We left PERFORMANCE out
– We thought about new features instead of thinking about the user
experience
99
1010
DesignDesign
• We wanted to continue using Confluence
– 3rd party software �major design decisions made elsewhere
– Extended by custom plugins � ensure compatibility
• Our mistake: assuming that resource usage will remain stable in the new version
1111
Build/Integrate/TestBuild/Integrate/Test
• Testing was focused on FUNCTIONALITY
– “Does the site work as expected?”
– Tests in small-scale staging environment � some findings disregarded because of environmental differences
• Our mistake: we didn’t test from the end-user perspective in a realistic environment
1212
DeployDeploy
• Actual rollout was documented and well-tested in staging environment
• Also part of deployment phase: User Training
– We thought the new features are so obvious that people will find them
• Our mistake: we didn’t make sure that features are accessible
1313
OperateOperate
• Mistakes we made earlier added up…
– Slow response times
– High resource usage (CPU, memory)
– Timeouts
– Crashes
– User complaints
• Blame it on IT?
1414
What problems did we see?
1515
Resource Pool ExhaustionResource Pool Exhaustion
1616
Compatibility Issues with custom pluginsCompatibility Issues with custom plugins
1717
Requesting too much data from DBRequesting too much data from DB
1818
Oversized PagesOversized Pages
17! JS Files – 1.7MB in Size
Useless Information!
Even might be a security risk!
1919
CachesCaches
62! Resources not cached
49! Resources with short expiration
2020
3rd Party Content3rd Party Content
2121
2222
OptimizeOptimize
• Monitor the impact of new features on server-side performance and end-user experience
• Monitor actual usage and consider tradeoff between performance and features
• Our success: after rollout, we were able to identify the bottlenecks and make necessary changes
2323
2424
Very “expensive” to work on these issuesVery “expensive” to work on these issues
~80% of problems
caused by ~20% patterns
YES we know this
BUT
2525
Lots of Problems that could have been avoidedLots of Problems that could have been avoided
• BUT WHY are we still dragging them along across the lifecycle – up into production?
2626
Requirements EXCLUDED in Stories / TasksRequirements EXCLUDED in Stories / Tasks
PerformancePerformance ScalabilityScalability
TestabilityTestability
DeployabilityDeployability
DeployabilityDeployability
2727
Test and Ops EXCLUDED from Agile ProcessTest and Ops EXCLUDED from Agile Process
Stand-UpsStand-Ups Sharing ToolsSharing Tools
FeedbackFeedback
2828
Automation EXCLUDESAutomation EXCLUDES
PerformancePerformance ScalabilityScalability
Shared ToolsShared Tools Automatic FeedbackAutomatic Feedback
2929
Measurement: Define KPIs accepted by all teamsMeasurement: Define KPIs accepted by all teams
# of SQL Executions
# of Log Lines
MBs / Uses
Time for Deployment
Time for Rollback
Response TimesPerf Test Code Coverage
3030
DevOps Automation in ActionDevOps Automation in Action
• Most problems can be identified in CI
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurchase OK
testSearch OK
Build 18 testPurchase FAILED
testSearch OK
Build 19 testPurchase OK
testSearch OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
75 0 230ms
3 1 68ms
Test Framework Results Architectural Data
We identified a regresesion
Problem solved
Lets look behind the
scenes
Exceptions probably reason
for failed testsProblem fixed but now we have an
architectural regression
Problem fixed but now we have an
architectural regression
Now we have the functional and
architectural confidence
3131
Performance Focus in Test AutomationPerformance Focus in Test Automation
Analyzing All Unit / Performance Tests
Analyzing Metrics Analyzing Metrics such as DB Exec
Count
Jump in DB Callsfrom one Build to the
next
3232
Performance Focus in Test AutomationPerformance Focus in Test Automation
Cross Impact of KPIs
3333
Performance Focus in Test AutomationPerformance Focus in Test Automation
Embed your Architectural Results
in Jenkins
3434
Performance Focus in Test AutomationPerformance Focus in Test Automation
Here is the difference!
Compare Build that shows BAD Behavior! With Build that shows GOOD Behavior!
3535
Performance Focus in Test AutomationPerformance Focus in Test Automation
CalculateUserStats is the new Plugin that causes
problems
36© 2011 Compuware Corporation — All Rights Reserved © 2011 Compuware Corporation — All Rights Reserved
36
Participate in Compuware
APM Discussion Forumsapmcommunity.compuware.com
Like us on Facebookfacebook.com/CompuwareAPM
Join our LinkedIn group
Compuware APM User Group
Follow us on Twittertwitter.com/CompuwareAPM
Read our Bloghttp://apmblog.compuware.com
Watch our Videos &
product Demosyoutube.com/Compuware
www.compuware.com/APM
Thank YouThank You
37© 2011 Compuware Corporation — All Rights Reserved © 2011 Compuware Corporation — All Rights Reserved
37COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
