Embed Size (px)
DESCRIPTION
Apache CloudStack (ACS) is one of the best-of-breed in the current IaaS landscape. However, despite its high functional quality, its technical quality (the one you observe when you look "under the hood") is bellow industry average. Miguel Ferreira was initiated in ACS development in the early days of 2014 and presents, in this talk, his perspective, as a newcomer, on how it felt to him like he was working on a skyscraper in the early days of the XX century, without safety-net, nor rope. Together with his former colleague Dennis Bijlsma (SIG), they dived into the code of ACS to find evidence to support a change in the way ACS is maintained, that combines the best of modern software engineering practices and cutting-edge innovation. Miguel proposes that the ACS community take charge of keeping to the highest standards in software technical quality, while continuing to foster creativity.
Citation preview
CCCE 2014
CCCE – 2014 – Budapest
Talk: Miguel Ferreira (SBP)
CloudStack evaluation: Dennis Bijlsma (SIG)
WORKING ON A SKYSCRAPER IN THE EARLY YEARS OF THE XX CENTURY
CCCE 2014
It’s about how I’ve experienced CloudStack development
THIS TALK
CCCE 2014
Portuguese
Mission Critical Engineer at Schuberg Philis
Interests:• Software maintainability
• Metrics-driven engineering
• Testing
WHO AM I
Studied systems and software engineering
Computer science research
Learned to see code as mathematical structures
Focus on building the right software
Real world software was not so mathematical
Software is a mess of building blocks
/legacy
Focus on building software right
Tremendous mental shift!
Code as building blocks
Size
ComplexityDuplication
Coupling
Categorize & Compare
CloudStack
CCCE 2014
Maintainable software is made of tiny blocks of code
Simple blocks, with meaningful names
That blend in together to create a consistent picture
Each piece is independently testable
WHAT SOFTWARE TAUGHT ME
CCCE 2014
Maintainability enables all other software quality characteristics
Security, Reliability, Performance, Portability, …
MAINTAINABILITY IS THE ENABLER
CCCE 2014
Changes often
Changes fast
MAINTAINABLE SOFTWARE
by Arthur John Picton, source http://www.flickr.com/, licensed under CC BY 2.0
CCCE 2014
The single most effective feedback loop
Without it the risk of continuously grooming
code is prohibitive
UNIT-TESTS ENABLE MAINTAINABILITY
System
Integration
Unit
Wasn’t this about CloudStack?
Healthcare domain
Mission critical cloud (with CloudStack)
CCCE 2014
What kind of software system is it?
CLOUDSTACK
“CloudStack is an open source software
platform that pools computing resources to build
public, private, and hybrid Infrastructure as a
Service (IaaS) clouds.”
CCCE 2014
CloudStack coordinates the work of multiple computing systems in order to offer computing
resources in a seemingly integrated fashion
What kind of systems are we talking about?• Hypervisors
• Storage devices
• Network devices
IN OTHER WORDS
Pretty complex stuff
CCCE 2014
Excitement to work on such an important piece of software
It must be really good, given the adoption
MY FIRST REACTION
CCCE 2014
Excitement to work on such an important piece of software
It must be really good, given the adoption
CloudStack is “the Bomb”!!
MY FIRST REACTION
CCCE 2014
Code is cluttered• Poor conventions (e.g. naming: ‘_*’, ‘s_*’)
• Commented out code
Poor domain model• Long methods
• “God classes”
• Duplicated objects
• “Stringly typed” methods
Poor error handling• “Pokémon” exception handling
Build system adoption was left halfway
THEN I LOOKED UNDER THE HOOD
CCCE 2014
Code is cluttered• Poor conventions (e.g. naming: ‘_*’, ‘s_*’)
• Commented out code
Poor domain model• Long methods
• “God classes”
• Duplicated objects
• “Stringly typed” methods
Poor error handling• “Pokémon” exception handling
Build system adoption was left halfway
THEN I LOOKED UNDER THE HOOD
Barely any unit-testing
How can that be?
Code is my work environment
CCCE 2014
Is not a safe work environment
The feedback loop is way too long
It requires extensive amount of debugging
It lacks unit-tests (which keep me sane)
CLOUDSTACK WORK ENVIRONMENT
CCCE 2014
CloudStack code is tightly coupled to specific hardware devices
It has a database
Plug-in architecture
Distributed development teams
TESTING CHALLENGES
CCCE 2014
There are other plug-in system out there
Lets look at a study of test practices within the Eclipse project
HOW DO YOU TEST SUCH A SYSTEM?
CCCE 2014
Test responsibilities• “Tester and developer, that’s one person. (…)”
Unit testing• “We have unit testing, and that’s where we put the main effort”
• “Ultimately, unit tests are our best friends, and everything else is already
difficult.”
• “What cannot be encapsulated is not tested.”
Influence of the plug-in architecture• “The small glue code (..), that’s not tested, because it is just hard to test
that. And for these untested glue code parts we had the most bugs.”
TEST PRACTICES IN ECLIPSE
CCCE 2014
Why doesn’t it have more unit tests?
GETTING BACK TO CLOUDSTACK
The code does not lend itself to
be unit-tested!
CCCE 2014
According to model used for certifying software maintainability
Focus on simple low-level source code metrics
Aggregation of metrics to system level respecting statistical properties
SOURCE CODE EVALUATION
CCCE 2014
MAINTAINABILITY RATING
http://www.sig.eu
CCCE 2014
VOLUME
Java 670 KLOC
JavaScript 70 KLOC
C# 44 KLOC
SQL 16 KLOC
CCCE 2014
DUPLICATION
Not Duplicated
Duplicated
CCCE 2014
UNIT SIZE
1 - 20
21 - 50
51 - 100
101 +
CCCE 2014
MODULE COUPLING
0 - 10
11 - 20
21 - 50
51 +
CCCE 2014
HOW DOES THIS TIE IN WITH UNIT-TESTING?
CCCE 2014
LET’S WRITE A UNIT TEST TOGETHER
CCCE 2014
Low complexity (barely no branching at all)
3 blocks• Setup: define the input
• Execution: call the method under test
• Verification: check expectations
WHAT DOES A UNIT-TEST LOOK LIKE
CCCE 2014
com.cloud.network.NetworkServiceImpl
updateGuestNetwork(…)• 11 parameters,
• 286 lines of code
• 15 blocks of code (with header comments)
Unit test• Setup: build/mock/abstract 11 input parameters (if you consider that 10 can be null, combinations = 2^10!)
• Execution: call updateGuestNetwork(…)
• Verification: what to expect from a 200+ line long method?
UPDATE GUEST NETWORK
Easy answer: an updated guest network!
CCCE 2014
In which state was it? Was it a system network? Was it any other kind of network?
Did it already have a name? Display name? Custom ID?
Did the network offering change?
Changing from VPC to non-VPC?
Network domain, IP reservation, …
Update life-cycle: shutdown all elements, make the update, turn back on
UPDATE GUEST NETWORK
CCCE 2014
I have some ideas…
NOW WHAT?
CCCE 2014
There must be something the community can do
Wants:• Better code, more tests, happier developers
Don’t wants:• Strangle innovation, chase current developers away
APACHE IS ALL ABOUT THE COMMUNITY
CCCE 2014
Popular in several open-source projects
Allows for cutting-edge innovation
Allows for better structure and quality control
LTS MODEL
Freeze the current
master branch
Freeze the current
master branch
Automate code
quality checks
Refactor on a need-
to basis
Development as
usual in innovation
branches
When releasing in
LTS, code has to
meet quality standard
At some point, swap
devs between the
two teams
CCCE 2014
Freeze the current release branch
Automate code quality standard• Measure unit size, complexity, coupling, duplication, etc.
• Measure unit-testing coverage
Keep the code as is, but• Every time the code is touched it is refactored according to the defined quality standard
• No new code gets in the LTS branch without meeting the tech quality requirements
Maintain two groups of developers (LTS and Innovation)• As time progresses swap people between the two
HOW TO MAKE IT WORK
CCCE 2014
CloudStack is functionally quite good, but the code is not keeping up
Feedback loops for developers are way too long
More and better unit testing will help to increase quality and developer satisfaction
Only the CloudStack community has the power to change the game
Schuberg Philis is more than interested in helping
SUMMARY
Thank you!
CCCE 2014
Skyscraper construction work photos by Charles C. Ebbets, source http://nowyorkcity.com/, © All rights reserved
Photo (and sand castle!) on slides 5 and 6, by Calvin Seibert, source http://www.flickr.com, © All rights reserved
Photo on slide 7, by Pedro Moura Pinheiro, source http://www.flickr.com, licensed under CC BY 2.0
Photo on slides 8 to 10, by Benjamin Esham, source http://www.flickr.com/, licensed under CC BY 2.0
Photo on slides 12 to 14, source http://legoquestkids.blogspot.nl/, © All rights reserved
Photo on slide 19, by Arthur John Picton, source http://www.flickr.com/, licensed under CC BY 2.0
Photo on slide 10, by Steven Zucker, source http://www.flickr.com/, licensed under CC BY 2.0
Photo on slide 22, by Murray Barnes, source http://www.flickr.com/, licensed under CC BY 2.0
Photo on slides 27 and 28, by Kristian, source http://www.flickr.com/, licensed under CC BY 2.0
Photo on slides 29 and 30, by Ben Wojdyla source http://jalopnik.com/5182381/adventures-with-wildlife-surprise-underhood-opossum-nest, © All rights reserved
CREDITS
CCCE 2014
Miguel Ferreira, SBP, [email protected], @miguel_f
Dennis Bijlsma, SIG, [email protected]
Ping us!
CONTACTS
