IT DOESN'T DO WHAT YOU THINK ITDOES
A Survey of Strategies for Gaining Confidence in Applicationsand Systems
Justin Stoller@juststoller
YOUR TITLE NEEDS A TL;DRway to go...
So what do we mean by "Gaining Confidence" and whyshould you care?
First that we have a clear idea of what we are trying to gainconfidence in?
What is the value of our software?
What are the acceptance criteria?
InputsOutputsPerformance CharacteristicsVisual Style
Does it meet the acceptance criteria?
What are the Rewards?What are the Costs?Where are the Risks?
A Few of the Topics Out of Scope
Organization & CommunicationArchitecture & Design
The Presentation
1. We will start with Language Paradigms and how theyaffect our abilities farther up the stack
2. Using Language constructs to perform both static analysis(type systems) and dynamic analysis (error handling)
3. Working our way up the stack to explicit low level tests4. And farther up to higher level integration tests5. We will look at combining what we've seen above into a
simulation testing and the emergent behaviors that arefound with these
6. And finally continuing the analysis from simulation intoproduction via monitoring
You forgot to say why should care...
Language Paradigms
Imperative vs DeclarativeLow Level vs High Level
Procedural
Type Systems that Model the Components of a ComputerTests assert the state of the worldError Handling by way of return codesGoto like statements
Object Oriented
Type Systems that Model the Application DomainTests assert the state of an objectTests assert the passing of messagesMore reliance on type systemsLimit exceptional behaviors to the boundaries of thesystem
Functional
Type Systems that Model the Application DomainTests assert the state of a returned data structureTests assert the distribution returned data structuresMore reliance on type systems
Logical/Relational
Type Systems as Theorem ProversTests assert the end state of the application
Testing Layers
A single unit test of logic: conditional, etc
Simple units of logic require simple tests
However low level tests tend to be very tied toimplementations
Good design facility but rarely good at ensuring quality
Often written in the language of the software and containsthe pros and cons of the software's language
The behaviors of Integrated Units
Tests are tied to behaviors of the systems
Increasing complexity of tests and prerequisites
Requires decent design, good at gaining confidence
The System as a Whole
What properties does it display
Analysis of emergent behaviors
A wide ranging spectrum
Higher level languages start farther up the level of strategies
The performance characteristics of the tests determine howoften they are ran
Higher level tests with simple ways to instantiate themprovide more value as ways to gain confidence in the
software.
Novelty in combining concepts
Instrumented or "monitored" development environments
Recommended
