What It Takes to Design High Assurance Applications

8/3/2019 What It Takes to Design High Assurance Applications

1/24

What It Takes To Design HighAssurance Applications

MotivationHigh assurance applications are those on which the entire organization depends

How Is a High Assurance Application Different?

building, deploying, and operatingthe processes that are used to design and build high assurance

applications are different than those that are used to build small departmentalapplications


2/24

led to a degradation in the quality of application designs in exchangefor rapid turnaround in functionality

Applications that are poorly designed internally degrade over time

symptoms that indicate the deficiencies

1. Our large applications take too long to make changes to,and are too costly to maintain. As a result, we are gettingpressure to 'buy instead of build' in order to save costs, butthat leads to operational rigitity.

2. Our outward-facing applications represent a risk, in terms ofsecurity, reliability, and assurance in general. We are notconfident that nothing serious will go wrong one day.

3. We would like to oursource some work, but are leery of therisks and complexities of doing so.

4. Our most complex internal mission-critical applications havechronic problems, yet we have invested a lot of moneyimproving them.

5. We are not sure what technologies to use, becausetechnologies change, and the landscape is so complex. Ourtechnical leads have not been able to provide a directionthat is compelling, short of always wanting the 'next greatest

thing'.6. Accounting rules often change, and we have had a hard

time supporting this while also supporting the requirements ofoperations. These two different sources of requirements seemto be in conflict.

Table 1: The Top Six Symptoms


3/24

What the Symptoms Indicate

Symptom: Non-Agility and Cost Of Enhancement!

many organizations that believe that they employ agilemethodologies find over time that their software project teams have become just asunresponsive as before an agile approach was used

new development staff do not trulyunderstand the application's design, and are unable to accurately assess the impact of

new requirements

Agile processes assume that developers collaborate

when an application progresses to amaintenance mode key people may leave the team

leave behind little useful documentation


4/24

Symptom: Lack Of Confidence In Design!

do not know how toapply unit testing judiciously and so the unit test suite itself becomes a significant cost

unit testing is not sufficient for a high assurance application

failure response must be specified at a detailed level as requirements

if users have specified that the application should be secure,reliable, and scalable, there needs to be a process that is acceptable to the users forassessing the security, the reliability, and the scalability of the delivered system

in-house experts will ignore risks because they are almost always engaged in

demanding operational tasks or crisis responses that require their full attention, and theyhave also learned that the business does not like to hear about risks, but prefers to hear


5/24

about new capabilities

Symptom: Inability To Outsource When Desired!

there must be arobust process for ensuring that adequate integration and end-to-end testing is done

before the outsourced component leaves the outsource environmentwell-developed acceptance test plantesting must not be confined to functionality, but must include the range of

conditions that are expected to occur during operation

Symptom: Chronic Operational Hickups!

Do not let your development teams tell you that chronic unexpected problems

are unavoidable even if the same problem never occurs twice


6/24

If programmersneed to respond to your application's failures, the application is surely built withoutadequate failure handling

reliability and failure handling both need to be specified at a requirementslevel, and both need to be explicitly addressed in an application's design

if the real problem is that the application code itself does not anticipate andhandle error conditions but merely terminates after writing an unintelligible message to alog, then chronic problems will continue

chronic problems can also be a sign of a degraded design

Developers often unfairlyp p p Ipisgchiigh igDur wgap

reliaop oigh igwn g i ' 'h zr' thci

pH@pisgch catipp faiHoAsihgoi hTsc 'i i @cH'zd

ifd apAp 'ghau plA p @csH

Dchi cn s tnD cW

'sdsc ' ? 'a


7/24

Architects need to be in the loop with regard to business plannin

Symptom: Difficulty Satisfying Both Operational and

Financial Requirements!

Business users increasing want to know the true impact ofdecisions, taking into account the full consequences tax and otherwise

the kinds of queries that areperformed by accounting functions are usually very different from those performed byoperations, leading to potentially incompatible requirements

Application architects need tounderstand the needs of each community equally, and have the opportunity to addressthose needs in a comprehensive manner


8/24

So What Does One Do?

software development practices and remedies

do not promote a particular software development methodologyInstead, we

take the approach of identifying the things that must be done to achieve higherassurance

Prevention Of Key-Person Dependenciesyour best staff are often the greatest

source of problems

guru-built systems run well until the key staff move onuntil the systems grow too large

it is a natural trait of themiracle worker personality to focus on handling crises,rather than on preventing crises

Such applications cannot beoperated at a high assurance level

Addresses: The maintenance s taff do

not truly understand theapplication's design, andtherefore are unable toassess the impact of newrequirements.

Key people have probably

moved to other assignments,and left behind little usefuldocumentation.


9/24

move the key-person individuals off of the maintenance of components as soon asthe components are built

Separation Of Duties, and Separation Of Environments

separation of duties separation of environments

a developer should noteven have access to deployment resources

separate departments that specialize in each lifecycle phase

test environment should not be able to access any resources in anyenvironment other than the test environment

user acceptance and production environments should not be able toaccess the development environment

Programmers should never be expected to write integration tests for their owncode

separate test suite designed by someone otherthan those who built the code complete coverage

tests for all non-functionalrequirementsSome

Addresses: Possible inadequate

processes for configuration,build and deployment.


10/24

Specify Non-Functional Requirements

The people who will manage, maintain, and secure your applications areimportant stakeholders they need tobe involved when requirements for an application are gathered

explicit requirements for security, reliability, andmanageability spelled out in a concrete manner

application design should explicitly address all of these requirements

require that applications detect and

respond to any kind of abnormal condition that can be expected to occur during thelifetime of the application

error messages that are intelligible tooperators

some

Independent Design-Level Reviews

Addresses: Inadequately expressed

requirements for reliabilityand failure handling. Possible overly complex

configuration andadministration design.Applications that aredifficult to manage andmonitor.


11/24

outside party review a design independently

design review is so

critically important for reliable systemstwo different perspectives:

(1) quality, and (2) requirements

review activities should be done outside of ameeting

more than one person understands thedesign

leads toincreased agility confidence assessing impact of changes and new features

original developers have moved on

Regression Testing Is Absolutely Critical

acceptance testing

not generally trained to test for non-functional requirements

Testing for error response, security, and manageability all require specialtechnical expertise test programs

Addresses: The des ign may employ too

little encapsulation offunction, with databaseaccesses s trewn everywhere.

Possible degraded design.Poor separation offunctionality, with businesslogic interspersed withinfrastructure code. Poorencapsulation of functionand database access.

Addresses: Inadequate integration test

suite.


12/24

run tests daily to maintain an application under development in a high quality working

stateconfidence in an

application's design

integration test suite also makes it possible to verify that emergency bug fixesmade by support personnel do not introduce other problems

exercises every major application interfacetransactions are invoked user

interface itself

Concrete Acceptance Criteria

acceptance criteriamake such non-concrete requirements easier tobound

Well-defined acceptance criteria are essential for any development work that isoutsourced

Addresses: Inadequate acceptance

testing process. Inadequate processes to

asses s and assure security,reliability, and scalability.


13/24

problems can beavoided by defining the environmental test conditions and acceptance criteria

Documentationwhat

howgood documentation of the broad base of anorganization's applications is a strategic asset

no more than isnecessary

intent design decisions

persistent business model

elements that should be addressed

Good documentation standards facilitate

information sharing during development, and are therefore not a burden

without having to tap the original developers

structural

processes that assure that design documentation is maintained alongwith the application documentation maintenance should be viewed asa critical deliverable, alongside the code developed andmaintained using the same processes as are used to develop the code

assure that as-built alwaysequals as-designed design reviews

Addresses: The maintenance s taff do

not truly understand theapplication's des ign, andtherefore are unable toasses s the impact of newrequirements.

Possible degraded des ign.Poor separation offunctionality, with bus iness

logic interspersed withinfrastructure code. Poorencapsulation of functionand database access.

The des ign may employ toolittle encapsulation offunction, with databaseaccesses strewn everywhere.


14/24

Business Model Quality

only as useful as the quality andcompleteness of the design

with relational databases it is possible to create schema and omit importantde-facto relationships between tables

The completeness of and accuracy of thedocumentation of the business model, as embodied inthe relational schema or object model, is by far the mostimportant aspect of any application design

leads to theembedding of logic about data relationships withinapplication code death knellfor an application's maintainability and agility of enhancement

Overly Complex Infrastructurenatural tendency

most talented software development teamto be inclined to develop infrastructure instead ofbusiness functionality

separate technology from business logic

often designed in such a way that they add flexibility at theexpense of design safety, and such designs have a tendency to greatly obscure how theapplication works and make it hard for new developers to understand.

important that a project manager communicate to the technicalarchitects that infrastructure should be scrutinized

Addresses : The maintenance staff do

not truly understand theapplication's design, andtherefore are unable toassess the impact of newrequirements.

Addresses : The maintenance staff do

not truly understand theapplication's design, andtherefore are unable to

assess the impact of newrequirements.


15/24

add clarity never circumventthe design safety mechanisms

Retain Agility

omissions

result inunmaintainability later on consequences for security,reliability, and the agility of enhancements by new teams

Agile projects can still have a controlled release process,with design reviews, full integration tests, formal user acceptance, and explicitly statedrequirements for security, manageability, and failure response

Educate Staff

how to design sound applications

Programmers arrive on the job

almost

Addresses: The requirements , design,

and development processesthemselves may employ to omuch overhead.

Addresses: Possible degraded design.

Poor separation offunctionality, with businesslogic interspersed withinfrastructure code. Poorencapsulation of functionand database access.


16/24

no knowledge about what makes an application maintainable, secure, manageable,and reliable

You cannot assume that your architects and programmers know how to achievethe goals of reliability, security, manageability, and maintainability without leadershipfrom management apply constant pressure to the technical staff

that these priorities are important

Project managers need to be educated in these aspects as well

The pressure to learn about and address thisconnection needs to be persistent and originate from a higher level than projectmanagement

tendency to gloss over issues that are invisible to or poorly understood by that enduser

real pressure to project

managers

Liaise Business and Technical Staff

Technical ITstaff are often very disconnected from the business area

If the IT staff understand the business better,their tendency to create technology will be replaced

with a tendency to use technology for the benefit of thebusiness

Includearchitects, on a regular basis

dialog, relationships, and shared visionbetween technical staff and the business

know the business processes inadvance of the start of a project

Addresses: Application architects do

not have a clear picture ofbusiness objectives.

Inadequate planning withregard to architecture andhow to meet businessobjectives.

Application architects to notequally understand theneeds of each community, orhave not had theopportunity to addressthose needs in acomprehensive manner.


17/24

offsite sessions joint whitepapers

dialog must be driven by a purpose visibleoutcome

Summary

An organization's critical software applications are those applications on whichthe organization depends for its bread and butter and its survival, built

and maintained with a higher level of diligence

cannot be added externally through operational practices designed into

symptoms are telltale large costs to add small featuresdifficulties in estimating LOE lack of confidence in theapplication hesitancy to outsource

operational hickups inability to justify new technology directionsdifficulty in satisfying competing business requirements

unsoundness can be traced to situations

maintenance staff do not truly understand theapplication's design; ittle useful documentation

design may be overly complexprocesses themselves may employ too

much overhead nadequate regression testing acceptancetesting inadequate processes to specify requirements

security, reliability, and manageability design hasdegraded over time inadequate processes forrelease management architects do not have a clear picture of


18/24

business objectives

changes to theorganizationprevent key-person dependencies separation of duties andenvironments requirementsfor reliability, security, manageability, and maintainability are adequately expressed

independent design-level reviewsregression testing

non-functional requirementsconcrete acceptance criteriadocumentation for software business data model

appropriate diligence and qualityinappropriate infrastructure

etaining agilityboth technical and management staff learn how to address reliability, manageability,security, and maintainability technical staff learn andunderstand the business

Achieving Change ToPromote Assurance


19/24

Appendix: Indications

Symptom!

Indications

Our large applications taketoo long to make changes to,and are too costly to maintain.

As a result, we are gettingpressure to buy instead ofbuild in order to save costs,but that leads to operational

rigidity.

Our outward-facingapplications represent a risk, interms of security, reliability, andassurance in general. We arenot confident that nothing

serious will go wrong one day.

We would like to outsourcesome work, but are leery of therisks and complexities of doingso.


20/24

Symptom!

Indications

Our most complex internal

mission-critical applicationshave chronic problems, yet wehave invested a lot of moneyimproving them.

We are not sure whattechnologies to use, becausetechnologies change, and thelandscape is so complex. Ourtechnical leads have not beenable to provide a direction thatis compelling, short of alwayswanting the 'next greatestthing'.

Accounting rules oftenchange, and we have had ahard time supporting this whilealso supporting the

requirements of operations.These two different sources of

requirements seem to be inconflict.

Table 2: What the Symptoms Indicate


21/24


22/24

Appendix: Solutions

Indication Solutions do not

truly understand theapplication's design

design decisions intent

little usefuldocumentation

too littleencapsulation of function

includingreviews by an outside party.

processesthemselves may employ toomuch overhead

but without giving upimportant controls, artifacts, andtesting.


23/24

Indication Solutions Inadequate integration test suite automated

including both the application

facade level and the user interfacelevel

Inadequate acceptance testingprocess

Inadequate processes to assessand assure security, reliability,and scalability

Inadequately expressedrequirements for reliability andfailure handling

degraded design

inadequate processesfor configuration, build anddeployment

separation of dutiesseparation of environments

Applications that aredifficult to manage and monitor

manageability


24/24

Indication Solutions architects do not

have a clear picture of businessobjectives

Inadequate planning with regardto architecture

architects to notequally understand the needs ofeach community nothad the opportunity to addressthose needs in a comprehensivemanner

Table 3: Solutions

Documents

What It Takes to Design High Assurance Applications