IIS Approach to Process Right-Sizing Lawrence Goldstein Southern California SPIN February 6, 2004

IIS Approach to Process Right-SizingLawrence GoldsteinSouthern California SPINFebruary 6, 2004

2

Agenda

• Northrop Grumman Internal Information Services–What is it?–Geographic scope–Some historic data on software process improvement

at IIS• IIS Process Right-Sizing

–General approach–Process size–Size-a-matic™–Tailoring options–Waivers

–Special cases–Maintenance–Web / RAD

–Summary

3

Who is Northrop Grumman IT IIS?

• Northrop Grumman is a $25 billion global defense company– Operating in 50 states and 25 countries– Approximately 120,000 employees– Defense product focus

– Advanced aircraft, shipbuilding and space technology

– Systems integration– Defense electronics– Information technology

• Northrop Grumman’s Internal Information Services (IIS) is the business unit responsible for providing the IT infrastructure to the rest of the corporation– Our only customer is the rest of Northrop

Grumman– Our mission is to provide strategic

information solutions and technologies which contribute to Northrop Grumman’s competitiveness

4

IIS Geographic Scope

IIS Locations – North America

Reston, VA

Sunnyvale, CA

Century City, CA

Hawthorne, CAEl Segundo, CA

Palmdale, CA

Torrance, CASan Pedro, CA

San Diego, CA

Norwalk, CT

Melbourne, FL

GA

Rolling Meadows, IL

Lake Charles, LA

Baltimore, MD

Albuquerque

Bethpage, NY

San Angelo, TX

Dallas, TX

Herndon, VA

Reading, MAToronto, ON

Chantilly, VA

Gaithersburg, MD

College Park, MD

Garland, TX

Goleta, CA Woodland Hills, CANorthridge, CA

Tempe, AZ

Salt Lake City, UT

Apopka, FL

Pascagoula, MSAvondale, LA

Charlottesville, VA

San Jose, CA

Enfield, NS

Ocean Springs, MS

Amherst, NY

Bellevue, NE Greenbelt, MD

Bethesda, MD

Stuart, FL

St. Augustine, FL

Newport News, VAPt. Mugu, CA

Azusa, CA

Huntsville, AL

Over 3,700 Employees

Redondo Beach

Fairfax, VA

Clearfield, UT Aurora, CO

Carson, CA

Fort Hood, TX

San Antonio, TX

Austin, TX

Wheeling, WV

Kettering, OH

Troy, MI

Colorado Springs, CO

San Bernardino, CA

Columbia, MD

Sierra Vista, AZ

5

IIS Process Improvement Path

1992 1994 1996 1998 2000 2002 2004 2006

CMMI L3

NISC*

Grumman*

Westinghouse*

Rolling Meadows*

Litton*

Newport News*

TRW*

Vought*

IIS Infrastructure(Operations, N/W, hardware, etc.)

L2CBA-IPI

L2CBA-IPI

L2CBA-IPI

L3CBA-IPI

* heritage organizations

6

IIS Process Right-SizingGeneral Approach

7

The Problem

Project Labor hours

Num

ber

of P

roje

cts

00

100,000+

8

The Problem

Project Labor hours

Num

ber

of P

roje

cts

00

100,000+

Maintenance

New Development

Enhancements

Data D

riven

Complex Algorithms3D Graphics

Systems of Systems

Business Systems

CO

TS In

tegr

atio

n

Engineering Systems

Manufacturing Systems

Quality Assurance Systems

Financia

l Syste

ms Mainframe

Distrib

uted

Web

PC Stan

dalo

ne

Classified

Unclassified

SAP

PeopleSoft

Lawson

9

The Problem

Project Labor hours

Num

ber

of P

roje

cts

00

100,000+

Maintenance

New Development

Enhancements

Data D

riven

Complex Algorithms3D Graphics

Systems of Systems

Business Systems

CO

TS In

tegr

atio

n

Engineering Systems

Manufacturing Systems

Quality Assurance Systems

Financia

l Syste

ms Mainframe

Distrib

uted

Web

PC Stan

dalo

ne

Classified

Unclassified

SAP

PeopleSoft

Lawson

10

IIS Process Tailoring Philosophy

• The main purpose behind the process is to protect the project from failure–Deliver the product on-time, agreed to cost,

agreed to quality• If less work is needed to create the product, then

less process should be needed to protect the project–Less to go wrong–Consequences of failure are lower

–High dollar value projects with little labor are treated as special cases

But …• The process also is needed for organizational

purposes–Some process is always required regardless of

amount of work to create the product

11

The Devil is in the Details

12

IIS General Approach

The project team determines whether the project is Minor, Small, Medium, or Large.

Minor:<151

Minor:<151

Small:151-1800

Small:151-1800

Medium:1801-7200

Medium:1801-7200

Large:>7200

Large:>7200

Adjust SizeAdjust Size

Size

13

Determine Project Size 1 of 2

• Determine and describe the effort estimating approach and data to be used and how it will be used–Others should be able to recreate

estimate from data provided• Estimate size of software work products• Calculate estimated effort

–Convert work product size to labor hours to produce the products

–Size any other project activities and convert them into labor hours

14

Determine Project Size 2 of 2

• Use Size-a-matic™–Effort Estimate

– Key-in total project effort estimate (hours)

–Size Adjustments– Evaluate risk levels for 6

key areas– Describe rationale for

levels selected

–Resolves the issue of a “large Minor” vs. a “small Medium”

15

Tailoring Building Blocks

• Regardless of development lifecycle chosen by a project, the same general sets of activities always have to be done

• The differences usually revolve around the ORDERING of those activities and the scale of those activities

• All lifecycles have to – Define requirements– Design solutions– Construct code– Discover defects– Etc.

• So why not define plug-and-play processes , tailorable by size, for the different project lifecycles to use?

• Then lifecycle definition becomes a case of pre-defined process ordering with minimal process development required

16

Tailor, Tailor, Tailor

Project size is used to identify the specific activities and deliverables required for each CMMISM Process Area involved in the project.

Minor

Small

Medium

Large

RM PP PMC M&A PPQA CM …•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

•Do

•Don’t do

17

And Then There Are Waivers

• Waivers provide the option for tailoring “outside of process boundaries”–Waive compliance with required process–Based on a substantiated business case which

requires deviation from the standard–Approved by someone high enough in the

management chain to understand the implications• If you define your process set correctly, waivers

should rarely be needed–All the common tailoring should be pre-defined,

balanced, with the trade-offs well understood and accepted

–Waivers should definitely be the exception rather than the rule

18

IIS Process Right-SizingSpecial Cases

19

Special Cases

• Special circumstances require special methods–Maintenance of production systems–Web / Rapid Application Development (RAD)

• A separate lifecycle methodology is applied–Pre-tailored to address the special circumstances

with just the right amount of process in all the right places

–Project sizing is not used to determine tailoring because the lifecycle is pre-tailored

–Less project-by-project options means simpler to plan

Process Costs Risk of Failure

20

2003 Distribution by Project Size

MINOR7%

SMALL19%

MEDIUM9%

LARGE10%

UMBRELLA46%

WADM9%

21

Maintenance Umbrella Projects 1 of 4

• Covers “level of effort” maintenance work efforts– Application systems in production– Supported by pre-allocated labor pool of

experts– Budgeted on an annual basis

• Each new customer request is either– New or changed requirement– Reported system defect– Service request (data load, table change,

etc.)• Groups minor-sized work efforts occurring

within a specified period of time, range of hours, and/or cost amount– Annual renewal– Pre-determined fixed cost (e.g. 500 hours)– No upper limit on aggregate work covered,

only on each discrete work effort• Special maintenance umbrella project plan

template

22


• Addresses a collection of minor work efforts on production systems which are related in one or more ways–Common customer–Common system–Common hardware–Common support organization

23


• Allows for the one-time definition of a standard approach pertaining to all of the related work efforts–Common roles and responsibilities–Common PPQA approach–Common CM approach–Common RM approach–Common RSKM approach–Etc.

• Requires more process than any single minor work effort, but less than if all work performed were treated as individual minor projects–Recognizes that more process is needed to protect

production software than is required for minor work efforts

–Better protection for less cost

24


• Work requests which size as larger than Minor are spun off as separate projects–Separate project plan–Separate funding–Separate QA–But can still use common CM, RSKM, etc.

Customer Request

Sizing Process

Minor

Not Minor

25

Eat at Joe’s

Reg. Double 1/3 lb

Hamburger

Soy-burger

Small Medium Large

Soda (coke, orange, root beer, sprite)

Milkshake (chocolate, vanilla)

Malted (chocolate, vanilla, peanut butter)

$1.25 $1.75 $2.50

$1.05 $1.50 $2.00

$ .75 $ 1.25 $ 1.85

$ 1.75 $ 2.25 $ 3.00

$ 2.25 $ 2.75 $ 3.50

26

Jo’s Estimating Table

Simple Average Complex

DB Segment Change

DB Segment Add

Screen Change

Screen Add

Screen Delete

10 hrs 15 hrs 25 hrs





DB Segment Delete 10 hrs 12 hrs 15 hrs

< 5 elmts < 10 elmts < 20 elmts

27

Umbrella Special Tailoring Example

Standard process–Must perform a Post-Implementation Evaluation (PIE)

n days following implementation

Umbrella process–Allows for the “bundling” of multiple small changes

into quarterly PIEs instead–Major changes are still be evaluated separately

28

Web Application Development

Unique set of problems and opportunities

Q: How do you marry structured process to the agile philosophy?

A: Very carefully!!

29

Basic Terminology

• WADM™ – Web Application Development Methodology– A lightweight, rapid application development process for use in

web application development, which is SW CMM / CMMISM level 3 compliant

• Lightweight – Minimal amount of intermediate “stuff”

– Steps– Artifacts– Reviews

• Rapid application development– A series of short, incremental development cycles

• Web application– An application, possibly linked to one or more databases, which

uses the web as the user interface• SW CMM / CMMISM level 3 compliant software development

methodology – A structured set of processes and procedures, which has the

goal of reliably producing high quality software and other work products, on schedule, and within cost.

30

WADM™ Overview 1 of 2

• The WADM™ is a full lifecycle process for web development projects, and for non-web development projects having similar characteristics

• It calls for a high level of customer involvement, short development cycles, and rapid development.

• General criteria for using the lifecycle– A high degree of customer involvement– Sufficient size and work scope to allow functionality to be

allocated to progressive production releases– Simple design– A requirement for rapid delivery of functioning releases (get

something up and running quickly)– The overall program can be compartmentalized into stand-alone

projects with small teams of developers

31

WADM™ Overview 2 of 2

• The WADM™ is not appropriate if the project– Has high algorithmic

complexity– Involves integrated

database design– Is solely maintenance– Has low customer

involvement– Involves conversion or

platform migration – Must be implemented as a

whole, at one time– Cannot be divided into

coherent functional units that can be implemented in successive releases

– Plans to subcontract all or a portion of the work

Read the label!

32

WADM™ Key Elements

• Customer Focus– Customer has ownership of, and a high degree of visibility into, the

project.– High level of customer commitment and involvement in the

development team. – Customer can add, delete, modify and/or reprioritize functions as the

application is being developed. – Meets customer’s real demand due to the customer’s degree of

influence and control over the development effort.• Short Time Span

– The development cycle is divided into successive, short-duration builds.– The first build and each successive build are usable.

• Low Cost– No unnecessary functions are allowed. Functions that are not required

are not developed. – Project documentation and reviews are minimal.– Works with virtual teams.

• Produces High Quality Product– Product has minimum number of defects. – Testing and defect prevention are integral to the process.

33

WADM™ Structure 1 of 5

Development Cycles

ProposalPhase

StartupPhase

Kick Off Activities•Project Plan•Form Team Activities•Kick Off Meeting

Proposal•SOW

• XXX• XXX

•Budget• XXX• XXX

•Schedule• X-----

• X----- Startup Meeting• Elicit Initial Set of

Use Cases and Constraints

• Initial Release Plan

Release 1.0

Releases of Usable Software

DevelopmentCycle 1

Release 2.0DevelopmentCycle 2

Release n

Release n-1

DevelopmentCycle n

DevelopmentCycle n-1

Additional Development Cycles

34


AcceptanceTesting

AcceptanceTest

Development

CompletedBuild

DevelopmentCycle

Design CyclePlanningMeeting

DesignCycle

Construction

StandupMeeting

ConstructionCycle

ThePlanning

Game

ProposalPhase

StartupActivities

35


AcceptanceTesting

AcceptanceTest

Development

CompletedBuild

DevelopmentCycle


DesignCycle

Construction

StandupMeeting

ConstructionCycle

ThePlanning

Game

ProposalPhase

StartupActivities

AcceptanceTesting

AcceptanceTest

Development

CompletedBuild

AcceptanceTesting

AcceptanceTest

Development

CompletedBuild

DevelopmentCycleDevelopmentCycle


DesignCycle


DesignCycle

Construction

StandupMeeting

ConstructionCycle

Construction

StandupMeeting

ConstructionCycle

ThePlanning

Game

ThePlanning

Game

ProposalPhase

StartupActivitiesStartup

Activities

• Three nested cycles of iterative activities:– The Development Cycle– The Design Cycle

– Enclosed within the Development Cycle

– The Construction Cycle– Enclosed within the Design Cycle

• Every cycle in the WADM™ begins with a planning / management meeting followed by the work activities of the cycle.

• WADM™ projects are typically composed of several sequential Development Cycles

• Project Management, Risk Management, Issue Management, Requirements Management, Configuration Management, and Measurements & Analysis activities occur throughout

36


• Development Cycle– Objectives

– To ensure that the customer needs are understood by all parties (identify customer requirements)

– To ensure that the customer needs are prioritized by the customer

– To ensure that the customer needs are met (validate and verify the requirements)

– Duration: ~ 1 Month– no Development Cycle

should exceed six weeks in duration

– Made Up of:– 1 Planning Game– 3-5 Design Cycles

• Design Cycle– Objectives

– To design an application capability to meet a selected subset of the customer requirements

– Duration: ~1 Week– Made Up of:

– 1 Design Cycle Planning Meeting

– 4-5 Construction Cycles

1 Cycle ~Every Week

1 Cycle ~Every Month

37


• Construction Cycle– Objective

– Code and Unit Test Modules of the Application of the Design Cycle

– Duration: Daily– Made Up of:

– Daily standup meetings– Daily construction activities– Continuous integration

1 CyclePerDay

38

Planning Game’s 5 Standard Moves

IdentifyUse Cases Estimate

Use CasesPrioritizeUse Cases

PlanIterations Commit to

ReleasePlanBusiness

Team Members(The Customer)

DevelopmentTeam Members(The Developer)

BusinessTeam Members(The Customer)

DevelopmentTeam Members(The Developer)

ProjectGoals

ProjectConstraints

Complexity

Risks

BusinessNeeds

LoadingFactor

ReleasePlan

1

2

3

4

5

39

WADM™ Plow-Back

AcceptanceTesting

AcceptanceTest

Development

CompletedBuild

DevelopmentCycle


DesignCycle

Construction

StandupMeeting

ConstructionCycle

ThePlanning

Game

ProposalPhase

StartupActivities

New and Deferred Use Cases

40

WADM™ QA

AcceptanceTesting

AcceptanceTest

Development

CompletedBuild

DevelopmentCycle


DesignCycle

Construction

StandupMeeting

ConstructionCycle

ThePlanning

Game

ProposalPhase

StartupActivities

QAProject Plan &

Schedule Review

QAConsultatio

n

SQAEnd of Cycle

Review

SQAEnd of Cycle

Review

SQAEnd of Cycle

Review

QAEnd of Cycle

Reviews

41

WADM™ Defect Management

• defect avoidance –A practice where the focus is to minimize the number

of defects created during the construction of a work product.

– It avoids the creation of the defect in the first place. • defect detection and resolution

–A post-construction activity to identify and disposition the defects which were created during construction.

– It identifies and resolves defects that were created despite the project’s defect avoidance practices.

• defect management approach –Defines both the defect avoidance and the detection

& resolution processes to be followed by the project team.

– It also defines the criteria for deciding which processes are to be applied for each work task.

AGILE

CMMIS

MWADM™

42

WADM™ Minimum Deliverables 1 of 3

Phase Minimum Required Deliverable/Work ProductFeasibility(Proposal)

High Level Statement of WorkHigh Level Estimate

Start Up Activities

Project Plan*, includingStatement of WorkEstimated CostHigh Level ScheduleDesign and Code StandardsTeam Operating Rules

Initial Use Case-Based Requirements Document*Initial Use CasesDesign Constraint Based Requirements

Initial Release Plan (found in Use Case Based Requirements Document*)Detailed Loaded ScheduleCommitted to by Business Team Members and Developers

Initial List of Risks*Initial List of Issues*

* WADM™ Template required

43


Phase Minimum Required Deliverable/Work ProductDevelopment Cycle: Planning Game

Updated Risks*, Issues*, Action Items*Use Case Based Requirements Document*

Use Cases, Estimates, and Design Constraint Based RequirementsUpdated Release PlanAcceptance Test Scenarios (Inspection, Procedural, & Automated)

Use Case Estimate and Rationale Document*

Development Cycle:Design and Construction Sub-Cycles

Updated Risks*, Issues*, Action Items*Design

CRC CardsUML Graphics

Unit Test PlansUnit Test resultsConfiguration Management

Code migrated from Working to Released to Submitted for Acceptance Testing


44


Phase Minimum Required Deliverable/Work ProductDevelopment Cycle:Test

Updated Risks*, Issues*, Action Items*Test Class UpdatesTest Procedure Document UpdatesUse Case Test Scenario Result Summary Log*

End of Development Cycle

Approved SoftwareConfiguration Management

Code migrated from Submitted to Approved (in production)Updated Use Case Based Requirements Document*, if changedUpdated Release Plan, if changed


45

WADM™ Process Tailoring 1 of 2

• WADM™ is a pre-tailored version of the IIS process set – Specific to the rapid development / agile environment– All team members need to be trained in this methodology

• Normal processes apply . . . except where superceded explicitly by WADM™ processes– Project planning checklist does not apply– Size-a-matic™ tool does not apply

• Prescribed templates– Boilerplate text must be followed as is – Required WADM™ Templates

– Project Plan – Use Case Based Requirements Document – Risk Management Log – Issue Management Log – Use Case Estimate & Rationale Document – Use Case Test Scenario Results Summary Log– Action Tracking Log

46

WADM™ Process Tailoring 2 of 2

• There is very little additional tailoring that the individual project can perform– Decision on the use of CRC cards, UML, or both– Choice of defect avoidance alternatives– Choice of defect detection approaches– Identification of any project specific standards and tools– Production implementation approach and deployment strategy

• The WADM™ is agile, but not flexible– Most flexibility has been tailored out of the WADM™

– To decrease documentation needs– To ensure SW-CMM / CMMISM conformance

– The WADM™ must be used as is

47

IIS Process Right-SizingSummary

48

Experiment

49

Summary 1 of 6

• One size does not fit all– It never has, and never will!–And even if it did, different situations call for different

dress …

50

Summary 2 of 6

51

Summary 3 of 6

• Look for the Rosetta Stone to your environment–Analyze your project types–Maybe effort is not the driver for your process

tailoring decisions, but something is …–Develop varied and flexible approaches that match

your business needs

52

Summary 4 of 6

Tame that SW-CMM / CMMISM monster–Turn it into

something that your programmers and engineers will relate to

53

Summary 5 of 6

• Apply the intelligence principle to your process definitions–What is the purpose of all this?–You don’t have to follow the SW-CMM or the CMMI

slavishly at the sub-practice level … meet the intent– If you have families of projects, customize your

process definitions to make planning them easier• Allow your projects to apply the intelligence

principle as well–Keep your eyes on the prize: better run and more

predictable projects that meet your customers needs–Avoid process for process sake

54

Summary 6 of 6

Trust in the Force …but make it easy to comply!

Documents

IIS Approach to Process Right-Sizing Lawrence Goldstein Southern California SPIN February 6, 2004