Why Use TRIMS? The Importance of the Technical Risk Identification and Mitigation System

Why Use TRIMS?The Importance of the Technical Risk Identification and Mitigation System

Presented By:

Ralph SickingerBMP Center of Excellence

July 2005

Anne Marie T. SuPrise, Ph.D.Director

BMP Center of [email protected]

301.405.9990

www.bmpcoe.org Why Use TRIMS? 2

Do You Know This Person?

It is halfway through year 1 of the All-Purpose Cruise Missile (APCM) program, and “Pete the Program Manager” finds himself facing a lot of decisions. Time is money, and he wants this development program to be completed as soon as possible (under three years), so that the cruise missile can be fielded and help keep the world safe from tyranny. What can he do to shorten the schedule?

Over the next 6 months, Pete will make the following decisions:

• Eliminate the Design Reference Mission Profile (DRMP) – The mission goals for this cruise missile are pretty straightforward and well understood, so the time spent developing and documenting the DRMP would be a complete waste.

• The design engineers want the body of the cruise missile to be fabricated from a new high-tech plastic-composite which can be easily formed; this should result in less weight and cheaper production costs. Pete approves the design.

• In order to keep personnel costs down, Pete specifies a minimum number of personnel in the design contract: mostly design engineers and few admin/support staff.

Pete completes the design phase 3 months ahead of schedule, so he’s off to a good start!

(Or is he?)


A Look Into The Future

What if you had a crystal ball, and could see into the future? If Pete could do this, he would see that it is 5 years later, the APCM program is two years overdue, and a few hundred million dollars over budget. How did this happen? Pete’s “future self” explains what happened:

“We finished the design phase ok, and then tried to manufacture some initial prototypes. We lost a couple of months trying to get the hang of producing plastic missile bodies, but the design engineers got pretty good at tweaking the machines every time they produced a missile shell. Because the plastic shells can’t really be bolted together, the design called for the two halves to be glued together with adhesive. Most of the prototypes worked well in testing, but it was kind of a pain to try to separate the two halves when a missile failed to launch, in order to try to figure out what was wrong. Still, most of the missiles worked, and we didn’t want to lose any more time, so we went ahead to production. That’s when the problems really started! We wanted an initial production run of 300 missiles, but when the contractor ran the line at full speed, the plastic forming machines would start to gum up after 25 missiles; they would jam, causing us to shut the entire line down while the machines were cleaned. And when the machines didn’t jam, we had problems manufacturing the shells themselves! Sometimes the shell would loosen itself from the vacuum assembly before it had completely cooled, leaving a slight indentation in the body. Some of these deformed shells didn’t have enough room for the electronics or the warheads to fit properly and the shell would have to be thrown out, or the body would be forced together anyway, damaging the electronics inside! Eventually we figured out what was happening, and started screening the shells before they went into assembly; but we’re still discarding 10% of the shells, which has slowed us down and increased the cost of each missile.”


A Look Into The Future

“Things didn’t get any better when we deployed the missile. The same missiles which were 99% reliable in testing, only launched correctly 60% of the time in the field. And, since the missile bodies were sealed, the depots had no choice but to return them to us. After breaking several dozen of them open and inspecting them, we found lots of cracked solder joints and loose circuit cards. This didn’t make any sense at first since the contractor has a QA team that inspects every aspect of the missile as it’s being assembled, to make sure that every one that goes out is working perfectly. We finally got some failure-analysis experts in to diagnose the problem. That’s when we learned that out in the field, our missiles were being subjected to lots of bumps and jolts during transport as the vehicles they were in hit pot-holes, had to drive off-road, went over rubble, etc. When we asked the design engineers about this, they admitted that the subject of “rugged transport” hadn’t ever come up. They were able to modify the design so that the electronics are better protected during transport, but that made the missile a little bigger, so the production line had to be readjusted, and of course the new missiles no longer fit in the originally-designed shipping containers, so we have a warehouse of those just gathering dust...”

Pete is jolted back to the present. “Hmmm... maybe I should revisit those decisions I just made.”

Then a new thought strikes him: “If I missed these things, I wonder what else I’ve missed...”


In Summary

• Program is 2 years behind schedule (and counting)

• Program is $250+ Million over budget

• Reasons (Testing):• Initial testing suffered delays because of new plastics technology• Difficult to analyze test failures

• Reasons (Production):• Fabrication issues as a result of using unfamiliar technology• Delays caused by immature production process• Production errors result in increased delays, cost, waste• Production problems also create latent faults which surface during deployment

• Reasons (Deployment):• Failures in the field because of rugged deployment conditions• Storage/Transport induced failures necessitated redesign

5 Years Later...5 Years Later...


• Defense Science Board Studies Indicate that 80% of all Programs that Fail, Fail for Engineering not Technology Reasons

• Example: N43 Reports that Over 65% of all Fleet Reliability Problems are Due to Inadequate Design

• February 1998 GAO Report to the Subcommittee on Acquisition & Technology, Committee on Armed Services, U. S. Senate

• Since 1975 Schedule Overruns Average 24%• Weapon Systems Overruns Cost 20% - 40%

• March 2005 GAO Report:• Average Cost Increase Will be 28.5% in Only 4 Years• Reason: “Armed services order hardware with unproven technology,

with too little time allowed for perfecting that technology before contractors attempt to insert it into weapons systems...”

• With Acquisition Reform and Performance Specifications, We Must Monitor the Engineering Process to Understand Program Risks

Some Things To Consider

Do NotNot Get Technology Tunnel Vision


ScheduleCost

TechnicalIssues

Process

Issues

ReactiveReactiveProactiveProactive

Why Process-Based Risk Management?

Managing Risk Using Cost & Schedule Data Is Like Driving Down The Highway With Your Eyes In The Rear-View Mirror

Time

• Cost overruns and schedule slippages are a result of technical issues or problems

• Most technical issues are the result of engineering processes that either aren’t implemented, aren’t well understood, or are out-of-tolerance


Process Risks Problems

Product Risks

T1 T2 T3 T4 T5 T6

Evaluating Process Risk Permits More

Timely & Efficient Mitigation

“A ‘Problem’ Is A Risk Whose Time Has Come”


Time

Cos

t to

Miti

gate

Design Test Production

$1

$100

$10

Cost To Mitigate Increases Exponentially


Process Risks

Product Risks

Problems

Risk Management or Damage Control?

Cost To Mitigate Increases Exponentially

Time

Cos

t to

Miti

gate

Design Test Production


• Continuous Insight of Systems Engineering Process is Key

• Many Risk Management Actions are Problem Chasing Activities

• Projects Become Schedule-Driven

• People Become Over-Tasked

• Communications Become Clogged

• Product Quality Suffers

• Configuration Management Becomes Impossible

What We Have Found


• TRIMS (Technical Risk Identification & Mitigation System): A Methodology for Doing Knowledge-Based, Process-Oriented Risk Management

• Includes Set of Software Tools to Help Implement the Methodology (Navy Owned, Fixed Revision Cycle)

• Outgrowth of DoD 4245.7-M and NAVSO P-6071 Templates

• Fully Compatible with BMPCOE Combined Government / Industry Systems Engineering Model

• Fully Supported: Training Classes, Computer-Based Training,On-line Help, 24-hour Help Desk

What Is TRIMS?

Process Based And Predictive


TRIMS Proactive Methodology

TRIMS

Probability Effect

• Risk Management

• One Shots

• Multiple Program Rollups

• Automatic Updates

Milestone Dates

PersonnelAssignments

QuestionWeighting

Due Dates

References

ComplianceStatus

Risks

Next Actions

Responsibilities

Action ItemTracking

Mission ProfileScheduleQualityCost

Guidance Documents

EngineeringProcess Model

Statistics

PastExperience

Planning &Tracking

SystemArchitecture

Risk Management

Understanding The Future Effects Of Today's Decisions

Risk Assessment



TRIMS

ProbabilityProbability EffectEffect

• Risk Management

• One Shots



Milestone Dates


QuestionWeighting

Due Dates

References

ComplianceStatus

Risks

Next Actions

Responsibilities

Action ItemTracking


Guidance Documents


Statistics

PastPastExperienceExperience

Planning &Tracking

SystemSystemArchitectureArchitecture

Risk Management


Risk AssessmentRisk Assessment“Risk” is a function of probability and effect. Determining these for

your program is “Risk Assessment”

“Risk” is a function of probability and effect. Determining these for

your program is “Risk Assessment”



TRIMS

Probability Effect

• Risk Management

• One Shots



Milestone Dates


QuestionWeighting

Due Dates

References

ComplianceStatus

Risks

Next Actions

Responsibilities

Action ItemTracking


Guidance Documents


Statistics

PastExperience

Planning &Planning &TrackingTracking

SystemArchitecture

Risk ManagementRisk Management


Risk Assessment

To do “Risk ManagementManagement” you also have to do planning planning and trackingtracking: establish a plan to mitigate your risks, and then

track it to ensure that the plan is being followed, and that it is

having the desired effect!

To do “Risk ManagementManagement” you also have to do planning planning and trackingtracking: establish a plan to mitigate your risks, and then

track it to ensure that the plan is being followed, and that it is

having the desired effect!



TRIMSTRIMS

Probability Effect

• Risk ManagementRisk Management

• One ShotsOne Shots

• Multiple Program RollupsMultiple Program Rollups

• Automatic UpdatesAutomatic Updates

Milestone Milestone DatesDates

PersonnelPersonnelAssignmentsAssignments

QuestionQuestionWeightingWeighting

Due DatesDue Dates

ReferencesReferences

ComplianceComplianceStatusStatus

RisksRisks

Next ActionsNext Actions

ResponsibilitiesResponsibilities

Action ItemAction ItemTrackingTracking


Guidance Documents


Statistics

PastExperience

Planning &Tracking

SystemArchitecture

Risk Management


Risk Assessment

TRIMS provides all of these capabilities, plus analysis, reporting,

and workload-reduction features!

TRIMS provides all of these capabilities, plus analysis, reporting,

and workload-reduction features!


PRODUCT TQM

FUNDING MoneyPhasing

DESIGN TEST PRODUCTION FACILITIES LOGISTICS MANAGEMENT

TRANSITION PLAN

• Design Refer. Mission Profile

• Design Requirements

• Trade Studies

• Design Policy

• Design Process

• Design Analysis

• Parts & Mater. Selection

• Software Design

• Computer-Aided Design (CAD)

• Design for Testing

• Built-In Test (BIT)

• Configuration Control

• Design Reviews

• Design Release

• Integrated Test

• Failure Reporting System

• Uniform Test Report

• Software Test

• Design Limit

• Life

• Test, Analyze & Fix (TAAF)

• Field Feedback

• Manufacturing Plan

• Qualify Manufacturing Process

• Piece Part Control

• Subcontractor Control

• Defect Control

• Tool Planning

• Special Test Equipment (STE)

• CAM

• Manufacturing Screening

• Modernization

• Factory Improvements

• Productivity Center

• Logistics Support Analysis

• Manpower & Personnel

• Support & Test Equipment

• Training Materials & Equipment

• Spares

• Technical Manuals

• Manufacturing Strategy

• Personnel Requirements

• Data Requirements

• Technical Risk Assessment

• Production Breaks

The Original “Willoughby” Templates


PRODUCT TQM


CostAssessment


TRANSITION PLAN

New PMWSTemplatesNew PMWSTemplates



• Trade Studies

• Design Policy

• Design Process

• Design Analysis


• Software Design





• Design Reviews

• Design Release

• Concept Studies & Analysis

• Bread Board Development

• Brass Board Development

• Specification Dev/Allocation/ Validation

• Design for Assembly

• Prototype Development & Review

• Integrated Test



• Software Test

• Design Limit

• Life


• Field Feedback

• TEMP Development/ Execution

• Software Simulator





• Defect Control

• Tool Planning


• CAM


• Production Fabrication

• Environmental Issues

• Modernization



• Field Visits/Site Surveys





• Spares


• Logistics Analysis Documentation






• Determining Defining Need for System

• Prepare Requirement Documents

• Quality Assurance

• Design/Milestone Review Planning

• Make or Buy Decisions

• Technology Base Analysis

• Schedule & Planning

• Dim. Manuf. Sources & Mat. Shortage (DMSMS)

TRIMS Baseline Templates


PRODUCT TQM


CostAssessment


TRANSITION PLAN

New PMWSTemplatesNew PMWSTemplates



• Trade Studies

• Design Policy

• Design Process

• Design Analysis


• Software Design





• Design Reviews

• Design Release

• Concept Studies & Analysis

• Bread Board Development

• Brass Board Development

• Specification Dev/Allocation/ Validation

• Design for Assembly

• Prototype Development & Review

• Integrated Test



• Software Test

• Design Limit

• Life


• Field Feedback

• TEMP Development/ Execution

• Software Simulator





• Defect Control

• Tool Planning


• CAM


• Production Fabrication

• Environmental Issues

• Modernization



• Field Visits/Site Surveys





• Spares


• Logistics Analysis Documentation






• Determining Defining Need for System

• Prepare Requirement Documents

• Quality Assurance

• Design/Milestone Review Planning

• Make or Buy Decisions

• Technology Base Analysis

• Schedule & Planning

• Dim. Manuf. Sources & Mat. Shortage (DMSMS)

TRIMS Baseline Templates

CAM•Manufacturing Screening

•Production Fabrication•Environmental Issues

The TRIMS SE Model includes the original 47 Templates from DoD

4245.7-M, plus another 21 templates that BMPCOE has added over the years – the principles have

been around for 20 years, but the model is up-to-date!

The TRIMS SE Model includes the original 47 Templates from DoD

4245.7-M, plus another 21 templates that BMPCOE has added over the years – the principles have

been around for 20 years, but the model is up-to-date!


UserAdded

Templates

UserAdded

TemplatesUser

AddedQuestions

UserAdded

Questions

3 Tiers: Category

Template

Knowledge

Base

Question

4.0 Design

4.6 Design Analysis

4.6.1 Do the contractor's corporate standards identify design analysis as an integral part of the design process?

TRIMS TailoringTRIMS Hierarchy

Design Test Production Facilities LogisticsManagementFunding TransitionPlan

User Added

Categories

User Added

Categories


Tailoring TRIMS To Your Needs

• Identify Risks Before they Become Problems

• Build YOUR Knowledge into TRIMS– Use the Systems Engineering Knowledge Base as a Starting Point for

A Good Engineering Process

• Create a Concrete, Repeatable Process for Evaluating & Comparing Programs & Contractors

• Ease the Learning Curve for New Employees

• Retain Knowledge of Lost Employees

• Share Knowledge

• To Maximize Benefits of Using TRIMS– Apply as Early as Possible in Program Life– Apply to All Subcontractors, Divisions & Vendors Involved– Use Tool Regularly to Prioritize Resources & Make Decisions


Client Feedback

Benefits• Reliability Established as a Performance Parameter

• Manpower Shortfalls Identified

• Numerous Best Practices Adopted

• Engineering and Failure Analyses Implemented

• Special Test Equipment and Test Assets Identified Early

• Environmental Stress Screening Implemented

• Concurrent Engineering Gains Obtained

Problems• Not Easy to Implement (Socio-Politically)

• Resistance to Change (Not Invented Here)

• Process, Not Product Based

• Few Actually Doing Concurrent Engineering


So How Does TRIMS Help?

TRIMS SE Model Questions that would have helped Pete:

• Eliminate the Design Reference Mission Profile (DRMP)4.1.2Has the contractor used detailed mission functional and environmental profiles to establish requirements

and design margins for the system and its component parts?4.1.4Do environmental profiles cover all functional modes of operation including combat, transport, storage,

handling, training, maintenance, and production?

• Using a New Material for the Missile Body4.5.1Does the contractor's corporate policy include producibility as part of design reviews?4.5.3 Will proof of manufacturing models be required prior to production?4.5.7 Will effect of design on current manufacturing processes and facilities be assessed?10.1.1 Will a manufacturing strategy be developed during the conceptual phase and included in solicitations for

Engineering & Manufacturing Development (EMD)?10.1.3 Will critical manufacturing technologies be identified and expertise acquired early for their development?

• Design Contract: Mostly Design Engineers4.5.2 Will manufacturing and producibility personnel be involved in the design process?4.5.4 Will design and manufacturing engineers be collocated during development?4.5.9 Will operations and customer support personnel be involved in the design process?6.2.1 Will manufacturing processes be qualified at the prime contractor and major subcontractors?6.2.5 Did the Production Readiness Review (PRR) indicate a stable design and manufacturing process?

The design phase takes longer than originally planned, but the final design is easier to test, easier to produce, and better able to withstand the rigors of pre-launch deployment. Testing and Production proceed with only a few minor issues; when the cruise missile is finally deployed, it achieves a fielded reliability of 99.99% - the program is a complete success!


NOSURPRISES!

NOSURPRISES!


A Quick Look At TRIMS

High level view givesan instant “snapshot”

of your program!

High level view givesan instant “snapshot”

of your program!



Answering the “Expert Questions” helps determine risk levels;

“Next Actions” helps mitigate process risks.

Answering the “Expert Questions” helps determine risk levels;

“Next Actions” helps mitigate process risks.



A variety of filtering options make it easy

to review and analyze risk data.

A variety of filtering options make it easy

to review and analyze risk data.



APM Missile Body [Process Risks]

Likelihood of Occurrence

Impact (Cf) - Performance Impact/Risk Profile

5

4

3

2

1

1 2 3 4 5

Performance

2 A-1.1 Money Phasing1 A-1.2 Cost Assessment5 A-2.1 Design Reference Mission Profile4 A-2.2 Design Requirements3 A-2.3 Trade Studies4 A-2.4 Design Policy4 A-2.5 Design Process5 A-2.6 Design Analysis4 A-2.8 Software Design4 A-2.9 Computer-Aided Design (CAD)5 A-2.10 Design for Testing4 A-2.11 Built-In Test (BIT)4 A-2.12 Configuration Control4 A-2.13 Design Reviews3 A-2.15 Concept Studies and Analysis4 A-2.16 Bread Board Development4 A-3.1 Integrated Test4 A-3.7 Test, Analyze, And Fix (TAAF)4 A-3.9 TEMP Development/Execution4 A-4.3 Piece Part Control3 A-4.4 Subcontractor Control3 A-4.11 Environmental Issues1 A-6.4 Field Visits/Site Surveys4 A-7.1 Logistics Support Analysis4 A-7.2 Manpower and Personnel4 A-7.3 Support and Test Equipment4 A-7.7 Logistics Analysis/Documentation3 A-8.2 Personnel Requirements2 A-8.3 Data Requirements

A-1.2

A-6.4

A-8.3

A-1.1

A-4.11A-8.2

A-4.4A-2.3A-2.15

A-3.7A-2.8

A-7.3

A-4.3A-2.13

A-2.12A-2.11A-2.2A-2.5A-2.4A-7.1A-2.9A-3.1A-7.7A-3.9A-7.2A-2.16

A-2.6

A-2.10

A-2.1

Printed: 4 Apr 2005

Print reports at both the summary level...

Print reports at both the summary level...



APM Missile Body [Product Risks]B-1.1.2 SRB Fuel

Milestone

Last Updated

Safety Risk

5

4

3

2

1

Likelihood of Occurrence

1 2 3 4 5Impact Factor (Cf)

03/31/2005

Impacts: -P |

5Sa|

4C |

3Sc

Performer

Derek Designer

Risk ID: B-1.1.2.1 Fuel Components Mixing Risk Rating: High

Description: Fuel components mixing in present fac ility may not be possible without safety upgrade.

Notes: Just testing...

Action Due Date Mitigation Actions

1 3 Apr 2005 Look up current safety regulations

1

2 14 Apr 2005 Identify necessary upgrades

2

3 23 Apr 2005 Conduct Safety T raining

3

4 4 Jun 2005 Install new equipment

4

Page: 1 Printed: 4 Apr 2005

...and in more detail!...and in more detail!


TRIMS - Total Risk Management

PP33RMRM Technical Risk

Identification Mitigation

Problem, Product & Problem, Product & ProcessProcess Risk Management, Risk Management,

in a Single Tool!in a Single Tool!


• Reduce Risk Throughout Acquisition Process• KnowHow: What To Do • Database: Where To Get Help• TRIMS: Keep On Track

• Tested and Used by Government and Industry

• Taught at Defense Acquisition University and Industrial College of the Armed Forces

• Accepted Tool/Methodology Within the DoD Acquisition Knowledge Sharing System

• Acquisition Community Connection:• Program Management: http://acc.dau.mil/pm• Risk Management: http://acc.dau.mil/rm

An Integrated Problem Solver

Program Manager’s WorkStation


BMPCOE . . . Provides High ROI

A place to obtain information about available resources to help make advances and improvements without making costly mistakes

Call 1-800-789-4BMP

www.bmpcoe.org

1998 Winner

Innovations In American Government

2000 Winner

HammerAward

[ End of Presentation ]