Copyright © 2012 Boeing. All rights reserved.

Effective Verification and Validation Testing

Steve HoltBoeing Commercial AirplanesAugust 2013

Copyright © 2012 Boeing. All rights reserved.

Initiatives Were Put In Place to Address: Incomplete or unnecessary test requirements Immature test capability Inefficient test conduct

Success found in: Application of systems engineering tools and methods Teaming across design and test functions early in the design cycle Test engineering participation in requirements and V&V development

Questioning why past requirements are needed Questioning current methods of verification

Focusing on problematic design features and complex integration areas Optimizing affordability to include risk mitigation (cost avoidance) as well as

cost reduction

Effective Verification & Validation TestingSummary - Commercial Airplane Product Development

Copyright © 2012 Boeing. All rights reserved.

Navigate:

Requirements

DesignersLab Test Value

Stream

Flight Test Value StreamStart

validating the design

here!

• Inspection• Analyses• Modeling

• Modeling• Simulation• Hardware Bench

Testing• Integrated System

Testing

Full ScaleIntegrated Testing

Verification Planning

Verification & Validation CycleCommercial Airplane Product Development

Copyright © 2012 Boeing. All rights reserved.

Verification & Validation TargetEarly Verification and Risk Retirement

Typical Plan

Target

AnalysesInternalBenchTesting

SupplierBenchTesting

IntegratedSystems

Laboratory

AirplaneFlightTest

Modeling&

Simulation

Req

uire

men

t Ve

rific

atio

n

Copyright © 2012 Boeing. All rights reserved.

Verification ToolsEducating Design Engineering on Capability

Wind Tunnel

Bench Tests

Full Scale Test Articles

Integrated System Testing

Modeling & Simulation

Culture Shift

Copyright © 2012 Boeing. All rights reserved.

Requirement

Verification Plan

Facility Needs

SatisfyingData

Test Statement of Work

Test System Readiness Triggers

Integrated Test Plan

Data Source Determination

Test Optimization

Verification & Validation

Governance

• Verification Approach• Verification Culture• Verification Capture

Interactive Development Among Design and Test Engineers

Test article readiness ensures early risk

mitigation

Part of test efficiency initiatives

Verification & Validation DevelopmentCross-Functional Verification, Test, and Means of Compliance

Copyright © 2012 Boeing. All rights reserved.

Requirement Area

Analyses Modeling & Simulation

Laboratory Testing

Integrated System Tests

Flight Article

Flight Loads Concurrent

High Speed Mach Buffet

Multiple Failure Assessment

Safety of Flight

System Functionality

Crew Workload Determination

Pilot & Airport Operations

Concurrent

Spatial Integration

Structural & Actuation Loads

Structural Interfaces

Systems Interfaces

Verification Exercise – Folding Wing TipExample of Earlier Verification and Risk Retirement

Copyright © 2012 Boeing. All rights reserved.

• A culture change is often necessary to qualify “Affordability” as cost optimization, to include investment in early verification and risk mitigation

• Using test experts to scrutinize design requirements and means of verification leads to a more robust and mature product

• Shifting verification “to the left” as much as practical helps ensure a mature and cost-optimized product

• Focusing on test readiness and efficiency throughout the development life cycle helps ensure verification success

• Complex systems yield emergent effects. Some of them increase risks, some of them increase opportunities.

Effective Verification & Validation TestingConclusions