Download pdf - Nondestructive Evaluation: USAF Perspectives on …mapod.weebly.com/uploads/1/8/4/4/18447259/lindgren...1 Nondestructive Evaluation: USAF Perspectives on POD, Imaging, and Characterization

1

Nondestructive Evaluation:

USAF Perspectives on POD, Imaging,

and Characterization

E r i c L i n d g r e n

M a t e r i a l s a n d M a n u f a c t u r i n g D i r e c t o r a t e , A i r F o r c e R e s e a r c h L a b o r a t o r y

J u l y 1 3 , 2 0 1 9

Distribution A, Unlimited Release. Case Number 88ABW-2019-3348

2

Outline

• USAF Relevance

• POD Refresher• MIL HDBK 1823A

• Assumptions

• Method (Math)

• USAF Standardization• Why / How

• Image based NDI• Implementation

• POD Validation

• Characterization• Background

• Approach / Status

• MAPOD

• Summary

Photograph by author


3

US Air Force Relevance

• Nondestructive Inspection / Evaluation (NDI / NDE)

key input to risk management in USAF

• Aircraft Structural Integrity Program (ASIP)

• Propulsion Systems Integrity Program (PSIP)

• NDI / NDE used for new materials qualification

• Polymer matrix composites: 100% inspection

• Additive Manufacturing for critical parts

• Probability of Detection (POD): capability validation

• Facilitate with model-assisted POD (MAPOD)

• Image-based NDE / NDI methods in use

• Desired state: flaw / material characterization

• Augments value of data from NDE

• Accelerates decisions based on NDE


4

Example: USAF Approach for ASIP (MIL STD 1530Dc1)

• First flight to 1950s: design to static strength• Wright brothers

• Worked with finite use

• 1950s: fatigue in metals leads to safe life• DeHavilland Comet

• B-47 Stratofortress

• 1970s: flaws at manufacturing leads to durability and damage tolerance

• F-111

• F-5

• Commercial aviation uses variant of DADT


5

Durability and Damage Tolerance (DADT)

• Tolerate defects for some inspection-free period of service usage:

slow damage growth is USAF ASIP preferred approach

• NDE at critical locations based on DADT analysis to protect safety

How do we assess this value?


6

POD: A Key Factor in Risk Calculation

*www.afgrow.net


7

POD Refresher

Photograph by AuthorDistribution A, Unlimited Release. Case Number 88ABW-2019-3348

8

POD Introduction

Probability of Detection:

• Testing/evaluation procedures for assessing NDI capability

• Validation of NDI procedure when performed as intended

• Human factors affecting procedure

• Not human factors affecting inspector

• Objective: determine largest flaw missed during inspection

• Capability study, not a sensitivity study

• Improved sensitivity does not necessarily improve POD

• Statistics to determine value

• Must meet assumptions of math to be valid


9

MIL HDBK 1823A

• Current title “NDE System

Reliability Assessment”

• Personal perspective: “system

reliability” should be replaced by

“procedure validation”

• Provide guidance:

• How to set up a POD Study

• How to analyze the data

• Assumptions that MUST be met


10

Design of the experiment

• Includes all factors affecting detection

• Measurement system

• Probes, electrical noise, calibration

procedure, and many more

• Structure being assessed

• Variance: manufacturing, maintenance,

repair, modification, and use

• Defect being detected

• Geometry, morphology, and many more

Attributes of the data

• Independent samples

• Response increases with flaw size

• Linear models apply

• Noise is normal

• Constant variance

If data is binary, use hit/miss analysis

• Outcome: Probability that flaw of size

“x” will be called by inspector

Factors and Assumptions for Valid POD

≠ Find damage here

Sensors

Notch Plate


11

The Nature of Data

𝜀 is a

NORMAL

random

variable

(noise)

Parameters are

ESTIMATED,

not calculated

Ideal Data

Real POD

Curve

Ideal POD

Curve

Real Data


12

Confidence Bounds (Complex Math for Engineers)

Requires:

• Large data set

• Symmetric

distribution

Determines:

• Variance

• Standard Normal


13

POD: Cautions

Remember assumptions:

• Independent samples

• Response increases with flaw size

• Linear models apply

• Noise is normal

• Constant variance

Must have suitable data set to meet assumptions

Size

Response

POD

POD values can be calculated even if assumptions are

NOT met, but these values are wrongDistribution A, Unlimited Release. Case Number 88ABW-2019-3348

14

Additional Challenges for SHM POD

USAF definition: SHM is “a nondestructive inspection process or

technique that uses in-situ sensing devices to detect damage”

• Environment

• Temperature, loads, etc.

• Time variance in performance

• Includes durability

• Validation of Capability*

• Required for ASIP driven applications

• Qualification

• Initial documents available**

*Lindgren, et.al., “Demonstrating Capability Validation Protocol for in-situ Damage Detection,” presented at the 2011 ASIP Conference, San Antonio, TX

**Brausch and Steffes, “Demonstration, Qualification, and Airworthiness Certification of Structural Damage Sensing (SDS) Systems for

Air Force Applications, ” AFRL-RX-WP-TM-2013-0062

Installation:

• Qualification / certification

• Training

• Human factors

Sustainment:

• Training

• Maintenance / Calibration

• Durability

• Repair / modification

• Technical orders


15

Reminder: How USAF Uses POD for ASIP

When to Inspect


Calculation of Risk

16

USAF Standardization


17

Need for Standardized Capability

When critical to ensuring safety (i.e. risk)

USAF structures challenge:

• Large number of aircraft

• High mobility of inspectors

• Increased inspection requirements with aging fleet

• Possible variance in capability as a function of weapon system

USAF Solution:

• Structures Bulletin EN-SB-08-012• Baseline common procedures/equipment

• Assumed capability for general classes of structural geometric features

Inventory: 1017*

*http://www.af.mil/AboutUs/FactSheets.aspx

Inventory: 428*


18

Foundation of EN-SB-08-012

Best Practices common inspection methods using standardized equipment

• See "Recommended Processes and Best Practices for NDI of Safety-of-

Flight Structures", AFRL-RX-WP-TR-2008-4373

• Includes inspection implementation and capability estimation

• Detection capability quantified for most standard practices

• Part specific procedures reference appropriate standard practice

Applicability

• Detection capability assumptions based on standard equipment, training,

and procedures across USAF programs

• Not applicable for specialized equipment or procedures: they should have separate

validation

• Supersedes Table XXXII of JSSG 2006: Joint Services Specification Guide –

Aircraft Structures


19

Content of EN-SB-08-012

• Introduction

• Members of USAF Capability Task Group

• Description of assumptions in training, procedures, and

assessments made in determining the capability values

• Additional steps to be followed if alternative values of capability

for specific inspections

• Description of applicability

• Detection of fatigue cracks in metallic structures to support damage

tolerance or durability analysis

• Field and Depot, performed by USAF personnel

• Not for contractor personnel or commercial procedures


20

Content of EN-SB-08-012 (cont)

• NDI capability values, a90/95 for calculating inspection intervals by

inspection method, including

• Method specific requirements, e.g. surface preparation

• Material type, e.g. aluminum, titanium, or steel

• Representative geometry, e.g. flat surface, radii, edges, bolt hole

• Eddy current probes, e.g. pencil probes, conformal probes

• Fluorescent Penetrant Inspection

• Whenever possible, the a90/95 values and POD curves determined via

guidance of MIL HDBK 1823A

• Alternative published values can be used

• Values can be adjusted when consensus that published values are not

representative

• Includes guidance for the use of:

• Ultrasound, Magneto-optical Imaging, Visual, Computed RadiographyDistribution A, Unlimited Release. Case Number 88ABW-2019-3348

21

Representative Case Study: Raised Head Fasteners*

• Inspection around raised head fasteners

• New probes conform to geometry of

fasteners

• Engineering / demonstration

• Validation via POD IAW MIL HDBK 1823A

• 51 inspectors, 3 USAF depots, 2200 fasteners,

44 cracks

• Impact -- detectable crack size decreased:

0.200” to 0.100” exposed crack length

*Forsyth et.al., available at: www.meetingdata.utcdayton.com/agenda/asip/2010/proceedings/presentations/P4213.pdf

Representative structure

Raised head fastener probes and kit

POD: pencil/raised head fastener probes

POD results:

Raised head

fastener

probes


22

Benefit to USAF*

Standardized:

• Assessment of inspection capability

• Assessment of methods to improve

capability

• Consensus for all USAF weapon

systems

• Focused efforts to improve capability

• Additional examples in Dr. Jones’

2015 ICAF paper

*K. Jones, J.C. Brausch, W.A. Fong, B.L. Harris, “Probing the Future,: Better F-16 Inspections using Conformal Eddy Current Inspection Tools,” Proceedings of the 28th International Conference of Aerospace Fatigue Symposium Helsinki, Finland, June 2015.


23

Image-based NDI


24

Image-based NDI in USAF

Two general classes of image-based NDI

• Image direct from data capture system• Fluorescent Penetrant

• Radiography

• Magnetic Particle

• Thermography/ Shearography

• Visual

• Image from transposing other data• Ultrasonic B and C-scans

• Eddy current C-scans

• Computed tomography


25

Image Case Study*

Note: Requirement removed with

center wing replacement program

in the mid 2000’s

• Large area requirement

• ~8,800 fasteners in upper wing

• ~10,000 fasteners in lower wing

• Fasteners both raised and flush

• Fatigue cracks in both layers

• Skins and stringer flanges

• 0.070” corner cracks*E. A. Lindgren, et.al., “Validation and Deployment of Automated Ultrasonic Inspection of the C-130 Center Wing,” Proceedings of the Aircraft Structural Integrity Program Conference, Savannah, GA December 2003.


26

Approach: Portable Ultrasonic C-scan*

*E. A. Lindgren, et.al., “Validation and Deployment of Automated Ultrasonic Inspection of the C-130 Center Wing,” Proceedings of the Aircraft Structural Integrity Program Conference, Savannah, GA December 2003.


27

Criteria for Defect Detection*

• Elongated gate for first and

second layer indications

• Side lobes in amplitude C-scan

• Must have correct location

relative to fastener

• Time-of-flight gradient

• Shows reflections from crack as a

function of transducer position

• Initial analysis automated to

assist inspectors with large

data sets

Amplitude C-scan

Time-of-Flight C-scan



28

• 216 holes, 6 samples

• 36 per sample

• 432 inspection sites

• 3 different first layer thickness

• 0.15”, 0.25”, 0.35”

• Second layer constant

• Fasteners:

• Three samples: raised head

• Three samples: flush head

• 144 flaws: 24 over 36 holes

• 18 no, 12 one, and 6 two flaws

• 36 flaws per layer: near and far

on first layer, plus near and far

on second layer

• Flaw size distribution: uniform

log scale

• Four each of following (inches):

0.01, 0.013, 0.017, 0.023, 0.030,

0.039, 0.052, 0.068, and 0.090

Design of Experiment*


Set criteria and MIL HDBK 1823A methods apply


29

POD Execution and Results*

Execution

• 5 inspectors

• In Depot environment

• Upper / lower wing configuration

• Production equipment

Results

• 90/95 was 0.067” or smaller

• Exact number depended on location and fastener type



30

Image-based NDI: Lessons Learned

• Current methods for evaluating capability apply

• Select detection criteria

• Determine type of response: a vs. a-hat or hit/miss

• Leverage all available data

• e.g. use A-and/or B-scans for UT, impedance plane for ET

• Images can help detection processes

• Human eye very sensitive to change

• Caution when trying to automate image analysis

• Especially true when data direct from imaging system: relying on pixel

values to determine if defect is present


31

Characterization (MSA)


32

Characterization: MSA

Materials State Awareness: Digitally-enabled Reliable Nondestructive Quantitative Materials / Damage Characterization Regardless of Scale

• National Academies Workshop: 2007 (Wood Hole, MA)

• Defense Materials, Manufacturing, and Infrastructure Workshop: 2014 (Washington, DC)

• Hosted by the National Academies

Motivation:

• Condition-based Maintenance

• Enhanced risk management

• New materials qualification (e.g. additive manufacturing)


33

Approaches for Characterization

Multiple thoughts on technical method:

• Image-based pixel counts

• Provides an initial approximation

• Advanced signal processing

• Used in some applications in power generation

• Inversion of NDE signal

• Typically ill-posed: factors affect NDE signal

not related to defect

• USAF focused on Model-based inversion

• Use data to models to assist in addressing

confounding factors

Representative fatigue crack

Representative corrosion (intergranular)

Representative composite impact damage


34

USAF Technical Approach to MSA

Augmented Materials Design, Processing, and Performance

Efficient and Effective ASIP/PSIP/MX Actions

Model-driven Quantitative Representation of Material/Damage State with Statistical Metrics

3D Representation and Validation

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

crack length (in)

PO

D

MAPOD

exp.

Signal Analysis and Uncertainty Quantification

NDE Damage / Materials

Characterization


35

Considerations for Characterization

• One-dimensional experiments

• Limits change to one factor

• Excellent to determine sensitivity to that one factor

• Historical focus of lots of talks at QNDE

• Example: residual stress measurements with eddy current

• Correlation between conductivity and residual stress, but

• Also correlation between conductivity and microstructure, loads,

temperature, etc.

• Characterization problems are typically multi-dimensional

• NDE modalities are sensitive to many factors

• Need to address confounding factors


36

AFRL Characterization Projects

Past

• Microstructure from surface measurements (eddy current and Rayleigh waves)

• Consider capability of SRAS from Univ. Nottingham

Present

• Eddy current for crack sizing: turbine engine components and evolving to bolt-hole

eddy current

• Bulk wave ultrasound for impact damage in polymer matrix composites

Future

• Volumetric microstructure

• Localized atomic changes in ceramic-based materials

Multiple talks at QNDE 2019, plus in previous years in the proceedings


37

Validation of Characterization

• Consider metrics of accuracy in length, depth, and width,

plus volumetric location in a part for risk management

• Analogy to a 3D POD

• Focus on only important parameters

• For fatigue cracks: length and depth

• For location, depends on geometry

• Active efforts within AFRL to address the process


38

Model-assisted POD (MAPOD)


39

MAPOD

• Motivation: reduce time and cost of POD studies

• Approach:

• Transfer functions

• Full model assist

• Working group formed in 2004

• AFRL, NASA, and FAA

• Over 100 contributions

• Active for ~10 years

• Renewed interest to address POD of SHM

MAPOD

Logo was

not created


40

Unified Approach (Framework) for MAPOD

• Combines considerations

of transfer functions and

full-model assist

• Discussed in MIL HDBK

1823A

• Sample preparation and

flaw independence driving

renewed interest for SHM

• Presentation on Tuesday

afternoon with examples


41

Summary

POD, Imaging, and Characterization

• USAF Relevance

• POD Refresher

• USAF Capability Standardization

• Image based NDI

• Implementation

• POD Validation

• Characterization

• Background

• Approach / Status

• MAPOD

USAF Pay-off

• Safety of structures

• Integrated to realize ASIP/PSIP risk

(safety) metrics

• Availability of aircraft

• Extended inspection intervals

• Augmented management capability

• Reduced cost / time of inspection

• With NO compromise to safety

• New materials qualificationDistribution A, Unlimited Release. Case Number 88ABW-2019-3348

42

Discussion
