SpatialAnalyzer Advanced Uncertainty Analysis

USMN

Agenda

• Review of USMN• Open Questions• New USMN Features/Additions

Presentation Outline

• Background and Motivation• Instrument Uncertainty Characterization• Discrete Point Cloud Uncertainty Fields• Combining CASs – Traditional Approach• Unified Spatial Metrology Network

(USMN)• Case Studies

Open Question

• What is Ranking … how does it relate to measurement confidence?

• Statistics Review… Predicting local fit error on a daily basis (point RMS, uncertainty)

• Instrument uncertainty why should we use average

• explain usmn with nominal point groups when we can use it

• can I get summary stats out after I have shut down USMN

• some questions on instrument uncertainty

Measurement Tools

Theodolites & Total Stations

Digital Photogrammetry Laser Trackers

Portable CMMs

Digital LevelsLaser Scanners

Background

• Many instrument types and models in use.

• Each manufacturer has individual, incompatible, software applications.

• Users need to apply several devices to a single measurement task.

• Operators need to re-train on each software package. Software

ASoftware

BSoftware

C

Need: General SoftwareCommon User Interface

Unify Metrology Processes

CombinedResults

Motivation• Uncertainty statements must accompany

measurements. (NIST TN/1297, ISO Guide, ANSI GUM, NCSL RP-12)

• Coordinate measurements used to make important (and expensive) decisions

• Multiple systems are often used to perform a single measurement job.

• Current industry practice is to make guesses at (or ignore) overall combined uncertainty based on instrument manufacturer specifications.

• Needed:– Instrument performance in the “real-world” – Geometric representation of uncertainty– Combination of measurements and uncertainty– Task-Specific Uncertainty (geometrical fits, etc.)

Questions… Questions… Questions…

• What is the uncertainty of my instruments in the “real-world”?

• What is the effect of uncertainty propagation on the quality of my measurements?

• How can I make optimal use of my measurements to minimize uncertainty?

• Ok, its nice to know the uncertainty of a point, but I’m fitting a cylinder. What is the uncertainty of my fit?

• What about my hidden point bar?

Unified Spatial Metrology NetworkAnswers… Answers… Answers…

• Combine measurement systems• Characterize instrument uncertainty• Verify instrument performance• Determine uncertainty fields• Take advantage of the relative

uncertainty of the measurement components.

• Geometric fitting uncertainty (sphere, line, plane, cylinder, etc)

Coordinate Acquisition System (CAS) Uncertainty Characterization

• Measure the performance of the entire system under the conditions of interest.

• Include instrument, operator, environment, etc.

• Determine uncertainty of compensated instrument output values.

• Determine effect of these uncertainties on the measured coordinates.

Instrument Example: Laser Tracker

,,

m

kkiik mfp ,

Measurement Process

• Establish a field of unknown fixed points.

56 feet

Measurement Process• Measure the points from the first

instrument location.

Measurement Process• Measure the points from the second

instrument location.

Measurement Process• Measure the points from the third

instrument location.

Measurement Process• Measure the points from the fourth

instrument location.

Solve for Instrument TransformationsPoint Computation:• Find • Minimize

Instrument Transform Computation:

p

1

2

31

kkiik mfp , ,,

m

*ip

n

in

2

1000

),,(

z

y

x

Worldk B

BR

B

T

R

R

R

B

B

B

t z

y

x

k

Find

...

3

2

1

t

t

t

T

Minimize

m

mE2

...

2

1

0

Extract Uncertainty from Residual Errors

• Group residuals by component

fromallr

fromallr

fromallr

• “Type A” uncertainty evaluation

componentcomponent rU

J

jjcomponent r

JU

1

2)(1

1

UUU ,,• Result:

Coordinate Acquisition System Outputwith Realistic Uncertainty Statement

UUU ,,

,,

m

Uncertainties including all measurable effects: operator,

environment, target, mechanical backlash, etc.

Coordinate Acquisition System

Instrument Performance ComparisonManufacturer Measurement Horizontal Vertical Distance

Location (arcseconds) (arcseconds) (1,000th inch)

A 1 0.6 0.4 0.2

1 0.6 0.4 0.2

2 1.1 1.5 0.5

3 1.4 1.5 0.7

4 1.3 1.7 0.3

Average 1.0 1.1 0.4

B 1 1.8 2.9 1.5

1 0.9 0.8 0.1

2 1.0 1.4 0.7

2 1.1 0.5 0.4

3 2.2 1.3 0.5

3 1.8 1.0 1.0

3 1.8 1.0 1.0

Average 1.5 1.3 0.7

C 1 1.8 1.9 1.0

1 1.9 1.8 0.4

2 1.0 0.9 0.4

3 1.4 1.3 1.3

4 1.2 1.3 0.7

5 1.3 1.4 0.6

6 1.7 1.5 1.4

Average 1.5 1.5 0.8

A, B, C Average 1.3 1.3 0.6

Horizontal

Vertical

Distance

A

B

C

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Arc

sec.

or

1/10

00th

inch

Average Uncertainty Results by Manufacturer

A

B

C

Uncertainty Characterization as an Operational Check

Parameter Before Compensation After Compensation Delta

Vertical index (arcseconds) 17.820 20.736 2.916Axis tilt (arcseconds) 27.864 25.272 -2.592

Mirror tilt (arcseconds) -10.692 -10.692 0.000Home distance (inches) 6.09528 6.09661 0.00134

Component 1 Sigma UncertaintyBefore Compensation

1 Sigma UncertaintyAfter Compensation

Typical Performance(from Table 4.2)

Horizontal Angle

3.47 arcseconds 0.91 arcseconds 1.3 arcseconds

Vertical Angle 11.45 arcseconds 1.18 arcseconds 1.3 arcseconds

Distance 0.0087 inches 0.000598 inches 0.0006 inches

Totalmeasurements

32 32

Coordinate Uncertainty Fields

**** ,,

jm

U

U

U

U

randf

randf

randf

R j

jj RU

jj mm

,ijij mfp

X

Y

)(14

00220.00012.26

00145.09224.88

00077.09085.304

inchesip

)(

)(

)(

zz

yy

xx

CMZ

CMY

CMX

Uncertainty Field Density

Laser Tracker

Uncertainty Field with 100,000 Samples

X

Y

Uncertainty Field Component Standard Deviation:Percent Deviation from 100,000 Sample Result

0%

5%

10%

15%

20%

25%

30%

35%

40%

Sample Size

% D

ev

iati

on

fro

m 1

00

,00

0 S

am

ple

R

es

ult X

Y

Z

Mag

Field Density: How many field points are needed?

Combining 2 CASs – Traditional Approach

• Match common points by minimizing residuals.

• Apply transformation to all points and the instrument.

Measured Points (Moving)

Nominal Points (Fixed)

ii

ii

ii

zzi

yyi

xxi

MFe

MFe

MFe

)23(

)13(

)3(

1000

000

000

000

1000

),,(

s

s

s

B

BR

B

Tz

y

x

iMF

MiF PTP

*

Chain of CASs - Traditional Best-Fit• Transform tracker to

CAD• Transform Arm to

Tracker• Transform Scanner to

Arm• All transformations

based on XYZ coordinate residuals

• Usually performed using multiple software packages

TTTT CMMPScanner

TracCMMP

CADTrac

CADScanner

ker

ker

Unified Spatial Metrology Network:A Method for CAS Combination

• Simultaneous combination of CASs• Relative uncertainty weighting for

measurements• Determine uncertainty fields based

on CAS combination.• Task-Specific: Apply uncertainty

fields to downstream analysis.

Laser Tracker

Total Station

• Different instrument types

• Different uncertainty characteristics

• Weight measurement components based on relative uncertainty.

Laser Tracker

Total Station

L = 518 inches

L = 376 inches

Relative Uncertainty Weighting in Point Computation

Weighting Example

,,

m

PPMUUD

UD

UD

)sin(

)sin(

D

DW

D

DW

D

DW

U

U

U

U

Total Station Laser TrackerHorizontal Angle (arcseconds) 0.5 1.3Vertical Angle (arcseconds) 0.5 1.3Distance (inches) 0.03937 0.0006

(parts per million) 2 0

Total Station Laser TrackerHorizontal Angle (degrees) 306.418 54.737Vertical Angle (degrees) 100.192 123.621Distance (inches) 376.2864 518.2100

Total Station Laser TrackerHorizontal Angle (inches) 0.00091 0.00330Vertical Angle (inches) 0.00091 0.00330Distance (inches) 0.04012 0.00060

Total Station Laser TrackerHorizontal Angle Weight 10.77 2.97Vertical Angle Weight 10.77 2.97Distance Weight 0.24 16.33

Total StationResidual (inches) Weight Objective Contribution

Horizontal Angle -0.00055 10.77 -0.00592Vertical Angle -0.00061 10.77 -0.00657Distance 0.00825 0.24 0.00198

Laser TrackerResidual (inches) Weight Objective Contribution

Horizontal Angle 0.00409 2.97 0.01215Vertical Angle 0.00970 2.97 0.02881Distance 0.00003 16.33 0.00049

U

m

D

W

e

M500 Optimal Point Laser Tracker #5

Total Station #0

Total Station #4

Total Station #2

Horizontal Error Vertical Error Distance Error Magnitude

Total Station #0 -0.0017 0.0042 0.0032 0.0056Weights 11.2 11.2 0.24

Total Station #2 -0.0052 0.0036 -0.0784 0.0786Weights 5.0 5.0 0.24

Total Station #4 -0.0008 -0.0088 -0.0403 0.0413Weights 8.2 8.2 0.24

Laser Tracker #5 0.0072 -0.0094 0.0005 0.0118Weights 2.9 2.9 16.4

Weighting Example: 4 Measurements

USMN Uncertainty Analysis

Laser Tracker

Total Station

Common Targets

2 Instruments Before Combination

USMN Uncertainty Field Analysis

Network Solutionwith Actual

Measured Values

Inject U intoall measurements

Network Solutionwith Measured + U

Values

UncertaintyField

CompositeCoordinate Set

+

USMN Uncertainty Propagation

T1

T2

T3

A

B

C

Total Station

Laser Tracker

Composite Point, T1, and Uncertainty Field

2 Instruments After Combination

Fixed Reference

USMN Uncertainty Propagation

Add a 3rd Instrument to the Measurement Chain

Add a 3rd Instrument to the Measurement Chain

USMN Uncertainty Propagation

Fixed Reference

Close the Measurement Loop to Reduce Uncertainty

Fixed Reference

USMN Uncertainty Propagation

Task-Specific Measurement Uncertainty• Given point uncertainties, how is my

actual measurement job result affected?

• What is the uncertainty of a sphere fit?

• Hidden Point Bar?• Go/No Go Decision? How certain are

you it’s a GO?

USMN Task-Specific Uncertainty

Small Coverage Large CoverageSphere Center:Ux 0.0018 0.0003Uy 0.0027 0.0004Uz 0.0012 0.0002Umag 0.0035 0.0005

Udia 0.0067 0.0008

Analysis time 3.3 sec. 3.7 sec.Points in Fit 186 222Field Points 1000 1000

• Given coordinate uncertainty fields….

• What is the uncertainty of the sphere fit in part coordinates?

• What is the uncertainty of the measured cylinder axis and diameter?

Small Coverage Large CoverageCylinder:

U Axis (deg) 0.0021 0.0009U Diameter (inches) 0.0027 0.0005

Analysis time (P-4 1.3 GHz) 14.7 sec. 32.3 sec.Points in Fit 217 495Field Points 1000 1000

USMN Task-Specific Uncertainty

Analysis: Hidden Point Bar Uncertainty

• Uncertainty Fields Interact

• End-Point is extrapolated…

• And so is the uncertainty!

• Yikes!

USMN Software Integration

USMN Advanced Settings

Case Studies

• Aircraft Carrier Catapult Alignment (CVN-76)

• Disney Concert Hall Panel Positioning• Submarine Fabrication (SSN 774)• Nuclear Power: Steam Generator

Replacement

Tracker #0

Tracker #1

Tracker #2

Tracker #3

Aircraft Carrier Catapult Alignment (CVN-76)

• Long narrow structure

• 350’ x 6’ trough

• 4 laser trackers chained together

Catapult

1,797 Measured Points300 field samples15 minute run timeP-4 1.8 gigahertz

Disney Concert Hall (LA)

285 measured points300 field points11 minute run timeP-4 1.8 gigahertz

Submarine Fabrication (SSN-774)

296 points1.6 sec. for single solution300 field points28 minutes run timeP-4 1.3 gigahertz

Steam Generator Replacement

Uncertainty Chain

106 points300 field points9 minute run timeP-4 1.8 gigahertz

Additional Point Measured by #0 & #5

Uncertainty Reduced by Closing the Measurement Chain

Future Applications

• Wrap optimization around USMN to determine instrument type and placement.

• Expand instrument models to include the multitude of internal parameters.

• Extend Task-Specific analysis to point to surface fitting and other analyses.

• Extend Uncertainty to entire GD&T & FD&T analysis process – decision uncertainty.

Conclusions• It is now possible to obtain realistic

geometrical uncertainty statements for combined measurement systems.

• It is also possible to obtain these results on the shop floor at the technician level.

• Realistic uncertainty statements provide ISO / ANSI compliant measurements. Replaces uncertain uncertainty guesses.

• This information will help to educate measurement technicians and designers so they may reduce measurement uncertainty in the future.

X

Y

M500 Optimal Point Laser Tracker #5

Total Station #0

Total Station #4

Total Station #2

Questions?

Mersenne Twister Random Number Generator

Gaussian Random Number Generator:Percent Deviation from 100,000 Sample Result for 6 Data Sets

0%

5%

10%

15%

20%

25%

30%

35%

40%

Sample Size

% D

ev

iati

on

fro

m 1

00

,00

0 S

am

ple

Re

su

lt

1

2

3

4

5

6

Attempts at Mapping Numerical Uncertainty

Instrument reference frame

Ux = ± 1

Uy = ± 2

Part reference frame

Point in Part frame:X = -3, Y = 6Ux = ?, Uy = ?

Point in Instrument frame:X = 2.5, Y = 4Ux = ± 1, Uy = ± 2