CS 351/ IT 351 Modeling and Simulation

Technologies

Errors In Models

Dr. Jim Holten

CS 351/ IT 351

Errors in Models

• Sources of Errors

• Characterizing Errors

• Using Error Bounds

• Interpreting Error Implications

CS 351/ IT 351

Sources of Errors

• Input Values (measurements)

• Machine Inaccuracies

• Algorithm Inaccuracies

• Bad models

CS 351/ IT 351

Measurement Errors

• Measurement granularity

• Granularity accuracy ==> Error intervals

• Types of measurements

CS 351/ IT 351

Machine Errors: Representation

• Float: 7 decimal places, E+/-38, or

subnormal E-45 (fewer digits of precision)

• Double – 16 decimal places, E +/-308, or

subnormal E-324 (fewer digits of precision)

CS 351/ IT 351

Machine Errors: Representation

• Equality comparisons (does 0.0F == 0.0D?)

• Overflow (too big an exponent)

• Underflow (too small an exponent)

• Mismatch (1.000E19D + 47.3D = ?)

CS 351/ IT 351

Machine Errors

• Divide by zero (+/- Inf), or divide zero by zero (NaN)

• Propagate “bad” values

• Worst-case scenarios, not seen as errors

– Near zero results of add or subtract

– Near zero denominator

CS 351/ IT 351

Algorithm Sources of Errors

• Inaccurate representation of real world

• Inaccurate representation of ideal world

• Computational errors

CS 351/ IT 351

Real World to Ideal Model

• Math Models are Idealistic

• Real world has many perturbations

• Statistical estimates are only “best fit” to

observed measurements

• Results in an inaccurate ideal model

CS 351/ IT 351

Ideal Model to Implementation

• Machine errors in number representations

• Machine errors in arithmetic calculations

• Results in even worse implementation

model values

CS 351/ IT 351

Computational Errors

• Numerical calculation to approximate math

functions

• Numerical Integration

• Numerical differentiation

• Techniques used determine the error

behaviors

CS 351/ IT 351

Controllable Errors• Understanding sources and behavior of

errors empowers you to control them and

predict their effects on the results.

• Identifying sources and effects of errors

allows you to better judge the quality of

models.

CS 351/ IT 351

What Gives Bad Models?

• Wrong equations

• Wrong numerical methods

• Details gone awry

• All irrationally affect results.

CS 351/ IT 351

Characterizing Errors

• Error Forms (Probability Distributions?)

• Error propagation effects on error forms

• Limitations versus needs

CS 351/ IT 351

Error Characterizationss• Error probability distributions

• The normal distribution• Zoo of common other distributions• Arbitrary distributions

• Error bounds

• Generalized error estimation functions

• Enumerated values and “false negatives”

CS 351/ IT 351

Error Probability Distributions

• Measurement error characteristics

• Calculation error characteristics

• Introduced algorithmic error terms

CS 351/ IT 351

Measurement ErrorCharacteristics

• Discrete sample on a number line

• Spacing determines “range” for each

measurement point

• Actual value may be anywhere in that range

CS 351/ IT 351

Calculation ErrorCharacteristics

• Round-off

• Divide by near-zero

• Divide by zero

• Algorithm inaccuracies

CS 351/ IT 351

Algorithmic ErrorCharacteristics

• Depends on the algorithms/solvers used

• Depends on the problem size

• Depends on inter-submodel data sharing

patterns and volume

CS 351/ IT 351

Errors: Normal Distributions

• Easy to characterize

• Propagates nicely through linear stages

• Useless for nonlinearities, special

conditions

• Not always a good fit

CS 351/ IT 351

Errors:Generalized Distributions

• Not commonly used

• Easy to represent (histograms into PDFs)

• Propagate through nonlinear calculations?

• Awkward: histograms, PDFs, CDFs for

each variable

CS 351/ IT 351

Errors: Bounded

• Not commonly used

• Easy to represent (+/-error magnitude)

• Can be propagated through nonlinear

calculations

• Still awkward for some calculations

CS 351/ IT 351

Errors: Propagating a Distribution

• Highly dependent on the distribution and

the calculations being performed.

• Generally only linear operations give easily

predictable algebraic results.

• Others require piecewise approximations

CS 351/ IT 351

Error Bounds• Expected value, +/-error magnitude, or min/max

• Propagates through calculations?

• More complex forms may be needed after

propagation – bounded piecewise linear

distributions

CS 351/ IT 351

Errors: Unhandled Implications

• Misinterpretation of results

• Misplaced confidences

• “Chicken Little”, “The Boy Who Cried 'Wolf'”, and ignored real consequences