Assessment of Predictive Capability

James Paul Holloway

CRASH Review MeetingOctober 19 2009

Page 2

Several initial UQ projects are complete

• Investigated (with CRASH 1D) the importance of:– Initial radiation field, boundary conditions, EOS, initial

field fit

• Explored (with Hyades 1D) sensitivity of shock location at 13 ns over 15-D input space– Included experimental, physical and numerical parameters

• Constructed a physics informed emulator (PIE) for CRASH 1D initial fields at 1.3 ns– Identified 40 scalar parameters to describe Hyades output

fields– Constructed an emulator for these over the input variables– Investigated of sensitivity of PIE data to 15 Hyades inputs

• Constructed a predictive model over 8-D input space CRASH 1D

• Performed sensitivity study of 3D CRASH over a 4-D input space See later talks by Fryxell & Bingham

and various posters

Page 3

Predictive Roadmap

• Analysis elements:– Experimental design– Sensitivity studies

• Year 1 & 2– Integration of team members– Exercise of analysis tools using Hyades, CRASH 1D and CRASH

multi-D• Year 3

– Use analysis & experiments to identify and improve modeling elements – quantify sources of error• Numerical error estimation• Uncertainty due to experimental and physics parameters

– Suggest experiments based on UQ analysis– Improve preprocessor (PIE) and extend to 2D– Demonstrate improved predictions

• Year 4– Analysis of 5th year experimental regime to estimate

sensitivities and predictive distribution– Suggest experiments based on UQ analysis

• Year 5– Provide improved predictions of 5th year experiment and

prediction pdf

– Emulator construction– Calibration of physics

parameters– Estimating predictive

distributions

Page 4

We are heading towards the year 5 elliptical tube

• Do computations and experiments with circular tubes

• Can do computations with elliptical tubes

• Must predict output y and its pdf for the elliptical tube

Year 1-4experiments

Year 1-4experiments

Year 5 experimentYear 5 experiment

Input space x

SimulationsSimulations

Page 5

We will use statistical modeling to guide improvement of the CRASH code

• Use assessment of predictive capability to guide– Improvement in CRASH code physics and numerics– Identification of needed field experiments or

diagnostics

to produce predictions for an extrapolated field experiment

• Aim towards predicting results in an unexplored region of the field experimental input space (the oval tube)

• Carry this out with a first principles physics code, avoiding the use of tuning parameters or unphysical choices of physics parameters

CodeAssessmentExperiments

Page 6

Predictive science is more than prediction

• Knowledge of the sensitivity of the output to variations in the input– Which input parameters, when varied over their

probable ranges, generate the significant variations in output

• An understanding of the significant sources of uncertainty in output

• An estimate of the predictive distribution of outputs over– Uncertainty in physics parameters– Uncertainty in experimental parameters– Uncertainty in statistical fitting parameters

• Successful prediction means that the experimental result is, with reasonable probability, within the range predicted by the code

Page 7

• CRASH maps inputs to output

• We want to characterize the variations in around interesting ranges of inputs – Sensitivities– Predictive distributions

• Numerics converged sufficiently to be small contributor to overall uncertainty

We understand CRASH as a map from input to output

Focus ofcurrentUQ efforts

Page 8

Some of our key inputs include:

• Laser energy and pulse shape

• Initial Xe pressure

• Be drive disk thickness

• Time of radiograph observation

• Tube geometryThese are uncertain

These are uncertain

These introduce error

• EOS data• Opacities

• Mesh parameters• Limiter choice• Time step controller

Page 9

Key outputs include:

• Shock position at observation time

• Distance of edge of shock from tube wall

• Angle of xenon edge just downstream of shock

• Average thickness of Xe layer

X-ray radiograph from experiment with average velocity 140 km/sec at 14 ns

Determine sensitivities of andpredict pdf of outputfor a new experiment ,accounting for uncertain inputs

Page 10

We have explored the influence of problem setup

• Initial radiation field• Left (irradiated) boundary location• EOS treatment• Initial field fit Varied mesh, initial field, boundary location

Page 11

• Use a set of 10 key spatial locations and interpolate hydrodynamic fields at 1.3 ns

• Create mapping of Hyades inputs to these 40 parameters

• Uncertainty in fit will contributeuncertainty to CRASH results

We have build a physics informed emulator: PIE

Page 12

Sensitivity analysis using statistical fitting

• Sample input space (Latin hypercube sampling)• Compute outputs• Fit a model

– Gaussian Process model – Flexible regression models

• Estimate global sensitivity due to an input by:1 perturbing input variable randomly over samples2 refitting3 computing RMS difference

• Estimate local sensitivities using length parameters from GP model

• Marginalize over other variables to explore variation due to one or two variables

The LHD used for CRASH 3D sensitivity study

Page 13

We can identify important influences on shock location

Based on 512 Hyades runs and MART fit

•Be Thickness•Laser Energy•Xe Density•Be and Xe gammas•Be and Xe opacity scale factors

Informs selection of parameters for 3D study

See related study on CRASH 3D in Fryxell’s talk

Page 14

• Construct predictive model of output

along with posterior calibration pdf

• Combine with estimated or input ranges to account for input uncertainty

• Such a model provides:– Mean and variance of output – predictive distribution– Tool to explore error in the model relative to

experiment– Approach to determine a next experiment or simulation

to reduce uncertainty

A predictive model provides a pdf of outputs

Page 15

CRASH 1D Predictive Model Construction

Page 16

Measurements

• Select 8 as training data, predict a 9th.

Page 17

The predictive model gives us a way to explore physics

• Data for model are field experiments and set of code runs

• Model data as sum of 3 terms:• Emulator of the code (Gaussian process model)• Model discrepancy (Gaussian process model)• Measurement error

• Large discrepancy compared to measurement uncertainty implies prospects for prediction are poor– physics or numerics of code is wrong– or measurements are misleading

• Code improvements can be judged against reduction in discrepancy in key regions of input space

• Must avoid conflating discrepancy with calibration

Results using CRASH 1D in Bingham’s talk

Page 18

Our immediate next steps build on this work

• Explore the CRASH 1D Predictive Model– Explore discrepancy over inputs and reflect on the

physics– Compare joint computation of discrepancy and calibration

with sequential calibration and computation of discrepancy

– Repeat with multi-group diffusion and explore discrepancy

– Include uncertainty in initialization– Quantify sources of uncertainty

• Use predictive model to suggest new code runs and experiments– Minimize uncertainty– Minimize discrepancy

• Move towards a predictive model using CRASH multi-D• Propagate uncertainty information through EOS/Opacity

model

Page 19

We are heading towards the year 5 elliptical tube

• We can perform– Experimental design– Sensitivity studies– Emulator construction– Calibration of physics parameters– Estimating predictive distributions

• Next steps– Use analysis & key experiments to identify and

improve key modeling elements – quantify sources of error• Numerical error estimation• Uncertainty due to experimental and physics parameters

– Suggest experiments based on UQ analysis– Improve preprocessor (PIE) and extend to 2D– Demonstrate improved predictions– Analysis of 5th year experimental regime to estimate

sensitivities and predictive distribution– Suggest experiments based on UQ analysis– Provide improved predictions of 5th year experiment

and prediction pdf

Year 1-4experiments

Year 1-4experiments

Year 5 experimentYear 5 experiment

Page 20

Related Posters

• Automatic Detection of Shock Wave FeaturesN Patterson, R. P. Drake, K Thornton

• Building a Dataset of Results Modeling Radiative Shocks in 1D HYADES for Uncertainty Quantification

M. J. Grosskopf, R. P. Drake, B. Fryxell, F. W. Doss, C. C. Chou, D Bingham

• Calibrating HYADES as initial conditions for CRASHMJ Grosskopf, R. P. Drake, B Torralva, F. W. Doss

• Gaussian Process Regression and Bayesian MARS for CRASH Initialization with HAYDES

Duchwan Ryu, Ryan McClarren, Anirban Mondal, Zach Zhang, Ashin Mukherjee, Jason Chou, Bani Mallick, Derek Bingham, Bruce Fryxell, Vijay Nair, Ji Zhu

• Gradient-Enhanced Stochastic Expansion: A Survey of Response Surface Construction Methods

Colin Miranda, Krzysztof Fidkowski, Kenneth Powell• Preliminary 1D CRASH Resolution Study / Variable

ScreeningChuan-Chih (Jason) Chou, Paul Drake

• Verification of Uncertainty Quantification SoftwareBruce Fryxell, Jason Chou, Ashin Mukherjee, Zach Zhang, Vijay

Nair, Mike Grosskopf, Derek Bingham, Bani Mallick, Duchwan Ryu

Page 21

Questions?

Page 22

Additional Slides Follow

Page 23

CRASH Personnel directly engaged in UQ

• Bruce Fryxell, James Holloway, Paul Drake• Derek Bingham, Vijay Nair, Bani Mallick• Marvin Adams, Ryan McClarren• Jason Chou• Chris Fidkowski• Mike Grosskopf• Anirban Mondal• Ashin Mukherjee• Avinash Prabhakar• Duchwan Ryu• Zach Zhang• Ji Zhu

Page 24

Sensitivity study shows mesh convergence

Average shock location at 13 ns converged

Average shock location at 13 ns converged

Page 25

Hyades emulator construction and sensitivity runs

• 15 dimensional input space sampled

• 512 point space filling LHD constructed

• TAMU constructed PIE emulator for 1.3 ns

• UM carried out sensitivity analysis for shock location at 13 ns

Page 26

Experimental Design for CRASH 1D analysis

• 256 Points over 8 dimensions (4 thetas, 4 x’s) orthogonal array based space-filling LHD8 purely for computer run inputs

• 64 Points over the product space