Uncertainty Analysis Monte Carlo simulation

User guide English version

Iva Nachtigalová Miloslav Suchánek

ICT Prague Metrological laboratory associated laboratory of CMI

Uncertainty Analysis – user guide

ICT Prague 2/12

Table of Contents

Introduction ............................................................................................................................ 3

1 Application requirements .............................................................................................................. 3

2 Manipulation with application ...................................................................................................... 3

3 User interface ................................................................................................................................ 4

4 Inserting input data ....................................................................................................................... 5

4.1 Measurement ................................................................................................................. 5

4.2 Calibration ...................................................................................................................... 6

5 Running analysis and interpretation of results ............................................................................. 7

5.1 Measurement ................................................................................................................. 7

5.2 Calibration ...................................................................................................................... 9

6 Recalculating analysis .................................................................................................................... 9

7 Report messages ......................................................................................................................... 12

8 Printing ........................................................................................................................................ 12

9 Saving........................................................................................................................................... 12

Uncertainty Analysis – user guide

ICT Prague 3/12

Introduction Uncertainty Analysis is a spreadsheet application which performs analysis of the uncertainties in the measurement and calibration based on propagation of probability distribution. In accordance with documents

ISO/IEC 98-3:2008 Uncertainty of measurement – Part 3: Guide to the expression of uncertainty in measurement (GUM 1995)

ISO/IEC Guide 98-3/Suppl.1:2008 Uncertainty of measurement – Part 3: Guide to the expression of uncertainty in measurement (GUM 1995), Supplement 1: Propagation of distribution using Monte Carlo method

the application evaluates

in case of the measurement value, standard uncertainty and probability distribution of a measurand from a measurement function and from estimations of input quantities

in case of the calibration

1. values and standard uncertainties of coefficients of linear calibration relationship between estimates of measurand and estimates of corresponding signal of a measuring instrument or a system provided by etalon

2. corrected value for recovery and standard uncertainty of measurand provided by sample from estimation of corresponding signal of a measuring instrument or a system and from linear calibration relationship.

All estimates are simulated from supposed probability distributions (or normal probability distribution in case of calibration) by Monte Carlo method, values and standard uncertainties of calibration coefficients are calculated by bilinear regression method with uncertainties on both axes.

Note: Measurement model assumes equation with 1 measurand and max. 10 input quantities with normal, rectangular or symmetrical triangular probability distribution. Calibration table assumes max. 10 pairs of calibration measurements provided by an etalon.

Note: As in case of the measurement, so in case of the calibration application generates always 5 000 estimations.

The application consists of one file UncertaintyAnalysis.xltm which complements this PDF user guide.

1 Application requirements File UncertaintyAnalysis.xltm is a document of the program MS Excel 2007 and MS Excel 2010 that contains macros (series of VBA commands). Therefore, the spreadsheet application requires installation some of mentioned versions MS Excel and enabled macros (permanently in the Trust Center or temporarily in the Message Bar when you create new workbook – see on-line help for Office 2007 or Office 2010).

2 Manipulation with application Essentially, the file UncertaintyAnalysis.xltm is the workbook template (the file with XLTM extension), i.e. the model that serves as the basis of other workbooks. If user wants to insert input data and to get results of uncertainty analysis he must create a new workbook based on the template and then manipulate only with new workbook.

Uncertainty Analysis – user guide

ICT Prague 4/12

Note: New workbook can be created by double-clicking the template in any file manager (e.g. Windows Explorer, Total Commander etc.) or in the dialog box under MS Excel command New > New from existing file. The new workbook copies content, formatting, VBA codes and customized user interface from the template, so the template remains unchanged. However, if user accidentally changes the template the full application functionality cannot be provided and the user must download the template again.

Way to manipulation with a new workbook is a simple: firstly, user must insert data (see Chap. Chyba! Nenalezen zdroj odkazů.), and secondly, user must run chosen analysis (see Chap. 5) and – if it is possible – he can recalculate it (see Chap. Chyba! Nenalezen zdroj odkazů.). Each new workbook generates sheets that can be handled in a standard way (more about printing sheets and saving in Chap. Chyba! Nenalezen zdroj odkazů. and Chyba! Nenalezen zdroj odkazů.).

3 User interface Each new workbook based on the template has customized ribbon interface with the new tab Uncertainty Analysis (at the end of the ribbon) that contains new controls (see Fig. 1). Controls are divided into two groups – Measurement intended for uncertainty analysis in measurement and Calibration intended for uncertainty analysis in calibration. Functions of controls are following:

New Input Data Button that opens the new sheet named MData # or CData # for inserting input data.

New Analysis Menu with two buttons: Full with intermediate and final results that runs analysis and generates new report sheet MReport # or CReport # with Excel formulae, intermediate and final results, and Quick with final results which runs the same analysis but generates new report sheet only with final results. Menu is accessible only if some sheet MData # or CData # is active.

Repeat Analysis Button that recalculates analysis. Button is accessible only if report sheet generated by button Full with intermediate and final results is active.

Fig. 1 New ribbon tab Uncertainty Analysis and its controls

Uncertainty Analysis – user guide

ICT Prague 5/12

Note: It is also possible to run the macros associated with ribbon controls by shortcuts displaying in the tooltips (see Fig. 1), or from the list of macros in the dialog box of the button Macros on the tab Developer.

Note: If workbook or MS Excel crashes the tab Uncertainty Analysis or its controls may not be accessible. In this case user must restart the program MS Excel and reopen workbook.

4 Inserting input data Inputs for analysis must be inserted into colored cells of the sheets MData # or CData # that are generated by button New Input Data (see Chap. Chyba! Nenalezen zdroj odkazů.).

Note: Cells for inserting input data are highlighted by the same color as buttons in the ribbon tab Uncertainty analysis. Similarly, they are highlighted as well as the final results of analyses.

Note: Sheets are protected against insertion data of other types than required ones. For this reason user can insert data only from keyboard, not from other sheets, files or programs.

4.1 Measurement

Uncertainty analysis in measurement requires these inputs:

Name Names or descriptions of input quantities (in table Input Quantities) and measurand (in table Measurand).

Units Physical units of input quantities (in table Input Quantities) and measurand (in table Measurand).

Value Values or (arithmetic) means of input quantities (in table Input Quantities).

Distribution Supposed probability distributions of input quantities (in table Input Quantities).

Standard Uncertainty Standard uncertainties of input quantities. In case of normal distribution, user inserts the standard deviation; in case of rectangular and symmetrical triangular distribution, user can insert the Excel formula =x/3^0,5 or =x/6^0,5 respectively, where x is the value of the half-width of distribution interval (see Fig. 2).

Function Excel formula of the measurement function (i.e. formula for calculation of measurand) of which arguments must be cell address from the column Name of the table Input Quantities. Formula must be inserted into the dialog box and then it is displayed in the cell in symbolic form (see Fig. 3).

In addition to these required data, user can insert optional notes into cells labeled Your Notices.

Note: The formula of the measurement function must be really Excel formula, i.e. must starts by sign = and can contain only numbers, basic mathematical operators, rounded brackets and cells addresses. User does not need to insert addresses from the keyboard - if user types the formula he can click on a cell in the column Name and MS Excel inserts corresponding address into the formula on actual cursor position.

Uncertainty Analysis – user guide

ICT Prague 6/12

Fig. 2 Sheet MData – insertion of Excel formula for calculation of standard uncertainty of input quantity b with rectangular distribution

Fig. 3 Sheet MData – insertion of Excel formula of measurement function

4.2 Calibration

Uncertainty analysis in calibration requires these inputs:

Name Name or description of a measurand.

Units Units of a measurand and signal of measurement instrument or system.

Value Values or (arithmetic) means of a measurand provided by etalon and corresponding signal of measuring instrument or system (in table Calibration Table) and value of signal of measuring instrument or system provided by a sample (in table Measurement).

Recovery Recovery of a measurand provided by etalon.

Uncertainty Analysis – user guide

ICT Prague 7/12

Standard Uncertainty Standard deviations of a measurand provided by etalon and standard deviations associated with signals of a measuring instrument or system (in table Calibration Table), standard deviation associated with a recovery of a measurand and a standard deviation associated with a signal of a measuring instrument or system provided by the sample (in table Measurement).

In addition to these required data, user can insert optional notes into cells labeled Your Notices (see Fig. 4)

Fig. 4 Sheet CData –insertion of value of signal of a measuring instrument or system

5 Running analysis and interpretation of results Uncertainty analysis is triggered by one of the two buttons in the menu New Analysis (see Chap. 3) if any of sheet with input data MData # or CData # is active. User is informed about status of the analysis on the Status bar of MS Excel. Both buttons generate report sheets with identical final results, only report sheets generated by button Full intermediate and final results contain intermediate results and Excel formulas in addition which enable recalculation of analysis (see Chap. Chyba! Nenalezen zdroj odkazů.).

5.1 Measurement

Report sheets with results of uncertainty analysis in measurement (see Fig. 5) contain all input data in the section labeled DATA; results are located in the other sections. Their meaning is as follows:

ESTIMATION OF INPUT QUANTITIES The table Input Quantities with names, units, distributions, simulated means and simulated standard deviations of input quantities, and table Measurand with measurand name and unit and measurement function. Report sheets generated by button Full with intermediate and final results contain also estimates simulated from distribution in each Monte Carlo trial.

Uncertainty Analysis – user guide

ICT Prague 8/12

Fig. 5 Sheet MReport – intermediate and final results generated by button Full with intermediate and final results

Uncertainty Analysis – user guide

ICT Prague 9/12

ESTIMATION OF MEASURAND Table Measurand with name, unit, simulated mean and simulated standard deviation of measurand. Report sheets generated by button Full with intermediate and final results contain also estimates of measurand in each Monte Carlo trial which were simulated from estimates of inputs and measurement function.

ESTIMATED MEASURAND HISTOGRAM, PROBABILITY DENSITY FUNCTION AND DISTRIBUTION FUNCTION Tables Parameters and Bins, frequencies and distribution function with parameters, bins and frequencies from which are plotted chart Histogram of Measurand and chart Probability Density and Distribution Functions of Measurand.

5.2 Calibration

Report sheets with results of uncertainty analysis in calibration (see Fig. 6 a 7) contain all input data in the section labeled DATA; results are located in the other sections. Their meaning is as follows:

ESTIMATION OF MEASURAND AND SIGNAL PROVIDED BY MEASUREMENT STANDARD Table Calibration Table with names, units, simulated values and simulated standard deviations of measurand provided by etalon and corresponding signal of a measuring instrument or system. Sheets generated by button Full with intermediate and final results contain also estimates of quantities simulated from normal distribution in each Monte Carlo trial.

ESTIMATION OF COEFFICIENTS OF CALIBRATION FUNCTION Table Calibration Function with values of coefficients b2 and b1 and their standard deviations su(b1) and su(b2) of calibration relationship Y = b2X + b1 which were calculated from estimates of table Calibration Table by bilinear regression method with uncertainties on both axes. Sheets generated by button Full with intermediate and final results contain also table Bilinear Least Squares Regression which calculates next regression data – correlation coefficient ro(b1,b2), covariance covar(b1,b2), sum of squared deviations S and number of degrees of freedom df.

ESTIMATION OF MEASURAND AND SIGNAL PROVIDED BY SAMPLE Table Measurement with name, unit, simulated value (corrected for recovery) and simulated standard deviation of measurand provided by sample and corresponding signal of a measuring instrument or system. Sheets generated by button Full with intermediate and final results contain also estimates of quantity simulated from normal distribution in each Monte Carlo trial.

6 Recalculating analysis Report sheets with intermediate and final results which were generated by button Full intermediate and final results contains Excel formulae and thus can be recalculated. Analysis recalculation is triggered by button Repeat Analysis (see Chap. 3) if any of sheets with intermediate and final results is active. User is informed about status of the analysis on the Status bar of MS Excel.

Note: Analysis are based on estimates of quantities simulated by Monte Carlo method that uses built-in RAND function to generate random numbers from the interval < 0, 1 >. Therefore at each sheet recalculation results of uncertainty analyses are slightly different.

Note: Report sheets with final results generated by button Quick with final results do not contain Excel formulae and thus it is not possible to recalculate them.

Uncertainty Analysis – user guide

ICT Prague 10/12

Fig. 6 Sheet CReport – left part with input data, estimates of pairs of calibration table and estimates of coefficients of calibration relationship generated by the button Full with intermediate and final results

Uncertainty Analysis – user guide

ICT Prague 11/12

Fig. 7 Sheet CReport – right part with values and standard deviations of measurand provided by sample, its estimations and signal of a measuring instrument or system generated by button Full with intermediate and final results

Uncertainty Analysis – user guide

ICT Prague 12/12

7 Report messages Report sheets confirm successful analyses, if not warn the user to possible errors. Messages are always displayed in the row no. 4 under the sheet identification header and they differ in color (see Fig. 8)

Fig. 8 Report message warning to errors in input data (on the left) and confirming successful analysis (on the right)

8 Printing If it is necessary sheets with input data and report sheets with results can be printed (it is better to print report sheets with a less volume of data that were generated by button Quick with final results). Print preview or printing is invoked using standard MS Excel commands.

Note: Print parameters are set so analysis inputs and outputs to include sheet identification header of the current sheet and to cover just one A4 page. However after connection the user‘s printer, the page margins can be automatically changed and printed data can be distributed to multiple pages. Then user must adjust print parameters manually.

9 Saving Each workbook based on the spreadsheet application template contains VBA codes which treat input data, generate estimates of the quantities by Monte Carlo method, calculate coefficients of the calibration curve, plot charts and ensure functions associated with the ribbon controls. So workbook remains fully functional it must be saved as sheet with enabled macros (with file extension XLSM).

Note: If a workbook is saved as a standard one without enabled macros (with extension XLSX) it irreversibly (!) loses all macros and so its functionality.

Note: Report sheets with a big volume of data generated by button Full with intermediate and final results slow opening and saving a workbook and significantly increasing its size. In the case of a large number of analyses is therefore preferable to work with several workbooks.