A.Murari 1 (24) Frascati 27 th March 2012 Residual Analysis for the qualification of Equilibria A.Murari 1, D.Mazon 2, J.Vega 3, P.Gaudio 4, M.Gelfusa

A.Murari 1 (24) Frascati 27th March 2012

Residual Analysis for the Residual Analysis for the qualification of Equilibriaqualification of Equilibria

A.MurariA.Murari11, D.Mazon, D.Mazon22, J.Vega, J.Vega33, P.Gaudio, P.Gaudio44, , M.GelfusaM.Gelfusa44, E.Peluso, E.Peluso44, F.Maviglia, F.Maviglia55, M. , M. FalschetteFalschette66

1

32

4 University of Rome

“Tor Vergata”

7

5

6 École Centrale de Nantes44000 Nantes, France


Question: how to choose the value of the weighting parameter K1=Wfar/Wcoils?

Goal of the analysis:

Identify a statistically sound methodology to determine the quality of the equilibrium

reconstructions.

The approach is based not only on the 2 but also on high order correlations of the residuals, which have been proved to be adequate for nonlinear

systems.

Statistical method from Billing and Zu (1995)

Statistical Assessment of the Magnetic Reconstructions


Statistical estimator:

• For each probe (i) the following variable i has been computed:

while for each shot the average over all n coils is:

having the following statistical error:


is not fully adequate

In case of nonlinear systems, indicators of the 2 type are not fully satisfactory. They take into account only the amplitude of the residuals.

The time evolution of the residuals can also

provide very interesting information about the quality of the models.

t

y


The correlation tests method Hypothesis: the noise is random and additiveConsequence: the residuals of a perfect model should be randomly distributed

The model with the distribution of the residuals closer to a random one is preferred

Cost function: correlation functions of the following type


The correlation tests methodTheory: for an infinite series of random number the autocorrelations should be zero

With finite samples the autocorrelations will not exactly be zero

Anderson, Bartlett and Quenouille showed in the 40s that the autocorrelation coefficients of white noise data can be approximated by a normal curve with mean zero and standard error 1/√n wher n is the number of samples

95% confidence level can be calculated 1.96 1/√nAdvanced correlations for nonlinear systems

Autocorrelations


where q is the number of the dependent variables and r is the number of the independent variables.

A complete and adequate set of tests for a nonlinear, MIMO system is provided by the higher order correlations between the residual and input and output vectors given by the following relations ( residuals, u inputs, y outputs):

tE)(

tE)(

ttt q22

1 ...

ttyttyt qq ...11

tutut r22

1 ...

If the non linear model is an adequate representation of the system, in the ideal case, should be:

otherwise

k

0

0

,)( ,)( 0

New model validation method


EFIT

yu

Inputs:Pickup coils

Faraday measurements

Outputs:Pickup coilsFaraday chords reconstructed by EFIT

Residuals:Difference between measurements & EFIT

for pickup coils and Faraday

Implementation of the correlations for equilibrium

Implementation of the correlations for the case of EFIT:

In our case the analysis consists of assessing the quality of the equilibrium reconstructions of EFIT by analysing the distribution function of the residuals


The correlation tests method• 2 different points of view:– Global: all data are computed for all coils– Local: data are computed independently for

individual coils


EFIT:

EFIT

EFIT version:

EFIT-J

Pressure constraints

Polarimeter constraints (ch 3, 5, 7)

P’ and FF’ equal to 0 at the separatrix


Residuals

-6 -5 -4 -3 -2 -1 0 1 2

x 10-3

0

100

200

300

400

500

600Coil n°21 [TP203 ] - Error distribution

Error (T)

Poin

ts n

um

ber

-3 -2 -1 0 1 2 3 4 5 6

x 10-3

0

100

200

300

400

500

600Coil n°40 [TN211 ] - Error distribution

Error (T)

Poin

ts n

um

ber

Monomodal error type Multimodal error type

Residuals: differences between the experimental values and the model estimates or predictions

Residuals in the case of the equilibrium (EFIT) and the magnetic measurements for two coils

Residuals are often presented as histograms: x axis the value of the residual, y axis the number of occurrences of

that value


Monomodal / Multimodal error shapes No clear tendency

Residuals: monomodal and bimodal pdf


Example of global correlations

42 44 46 48 50 52 54 56 58 60-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Linear auto-correlation , (residuals & residuals) - Pulse 68671

Delay (s)

Corr

ela

tion

42 44 46 48 50 52 54 56 58 60-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Linear cross-correlation u, (measurements & residuals) - Pulse 68671

Delay (s)

Corr

ela

tion

, u,

Outside the 95% confidence interval Problems in the reconstructions


60%outsi

de

outside

80%

Many points outside the 95% confidence interval Failings in the model

Overview of the database


Example of local correlations

u,,

NO clear trend: the pattern changes from shot to shot and even during the same discharge (more than

120 shots analysed)


Comparison of ELM-free and ELMy phases

• Hypothesis: ELMs are of the the causes of the multimodal distribution

• Comparison of the EFIT quality during ELMs and during ELM-free periods

Figure: Shot 75412. Top: D channel; Bottom: ELMs in the EHTR channel.


Residual distributions: visual analysis

• The residual distribution function of the pick-up coils shows typically a multimodal shape. The ELMs typically account for one of the peaks

Total Residual distribution

ELMs free ELMy phase


Summary of the shots analysed

• Details of the shots analysed:

• Abut 350 type I ELMs studied. Results are consistent not only for the shots but also for the individual coils so the statistical basis is considered sufficient of the shots


Utility function: the Z-test

• In order to check if two physical quantities, two measurements

etc are different, the Z-test is normally used ( 1,2 are the

averages of the in the ELMy and ELM-free phases):

• If the Z variable is higher than 1.96, the two quantities are statistically different with a

confidence exceeding 95%.

FEyEy

FEyEyZ22


Z-test for the ELMy and ELM free periods

• The variable has been computed separately for the ELMy (Ey ) phase and for the ELM free one (FEy ):

Results: the for ELMy and ELM-free periods are different with a confidence well in excess of 95%.Not an academic exercise: in statistics quantity is

quality

Details of this application by

M.Gelfusa, A.Murari et al to be submitted

to NIMA

Results: the during ELMs is always higher than in ELM-free periods


- ELMs effects on the equilibrium. Three main causes:- a) EFIT hypotheses not valid: equilibrium, toroidal symmetry,

current at the boundary etc- b) Coils: delays, eddy current in metallic structures etc. - c) Not optimal constraints in EFIT

ELMs

- A specific dry shot in which the currents in the divertor coils have been modulated is being used to assess the time response of the coils


Relation between residuals in the ELMy and ELM-free phases versus time constant of the coils

Fast and slow coils

Difference m of the residual means between the ELMy and ELM –free phases versus the difference between the rise time of the signals of the pick-up coils and the divertor currents.

Fast coils reconstructed more poorly


The constraints of p’ and ff’ to go to zero at the separatrix have been relaxed.

11 both zero 00 both parameters free

Constraints at the edge

Freeing p’ and ff’ improves the situation in ELM-free periods but does not have a major effects for the reconstructions during ELMs

Monomodal residual pdf

Bimodal residual pdf


00 17 19

01 14 19

10 17 19

11 8 17


00 37 30

01 39 30

10 37 30

11 45 34


Question: how to choose the value of the weighting parameter K1=Wfar/Wcoils?

Summary:

The approach based not only on the 2 but also on high order correlations of the residuals increases

the confidence in the results

However, no principle method has been found yet to determine the relative importance of the 2 and

the high order correlations.

The application to the investigation of the ELMs seems to indicate that the main issue resides in the limited physics in EFIT more than in the coils

or the constraints.

Statistical Assessment of the Magnetic Reconstructions:

Summary


Example: pendulum

0 5 10 15 20 25 30 35 40-2.5

-2

-1.5

-1

-0.5

0

Time

Solu

tion

Accurate model

100 200 300 400 500 600 700 800 900 1000

-0.1

-0.05

0

0.05

0.1

Linear auto-correlation , (residuals & residuals)

Delay (s)

Corr

ela

tion

0 100 200 300 400 500 600 700 800 900 1000-0.1

-0.05

0

0.05

0.1

Linear cross-correlation u, (inputs & residuals)

Delay (s)

Corr

ela

tion

Residual for Accurate model Good correlations(inside the 95% confidence interval)

y’’ + ∙y’ + a sin(∙ y) = b sin(∙ ∙t)

25

Nonlinear pendulum plus 10% of Gaussian noise. Black curve: exact solution Red curve: exact solution plus noise


Example: pendulum

0 5 10 15 20 25 30 35 40-2.5

-2

-1.5

-1

-0.5

0Wrong model

Time

Solu

tion

0 100 200 300 400 500 600 700 800 900 1000-0.5

0

0.5

1

Linear auto-correlation , (residuals & residuals)

Delay (s)

Corr

ela

tion

0 100 200 300 400 500 600 700 800 900 1000-0.5

0

0.5

Linear cross-correlation u, (inputs & residuals)

Delay (s)

Corr

ela

tion

Error added on parameter a Poor correlations (outside the 95% confidence interval)

y’’ + ∙y’ + a sin(∙ y) = b sin(∙ ∙t)

Details of application to equilibrium in the paper A.Murari et al Nucl. Fusion 51 (2011) 053012 (18pp)

Documents

A.Murari 1 (24) Frascati 27 th March 2012 Residual Analysis for the qualification of Equilibria A.Murari 1, D.Mazon 2, J.Vega 3, P.Gaudio 4, M.Gelfusa