© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 1

Analysis and graphicalrepresentation of S-parameter data

for Euromet 555 measurementcomparison exercise

Topics :– E555: what is it?– How data was provided.

• Problems.

– Analysis• Outlier elimination• Comparison Reference Value and Confidence Region• Monte Carlo

– Draft A representation

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 2

E555: What is it?

• Measurement comparison exercise– European Metrology Collaboration

Project 555– Supplementary Comparison

Euromet.EM.RF-S16

• Measurement of Scattering Parameters

• 12 Participants: BNM-LNE/LAMA,Nmi VSL, INMS NRC, NIST, METAS,CSIR-NML, EPSG/MTA, IEN, SQF,NPL, INTA, UKAS No“0147“

• Status: Measurements done. Draft A in preparation.

• Devices: 2.4 mm system– Attenuator: 3, 20, 40 dB– Load: male, female– Missmatch: male, female

• Frequencies: 50 MHz - 50 GHz

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 3

Measurements performed

• Full Data Set: 63 frequencies– 50, 100, 500 MHz– 1, 2, 3, .. , 39, 40 GHz– 40.5, 41.0, 41.5, .. , 49.5, 50.0 GHz

Euromet 555Visualisation Tool

Further Analysis:Outlier eliminationComparison Reference Value (CRV)95% Confidence regionresults shown in draft A

• Annex Data (7 frequencies) withuncertainties:

– 0.05, 1, 10, 20, 30, 40, 50 GHz

• Required Format:– magnitude and phase (r, φ)

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 4

Problems with (r,φ) data

Small r:

• r-u(r) < 0

0.03

0.02

0.01

0.00

-0.01

-0.02

S11

mag

nitu

de

LA

B 01

LA

B 02

LA

B 03

LA

B 04

LA

B 05

LA

B 06

LA

B 07

LA

B 08

LA

B 09

LA

B 10

LA

B 11

LA

B 12

Load 407, f = 0.05 G Hz

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 5

Problems with (r,φ) data

Small r:

• r-u(r)<0

• u(φ) overestimated

• Large variations inu(r) and u(φ)

0.030

0.020

0.010

0.000

S11

mag

LAB

01

LAB

02

LAB

03

LAB

04

LAB

05

LAB

06

LAB

07

LAB

08

LAB

09

LAB

10

LAB

11

LAB

12

300

200

100

0ph

ase

(deg

)

-0.004

-0.002

0.000

0.002

0.004

Im[S11]

-0.004 -0.002 0.000 0.002 0.004Re[S11]

3 dB, f = 0.050 GHz

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 6

Overview of analysis of annex data

• Ignore uncertainties in r and φ• Transform data from (r,φ) to complex representation (x,y)• Eliminate outliers using robust statistics• Calculate mean for real (x) and imagenary (y) component to determine

comparison reference value (CRV)• Calculate standard uncertainties and correlation for CRV• Determine 95% confidence region in x-y for CRV• Determine 95% confidence interval for r and φ using monte carlo

method

References:N.M. Ridler and M.J. Salter, Metrologia 39, 2002, 295-302M.J. Salter, Anamet Report 033, April 2001

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 7

Transformation

• Data Points: 12..1),( =Φ ir ii

• Transformation:– Real component:– Imagenary Component:

),(),( iiii yxr �Φ

iii

iii

ry

rx

Φ⋅=

Φ⋅=

sin

cos

0.030

0.020

0.010

0.000

S11

mag

LAB

01

LAB

02

LAB

03

LAB

04

LAB

05

LAB

06

LAB

07

LAB

08

LAB

09

LAB

10

LAB

11

LAB

12

300

200

100

0

phas

e (d

eg)

-0.004

-0.002

0.000

0.002

0.004

Im[S11]

-0.004 -0.002 0.000 0.002 0.004Re[S11]

3 dB, f = 0.050 GHz

a

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 8

Analysis II: Outlier Elimination

),( mm yx• Determine spatial median:

• Median of absolute deviation (MAD):

-0.004

-0.002

0.000

0.002

0.004

Im[S11]

-0.004 -0.002 0.000 0.002 0.004Re[S11]

-0.004

-0.002

0.000

0.002

0.004

Im[S11]

-0.004 -0.002 0.000 0.002 0.004Re[S11]

-0.004

-0.002

0.000

0.002

0.004

Im[S11]

-0.004 -0.002 0.000 0.002 0.004Re[S11]

-0.004

-0.002

0.000

0.002

0.004

Im[S11]

-0.004 -0.002 0.000 0.002 0.004Re[S11]

( ) ( ){ }NiyxyxmedMAD mmii ,,1;,, �=−=

• Outlier criterion:

( ) ( ) MADyxyx mmii ⋅>− 75.3,,

• Use robust estimator

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 9

Analysis III: Comparison Reference Value (CRV)

• CRV:

• Standard uncertainties in CRV:

∑=

=N

iix

Nx

1

1 ∑=

=N

iiy

Ny

1

1

( )∑=

−−⋅

=N

ii xx

NNxu

1

2

)1(1

)(

( )∑=

−−⋅

=N

ii yy

NNyu

1

2

)1(1

)(

-0.004

-0.002

0.000

0.002

0.004

Im[S11]

-0.004 -0.002 0.000 0.002 0.004Re[S11]

0.0015

0.0010

0.0005

0.0000

-0.0005

Im[S11]

-0.0030 -0.0025 -0.0020 -0.0015 -0.0010Re[S11]

0.0015

0.0010

0.0005

0.0000

-0.0005

Im[S11]

-0.0030 -0.0025 -0.0020 -0.0015 -0.0010Re[S11]

0.0015

0.0010

0.0005

0.0000

-0.0005

Im[S11]

-0.0030 -0.0025 -0.0020 -0.0015 -0.0010Re[S11]

1-dim 68% confidence intervalsfor normal distribution

• Correlation coefficient:

( )( )( ) ( )yuxu

yyxxNNyxr

i

N

ii

⋅

−−−⋅=

∑=1)1(

1

),(

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 10

Two-dimensional normal distribution: Formalism

• 1-dim Gaussian: ( )Mean:Variance,:�

2exp)( 2

2

µσ

µ

−−∝

xxf

• 2-dim Gaussian: ( )( ) ( )( ) ( )

nCorrelatio:�

2

121

exp),( 2

2

2

2

2

−−

−−−

−−

−∝y

y

yx

yx

x

xyyxx

yxfσ

µσσ

µµρσ

µρ

– Contours of equal density:

( ) ( )( ) ( )const

yyxxconstyxf

y

y

yx

yx

x

x =−

−−−

−−

⇔= 2

2

2

2 2),(

σµ

σσµµρ

σµ

Ellipse equation:

circle,0y x, toparallelaxes0

⇔==⇔=

yx σσρρ

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 11

Two-dimensional normal distribution: Example

• S11 3dB 0.05GHz: ( )( )

( ) 4573.0,

0001.0,0002.0

0002.0,0024.0

−==

==−==

==−==

yxr

yuy

xux

yy

xx

ρ

σµ

σµ

1.5x10-3

1.0

0.5

0.0

-0.5

Im[S

11]

-3.0x10-3

-2.5 -2.0 -1.5 -1.0Re[S11]

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 12

Analysis IV: 95% Confidence Region

• Ellipse equation:(3dB f=0.05GHz)

),21,2(21

2 222

2

clnnFn

nk −+⋅

−+⋅

= υυ

υ

• Coverage factor k:– k determines confidence level– confidence level (cl) depends on:

• dimension n• degrees of freedom ν

S11 3dB f=0.05GHz:n=2, ν=N-1=10, cl=0.95

k=3.08

( ) ( )( ) ( ) 282210440.0966.1949.4 kxxxxyyyy ⋅=−⋅+−−⋅+−⋅ −

F distribution

0.0015

0.0010

0.0005

0.0000

-0.0005

Im[S11]

-0.0030 -0.0025 -0.0020 -0.0015 -0.0010Re[S11]

0.0015

0.0010

0.0005

0.0000

-0.0005

Im[S11]

-0.0030 -0.0025 -0.0020 -0.0015 -0.0010Re[S11]

0.0015

0.0010

0.0005

0.0000

-0.0005

Im[S11]

-0.0030 -0.0025 -0.0020 -0.0015 -0.0010Re[S11]

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 13

Analysis V: 95% Confidence Region for r and φ

Monte Carlo Method:Create 2-dim gaussian random distribution

22 yxr +=

1.5x10-3

1.0

0.5

0.0

-0.5

Im[S

11]

-3.0x10-3

-2.5 -2.0 -1.5 -1.0Re[S11]

4x10-3

3

2

1

0

coun

ts

3.2x10-3

3.02.82.62.42.22.01.81.6magnitude

8x10-3

6

4

2

0

coun

ts

185180175170phase

=Φ −

xy1tan

3dB f=0.05GHz:

46.00001.0)(0002.0)(

0002.00024.0

−===

=−=

ryuxu

yx

3.22.2

0004.00004.0 )()(

deg7.1750024.0+−

+− =Φ=

=Φ=

uru

r

95% of areabounded

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 14

Analysis V: 95% Confidence Region for r and φ

• Another example

22 yxr +=

=Φ −

xy1tan

S11 Load 446, f=1GHz:

17.000021.0)(00038.0)(

00054.000038.0

===

==

ryuxu

yx

5.762.39

0007.00004.0 )()(

deg4.550007.0+−

+− =Φ=

=Φ=

uru

r

-2x10-3

-1

0

1

2

Im[S

11]

-2x10-3

-1 0 1 2Re[S11]

7x10-3

6

5

4

3

2

1

0

coun

ts

2.0x10-3

1.51.00.5magnitude

6x10-3

5

4

3

2

1

0

coun

ts

150100500-50phase

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 15

Representation in draft A: Graphics

0.030

0.020

0.010

0.000

S11

mag

LAB

01

LAB

02

LAB

03

LAB

04

LAB

05

LAB

06

LAB

07

LAB

08

LAB

09

LAB

10

LAB

11

LAB

12

-200

0

200

phas

e (d

eg)

0.006

0.004

0.002

0.000

-0.002

-0.004

Im[S11]

0.0060.0040.0020.000-0.002-0.004Re[S11]

Device: Load 446, f: 1.000 GHz

© metas, M. Zeier ANAMET 20th Meeting September 2003 slide 16

Representation in draft A: TableLo ad 4 4 61 . 0 0 0 GHz

S 1 1 mag u(mag) phase u(phase) Re Im Flag(deg) (deg)

LAB 0 1 0 .0019 0 .0114 147 .84 180 .00 -0 .0016 0 .0010 1

LAB 0 2 0 .0008 0 .0177 -168.17 180 .00 -0 .0008 -0 .0002 1

LAB 0 3 0 .0024 0 .0075 18.00 180 .00 0 .0023 0 .0007 1

LAB 0 4 0 .0004 0 .0114 -39 .28 180 .00 0 .0003 -0 .0003 1

LAB 0 5 0 .0008 0 .0081 159 .10 11.69 -0 .0007 0 .0003 1

LAB 0 6 0 .0022 0 .0013 25.96 43.82 0 .0020 0 .0010 1

LAB 0 7 0 .0020 0 .0060 56.00 180 .00 0 .0011 0 .0017 1

LAB 0 8 0 .0008 0 .0080 45.77 180 .00 0 .0005 0 .0005 1

LAB 0 9 0 .0023 0 .0111 12.13 174 .86 0 .0022 0 .0005 1

LAB 1 0 0 .0012 0 .0165 143 .13 180 .00 -0 .0009 0 .0007 1

LAB 1 1 0 .0014 0 .0057 92.16 180 .00 -0 .0001 0 .0014 1LAB 1 2 0 .0009 0 .0300 -79 .78 180 .00 0 .0002 -0 .0009 1

S 1 1 mag u(mag) phase u(phase) Re u(Re) Im u(Im)

CRV 0 .0007 +0.0007 55.35 +76.53 0 .0004 0 .0004 0 .0005 0 .0002

-0 .0004 -39 .24

S 1 1 Re Im OutB

M e d ian 0 .0004 0 .0005 0 .0051

Co rre lat io n Re[S11] Im[S11]M at rix Re[S11] 1 .0000 0 .1660

Im[S11] 1 .0000