Embed Size (px)
DESCRIPTION
Repeatability and Reproducibility Analysis of the Round Robin Results. The familiarization of the participating laboratories within the project with the specific sampling system use (“particulates” system) - PowerPoint PPT Presentation
Citation preview
Repeatability and Repeatability and Reproducibility Analysis Reproducibility Analysis
of the Round Robin of the Round Robin ResultsResults
Round Robin Objectives
The familiarization of the participating laboratories within the project with the specific sampling system use (“particulates” system)
The examination of the consistency of the results obtained when applying this methodology in different physical environments
Participating Laboratories
VTT Energy (VTT) – Finland MTC – Sweden IFP – France EMPA – Switzerland Technical University Graz (TUG) –
Austria AEA Technology (AEAT) – UK SHELL – UK Lab of Applied Thermodynamics
(LAT) - Greece
Round robin execution table
Laboratory Arrival DayDeparture
Day
VTT 11/09/2001 24/09/2001
MTC 28/09/2001 05/10/2001
IFP 12/10/2001 22/10/2001
EMPA 02/11/2001 09/11/2001
TUG 09/11/2001 20/11/2001
AEAT 29/11/2001 07/12/2001
Shell 08/12/2001 15/12/2002
LAT 16/12/2002 29/01/2002
Test Vehicle
VW Golf TDI 1.9 l, M Y 1996 with the oxidation catalyst removed
Fuel: Fulfilling the specification EN590 (Year 2000 specs)
Lubricant: BP Vanellus C4 multi SAE 20W – 50 grade (pre aged for 874 km)
Instrumentation
Primary Porous Diluter TYLAN MFC 2920 dilution air mass
flow controller Ageing Chamber Thermodenuder Dekati’s Electrical Low Pressure
Impactor (ELPI) & ELPI Filter Stage Vacuum Pump 2 Ejector dilutors Dekati’s Diffusion Charger (DDC) TSI’s 3022 Condensation Particle
Counter (CPC) Dekati’s Gravimetric Cascade
Impactor (DGI)
Testing
Transient Cycles: NEDC Artemis Road
Steady State Modes: 50 km/h 120 km/h
• (road loads)
Properties measured in round robin
Particulates System Mass emission rate (mg/km) [DGI] Active surface (cm2/cm3) [DDC] Number Concentration (cm-3) [ELPI stages
1 to 8] Number Concentration (cm-3) [ELPI end
filter stage] Mean aerodynamic diameter (nm) Number concentration (cm-3) [CPC] Total number concentration (cm-3) – mean
& median mobility diameters [SMPS] Regulated emissions:
NOx
CO2
CO HC PM
Repeatability – Reproducibility
Analysis proposed in
ASTM E691-99 Standard
Inspection of the repeatability and reproducibility of the results obtained from each laboratory.
Derivation of a precision statement in terms of the total within laboratory and between laboratories variances.
Factors affecting the outcome of a test method
The equipment used
The calibration of the equipment
The operators using the equipment
The environmental conditions
Precision and Accuracy
Accurate but Accurate but imprecise imprecise
resultsresultsTrue True ValueValue
Precise but Precise but inaccurate inaccurate
resultsresults
Within Lab and between
laboratories variability
Within Laboratory Variability: (σ2).
Between Laboratories Variability (στ
2).
Within Laboratory – Between Laboratories Variances reflect: The precision of the test method Any vagueness in the protocol Departures from the protocol
Coefficient of Variance: σ/μ
Repeatable Results
Distribution expected due to w ithin lab variability Case 1 Case 2
AveragAverage valuee value
Reproducible Results
Lab 1 Lab 2 Lab 3 Lab 4
Overall Overall mean mean valuevalue
Repeatability – Reproducibility Statistics and their critical
values
Repeatability Index: k
x… laboratory
Reproducibility Index: h
Critical h* and k* values.
2σx
x
sk
x
22
...xx
n
yyh
h plots by laboratories
(reproducibility)
CO2
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
120
km/h
50 k
m/h
Art
emis
Roa
dN
ED
C C
old
NE
DC
Hot
120
km/h
50 k
m/h
Art
emis
Roa
dN
ED
C C
old
NE
DC
Hot
120
km/h
50 k
m/h
Art
emis
Roa
dN
ED
C C
old
NE
DC
Hot
Art
emis
Roa
dN
ED
C C
old
NE
DC
Hot
120
km/h
50 k
m/h
Art
emis
Roa
dN
ED
C H
ot12
0 km
/h50
km
/hA
rtem
is R
oad
NE
DC
Col
dN
ED
C H
otN
ED
C H
ot12
0 km
/h50
km
/hA
rtem
is R
oad
NE
DC
Col
dN
ED
C H
ot
AEAT EMPA IFP LAT MTC Shell TUG VTT
h s
tati
stic
h plot by driving cycles
CO2
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
AE
AT
EM
PA
IFP
MT
CS
hell
VT
TA
EA
TE
MP
AIF
PLA
TM
TC
She
llT
UG
VT
TA
EA
TE
MP
AIF
PLA
TM
TC
She
llV
TT
AE
AT
EM
PA
IFP
MT
CS
hell
VT
TA
EA
TE
MP
AIF
PLA
TS
hell
VT
T
50 km/h NEDC Hot Artemis Road 120 km/h NEDC Cold
h s
tati
stic
k plots by laboratories
(repeatability)
CPC
0
0.5
1
1.5
2
2.5
3
120
km/h
50 k
m/h
Art
emis
Roa
d
NE
DC
Col
d
NE
DC
Hot
120
km/h
50 k
m/h
Art
emis
Roa
d
NE
DC
Col
d
NE
DC
Hot
120
km/h
50 k
m/h
Art
emis
Roa
d
NE
DC
Hot
Art
emis
Roa
d
120
km/h
50 k
m/h
Art
emis
Roa
d
NE
DC
Hot
120
km/h
50 k
m/h
NE
DC
Col
d
50 k
m/h
NE
DC
Hot
Art
emis
Roa
d
NE
DC
Hot
AEAT EMPA IFP LAT MTC SHELL TUG VTT
k st
atis
tic
k plots by driving cycles
CPC
0
0.5
1
1.5
2
2.5
3
AE
AT
EM
PA
IFP
MT
C
SH
ELL
TU
G
AE
AT
EM
PA
IFP
MT
CT
UG
VT
T
AE
AT
EM
PA
SH
ELL
AE
AT
EM
PA
IFP
LAT
MT
C
VT
TA
EA
T
EM
PA
IFP
MT
C
SH
ELL
50 km/h NEDC Hot NEDCCold
Artemis Road 120 km/h
k st
atis
tic
Repeatability problems in regulated emission results
VTT IFP EMPA
CO2 NEDC cold
CO
HC NEDC cold 50 km/hArtemis & NEDC hot
NOx NEDC cold Artemis
PM Artemis & NEDC hot
Reproducibility problems in regulated emission results
EMPA
SHELL
VTT IFPAEA
T
CO2 50 km/h
All cycles
CO 50 km/h
HC NEDC hot
NEDC cold
NOx Artemis
NEDC cold
PM All cycles
Within Laboratories
Coefficients Of Variance for Regulated Emissions
Results
Within laboratories COV
0%
10%
20%
30%
40%
50%
60%
CO2 CO HC NOx PM
Emission
% C
OV
Within laboratories COV
0%
10%
20%
30%
40%
50%
60%
CO2 CO HC NOx PM
Emission
% C
OV
120 km/ h 50 km/ h Artemis RoadNEDC Cold NEDC Hot
Within laboratories COV
0%
5%
10%
15%
CO2 CO HC NOx PM
Emission
% C
OV
Between Laboratories
Coefficients Of Variance for Regulated Emissions Results
Between Laboratories COV
0%10%20%30%40%50%60%70%
CO2 CO HC NOx PM
Emission
% C
OV
Between Laboratories COV
0%10%20%30%40%50%60%70%
CO2 CO HC NOx PM
Emission
% C
OV
120 km/ h 50 km/ h Artemis RoadNEDC Cold NEDC Hot
Repeatability Problems in Particulates instruments
results
MTC IFP EMPA AEAT SHELL TUG LAT
CPC120 km/h
50 km/h
NEDC hot
DDCNEDC cold
50 & 120 km/h
ELPI 1-8
Artemis &
NEDC cold
50 & 120 km/h
NEDC cold
ELPI filter
Artemis &
NEDCELPI
mean dp
120 km/h
SMPS total conc.
120 km/h
50 & 120 km/h
Reproducibility Problems in Particulates instruments
results
MTC IFPEMP
AAEAT TUG SHELL
CPC120 km/h
50 km/h
NEDC hot
NEDC cold
DDCArtemis
NEDC
120 km/h
ELPI Filter
Artemis
NEDC
ELPI 1-8
Artemis
NEDC hot
& 120 km/h
NEDC cold
ELPI mean dp
50 km/h
NEDC
cold
DGI total mass
NEDC Hot
Artemis
Within laboratories
Coefficients Of Variance for
Particle properties
Within Laboratories COV
0%
50%
100%
150%
200%
% C
OV
Within Laboratories COV
0%
50%
100%
150%
200%
% C
OV
120 km/ h 50 km/ h Artemis RoadNEDC Cold NEDC Hot
Within Laboratories COV
0%
25%
50%
75%
% C
OV
Between Laboratories
Coefficients of Variance for
particle properties
Between Laboratories COV
0%
50%
100%
% C
OV
Between Laboratories COV
0%
50%
100%
% C
OV
120 km/ h 50 km/ h Artemis RoadNEDC Cold NEDC Hot
Comparison of the Within Laboratory
Coefficients of Variance of the
particulates instruments
results with the regulated PM
ones0123456789
10
CO
V/C
OV
PM
0123456789
10
CO
V/C
OV
PM
120 km/ h 50 km/ h Artemis RoadNEDC Cold NEDC Hot
Comparison of the Between Laboratory
Coefficients of Variance of the
particulates instruments
results with the regulated PM
ones0123456789
10
CO
VT/C
OV
T,P
M
0123456789
10
CO
VT/C
OV
T,P
M
120 km/ h 50 km/ h Artemis RoadNEDC Cold NEDC Hot
Conclusions - Recomendation
s
Great variability observed in ELPI results (possibly thermodenuder?).
DDC results were the most repeatable and reproducible.
Possible dilution ratio determination problems.
Indications that the method is robust.
Importance of conducting another round robin now that all laboratories are familiar with the protocol.
![Gauge & R&R [Repeatability & Reproducibility] Analysis](https://img.dokumen.tips/doc/110x75/54becf3e4a7959a67f8b4696/gauge-rr-repeatability-reproducibility-analysis.jpg)
