Upload
luis-erickson
View
215
Download
1
Tags:
Embed Size (px)
Citation preview
INITIALIZATION AND ONGOING ACCEPTANCE
TESTING
In order to use the contractor’s test results for acceptance and payment, ODOT must ensure that the contractor’s test results compare favorably with their own.
Favorable comparability will be determined by means of paired testing of split samples.
INITIALIZATION TESTING
Minimum of 10 tests
1st Lot testing divided into 10 sublots
Optionally, initialization can be made
up of the first 2 lots of 5 sublots
Include plant start-up testing
Each set of asphalt testing includes:
INITIALIZATION TESTING
✓1 AIR VOID TEST
✓1 AC CONTENT TEST
✓3 ROADWAY DENSITY TESTS
At the conclusion of the sampling and associated
testing, a statistical “paired-t test” will be performed by ODOT using the pairs of initialization test data.
If there is a significant bias:
ODOT’s test results will be relied upon until the source of the bias
has been identified and corrected. The lack of bias must then be validated by further “paired-t
testing”
If Independent Evaluation requested:
The Contractor may request evaluation of the testing bias via three-way split sample testing with an independent laboratory.
ODOT will select the independent laboratory.
All initialization testing will be performed by all three laboratories using split samples.
If ODOT testing shows a significant bias:
●The Contractor will not be held responsible for any additional costs
incurred by ODOT in conjunction with the 3-way split-sample testing
●The Contractor’s test results will be relied upon for acceptance and pay
adjustments
If Contractor testing shows a significant bias:
●The Contractor shall reimburse ODOT for any additional costs incurred by
ODOT in conjunction with the 3-way split-sample testing
●ODOT’s test results will be relied upon for acceptance and pay adjustments
If paired-t testing validates the Contractor’s test methods, the Contractor’s test results
will be used for acceptance and pay adjustment subject to:
●ODOT will perform ongoing testing at a minimum frequency of 1 independent test
per lot.
●Ongoing process verification over time will be performed using the F-test and the
t-test.
If ODOT’s testing-bias Control Charts indicate an out-of-control condition
for testing bias:
Both the Contractor and ODOT will immediately investigate the probable
cause.
If the probable cause is identified and corrected or if the resulting bias is, in ODOT’s judgment, not likely to exceed
the Allowable Testing Bias:
Acceptance and pay adjustments will continue to be based on the Contractor’s
results
At ODOT’s discretion, errant test results may be discarded. If replacement test results are
available, they may be substituted for discarded test results
If the probable cause is not corrected or if the resulting bias is, in ODOT’s judgment, likely to exceed the allowable testing bias:
All subsequent testing will be paired testing until the source of the
unaccepatable testing bias is corrected.
During this period, acceptance and pay adjustments will be based on ODOT’s test
results.
GUIDELINES FOR INITIAL VALIDATION OF CONTRACTOR’S TEST METHODS
●A paired test shall consist of 2 separate tests by ODOT and the
Contractor on a split sample
PAIRED TESTING
On split samples test:●Air Voids
●Asphalt Cement Content
For Roadway Densities, perform paired tests on the same cores. The
first lab to test the cores should make sure to oven-dry the cores at
140°F instead of 230°F
GENERAL PROCEDURE
Calculate , the average of thedifferences between the paired tests
pX
STEP 1
Where:
np = Number of paired tests
Xd = ODOT’s individual test result for
Xp = The average of the differencebetween paired tests
Xc = The Contractor’s individual test result for each sample.
each sample.
p
n
1 idc
pn
)X(XX
p
GENERAL PROCEDURE
Calculate Sp, the standard deviation of the differences between the
paired tests
STEP 2
Where:
Sp = The Standard Deviation of the
np - 1 = Degrees of freedom, and
difference between paired tests,
Xc, Xd, np, and Xp as previously defined.
1n
XX(XS
p
n
1i
pdc
p
p
2)
GENERAL PROCEDURE
Calculate tp, the paired-t teststatistic
STEP 3
Where:
tp = The paired-t test statistic, andnp, Xp, and Sp as previously
defined. The vertical lines on either side of the equation indicate the need to take the ABSOLUTE VALUE of the
equation
Simply put, if the result of the calculation is NEGATIVE, change it to
a POSITIVE number
p
p
pp S
Xnt
GENERAL PROCEDURE
Obtain the critical t-value, tcrit, from Table 1 in Appendix A
using np-1 degrees of freedom
STEP 4
GENERAL PROCEDURE
Compare the calculated paired-t statistic (tp) to the critical t-value tcrit
STEP 5
If tp is less than tcrit, there is no statistically significant bias
If tp is more than tcrit, there is a statistically significant bias
Compare the testing bias (Xp) to the associated allowable testing bias (ATB) from
the table in Appendix A:
If tp is more than tcrit
:
AC Quality Char. ATB Units
AC Content ± 0.15 % by weight Air Voids ± 0.50 % by volume
Roadway Density ± 0.50 % of Gmm
Note: ATB’s for aggregates are for control charts and
information only.
If the magnitude of Xp is less than the magnitude of the corresponding ATB, the Contractor’s bias is statistically,
but not practically significant.
Contractor test results shall then be relied upon for acceptance and pay
adjustments
If the magnitude of Xp is greater than or equal to the magnitude of the
corresponding ATB, the Contractor’s bias is practically significant and unacceptable.
Contractor test results shall not be relied upon until certified by a qualified
independent calibration specialist and validated through further paired-t testing.
EXAMPLE - % Air Voids
TEST #
1 2 3 4 5 6 7 8 9
10
Xc 4.14.24.95.85.04.83.75.14.9
4.5
Xd 3.94.45.46.05.14.53.95.44.7
4.7
Step 1 – Average DifferencesTEST #
1 2 3 4 5 6 7 8 9
10
Xc 4.14.24.95.85.04.83.75.14.9
4.5
Xd 3.94.45.46.05.14.53.95.44.7
4.7
Xc - Xd
0.2
-0.2-0.5-0.2-0.10.3
-0.2-0.3
0.2-0.2
Xp = -0.1
Step 2 – Standard DeviationTEST #
1 2 3 4 5 6 7 8 9
10
Xp -0.1-0.1-0.1-0.1-0.1-0.1-0.1-0.1-0.1
-0.1
Xc - Xd - Xp 0.3
-0.1-0.4-0.1
0.00.4
-0.1-0.2
0.3-0.1
Xc - Xd
0.2
-0.2-0.5-0.2-0.10.3
-0.2-0.3
0.2-0.2
(Xc - Xd - Xp)2 = 0.58
(Xc - Xd - Xp)2
0.090.010.160.010.000.160.010.040.09
0.01
Sp = 0.25386
1n
XX(XS
p
n
1i
pdc
p
p
2)
110
0.58Sp
tp = 1.246
p
p
pp S
Xnt
0.25386
0.1-10tp
Degrees ofFreedom(np – 1)
Criticalt-value
(tcrit)
9.925
3.250
23 5.8414 4.6045 4.0326 3.7077 3.4998 3.3559
3.16910
tp = 1.246 < tcrit = 3.250
Since tp < tcrit, there is no statistical bias. Therefore the contractor’s
tests on this quality characteristic are valid and will be relied upon for acceptance and pay adjustments.
ONGOING VALIDATION OF CONTRACTOR’S TEST METHODS
GUIDELINES FOR ONGOING VALIDATION OF CONTRACTOR’S TEST METHODS
●Ongoing process verification over time will be performed using the F-test
and the t-test.
●The F-test provides a method for comparing the variances of the two data
sets.
● The t-test provides a method for comparing the means of the two data
sets.
Contractor
Department
One independent
Contractor: Collect samples to split and test ¼, discard ¼, store 2nd half.
ODOT: Collect one sample per lot, independent of Contractor’s samples.
Test ¼, discard ¼, store 2nd half.
GUIDELINES FOR ONGOING VALIDATION OF CONTRACTOR’S TEST METHODS
●First two lots do not yield enough data for F-test and t-test.
●These lots will be validated by comparing to D2S limits.
● If │Xc – Xd │≤ D2S limit, assumed from same population – Contractor’s accepted.
● If │Xc – Xd │> D2S limit, assumed from different populations – ODOT’s used.
D2S LIMITS
Quality Characteristic D2S Limits
Asphalt Cement Content 0.30%
Air Voids (Lab Molded) 1.40%
Roadway Density 1.40%
THIRD - FIFTH LOT AFTER INITIALIZATION
●Using the F-test and t-test for process verification.
●Use all data after Initialization, including that used for D2S comparisons.
● By third lot after Initialization there should be 15 contractor results and 3
ODOT results.
SIXTH LOTS & FOLLOWING AFTER INITIALIZATION
●Still using the F-test and t-test for process verification.
●Use the current lot results plus the previous four for a total of five lots.
● Each of these analyses should include 25 contractor results and 5 ODOT results.
F-test● Calculate the mean of each test set.
c
cc
n
XX
d
dd
n
XX
Xc = the mean of the Contractor’s results
Xd = the mean of the Department’s results
Xc = individual Contractor result
Xd = individual Department result
nc = the number of Contractor results
nd = the number of Department results
F-test● Calculate the variance of each test
set.
nc - 1 = degrees of freedom of Contractor results
= the variance of the Contractor’s results2cs
nc - 1 = degrees of freedom of Department results
= the variance of the Department’s results2ds
1n
)XX(s
c
2cc2
c
1n
)XX(s
d
2dd2
d
F-test● Calculate the F-statistic.
Use the larger of the variances in the numerator so that the F-statistic is always greater than one.
2d
2c
s
sF
2c
2d
s
sF or
Compare the calculated F-statistic to the critical F-value Fcrit
If F is less than Fcrit, the data is assumed from the same population.
If F is more than Fcrit, the data is assumed from a different population
F-test Determine Fcrit from the F-distribution
table using a 0.01 significance level
t-test
● Calculate the pooled variance, .
Where nc + nd - 2 = pooled degrees of freedom
2ps
2nn
1)(ns1)(nss
dc
d2dc
2c2
p
t-test● Calculate the t-statistic.
d
sp
c
2p
dc
n
s
n
s
XXt
Compare the calculated t-statistic to the critical t-value tcrit
If t is less than tcrit, the data is assumed from the same population.
If t is more than tcrit, the data is assumed from a different population
t-test Determine tcrit from the t-distribution
table using a 0.01 significance level and the pooled degrees of freedom
Both the F-test and the t-test must show the results to be from the same population to
continue accepting the Contractor’s results
If either test shows the results to be from different
populations, ODOT will test the remaining half of
the Contractor’s split samples for acceptance until the reason for the difference is resolved.
EXAMPLE - % Asphalt Content1st Lot after Initialization – D2S Limits
Test # Xc Xd
11 4.18 4.08
12 4.37
13 4.27
14 4.19
15 4.21
After initialization, ODOT and Contractor samples are no longer split
samples but obtained independently
Step 1: Compute Averages
Test # Xc Xd
11 4.18 4.08
12 4.37
13 4.27
14 4.19
15 4.21
Avg. 4.244 4.08
Step 2: Compute D2S Comparison
S2DX-X dc
30.008.4244.4 - (D2S=0.30 from
Appendix A Part 2 for Asphalt Cement content)
True 30.0164.0
● Since is true, we assume that the test results come from the same population.
The Contractor’s results are accepted.
D2SX- dc X
EXAMPLE - % Asphalt Content2nd Lot after Initialization – D2S Limits
Test # Xc Xd
12 4.21
16 4.14
17 4.27
18 4.21
19 4.14
20 4.27
Step 1: Compute Averages
Test # Xc Xd
12 4.21
16 4.14
17 4.27
18 4.21
19 4.14
20 4.27
Avg. 4.206 4.21
Step 2: Compute D2S Comparison
30.021.4206.4 -
True 30.0004.0
● Since is true, we assume that the test results came from the same population.
The Contractor’s results are accepted.
D2SXX dc
S2DX-X dc (D2S=0.30 from
Appendix A Part 2 for Asphalt
Cement content)
EXAMPLE - % Asphalt Content3rd Lot after Initialization – Begin F & t
Test #
Xc XdTest
#Xc
Test #
Xc
11 4.18 4.08 16 4.14 21 4.19
12 4.37 4.21 17 4.27 22 4.21
13 4.27 4.11 18 4.21 23 4.14
14 4.19 19 4.14 24 4.00
15 4.21 20 4.22 25 4.50
Step 1: Compute Mean for Each Set
c
cc
n
XX
d
dd
n
XX
4.224.219333315
63.29Xc
4.134.13333333
12.40Xd
Step 2: Compute s2 (Variance) for Each Set
1n
)XX(s
c
2cc2
c
1n
)XX(s
d
2dd2
d
Step 3: Compute sc2
Test # Xc(Xc-Xc)2
11
12
4.08
4.21
0.0025
0.0064
Sum
Sum/(nc-1)
25 4.50 0.0784
63.29 0.1797
0.1797 / (15-1)
0.01284sc2
Xc ≈ 4.22
Step 4: Compute sd
Test # Xd (Xd-Xd)2
11 4.08 0.0025
12 4.21 0.0064
13 4.11 0.0004
Sum 0.0093
Sum/(nd-1)
0.0093/(3-1)
sd 0.00465
2
2
Xd ≈ 4.13
Step 5: Compute F-statistic
2d
2c
s
sF
2c
2d
s
sF or
F NumeratorDenominat
orResult
1 0.01284 0.00465 2.76
2 0.00465 0.01284 0.36
Note: Pick F with largest numerator (highest result).
Step 6: Pick Fcrit from F-distribution (2-tailed, =0.01) Table
n-1 13 14 15
2 199 199 199
3 43.3 43.2 43.1
n-1 2 3
13 8.19 6.92
14 7.92 6.68
15 7.70 6.48
For this example, contractor variance
was in the numerator with 14 degrees of
freedom and ODOT’s variance was in the denominator with 2
degrees of freedom. Read the table like this
Not like this!
Step 7: Perform F-test
● Since F-test is true, we must perform
the T-test.
● Had F-test been false, the tests would not have come from the same population. The
Contractor’s results would not be accepted.
F ≤ Fcrit ? 2.75 ≤ 199 = True
Step 8: Compute Pooled Variance
2ps
2nn
1)(ns1)(nss
dc
d2dc
2c2
p
2315
1)0.00465(31)-0.01284(15S2
p
0.0118s2p
Step 9: Compute t-statistic
d
sp
c
2p
dc
n
s
n
s
XXt
30.0118
150.0118
4.134.22t
1.31t
Note: Without rounding, t ≈ 1.25.
Step 10: Pick tcrit from t-distribution (2-tailed, =0.01) Table
DF tcrit
15 2.947
16 2.921
17 2.898
• Degrees of freedom = nc + nd – 2
DF = 15 + 3 – 2 = 16
Step 11: Perform t-test
● Since both t-test and F-test are true, we assume
that the test results came from the same population. The Contractor’s results are
accepted.
t ≤ tcrit ? 1.31 ≤ 2.921 = True