87
National Analytical Management Program (NAMP) U.S. Department of Energy Carlsbad Field Office In Cooperation with our University Partners Radiochemistry Webinars Data Verification and Validation

Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

  • Upload
    ngokiet

  • View
    228

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

National Analytical Management Program (NAMP)

U.S. Department of Energy Carlsbad Field Office

In Cooperation with our University Partners

Radiochemistry Webinars Data Verification and Validation

Page 2: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Meet the Presenter… Tom Rucker

2

Contact information: Phone: (865)481–2993 Email: [email protected]

Dr. Rucker is the manager of the Radiological Assessment and Protection business area for Science Applications International Corporation (SAIC) in Oak Ridge, TN. Dr. Rucker earned a PhD in Analytical Chemistry (Radiochemistry Emphasis, Health Physics Minor) in 1988 from the University of Tennessee. Dr. Rucker joined SAIC in 1987 as a Senior Radiochemist, where he now provides technical and project leadership for radiological characterization, assessment, and protection services. Dr. Rucker specializes in radionuclide measurement and dose/risk assessment for human health and environment protection. He also provides expertise in other areas of analytical chemistry, environmental chemistry, and health physics including environmental monitoring, waste management, and nuclear safeguards and security. Before joining SAIC, Dr. Rucker served as a Research Associate at the Oak Ridge National Laboratory from 1984 to 1987 where he provided radiochemistry procedures and radiation counting facilities development and upgrade. Dr. Rucker also worked at the Oak Ridge Gaseous Diffusion Plant from 1977 to 1984 as Group Leader of the Radioanalysis Group. Dr. Rucker has extensive experience in radioanalytical data evaluation, validation, and management and served as co-chairman of the ANSI/ANS-41.5-2012 writing group.

Page 3: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Verification and Validation of

Radiological Data for Use in Waste

Management and Environmental

Remediation

Thomas Rucker, PhD

National Analytical Management Program (NAMP)

U.S. Department of Energy Carlsbad Field Office

TRAINING AND EDUCATION SUBCOMMITTEE

Page 4: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

ANSI/ANS-41.5-2012 Sponsorship

• American Nuclear Society (ANS)

−Nuclear Facilities Standards Committee (NFSC)

Subcommittee on Decommissioning and Site Remediation Standards (ANS-23)

−Manages the development and maintenance of standards that address the cleanup of radioactive materials and radioactivity mixed with hazardous substances.

4

Page 5: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

ANS-41.5 Working Group Members • S. R. Salaymeh (Chair), Savannah River National Laboratory (Retired)

• T. L. Rucker (Co-Chair), Science Applications International Corporation

• A. E. Rosecrance, Oilfield Environmental Compliance

• D. E. McCurdy, Independent Technical Consultant

• J. E. Chambers, Fluor-B&W Portsmouth

• D. W. Poyer, U.S. Army Center for Health Prevention and Preventive Medicine

• C. King Liu, U.S. Department of Energy

• J. G. Griggs, U.S. Environmental Protection Agency

• J. C. Jang, U.S. Nuclear Regulatory Commission (Deceased)

• P. D. Greenlaw, U.S. Department of Homeland Security

5

Page 6: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

A 20-year Process • Working group first assembled 1992.

− Volunteers wrote sections that were reviewed and revised at working group meetings once or twice a year.

• Initial draft completed in 2002 and submitted to ANSI Committee for review.

− Working subgroup reviewed committee comments and revised the draft at meetings once or twice a year.

• Obtained ANS committee approval February 2012.

• Published August 2012 and available for purchase from the ANS Store: http://www.new.ans.org/store/i_240288

6

Page 7: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Contents

Section 1 – Purpose and scope

Section 2 – Acronyms and definitions

Section 3 – General principles

Section 4 – Sample-specific parameters

Section 5 – Batch control parameters

Section 6 – Instrument parameters

Section 7 – Personnel qualifications

7

Page 8: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Appendices Contents

Appendix A – Discussion of Data Life Cycle and Supporting Documents and Information

Appendix B – Recommended Validation Report Contents

Appendix C – Discussion of Compliance Verification and Validation Parameters

Appendix D – Explanation of Equations for Verifying Compliance to Required Sample-Specific Detection Level

Appendix E – Explanation of Equations for Decision Level and Detection Decisions

8

Page 9: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Purpose of the Standard • Provide requirements (shall statements) and

recommended practices (should statements) for determining the validity of radioanalytical data for waste management and environmental remediation.

• Applications include site characterization, waste acceptance, waste

certification, waste treatment

design, process control, litigation,

and other applications requiring

data verification and validation.

9

Page 10: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Purpose of the Standard (Cont.)

• Provide a minimum set of checks and tests that will ensure a consistent approach for compliance verification and validation of data produced by any radioanalytical laboratory.

• Eliminate many of the inconsistencies in the approaches, evaluation algorithms, parameters evaluated, qualifiers, and qualifications of validators used in existing site-specific data compliance verification and validation programs.

10

Page 11: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Scope of the Standard • The standard applies to radioanalytical data for

waste management and environmental remediation.

• The standard applies to data generated by field measurements and radioanalytical laboratories, which require independent review as specified by the data quality objectives (DQOs).

• Some of the elements of the standard may apply to non-destructive assay and in situ measurements.

• The standard does not apply to non-radioassay measurement methods (i.e., ICP-MS, KPA, X-ray diffraction, etc.).

11

Page 12: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Scope of the Standard (Cont.)

• Applies only to independent compliance verification and validation processes and should not be construed to apply to any actions taken by laboratories to internally generate or review data, including audits and performance evaluation studies.

• However, this standard expects that certain laboratory quality control (QC) and programmatic quality assurance (QA) measures have taken place that feed data for review by the data verifiers and validators, including results of audits and performance evaluation studies.

12

Page 13: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Scope of the Standard (Cont.) • Acceptance criteria for the test and checks are

intentionally not provided in most cases.

−Waste management or environmental remediation programs or projects may have unique measurement quality objectives (MQOs) based on the intended use of the data.

−The Standard assumes a DQO process has been used by the project to define the quality of data needed for the decision process and to develop corresponding MQOs of accuracy, precision, sensitivity, selectivity, and representativeness to be met. The DQO process should also provide guidance for the frequency, percentage, and extent of data validation.

13

Page 14: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Data Life Cycle

14

Page 15: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Compliance Verification

• The systematic process of checking data for completeness, correctness, consistency, and compliance with written analytical specifications (e.g., SOW, contract, project plans). (Does not assume or require laboratory work be performed under contract.)

• Verification evaluates all aspects of compliance and attempts to resolve non-compliance.

• Verification does not provide qualification of the data.

• Non-correctable non-compliance items are flagged and forwarded to the validation step in a verification report.

15

Page 16: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Validation

• Process of examining a verified data package to provide a level of confidence in the reported analyte’s identification, concentration (including detection status), and associated measurement uncertainty.

• Analyte- and sample-specific, and extends beyond the method or written analytical specification (e.g., SOW, contract, project plans) compliance.

• Produces a data set with a limited number of qualifiers associated with the result based on the data’s fitness (suitability) for their intended use, as defined by the MQOs and DQOs.

• Results are documented in a validation report and forwarded for use in data quality assessment.

16

Page 17: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Validation Qualifiers • The standard recommends use of traditional EPA

Contract Laboratory Protocol qualifiers.

• The actual qualifiers and associated reasons assigned to each result shall be recorded in an organized manner for final evaluation and reporting.

• One result may receive multiple (even repeated) qualifiers, each with its own reason.

• If qualifiers are combined for recording purposes, the rules should be developed during project planning and addressed in the validation plan.

17

Page 18: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Validation Qualifiers (Cont.)

• <none>: The analyte has been detected and any problems that exist are minor or irrelevant to the intended use. The uncertainty in the result is fairly represented by the reported uncertainty.

• U: Undetected. The analyte result is less than the critical level.

• J: Estimated. An unusually uncertain or biased, but usable, result. The uncertainty associated with the result significantly (relative to the MQOs) exceeds the reported uncertainty.

• R: Unusable. The problems are so severe that the data cannot be used because they significantly affect the decisions based on them.

18

Page 19: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Use of Audit Information • The standard anticipates that specific

information gathered and evaluated during audits will be available and referenced during the compliance verification and validation processes, rather than contained in the data package for evaluation.

• Audit information may include generic audit items, items reviewed during a post-award lab audit, or items reviewed on a regular basis for updates based on a desk audit.

• Desk audits may be included in the validation process if included in the validation plan.

19

Page 20: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

20

Page 21: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Generic Audit Information

• Radiochemical procedures

• Equations

• Initial Instrument calibrations

• NIST traceability for equipment and standards

• Historic internal QC and external performance testing sample results

• When an audit report is available, deficiencies are evaluated during verification for corrective actions and during validation for data qualification relative to MQOs.

21

Page 22: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Results of External PE Programs to be

Forwarded to the Verifier

• PE program results reviewed during compliance verification and used as a feedback mechanism to the laboratory to correct any major deficiencies.

• Data verifier should verify that participation meets the requirements of the SOW.

22

Page 23: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Results of External PE Programs to be

Forwarded to the Validator

• The magnitude of the bias and precision shown by the PE program should be viewed in terms of project-identified action levels and the magnitude of the sample data results.

• External PE program results cannot be applied to qualify any one batch of samples, but the validator may make recommendations to the data quality assessor based on the PE results and their effect on overall data usability.

23

Page 24: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Questions?

24

Page 25: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample Specific Parameters • Sample Preservation • Holding Times • Sample-Specific Chemical Yield • Required Detection Limit • Nuclide Identification • Quantification and CSU • Detectability • Sample Aliquot

Representativeness

25

Page 26: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample Preservation

• Audit Information

−Relevant procedures

−Sample preservation documentation

−Documentation on calibration and maintenance of relevant equipment or instrumentation (refrigerators, pH meters, thermometers, etc.)

26

Page 27: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample Preservation (Cont.)

• Compliance Verification

−Review laboratory data sheets and/or chain-of-custody records for evidence of sample preservation.

27

Page 28: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample Preservation (Cont.)

• Validation

−If sample preservation requirements were not followed, all affected sample results are questionable. Qualify all affected sample results as either estimated (J) or unusable (R), depending on the magnitude of the uncertainty introduced compared to the established MQOs.

28

Page 29: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Holding Times

• Audit Information

−Relative procedures

−Sample documentation

• Compliance Verification

−Determine total elapsed time from date of sample collection to date of analysis.

− If this time exceeds the specified holding time for a given nuclide or matrix, a notation should be made in the verification report.

29

Page 30: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Holding Times (Cont.)

• Validation

−Determine if the results were adversely effected by exceeded the holding time.

−Qualify all affected sample results as either estimated (J) or unusable (R), depending on the magnitude of the uncertainty caused by the holding time exceedance compared to the established MQOs.

30

Page 31: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample-specific Chemical Yield

• Audit Information

−Relevant procedures

−Sample documentation

−Relevant QC data and corrective action on outliers

−Certification and traceability of tracers

−Tracer preparation log

−Chemical yield trending

31

Page 32: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample-specific Chemical Yield (Cont.)

• Compliance Verification

−If the chemical yield does not meet the method or project requirements, a notation should be made in the verification report.

32

Page 33: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample-specific Chemical Yield (Cont.)

• Validation

−Calculate the uncertainty of the chemical yield.

−If the tracer uncertainty (1 σ) is > 10% (or other limits as specified by the APS or MQOs), qualify the sample result as estimated (J) unless the tracer uncertainty has been propagated into the reported measurement uncertainty. If chemical yield is greater than 110%, qualify the sample result as estimated (J) or unusable (R) based on the amount of bias allowed by the MQOs.

33

Page 34: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Required Detection Limit

• Audit Information

−SOPs

−Documentation for the equations and experimental data from which the typical values of the parameters are obtained for calculation of the Lc.

34

Page 35: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Required Detection Limit (Cont.)

• Compliance Verification

−For each result that is less than Lc, test to determine if the RDL has been met

k • CSU ≤ RDL ,

where:

CSU is the combined standard uncertainty.

k is 3.5 for most applications assuming α and β probabilities of 0.05 each. However, the appropriate value can vary depending on the number of background counts (see MARLAP) .

−If this test is not met, a notation is made in the compliance verification report.

35

Page 36: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Required Detection Limit (Cont.)

• Validation

−If RDL is not met, note in the validation report that the RDL has not been met.

−If the result is less than Lc and the result plus 1.65 times its CSU is greater than the action level, qualify as unusable (R).

36

Page 37: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Nuclide Identification • Audit Information

−Documentation on measured resolution of the various detectors and the achieved process alpha, beta, and gamma-ray resolutions for typical final sample mounts.

−Spectral or mathematical unfolding routines/algorithms used in the identification of radionuclides.

−Basis and/or mathematical algorithms for energy determinations of alpha, beta, and gamma-ray spectra.

37

Page 38: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Nuclide Identification (Cont.)

• Compliance Verification

−Verify that the raw spectral data and/or peak search and identification reports have been included in the data package for each analysis, if required by the SOW or other planning documents.

−Exceptions are noted in the compliance verification report.

38

Page 39: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Nuclide Identification (Cont.)

• Validation

−The alpha, beta, or gamma-ray spectra are inspected for the following determination, if required by the validation plan:

Obvious misidentification due to improper position of peaks, nonlinear energy response or skewed spectral peak positions.

Unresolved multiple peaks overlapping peak interferences.

Degradation of resolution resulting from improper sample mounts or final geometry.

Quenching of liquid scintillation solutions insufficient counts in the peak for proper peak centroid.

39

Page 40: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Nuclide Identification (Cont.)

• Validation (Cont.)

−For alpha spectrometric applications involving radiotracers, the resolution and centroid position of the peak associated with the radiotracer are evaluated, if required by the validation plan.

−Independent calculations are performed from instrument QC data to verify the detector resolution and energy calibration parameters (gain and offset values) of the spectrometry system, and the peak centroid energy, if required by the validation plan.

40

Page 41: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Nuclide Identification (Cont.)

• Validation (Cont.)

−If the analyte has been misidentified or its identification is highly questionable, the results are qualified as unusable (R).

−If there is a possibility of several radionuclides present in the sample and the energy resolution of the measurement does not permit proper identification, the affected results are qualified as unusable (R).

41

Page 42: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Nuclide Identification (Cont.)

• Validation (Cont.)

−If the quench of a sample being counted by liquid scintillation is severe and no corrections have been made for energy correction, the affected results are qualified as estimated (J) or unusable (R), depending on the severity of the problem.

−If the energy resolution of the alpha spectral measurement has deteriorated to the point that multiple radionuclide peaks overlap significantly, the affected results are qualified as estimated (J) or unusable (R), depending on the severity of the problem.

42

Page 43: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Quantification and CSU • Audit Information

− Detailed radiochemical and/or radiometric procedures.

− Documentation of procedure validation.

− Equations used to calculate the analytical result, CSU, MDC, and Lc or DL.

− Documentation relative to the expected range and boundary values for various parameters used in the quantification process and the calculation of the CSU and a priori MDC.

− Verification of accurate transfer of information from analytical instrument to a database or reporting system.

43

Page 44: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Quantification and CSU (Cont.)

• Compliance Verification

−“Spot” checks (a percentage defined by the validation plan) are performed to evaluate:

The occurrence of transcription errors.

The consistency between hard-copy and electronic data submissions.

The quantification calculations by independent calculations if required by the verification and validation plan.

44

Page 45: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Quantification and CSU (Cont.)

• Compliance Verification (Cont.)

−All quantification data and calculation parameters are verified against requested analyses and reporting requirements.

45

Page 46: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Quantification and CSU (Cont.)

• Compliance Verification (Cont.)

−The raw data are reviewed to ensure the following:

Procedures and equations are consistent with those required in the written analytical specifications or validated in the audit.

Correct dates and time intervals are used in the equations for radioactive decay and ingrowth.

46

Page 47: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Quantification and CSU (Cont.)

• Validation

−For parametric, spectral resolution, and calculation outliers/errors, the data are qualified as estimated (J) or unusable (R), depending on the magnitude of the bias introduced compared to the established MQOs.

−For concentrations greater than ten times the MDC, if CSU > 0.25 • RS then there is excessive uncertainty in the measurement and further review may be necessary. An estimated (J) qualifier may be assigned to the sample data.

47

Page 48: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Quantification and CSU (Cont.)

• Validation (Cont.)

−If the net negative result is more negative than the 2σ CSU, there is a negative bias resulting from improper background subtraction and the data shall be qualified as estimated (J) or unusable (R), depending on the magnitude of the error introduced compared to the established MQOs.

48

Page 49: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Detectability • Audit Information

−When Lc is calculated by the laboratory, the equation and data used for its derivation are normally reviewed during the audit or desk audit.

−When an audit report is available, Lc calculation deficiencies identified are evaluated relative to the MQOs during verification and validation to determine when corrective actions and/or data qualification are to be performed.

−When documentation of changes in the method used to calculate Lc is provided in the data package, the experimental data and equations from which the Lc values are obtained are reviewed.

49

Page 50: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Detectability (Cont.)

• Compliance Verification

−When Lc is not calculated by the laboratory, the sample CSU may be used for its derivation:

or by using a set of blank data:

• Validation

− If the analyte concentration is found to be less than Lc, an undetected (U) qualifier shall be applied to the data result.

Lc = 1.65 • CSUR

Lc = [(t • SB) + RB] / (E • R • IDF • W)

50

Lc = Critical Level; CSU = Combined Standard Uncertainty; t = student t factor; Sb = standard deviation of set of blanks; Rb = average blank count rate; E = fraction detector efficiency; R = fractional chemical yield; IDF = ingrown/decay factor; W = weight or volume

Page 51: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample Aliquot Representativeness

• Audit Information

−Subsampling procedures and sizes and homogenization methods.

• Compliance Verification

− Verify that required homogenization techniques and aliquot sizes were used.

− Exceptions are noted in the compliance verification report.

51

Page 52: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Sample Aliquot Representativeness (Cont.)

• Validation

−Results from different but comparable analytical techniques from different subsample aliquots of the same sample should be compared for consistency.

−If the laboratory did not follow the required homogenization techniques or use the required aliquot sizes qualify the affected results as estimated (J) or unusable (R), depending on the magnitude of the uncertainty introduced compared to the established MQOs.

52

Page 53: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Questions?

53

Page 54: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Batch Control Parameters • Laboratory Control Sample

• Matrix Spike Sample

• Duplicate and MS Duplicate Sample Analysis

• Batch Method Blank Analysis

54

Page 55: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Batch Control Parameters (Cont.)

• Batch = A group of samples prepared at the same time, by the same analyst, in the same location, and using the same method.

55

Page 56: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Laboratory Control Sample

• Audit Information

−Algorithm used to calculate the LCS percent difference (LCS %D).

−Quality control charts.

56

LCSM = LCS measured value; LCSE = LCS expected value

Page 57: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Laboratory Control Sample (Cont.)

• Compliance Verification

−Review the results for each batch to ensure that the required number or frequency of LCSs was included with the sample batch.

−Review the results for each batch to determine if the percent difference was within the QC acceptance limits.

−Exceptions are noted in the compliance verification report.

57

Page 58: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Laboratory Control Sample (Cont.)

• Validation

−If LCSs were not performed at the frequency specified in the MQOs, qualify the data for all samples analyzed with the batch as estimated (J).

−If the percent difference for the LCSs was not within the QC acceptance limits as established in the MQOs (accuracy), qualify the data for all samples analyzed with the batch as estimated (J).

58

Page 59: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Matrix Spike Analysis

• Audit Information −Algorithm used to calculate the MSS percent

difference (MSS %D).

−Quality control charts.

59

SSR = Sample spike result; SR = Sample result; SA = Spike activity

Page 60: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Matrix Spike Analysis (Cont.)

• Compliance Verification

−Review the results for each batch to ensure that the required number or frequency of MSSs was included with the sample batch.

−Review the results for each batch to determine if the percent difference was within the QC acceptance limits.

−Exceptions are noted in the compliance verification report.

60

Page 61: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Matrix Spike Analysis (Cont.)

• Validation

−If MSSs were not performed at the frequency specified in the MQOs, qualify the data for all samples analyzed with the batch as estimated (J).

−If the percent difference for the MSSs was not within the QC acceptance limits as established in the MQOs (accuracy), qualify the data for all samples analyzed with the batch as estimated (J).

61

Page 62: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Duplicate and MSD Sample Analysis

• Audit Information

−Algorithm used to calculate the relative percent difference (RPD) and duplicate error ratio (DER).

−Quality control charts.

62

S = Sample result; D = Duplicate result; CSUS = Combined standard uncertainty of sample; CSUD = Combined standard uncertainty of duplicate

Page 63: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Duplicate and MSD Sample Analysis (Cont.)

• Compliance Verification

− Review the results for each batch to ensure that the required number or frequency of duplicates and/or MSDs was included with the sample batch.

− Review the results for each batch to determine if the duplicates (laboratory, field, or MSD) were within the QC acceptance limits. If the RPD was not within QC limits, verify that the DER is within limits set by the DQO process.

A limit of 2 provides a 5% false conclusion rate and a limit of 2.58 provides a 1% false conclusion rate.

− Exceptions are noted in the compliance verification report.

63

Page 64: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Duplicate and MSD Sample Analysis (Cont.)

• Validation

−If laboratory duplicates and/or MSDs were not performed at the frequency specified in the MQOs, qualify the data for all samples analyzed with the batch as estimated (J).

−If the precision for the duplicates (laboratory, field, or MSD) was not within the QC acceptance limits as established in the MQOs (precision), qualify the data for all samples analyzed with the batch as estimated (J).

64

Page 65: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Batch Method Blank Analysis • Compliance Verification

−Review the results for each batch to ensure that the required number or frequency of blanks was included with the sample.

−Review the results for each batch to verify the batch method blank is less than 1.65 • CSU (when background is subracted) and/or within control limits (when the method blank is subtracted).

−Exceptions are noted in the compliance verification report.

65

Page 66: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Batch Method Blank Analysis (Cont.) • Validation

−If batch method blanks were not performed at the frequency specified in the MQOs, qualify the data for all samples analyzed with the batch as estimated (J).

−If the net batch method blank result was not less than 1.65 • CSU or was outside control limits (depending on applicable test):

Qualify the results for all samples analyzed with the batch that are less than ten times the net batch method blank value as estimated (J).

Qualify the results for all samples analyzed with the batch that are less than the sum of the net batch method blank and its 1.65 CSU as undetected (U).

66

Page 67: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Questions?

67

Page 68: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Instrument Parameters • Counting Efficiency Calibration

• Efficiency Calibration

• Background Determination

68

Page 69: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Counting Efficiency Calibration • Audit Information

−Date of efficiency calibration and date that the new calibration factors were effective.

−Standard preparation log for the counting geometry standard with traceability to a certified reference material or standard.

−Certificate for certified reference material or standard.

−Counting time for the standard.

−Raw count results for the standard.

−Calculations showing derivation of the counting efficiency factor or statistical curve fit.

69

Page 70: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Counting Efficiency Calibration (Cont.)

• Audit Information (Cont.)

−If the laboratory recalculates the efficiency calibration after the audit and documentation of changes is provided in the data package, the experimental data and equations from which the efficiency calibrations are obtained are reviewed via a desk audit or during data validation.

70

Page 71: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Counting Efficiency Calibration (Cont.) • Compliance Verification

− Verify that the instrument’s most recent efficiency calibration was performed at the required frequency.

− Verify that efficiency performance checks are analyzed prior to the counting of samples each day that samples are counted.

− Verify performance check count-rate results data are within properly established tolerance limits (based on system performance and analytical MQOs) or that recalibration was performed whenever the limits were exceeded.

The limits are related to the mean count-rate value established at the time of calibration for each detector.

− Evaluate check source counting statistics to verify that the counting uncertainty (1σ) was less than or equal to one-fifth of the MQO.

71

Page 72: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Counting Efficiency Calibration (Cont.)

• Validation

− If the specified efficiency calibration and/or verification frequency is not followed, the efficiency or quench curves are not smooth, or the QC performance check results fall outside the appropriate tolerance limits, qualify the results for all samples analyzed between acceptable calibration verifications as estimated (J) or unusable (R), depending on the magnitude of the error based on the established MQOs.

− When significant errors are found in the calculation, qualify all affected results as either estimated (J) or unusable (R), depending on the magnitude of the error based on the established MQOs.

72

Page 73: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Energy Calibration • Audit Information

− Date of energy calibration and date that the new calibration factors were effective.

− Certificate for calibration standard.

− Peak centroid for all peaks used for calibration.

− Procedure and calculations showing derivation of the energy calibration gain and offset factors or other curve-fit parameters.

− Voltage, gain, and cross-talk calibration data, depending on specific instruments.

− Discriminator or region of interest setting determinations.

− Energy resolution (full-width at half-maximum) calibration data for spectroscopy systems.

73

Page 74: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Energy Calibration (Cont.)

• Audit Information (Cont.)

−If the laboratory recalculates the energy calibration after the audit and documentation of changes is provided in the data package, the experimental data and equations from which the energy calibrations are obtained are reviewed via a desk audit or during data validation.

74

Page 75: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Energy Calibration (Cont.) • Compliance Verification

−Verify that the instrument’s most recent energy calibration was performed at the required frequency.

−Verify that energy performance checks are analyzed prior to the counting of samples each day that samples are counted.

−Verify performance check peak centroid or calculated energy is within properly established tolerance limits (based on system performance and analytical MQOs), or that recalibration was performed whenever the limits were exceeded.

The limits are the energy tolerance used for peak identification for the samples.

75

Page 76: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Energy Calibration (Cont.)

• Validation

− If the specified energy calibration and/or verification frequency is not followed, the efficiency or quench curves are not smooth, or the QC performance check results fall outside the appropriate tolerance limits, qualify the results for all samples analyzed between acceptable calibration verifications as unusable (R) if the error is great enough to cause misidentification of the radionuclide (outside the peak identification energy tolerance limit).

− When significant errors are found in the calculation, qualify all affected results as either estimated (J) or unusable (R), depending on the magnitude of the error based on the established MQOs.

76

Page 77: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Background Determination • Audit Information

−Date of background determination and date that the new factors were effective (for calculation).

−Counting time for the background determination.

−Raw background count results.

−Calculations showing derivation of the background count-rate factor.

−Review data fit when multiple factors are used to produce a background (i.e., quench) curve.

77

Page 78: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Background Determination (Cont.)

• Audit Information (Cont.)

−Review of background individual energy ranges for spectroscopy measurements.

−If the laboratory recalculates the background values after the audit and documentation of changes is provided in the data package, the experimental data and equations from which the background values are obtained are reviewed via a desk audit or during data validation.

78

Page 79: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Background Determination (Cont.)

• Compliance Verification

− Verify that the instrument’s most recent background determination was performed each time there is a significant instrument operational and at the required frequency as stated in the SOW or QAPP.

− Verify that background performance checks are analyzed at the required frequency as stated in the SOW or QAPP.

− Verify background performance check count rates are within properly established tolerance limits (based on system performance and analytical MQOs) or that redetermination was performed whenever the limits were exceeded.

The limits are related to the mean count-rate value established at the time of background determination for each detector.

79

Page 80: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Background Determination (Cont.) • Validation

− If the specified background determination and/or verification frequency is not followed, the background quench curves are not smooth, or the QC performance check results fall outside the appropriate tolerance limits, qualify the results for all samples analyzed between acceptable background verifications as either estimated (J) or unusable (R), depending on the magnitude of the error based on the established MQOs.

− When significant errors are found in the calculation, qualify all affected results as either estimated (J) or unusable (R), depending on the magnitude of the error based on the established MQOs.

80

Page 81: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Questions?

81

Page 82: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Personnel Qualifications

• Verifier

• Validator

• Auditor

82

Page 83: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Verifier Qualifications

• A high school diploma or AA degree.

• 2 years of radiochemical laboratory experience including chemical separations, nuclear instrumentation, and record keeping.

• Familiarity with radiochemical, nuclear instrumentation, and QC procedures.

83

Page 84: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Validator Qualifications

• BS or BA degree in chemistry or related physical sciences or engineering disciplines.

• 3 years of radiochemical laboratory experience including sample preparation, radiochemical procedures, and measurement instrumentation.

• 2 years of experience in data interpretation and review.

• Familiarity with the DQO process and statistical concepts, inferences, interpretation, and tests.

84

Page 85: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Auditor Qualifications

• BS or BA degree in chemistry or related physical sciences or engineering disciplines (years of related experience may substitute).

• 4 years of radiochemical laboratory experience including sample preparation, radiochemical procedures, and measurement instrumentation.

• 3 years of experience in data interpretation and review.

• Completion of internal or external auditor training.

• Familiarity with the DQO process and statistical concepts, inferences, interpretation, and tests.

85

Page 86: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Questions?

86

Page 87: Data Verification and Validation - HOME - ICLN Data... · data verification and validation. 9 . Purpose of the Standard (Cont.) •Provide a minimum set of checks and tests that will

Upcoming Webinars in the

Environmental/Bioassay

Radiochemistry Series

• EPA Incident Response Guide and Rapid Methods Overview • Traceability and Uncertainty • Subsampling • Mass Spectrometry • Gamma Spectrometry (Parts I & II) • Radiobioassay • GUM