SANAS Page 1 of 12

TG 26-01

CRITERIA FOR VALIDATION OF METHODS USED BY CHEMICAL LABORATORIES IN THE COAL, OIL

PETROLIUM, METALS AND MINERALS INDUSTRY

Approved By:

Acting Chief Executive Officer: Ron Josias Senior Manager: Christinah Leballo

Date of Approval: 2009-08-07 Date of Implementation: 2009-08-07

TG 26-01

SANAS Page 2 of 12

CONTENTS:

1. Purpose and Scope

2. References, Definitions and Abbreviations

3. Introduction

4. Performance Characteristics and Criteria of a Test Method

5. Validation Plan

6. Implementation and Review

7. Summary Report

8. Guidelines for Assessors

9. Definitions

ADDENDUM 1: Amendment Record

TG 26-01

SANAS Page 3 of 12

1. Purpose and Scope

The purpose of this document is to define the concepts and processes of method validation, and to provide practical guidelines in order to facilitate a uniform approach. This document amplifies ISO/IEC 17025:2005 requirements and lists SANAS requirements applicable to chemical laboratories. It is not intended as a training guide. The following do not form part of this document: Verification, sampling, sample handling and transportation.

2. References, Definitions and Abbreviations 2.1 References

a) Eurachem Guide. The Fitness for Purpose of Analytical Methods. A laboratory guide to method validation and related topics. Copyright LGC (Teddington) Ltd 1998

b) ISO/IEC 17025. General Requirements for the competence of testing and calibration laboratories.

c) Ludwig Huber, Validation and Qualification in Analytical Laboratories, second edition. Ludwig Huber. Agilent Technologies. Waldbronn, Germany.

d) TG 41-01 “Recommended guidelines for the verification and validation of methods in forensic chemistry”.

e) Quantitative Chemical Analysis Second Edition Gilbert H. Ayres, Prof of Chemistry, University of Texas at Austin

f) PANCAL Guidance for the Validation of Test Methods. Can-P-1629. November 2009. Standards Council of Canada.

g) Skoog et al, Fundamentals of Analytical Chemistry, 7th Edition, 1996.

2.2 Definitions

2.1.1 Accuracy - The accuracy of an analytical method is the extent to which test results

generated by the method and the true value agree. Accuracy can also be described as the closeness of agreement between the value that is adopted, either as a conventional, true, or accepted reference value, and the value found. (3)

2.1.2 Precision - The closeness of agreement between independent test results obtained under stipulated conditions (how close the measured values are to each other). It is usually expressed as the standard deviation or relative standard deviation (co-efficient of variance) and may be a measure of either the degree of reproducibility and /or repeatability (1).

Figure 1: Graphic presentation of the difference between precision and accuracy.

From http://elchem.kaist.ac.kr/vt/chem-ed/data/graphics/acc-prec.gif)

TG 26-01

SANAS Page 4 of 12

� Precision, in general, is a complex property involving variation between laboratories, variation within laboratories, and laboratory/sample interaction. Method precision is a key indication of method ruggedness and should include as many variables as will be encountered during the daily application of the method.

� The required precision for a method is determined by the role the test results are

going to play in making the decision regarding the release of the product. Precision may not be relevant to an identification test, but if may be highly relevant to an assay.

� Under normal operating circumstances a reference material is not required. All

that is needed, from a reference standpoint, is something that produces the same response each time the method is applied to it.

� There are different types of precision studies:

• Instrument precision or injection repeatability • Repeatability or intra-assay precision • Regression precision • Ruggedness • Intermediate precision • Reproducibility

� It must be noted that the RSD is usually high at low concentrations and low at

higher concentrations. 2.2.2.1 Instrument Precision

i) A typical criterium for instrument precision would be RSD ≤ 1%. For

an impurity method, at the limit of quantitation, the instrument precision would be RSD ≤ 5%.

Example: Injection repeatability

ii) The measurement of instrument precision is most easily made by

injecting replicate aliquots of the same solution (minimum of ten). The response ratios derived from those injections are determined, and their RSD indicates the instrument’s precision.

iii) The repeatability of replicate injections of the analytical solution is

best expressed as the relative standard deviation.

2.2.2.2 Intra-assay Precision (Repeatability) i) Intra-assay precision is obtained by repeatedly analyzing aliquots of a

homogeneous sample, each of which has been independently prepared according to the method procedure.

ii) The analysis is carried out in one laboratory by one operator, using

one piece of equipment, over a relatively short time span (normally one day).

iii) A typical criterium for repeatability would be RSD ≤ 2%. For an

impurity method, at the limit of quantitation, the instrument precision would be RSD ≤ 10%.

2.2.2.3 Regression Precision

i) A regression line has errors in the slope and the intercept and

when an unknown concentration (x0) is determined by using the regression line, a regression error (Sx0) is present.

TG 26-01

SANAS Page 5 of 12

(Sx0)

Regression precision (RSD) = x0 x 100 %

ii) A typical criterium for regression precision would be RSD < 2.5%.

2.2.2.4 Ruggedness

i) Ruggedness is normally expressed as the lack of influence on test results of operational and environmental variables of the analytical method. Ruggedness is a measure of reproducibility of test under normal, expected operational conditions from laboratory to laboratory and from analyst to analyst.

ii) A rugged method is one that has built in buffers against typical

abuses, that is, against differences in care, technique, equipment, and conditions.

2.2.2.5 Intermediate precision

i) The intermediate precision of an analytical method is determined by

analysis of aliquots from homogeneous lots by different analysts, using operational and environmental conditions that may differ but are still within the specified parameters of the assay. For example: multiple instruments, different sources of reagents, different chromatographic columns and multiple days in one lab.

ii) If a true (accepted) value is not available apply the paired t test and

F test to the results generated by different analysts.

iii) If the calculated F value exceeds the tabulated F value at the selected confidence level, then there is a significant difference between the variances of the two analysts.

iv) The statistical calculated paired t value shall not exceed the tabulated

value at the desired confidence level.

2.2.2.6 Reproducibility

i) It is determined by testing homogeneous samples in multiple laboratories (inter laboratory studies).

ii) Evaluation of reproducibility results often focuses more on measuring

bias in results than on determining differences in precision alone.

iii) This reproducibility may be compared to the precision of the assay under normal conditions to obtain a measure of the ruggedness of the analytical method.

TG 26-01

SANAS Page 6 of 12

2.2.3 Bias – It is the tendency of a method towards delivering a result that is skewed from the true value. It is the difference between the experimental mean and the true value and is generated from a total systematic error as contrasted to random error. There may be one or more systematic error components contributing to the bias.

2.2.4 Reproducibility - This refers to replicate analysis performed using the same method on identical test items using different operators and/or instruments and/or laboratories over a longer interval of time

2.2.5 Repeatability - This refers to replicate analysis performed using the same method on identical test items using the same operator, the same instruments and within the same laboratory over a short interval of time.

2.2.6 Linearity - Ability of a method to obtain test results proportional to the concentration of the analyte within a given working range.

2.2.7 Working range - The range of an analytical method is the interval between the upper and lower levels of an analyte, including these levels that have been demonstrated to be determined with a suitable level of precision, accuracy and linearity, using the method as written. The working range is normally expressed in the same units as the test results obtained by the analytical method.

2.2.8 Limit of Detection (LOD) - The lowest concentration of analyte that can be detected but not necessarily quantitated under the stated conditions of the test. It is a point at which a measured value is larger than the uncertainty associated with it.

2.2.9 Verification - Confirmation by examination and provision of objective evidence that specified requirements have been fulfilled 1).

2.2.10 Uncertainty of Measurement (Measurement Uncertainty) - Parameter associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand (1).

2.2.11 Limit of Quantification LOQ - The lowest concentration of analyte that can be determined with acceptable precision (and accuracy) under the stated conditions of the test (4).

2.2.12 Sensitivity - Capability of the method to discriminate between small differences of concentrations of analyte.

2.2.13 Specificity / Selectivity - The ability of a method to respond to a particular analyte of interest in the presence of possible interferences such as impurities, degradents and matrix effects.

Figure 2: The Definitions for linearity, range, LOQ and LOD displayed.

TG 26-01

SANAS Page 7 of 12

Figure 3: The definitions for LOD and LOQ displayed: Limit of detection and limit of quantitation via signal to noise.

3. Introduction

3.1 Method validation is the process of establishing the performance characteristics and limitations of a method and the identification of the influences which may change these characteristics and to what extent. It is also the process of verifying that a method is fit for purpose, i.e. for use for solving a particular analytical problem (1).

3.2 Verification refers to a process that provides evidence that the laboratory can achieve the performance characteristics given in a specific analytical method, especially accuracy and precision, and demonstrating that the method is suitable for the intended use. The extent and nature of such verification work depends on the needs of the customer, and the intended use (4).

3.3 It is preferable to use this guideline, but if an individual laboratory uses a different approach, it is their responsibility to prove the validity of their approach, with the necessary literature references and/or historical data. Validation is always a balance between costs, risks and technical possibilities.

3.4 For clarification on when to perform validation or verification see Table 1. This Table has been adapted from PALCAN Guidance for the Validation of Test Methods (3).

Table 1: When should methods be validated? (3)

Test method description Validation or verification requirements

Standard published method.

Confirmation of published performance characteristics (verification) in accordance with the requirements of ISO/IEC 17025:2005 section 5.4.2.

Standard published method plus additional documentation for optional steps.

Full validation may be required only if changes made.

In-house developed method. Full validation (See Section 5).

Method published in the scientific literature without any performance data.

Full validation (See Section 5).

Methods published in scientific literature with performance data.

Confirmation of published performance characteristics (verification) but more likely full validation required.

Changes in implementation of previously validated methods – i.e. changes to

Extent of validation will vary to demonstrate change does not have a significant impact on performance

TG 26-01

SANAS Page 8 of 12

equipment, reagents, lab environment or staff.

characteristics.

Standard published method applied to different matrices, different concentration ranges, analytes or standard published method used for a similar purpose but different conditions.

Validation is required and the extent will vary, e.g. having similar properties to those of representative matrices and analytes.

Archived standard published or previously validated method that is reinstated.

Confirmation of previous performance characteristics (verification).

Ad hoc or special analyses (technique accreditation).

Extent of validation limited by circumstance.

Commercial Test Kits – collaboratively tested, third party evaluation (e.g. AOAC).

Confirmation of published performance characteristics (verification) but validation may be required if and changes are made or the matrices differ.

Commercial Test Kits – no performance data available, incomplete or not applicable.

Validation.

4. Performance characteristics and criteria of a test method.

4.1 Performance characteristic “means functional quality that can be attributed to an analytical method” (EC Directive). Examples of typical performance characteristics include: selectivity, accuracy, trueness, recovery, precision, repeatability, reproducibility, detection limit, limit of quantitation, detection capability, ruggedness and stability. The laboratory may also be required to evaluate sampling, subsampling and transportation of samples to the laboratory as part of the validation plan.

4.2 Performance criteria “means requirements for a performance characteristic according to which it can be judged that the analytical method is fit for the purpose and generates reliable results.” (EC Directive).

5. Validation Plan

5.1 The scope of the method and its validation criteria should be defined early in the process. These include the following questions:

i) Purpose of measurement (what analytes should be detected and why?).

ii) What are the sample matrices?

iii) Are there interfering substances expected, and, if so, should they be detected and quantified?

iv) Are there any specific legislative or regulatory requirements?

v) How robust should the method be?

vi) Measurement scope (What are the expected concentration levels?)

vii) Are there specific equipment accommodation and environmental conditions that need to be considered?

viii) Which type of equipment should be used? Is the method for one specific instrument, or should it be used by all instruments of the same type?

TG 26-01

SANAS Page 9 of 12

ix) Method used for the sample preparation, sub-sampling, procedure and including instruments to be used.

x) Identification of performance characteristics:

• Accuracy

• Bias

• Precision

• Reproducibility

• Repeatability

• Linearity

• Working Range

• Limit of Detection

• Limit of Quantification

• Sensitivity

• Specificity

• Uncertainty of Measurement

Note: How test method performance characteristics are evaluated, along with the criteria against which they will be assessed, are usually described in your standard published methods, scientific literature or equipment specifications. It is therefore the responsibility of the laboratory, with input from clients, to seek out the relevant characteristics to be evaluated with respect to the laboratory’s specific situation and client’s needs. The laboratory must have a documented validation plan, either to be used generally or applied to a specific project or client. Test method performance characteristics to be evaluated will vary with the type of test and its intended use. Discipline-specific or client required performance criteria are to be applied to demonstrate fitness for purpose. (3)

xi) Experimental design.

• Approaches that can be followed: Select a suitable technique for determining the performance of the methods, which could be one of or a combination of the following (2):

� Calibration using reference standards or reference materials.

� Comparison of results achieved with other methods.

� Inter-laboratory comparisons.

� Systematic assessment of the factors influencing the result.

� Assessment of the uncertainty of the results based on scientific understanding of the theoretical principles of the method and practical experience.

• Guidelines on how to do the design can also be found in the literature, e.g. Eurachem Guide, The fitness for Purpose of analytical Methods (1) and Validation and Qualification in Analytical Laboratories, second edition. Ludwig Huber (3).

Note 1: Ensure measurement traceability of the critical measuring equipment & standards

Note 2: Ensure that data is trended and reviewed to ascertain whether customer requirements are met, whether methodology requires changes or corrective action is required.

TG 26-01

SANAS Page 10 of 12

Note 3: There are many cases in which the range and uncertainty of the values (e.g. accuracy, detection limit, selectivity, linearity, repeatability, reproducibility, robustness and cross-sensitivity) can only be given in a simplified way due to lack of information. (Reference 2)

6. Implementation and Review

6.1 Analyse the data applying the appropriate statistical tools, e.g. Analysis of Variance (ANOVA tool, on Excel, for statistics), linear regression, t- test, f- test, etc.

Note 1: Ensure that conventions regarding use of significant figures and rounding data is applied before analysing the data. Remember that when rounding data from chemical computations it may be necessary to carry one extra digit through all the computations to avoid rounding error. In rounding a number ending in 5, always round so that the result ends with an even number. (7)

Note 2: Units give dimensions to numbers therefore do not forget your measurement units.

6.2 Make a statement on fitness for purpose.

6.3 Keep records of the validation, including raw data and other suitable evidence.

6.4 The final validation report shall contain conclusions, summaries of experimental data and calculations substantiating each of the applicable analytical performance parameters.

7. Summary Report

7.1 The laboratory must have available for review a report, summarizing all the detailed method validation data for all non-standard, in-house developed or modifications and amplifications of standard published methods. The report should include:

7.2 The test method as validated. This includes information about equipment, reagents, calibration etc. (Confusion may arise if the method does not meet performance criteria and further method development is required).

7.3 Reference to the validation procedure or plan used to generate the test method performance characteristics.

7.4 A summary of the test method performance characteristics and how these were calculated or defined. The raw data should also be available for review.

7.5 The test method performance criteria against which the characteristics were evaluated and whether or not the method is fit for purpose.

7.6 The intended use of the method.

7.7 Estimates of uncertainty based on interpretative documents of the ISO GUM (for example, the EURACHEM CITAC Guide).

7.8 If the method that is not a standard published method is used routinely, it is expected that over time there will modifications or improvements made. This information needs to be documented and available for assessment. Ongoing proficiency testing data and quality control data should be reviewed by the laboratory to confirm the fitness of the method.

7.9 SANAS requires is that all validation data should be readily available in the laboratory for a minimum of at least one assessment cycle (5 year cycle).

8. Guidelines for Assessors

What should assessors be looking for?

• How are test methods selected by the laboratory?

TG 26-01

SANAS Page 11 of 12

• Is the laboratory knowledgeable about best practices for validation in the applicable discipline and do they have access to relevant documents? Is the client providing any information?

• Does the laboratory have procedures for assuring the quality of test results generated by test methods used in ad hoc/ non-routine testing?

• Who is assigned responsibility for validations? Are the staff trained in conducting validations and evaluating data packages?

• Is there a separation in the technical records between method development and validation?

• Is the validation documentation package complete?

• Is there evidence that the method has been successfully transferred to routine use, transferred to another laboratory or undergone some type of peer review, where appropriate?

• Is there a process to review performance data generated for methods in routine use to demonstrate to clients ongoing fitness for purpose?

• Is the method declared fit for purpose? (6)

TG 26-01

SANAS Page 12 of 12

ADDENDUM 1: AMENDMENT RECORD Proposed By: Section Change

FM All New Document