EUROPEAN COMMISSION DIRECTORATE-GENERAL CLIMATE ACTION Directorate C - Climate strategy, governance, and emissions from non-trading sectors CLIMA.C.2 - Governance & Effort Sharing

AV Training Handbook

(covering four scenarios on verification issues)

This handbook is intended for use in the training of verifiers involved in verification of

GHG annual emissions reports under the EU ETS, for (lead) assessors of accreditation

bodies responsible for oversight and witnessing of verifiers as well as for practitioners of

competent authorities responsible for the review of GHG emissions reports and

verification reports.

This handbook has been composed on the basis of four scenarios and their model answers developed for

the 2017 Accreditation and Verification Training Event that took place on the 7th and 8th September 2017

in Brussels, Centre Albert Borschette. It includes an introduction (overview) of the event, the outline and

content of the four scenarios, the model answers on the questions of the four scenarios, as well as the

programme of the 2-day training event and presentations outlining each of the scenarios at the start of

each session.

1

I. OVERVIEW

This handbook is intended as guidance for training of verifiers, (lead) assessors of national accreditation

bodies (NABs) as well as for practitioners from Competent Authorities (CAs) in the area of Verification and

Accreditation for theEU ETS (European Union Emission Trading System). It was prepared for the

Accreditation & Verification (A&V) Training Event organised by the EU ETS Compliance Forum secretariat

on behalf of the European Commission.

The motivation for this handbook and the 2017 Accreditation & Verification (A&V) Training Event was to

discuss some of the more complex issues associated with EU ETS verification. The training aimed at

providing an up to date and shared understanding of the following main topics:

▪ The scope, detail and types of checks performed during verification (for both data validity and rules

compliance);

▪ Minimum expectations in relation to verifier site visits;

▪ The type of checks to perform on specific M&R issues (e.g. distinctions between the role of the

verifier and that of the CA in relation to, for example the operator’s risk assessment, control

procedures, unreasonable cost claims, CEMs, biomass and evidence for acceptance of non-accredited

laboratories;

▪ Potential for limitation(s) of verification scope and consequences for conclusions expressed in the

verification opinion statement.

The verification issues raised during the training were explored and explained on the basis of four case

studies (scenarios) and a set of questions related to each scenario. The training started with issues related

to a Power Plant - this is outlined in Scenario 1. Scenarios 2 and 3 considered verification issues for two

Cement Plants and Scenario 4 an Oil Refinery.

The four scenarios and related questions were developed to cover a range of interrelated verification

issues. However, each scenario and the questions listed under its content description should be

considered on their own, implying that information provided for Scenarios 2-4 is not relevant for

answering questions under Scenario 1, etc.

The training was arranged to allow exchange of views and discussion between the participants on the

subjects covered by the training. Each participant was assigned to one of seven Discussion Groups, and

each Discussion Group consisted of a balanced mix of representatives from CAs, verifiers and NABs from

different MS so as to maximise the training, exchange of experience and learning from the event. On

average each discussion group was made up of 3-5 verifiers (including the trainers), 3-4 practitioners from

the CAs and 1-2 representatives from NABs.

Each Discussion Group was invited to address the verification questions and issues listed for each

scenario. Each session of the training started with a short introduction to the scenario and the questions

pertaining to that scenario that the participants were then asked to resolve in their Discussion Groups.

Before the event the scenario information had been made available to each participant. All participants

had been advised to study each scenario carefully and consider in detail the questions formulated for

each one, and to come to the training well prepared to address and resolve the questions formulated for

each of the four scenarios. They were advised to acquaint themselves with the requirements in the MRR

and AVR and to study the relevant guidance documents. The regulations and guidance documents can be

found on the EU ETS MRV website of the European Commission:

https://ec.europa.eu/clima/policies/ets/monitoring_en#tab-0-1

Participants were instructed to ask themselves “does the verifier have the information needed to answer

the question, should the verifier ask the operator for additional information, or should the verifier do

2

additional tests to ensure that it is able to address the issue.” However, the information provided for each

scenario should be sufficient to allow the participants to answer the questions.

The agenda of the 2-day training event (see Annex II) contained different sessions. Session I started with

an introduction to Scenario 1 and the issues to address for that scenario (scope of verification, areas of

risk and specific checks to be done). Session II dealt with Scenarios 2 and 3, in particular the verifier’s risk

analysis & verification plan; non-accredited labs; and checks relevant to the CA and verifier on key

elements, while Scenario 4 (session III, Day 2) discussed issues concerning on-site verification, CEMS,

uncertainty analysis and scope limitations. All the scenarios considered more broadly the questions of

scope and minimum expectations of what the verifier should do when on site.

At the end of each session, the Discussion Groups were invited to summarise their key findings on the case studies from which the conclusions of the training event were formulated.

At the end of day I a plenary discussion was organised on dealing with complaints about verifiers (e.g. when is an issue identified by the CA to be considered as a formal complaint to the NAB in accordance with Article 72(2) AVR concerning non-compliance of the verifier with the AVR; what feedback should ideally be provided by the NAB to the CA etc.). The main conclusions from that session are incorporated in Chapter V of this handbook.

This handbook consists of the following chapters:

I. Overview

II. Outline and content of the four scenarios

III.1 Model answers for Scenario 1, i.e. instructions to trainers

III.2 Model answers for Scenario 2, i.e. instructions to trainers

III.3 Model answers for Scenario 3, i.e. instructions to trainers

III.4 Model answers for Scenario 4, i.e. instructions to trainers

III.5 Suggested priorities for addressing questions in scenarios

IV. Main conclusions from the plenary session on information exchange

V. Main conclusions and summary of the AV training event

Annex I Programme of 2017 EU ETS Training Event on Accreditation and Verification

Annex II Introductory presentations to explain Objectives and Aims of the Accreditation and

Verification Training

Annex III Presentations from the plenary session on information exchange

Suggestions for the use of this handbook. To maximise the benefits of this handbook, verification bodies, NABs and CAs are advised to make the scenarios (case studies) and the questions for each scenario available to their staff. The trainers are advised to make use of the model answers (instructions to trainers) provided in chapter III.

3

II. Outline and content of the four scenarios

Scenario 1: Power Generation

The installation: The installation is a power station generating electricity for export to the national grid; the installation is mainly solid fuel fired. It is regulated as a combustion installation under Annex I of the Directive and produces around 2,800,000 tonnes CO2 per year.

The equipment for electricity generation comprises:

three dual fuel units of 530MW which can burn coal, HFO and “solid/liquid biomass” which feed into the steam turbines; and

three smaller turbines of 17.5MW each which run on gas oil or bioliquid.

In addition to the equipment for electricity generation there is a propane-fired heating system installed in the workshop area; and three fixed fire-fighting pumps to supplement water pressure and flow in the event of an emergency. These pumps are powered by engines which burn gas oil.

Very small quantities of gas oil and propane are used on site by portable combustion units used during maintenance.

Fuels listed in the approved MP:

Coal is delivered to site by rail and goes to the main stockpiles from which it is taken and blended to meet operational/pollution control requirements before being sent to bunkers for milling and delivery to the generation units. (MAJOR SOURCE STREAM)

HFO is delivered to site by rail and occasionally by road tanker and stored in bulk tanks with a smaller quantity transferred to a day tank for immediate use; HFO is used for boiler start up and to stabilise combustion. (MINOR SOURCE STREAM)

Gas Oil is delivered to site by road tanker and stored in bulk tanks. (DEMINIMIS SOURCE STREAM)

“Solid biomass” –is delivered by road truck and varies according to availability, but is normally logs, woodchip or sawdust; with occasionally oat husk and other agricultural wastes. (DEMINIMIS SOURCE STREAMS)

“Liquid Biomass” is delivered to site by road tanker and stored in bulk tanks as a blend with their respective fossil equivalent fuels. Tall Oil and Cashew Nut Oil are fired as a substitute for HFO and Rape Seed Oil as a substitute for Gas Oil, whenever available. (DEMINIMIS SOURCE STREAMS)

The outline and boundary of the Power Plant for Scenario 1 are provided below

4

Fuel accounting:

Coal delivered to site by railcar is weighed in on delivery using calibrated rail weighbridges; a stockpile survey is conducted quarterly to determine density and volume of coal in stock; and coal sent to the generation Units is weighed on belt-weighers on the conveyor to the Units being fired, enabling a mass balance to be determined for coal stock accounting. A representative sample of coal is taken using automatic samplers on the conveyor belt to the Units being fired.

“Solid biomass” delivered to site by road trucks is weighed in on delivery using calibrated road weighbridges; and when sent to the Units is weighed on belt-weighers on the conveyor to the Units being fired; a representative sample is taken on delivery and when transferred to the bunkers for use in order to determine the moisture content and calorific value.

Heavy Fuel Oil (HFO) and “liquid biomass” delivered to site by rail or road tanker are weighed into site on calibrated rail/road weighbridges; bulk tank levels are measured before and after deliveries using an independent third party to take tank dips and accounting for fuel density and temperature using tank tables with a reference temperature of 15oC. In addition : o Changes in bulk tank stocks are measured by automatic level & temperature gauges, and

cross-checked by manual tank dips at the end of each month. o Representative samples of liquid fuels are taken upon delivery into the storage tanks and at

the end of each month when the tanks are dipped by an independent third party. A mass balance is maintained for the “liquid biomass” blends in order to determine the amount of fossil and non-fossil fuel burnt at any one time.

Gas Oil is delivered by road and consumption is calculated based on weighbridge values which are cross checked back to information on delivery notes from the supplier.

Consumption (Activity Data) of all solid fuels during power generation is calculated based on the heat content of the fuel delivered to the generation Units and the generation output of the Units.

Consumption (Activity Data) of all liquid fuels is based on a stock balance using the data collected from deliveries and tank level changes.

Emissions factors and NCV for all fuels (except gas oil) are determined based on the results of laboratory analysis of samples taken as outlined above; these samples are sent for analysis at an external laboratory to determine the composition and NCV. Gas Oil emissions factor and NCV are taken from the National Inventory default values.1

In order to determine the oxidation factor for the solid fuels, ash from the generation Units is also sampled and sent for analysis at an external laboratory to determine its residual carbon content.

Data from the weighbridges and laboratories are handled as follows:

Rail weighbridges – reading is taken in the Movements control room by the operator and logged on the Delivery sheet, then manually entered into the Fuel stock & Energy accounting database.

Road weighbridges – readings are transferred daily as an electronic spreadsheet which is cross checked by the administrative team to the supplier’s invoices and uploaded to the database.

Sample results are received by the administrative team, checked off against the sample register and entered into the database.

Data is downloaded from the database into an excel spreadsheet to calculate the CO2 emissions for reporting.

1.0 Questions:

a) What questions should a verifier ask itself during the strategic analysis to determine the depth and scope of verification for this year? What elements are important given the information in the context?

b) Given the context in this case, what inherent and control risks should the verifier identify and on which elements should the verifier focus its verification checks? Please indicate the reasons why the verifier should focus on these elements and what questions the verifier should ask itself when identifying the risks and determining the scope.

1 Standard factors used by the Member State for its national inventory submission to the Secretariat of the United Nations

Framework Convention on Climate Change.

5

c) Which specific checks should a verifier do on the monitoring methodology and the control activities

a. how should a verifier do those checks?

b. Which checks should be done on site during the visit?

c. How should the verifier check that MRR requirements are being met?

d) List the documents the verifier should ask for copies of and what checks should they make in relation to those documents

e) What data should the verifier ask for and what specific checks should they do on the data provided? How would the verifier perform these checks?

f) What should the verifier consider when preparing for the site visit?

a. Who should be included in the site visit?

g) What checks should the verifier make in relation to the installation’s sampling plan?

h) What checks should the verifier make in relation to the “solid biomass” and “liquid biomass”?

a. Do verification activities change if the operator claims that the liquid alternative fuels are bio-liquids with zero-rated emissions for the verification?

6

Scenario 2: Cement Manufacture 1 – Non-accredited lab

The Installation

The installation produces cement clinker in a rotary kiln with an annual production of 615,000 tonnes of CO2.

Description of processes and activities

Limestone (calcium carbonate and magnesium carbonate) is the basic raw material in cement making. It is taken from quarries located adjacent to the main factory; where it is quarried, crushed and stockpiled according to grade. Clay is combined with the limestone in a tertiary crusher to reduce the materials to 1.5 cm or less. The mixture is transported to the pre-blend dome where it is blended in semi-circular beds before being sent to raw milling where additives are mixed with the limestone/clay in mills and pulverized to a fine powder called raw ‘meal’. The raw meal is further blended in homogenising silos. A representative sample of material is taken from the silos.

The raw meal is then fed into the top of the preheater tower where it is swirled through with hot kiln exhaust gases. This pre-heats the material before it enters the kiln so that the necessary chemical reactions to convert the carbonates to oxides and CO2 will occur more quickly and efficiently. This 'hot' material then travels through the rotary kiln for approximately 20-25 minutes reaching 1500oC at which stage the material converts to nodules of mainly calcium silicate – the clinker. The red hot clinker is partially cooled on grates, then over air tubes in gravity coolers; after cooling it is stored in steel silos while it awaits transport to the finishing mills.

A rotating ball mill and finish mills, equipped with high efficiency separators, are used to process clinker and a small amount of gypsum is added to control setting times of the cement. Storage silos hold the finished cement until shipped. Most cement is sold in bulk and shipped by rail or truck; only about 5-10% is sold in sacks.

Management, quality control and control room operations are directed from the Plant office which also contains the small plant laboratory.

The site has a range of small equipment that uses LPG (F6) and Gas Oil (F7) as well as an emergency generator and Gas Oil fired fire pumps.

Fuels

In addition to the Gas Oil and LPG, the kiln is able to co-fire a range of different fuels including:

Solid Liquid

(F1) Coal (F3) Kerosene*

(F2) Pet Coke (F9) Waste solvent

(F4) Waste Tyres Gaseous

(F8) Sewage pellets* (F5) Natural Gas

(F10) Municipal waste derived fuel *Deminimis Source Streams

The fuel used to fire the kiln depends on availability and economics at the time of operation.

The outline and boundary of the Cement Plant for Scenario 2 is provided on the next page.

Materials and Fuel accounting:

The fuels are weighed into site over calibrated road and rail weighbridges and Activity Data is determined using a stock balance method. With the exception of tyres, a representative sample is taken of these fuels and sent to the lab to determine NCV, carbon content and Emissions Factor (EF). The EF of tyres uses a country specific default value due to the difficulty in obtaining a representative sample.

For deminimis fuels (F3) an estimate is applied or a non-calibrated flow meter is used to determine activity data and national default values are used for the calculation factors.

Sewage pellets (F8) are 100% solid biomass and the operator has used an EF of zero.

The approved MP refers to internal procedure(s); within these procedures the following items are outlined:

The operator determines Activity Data associated with the clinker as follows:

a) Cement despatched is weighed over calibrated road and rail weighbridges and a stock balance is done taking account of any imported cement; the result of the stock balance is the manufactured cement (CEM1).

b) Cement extenders (additives) used are weighed over calibrated road weighbridges (CE1). c) Limestone extender (additive) is weighed using calibrated weighing equipment (LE1). d) The clinker used for cement production (CL2) is calculated using the following formula :

CL2 = CEM1-[CE1 adjusted for stock balance]

e) Clinker produced (CL1) is calculated using the following formula :

CL1 = CL2 + clinker opening stock + clinker exports – clinker imports- clinker closing stocks

All cement Kiln dust and bypass dust are recycled.

The calculation factors are determined from samples as follows:

Clinker – a daily sample is taken from which a monthly composite is produced and sent to the lab for analysis of MgO and CaO concentrations.

Chalk and raw meal- – a daily sample is taken from which a monthly composite is produced and sent to the lab for analysis of CaO and MgO content, carbon dioxide content and total organic carbon

Coal, petcoke and waste derived fuels (F1, F4 and F10)– representative samples are taken from each delivery along with a record of the moisture of the delivered fuel; a weekly and monthly composite samples are created and sent to the lab for analysis of - calorific value; and % content of carbon, ash, moisture, biomass, non-biomass and inert mass.

Samples taken are analysed in the installation’s internal lab; this lab is not accredited to EN ISO 17025 and the operator has agreed with the competent authority that it can be used on the grounds of unreasonable cost to obtain accreditation and the lack of alternative accredited labs in the region. However, the operator has been able to convince the competent authority that its own corporate Management System as applied to the internal laboratory will enable it to produce valid laboratory results.

2.0 Questions:

a) What is included in the installation boundaries and thus subject to monitoring under EU ETS? How should the verifier check the installation boundaries?

b) What questions should a verifier ask itself during the strategic analysis to determine the depth and scope of verification for this year? What elements are important and on which areas should the verifier focus given the information in the above context? Is there any other information that the verifier would need?

c) Using the attached risk analysis template2, identify the main inherent and control risks associated with the data flow described in this scenario; then annotate with a verification plan covering:

i. What checks a verifier should do on the data and the monitoring methodology? State which of those should be done by the verifier on site?

ii. What type/size of sample should the verifier check?

d) On the basis of the information provided, list the specific control processes, procedures and documents you would expect to find on site? Please list in the verification plan the procedures, processes and control activities identified and state what checks the verifier would do on these? Which of those ’checks’ does the verifier need to do on site? Which can the verifier ignore based on them being checked and approved by the CA?

2 You have been provided with a template for the Risk Assessment and Verification plan based on the European Commission’s

exemplar. Please note that this is not a template for a complete verification plan as this should contain more information.

1

e) How does the verifier check representativeness of samples taken of major source streams before these are sent to the laboratory? On what areas should the verifier focus when performing checks on the sampling plan?

f) What checks should the verifier do regarding the use of a non-accredited lab?

i. What impact would this have on the scope of the verification and the time allocated to complete the verifier’s work?

Data Flow & Accounting area3 Inherent Risk 4 (High/Medium/Low)

Controls Risk5 (High/Medium/Low)

Verification Risk6 (High/Medium/Low)

What will verifier check?7 What type/size of sample?

State what area of the data flow is being considered

State what the inherent risk is and whether it is likely to be H/M/L risk

Describe the (likely) activity to control the inherent risk and state how it might potentially break down and whether this is a H/M/L risk

State in what way a verifier might reach an incorrect conclusion in this area and whether it is likely to be H/M/L risk

State what you would plan to check, what type of activity/item you would sample and how large a sample you would check

Example : manual meter readings of Minor Source stream

Incorrect read Incorrect logging of read etc.

Substitute personnel not trained Failure to use formal log sheet Failure to conduct regular data validation checks etc.

Medium – source is reasonably significant; lack of documented procedures; personnel turnover etc.

Interview operative Review training provided Check period opening and closing readings to calculate total; compare to recorded information Compare to delivery invoices etc. – 100%

3 The part of the accounting data flow being considered e.g. meter reads; sampling; maintenance of instruments etc.? 4 What is the risk inherent in the accounting element if there is no other control in place? 5 Where controls are in place (make assumptions about reasonable likely controls) - where and how could they breakdown and fail to control the inherent risk? 6 What is the risk that the verifier might reach an incorrect conclusion for this area based on the information provided? 7 This is the verifier’s plan – what to check includes who to interview, what to inspect, documents to review and data set(s) to evaluate

1

Data Flow & Accounting area3 Inherent Risk 4 (High/Medium/Low)

Controls Risk5 (High/Medium/Low)

Verification Risk6 (High/Medium/Low)

What will verifier check?7 What type/size of sample?

State what area of the data flow is being considered

State what the inherent risk is and whether it is likely to be H/M/L risk

Describe the (likely) activity to control the inherent risk and state how it might potentially break down and whether this is a H/M/L risk

State in what way a verifier might reach an incorrect conclusion in this area and whether it is likely to be H/M/L risk

State what you would plan to check, what type of activity/item you would sample and how large a sample you would check

Delegates should review the EC’s Risk Assessment & Plan exemplar and consider likely areas of the data flow described in the scenario above; examples of risk areas include: * Materials delivery * Consumption calculations * Laboratory * Instrument maintenance

2

Scenario 3: Cement Manufacture 1 – Accredited lab

The Installation

For this scenario, the installation is as described in Scenario 2 and as shown in the diagram above with the exception that now:

The site operates an ISO9001 quality management system that undergoes external, accredited, certification; alongside this is a certified ISO14001 environmental management system; both systems are integrated with overall business governance and management processes. All company management systems are subject to regular external audits (corporate or certification body); internal audits of issues and processes; and periodic reviews to ensure that they are fit-for-purpose.

The lab has now undergone a full assessment to obtain ISO17025 accreditation for its laboratory for the following tests:

o Purity of calcium carbonate and magnesium carbonate o Total organic carbon content of input limestone o Carbon content and NCV of coal, pet coke, waste solvent

3.0 Questions:

a) What difference does this make to the depth and scope of verification, and the type of checks to be made for this year?

b) What elements are important to be checked given the information in the above context?

c) What questions should the verifier ask of lab and other personnel? and what documents and data should the verifier check and test?

d) What other checks should the verifier make?

3

Scenario 4: Refinery

The Installation

The refinery is located a mile inland from the coast and has an annual crude oil throughput of around 210,000 barrels per day (bbl/day). The refinery receives crude oil by ship, primarily from the North Sea, and exports refined products by ship, road tanker, wagon and pipeline across Europe and to other parts of the world.

The technical units at the refinery consist of: a 210,000 bbl8/day Crude Distillation Unit (CDU), 3 Hydro-treaters (HT), Vacuum Distillation Unit (VDU), Visbreaker Unit (VBU), Fluidised Catalytic Cracking Unit (FCCU), Unifiner Unit, Catalytic Reforming Unit (CRU), Isomerisation Unit, Hydrogen Recovery Unit, Merox Units, Utilities Plant generating power for the refinery, LPG Recovery Unit, Butamer Unit, Alkylation Unit, Amine Regeneration and Sulphur Recovery Unit, Wastewater Treatment Unit, Crude Oil and Product Storage, three flares, two jetties and a road tanker loading facility.

The fuels combusted and materials used within the refinery9 are : heavy fuel oil (0.3%), refinery fuel gas (51.3%), natural gas, refinery flared gas (2.3%)10; gas oil (0.2%); acid gas (0,0%), LPG (0.1%), FCCU Coke (45.5%) and CRU regen coke (0.3%).

The outline and boundary of the Refinery for Scenario 4 is provided below.

Fuel accounting

Heavy fuel oil (HFO) for use in the Utilities power plant is stored in a dedicated storage tank; HFO is taken from production stocks and transferred to the utilities day tank, tank levels are measured before and after deliveries using automatic level & temperature gauges and accounting for fuel density and temperature using tank tables with a reference temperature of 15oC. Representative samples of liquid fuels are taken

8 Barrel per day. 9 % values indicate the proportion of aggregate emissions arising from each fuel source stream 10 Early in the year the Refinery had a major shut down which resulted in depressurising the whole plant and also process upsets during start up.

4

upon delivery into the storage tank and at the end of each month when the tank is independently dipped by a third party; these samples are sent for analysis at the site’s accredited laboratory to determine composition and net calorific value (NCV).

Refinery fuel gas (RFG) is derived from off gases collected from the various process units that are blended in Vessel 30-V-4809 and put into the gas ring main piping which also allows the intake of Natural Gas for pressure balancing11. RFG is measured into consuming units (e.g. heaters) using differential pressure meters comprising Orifice Plates with temperature and pressure compensation; continuous meter readings are transferred via the flow computer to the Plant Information (PI) system from where it can be downloaded into spreadsheets and databases for use. The RFG gas used in the VBU unit amounts to about 2.5% of the total RFG consumption of the refinery.

RFG composition is analysed twice a day from samples taken from Vessel 30-V-4809 and the results are used to generate a monthly average composition. Analysis is undertaken by the on-site laboratory and the results are uploaded to the Laboratory Information Management System (LIMS) and exported to the PI system.

The flow of refinery flared gas is measured through ultrasonic meters in the pipelines to the flare stacks. The pilot gas for the flare is refinery fuel gas. Flare gas composition is assumed to be similar to RFG gas during normal refinery operation. During upset conditions the composition of the flare gas is determined depending on the vessel/process that has been blown down to the flare.

Coke make12 is calculated on the basis of the input airflow rate and CO2 concentration in the output FCCU/CRU flue gas stack, which is determined by online analysers (these are installed for operational control reasons and to meet regulatory requirements but are not used specifically for continuous emission measurement of CO2 via CEMs).

The concentration is multiplied by the online air flowrate and the flue gas to air ratio. :

CO2 = (Flue CO2 – Air CO2) x Input Air Flow Rate x hours of operation

The refinery is in the middle of a shutdown for major maintenance on two of its three core process units; and due to the works part of the site has access restrictions limited to qualified personnel only; in addition key personnel responsible for instrumentation are intermittently not available.

4.0 Questions:

a) Given the context in this scenario, what inherent and control risks should the verifier identify and on what elements would the verifier focus its verification checks? Please indicate the reasons for focusing on these elements and indicate what questions the verifier should ask itself when identifying the risks and consequently determining the scope and detail of verification.

b) What parts of the installation should the verifier visit whilst on site and how should they determine what they look at and who they should interview?

c) Should the verifier check anything in relation to the CEMS? If yes, what should they check?

d) In the context of this scenario, list the things that the verifier should check as part of the analysis of uncertainty for :

i. Measurement instruments13 ii. Calculation factors based on sampling

e) Given the circumstances described on site, is there a limitation being placed on verification activities; if yes, how might these be overcome, or what could the consequences be?

11 Natural gas is only used for pressure balancing in the RFG ring main and does not constitute a source stream on its own. 12 Coke made refers to the process whereby during the cracking of the oil in the FFCU carbon is formed on the surface of the

catalyst, quickly de-activating the catalyst. The catalyst is “regenerated” in the regenerator vessel of the FFCU, whereby steam is led over the hot catalyst and the carbon is burnt off.

13 i.e. flow and weighing instruments etc.

5

III.1 Model answers for Scenario 1, i.e. instructions to trainers

Power Generation Scenario 1 introduces a power station generating electricity for export to the national grid that is mainly solid fuel fired. It is a category C installation producing around 2,800,000 tonnes CO2 per year.

Question 1a as asked in Scenario 1

What questions should a verifier ask itself during the strategic analysis to determine the depth and scope of verification for this year? What elements are important given the information in the context?

Article 7(4) of the AVR outlines most of the scope of verification. The verifier is required to assess:

Completeness of the emission report (AER) and its compliance with Annex X of the MRR;

Compliance with the approved MP and permit;

Whether data in the AER is free from material misstatements;

Information in support of improving the operator’s performance on monitoring and reporting.

Key questions that a verifier should ask itself include:

What is the latest version of the approved MP? Article 10 AVR requires the operator to send the latest version to the verifier;

What is the latest version of the permit? Have there been any changes to the permit?

What is the installation’s category and applicable materiality level? The verifier must apply reasonable level of assurance (Article 7(1) AVR) and the materiality level is 2 % for Category C installations (Article 23 AVR);

The installation boundaries: What are the emission sources, units and source streams listed in the MP? Does this include everything on site that should be in the MP? During verification (by walking around the site) the verifier will check whether the installation’s boundaries as listed in the MP reflect the actual situation on site and whether there are any other technically connected elements that have been omitted (e.g. is there a delivery jetty with combustion sources; should the supplier’s port of delivery be included, is there more than one pile of coals14 etc.). This is connected to the data flow and the procedures used to manage the data flow activities as mentioned below;

What are the risks of misstatements and non-conformities (e.g. are there significant inherent and control risks in the accounting process?)? Please see AVR KGN II.2 on risk analysis for information on how to assess inherent risks and control risks. These issues will also come up under the next questions;

What are the specifics of the monitoring methodology, e.g.:

o type of monitoring methodology applied for different source streams (e.g. mass balance for determining the coal stock accounting; measuring bulk tank levels for heavy fuel oil and liquid biomass before and after deliveries; mass balance for liquid biomass blends);

o whether Tall Oil and Cashew Nut Oil fired as substitute for HFO, and Rape Seed Oil fired as substitute for Gas Oil, are bio-liquids and whether a zero rate for biomass has been claimed;

o whether the biomass is a mix or pure biomass; whether solid biomass can be classified as solid biomass; how the quantity of biomass is determined and tracked;

o the use of sampling for determining the density and volume of coal as well as moisture content and calorific value of solid biomass; and any agreed alternate methods for taking samples if the auto-sampler fails: whether the sampling plan is approved by the CA;

o type of default factors used and where they are sourced from;

14 In practice the other location for coal once removed from this main pile are the bunkers.

6

o whether the external lab is EN ISO 17025 accredited or not;

o type of measurement instruments used; and whether these instruments are covered by national legislation on national legal metrological control.

Dataflow - the route by which data from primary sources end up in the emission reports (e.g. including manipulation15, aggregation, collation etc.);

Whether the operator has established and implemented procedures used to manage the data flow activities in accordance with Article 57 MRR and whether these procedures are effective to mitigate the inherent risks;

Whether the operator’s risk assessment is adequate for their internal control and to assist the verifier in understanding the operator’s reasons for implementing (or not) control activities. Please note that this does not exempt the verifier from doing its own risk analysis, but the operator’s analysis should be used as the starting point. Where the operator’s risk assessment is found to be limited or incorrect, the verifier will take this into account in the planning of the verification activities and in their findings;

The control system and control environment - a first impression on the robustness and quality of control activities and procedures: e.g.;

o whether manual controls or automatic controls are used;

o whether double checks are performed by a different persons (four eyes principle), including plausibility checks;

o the way (data for) the emission report is extracted from the data management system;

o the frequency and type of calibration of measurement instruments and their fitness for purpose based upon original design and installation;

o whether part of the monitoring activities within an installation have been outsourced; and the type of control activities in place to ensure the quality of outsourced activities;

o whether correlation and comparison checks (against other appropriate data or sub-sets) and completeness checks on data have been performed by the operator in accordance with Article 62 MRR;

o the type and quality of controls on recording and transmitting data into IT systems; and the control of black box databases, archives and source data in other IT systems and advanced process control systems.

Have there been any changes in the reporting period? If so, were these changes significant and was CA approval obtained? Have the other changes ('not significant' changes) been notified to the CA?

Has there been communication between the CA and the installation?

What were the results of previous verification? Is there an improvement report from previous year? (this information may also be asked for in the pre-contract stage). And has it been acted upon in accordance with the agreed deadlines?

Question 1b as asked in Scenario 1

Given the context of this case, what inherent and control risks should the verifier identify and on which elements should the verifier focus its verification checks? Please indicate the reasons why the verifier should focus on these elements and what questions the verifier should ask itself when identifying the risks and determining the scope.

Participants should ask themselves generic questions about what they would be looking for in terms of inherent and control risks.

15 Data manipulation is the process of changing data to transform it into something meaningful in the context of the emission

report, e.g. conversion of units, adjusting orders of magnitude, determination of emission factors from analytical data by means of

calculation etc.

7

Examples of inherent risks include:

Relevance and proportional size of emissions related to each source stream or emission source. The source stream: coal used by the dual fuel units of 530 MW is larger than the fuels HFO and gas oil, and the impact of any error etc. in this source stream on declared emissions is therefore greater so any inherent risk will have a greater effect. It is important to check categorisation of source streams (major, minor and de-minimis);

Not including emissions from portable combustion units, although their impact on the total reported emissions is not great; and it should be assessed whether those sources count as ‘mobile16’ (and so are not eligible for EU ETS);

Omitting source streams: e.g. note that there is a propane-fired heating system installed in the workshop area. So there should be a source stream: propane, but this is not mentioned in the description of fuel monitoring;

Additional risks associated with liquid fuel, in particular if the operator claims a zero-rate EF; also, whether the energy balance between fossil and non-fossil fuels consumed matches the CO2 balance declared;

Inaccurate measurements (e.g. by not accounting for fuel density and temperature);

Incorrect default values for Emissions factors and NCV for gas oil;

Risks of stock balance not being accurate (for example, stock adjustments between years meaning prior year closing balance doesn’t match current year opening balance in records);

Risks of samples not being analysed, risks of forgetting to send samples for analyses, risks of non- representative sampling, risks of a sample not being controlled through to the lab (chain of custody);

Measurements are not read correctly, e.g. temperature is not taken which means tank table adjustments for volume cannot be properly made;

Samples are not taken correctly and/or the samples are not representative of the fuel consumed;

Manual transfer of data;

Risk that calculation of activity data based on back calculation from energy and heat input is not entirely in line with MRR;

Complexity of operator’s operations (in Scenario 1 the operations are not particularly complex although solid fuel accounting might be more complex than for a simpler combustion installation that does not use solid fuels and quite a number of different source streams are involved overall making for more complex data flow to control).

Examples of control risks include:

Controls in place for managing the quality and competence of an external lab. For example, is the lab accredited according to EN ISO 17025? If not what controls might be expected;

Competence of the administrative team carrying out cross checks of the readings of road weighbridges to supplier’s invoices; checking off sample results against the sample register and entering this information into the database;

Readings from the movements control room are incorrectly logged on the delivery sheet or incorrectly entered into the fuel stock and energy accounting database;

The database from which data is downloaded to an excel spreadsheet template to calculate CO2 emissions for reporting is not functioning effectively; or that data is re-downloaded and overwrites adjustments or calculations already made in the excel template;

Risks that the controls in place in the database to stop unauthorized access of data or to change data are not effective or can be overridden;

Procedure for checking completeness and appropriateness of monitoring plan is not robust;

16 Ie the fuel is used to drive the wheels to make it move

8

Internal auditing is not effective;

Risks of calibration of measurement/sampling equipment not being carried out or not being carried out properly (e.g. infrequent calibration, malfunctioning of equipment, failure of bias tests).

Focus of verification checks include:

Major source stream : coal, but don’t forget the smaller ones and emissions from the portable plant equipment;

Testing of control activities (e.g. focus on manual transposition of data and anywhere that involves human action, e.g. manual cross checks, manually entering into fuel, stock and energy accounting database etc.);

Checks related to solid and liquid biomass, including energy and CO2 balances;

Checks on measurement equipment, calibration and measurement results;

Checks on competence of labs and associated personnel (depending on the type of lab);

Database and spreadsheet used for CO2 calculation (correctness of information and applied calculation methods).

Question 1c as asked in Scenario 1

Which specific checks should a verifier do on the monitoring methodology and the control activities? How should the verifier do these checks?

Monitoring methodology:

Information on checks to be carried out on the monitoring methodology is included in section 3.2 of AVR KGN II.3 on process analysis. The verifier will need to check whether the monitoring methodology described in the approved MP has been correctly applied. Usually the verifier will first do high level checks on the methodology and data (e.g. checking the data flow) and then do detailed checks as a result of the verifier’s risk analysis and/or issues identified during the verification (e.g. errors in the data, controls not effective). For this scenario, this can include:

Which checks Checks to be carried out:

Checking the spreadsheets and other tools or software used to calculate emissions

whether spreadsheets etc. are functioning and formulae have been correctly set up to meet the approved monitoring method in the MP;

validation and other tests done for software databases etc. before they go live;

comparisons between the outputs of data calculation spreadsheets used for different purposes (e.g. internal reporting vs external reporting, where separate spreadsheets are used);

Please see the table of control activities for the type of checks that should be carried out on software and IT used to calculate the emissions.

Checking correct total and subtotals used in the formulae to calculate emissions and parameters

re-calculation of the totals and sub-totals by the verifier to confirm outputs of spreadsheets (applying the formulae as listed in the approved MP), etc.;

review work instructions etc. and check how operator personnel have calculated totals and sub-totals; whether and what validation/ control activities have been applied;

cross check emissions produced with other data (e.g. energy generated, fuel related energy balance etc.);

completeness checks, checks on whether the data represent all items (e.g.. deliveries; number of invoices; samples) etc.

cross check emissions with production data; checks on capacity changes;

Checking application of correct tiers according to the approved MP and checking whether uncertainty thresholds have been met

checking category of source streams (major, minor and de-minimis);

checking correct application of approved tiers and assess whether the actual situation at the installation reflects the approved MP.: i.e. is the tier mentioned in the approved MP the correct one?;

review calibration reports, bias test, stock surveys etc. to confirm the basis of

9

Which checks Checks to be carried out:

calculations;

inspect measurement equipment and assess how the uncertainty assessment has been prepared for activity data. Please note that the verifier would not repeat the uncertainty calculations or do a detailed check on the calculation methodology; they should check that the input data for the calculation is reasonable. However, if in assessing that an appropriate uncertainty has been completed the verifier identifies that there might be a problem with the approach to calculating uncertainty, this should be raised as an issue that the operator should look at.

Checking application of corrects units of measurement for parameters

checking calculation formulae etc. against approved MP and MRR to ensure the correct units of input and output data are used;

checking meter equipment readings.

Checking storage of fuels locations and conditions;

whether records and MP reflect the actual situation.

Checking the measurement equipment and how data gathering is carried out.

checks are performed depend on the type of measurement equipment. In any case checks are performed on:

o measurement equipment itself (physical inspection during installation’s site visits: e.g. type, construction data, manufacturer, serial number, meter positions);

o calibration (records, correct references to installed equipment) (see control

activities – review documentation);

o alternate instruments / methods (e.g. in case of instrument failure);

o configuration of measurement equipment in the flow computer or DCS17 etc.

Check application of stock pile survey to determine density and volume coal

whether stockpile survey has been carried out according to specified and recognised standards;

whether the stockpile survey has been carried out by qualified and competent personnel;

whether the stockpile survey has been carried out on a periodic basis (e.g. quarterly basis);

whether a process of smoothing is applied e.g. rolling annual average;

whether the required uncertainty has been met (5% rule).

Check weigh readings of coal, biomass and heavy fuel oil

how and where results are entered so that they end up in the accounting system;

see control activities section for checks on weighbridges.

Check meter readings of bulk tank levels of HFO

how and where results are entered so that they end up in the accounting system

see control activities section for checks on control activities implemented in relation to meter readings

Check application of mass balance applied to determine the stock coal accounting

how mass balance has been applied by the operator;

re perform calculating of mass balance (if possible) and/or validate the input data and calculations;

check how energy generated is calculated to determine back calculation of activity data.

Checking invoices and delivery notes check amount of fuel/ material listed in invoices and delivery notes;

check the time period to which invoices relate;

cross checks to meter readings/ weighbridge tickets, if relevant;

check any adjustments made to invoices (e.g. credit notes etc.);

KGN II.3 on process analysis explains what to do if the invoiced quantity does not entirely tally at the start of the reporting period (i.e. invoice is not related to a period starting/ending 1/1/## or 31/12/##).

Checks on sampling of coal and solid biomass

proper implementation of approved sampling plan;

sampling positions;

17 Distributed/Digital control system

10

Which checks Checks to be carried out:

results of sampling and analysis, have results been applied to appropriate batches, if the operator is required to determine the calculation factor by analysis of samples;

whether analysis and sampling have been carried out according to applicable standards;

whether proper documentation is retained from laboratory tests to calculate emissions data, e.g. results from tests for establishing calculation factors;

representativeness of sampling;

sample uncertainty calculations to confirm frequency of sampling is appropriate;

sampling documentation - to assess how sampling is carried out by the operator, conduct compliance checks with sampling standards and assess control activities/ procedures for sampling.

Checking whether activity data and related data are properly documented

review how documentation is stored (see control activities);

review documentation to see whether relevant information has been documented.

Checking proper use of default values for the emission factors and NCV of gas oil

check with approved MP and cross check with MRR.

Control activities:

This includes, for example, checks on:

Which checks Checks to be carried out:

Whether calibration on weighbridges has been performed and whether this was done properly

whether the weighbridges are covered by national legal metrological control;

documentation on whether weighbridges have been calibrated, checked and adjusted prior to use, at regular intervals, and according to the frequency in the approved MP or in written procedures;

whether maintenance has been carried out according to the manufacturer’s recommendations and specifications;

whether calibration is carried out and whether there are regular on site loop calibrations;

whether there are limitations to weighbridge calibrations;

documentation to confirm whether measurement equipment has been checked against appropriate calibration standards;

documentation to confirm that checks have been carried out in accordance with requirement measurement standards and procedures (e.g. reports, certificates, signed off work instructions);

whether prompt corrective action has been taken by the operator if measurement equipment was found not to function properly;

check on age of weighbridge (i.e. when was it installed);

whether the weighbridge resembles the information on the calibration documentation (e.g. type, manufacturer, construction of data, serial number);

visual inspection of weighbridge & interview weighbridge operator (e.g. is it at the correct location, is the weighbridge installed properly, is it functioning effectively).

Check on automatic samplers and control activities implemented to ensure the automatic samplers are functioning effectively

whether samplers are functioning, have been installed correctly and have passed recent bias tests, including representativeness of sampling; whether data gaps have occurred; and if yes, whether corrective action or another approach has been applied;

control activities and maintenance tools built into the automatic samplers: e.g. recovery, continuity;

maintenance of automatic samplers (e.g. whether maintenance and bias testing has been carried out and at what frequency).

Checking the cross checks between measurements of changes in the bulk

whether persons responsible for doing cross checks are competent, and do not perform conflicting duties (e.g. recording, processing and reporting are carried out

11

Which checks Checks to be carried out:

tank stocks with manual tank dips by different persons) – through interviews and review of documentation;

whether a four eye principle is applied (double check by another person);

observation on how responsible persons are doing the cross checks;

if necessary, re-performing cross checks to confirm they have been applied correctly.

Checking cross checks between weighbridge values and delivery notes from the supplier, manually entering into the fuel stock and energy accounting database

whether persons responsible for doing cross checks and manually entering data are competent, and do not perform conflicting duties (e.g. recording, processing and reporting are carried out by different persons) – through interviews and review of documentation;

whether a four eye principle is applied (double check by another person);

observation on how responsible persons are doing the cross checks;

if necessary, re-performing cross checks to confirm they have been applied correctly.

Checking off sample results against the sample register and checking the readings from road weighbridges to the suppliers’ invoices

whether persons responsible for doing the checking are competent, and do not perform conflicting duties (e.g. recording, processing and reporting are carried out by different persons) – through interviews and review of documentation;

whether a four eye principle is applied (double check by another person);

observation on how responsible persons are doing the cross checks;

if necessary, re-performing cross checks to confirm they have been applied correctly;

cross checking rail vs belt weighed data.

Checking automatic downloading of data from database to excel spreadsheet and uploading data to database

Assess and consider risks related to using automatic controls and downloading of data. Verifiers need to understand extent of the risks and control of these risks in relation to IT systems. In addition verifiers will also consider:

proper use of calculation formulae and access controls; possibility of recovering data; continuity planning; and security with respect to IT and advanced process control systems;

whether IT systems and processes are managed under an effective IT Management System such as ISO 20000;.

Verifier checks control activities that are implemented in the IT system and electronic interfaces to ensure:

timeliness, availability and reliability of data;

correctness and accuracy of data, e.g. avoid double counting etc.;

completeness of data;

continuity of data to avoid data being lost and to ensure traceability of data;

integrity of data: i.e. data is not modified by unauthorised persons.

Control activities could also include a manual check on whether the IT system is functioning and whether the above points are met. It will include control activities and maintenance tools built into the IT system such as access controls, backups, recovery, continuity planning, change management and security. The type of testing carried out by the verifier depends on whether these control measures are manual or electronic.

Where, detailed checks/validation of IT systems is required the verification team may require the addition of an appropriate Technical Expert.

Checks on external lab activities. Type of checks depend on whether the lab is accredited or not

Scenario 2 and 3 outline what checks should be carried out on labs. Therefore this does not need to be elaborated too much under Scenario 1.

Please also note that where an external lab is used this is considered an outsourced process; so the verifier will check:

what activities have been outsourced to the lab;

control activities in place to ensure the quality of outsourced activities, e.g. assessing procedures for procurement, internal audit (including frequency of audits), carrying out plausibility checks on returned analysis data, checking contracts with external lab, instrument engineers, checking how an operator ensures that the party to which the activity is outsourced, carries out the activities

12

Which checks Checks to be carried out:

according to the MRR and other requirements;

confirmation with the lab that relevant elements of the sampling plan are acceptable, where changes have been made [MRR Article 33(1)].

Transfer daily readings from road weighbridges to electronic spreadsheets

ensure no mis-upload of data, including :

o standard template used for data upload has not been changed;

o upload macros have not been adjusted.

Check on whether corrective action have been taken in the case of malfunctioning of equipment or identification of mistakes

corrective action has been indeed taken in those situations and there is confirmation that emissions are not underestimated;

effective control activities have been implemented to prevent data flow activities and control activities from not functioning properly or from being outside the boundaries set in the relevant procedures;

criteria in the procedures for data flow activities and control activities are addressed and met by the operator, and whether details of these procedures are effective to avoid malfunctions;

operator has notified the CA of any equipment failures or drops to lower tiers during the reporting period, and that efforts were made to correct the failures as promptly as possible.

Checking competence of personnel assess in interviews and through document review and observation on whether personnel carrying out control activities are competent

assess whether there is a control process that defines who can do which activities and what competence is needed for these activities.

Which checks should be done on site during the visit?

Checks on site include the following:

Interviews with relevant operational personnel and administrative team;

Checking implementation of the monitoring plan in actual practice;

Cross-check whether the installed equipment matched the information on the calibration records (type, manufacturer, serial number, ...);

Visual inspection on whether the installation boundaries approved in the MP and other elements of the MP reflects the actual situation of the installation (e.g. source streams, emission sources). The verifier will not solely focus on the MP, but will also do checks on fuel purchase records etc. and other checks to see whether any evidence can be found of unlisted source streams coming on site and perhaps being combusted with or without the emissions being reported;

Checking implementation in practice of procedures and control activities through interview and observation (e.g. appropriateness of procedures, completeness and implementation of procedures, effectiveness of control activities);

Following a sample through the process; checking existence of maintenance plans, invoices, checking locations and conditions of storage of fuels;

Visual inspection of meters, and weighbridges; collection of serial numbers and Field Tags from instruments in situ to cross check to records.

How should the verifier check that MRR requirements are being met?

AVR KGN II.1 on scope of verification explains the extent to which verifier will check the MRR requirements. It is the responsibility of the CA to approve the MP (and associated documents) and to confirm whether the proposed MP is in line with the MRR. The verifier takes the approved MP as the starting point to assess whether the reported data is free from material misstatement. However this does not exempt the verifier from crosschecking against the MRR to some extent. The verifier will, for example, check:

13

Implementation and robustness of control activities and procedures, and ensure that what is happening ‘on the ground’ reflects the summary in the MP; and whether there have in fact been any changes since procedures were approved.

Installation boundaries (e.g. emission source streams and emission sources): verifiers will check whether emission sources and source streams listed in the approved MP reflect the actual situation within the installation. They will for example check:

o whether all emission sources and source streams were included in the MP, no matter how small;

o the correct categorisation of the source streams.

When checking the installation boundaries the verifier will not solely restrict itself to what is described in the MP but will thoroughly check and consider the actual situation of the installation by means of a walk over inspection.

Inputs to the instrument uncertainty analysis to ensure they are reasonable and evidenced;

Do some cross-checks to MRR rules when checking the application of the monitoring methodology; e.g. representativeness of sampling, evidence of non-accredited labs, compliance with standards, uncertainty thresholds for tiers etc.

This means that the verifier will do cross checks against the MRR and that auditors carrying out verification activities must have full knowledge of both the MRR and the guidance material. Please note it is not the verifier’s responsibility to actively check every element and detail of the operator’s situation or operations against the MRR; but they need to have confidence in their conclusion if they state that the monitoring and reporting for the year is in compliance with the approved plan and the MRR.

So a balance is required between the level of effort input by the CA when approving the MP and that input by the verifier when checking its implementation to avoid duplicating tasks carried out by the CA – this may vary depending on the approach of the CA in an individual MS.

However, it is the responsibility of the verifier to assess whether the data is fairly stated and in that respect the verifier is entitled to, and required by the AVR, to list its observations in the verification report where non-compliance is identified, during the course of the verifier’s work, between the approved MP and the MRR [Article 7(5) AVR].

Where the verifier has identified non-compliance with the MRR, the verifier must report this in the verification report in accordance with Article 7(5) MRR, even if the MP was approved by the CA.

If the verifier identifies weaknesses in the application of the control activities or other parts such as the monitoring methodology etc., the verifier must report this as a recommendation of improvement in the verification report in accordance with Article 29 AVR.

Question 1d as asked in Scenario 1

List the documents the verifier should ask for copies of, and what checks should they make in relation to those documents

According to Article 10 AVR , the verifier should request the following documents from the operator, who is required to provide them:

The latest MP approved by the CA;

Emission reports;

Operator’s risk assessment;

Operator’s sampling plan that is approved by the CA;

Operator’s uncertainty assessment for measurement instruments;

Procedures and corresponding documentation that are referred to in the approved MP;

Calibration certificates;

Accreditation report of lab if the lab is accredited;

Evidence of competence of personnel;

14

Delivery notes, invoices, stock surveys., downloads of spreadsheets;

Relevant communication with CA including formal notifications in terms of instrument failures, tier changes etc.;

Improvement report and verification report from the prior year.

This part of the question is important for the discussion groups since there is a lot of confusion about the roles of the CA and verifier in relation to different elements of the MP and supporting documentation; and in relation to checking compliance with the MRR.

Checks to be carried out by verifiers on these documents include:

Document Checks to be carried out

Latest MP whether all elements of the approved MP have been correctly applied, as approved;

cross checks with other data sets (e.g. emission report data);

cross checks with MRR (see question 1c);

Permit whether permit conditions have been applied;

cross check of emission sources and emission source streams, capacity levels between permit and MP (where these are separate documents);

cross check with the actual situation at the installation.

Operator’s risk assessment confirm that risks identified by operator and their magnitude as assigned by the operator match those identified by the verifier in its risk analysis; and discuss mismatches with the operator;

assess whether risk assessment reflects the actual situation;

identify weaknesses in the control activities and procedures.

Operator’s sampling plan See question 1g

Operator’s uncertainty assessment

check the validity of input information used to calculate uncertainty levels as stated in the approved MP. The type of information used depends on the methodology, the type of measurement instrument, and on the approach the operator applies to uncertainty calculations used to demonstrate that the required tier and associated uncertainty threshold is met.

The approach used to assess whether the required uncertainty for activity data is being met, depends on whether the measurement instrument is under the operator’s own control or under the control of other parties. For measurement instruments under the operator’s, control different routes can be applied:

1. using the maximum permissible error in service allowed by the national legal metrological control, if the measurement instrument is subject to that control;

2. using a maximum permissible error specified for that measurement instrument in service, or using an uncertainty assessment obtained by calibration multiplied by a conservative factor, if the measurement instrument is not subject to national legal metrological control and the instrument is installed in an environment and under conditions that is appropriate for its use specifications;

3. undertaking a specific and extensive uncertainty assessment.

For point 1, the verifier checks whether the measurement instrument is covered by legal metrological control, e.g. checking the certificate of official verification of the instrument and checking specifications from the national legal metrological control institute. The verifier must be sufficiently confident that the instrument is regularly calibrated.

For point 2, the verifier would, for example, check the manufacturer’s specifications, specifications from the legal metrological control, and procedures implemented by the operator to ensure that the activity data are measured against the relevant standards.

For point 3, the information to calculate the uncertainty level using this more complex assessment process is more substantive than the information needed for measurement instruments using points 1 & 2 (for which a simplified uncertainty assessment can be applied). This means that the verifier will have to check all information input to this specific uncertainty assessment to confirm that it is reasonable and valid.

15

Operator’s uncertainty assessment

If the measurement instruments are outside the control of the operator, the verifier must check the validity of the information provided to the operator. The operator needs to make available to the verifier all relevant evidence and documents. This means that the operator must obtain information on, for example, the calibration results and the manufacturer’s specifications of the instruments used by the trade partner; or at least a valid statement from, e.g. a gas supplier, attesting to the uncertainty if their instruments The verifier will also check whether the operator has implemented control activities to ensure the quality of measurement results.

Procedures That these procedures:

are fit for purpose in relation to the verifier's risk analysis;

available for use, properly documented, maintained and retained;

contain information listed in the summary of the procedures in the approved MP;

have been correctly implemented in practice and are up to date;

are applied throughout the reporting year;

are effective to mitigate associated inherent and control risks.

This includes review of procedures and documentation, inspection on-site on to check whether procedures have been implemented.

Calibration certificates • documentation on whether metering equipment has been calibrated, checked and adjusted prior to use and at regular intervals and according to the frequency in the approved MP or in written procedures;

Calibration certificates – cross check :

o serial number on certificate to that on field instrument (not the same as Field TAG id);

o whether calibration equipment is traceable to a certified standard;

o whether it is still in date.

age of measurement equipment;

range of calibration;

competence of personnel carrying out calibration;

conditions at the time of calibration;

whether measurement standards were applied;

whether maintenance has been carried out according to the manufacturer’s recommendations and specifications.

Accreditation report of lab Checks include:

analyses conducted;

validity of accreditation;

variation results;

standards applied.

Correspondence with the CA whether there has been any relevant correspondence with the CA regarding changes to the MP (e.g. approval of significant changes to the MP, lack of approval) or any other issues that have arisen during the year that could impact on monitoring.

Improvement report and verification report from last year

have comments made by the verifier in the prior year verification report been fully addressed in the improvement report;

whether non-conformities and improvement recommendations have been addressed in accordance with the improvement report approved by the CA;

whether there are any remaining weaknesses that are relevant for planning of verification activities

16

Question 1e as asked in Scenario 1

What data should the verifier ask for; and what specific checks should they make on the data provided? How would the verifier perform these checks?

The verifier has to assess whether the data flow as described in the approved MP meets the actual process in practice by testing data flow activities, checking data trails and following the sequence and interaction of the data flow activities from primary data generation, through manipulation to final output reporting. The verifier traces data back to primary sources, checks the existence, consistency and validity of primary data sources and follows data points though each subsequent step in the data flow.

In addition, the verifier will check which persons are responsible for specific data flow activities. An important aspect which the verifier shall take into account when assessing inherent risks related to data flow activities, is whether these persons are accustomed to, and competent, to deal with specific data flow activities assigned to them. This will be done by inquiry (interviews with personnel), checking competence criteria and document, and observation on how the data flow activities are carried out by the personnel.

Other checks on data may include:

Completeness checks;

Cross checking between different data sets (e.g. month on month comparisons against production data; comparison to installation mass balance done for e.g. Solomon Index or other internal purpose);

Cross checking with external data (e.g. fuel supplier data);

Reconciliations and re-performance (check on whether the verifier achieves the same results);

Checking measurement results and readings;

Checking accuracy of calculations and associated formulae, checking CO2 calculation spreadsheets or other database output;

Checking whether there are data gaps or double counting, and checking whether/how these data gaps or double counting were addressed;

Analytical procedures (e.g. plausibility checks, checks on fluctuations, trends, comparing GHG emissions with previous year emissions, comparing emissions with fuel consumption (and – for the power sector - energy produced), comparing operational conditions with trends in fuel consumption over time), are other databases available to use as proxies;

Checking laboratory results; what tests is the lab applying? If the lab results do not contain uncertainties, is there an inherent uncertainty that is accepted?

Transfer and conversion to factors, stoichiometric calculation factors; stock pile surveys;

Checking extremes, see whether these are included or excluded, what is the procedure for handling such data points;

Checking installation boundaries (completeness and categorisation of source streams and emission sources) – through walk-around inspection and review of relevant operational/contractual lay-out plans and piping and instrumentation diagrams followed by cross check of the data on source streams and emission sources, other data verification checks and plausibility checks.

Question 1f as asked in Scenario 1

What should the verifier consider when preparing for the site visit? Who should join the site visit?

When preparing for the site visit, the verifier should consider:

who from the verification team should join the site visit, whether a technical expert is needed – for example, to check measurement equipment or other elements during the verification;

whether more than one site visit is required (to different locations, often dependent on whether information is held centralised or decentralised);

17

what activities to carry out during the site visit (e.g. interviewing staff, reviewing documents and assessing implementation of procedures, physical inspection of meters, automatic samplers & associated records, checking installation boundaries and the data flow, actual testing of control activities and obtaining physical evidence through assessment of measurement equipment);

what locations to visit on site and which areas within the location: e.g. sampling different processes/facilities on site to assess the installation boundaries and metering equipment etc.; sampling, as relevant, at the installation’s head office (e.g. for data aggregation) and at facility level (e.g. for data generation) to review procedures and interview personnel at each stage of the data flow; sampling at any other location where verification work may be necessary (e.g. fuel supplier facilities, external non-accredited labs etc.).

In most cases the EU ETS lead auditor will conduct the site visit themselves which means that all on site activities must be carried out by the lead auditor. If the EU ETS lead auditor takes other persons from the team to the site visit, the lead auditor must instruct those team members on what activities they have to carry out during the site visit. The lead auditor would be responsible for assigning tasks to other team members and implementing the verification plan. Based on the risk analysis the lead auditor will decide whether s/he carries out the site visit by themself or whether and, if so, which team members will take part in the site visit. The lead auditor will also determine whether s/he needs a technical expert during the site visit. This could include measurement equipment expert, IT-expert etc.

Question 1g as asked in Scenario 1

What checks would the verifier make in relation to the installation’s sampling plan?

The verifier must check, for example, whether:

the sampling plan includes all items specified in the MRR; any relevant good practice identified in the Commission’s guidance18; and that it is effective to deliver the quality required for the specified and approved monitoring methodology;

the sampling plan is still appropriate and can be justified by the operator as delivering the most representative samples for the current circumstances19;

sampling is being consistently carried out according to the latest version of the sampling plan approved by the CA (e.g. frequency, approach, pattern of sampling). The verifier basically checks what samples are taken, how frequent samples are taken, what standard is used, where the samples go, how the samples are handled, whether there is a chain of custody, what laboratory the samples go to, etc.;

personnel carrying out sampling are competent to do so;

the sampling plan has changed in a significant way20; whether these changes are reflected in an update to the approved MP; and where required the update has been approved by the CA;

where elements of the sampling plan are distributed across different departments/ existing operational procedures (as is the case for many complex installations), transparency is ensured by a central reference document or table (that acts as the single point ‘plan’) to signpost to where each key element is managed (e.g. who is responsible, what existing procedure is used, etc.); and that – from an EU ETS compliance and verification perspective - those underlying procedures are appropriate and effective, and the relevant personnel are appropriately trained and competent;

the procedure(s) underlying the sampling plan are documented, implemented, maintained and effective;

in the case of problems identified in sampling, these have an effect on the data and if so, whether this is material.

18 Where there is evidence that the CA has undertaken a detailed evaluation of the sampling plan, then the verifier may consider that this is an area of lower verification risk and take account of this in the planning of their work 19 The verifier cross-checks this information with other information on the Installation’s site. 20 In particular where analytical results indicate that the heterogeneity of the fuel or material significantly differs from the information on heterogeneity on which the original sampling plan for that specific fuel or material was based and approved.

18

Question 1h as asked in Scenario 1

What checks should the verifier make in relation to the “solid biomass” and “liquid biomass”?

Please note that a liquid fuel for power generation can only be considered biomass (i.e. eligible for zero-rating of related emissions) if it is deemed a bioliquid; and the Renewable Energy Directive (RED Directive) requires in the case of bioliquids that mandatory sustainability requirements are satisfied. A liquid fuel that does not meet the RED required sustainability requirements would have to be treated as fossil. So legally the term liquid biomass can only be used if sustainability criteria have been met. Therefore the terms used in the scenario are highlighted in italics as they are the site’s name for the fuel and not the legal designation.

The verifier will for example check:

completeness information of solid and liquid biomass used (including sustainability information);

whether solid biomass is pure biomass or part of mixed fuels – “liquid biomass is delivered to site by road tanker and stored in bulk tanks as a blend with their respective fossil equivalent fuels” (this means that these are mixed fuels);

the delineation of biomass source streams. If the operator uses biomass that is delivered in batches, verifiers should check if these batches are considered to be the same or should be treated as different source streams;

if the monitoring methodology applied for liquid and solid biomass has been done in accordance with the approved MP – this will entail some cross checks to the MRR by the verifier.

Question i under question 1h as asked in Scenario 1

Do verification activities change if the operator claims that the liquid alternative fuels are bio-liquids with zero-rated emissions for the verification?

Inherent risks are increased because sustainability criteria apply, so the verifier needs to check if the operator has sufficiently demonstrated that the fuels concerned have met the applicable criteria, and whether reporting zero rated emissions is justified. Detection risk must be set lower because of the increased inherent risk which means more detailed verification activities are required focusing on claimed bio-liquids. For solid biomass no sustainability criteria apply and zero-rating can be allowed without having to demonstrate compliance with criteria.

The verifier should check and address following issues:

Can the Rape seed oil, Tall oil or Cashew nut oil come from a source that has a sustainability compliance scheme associated with it?

o If yes, did the fuel used come from such a source? Has a certificate been obtained as part of the supply? Was the certificate issued by a national certification system, or by a voluntary certification scheme that is recognised by the Commission? Was the certificate still valid at the time that the fuel was burnt? Is the geographical scope of the relevant oil in line with the scope identified for the certification scheme?

If a certificate was not issued? How did the operator demonstrate compliance with a national system by other means: e.g. evidence of a third party chain of custody audit21, did that audit cover the full scope that would be required by MRR to reasonable levels of assurance; and was the result of the third party audit acceptable?

In this case the claimed liquid biomass is blended with HFO or gas oil (depending on type). This means there is a mixed fuel and there can be different possibilities with respect to application of sustainability criteria. The following options apply:

21 Chain of custody audit means that there has been an audit check back though each stage from the installation to the original

supplier and that all evidence is in place to show at each stage that the fuel meets the sustainability requirements; and that evidence is associated with a specific batch of fuel and this has been passed along the chain of supply.

19

a) Fossil / claimed bio-liquid 22 mix where sustainability criteria apply and are not satisfied: In this case, the emission factor to be used for all of the fuel is the emission factor based upon the carbon composition of the blended fuel (or the equivalent factor for each fraction of eg HFO and other ‘bio-liquid’ fuel); i.e. there is no zero factor to be applied;

b) Fossil / bio-liquid mix where sustainability criteria apply and are satisfied: In this case, the emission factor to be used is zero for the bio-liquid fraction and an emission factor based upon the carbon composition of the fossil fraction (eg HFO);

c) Claimed bio-liquid mix or fossil / claimed bio-liquid mix, where only a part of the claimed bio-liquid satisfies applicable sustainability criteria (i.e. part does not): In this case, the emission factor to be used is the emission factor, based on each of their compositions, for the proportion of the mix corresponding to the fossil fraction and the part of the claimed bio-liquid that does not fulfil applicable sustainability criteria; for the proportion of bio-liquid meeting relevant sustainability criteria an emission factor of zero could be used.

See MRR guidance document 3 (GD 3) and section 4.3 of KGN II.3 on process analysis.

Discussion could also evolve on how the verifier would check these elements; including whether a fuller

chain of custody audit would need to be completed as part of verification work, or whether the verifier

can just accept the operator’s supply documentation if it has sufficient information in it. Key to the

discussion is what the role of the verifier versus the role of the CA is.

22The definition of bio-liquids under EU ETS is liquid fuel for energy purposes other than for transport, including electricity and

heating and cooling, produced from biomass (Article 3(21) MRR). Please note that not all liquid biomass are necessarily deemed bio-liquids under EU ETS definitions. If no sustainability criteria apply according to the RES Directive, the emission factor to be used is the emission factor that applies solely to the fossil fraction of the mix.

20

III.2 Model answers for Scenario 2, i.e. instructions to trainers

Cement Installation with non-accredited laboratory

In Scenario 2, an installation producing cement clinker in a rotary kiln is presented. The annual production of this installation is 615,000 tonnes of CO2. This installation is a poorly run installation where high inherent and control risks are likely. The idea is to demonstrate to participants that a verification plan for a badly run installation, and in particular where there is no lab accreditation, is likely to be more extensive than one for a well-run installation (and/or accredited lab); and which will therefore be likely to make the verification more extensive and complex.

Scenario 3 presents a similar cement installation but which has significantly higher levels of internal control, an accredited laboratory and hence lower risks. This means the verification plan will be much simpler in that case.

For Scenario 2 the discussion groups will be asked to develop a verification plan using the risk analysis template provided as part of the Scenario 2 papers). For examples on how to complete this type of verification plan please see the exemplars on verifier’s risk analysis and sampling on the Commission’s website.

Question 2a as asked in Scenario 2

What is included in the installation boundaries and thus subject to monitoring under EU ETS? How should the verifier check the installation boundaries?

The first question that the verifier should ask is what falls inside the scope of EU ETS. The verifier should check the installation boundaries: e.g. whether:

all emission sources and source streams are covered in the latest approved MP and whether the actual situation at the installation is reflected in the approved MP. For example, are there any technical units at the quarry or in the preliminary part of the process (preparation of quarried materials) that should be inside the installation EU ETS boundary? Should the truck/rail loading yard be included or excluded? Are there technical connections with the site?

the categorisation of source streams in the MP corresponds with the actual situation on site;

the emissions from all source streams and emission sources are included in the total emission figure in the emission report.

In this particular situation the verifier needs to check whether the quarries located adjacent to the main factory, where limestone is quarried, crushed and stockpiled are part of the installation boundaries. Whether the quarries fall within the scope of EU ETS depends on what technical units are placed at the quarry and how these are technically connected. If the activity at the quarry includes mobile equipment or units powered by electricity, it falls outside the scope of EUETS. However stationary combustion units included at the quarry would be covered by EU ETS. Please see Annex I of Guidance on interpretation of Annex I activities provide further information on this. The verifier will check the same for the tertiary crusher and product loading yards. The permit and the approved MP should provide information on this, but may not have been set up properly if information has been omitted from the application.

Additional questions that could be asked to prompt discussion

If the quarries and locations where raw material is processed fall outside the scope of EU ETS and are not included, to what extent should the verifier consider data and information on these processes for the verification?

The verifier is expected to check the transfer of raw material and pre-blends of semi-circular beds to the raw milling (e.g. how is the transport to the raw milling carried out, the data flow, where is the raw material weighed).

21

If the quarries and locations where raw material is processed fall inside the scope of EU ETS but are part of another installation, should the verifier check anything in that case?

The verifier should check whether there are indeed two separate installations with two separate permits or whether this in fact constitutes one operator under the EU ETS that has control of the installation as a whole and has decisive economic power over the installation. Although the permits include that information, the verifier is still expected to check the interface arrangements as it is possible that there could be shared source streams. Please note in the discussion that the permits might not be up to date in which case the verifier will have to do additional checks and refer the operator to the CA.

Question 2b as asked in Scenario 2

What questions should a verifier ask itself during the strategic analysis to determine the depth and scope of verification for this year? What elements are important and on which areas should the verifier focus given the information in the context?

This question is intended as an introduction to the case and should not take too long. It is just outlining what elements are important. Try to focus on the specifics of the case. The following questions are more crucial and should take more time.

An important note in this question is whether there is a non-compliance with the MRR in the monitoring methodology described in the case study. See below for clinker/cement ratio missing and deviation of tier because of incorrect reasons.

Key questions that a verifier should ask include:

what is the latest version of the approved MP? The operator must provide the most up-to-date approved MP to the verifier according to Article 10 AVR;

the installation’s category and applicable materiality level (category C installation, the materiality level is 2%);

the installation boundaries: what emission sources, technical units and source streams are listed in the MP; and is it complete compared to what is in place on the ground? What is included in EU ETS and what not? Are quarries adjacent to the location and tertiary crusher included?

what are the risks of misstatements and non-conformities (and how high are inherent and control risks in the accounting process?)? Please see KGN II.2 on the verifier’s risk analysis for information on how to assess inherent risks and control risks. These issues will also come up in the next questions.

what are the specifics of the monitoring methodology, e.g.:

o the type of monitoring methodology applied for different source streams (e.g. mass balance, weighing of fuels);

o is the monitoring methodology described in the case in line with the MRR (e.g. is the determination of clinker/cement ratio missing or can it be derived from the information presented? (See the formula in MRR, Annex IV, section 9 calculation meth B - clinker output (b));

o the fact that sampling has been used for determining the calculation factors and an operator’s sampling plan was drafted;

o the fact that zero rated biomass has been claimed for the sewage pellets used;

o the fact that country specific default factors are used for emission factor of tyres because it is difficult to sample. Question should be raised on whether this situation was approved by the CA and if this is in line with the MRR (deviation of tier because of unreasonable costs or technical infeasibility);

o type of default factors used;

22

o use of non-accredited lab that is not certified to any standard. In the following questions the discussion should go deeper into this topic ( eg how certification to an acceptable standard could mean less extensive tests on technical competence and quality management);

o type of measurement instruments used and whether these instruments are covered by national legislation on legal metrological control.

the dataflow - route by which data from the primary sources ends up in the emission report (i.e. including manipulation, aggregation, collation etc.), the range of the data set, the number of items in each population (as part of the verification plan a data sampling plan needs to be developed taking into account inherent and control risks);

control system and control environment- a first impression on the robustness and quality of these activities and procedures: e.g.

o whether controls used are manual or automatic;

o whether there is proper segregation of duties and responsibilities (e.g. management quality control and control room operations are directed from the plant office which also contains the small plant laboratory (e.g. whether responsibilities for recording, processing and reporting are carried out by different persons);

o the fact that the operator has its own corporate management system but it is not clear what other control activities are implemented (e.g. four eyes principles, corrective action) and whether the management system applies recognised best practice and/or does in fact properly control the laboratory and the monitoring process effectively;

o the way the emission report is extracted from the data management system;

o the frequency and type of calibration of measurement instruments and their fitness for purpose based upon original design and installation.

have there been any changes in the reporting period that are relevant to monitoring and reporting? Were these changes significant and was CA approval obtained for update to the MP? Have the other changes ('not significant' changes) been notified to the CA?;

have there been any communication between the CA and the installation?;

What were the results of previous verification? Is there an improvement report from previous year? (this information may also be asked in the pre-contract stage). And has it been acted upon in accordance with the agreed deadlines?

Is there any other information that the verifier would need?

In addition to the latest approved MP the verifier should seek information on:

a diagram of the data flow activities of the installation and/or a detailed description;

the procedures the operator has to establish and implement to manage the data flow activities in accordance with Article 57 MRR;

the operator’s risk assessment;

the operator’s uncertainty assessment(s);

approved permit to assess the installation’s boundaries (two permits if it turns out that the quarry etc. are controlled under a different permit). This could be the same operator or different operators depending on the situation (technical connections, interface arrangements, whether one operator has control of the installation as a whole and has decisive economic power over the installation definition of operator etc.;

tier improvement report (that should be compiled because the tier for emission factor of tyres has not been met, presumably because of technical feasibility and hence unreasonable costs). Trainers should be aware that the methodology in the case study is not clearly described and could have been better outlined in the monitoring plan;

procedures and documentation, in particular relating to data flow activities, control activities and the control and use of the non-accredited laboratory;

the operator’s sampling plan;

23

laboratory analysis results and sales data for clinker/cement;

correspondence with the CA, in particular in relation to changes to the MP and whether there was a claim for unreasonable costs for the deviation of the tier for tyres.

Question 2c as asked in Scenario 2

Using the attached risk analysis template23, identify the main inherent and control risks associated with the data flow described in this scenario; then annotate with a verification plan covering:

iii. What checks a verifier should do on the data and the monitoring methodology? State as a minimum which of those must the verifier do on site?

iv. What type/size24 of sample should the verifier check?

The idea is to complete the verification plan with the participants as outlined at the start of this section about Scenario 2.

Additional questions that you could raise if there is time for discussion:

How should the verifier deal with the non-compliance issues that were identified?

The verifier should direct the operator to talk to the CA about how to correct non-compliance issues identified and to adapt the monitoring methodology as required. If the non-compliance issue is not corrected by the time the verification report is finalised the verifier must assess the impact on the data and report this in the verification report.

The table below contains some examples that were discussed in the training event.

23 You have been provided with a template for the Risk Assessment and Verification plan based on the European Commission’s

exemplar. Please note that this is not a template for a complete verification plan as this should contain more information. 24 If you think you have insufficient information to determine the size of sample please indicate what information you need to

determine the size of sample.

24

Examples from the discussion groups on question 2c

Data Flow & Accounting area Inherent Risk (High/Medium/Low)

Controls Risk (High/Medium/Low)

Verification Risk (High/Medium/Low)

What will verifier check? What type/size of sample?

Determination of activity data for clinker

Method for calculation (method B) is not applied correctly/ incorrect data is used in the method B methodology The risk is high

Control activity is calibration of measurement instrument used for weighing cement clinker Management system and procedure for ensuring competence of personnel and internal review of data (e.g. cross checking the calculation) as well as controls on the transfer of data are other control activities The risk that these control activities are not functioning correctly is high (the overall QA/ QC is not organised well)

Verification risk consisting of a high inherent and control risk is high. This indicates a likely high risk that there are issues so the verification risk is high unless sufficient work is done; more information is needed to determine the verification risk effectively

Calibration certificate Calibration plan/management system Procedure for corrective action and data gaps Check calculations (formula and numbers) Checking transfer of data (checks on manual controls are different compared to checks on IT systems) Sample 20% and increase of sample if problems are identified

Determination of emission factor Possible inherent risk related to sampling carried out to determine the emission factor are:

Sampling plan(s) not in place

Samples are missing

Samples or sample are not representative

Sampling plan is not agreed with laboratory

Inappropriate composite of samples

Insufficient number of analysis The inherent risks are high

Possible control activities include:

Procedure for ensuring competence of personnel

Competence of personnel

Training

Appropriate procedures for sampling

Checks that samples are being taken through unique id logging & tracking etc

Sampling plan is not implemented or implemented correctly

The risks are likely high because overall the quality of the control activities is not appropriate

Verification risk consisting of a high inherent and control risk is medium to high. This indicates a likely high risk that there are issues so the verification risk is high unless sufficient work is done; more information is needed to determine the verification risk more effectively

Interviews with personnel to test their competence Checks on implementation of procedures Check between register of samples taken and results reports returned by the lab Training records Check of actual practice versus procedures Sample 100%

25

Application of stock balance

Incorrect measurement of clinker opening stock

Control activities would have been:

Competence of personnel and training

Four eye check on calculation

Defined roles and responsibilities of personnel

Check on the closing stocks of the prior period

Insufficient information available to determine the extent of the control risk. It is likely high because the overall quality of control activities is not good

More information required Interviews with personnel to test their competence Checks on implementation of procedures Assessment of training records Check of actual practice versus procedures

Collection of cement and clinker data

Failure of measurement systems

Calibration is a control activity: a control risk could be that calibration is not functioning properly Inappropriate methodology (cement/clinker ratio missing)

Verification risk is likely high because the QA/QC system is not robust

Assessment of:

Dispatch notes

Production data (cross check)

Internal review/ checks on data

Sales data, stockpiles

Calibration (e.g. frequency, approach)

Competence of personnel. Because of high risks sample size will be large.

Determination of emission factor

Possible inherent risks include:

Non-representative samples

Results used to calculate EF are not from the same batch of materials

The laboratory does not apply the correct methods for analysis

The lab is not accredited against ISO 17025

The risk is high because the impact on the data can be high.

Control activities include:

Robust sampling plan

Checks on representativeness of samples

Chain of custody of individual samples

Requirements of equivalence in Article 34 MRR

Competence of personnel

Relevant standards

Procedures

Inter-comparison tests

Daily quality control

Maintenance of sampling equipment

Cross check that calculation of EF from analytical results is appropriate

The risks are likely high because overall the

Verification risk consisting of a high inherent and control risk is medium to high. This indicates a likely moderate to high risk that there are issues so the verification risk is moderate to high unless sufficient work is done; more information is needed to determine the verification risk more effectively

See question 2f of scenario 2. It would include checks on:

Requirements concerning equivalence of lab (Article 34 MRR)

Procedures

Certificates and lab reports

Competence of staff

Approval of the CA to use non-accredited labs

Recalculate the emission factor from analytical results

Check population of analytical results matches the number of samples taken

Higher sample size because of non-

26

quality of the control activities are not appropriate or missing (e.g. daily quality control missing, failure to check labels or to maintain equipment, failure to calibrate)

accredited labs

Fuels

Fuels are missing Lack of information from providers

Control activities would be procedures to check completeness of source streams and periodic review of the monitoring plan for completeness Insufficient information available to determine the control risks

Verification risk likely medium. Fuels that are likely to be omitted are de-minimis source streams. More information is needed to determine the verification risk more effectively

Site visit of verifier to check boundaries of installation and completeness of source streams by walking over the installation and interviewing operatives on site Check of permits Check of MP against the actual reality

27

Question 2d as asked in Scenario 2

On the basis of the information provided, list the specific control processes, procedures and documents you would expect to find on site?

Likely procedures verifiers would expect to find are those that cover:

management of responsibilities and competency of personnel;

data flow activities

quality assurance of measurement equipment (e.g. weighbridge maintenance and calibration)

uncertainty assessment (both for activity data and calculation factors)

regular review of the MP for its appropriateness

sampling of fuels and materials

methods of analyses if applicable;

demonstrating evidence for equivalence of the laboratory controls to EN ISO/IEC 17025 on accreditation of laboratories;

internal review of data and taking corrective action.

Specific control activities that are important for the verifier to check are:

Calibration and maintenance of the measurement equipment

the corporate management system, and in particular how it controls the laboratory and demonstrates evidence for equivalence of the laboratory controls to EN ISO/IEC 17025 on accreditation of laboratories

Of specific importance in this Scenario is the potential lack of robust control activities for both data monitoring and for the laboratory. On the basis of the scenario there appear to be no cross checks (using the four eye principle). In addition, it looks like there is no automatic transfer of data which means that the verifier will have to do substantive testing of the data trails across the data flow.

What checks would the verifier do on these controls and control procedures?

The verifier must check as a minimum that these procedures:

are present, properly documented, maintained and retained;

provide detailed and complete information in line with the summary of procedures in the approved MP;

have been correctly implemented and periodically reviewed to ensure they are up-to-date;

are applied properly throughout the year;

are effective to mitigate the inherent and control risks.

The verifier should do thorough checks on these procedures using the following methods: interview of personnel to assess competence and what they say they do in practice, review of documentation (including procedures and work instructions etc.), observation of how procedures and control processes are applied (where practicable), check of instruments and tools used etc.

Which of those ’checks’ does the verifier need to do on site?

Checks on site include the following:

Interviews with relevant operational personnel and administrative team;

Observation of how data flow activities and control activities are carried out by operator’s personnel;

Checking implementation in practice of procedures and control activities through interview and observation;

28

Testing whether the procedures are effective to mitigate the inherent and control risks by interviewing relevant persons/functions who are carrying out activities listed in these procedures;

Visual inspection of meters, and weighbridges etc.; collection of serial numbers and Field Tags from instruments in situ to cross check to records;

Review of records available on site.

Which can the verifier ignore based on them being checked and approved by the CA?

Trainers should promote discussion on the role of the CA versus the role of the verifier concerning procedures. What should a CA check in relation to procedures (and in how much detail) : do CA look differently at procedures when they relate to the monitoring methodology as compared to procedures on quality assurance and quality control? What are the responsibilities of the verifier and what of the CA?)

The verifier cannot ignore any controls or procedures. Not all procedures will necessarily have undergone a detailed review process by the CA as part of the MP approval process. So a thorough check by the verifier is justified. Some procedures may be included within a certified management system (e.g. EN ISO 14001, 50001etc.) which can be more easily checked by the verifier.

Moreover it is the verifier’s responsibility to check implementation of control activities and procedures and associated documentation and records in order to assess whether controls etc. are effective to mitigate identified risks.

There will of course be more focus on control activities and procedures where risks are high or weaknesses are identified or where control is not sufficiently robust. The verifier will also do cross checks against the MRR in this respect to ensure that practice is in compliance with requirements.

Question 2e as asked in Scenario 2 (consisting of two sub-questions)

How does the verifier check representativeness of samples taken of major source streams before these are sent to the laboratory?

In general, where field operatives take samples there is likely to be a higher risk that a non-representative sample is taken as compared to samples that are taken by specialist contractors or laboratory personnel.

The verifier should – if possible – observe samples being taken in order to evaluate whether the approach to sampling meets the specified standard that would result in a representative sample; and that persons taking the sample are applying the requirements of the sampling plan.

If it is not possible to observe (e.g. there is no delivery of fuel/material due) then a reasonable range of field operatives responsible for sampling should be interviewed to establish what they each do; and to compare responses to each other and to the sampling plan requirements.

In particular the verifier needs to establish that for batch deliveries of fuel/materials if consolidated samples are being created to send to the laboratory, the consolidation only relates to samples from an individual batch.

On what areas should the verifier focus when performing checks on the sampling plan?

Areas to focus on include:

Appropriateness of sampling to deliver quality required for the approved monitoring methodology (e.g. where sampling is occurring, conditions under which samples are taken and how samples are taken, whether sample seal bags/containers are used, how conditions of moisture were taken into account);

Correct and consistent application of frequency and method of sampling;

Representativeness of sampling and sampling patterns;

Application of specified sampling standards;

Changes to sampling plan and whether these have been agreed with the laboratory conducting the analysis, and also approved by CA;

Procedures in place for sampling to ensure it is controlled;

29

Competences of individuals taking samples;

Sample containers25, transport of samples to the lab, storage and other aspects of chain of custody26.

Question 2f as asked in Scenario 2

What checks should the verifier do regarding the use of non-accredited labs?

The operator is using an internal non-accredited laboratory to analyse fuel or material characteristics for

deriving the calculation factors. There is a corporate management system but this is not certified in line

with EN ISO/IEC 9001 or another equivalent certified quality management system and it is not clear how

effective it is at controlling the quality of the lab’s work. This means that the verifier has to perform

additional checks to ensure that the requirements listed in Article 34(2) and (3) of the MRR, as approved

by the CA in the MP, are actually being applied and proper quality control is being achieved.

Annex I of the FAQ AV provides examples of checks that a verifier will do regarding these articles. This

annex is based on the checklist of MRR Guidance Document 5 on sampling.

If the CA has approved the use of the non-accredited lab, the verifier will have to check and obtain

evidence that:

the laboratory is capable of managing its personnel, procedures, documents, laboratory standards (chemicals etc.), equipment, tasks and output reporting in a reliable manner. The verifier will do spot checks on this evidence. As the risks are high and the robustness of the corporate management system is unclear, spot checks will need to be quite extensive;

The requirements in Article 34(3) of the MRR in relation to determining the technical competence of the non-accredited laboratory and its personnel are met;

Inter-comparison or proficiency tests have been undertaken with other accredited (or recognised) laboratories.

It is not technically possible to use an accredited lab or the costs of doing so are unreasonable. If this is not the case, it is a non-compliance which should be reported in the verification report.

If the verifier is sufficiently confident in the technical competence and quality management of the laboratory, no further detailed checks will be carried out. However if the verifier is not sufficiently confident - which might be the case in this scenario - the verifier will perform further detailed tests and will assess the impact on the data of anything going wrong with analysis/reporting at the laboratory.

If the CA has not approved the use of a non-accredited laboratory, the verifier will direct the operator to

the CA to obtain required approval. If such approval is not obtained by the time the verification report if

finalised, the verifier reports the position in the verification report.

What impact would this have on the scope of the verification and the time allocated to complete the verifier’s work?

The risks are higher because it is an internal non-accredited laboratory that is not certified to EN ISO 9001

or another equivalent certified management system. The corporate management system does not seem

to operate in line with recognised standards and is rather vague which increases the inherent and control

risks and extends the depth and scope of verification. This in turn will have an impact on the time

allocation.

If in this scenario, the laboratory was an external one, there are additional checks that would be required

including:

25 Samples should be analysed as quickly as possible, containers should be correctly sealed and opened to avoid introduction of impurities, loss of quality etc. 26 Chain of custody means that there is a traceable chain of control from the point at which a sample is taken all the way through to the laboratory. In principal the term refers to the chronological documentation or paper trail, showing the trail, custody, control, transfer, analysis, and disposition of the sample; generally the trail using standard forms, unique reference identification numbers enabling tracking of an individual sample across the chain and knowing who did what and when etc.

30

The operator should –

o provide evidence of how they have satisfied themselves that there is no alternate to using

a non-accredited laboratory;

o provide evidence of the basis on which they have selected the specific laboratory and

how they have determined it is an appropriate laboratory; and how they have confirmed

the lab’s ability to control the quality of its processes and the competence of its

personnel. This would include the evidence supplied to the CA when the MP was

submitted for approval.

If the operator is not able to provide satisfactory evidence that would enable the verifier to confirm that

sampling and analysis is done in accordance with MRR requirements, the verifier would need to go to the

laboratory to conduct interviews and document checks etc. to check on the laboratory’s ability to deliver

the required quality and robust outcomes. This would also impact on the amount of time required for

verification and the number of locations that would need to be visited. It may also be necessary to involve

a lab expert in the verification team.

31

III.3 Model answers for Scenario 3, i.e. instructions to trainers

Cement Installation with an accredited laboratory and good internal controls

Scenario 3 presents the analogous installation as described in Scenario 2 but the installation is functioning much more effectively; it has more effective procedures and control activities and has lower control risks due to the fact that it has certified management systems and conducts significant levels of internal validation and audit of its GHG monitoring and reporting processes.

The focus of the discussion should be on how the verification will become simpler as a result, due to the lowering of inherent and control risks through more robust control activities and procedures that have been implemented. The verification risk would therefore be reduced requiring less extensive verification activities (please see KGN II.2 on risk analysis and KGN II.4 on sampling).

A key point for discussion relates to the reduction in the level of detailed tests that are required in the plan and therefore the amount of time that is required for verification – on both sides (the verifier in delivering verification and the operator in preparing for verification).

Question 3a as asked in Scenario 3

What difference does this make to the depth and scope of verification, and the type of checks to be made for this year?

The site operates an EN ISO 9001 quality management system that has been certified by an accredited external party. There is also a certified EN ISO 14011 environmental management system. All company management systems are subject to regular external audits, internal audits and periodic reviews. This means that the control activities are more robust. The inherent and control risk has decreased which means that the detection risk can be set higher and less detailed verification activities may be necessary.

Please ensure that you have a short discussion about the differences between certification (of the operator’s management system) and accreditation (of the external third party to confirm that they are competent to do certification). And also highlight the fact that it is possible to have a management system certified by an un-accredited third party (this is common) and therefore it is necessary for the verifier to see a copy of the certificate and check that the auditor is (or is not) accredited, before placing reliance upon the certification for the purposes of the legal verification.

In addition, the laboratory has undergone a full assessment to obtain EN ISO 17025 accreditation for the tests covered in the approved MP. This means that the verifier would have more confidence in the technical competence and quality management of the lab. The verifier will check:

the laboratory is accredited according to EN ISO/IEC 17025 by reviewing the laboratory’s accreditation certificate and scope and validity of accreditation;

the analytical tests agreed with the accredited laboratory have been carried out in accordance with the approved MP;

that the scope of the laboratory’s accreditation covers the required test methods and sample analyses mentioned in the approved MP.

Question 3b as asked in Scenario 3

What elements are important to be checked given the information in the context?

With respect to the accredited laboratory, the verifier will check whether the accreditation certificate is still valid and whether the laboratory’s accreditation covers the required methods and analyses that are applied to determine the calculation factors. In addition, for an internal laboratory it is reasonable for the verifier to check:

the most recent accreditation assessment report to identify if there were any issues identified during surveillance that might impact on emissions accounting, or that the verifier should be aware of;

maintenance and calibration of relevant laboratory equipment to ensure that it is up to date and

32

valid (e.g. sampling containers, analysers and gas bottles used for self-calibration of equipment) – the accreditation surveillance will not necessarily check this level of detail every time.

In this scenario one of the source streams listed in the approved MP is not mentioned under the list of source streams for which tests the lab is accredited. This concerns the municipal waste derived fuel.

If the verifier identifies that the lab’s accreditation does not cover the required methods and analyses for municipal waste derived fuel, the verifier will have to perform additional checks on the quality management and technical competence of the laboratory to assess the risks of material misstatements. This means that the verifier will do spot checks on compliance with Article 34(2) and 34 (3) MRR elements. See Annex I FAQ AV for examples of checks to be carried out.

In any case the verifier must assess whether the deficiency in the laboratory’s accreditation has an effect on the emissions data and must report the deficiency as a non-compliance27 in the verification report regardless of whether this has an effect on the data. Where the situation leads to material misstatements or one of the other negative verification opinion statements listed in Article 27(1) (c) or (d) of the AVR, the verifier must select “not verified” in the Commission’s verification report template.

With respect to the EN ISO 9001 management system and EN ISO 14011 management system as well as the audits, the verifier will have to be aware that these systems and audits do not relieve the verifier from testing control activities and especially EU ETS adaptations. The verifier will thus carry out checks on both management systems, the internal and external audits and the periodic reviews, for example, reviewing planning of audits and evidence of outcomes etc.

Question 3c as asked in Scenario 3

What questions should the verifier ask of lab and other personnel? And what documents and data should the verifier check and test?

Questions to ask laboratory and other personnel include:

if there have been any changes of significance that are relevant to the emissions data;

what changes of personnel have happened in the laboratory;

if they have any issues in relation to the quality of sampling that is happening out on the site.

Documents and data to check and test include :

the latest NAB assessment report;

laboratory procedures;

maintenance reports for relevant equipment;

documentation of sample results etc.

See annex I FAQ AV for examples of checks.

Question 3d as asked in Scenario 3

What other checks should the verifier make?

EN ISO 9001 management system

The verifier will check the third party auditor’s certificate and most recent audit report and assess what the scope of certification was, which parts were checked by the certifier and which were not. The verifier will also check to confirm that the system does in fact cover EU ETS monitoring and reporting.

EN ISO 14011 management system

The verifier will check the third party auditor’s certificate and most recent audit report and assess what

27 Whether it is a non-compliance depends on the extent to which information on the lab’s accreditation is included in the approved MP. If the information is provided in the MP, then it is a non-conformity.

33

the scope of certification was, which parts were checked by the certifier and which were not. The verifier will also check to confirm that the system does in fact cover EU ETS monitoring and reporting.

Note – it is possible for the two ‘systems’ to be integrated in practice so that there is only one system is in place but it is audited against the two different standards.

External and internal audits

The verifier will check the scope and subject matter of the audits, e.g. what parts were checked and which were not and which was done by the internal auditor and which by the external auditor; what alterations were made for EU ETS; who conducted the audits and whether they were competent to do so from an EU ETS perspective; if relevant documentation is available; what audit procedures were carried out by the third party; what recommendations were made in the audit report that are also relevant for EU ETS, and if so, whether these recommendations have been implemented; whether corrective action as a result of audits were implemented etc.

With respect to external audits the verifier will also check that the EU ETS mandated controls on outsourced processes have been applied: i.e. whether the operator has implemented control activities to ensure the quality of outsourced processes, e.g. assessing procedures for procurement, checking contracts with external parties, checking how an operator ensures that the party to which the activity is outsourced, carries out the activities according to the MRR and other requirements, if relevant.

Periodic reviews

The verifier will check whether the reviews have been performed with the frequency listed in relevant procedures; how these reviews were performed and by whom; where corrective action was taken as a result of these reviews; whether reviews were carried out by a competent person; whether a four eyes principle was applied to period reviews.

Where possible, the verifier will interview the operator personnel carrying out these reviews but should also look at the plans for review and the outcome findings and actions. The verifier will also assess to what extent internal review is relevant for EU ETS and how associated findings/corrective actions/proposed system adjustments are documented. This documentation should contain proof that internal reviews were performed and if changes are proposed/made to systems that affect EU ETS data monitoring, how the impact of changes on monitoring and compliance have been evaluated. Examples of such proof are visible signoffs after review, approvals by email and/or meeting minutes where amendments were discussed and agreed etc.

34

III.4 Model answers for Scenario 4, i.e. instructions to trainers

Refinery Scenario 4 presents a refinery which produces around 2,409,670 tonnes CO2 per year. The refinery is located a mile inland from the coast and has an annual crude oil throughput of around 210,000 barrels per day (bbl/day). The refinery receives crude oil by ship, primarily from the North Sea, and exports refined products by ship, road tanker, wagon and pipeline across Europe and to other parts of the world.

In considering this scenario, delegates should not consider that a refinery is more complex than it really is. With some exceptions (such as catalytic cracking), a refinery is basically a large collection of combustion sources al-be-it with unusual self-generated fuels. Technical competence on refinery activities should be represented in the verification team.

Question 4a as asked in Scenario 4

Given the context in this scenario, what inherent and control risks should the verifier identify and on what elements would the verifier focus its verification checks? Please indicate the reasons for focusing on these elements and indicate what questions the verifier should ask itself when identifying the risks and consequently determining the scope and detail of verification.

This question is just to introduce the case and is meant to identify the risks concerned. The following questions are more important. Question 4a is intended to show participants that the verifier’s risk analysis is key to determine what activities are carried out on site, which locations will be visited and how the site visits will be organised.

The verifier needs to consider the different risks, including:

Inherent risks:

Different types of source streams and emission sources compared to general combustion installations (i.e. both combustion and process emissions). The total emissions of the refinery and its internal power generation make it a category C installation;

Complexity of the data flow: e.g. multiple different points at which individual source streams might be measured and aggregated up to a single data point per obligated source stream;

Non-standard fuels e.g. Refinery Fuel Gas (RFG) derived from off gases collected from various processes, variability of gas;

Complexity of installation boundaries and the number of process/technical units that are involved which mean that there is usually a high number of measurement instruments that are involved in EU ETS monitoring (>50 instruments);

Complex data management systems: e.g. Distributed Control Systems (DCS), PIant Information Systems (PI) as well as potentially ‘black box’ databases and multiple spreadsheets;

Multiple different people handling different parts of the data flow (and different process unit monitoring within the installation).

Control risks:

Meters measuring RFG and Natural Gas not functioning properly; especially temperature and pressure probes which impact volumetric to mass conversions;

Unprotected IT systems;

Online gas analysers not functioning properly; not being calibrated with certified gases or the calibration gases being out of validity;

Technical competence of external lab – Is the lab accredited or not? Non-accredited lab means additional checks (see question 4.2 FAQ AVR);

Competence of personnel carrying out control activities;

35

Quality of management of the refinery; the way the control activities in the refinery are organised and maintained; the number of different people involved in aspects of data flows and QA/QC; the quality of the internal communication and training, as these are often not seen to be properly arranged in refineries and could have weak points and thus control risks.

There are a number of high inherent risks/ control risks involved with some steps in the data flow of the refinery: e.g.

Each refinery is a complex installation with various process units, source streams, emission sources, control systems and a huge data set; in first instance these are developed to control the process conditions and operations, thus they have different control objectives which have to be aligned with the control and quality objectives of emissions trading;

Some of the input materials might be both a source stream for combustion and product material input (e.g. Methane (CH4) for Hydrogen (H2) production);

In the scenario, the RFG is 51.3% of the fuel used and is collected from several process sources with different gas composition. The likelihood of material misstatements is high because in each refinery the data collection system and the data flow are unavoidably complex; and any error in the RFG accounting would have a high impact on the reported data due to RFG’s high contribution to the overall declared emissions value. In addition, homogeneity of RFG depends on whether there is a single point of blending and a ring main around the refinery or whether there is no ring main and multiple different points at which RFG is fed into the heaters etc. (meaning the emission factor could be different for each RFG source). In addition in this scenario, Natural Gas is used for pressure balancing in the RFG ring main and so does not necessarily constitute a source stream on its own;

Transfer of continuous meter readings of RFG to the PI system (via Flow Computers and DCS) which is then downloaded in multiple spread sheets and/or databases: High inherent and control risk because of the complexity of data management and data flow, and the likelihood of misstatements as well as high impact on reported data; more so if there is an element of manual transfer of data within the data flow and/or manual data validation/ substitution of data;

The extensive use of computer systems (both process control and ‘Microsoft’ type desktop applications) means that the risk of a breakdown of control within IT systems is high, verifiers should be looking for archive, backup systems, access control and other IT System and Information Security mechanisms, software validation;

Determination of the quantity and composition of the gas being flared during upset conditions, as this is very dependent on duration of outage and the vessel/process that has been blown down to the flare. The likelihood of misstatement is high, although the impact on the reported emission of the refinery may be low.

On the basis of its assessment of the inherent and control risks, the verifier needs to determine the nature, timing and depth of the verification activities and, through those activities, lower the verification risk28 to an acceptable low level in order to be able to issue a verification report with reasonable assurance that the operator’s report is free from material misstatements.

The greater the risk of a material misstatement as a result of high inherent and control risks, the more extensive the number of verification activities (testing, sampling) needs to be. The materiality level of 2% means that the detection risk must be set even lower). The verification should in particular focus on those areas that have the higher risks.

Question 4b as asked in Scenario 4

What parts of the installation should the verifier visit whilst on site and how should they determine what they look at and who they should interview? The verifier’s risk analysis should indicate that the verifier would visit the following locations:

28Verification risk is the overall risk that the verifier issues an inappropriate verification opinion. It consists of three components,

i.e. inherent risk, control risk and detection risk.

36

The head office to review documentation and procedures;

Site to inspect the metering equipment, to assess the installation boundaries (e.g. emission sources and source streams), to assess the data flow and control activities, the installation boundaries (e.g. the different units, source streams, flares);

Accredited lab to assess the procedures and to interview personnel on their assessment of the quality of samples coming into the lab from field operators;

Process control room, finance department, location where maintenance records of equipment are stored, location where IT systems are placed;

Observation on how samples are taken following the chain of custody in the sampling process;

Potentially, other locations associated with an installation (e.g. jetties, storage yards, materials handling/load/unload locations) to confirm that there are no obligated sources/source streams (although it may be possible to determine this using plans, maps and piping/instrumentation diagrams etc.);

Other locations in relation to particular risks identified as part of the verifier’s risk analysis.

Question 4c as asked in Scenario 4

Should the verifier check anything in relation to the CEMS? If yes, what should they check?

It is not necessary to check anything in relation to CEMS because the on line gas analysers are not used for continuous emission measurement of CO2 so the approved methodology is not considered to be via a CEMs. The EU ETS requirements in relation to CEMS are therefore not applicable; but any available evidence from CEMs instrument testing may provide relevant evidence of the quality control of the analysers (see also additional question below). However, verifiers should be aware and take into account the annual validations required in relation to using on line gas analysers regarding MRR Article 31(2), and the scope of using such analyses as further set by MRR Article 32(3).

Additional question that could be addressed in the discussion:

What checks would the verifier perform on gas on line analysers?

The verifier would for example check:

the in-situ instrument to ensure the serial number is the same as listed in the records;

whether the online analyser is maintained and calibrated by an appropriately qualified engineer;

whether self-calibration has been done using calibrated standard gases;

whether the correct standards are applied;

whether the calibration services are accredited and the suppliers of calibration gases are accredited in accordance with ISO 17025;

whether the composition of calibration gas resembles the components to be measured within the test medium;

whether the operator has done an inter-lab comparison on a sample of gas (i.e. running it through the analyser and getting it analysed in an accredited lab.

Additional question that could be raised in the discussion:

This question could be interesting to discuss since there appears to be considerable misunderstanding about what a verifier should check when CEMS is applied. If there is time, please add this question to the discussion.

If the operator was using CEMS to measure the CO2 concentration, what would the verifier have to check in that case?

37

The verifier has to assess whether the data flow as described in the approved MP meets the actual practice by testing the data flow activities, checking the data trail and following the sequence and interaction of the data flow activities. The verifier must carry out the same activities as explained in section 2.1 of KGN II.3. The verifier will for example check location of stacks/ducts, diagrams of emission points, process types and variations, how meter readings are transferred to the data management system; and if it is automatic, what validation checks have been done to ensure that hand off of data room one system to another is properly done.

The verifier must test the control activities, based on the verifier’s analysis of the inherent and control risks involved. Section 2.2 of KGN II.3 outlines the different control activities and the checks that verifiers carry out on these activities. Application of EN 14181 is the key element in the quality assurance of continuous measurement systems. When checking the control activities the verifier must include certain checks on the application of the QALs and Annual surveillance test (AST). See Guidance Document 7 on CEMS for more information.

The verifier will check procedures that are specifically relevant for a measurement based methodology. See Guidance Document 7 on CEMS for more information.

The verifier will also :

do checks related to flue gas flow and other peripheral measurements and calculations. See Guidance Document 7 on CEMS for more information.

substantive data testing consisting of analytical procedures, data verification and assessing correct application of the monitoring methodology to detect misstatements and non-conformities. The extent to which this data testing is carried out depends on the outcome of the verifier’s assessment of the data flow; control activities and the procedures; and the subsequent verifier risk analysis. More information on these activities is provided in KGN II.3. Basically, the same checks are performed by the verifier when assessing application of a measurement based methodology and verifying the relevant data. An additional check, specifically required for a measurement based methodology, is outlined in Article 16(2)(g) of the AVR – which states that verifiers must check the measured values by using the results of corroborative calculations performed by the operator (see section 4 Guidance Document 7 on CEMS for the requirements on corroborative calculations).

Examples of CEMS-specific checks that the verifier will make during analytical procedures, data verification and assessment of a measurement based methodology are:

what standards are applied and whether these standards are complied with; check on representativeness of measurements;

completeness of hourly data and of substitution data for incomplete hours;

calculations and underlying measurements if the flow rate is calculated;

calibration and maintenance documentation for flow and concentration measurements;

whether correct substitute values have been used if there have been missing data;

whether the CA has been notified if any part of the CEMS has been out of operation for more than five consecutive days.

Question 4d as asked in Scenario 4

In the context of this scenario, list the things that the verifier should check as part of the analysis of uncertainty for:

i. Measurement instruments (e.g. Flow instruments etc. other than CEMS and online gas analysers The verifier must check the validity of information used to calculate the uncertainty levels and confirm that they match the levels approved in the MP. The type of information used depends on the methodology; the type of measurement instrument; and on the approach the operator applies to calculate uncertainty levels and to demonstrate that required tier and associated uncertainty threshold is

38

met. The approach used to assess whether required uncertainty is being met, depends on whether the measurement instrument is under the operator’s own control or under the control of other parties. Three different routes are available

If the measurement instrument is subject to national legal metrological control, the verifier would check if the measurement instrument is covered by national legal metrological control, e.g. checking the certificate of official verification of the instrument; and the required specifications from the national legal metrological control institute. The verifier must be sufficiently confident that the instrument is regularly calibrated.

if the measurement instrument is not subject to national legal metrological control but the instrument is installed in an environment and under conditions that are appropriate for its use specifications. The operator can use a maximum permissible error specified for that measurement instrument in service, or use an uncertainty assessment obtained by calibration multiplied by a conservative factor. The verifier would then check :

o manufacturer’s specifications, calibration certificates;

o specifications from legal metrological control and specifications of meters (flow range, checking what can influence the meter and how measurement results are determined); and

o procedures implemented by the operator to ensure that activity data are measured against the relevant standards.

If the operator undertakes a specific and extensive uncertainty assessment, the verifier would check the input information used to calculate the uncertainty level. This means that the verifier will have to check all relevant information used for this specific uncertainty assessment. The verifier is not specifically required to check the calculation itself; but to manage the verifier’s risk it is sensible to do so to ensure that both the input information is reasonable and that the output data are a reliable basis for the verifier confirming that the installation is compliant with its approved tier thresholds.

If the measurement instruments are outside the control of the operator, the verifier must check the validity of the information being provided as evidence by the operator. In order to sign off on this aspect of the monitoring process, the operator must make relevant evidence and documents available to the verifier. This means that the operator must obtain information on, for example, calibration results and the manufacturer’s specifications of the instruments used by trade partners etc.

ii. Calculation factors based on sampling For determining calculation factors in a calculation based methodology, uncertainty plays a role in situations where the operator chooses to apply a frequency different to the minimum frequencies for the analysis listed in Annex VII of the MRR. In this case the operator determines the frequency of analysis by using the approach outlined in the IMPEL/ETSG Excel tool on frequency of analysis which is based on section 4.2 of the MRR Guidance No. 5 on Sampling and Analysis (GD 5).

This tool enables the operator to demonstrate that any variation in the analytical results (or the calculation factors derived from them) for a fuel or material does not exceed 1/3 of the uncertainty value which the operator must meet for the monitoring of activity data for that specific fuel or material.

The determination of this variation has to be based on historical data and can include analytical values for the monitoring of the respective fuels or materials in the reporting period immediately preceding the current reporting period. The verifier checks that the historical data and other information used in the tool/calculation matches prior and current year evidence of analytical results.

Question 4e as asked in Scenario 4

Given the circumstances described on site, is there a limitation being placed on verification activities; if yes, how might these be overcome, or what would the consequences be?

The refinery is in a shutdown for major maintenance on two of its three core process units. The verifier will assess the impact on the data and assess the risks to the verification as parts of the installation are not accessible for inspection and key personnel are not available for interview or to observe their performance in monitoring, reporting and carrying out control activities. The verifier may not be able to

39

collect evidence necessary to reduce verification risk and/or to come to an appropriate verification opinion statement with reasonable assurance that the data is free from material misstatements: e.g. the verifier will not have the choice to be able to visit and inspect all parts of the installation they determine are necessary; as the whole installation is not in operation the verifier will not be able to check all relevant processes; and unless the visit is timed carefully the verifier will not be able to interview certain key personnel. However, it is noted that it might be possible to arrange telephone interviews before or after the scheduled visit and to follow up with requests for information once the visit is complete.

The verifier will check whether the operator needs to notify the shutdown29 to the CA; and if yes, whether this has happened; whether agreements are required from the CA (and obtained) for alternate approaches etc. (e.g. applying an interim methodology for the parts that are still working if it is not possible to apply the approved methodology). If CA notification and agreement is required, but has not happened the verifier will direct the operator to notify the CA about the shut-down.

Furthermore the verifier needs to determine if, due to an inability to visit and/or interview key aspects of the installation, there is a limitation of scope the verifier will have to determine whether it has to stop the verification.30

If there is a limitation but the situation is resolved before the verification report must be issued, verification can continue. Whether the verifier can still give an affirmative verification opinion statement depends on the time left available to complete work, impacts on data, the verification risks involved; any specific issues identified; and the ability of the verifier to collect sufficient evidence. The verifier will have to check the impact of the shut down on the overall data (and compare to a reasonable proxy e.g. production data) as well as report the shutdown situation in the verification report.

If the verifier is not able to collect sufficient information to carry out the verification and/or the operator cannot provide evidence required for the verifier to continue its work, and to reduce verification risk, the verifier will have to report a limitation of scope in the verification report and give a declined verification opinion statement. A declined ("not verified") verification opinion statement must also be issued if the shutdown provides insufficient clarity and prevents the verifier from stating with reasonable assurance that the data is free from material misstatements (Article 27 (1) (d) AVR).

In the case of a declined verification opinion statement the CA has to conservatively estimate the emissions according to Article 70 MRR.

Essentially a limitation is imposed on the verification if the verifier does not have the full set of information about how the reported emissions data has been compiled; a free hand to evaluate the inherent and control risks associated with a set of emissions accounts; and freedom to determine what the verifier will test, where they will go to inspect/check etc., and who they will talk to etc.

Limitations can be actively imposed (e.g. operator preventing a verifier from doing something), or passively imposed (e.g. information has been lost somehow and is not recoverable). Other examples of limitations include:

Important data is missing which means that the verifier cannot obtain evidence needed to reduce verification risk to a level required to enable them to have reasonable level of assurance over the reported emissions, e.g. some or all primary source data is missing and data is only available at an aggregated level; or the verifier has been prevented from trailing the data back to source.

the MP has not been approved by the CA so cannot be used as the criteria against which assurance is provided i.e. there is no proper reference document for the verifier to check the report against.

the MP does not provide sufficient scope or clarity about what should be happening at the

29 Whether notification is required depends on the nature of the shutdown and how long it goes on for. A shutdown does not necessarily lead to a change in the MP (there is no fuel/material being consumed and there are no emissions during that period. In some cases the shutdown can lead to a (significant) change to the MP (e.g. if a new emission source is installed, if additional source streams are used or if new meters are being installed). In those cases notification is required. In this particular case there is no change in the MP so there would be no need for notification. If the shutdown lasts for more than 6 months (e.g. explosion), this could mean a temporary cessation of operations which would also require notification to the CA. 30 Note – it is unlikely that in general a plant shutdown would constitute a limitation on the scope of verification as when the plant is shut down there is usually no activity data to be verified.

40

installation and therefore the verifier cannot conclude whether what is actually happening on site is what is supposed to be happening on site; and therefore the verification cannot be completed.

the operator has failed to make sufficient information available to enable the verifier to carry out the verification: e.g. the operator has not provided the verifier with:

o the latest version of the approved MP;

o primary source data needed to check accuracy of reported data e.g. fuel invoices, or results of online measurements;

o information on measurement equipment and quality assurance that has been applied to it (e.g. manufacturer’s information, calibration records, maintenance information);

o failure to provide information may be as a result of a ‘hostile’ operator, who does not wish to support verification activities, or more likely, of a disorganised operator who does not have good internal control of emissions accounting, in practice.

41

III.5 Suggested priorities for addressing questions in scenarios

Scenario 1 Question 1a: only treat as introduction to get the participants familiar with the case scenario’s: Of particular interest: installation boundaries (role verifier), data flow and procedures for managing data flow, operator’s risk assessment, control activities Main priority is with question 1c, 1e, 1f, 1g and 1h. Please note the focus points in the text when addressing the questions. For question 1d the role of the CA versus the role of the verifier is important to address here as there is sometimes much confusion on the roles. Some issues such as procedures, sampling plan and uncertainty assessment are covered in later scenario and could be picked up under these scenario’s. Question 1g is a question which is also asked in scenario 2 which goes in much more detail and concerns a badly run installation. If time is short you could treat the detail under scenario 2. Scenario 2 Question 2a: please focus on the specifics of the case since installation boundaries was also addressed under scenario 1 (e.g. the quarries and the tertiary crusher and product loading yards). What falls under the scope of EU ETS and how does the verifier check this. Question 2b: You could make a choice to treat this together with 2c. Please note the potential non-compliance issues Main focus is question 2c. As inspiration for completing the verification plan template with the discussion group you could use the Example of risk analysis and sampling published on the Commission website. Other main priorities are question 2d, 2e and 2f (role verifier and CA, see instructions) Scenario 3 Could take less time than scenario 2. The idea is to show that the verification plan can be much simpler for the well-run installation with lower risks. Just focus on the differences between checks compared to scenario 2. Scenario 4 Question 4a is just an introduction to the case scenario and should not take much time. Main focus is also question 4b (locations of site visits) and question 4e limitation of scope (try to show the participants to meaning of limitation of scope, when can it arise (see instructions). Main focus is question 4c (highlighting the differences between CEMS/online gas analysers). Try to also capture the additional questions. Question 4d: Question is also asked in scenario 1. Depending on time available, you could either choose to treat this in full under this case scenario or only address the specifics if you have already covered this under case scenario 1.

42

IV. Main conclusions from the plenary session on information

exchange

The discussion showed that:

NABs sometimes have difficulty understanding information provided by the CA to the NAB

according to Article 72 AVR.

Adding more detail to the information reported in the information exchange according to Article

72 AVR (by e.g. explaining the complaint or issue identified in the review of AERs) could improve

the NAB’s understanding of the information.

Direct communication between the CA and the verifier on issues that the CA identifies in their

review of AERs or inspection could make the process of information exchange more efficient.

However, it is important for the NAB to always be copied in on that information exchange as the

NAB is the responsible party to take further action, if required.

However, direct communication between the CA and the verifier could mix up the roles and

responsibilities of the CA and the NAB and care would be necessary to avoid compromise of the

independent role of the verifier as required by Article 7(3) AVR.

There are differences between MS on how NABs address complaints and provide feedback to the

CA on how these have been addressed.

It is important to collect more information on how complaints about verifiers are being addressed

in different MSs and what information is exchanged between CA and NAB on this subject. The

managers of the Task Force AV of the Compliance Forum and the EU ETS Network of the

European cooperation for Accreditation are invited to organise trawls of their respective

memberships for this further information.

More harmonisation between MS on how NABs report back to the CA on information provided by

the CA and at what level of detail CA and NABs exchange information could improve the quality of

the information exchange process. The need for additional guidance or amendment in the

templates of information exchange should be assessed.

If the NAB does not provide feedback to the CA on a complaint they have made, or if the CA has

concerns about how the NAB is dealing with a complaint, the CA can submit a complaint to the

European co-operation for Accreditation about the NAB's approach. The European cooperation

for Accreditation will then investigate the complaint about the NAB.

43

V. Summary and conclusions

The training was highly appreciated with 90 % of those providing feedback rating the training material

excellent or very good. Delegates confirmed appreciation of the opportunity to exchange views and

information between representatives of NABs, verifiers and CAs (from different Member States). The

discussion group approach to the training once again worked well. Good case scenarios and the

competence of dedicated trainers play an important role in allowing efficient coordination of

discussion group activities. Nearly 90% of participants expressed the wish to cascade the training

material to their own organisation.

The slides on summary and conclusions of the Training Event presented on the day are enclosed

below.

44

45

46

47

48

Annex I: Programme of the 2017 EU ETS Compliance Forum Training Event on Accreditation and Verification

Day I of the A&V Training on Verification Issues

Time Session Who

10:00 - 10:15 Opening, welcome, agenda and objectives of the training event Chair

10:15 -10:30 Introduction of the base case study (Scenario 1) and the issues for discussion in Session I

Machtelt Oudenes

10:30 - 12:30 Session I Group Discussion on the base case study (addressing the least complex situation)

Participants

12:30 - 13:30 Lunch

13:30 - 13:35 Opening of the afternoon training Chair

13:35 -13:45 Introduction of further complexity in the case studies under Scenario 2 and 3, and issues for discussion in Session II

Machtelt Oudenes

13:45 - 15:45 Session II Group Discussion of issues and related questions Participants

15:45 - 16:00 Tea break

16:00 - 16:45 Main conclusions from Session I and II and plenary discussion All

16:45 -17:45 Plenary discussion on complaints about verifiers and information exchange between CA and NAB

All

17:45 - 18:00 Final remarks and close of the Day I training Chair

Day II of the A&V Training on Verification Issues

Time Session Who

09:00 - 09:10 Opening and introduction of the agenda and objectives of Day II Chair

09:10 -09:20 Introduction of the more complex case study under Scenario 4 and the issues for discussion in Session III

Machtelt Oudenes

09:20 - 11:45 Session III Group Discussion of issues and related questions Participants

11:45 - 12:05 Coffee break

12:05- 13:15 Main conclusions from Session III and Event/ Lessons learnt All

13:15 -13:30 Final remarks and close of the Day/Event Chair

49

Annex II: Introductory presentations to explain Objectives and

Aims of the Accreditation and Verification Training

50

51

52

53

54

55

56

57

58

59

Annex III: Presentations from the plenary session on

information exchange

60

61

62

63

64

65

66

67

68

69