Pilot Audit Program Evaluation - Clean Energy Regulator Clean Energy … · 2015-04-15 · Pilot Audit Program Evaluation . ... 1. introduCtion 7 1.1 Background 8 2. ... as part of

1 RENEWABLE ENERGY TARGET

National Greenhouse and Energy Reporting

Pilot Audit Program Evaluation

Pilot Audit ProgrAm EvAluAtion 2

This evaluation includes the views or recommendations of third parties and does not necessarily reflect the views of the Australian Government, or indicate a commitment to a particular course of action.

The material in this evaluation is provided for general information only, and on the understanding that the Australian Government is not providing professional advice. Before any action or decision is taken on the basis of this material the reader should obtain appropriate independent professional advice.

While reasonable care has been taken in preparing this evaluation, the Commonwealth provides no warranties and makes not representations that the information contained is correct, complete or reliable. The Commonwealth expressly disclaims liability for any loss, however caused and whether due to negligence or otherwise, arising directly or indirectly from the use or reliance on information contained in this evaluation by any person.

ISBN 978-1-922003-41-6

The information in this document was prepared before the Clean Energy Regulator was established. Corporations that were registered under the National Greenhouse and Energy Reporting Act 2007 (http://www.comlaw.gov.au/Series/C2007A00175) were required to report to the Greenhouse and Energy Officer (GEDO). From 2 April 2012, the GEDO has been replaced by the Clean Energy Regulator. For more information on current auditing requirements please visit www.cleanenergyregulator.gov.au/National-Greenhouse-and-Energy-Reporting.

This work is licensed under the Creative Commons Creative Commons Attribution 3.0 Australia Licence. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/au.

The Department of Climate Change and energy Efficiency asserts the right to be recognised as author of the original material in the following manner:

© Commonwealth of Australia (Department of Climate Change and Energy Efficiency) 2012.


Contents

exeCutive summary 4

Key recommendations 5

1. introduCtion 7

1.1 Background 8

2. overview of the pilot audit program outComes 9

2.1 Assessment of fitness for purpose and effectiveness of delivery of the audit types 12

2.2 Consistency of approach, particularly between auditors from differentbackgrounds and impact on final audit product 12

2.3 Recommendations: Audit program—promoting effective outcomes 13

3 audit proCess and reporting 14

3.1 Information provided in the formal audit report 15

3.2 Three-way process—confidentiality and liability 16

3.3 Recommendations: Auditor engagement and reporting 16

4 appliCation of materiality and modified opinions 17

4.1 Application of materiality 17

4.2 Application of category of opinion in auditor’s conclusion 18

4.3 Interpretation of legislation 21

4.4 Recommendations: Audit practice specific to greenhouse and energy audits 21

5 reporting entities and the audit proCess 22

5.1 Auditable records 22

5.2 Presubmission audit 23

5.3 Recommendations: Enabling audited bodies to get the most from the audit process 24

6 Common diffiCulties experienCed in preparing nger reports 25

6.1 Responsibility for reporting 25

6.2 Contractors and subcontractors 26

6.3 Complete identification of sources 27

6.4 Recommendations: Opportunities to address common reporting challenges 27

7 ConClusions 28


exeCutive summaryThe objective of a greenhouse and energy audit is to enable the auditor to express an opinion on whether an organisation’s reported information fairly represents its greenhouse and energy activity, and has been prepared in accordance with the framework prescribed by the National Greenhouse and Energy Reporting (NGER) scheme.

Between April and August 2011, the Department of Climate Change and Energy Efficiency (department) undertook a pilot audit program under s.74 of the National Greenhouse and Energy Reporting Act 2007. It was designed to test the methodology and provided input to a full audit program to commence in November 2011. The pilot audit program covered as wide a range of audited bodies, audit types, and auditors as possible. A total of 22 audits of reporting corporations were undertaken, including reasonable and limited assurance, and verification audits.

The methodology used to evaluate the pilot audit program was designed to ensure an accountable, independent and transparent evaluation process. A key element involved direct engagement with stakeholders including audit and auditing standard setters, the greenhouse and energy auditors, professional assurance and accounting bodies and the audited bodies themselves.

Overall, the audit reports demonstrated a fair level of data integrity and significant voluntary compliance with reporting obligations. Evidence of reporting entities processes and activities, provided by both the auditors and the corporations involved in the pilot program, demonstrates that there currently exists a well-developed understanding of the reporting obligations, and a developing capacity to compile and submit essentially compliant NGER reports, with data that is substantially accurate and complete.

The outcomes of the audits themselves show that the audit approach was deliverable and able to identify misstatement and issues with regard to the NGER reports, and be able to provide advice and recommendations to the audited body on potential improvements to the reporting process. This demonstrates that the greenhouse and energy audit process is able to contribute effectively to data improvement, and to promote voluntary compliance.

A number of positive and successful behaviours were identified in the pilot audit program, as well as a number of issues arising and challenges. These findings will be considered by the Regulator with the objective of identifying opportunities for development or improvement, particularly around efficiency and data integrity.

The approach to taking the recommendations forward will include direct engagement with stakeholders to identify practical and effective solutions, and working with the NGER community to communicate the outcomes to all audiences, both reporting corporations and auditors, and others who use the NGER scheme.

The evidence from the pilot audit program shows that presubmission engagement of auditors and the increasing knowledge base of reporters would put reporting entities in a strong position to be able to deliver on timely completion and submission of NGER reports.

Clear guidance on record-keeping requirements to collate and keep readily accessible all the relevant information, records and data could be developed using an ‘evidence pack’ approach. This approach would be of significant advantage to NGER reporting entities by ensuring there is an expeditiously retrievable, auditable trail of evidence. The benefit in the audit context is that the auditor is able to complete the audit engagement more efficiently, in both time and cost, with a reduced risk of an incorrect opinion.

Whilst there are inherent limitations in drawing conclusions from a small number of audits, the overall conclusion from the evaluation is that the current audit methodology and process is a reasonable basis to proceed with a full audit program to commence in November 2011. Changes and improvements could be developed concurrently with a full audit program with any changes to apply to the next reporting period. This approach would align with timing for a new international auditing standard, currently under consideration, and the introduction of the Clean Energy Future package.


Key recommendationsThe 11 key recommendations are based on findings from the pilot audit program as follows:

audit program—promoting effective outcomes

Recommendation 1

The current audit methodology and process is a reasonable way to continue in delivering a full audit program.

Recommendation 2

Develop communications for stakeholders to provide advice on factors contributing to a compliant NGER report and a successful audit outcome.

auditor engagement and reporting

Recommendation 3

Ensure that the content of the auditor’s report is as useful and informative to the Regulator as possible by working with auditors and assurance practitioners on options to enhance the content of the formal audit report and developing consistent reporting content for Part A and B of the formal audit report.

Audit practice specific to greenhouse and energy audits

Recommendation 4

Develop guidance for NGER auditors on the consideration of materiality in greenhouse and energy audit engagement, and assessment of the potential use of appropriate benchmarks which auditors may consider in determining materiality for the NGER report.

Recommendation 5

Collaborate with the Auditing and Assurance Standards Board (AUASB) and professional organisations to determine if further guidance is required for auditors in order to achieve consistency in the provision of qualified conclusions.

Recommendation 6

Develop a formal process for resubmission of data, where errors are identified, as part of a greenhouse and energy audit.

Enabling audited bodies to get the most from the audit process

Recommendation 7

Consider changes to the NGER requirements for record keeping, and the development of guidance in the form of an ‘evidence pack’ to ensure that record keeping contributes effectively to providing an expeditiously retrievable and auditable trail of evidence.

Recommendation 8

Collaborate with AUASB and professional organisations to develop guidance that would define circumstances where presubmission audits would be of particular benefit to reporting entities.


Recommendation 9

Promote the benefits to the reporting entity of providing an assurance report to the Regulator; or advising the Regulator that assurance has been undertaken on the submitted NGER report.

Opportunities to address common reporting challenges

Recommendation 10

Working with reporting entities to develop guidance or strategies that would contribute to improved data submission from contractors and service providers, and options for providing assurance over this third-party data input.

Recommendation 11

Consideration of the challenges associated with providing data on minor sources in the greenhouse and energy audit process. In particular, to address cases where emissions or activities have not been estimated, or estimated at an insufficient level of quality, and develop guidance on how this should be transparently documented and justified.

Dr Lesley Dowling

Audit Development and Management


1 introduCtionIn the approach to climate change and energy policy there is a need for thorough monitoring and reporting of greenhouse gas emissions, and energy activity. There are a number of existing national and global monitoring and reporting schemes for this type of information. Many of these schemes describe methodologies for collecting and compiling data for participants to use in reporting information. The monitoring and reporting methodologies are designed to deliver consistency, reliability and cost-effectiveness in information capture.

Many schemes also include a quality assurance process in the form of third-party assessment of reported information. The independent assessments generally provide evidence of the correct usage of a measurement and reporting methodology, and the accuracy of the data determined on this basis.

In the case of NGER data quality is pursued by the Regulator through the principles of transparency, comparability, accuracy, and completeness. The assessment of data quality is linked to compliance with these principles, evidenced by adherence to the monitoring and reporting methodologies described in the regulations.

The pilot audit program was conceived to provide feedback on the design and execution of the NGER audit framework prior to the introduction of a broader and ongoing NGER audit program in 2011–12.

The objective of this evaluation is to assess the outcomes of the pilot audit program using the results of the audit engagements undertaken as part of the pilot. A review of the procurement process has been undertaken separately and is out of scope for this evaluation.

The findings from the evaluation will be used to improve the effectiveness of the methodology, respond to feedback from both auditors and reporting corporations, and to shape the future ongoing risk and intelligence-based audit program.

The methodology used to assess the pilot audit program was designed to ensure an accountable, independent and transparent evaluation process. A key element involved direct engagement with stakeholders including audit and auditing standard setters, the greenhouse and energy auditors, professional assurance and accounting bodies and the audited bodies themselves.

The pilot audit program covered as wide a range of audited bodies, audit types, and auditors as possible in the scale of the program. However, there are inherent limitations in drawing conclusions based on a relatively small sample of audits. The findings of the evaluation should be considered in conjunction with relevant information from business outreach, report validation, and direct stakeholder engagement activities.

Under the Terms of reference of the evaluation, the recommendations were to focus on:

• describing the expected content and quality of audits prepared under the audit framework, and

• providing guidance on the effective exercise of audit powers.

In considering the outcomes of the pilot audit program it is important to note that audit quality is dependent on inter-related factors such as the effectiveness of the NGER audit framework; professional standards applying to audit practitioners; and the professional skills and experience of auditors.

A number of positive and successful behaviours were identified in the pilot audit program, as well as a number of issues arising and challenges. Issues and opportunities identified by the pilot audit program will be considered by the Regulator. The Regulator is looking to identify opportunities for development or improvement, particularly around efficiency and data integrity.


A broad range of stakeholders were consulted in preparing the evaluation. Their input will also be sought to identify practical and effective solutions to the issues identified in the evaluation. It is also proposed that a comprehensive process be put in place to work with the NGER community to communicate the outcomes of the evaluation to all audiences, both reporting corporations and auditors, and others who use the NGER scheme.

The report provides a summary of the audits undertaken and the auditor’s conclusions in Part 2. The sections following the overview consider report on the findings from the perspective of: the audit process and reporting (Part 3); the application of materiality and modified opinions (Part 4); the reporting entities and the audit process (Part 5); and common difficulties experienced by reporters in preparing NGER reports (Part 6). Recommendations are provided at the end of each of these sections.

1.1 BackgroundUnder the NGER scheme, reporting is mandatory for corporations that have operational control over facilities whose activities meet certain corporate or facility thresholds. The reporting entities must provide annual reports to the Regulator, prepared in line with the methods and criteria provided in the regulations.

The role of the greenhouse and energy audit is to provide evidence of the correct usage of the measurement and reporting methodologies, and the accuracy of the data determined on this basis.

The Regulator may choose to initiate audits on a risk management basis, or to gather information on the reporting entities general compliance with obligations. The Regulator may also initiate and audit where specific non-compliance is suspected.1

The greenhouse and energy audit framework may also be used on a voluntary basis by corporations, where a reporting corporation wishes to obtain a level of assurance that it is in compliance with its obligations, or to inform potential customers or investors.

The auditor undertaking an audit must be a registered greenhouse and energy auditor and must ensure that the requirements of the legislation, including the NGER Audit Determination, are satisfied. The reporting corporation must agree to be audited, and must provide the audit team with all reasonable facilities and assistance necessary for effective exercise of the audit team’s duties.

The primary purpose of the pilot NGER audit program was to trial audit methodology, examine the accuracy and completeness of data, and identify any areas where additional support for reporting corporations might be needed.

The evaluation of the pilot audit program outcomes was undertaken inhouse with peer review by an external expert. The evaluation was based on the following sources of evidence: desk top review of pilot program documents and audit reports; and interviews with auditors, audited bodies and other stakeholders, including experts in professional audit practice and development of international audit.

1 s.73 Audits of registered corporations—compliance; s.74 Audits of registered corporations—other; National Greenhouse and Energy Reporting Act 2007.


2 overview of the pilot audit program outComesThe objective of a greenhouse and energy audit is to enable the auditor to express an opinion on whether the organisation’s reported information fairly represents its greenhouse and energy activity, and has been prepared in accordance with the framework prescribed by the NGER scheme.

The National Greenhouse and Energy (Audit) Determination (2009) provides the requirements to be met by auditors in preparing for and carrying out greenhouse and energy audits, and in preparing assurance engagement reports or verification engagement reports.

The requirements were prepared with reference to a number of existing standards, in particular those prepared by the AUASB, the Accounting Professional and Ethical Standards Board (APESB) and the International Organisation for Standardisation (ISO).

There are two types of greenhouse and energy audit defined in the NGER legislation:

1. an assurance engagement• reasonable assurance—opinion: positive expression• limited assurance—opinion, negative expression

2. a verification engagement—no opinion provided.

In a verification engagement the audit team leader carries out specified procedures for the purposes of checking the matter being audited but does not provide an opinion.

In an assurance engagement, the audit team leader provides an independent opinion about the reliability, accuracy and completeness of the matter being audited and uses professional judgment in preparing for and carrying out the audit and preparing the assurance engagement report.

There are two types of assurance engagement a greenhouse and energy auditor may perform. They are consistent with ASAE 3000, Standard on Assurance Engagements:2

1. Reasonable assurance engagement

The objective of a reasonable assurance engagement is a reduction in assurance engagement risk to an acceptably low level in the circumstances of the assurance engagement as the basis for a positive form of expression of the assurance practitioner’s conclusion. Reasonable assurance means a high, but not absolute, level of assurance.

2. Limited assurance engagement

The objective of a limited assurance engagement is a reduction in assurance engagement risk to a level that is acceptable in the circumstances of the assurance engagement, but where that risk is greater than for a reasonable assurance engagement, as the basis for a negative form of expression of the assurance practitioner’s conclusion.

Both reasonable assurance and limited assurance engagements require the application of assurance skills and the gathering of sufficient appropriate evidence to obtain an understanding of the subject matter and assurance circumstances. The nature, timing and extent of evidence-gathering procedures however are deliberately limited in a limited assurance engagement relative to a reasonable assurance engagement.

The decision on whether to seek limited or reasonable assurance is usually made on a cost/benefit basis; however if the auditor suspects a material error as part of the engagement (limited or reasonable), they will perform evidence-gathering procedures sufficient to enable the auditor to report.

2 Standard on Assurance Engagements ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information,

July 2007, Auditing and Assurance Standards Board.


There are currently three types of greenhouse and energy auditor registered:1. Category 1 technical2. Category 1 non-technical, and3. Category 2.

For currently registered greenhouse and energy auditors, those auditors in Category 2 are generally qualified and experienced assurance practitioners, many have considerable experience of assurance in the financial sector. Category 1 auditors generally have a higher ratio of technical experience compared to assurance experience. For the pilot program the assurance audits were undertaken by Category 2 auditors, as this registration category is assessed against the requirements considered to provide the level of expertise necessary to lead assurance engagements. The verification audits were undertaken by Category 1—technical registered auditors.

A total of 22 engagements were undertaken as part of the pilot audit program. The audits were undertaken by 13 audit firms including: experts in international accountancy and professional services (‘Big 4’); accountancy services; engineering and environmental services (see Table 1 below).

The approach for selecting corporations was to use criteria that minimised the amount of subjectivity in the decision-making process. An initial filter of a significant drop in Scope 1 or Scope 2 emissions from 2008–09 to 2009–2010 was applied to create a manageable list from which to make a final selection. A risk and intelligence based approach was taken to further identify corporations based on size, sector, and geographical location to result in a mix that would enable sufficient testing and identification of possible issues.

Table 1: Description of audits undertaken and outcomes

type of audit sector Corporate/facility type of auditor* Conclusion

reasonable resource large assurance** qualified

reasonable resource large assurance clean

reasonable transport medium environmental adverse

reasonable oil and gas medium environmental adverse

reasonable energy medium accountancy clean

reasonable energy medium assurance clean

reasonable—part manufacturing small assurance qualified

reasonable waste large assurance qualified

reasonable construction small accountancy qualified

reasonable—part waste n/a engineering clean

limited energy medium engineering clean

limited oil and gas medium assurance adverse

limited manufacturing medium engineering clean

limited service large assurance clean

limited energy medium engineering clean

limited resource small assurance clean

verification manufacturing small engineering n/a

verification fugitive emissions large environmental n/a

verification fugitive emissions large engineering n/a

verification fugitive emissions large engineering n/a

verification manufacture medium engineering n/a

* 4 Assurance; 5 Engineering; 2 Accountancy; 2 Environmental** International accountancy and professional services


These results are in line with expectations, given the audits were conducted post-submission of the reports, rather than during the reporting period. It should be noted that there is an increased likelihood of qualified outcomes if the audit engagement is undertaken retrospectively on a finalised report. The benefit of presubmission audits, such as those undertaken on financial statements, is that auditors can observe some data collection activities, and identify manual errors, or omissions, which can then be addressed prior to finalisation of the report.

The results of the audits conducted provided a number of unqualified or ‘clean’ conclusions. An unqualified audit opinion would have been given when the auditor considered that, in all material respects, the report gave a true and fair view of the greenhouse and energy activity, and had been prepared in accordance with the NGER legislation.

The results of the audits also included four adverse and four qualified conclusions. The qualified conclusions were the result of the auditor identifying misstatements or errors in one, or more, parts of the report being audited. The reasons provided for qualified conclusions including: non-reporting of energy consumed without combustion; non-reporting of energy produced for a range of minor fuels types which resulted in material under-reporting; inability to test control procedures and completeness of subcontractor information. Both the adverse and qualified conclusions are discussed further in Part 4: Application of materiality and modified opinions.

The pilot audit program identified that a number of the audited reporting corporations had sought assistance from external consultants to develop reporting systems, undertaken voluntary audits of previous reports, or included NGER reporting in internal audit reviews. The incidence of this type of activity in the pilot program sample would suggest that it is likely that this behaviour is undertaken more broadly in the population of reporting corporations.

Auditors advised that many clients will undertake some type of activity to address risk associated with providing a complete and accurate NGER report. However the corporations would use not only the external assurance tool but also internal audit tools such as document review, or specific verification. The decision on the activity undertaken was based on an assessment of risk compared to the cost of activity, that is, whatever was cost-effective for the corporation to reduce risk to an acceptable level.

The reporting corporation is considering the frequency of auditing activity and the ‘type of assurance as required for comfort that the reporting process is on track’.

No formal audit procedure, however random checks are carried out on items that contribute to the NGER report, source data, calculations, formulas and factors selected. Entry into online reporting tool is checked against calculation spreadsheets before sign off by the CEO.

The results of the pilot program highlighted that there are differences between corporations approach to risk management based on size:

• Large corporations have longer experience with this type of reporting as they may have been previously involved in assessments undertaken under the Emissions Intense Trade Exposed (EITE) scheme, or state-run programs such as GGAS or VEET Scheme (Victorian Energy Efficiency Target). This is reflected in better reporting systems and documentation.

• Organisations with relatively small workforces tend to bring in external support to meet their NGER obligations, having neither the expertise nor capacity to undertake inhouse compilation. Commonly, external support includes assistance with development of the reporting process, and advice on systems and controls.


Key factors identified during the pilot audit program that contribute to a positive audit outcome include:

• Where the NGER reporting obligation is given a high-profile in the governance structure of the reporting corporation. For example, key person responsible on Board, or reporting directly to the Board—the segregation of duties act as a check and balance on the reporting process.

• Where the responsibility for NGER reporting process is shared, with clear roles and responsibilities identified. This was shown to be effective on complex multi-site corporations where each site manager was engaged.

• For smaller, less complex reporting corporations, the approach of having a single person responsible for the NGER reporting process, along with other duties, was cost-effective and efficient. However this approach carries a greater risk of error and loss of corporate knowledge, as well as a significant ‘key person’ risk, that is, when an employee leaves the organisation.

Evidence of reporting entities processes and activities, provided by both the auditors and the corporations involved in the pilot program, demonstrates that there currently exists a well developed understanding of the reporting obligations, and a developing capacity to compile and submit essentially compliant NGER reports, with data that is substantially accurate and complete.

2.1 Assessment of fitness for purpose and effectiveness of delivery of the audit typesThe pilot audit program undertook assurance engagements across a broad range of sectors. The outcomes show that the audit approach was deliverable, and was able to identify misstatement and compliance issues with regard to the matter being audited. Both reasonable and limited assurance engagements delivered qualified conclusions.

The pilot program also trialled an engagement approach where reasonable assurance was undertaken only over a specific part of the submitted NGER report, for example: emissions from an individual landfill site; and an individual saw-milling facility. This approach to assurance engagements proved highly cost effective as attention was focused over the identified area of potential risk.

Verification audit engagements were considered to be best suited to audit activities which are based on pre-identified issues, and had the potential to provide a cost-effective approach to targeted activities for example specific to the application of a particular methodology.

In the pilot program verification engagements were used to address specific technical issues identified by the inhouse NGER report validation process, and to assess the use of fugitive emission methodologies for coal-mining activities.

Despite the challenges in engaging an auditor3 the verification engagements proved to be very successful—delivering useful evidence at a relatively low cost. This outcome demonstrates that verification would be a useful tool to the Regulator for both improving data quality, and identification of improvements to measurement methodologies.

2.2 Consistency of approach, particularly between auditors from different backgrounds and impact on final audit productA key question to be addressed by the pilot audit program was whether the approach of auditors from different backgrounds would be different in undertaking the engagement, and whether any difference in approach would be significant in terms of the outcome delivered.

3 The legal and contractual arrangements preclude exchange of greenhouse and energy information prior to engagement of auditor—this is

discussed further in the NGER Pilot Program Procurement Evaluation.


The pilot audit program demonstrated that there is a high level of consistency in the approach of auditors in undertaking the audit in terms of engagement planning, risk assessment, determination of testing requirements, execution of procedures, and identifying discrepancies and misstatements. This is consistent with the expectation that the auditors would follow the process described in the NGER Audit Determination and the Audit Handbook.

The key identifiable difference arising from different backgrounds would be around the approach to the conclusion and the qualification of the results of the audit. In broad terms the assurance auditors and accountancy backgrounds were very consistent, with the engineering background showing a stronger tendency to the ISO ‘verification’ style, particularly in reporting findings. However, over the majority of the reports, these were not substantially different.

The environmental auditors, based on only one auditor, indicate the strongest difference in approach to identifying and presenting the conclusions of the audit engagement. The approach of this auditor focussed primarily on a systematic assessment of ‘strict compliance’ with applying the legislation in preparing the NGER report. Although the impact on the submitted data was considered, the outcome was delivered from the perspective of compliance rather than the accuracy or completeness of data provided in the NGER report (see Part 4.2).

For audits designed to assess risks to achieving compliance or data integrity, the style and approach from the non-environmental auditors was more informative and useful as it provided a risk hierarchy to the matters reported in terms of potential significance to the Regulator.

This finding illustrates the different strengths offered by the different auditor types—the assurance and accountancy practitioners potentially bring a relative strength in experience of analysing complex corporation structures under corporations law, which is critical to the ‘who is responsible’ element of an NGER report. Auditors with engineering and environmental backgrounds may have relative strengths in the interrogation of the technical elements and measurement methodologies which is critical to the data accuracy element of the NGER report.

It is professional practice, consistent with relevant standards, for auditors to engage whatever experts may be required to complete the skill set for the audit engagement. This practice was demonstrated during the pilot program with audit service providers engaging external technical experts to participate in the audit team, that is, an accountancy firm engaged a registered NGER Category 1—technical auditor to assist with their audit and another accountancy firm engaged an external engineer to provide advice. It is a positive outcome for the pilot program to be able to observe this practice applied to greenhouse and energy audits. The design of the auditor registration categories envisaged this as a delivery model for the audit services for NGER and the pilot has demonstrated that it can be used effectively.

2.3 recommendations: audit program—promoting effective outcomesRecommendation 1

The current audit methodology and process is a reasonable way to continue in delivering a full audit program.

Recommendation 2

Develop communications for stakeholders to provide advice on factors contributing to a compliant NGER report and a successful audit outcome.


3 audit proCess and reportingThe requirements for undertaking an NGER audit and reporting the outcomes of the audit activity are provided for in the NGER Regulations and the Audit Determination. Additional guidance is provided in the NGER Audit Handbook.

The format of the formal audit report, which details the auditor’s conclusions, is prescribed in the Audit Determination. The content and wording of Part A, which presents the auditor’s opinion, is prescribed by the legislation and contains the conclusions, limitations and summary of work performed. Part B requires a listing of findings and conclusions—with a template provided.

Section 3.23 of the NGER Audit Determination requires that auditors outline in Part B of the audit report the following:

a. details of the items or issues related to the matter audited that required particular attention during the assurance engagement

b. details of aspects of the matter audited that particularly impacted on the carrying out of the assurance engagement

c. details of any matter, related to the matter being audited, that the audit team leader has found during the carrying out of the assurance engagement that he or she believes amount to a contravention of the Greenhouse and Energy Reporting Act 2007 of the Regulations, and

d. any other matter, related to the matter audited, that the audit team leader believes should be mentioned in the assurance engagement report.

The major international accountancy and professional services firms have globally consistent approaches inhouse, including documentation, which provide quality assurance and protect confidentiality. In addition audit practitioners are required to apply auditing standard ASQC 1 Quality Control for Firms that Perform Audits and Reviews of Financial Reports, Other Financial Information, and Other Assurance Engagements.

The planning phase of the engagement includes detail of the assurance process including describing key sources or facilities, and risk profiling. The risks addressed cover both inherent risks plus control risks identified specific to the particular engagement, for example a complex organisational structure, or the use of manual spreadsheets. Each risk has an assurance procedure described which is designed to provide sufficient appropriate evidence for the auditor to assess the resulting combined risk.

The engagement plans submitted during the pilot audit program generally contained a significant volume of useful information regarding the audited body, its approach to NGER reporting and the resulting risk profile. The information in the engagement plan is of high value to the Regulator.

The use of Part A in the audit report was consistent across all the audits in the pilot program. The one minor exception was the section on the Peer Reviewer which in some cases was provided in Part A, although the Audit Determination specifies that the information is provided in Part B; the name of the peer reviewer for the assurance engagement and details of the outcome of the evaluation undertaken by the peer reviewer in accordance with paragraph 3.7 (2) (e). In any case the information was provided and most likely provided in a manner consistent with the audit firms international practice guidelines.

Although there were consistent elements in Part B reports (issue, risk, testing, findings, conclusions, and recommendations) this element of the audit reporting process showed the most variety in both the type and extent of information provided, the variation appearing to reflect the different auditing backgrounds of the audit providers.


Table 2: Summary of the content of Part B of the formal audit report

type of auditor Brief description of approach differences

international accountancy and professional services

Part B Detailed Findings:

• includes in response to 3.23 d) the recommendations to management of the audited body

• ‘recommendations and suggestions for improvement’

• the ‘documents reviewed’ list is not included.

>> information in the engagement plan

accounting firms Content consistent but presented in own format

• source, approach, procedures

• findings and recommendations.

Provided risk ratings, including for any recommendations.

significant detail—reserved own format

engineering services Audit findings and conclusions which includes all procedures and conclusions [even where nil findings].

Supporting evidence: includes list of documentation reviewed.

all activities reported >> detail

environmental services [Same as above]

+ Audit findings and conclusions which includes all procedures and conclusions [even where nil findings]

Supporting evidence: includes list of documentation reviewed.

level of significance or importance is not well differentiated

In many cases detailed recommendations and advice to the reporting corporation was provided in Part B. This information is invaluable to the Regulator as it contributes to assessing risk for individual reporting entities and developing risk profiles for different sectors and activities. In addition more generic advice could be of value to other stakeholders.

... These findings, conclusions and recommendations are designed to inform the corporation and the GEDO of any compliance issues and will be used, in part, to better inform regulatory decisions and broader advice to the regulated community.

3.1 Information provided in the formal audit reportDuring the course of the audit engagement the auditor expends considerable effort in gathering information and making judgements about the entity and its report, for example information about the organisational structure of the reporting entity, the identified systems and controls, and the key reporting risks and how they are being addressed e.g. in the recommendations provided to the reporting entity management by the auditor performing the audit.


There is value to the Regulator in having access to this information, and in having access to the auditor’s insight into the reporting entity to assist the Regulator in assessing the performance of the entity, and any potential risks to data integrity or compliance.

However, in practice much of this information is not included in the formal report, although some may be provided to the Regulator through the engagement plan. The challenge is to ensure that the content of the auditor’s report is as useful and informative as it could be.

There would be benefit in working with auditors and assurance practitioners to consider options to enhance the content of the formal audit report and develop a consistent approach to reporting content for Part A and Part B of the formal audit report.

3.2 Three-way process—confidentiality and liabilityTo provide independent assurance or verification, a three party relationship must exist between the Regulator, the audited reporting corporation, and the audit team leader.

The three-way relationship generally proved easy to manage and was effective in delivering audit outcomes. However, some areas of confusion were apparent, where questions were raised by auditors and corporations regarding the process: for example whether the audited body should be provided with a draft of the audit report, and the role of the Regulator in considering the draft or the report finalisation process.

Based on experience in the pilot program and to ensure consistency and transparency in approach, there would be value in developing some additional material to be provided to the audited body which would clarify the steps in the audit process and roles and responsibilities of the auditor and the Regulator. In addition, for each specific audit engagement the audited body could be informed as to what the auditor has been engaged to deliver, the timings and the expected engagement process steps.

A concern was raised by auditors regarding the ‘declaration letter’ that is not specifically referred to in the Audit Determination. The ‘declaration letter’ is the step in the process where the audited body confirms that to the best of its knowledge, the audited body has provided complete and accurate information to the auditor. That is:

... making relevant records available ... for the purpose of ... engagement. To the best of xx’s knowledge, all records and related information which might affect the completeness or accuracy of the information has been made available ...

As the Audit determination is silent on the declaration letter there was also some confusion as to the intended purpose of the management representation letter template provided in the Audit Handbook was.4

The pilot audit program has highlighted the importance of the ‘declaration letter’ as a part of professional audit practice, and consideration could be given as to whether this step in audit practice should be included in the formal process for an auditor undertaking an engagement on behalf of the Regulator.

3.3 recommendations: auditor engagement and reportingRecommendation 3

Ensure that the content of the auditor’s report is as useful and informative to the Regulator as possible by working with auditors and assurance practitioners on options to enhance the content of the formal audit report and developing consistent reporting content for Part A and B of the formal audit report.

4 The management representation letter is used where sufficient evidence cannot reasonably be expected to exist, and the auditor is seeking a

written representation from the audited body stating that the matter is reliable, accurate and complete.


4 appliCation of materiality and modified opinionsThe pilot audit program demonstrated a high degree of consistency in approach in undertaking the audit engagements in terms of engagement planning, risk assessment, determination of testing requirements, execution of procedures, and identifying discrepancies and misstatements. Generally the audit engagements were performed in a manner consistent with professional auditing and assurance standards and NGER audit requirements.

The evaluation process identified two areas for further considerations: the application of materiality in audit planning and reporting; and the qualification of audit opinion in cases where the auditor was unable to provide an unqualified opinion.

4.1 application of materialityAudit practice standards require the auditor to consider materiality throughout an audit to manage the risk of reaching and reporting an incorrect audit opinion. A misstatement or discrepancy is considered material when, in the context of the surrounding circumstances, it is probable that the decision of a user relying on the assertion would be changed or influenced by such misstatement or discrepancy.

Materiality depends on the size and importance of the item judged in the particular circumstances of its inclusion, omission or misstatement. Assurance providers determine materiality based on their own judgement of the needs of intended users of the information.

Materiality is used by auditors in the planning phase of an audit and is revised during the course of the assurance engagement, as required, to ensure that the extent and type of procedures conducted reflects the risk of material misstatement in the reported information, or risk of non-compliance.

Information is material if its omission or misstatement could influence the economic decisions of users of the NGER report. Materiality depends on the size and nature of the item or error judged in the particular circumstances of its emission or misstatement.

Materiality means that, in the context of an audit, if information is misstated or non-compliant, it has the potential to (adversely) affect the decision of users of the greenhouse and energy information.

In the pilot audit program the application of materiality was determined by the assurance providers engaged for each individual audit. The Regulator imposed no requirements with respect to the application of materiality.

The following provides a quick summary of the ‘materiality’ approach applied to the pilot program engagements, noting that some audit reports did not provide information on the materiality approach taken in the audit:

• ‘The 5 per cent quantitative materiality level has been decided as appropriate as it is consistent with assurance standards on materiality and on guidance provided by the DCCEE [Department of Climate Change and Energy Efficiency] for assurance provided for applications for emissions intensive trade exposed permits, which although not the same as NGER audits is considered indicative of the DCCEE’s view of what would constitute a material error.’

• ‘Three facilities contribute 95 per cent of xx emissions. The assurance will concentrate on the operations of these facilities.’

• ‘An overall materiality threshold of 2 per cent was established for this limited assurance audit.

• Material errors were summed and the total error was less than 2 per cent.

• As reporting under corporate threshold then planning materiality is calculated at the corporate group level. Applied to all facilities that form part of the corporate group, except those that individually meet the facility level thresholds. Corporate group materiality was set at 5 per cent.


• Quantitative materiality threshold of 5 per cent.

• Global GHG Audit Methodology requires us to apply a materiality threshold of 5 per cent with our testing threshold (tolerable error) set at 2 per cent.

• Quantitative materiality threshold of 10 per cent (limited assurance).

• Reasonable or limited assurance between 5 to 10 per cent; started with 2 per cent at the corporate level (not facility).

Note that in many cases the materiality was considered separately at both the corporate and facility level. A number of practitioners advised that the materiality used to develop an appropriate sampling regime was set at 2 per cent to address quality assurance risk for the audit company.

An auditor uses professional judgement in determining appropriate percentages—for example a lower percentage may be used for larger organisational units where the materiality would otherwise be very high absolute amount. For example, at facility level or where 5 per cent is considerable in terms of the absolute amount, that is, 40,000 which is greater than the facility threshold under NGER.

Planning materiality may be adjusted using the professional judgement of the auditor. The presence of one or more of the following factors may indicate that a lower materiality threshold may be appropriate: proximity of reported numbers to thresholds, internal control deficiencies, complexity of measurement.

A specific difficulty arose where the NGER report value was zero: reporting of zero energy production means that we cannot apply materiality thresholds to sources and sites in selecting our sample.

To ensure consistency in approach some greenhouse gas emission programs assist the decision-making process by including auditing materiality thresholds.5 These can be defined at overall level and/or varying thresholds depending on the level of disaggregation at organisation, facility or source level. Whilst a materiality threshold is useful for making a preliminary assumption about whether an item is likely to be material, all relevant considerations should be taken into account. As a result, discrepancies of smaller amounts can have a material effect on greenhouse gas assertions—for example, if a 1 to 2 per cent error were to prevent an entity from reaching a facility threshold then, this would most likely be considered material. In the audit process, as there is an increased risk to any possible understatement, the auditor would undertake additional testing around that activity to address the risk.

On the basis of the results from the pilot audit, further guidance would be helpful for greenhouse and energy auditors on the application of materiality for NGER audit thresholds. Existing standards could be used as a starting point (ASA 320 and ASA 450). It may even be appropriate to consider developing examples of appropriate benchmarks and percentages which may be applied in determining materiality for the NGER report as a whole and at the facility level. Noting that the use of materiality in any particular audit would remain the responsibility of the auditor.

4.2 application of category of opinion in auditor’s conclusionThe objective of the greenhouse and energy audit is to enable the auditor to express an opinion on whether the reported assertions fairly represent greenhouse and energy activity, and whether they have been prepared in accordance with the NGER legislated framework.

The Audit Determination (Section 3.17) provides for the following qualifications in an assurance conclusion:

1. a qualified reasonable or limited assurance conclusion—same as AUSASB Standards

2. an adverse conclusion—same as AUASB standards

5 An auditing materiality threshold is not the same as reporting materiality threshold, that is, a predetermined quantity of emissions that an entity chooses to omit.


3. a conclusion that he or she (the auditor) is unable to form an opinion about the matter being audited—a disclaimer of opinion or conclusion in AUASB standards.

The circumstances under which the Audit Determination qualifications are appropriate are described in the Auditing Standard ASA 705 Modification to the opinion in the independent auditor’s report. Under this standard the auditor is required to express an adverse opinion when the auditor, having obtained sufficient appropriate evidence, concludes that misstatements are both material and pervasive.

Further, the auditor is required to express a qualified opinion when the auditor:

• having obtained sufficient appropriate evidence concludes that misstatements are material, but not pervasive, or

• is unable to obtain sufficient appropriate evidence on which to base the opinion, but that the auditor concludes that the possible effects on the report of undetected misstatements, if any, could be material but not pervasive.

A disclaimer is required when the auditor is unable to obtain sufficient appropriate evidence on which to base the opinion, and the auditor concludes that the possible effects could be both material and pervasive.

The approach to classifying qualified assurance conclusion is described in the NGER Audit Determination such that there:

• are one or more misstatements in the matter being audited that are material but not pervasive enough to affect the matter being audited as a whole, or

• is insufficient evidence in relation to one or more aspects of the matter being audited, but although the lack of evidence may be material it is not pervasive enough to affect the matter being audited as a whole.

As described above, the auditor is considering whether assertions represent greenhouse and energy activity, and that they have been prepared in accordance with the legislation. It appears that the majority of auditors identified three criteria in regard to the matter being audited, the criteria being based on the NGER data requirements: and GHG emissions, energy production, energy consumption: that is, if not misstated then compliant. One auditor considered the matter in terms of compliance with the NGER legislation (see discussion below).

There were four adverse conclusions in the pilot audit program. Two of these were the outcome of limited assurance engagements on reports submitted by reporting corporations from the oil and gas sector. The basis of the adverse conclusions in these cases was supported by evidence of material misstatement occurring across more than one of the identified criteria, for example, non-reporting of energy production due to a transposition error; lack of documentation relating to emissions from venting; and the non-reporting of both emissions and consumption from fuel gas use at facility.

There were two illustrative examples of auditor conclusions that show the application of the auditor opinion in the NGER context:

a. A qualified conclusion on a reasonable assurance audit of a report submitted by a reporting corporation from the coal mining sector: The auditors reported a qualified conclusion, having identified an instance of under-reporting of greenhouse gas emission equivalent to ~75 per cent of emissions actually reported. The auditors found evidence of a significant, but precisely quantifiable, calculation error in a complicated methodology related to activity from the only underground mine operated by the company. As the corporation had only one activity of this type the auditors concluded that the misstatement there did not present a significant risk for the business as a whole. This conclusion being supported by evidence that across all other elements of the business that were examined, the NGER data was accurate and complete.


b. Two adverse findings by environmental auditor on reasonable assurance audits of corporations from different sectors: For one report the findings were adverse based on how the reporting corporation had applied the legislation. Even though the report acknowledged that there were no significant misstatements (most not material or borderline), in the auditor’s view the number of instances of minor non-compliance met the requirement of pervasive. The instances of non-compliance were: facility definition, omission of electricity from offices, and omission of vehicles supplied by contractor.

In the second report, where the auditor identified issues it was acknowledged that there was ‘no evidence’ or ‘insufficient evidence’, or that the findings only had the ‘potential to cause a material misstatement’.

The auditor, in applying the NGER audit regulations, and considering the lack of availability of evidence, or actual material misstatement, should have reached either a qualified opinion, or a disclaimer (depending on the auditor’s assessment of the likelihood that the unknown matter is both pervasive and material). The adverse conclusions reported by the auditor appears to reflect consideration of ‘strict liability’ in terms of compliance, that is, where the facilities aggregates were incorrectly structured with respect to a small number of minor facilities but all the data was reported, this was still regarded primarily as ‘non-compliance’. This approach coupled with a number of minor misstatements where there is no evidence as to whether they could be considered material, appears to have led the auditor to reach the adverse conclusion.

There were a number of instances in the pilot audit program where reports were qualified on the basis of data entry error, calculation error, or transposition error. This type of error is uncommon when the audit is undertaken presubmission and in part explains the relatively high incidence of qualifications in the pilot audit program. When the audit of the reported matter occurs prior to submission then the reporting entity can address errors or omissions prior to submission of the report (or audited matter).

In most cases of financial auditing practice the client is the audited body. While the audited body pays for the audit, the financial Regulator is one of the primary users of the audit conclusion, and provides guidance to the auditor, through requiring adherence to the accounting standards.

In the case of NGER audits, the direct client is the Regulator so it would be appropriate and important for the Regulator to provide guidance around the issues relating to adjustments of reported data, when data errors have been identified through the audit process.

Although the NGER Audit Determination is consistent with existing standards, and the Audit Handbook contains some further explanation of qualified conclusions, nevertheless, neither the Determination nor the Handbook provide the extent of requirements or guidance in relation to qualifications which is available in ASA 705 Modification to the Opinion in the Independent Auditor’s Report.

Note: the AUASB also allow for an emphasis of matter paragraph (see ASA 706) which highlights a matter to users of the report but does not affect the auditor’s conclusion. The Audit Determination does not allow for this circumstance. Neither does it allow for a ‘limitation of scope’ qualification, where there is not sufficient evidence on a proportion of the report but the matter is not pervasive enough to not provide a conclusion on the other part.

4.2.1 ConclusionsConsideration of the functionality of the current NGER audit conclusions would be useful to determine if further guidance is required for auditors in order to achieve consistency in the provision of qualified conclusions. ASA 705 could be used as a starting point for developing such guidance. In terms of correction of identified data errors there would be value to Regulator making adjustments to ensure accurate, complete data in the NGER data set. The Regulator could consider the circumstances where resubmission of data (reassertion) post-submission of the NGER report could be provided for in the greenhouse and energy audit process.


4.3 interpretation of legislationThe pilot audit program showed that the auditors demonstrated a high level of expertise and understanding in the application of the NGER legislation and the reporting requirements, including of relatively complex areas relating to facility definition and operational control.

There were only a few instances where discrepancies in interpretation occurred, for example, application of NGER to external territories, and application of the consistency principle. One item that was raised in three different audits was the reporting of minor sources of energy production. In this case this issue had been previously identified by the Regulator, and in the case of a threshold for reporting energy consumption, is being resolved by considering amendments to the legislation.

In the event, legal interpretation did not raise any significant concern but did serve to highlight a weakness in the audit engagement process from a contract management perspective, particularly where auditors were contacting the Regulator through the call centre to seek advice on legislative interpretation.

In terms of ensuring consistent advice and treatment these queries would be best managed through the audit contract manager whereby the contract manager would forward query and/or liaise with relevant areas, including policy or legal teams, to develop appropriate response to auditors. This approach would also allow for better contract delivery, through monitoring of auditor’s capability and expertise, as well as being able to resolve valid issues in a timely way, in line with any agreed contractual deadlines.

4.4 Recommendations: Audit practice specific to greenhouse and energy auditsRecommendation 4

Develop guidance for NGER auditors on the consideration of materiality in greenhouse and energy audit engagement, and assessment of the potential use of appropriate benchmarks which auditors may consider in determining materiality for the NGER report.

Recommendation 5

Collaborate with the Auditing and Assurance Standards Board (AUASB) and professional organisations to determine if further guidance is required for auditors in order to achieve consistency in the provision of qualified conclusions.

Recommendation 6

Develop a formal process for resubmission of data, where errors are identified, as part of a greenhouse and energy audit.


5 reporting entities and the audit proCessUnder the legislation a registered corporation is required to keep adequate records of activities undertaken to comply with the provisions of the Greenhouse and Energy Reporting Act 2007, including the preparation of the NGER report. The NGER fact sheet: Record keeping, provided in July 2009 at the end of the first reporting year, advised that records of activities must provide adequate evidence to the Regulator of a corporation’s compliance with the legislation, including information that could be used to verify the relevance, completeness, consistency, transparency and accuracy of reported data during an external audit. Further, corporations were encouraged to record both the decision making process and the details of the calculation and data analysis methods used in preparing the report.

5.1 auditable recordsThe issue of documentation identification and retrieval was raised by auditors in the majority of audits. In most cases the reporting entities have adopted a ‘fit for purpose’ approach to documenting and describing the process and procedures around the NGER reporting obligations. Generally reporting corporations appear to be looking primarily at internal requirements that are cost-effective, particularly in smaller organisations. In many cases the actions taken had not been optimised to accommodate external scrutiny.

In one case, the reporting entity had established a comprehensive system for collecting and collating data from its activities in the form of a purpose built database. The auditors reported that all requested supporting information was readily retrievable and available.

Many reporting entities rely on manual spreadsheet systems as the process to capture and calculate the group’s GHG emissions, and energy produced and consumed, for the purposes of NGER reporting. Auditors frequently identified this as increasing the risk of material error through transposition error and miscalculation, although acknowledging that quality assurance procedures could be used to reduce this risk.

...whilst we did not identify any errors in relation to the current manual reporting process, we recommend xx continue to investigate acquiring and implementing an automated database data collecting, aggregations, and conversion system to increase the efficiency of the reporting process and reduce the likelihood of human error.

Examples from the audit program include where the auditor reviewed the systems and procedures for the collection and collation of data, and a clear audit trail of documents was available to support the data and calculations used for NGER reporting but the process followed was not documented.

An NGER Submission Plan 2009–2010 have been developed by XX which describes the key emissions sources and entities within XX where the major emission activities occurred in the 2009–2010 financial year. However, there is no written procedure for the NGER calculation and reporting tasks.

Many auditors included in their report on the audit recommendations to the reporting entity to consider strengthening the governance processes and controls in place by:

• Ensuring relevant and accurate records are maintained as per the requirements of the regulations, including maintenance and calibration records; maintaining evidence to support any assumptions employed.

• It should be considered to document the quality controls for assuring that the NGER information provided through OSCAR is accurate. Also, those procedures should specify the document retention and record-keeping processes.

• Document a clear and detailed procedure for reporting processes, methods, roles and responsibilities.


The importance of security of records was also highlighted in the pilot audit program. One reporter’s end-of-month reconciliation worksheets had been saved over with the following month. In this case the source data was able to be retrieved and reproduced. The auditor advised that although the missing July 2009 reconciliation sheet is not significant in and of itself...it does point to a weakness of the record-keeping system. The auditor provided advice to the reporter of the need to ensure that data is securely stored and cannot be overwritten.

Despite legislative requirement and previous advice from the Regulator concerning the purpose of record keeping it would appear from the outcomes of the pilot audit program that there are still some improvements to be achieved in the provision of adequate, auditable records of compliance. There could be considerable benefit to Regulator, auditors and reporting entities to developing an approach to record keeping that is both efficient, and contributes effectively to providing an auditable trail of evidence.

The approach used in the UK Government’s Carbon Reduction Commitment (Energy Efficiency Scheme) is to require participants to keep an ‘evidence pack’ of records that show how the entity has complied with the scheme and enables the entity to be audited. The records are required to be up-to-date, accurate and easy to understand, and the evidence pack must be produced in the case of an audit.

The evidence pack must include, as a minimum:

• structural records—with clear definitions for the structure of the organisation

• data records—to show the annual organisation-wide energy consumption (specified), and

• special event records to identify unusual activity such as prolonged shutdowns and acquisitions.

Clear guidance on record-keeping requirements to collate and keep readily accessible all the relevant information, records and data could be developed using an ‘evidence pack’ approach. This approach would be of significant advantage to the NGER reporting entities by ensuring there is an expeditiously retrievable, auditable trail of evidence. The benefit in the audit context is that the auditor is able to complete the audit engagement more efficiently, in both time and cost, with a reduced risk of an incorrect opinion.

In terms of the reporting entity the evidence pack would be of benefit for:

• Internal use within reporting corporation: transparent decision making process, clear roles and responsibilities, and value to internal audit or governance processes. In the event that there is a change in personnel or structure it enables the easy transfer of the reporting task.

• External scrutiny: appropriately documented processes and procedures provides a ‘road-map’ for the auditor to follow in understanding the corporation’s business and reporting obligations as well as documenting assumptions made and business decisions, that is, reporting responsibilities for joint ventures and contractors, and selection of measurement methodologies.

5.2 presubmission auditIn professional audit practice, audit engagements often commence during the reporting period relevant to the matter being audited, and an opinion is reached on final reporting data immediately prior to reporting. These are sometimes referred to as ‘interim audit work’ and ‘final audit work’. Advice from AUASB is that there are a number of reasons why it is preferable for the greenhouse and energy auditor to be appointed prior to the commencement of the reporting period, or not long after commencement of the auditing period, for example:

• confirmation that the data is actually auditable, that is, that documents, recording systems and controls are in place

• testing—this would vary between sectors, however access to conduct procedures, for example, to test the accuracy or calibration of meters, may be necessary if the measurement cannot be tested subsequently, and


• logistics—in order to complete the work, and report within the required deadline, auditors may need to conduct a proportion of their work during the reporting period.

In the pilot audit program there were a number of qualified conclusions. Those relating to unintentional manual errors or omissions have already been discussed in Part 6. However, other qualifications occurred which were the result of other post-submission issues such the ability to observe control activities.

One of the audits under the pilot audit program has a ‘qualification of scope’ as the auditors were not able to observe control measures in place, that is, on a construction site that no longer exists.

The evidence from the pilot audit program suggests that reporting entities are gaining experience with compiling evidence and reporting under NGER legislation, and with each reporting cycle information about the entities greenhouse and energy activities increases. These trends provide an opportunity for entities to reduce risks and costs associated with producing a compliant NGER report on an annual basis. For example, the data cycles provide and option to consider consistency checks, that is, monthly comparisons, and year-on-year comparisons. Independent assurance activities undertaken by internal audit or external auditors are assisted by these kinds of data comparisons.

The evidence from the pilot audit program shows that the presubmission engagement of auditors and the increasing knowledge base of reporters would put reporting entities in a strong position to be able to deliver on timely completion and submission of NGER reports, in those cases where a reporting entity is seeking assurance over its report.

5.3 Recommendations: Enabling audited bodies to get the most from the audit processRecommendation 7

Consider changes to the NGER requirements for record keeping, and the development of guidance in the form of an ‘evidence pack’ to ensure that record keeping contributes effectively to providing an expeditiously retrievable and auditable trail of evidence.

Recommendation 8

Collaborate with AUASB and professional organisations to develop guidance that would define circumstances where presubmission audits would be of particular benefit to reporting entities.

Recommendation 9

Promote the benefits to the reporting entity of providing an assurance report to the Regulator; or advising the Regulator that assurance has been undertaken on the submitted NGER report.


6 Common diffiCulties experienCed in preparing nger reportsThere were a number of areas identified in the pilot audit program of difficulties that reporters commonly face in preparing NGER reports. The majority of these issues relate to the completeness and quality of data provided in regard to minor sources.

Key issues identified from validation of 2008–09 reports: energy production not reported; incorrect units inadvertently used; contractors’ emission and energy data not provided; greases and oils reported as non-combusted instead of combusted (NGER fact sheet 15: Analysis of NGER reports).

The types of issues identified by the pilot audit program were similar to those identified by validation. The key additional issue relates to the reporting entities activities in establishing the responsibility to report.

A summary of key issues identified from the NGER pilot audit program is provided below:

• establishing responsibility to report—partnerships, joint ventures, shared metering

• access to information from contractors and subcontractors

» complete identification of sources

» minor facilities (percentage emissions)

» minor emissions sources—oils and greases, acetylene, LPG, SF6 in switchgear, car fleet, temporary generators

» minor fuels for combustion: ignition fuels, alternative fuels which supplement main fuel supply

» Energy consumption—office locations with single solar panels, regenerative dynamometers, temporary generators

• quality of data for minor sources

» oils and greases

» fuels from contractors

» on-site generation, that is, single solar panel.

6.1 responsibility for reportingIn order to determine its obligations under the NGER legislation, a controlling corporation must assess whether any of its group members have operational control over facilities that would meet any of these reporting thresholds. Also central to determining these obligations are the concepts of a controlling corporation’s group, members of that group and affected group members.

For example, the participants in a joint venture may nominate a ‘responsible entity’ to undertake reporting obligations for a captured activity. A nomination form is provided to the Regulator, however the nominee can self-nominate on the form and although information on all participants is provided, the Regulator is not provided with a record of agreement by the other partners and so the auditor is dependent on these records being held by the corporations involved. Comments from auditors in relation to evidence of decision-making for these types of considerations were a common theme across the pilot audits:

• Where it appeared that decisions had been made, that is, not overlooked, but no documentation was available.

• A change of control procedure should be developed to address cases where control over new operations is acquired, and control over operations is lost, to ensure that the underlying documentary evidence is available to the controlling corporation.


In one case the reporting corporation had triggered the corporate threshold based on its responsibility for a large number of small activities, with sites at a large number of locations over a wide geographical area. The auditor made recommendations to the corporation around documenting an approach to determining operational control compliant with Section 11 of NGER Act 2007.

However, there was evidence of robust practice where the auditor documented that the reporting entity had extensively examined the issue of operation control, including seeking legal advice. The reporting corporation was also able to provide executive board papers on the issue, a letter seeking confirmation of operational control, and documentation in the subsequent service agreement regarding responsibility for reporting NGER data.

Guidance on appropriate evidence base for operational control could be considered as part of the ‘evidence pack’ approach to record-keeping approach considered in Part 5.

6.2 Contractors and subcontractorsThe pilot audit program identified a number of obstacles to complete and accurate reporting of data from service providers:

• a business model that preferentially uses small, local contractors for on-site activities—these small service providers are often not resourced to provide the information and not required to report independent of corporation

• contractual arrangements that do not include the provision of particular information which were in place prior to the NGER reporting obligation

• legal interpretation of contractors as where a reporting entity engages a wide variety of services, delivery models with resultant >> of variations in contractual arrangements.

One example from the pilot audit program illustrates a typical approach to acquiring data from contractors. The reporting entity sought information through an annual survey to contractors over a threshold contract value. Volumes of fuels were estimated for contractors that did not respond to survey, or who were below the threshold contract value. The system also identified contractors that already report the data under NGER, so that data was not reported twice. The auditor concluded that the assumptions and methodology used to estimate fuel from contractors appeared reasonable, however they recommended that the company consider conducting an audit on the major fuel using subcontractors.

The findings in the pilot audit program have highlighted the challenges faced by reporting corporations who include contractors and subcontractors in their reporting obligations. These challenges include identifying reporting responsibilities for contractors and subcontractors, and ensuring timely access to data once an obligation has been established.

Advice has previously been provided by the Regulator to reporting corporations on establishing reporting responsibilities (NGER fact sheet 7: Contracts and Leasing). However, the advice does not cover data acquisition or how to manage risks associated with third-party data sources.

There is the potential to consider working with reporting corporations to develop guidance on strategies that would improve data retrieval from contractors and service providers—acknowledging that this element of the reporting requirements has the capacity to improve with time as new contracts are negotiated to include provision of information, or included when existing contracts are renewed.

A significant issue to consider is the provision of assurance on information provided by contractors and subcontractors. Further assistance could be provided by considering the requirements for financial auditing with regards to providing assurance on information obtained from service providers, and the existing standards and guidance for financial statements: ASA 402 Audit Considerations Relating to Entities Using Service Organisations; and ASAE 3402 Assurance Reports on Controls at a Service Organisation.


6.3 Complete identification of sourcesA large number of the pilot audit reports identified either omission or poor quality data with respect to minor emissions sources such as oils and greases, acetylene, LPG, SF6 in switchgear, car fleets, and temporary generators. There were also frequently reported omissions for minor energy consumption such as office locations and temporary generators. In most cases the reporting entity had considered the activity to be too insignificant to report, in spite of the inclusive NGER requirements. Alternatively, for unplanned temporary activities, such as temporary generators the NGER reporting procedures did not identify the activity.

Consideration could be given to whether there is a benefit to recommending minimum accounting thresholds that would state that a source not exceeding a certain size be omitted from the audit. However, although it appears a useful approach in theory, the practical implementation of such a threshold is not compatible with the completeness principle of the NGER legislation. In addition, in order to utilise the threshold the activities would have to be quantified to ensure that they were under the threshold. However, once emissions or activities are quantified, most of the benefit of having a threshold is lost.

Reporting corporations need to make a good faith effort to provide a complete, accurate and consistent data report. For cases where emissions or activities have not been estimated, or estimated at an insufficient level of quality, it is important that this is transparently documented and justified. In such cases the auditor could then effectively determine the potential impact of their exclusion, or lack of quality, on the overall report.

6.4 Recommendations: Opportunities to address common reporting challengesRecommendation 10

Working with reporting entities to develop guidance or strategies that would contribute to improved data submission from contractors and service providers, and options for providing assurance over this third-party data input.

Recommendation 11

Consideration of the challenges associated with providing data on minor sources in the greenhouse and energy audit process. In particular, to address cases where emissions or activities have not been estimated, or estimated at an insufficient level of quality, and develop guidance on how this should be transparently documented and justified.


7 ConClusionsOverall, the audit reports demonstrated a fair level of data integrity and significant voluntary compliance with reporting obligations. These results are in line with expectations, given the audits were conducted post-submission of the reports rather than during the reporting period, which increases the likelihood of qualified conclusions.

The outcomes of the audits themselves show that the audit approach was deliverable and able to identify misstatement and issues with regard to the NGER reports, and be able to provide advice and recommendations to the audited body on potential improvements to the reporting process.

The approach to taking the recommendations forward will include direct engagement with stakeholders to identify practical and effective solutions, and working with NGER community to communicate the outcomes to all audiences, both reporting corporations and auditors, and others who use the NGER scheme.

The current audit methodology and process is a reasonable way to continue in delivering a full audit program. Changes and improvements could be developed concurrently with a full audit program, with any changes to apply in the next reporting period. This approach would align with timing for a new international standard, currently under consideration, and the introduction of the Clean Energy Future.

2001

4

Documents

Pilot Audit Program Evaluation - Clean Energy Regulator Clean Energy … · 2015-04-15 · Pilot Audit Program Evaluation . ... 1. introduCtion 7 1.1 Background 8 2. ... as part of