IAEA International Atomic Energy Agency THE IAEA INTEGRATED REGULATORY REVIEW SERVICE (IRRS) PROCESS Ivan LUX Regulatory Activities Section | Division

IAEAInternational Atomic Energy Agency

THE IAEA INTEGRATED REGULATORY REVIEW SERVICE (IRRS) PROCESS

Ivan LUX Regulatory Activities Section | Division of Nuclear Installation Safety

Department of Nuclear Safety and Security

Presentation to the EPREV Lessons Learned Workshop

16.07.2014

IRRS Process_EPREV LL Ws.pptx

IAEA IRRS Mission overview 2

Outline

• On the IRRS missions in general

• Analysis of past missions

• Continuous improvement of the IRRS programme


ON THE IRSS MISSIONS IN GENERAL


IRRS as successor of other missions

• IRRS – Integrated Regulatory Review Service, superseds a number of former missions:• Integrated Regulatory Review Team (IRRT)• Radiation Safety, and Security of Radioactive

Sources, Infrastructure Appraisal (RaSSIA)• Part of Emergency Preparedness Review (EPREV)

Core areaAdditional area(also in separate

missions)


IRRS Missions

• Integrated Regulatory Review Service to peer review the host country’s nuclear and radiation safety regulatory regime against the IAEA Safety Standards

• Modular review by international experts (15-20)• Core regulatory activities Modules + additional areas• Mission report with Recommendations, Suggestions,

Good Practices (observations)

IRRS Modules


What is an IRRS mission for?

• IRRS is meant to provide• an opportunity for continuous improvement for the

host nuclear and radiation safety regulatory body through self-assessment and through peer review by an international expert team

• a review of the technical and policy issues relevant for the host country

• an opportunity to share and exchange experience and discuss issues with international experts

• a contribution to the harmonization of regulatory approaches among Member States


What is not an IRRS mission for?

• The IRRS mission are not meant to• inspect regulatory or operator activity in the host

country• form any judgment on the host country regulatory

regime or practice• to provide comparison among Member States

Therefore the number of findings is in no way a measure of the status of the regulatory body. Comparisons of such numbers between IRRS reports from different countries should not be attempted.


IRRS Process

• Prep-meeting about 6 months before

• Self-assessment by host• First mission: 12 days• Report finalization about

3 months• Follow-up missions after

2-3 years, about 7 days


Mission outline

• Initial team meeting• Entrance meeting• Review of technical and policy

issues (interviews and meetings)• Site visits• Daily team meetings• Report preparation• Plenary discussion• Exit meeting


Missions in 2006-2014

Korea (f)

Zimbabwe

Niger UAE France

Mexico Germany Canada (f) Netherlands

Mauritius Ukraine Russia Switzerland UK (ff)

Cameroon Sierra Leone UK (part 2) Australia (f) Belgium Cameroon

Kenya Namibia Vietnam Slovenia Czech Rep. Vietnam (f)

Uganda Madagascar Lebanon Ukraine (f) Germany (f) Finland Russia (f) Slovenia (f)

France Gabon Botswana Canada USA Korea Greece UK (f) Jordan

UK Australia Spain Peru China Spain (f) Slovakia Poland Pakistan

Romania Japan Cote d'Ivoire France (f) Iran Romania Sweden Bulgaria USA (f)

2006 2007 2008 2009 2010 2011 2012 2013 2014

3 9 8 7 4 9 4 6 3 (11)


Missions in 2006-2014: Site Overview

Initial missionsFollow-up missions


IRRS Guidelines

• Published in May 2013

• A series of Targeted Consultancies was held for collecting experience and advice for a thorough revision

• Novel edition will be compiled based on the consultancies


Results of an IRRS mission

• Missions report, including observations (agreed with the host during the mission):• Recommendation: non-

compliance with IAEA Safety Standards

• Suggestion: a way of further improvement

• Good Practice: unique feature worth of recommending to others


IAEA Safety Standards used in IRRS


Basis for findings: GSR Part 1 Requirements

IRRS Modules

+ other IAEA Safety Standards


Participation in the missions (2006-2014)

• No. of experts (including IAEA): 362• No. of participations: 673• From 59 countries

010

2030

4050

ALB

ALG

ARG

AUS

AUT

BEL

BIH

BKF

BUL

BRA

CAN

CHI

CPR

CUB

CZR

DEN

EGY

ETH

FIN

FRA

GFR

GRE

HUN

ICE

IND

IRE

ISR

ITA

JOR

JPNLIT

MEX

MO

RN

ETNIR

NO

RN

ZEPA

KPO

RRO

KRO

MRU

SSA

FSL

OSL

RSP

ASU

DSY

RSW

ESW

ITA

NTU

NTU

RU

AEUK

UKR

USA

Expe

rts

in IR

RS m

issi

ons

Num

ber

of p

artic

ipati

ons

Num

ber

of e

xper

ts


ANALYSIS OF PAST IRRS MISSIONS - EXTRACTS


Scope of analysis

• Missions in 2006-2014 to MSs operating NPPs

• Modules and facilities related to nuclear installations

Niger UAEMexico Germany Canada (f)

Mauritius Ukraine Russia SwitzerlandCameroon Sierra Leone UK (part 2) Australia (f) Belgium

Kenya Namibia Vietnam Slovenia Czech Rep.Uganda Madagascar Lebanon Ukraine (f) Germany (f) Finland Russia (f)

France Gabon Botswana Canada USA Korea Greece UK (f)UK Australia Spain Peru China Spain (f) Slovakia Poland

Romania Japan Cote d'Ivoire France (f) Iran Romania Sweden Bulgaria USA (f)2006 2007 2008 2009 2010 2011 2012 2013 2014


Analysed topics

• Frequency of references from mission observations to IAEA safety standard requirements

• Distribution and correlation of observations (by types by missions, by IRRS modules)

• Contents of observations (most frequent subjects, coincidence of subjects in findings and GPs,…)

0

10

20

30

40

50

60

70

80

24 18 20 32 27 25 2 29 36 4 26 33 15 22 31 16 19 14 11 23 1 3 8 7 5 30 35 17 21 28 10 34 6 13 9 12

Num

ber o

f ref

eren

ces

GSR Part 1 Requirement number

All references to GSR Part 1 from all missions (ordered)

0

5

10

15

20

25

30

35

40

45

50

1.d 7.a 9.b 7.c 3.a 4.a 10.e 3.f 4.b 9.a

Num

ber o

f find

ings

per

mis

sion

s

Subject Group

Subject Groups with the most findings

0

10

20

30

40

50

60

70

A B C D E F G H I J K L M N O P Q R S T U V

Num

ber o

f ob

serv

ation

s

Missions

Observations in initial MissionsR S GP R+S Ʃ


Conclusions of analysis (examples)

• Conclusion: The mission reports analysed indicate that requirements which would be expected to form the basis of observations in particular IRRS Modules often appear in other Modules.

• Conclusion: Recommendations and Suggestions, referencing GSR Part 1 Requirements relating to core regulatory functions from initial missions, comprise a dominant part of all references.

• Conclusion: There is no need to set rules as for the desirable number of observations offered in missions, the practice so far followed ensures reasonable distributions for these numbers.


Conclusions of analysis (examples)

• Conclusion: The numbers of the Recommendations and those of the Good Practices are indeed negatively correlated in the missions analysed. Yet this seems to be less pronounced than expected, further analysis of selected missions may reveal the reasons for this.

• Conclusion: In the light of the limited data available it is likely that complying with findings addressed to the government needs more time, or other prerequisites than with those for the regulatory body.

• Conclusion: The Subject Group most frequently addressed by findings in the analysed missions is related to the legal framework for regulatory activities

• Conclusion: Activities related to inspection represent the most controversial part of the regulatory regime involving some fields with frequent non-compliance and others often commended by the missions.


CONTINUOUS IMPROVEMENT OF THE IRRS PROGRAMME


Ways to improve

• Basic IRRS Training• IRRS Mission Standards• Feedbacks• Measuring effectiveness and

efficiency• Prompt Evaluation Report


Basic IRRS Training Course

• First held in October 2013• 62 participants from 38 MSs• In the framework of the EC-IAEA

cooperation in the field of Nuclear Safety

• Covered all aspects of the IRRS programme, 16 Training Lectures, 3 Information Lectures

• Test Questionnaires for all Lectures• 2nd edition in October 2014• Refresher course (in missions)• E-learning modules


Feedback on the course

Evaluation of the BIT Course by participantsQuestion Mark

4.64

How do you rate the training course presentations from the point of view of

the information therein and their completeness 4.45

their clarity 4.30

their format 4.29

How do you rate the training course lecturers from the pointof view of

their competence in the field 4.69

the way they held the presentations 4.26

What is your opinion on the course-related organization and arrangements by IAEA? 4.69

Average: 4.47

Number of answers: 56

How useful you consider the training in your possible future IRRS activity?


Feedback from the Test Questionnaires

0.500.550.600.650.700.750.800.850.900.951.00

1 2 3 4 5 6 7 8 9 10 11

Solution ratios in Test Questions No.1

0.500.550.600.650.700.750.800.850.900.951.00

1 2 3 4 5 6 7 8 9

Solution ratios in Test Questions No.6


Ways to improve




Mission Standards – ARM Report Template

Under development / discussion


Mission Standards – Report Template

• Cross-contribution table• Instructions for reviewers

• Issues to cover• Format of observations


Mission Standards – Schedule


Ways to improve




Feedbacks on the IRRS process

• Questionnaire on the quality of the ARM – reviewers at mission start

• Questionnaire on effectiveness of the mission – reviewers at end of mission

• Round-table conclusions and suggestions – reviewers at end of mission

• Questionnaire on effectiveness of the mission – host after mission

• Questionnaire on effectiveness of the initial mission – host before follow-up mission

• Workshop for team leaders (January 2013)• Triennial LL Workshop (October 2014, Moscow)• Workshop for experienced reviewers (planned

for 2015)


Ways to improve




Effectiveness and efficiency of IRRS

• NSAP requires establishing criteria for efficiency of IRRS missions

• 16 Performance Indicators introduced to characterize the E&E of a mission, such as:• Team size; team experience;• Time available to prepare for the

mission;• Feedback from team members and from

hosts;• Coverage of the Action Plan by findings;• Balance of findings;• Conciseness of mission report; etc. Under development


Examples of Performance Indicators

3

3.5

4

4.5

5

Miss1 Miss2 Miss3 Miss4 Miss5 Miss6

mar

k

mission

Host Feedback

0.000.100.200.300.400.500.600.70

Fu-1 Fu-2 Fu-3 Fu-4 Fu-5 Fu-6 Fu-7

Ratio of open issues in follow-upsRecommendations Suggestions All

0.000.100.200.300.400.500.600.700.800.901.00

Fu-1 Fu-2 Fu-3 Fu-4 Ini-1 Ini-2 Ini-3 Ini-4 Ini-5 Ini-6 Ini-7 Ini-8 Ini-9

ratio

of m

embe

rs w

ith e

xper

ienc

e

missions

Team Experience

0306090

120150180210240270300330360390

FRA

UK

JPN

MEX

SPA

UKR GFR

FRA

-fu

CAN

UK-

fuR

US

CPR US

RO

MSP

A-f

uR

OK

GFR

-fu

SLO

SWI

CAN

-fu

SWE

SLR

FIN

BUL

UK-

fu2

RU

S-fu

CZR

BEL

com

pleti

on ti

me

in d

ays

Mission

Report Completion Time

Optimum Acceptable Need attention

0 0.2 0.4


E & E of past IRRS missions

Note: The Performance Indicator values presented here characterize the effectiveness and efficiency of the IRRS process and represent no indication whatsoever on the safety performance of the host countries of the missions

0.000

0.020

0.040

0.060

0.080

0.100

0.120

0.140

0.160

0.180

0.200

0.220

ROK GFR-fu SLO SWI CAN-fu SWE SLR FIN BUL UK-fu2 RUS-fu CZR BEL

Dist

ance

from

opti

mum

Mission

Efficiency and effectiveness of IRRS missions

to analyse

acceptable

effective

optimum


Ways to improve




Prompt evaluation report

• Basic mission data• Feedback from the team• Feedback from the hosts• Overall evaluation of effectiveness

and efficiency• Discussion, conclusions• Summary


Evaluation of a mission via PI’s


Pilot application in nuclear missions


IN SUMMARY


Conclusions

• IRRS is an effective means of reviewing compliance of RB status and activities with IAEA safety standards

• Regular invitation of IRRS missions by all IAEA MS is highly recommended

• The IRRS process is mature and effective – yet it is under continuous development by using novel standardized tools and feedback from participants

• Lessons learned from past missions as well as mission effectiveness and efficiency analysed

• Prompt evaluations of IRRS missions to nuclear countries were performed in a pilot phase


THANK YOU FOR YOUR ATTENTION

Ivan Lux, IAEA NSNI RAS, [email protected]

Documents

IAEA International Atomic Energy Agency THE IAEA INTEGRATED REGULATORY REVIEW SERVICE (IRRS) PROCESS Ivan LUX Regulatory Activities Section | Division