IAEA International Atomic Energy Agency
INTEGRATION OF SAFETY CULTURE INTO REGULATORY
PRACTICES AND THE REGULATORY DECISION MAKING
PROCESS
Technical Meeting
IAEA Guidance on Safety Culture Self-Assessment and
Independent Assessment Abida Khatoon
Regulatory Activities Section, Division of Nuclear Installation Safety
Monica Haage
Operational Safety Section, Division of Nuclear Installation Safety
6-8 October 2014,
Vienna, Austria
IAEA
Contents
• Training Material on Safety Culture Self-
Assessment
• Independent Assessment (IRRS/OSART)
• Summary
IAEA
Basis for Training Material on Safety Culture
Self-Assessment
Requirement 13: Continuous improvement of safety culture
“All individuals in the organization, from senior management
down, shall demonstrate leadership by promoting safety. A
normative framework based on international best practices
shall be described in the management system to promote and
support a strong safety culture”.
Requirement 14: Assessment of leadership and safety
culture.
“Senior management shall regularly commission
independent assessments and provide for self-
assessments of safety culture and leadership”.
IAEA
• Culture is seen as something we can influence,
rather than something we can control
• Culture work needs to encompass the whole
organization – not only as a top-down process
• Edgar Schein’s well established iceberg metaphor
helps to understand and how to continuously
improve safety culture
• observe visible aspects (artefacts and
behaviour, “above surface”) and;
• interpret this information to reveal the cultural
reasons behind (found in attitudes, values and
basic assumptions, “below surface”)
• For safety culture improvement, the IAEA
emphasises human interactions (shared space)
including trust, mindful communication, learning
attitude, inquiring attitude, self-accountability,
diversity, self-reflection etc.
IAEAs Approach to
Safety Culture Continuous Improvement
Artefacts,
Behaviour
Values
Attitudes
Basic
Assumptions
IAEA
Why assess RB safety culture?
• Regulators play an integral role in shaping safety culture within a
nuclear power program, including the culture of Licensees, through
their regulatory strategy, organizational culture, communication, and
style of engaging/influencing
• Regulatory bodies and Licensees require a common language and
understanding related to safety culture
• Regulatory Bodies must lead by example in terms of exploring and
understanding their own Safety Culture in order to influence
licensees in positive ways
• Regulatory Bodies need to stay at the forefront of understanding
Safety Culture and related human and organizational factors if they
are to assess and positively influence Licensees
IAEA
• IAEA Safety Standards and IAEA Safety Culture publications
• Social, Organization and Behaviour sciences
• Past experiences- IAEA and external SC expert knowledge and
experience of NPP Safety Culture Self Assessment training
approaches and materials
Course material is based on
IAEA
Methodology for Training
Each module is consist of:
• Core set of lectures based on the IAEA safety standards;
• Further illustration by country specific example(s) through
Application set presentations,
• Practical examples from experts and Case Studies and
interactive exercises to ensure that the training is based on
the ‘learning by doing’ approach..
• Group discussions for further clarification
• Outcome and Future initiatives
IAEA
Types of safety culture assessments
• Self-assessment
• Independent - internal / external assessment
• Mix of independent and internal assessment
• Meta analysis
IAEA
1. Ambassador Workshop (2 Weeks)
The course duration is 10 days, divided into two one week
blocks. In between the two training blocks a book about
culture is compulsory to read.
2. Senior Management Workshop (3 Days)
3. Train the Trainer (1 Week)
Training Course for Safety Culture Self Assessment
IAEA Training Course Package:
IAEA
To enhance the understanding of:
• Safety culture and the value of safety culture assessments;
• How improvement in safety culture can enhance safety
performance; and
• Everyone’s roles and responsibilities in cultural work.
To provide knowledge and practical skills in SCSA methods,
based on current international research and practices in
nuclear industries.
10 INLEP Module 1 19
June
2013
Objectives of the Training Course
IAEA
SCSA 1 To describe the concepts of culture and safety culture and explain their
relationship to safety performance
SCSA 2 To understand the role and impact of the Regulatory Body’s strategy
and safety culture on the National Nuclear Power Programme
SCSA 3 To explain the relationship between SCSA and an ‘inquiring attitude’
SCSA 4 To describe the methodology and process for SCSA, including roles
and responsibilities and critical success factors
SCSA 5 To describe and be able to use practical methods to identify safety
culture strengths and weaknesses
SCSA 6 To describe the various stages involved in SCSA, including
descriptive and normative analyses
SCSA 7 To understand the variables that impact the effective collection of data
and communication of findings in a SCSA
SCSA 8 To understand the basis for transformational change of culture
Learning objectives
IAEA
Prerequisite for Assessment of safety culture
GS-G-3.5
Safety culture self-assessement should:
• Include the entire organization
• Several different self-assessment tools should be used (e.g. interviews,
focus groups, questionnaires, observations and document reviews)
• A designated team representing all organizational levels and functions at
the installation should carry out the self-assessment
• A specialist in safety culture should be included in the team
• The self-assessment team should receive training
• The self-assessment team should summarize the results and identify areas
for improvement and may suggest actions to be taken
• The results should be reported to the management at an appropriate level
• A follow-up assessment should be performed
The independent assessment of safety culture should follow a similar approach
IAEA
The independent assessment of safety culture should follow a similar
approach as self-assessment
• The independence and qualification of the members of the
assessment team should be considered crucial for the success of the
assessment
• The team should be staffed with sufficient diversity of experience and
should include specialists in behavioural science, with knowledge of
statistical methods of analysis
• The independent assessment team should aim at identifying
strengths and areas for improvement
Prerequisite for Assessment of safety culture
GS-G-3.5
IAEA
Safety culture – multi-disciplinary expertise
• Anthropology
• Organizational theory
• Social Psychology
• Sociology
• Leadership and management theory
• Cognitive science
• Psychology
• Human Factor Engineering
• Resilience Engineering
• Organizational Factors
• ITO (interaction between Individuals, Technology and Organizations)
• Basic knowledge; Nuclear technology, nuclear organizations, regulatory
framework
Prerequisite for Assessment of safety culture
GS-G-3.5
IAEA
SCSA Process Step 1
• Scope
• Identify team
• Commit time and resources
• Strategy to address the results
• Communication strategy to inform staff
Step 2
• Train team members
• Clarify roles/ responsibilities
• Assessment strategy/methods
• Establish records management
Step 3
• Scope -cover activities from the
pre-launch to the communication
and follow-up of the results
• Minimize organizational disruption
• Adapt methods
• Plan to apply the methods in
parallel /independently
• Conduct a pre-job brief for the
team
• Step 4
• Orient the organization.
• Senior managers commitment
• Assessment is not an audit
• Learning about its existing
attitudes and behaviours and their
influence on safety
Step 5
• Capture information without interpretation or
judgment
• Maintain records
• Avoid personal biases
• Ensure access to information on progress of the
assessment, and respond to any questions
Step 6
• Analyse the data sources
• Draw conclusions according
to their logic
• Develop overarching themes
• Compare to the IAEA SCSA
framework
• Identify strengths and areas
for improvements
Step 7
• Develop key messages
• Develop communication packages
that suit the various target
audiences
• Team Lead to prepare a report with
input from team members
Step 8
• to management
• open dialogue regarding the
potential impact of the results
• Communicate with the organization
through various channels such as
discussion forums with groups,
cross-group training.
Step 9
• Engage the organization in action plan
finalization
• Depending on the approach preferred by
management, communication activities in Step 8
may include preliminary actions developed as a
consequence of the assessment
IAEA
Concept of Culture and Safety Culture
• The concept of culture is descriptive
• The concept of safety culture is normative
IAEA
Descriptive/Normative
Descriptive
‘is’
Based on data and a theory
of culture
Normative
‘should’
Based on data, a theory
of culture and a norm
Descriptive must come before the Normative!
• It sets standards for behaviour and
values
• It is related to practices known to lead
to safety
• It focuses on certain limited aspects of
the organization
• Being evaluative in the descriptive stage
can lead to exclusions
• Being descriptive helps to pick up weak
signals
• Being descriptive means having a broad
agenda for dialogue
IAEA
Normative analysis
Interview data
Cultural facts
Cultural themes
Survey data
Cultural facts
Cultural Themes
Focus group
data
Cultural facts
Cultural Themes
Document
data
Cultural facts
Cultural Themes
Observation
data
Cultural facts
Cultural Themes
Descriptive Analysis: Images of culture
Normative,
evaluative analysis
Safety culture is not a culture; it is a normative interpretation of a culture
IAEA
Implementation of SCSA
• IAEA SCSA methodology was first implemented by
Pakistan Nuclear Regulatory Authority (PNRA)
• IAEA Training was provided to PNRA in 2013
• Ambassador Workshop (2 Weeks)
• Senior Management Workshop (3 Days)
• Train the Trainer (1 Week)
• Regional Workshop on Safety Culture Self
Assessment for Senior Management was conducted
in Slovenia - November 2013
IAEA
Future Work for the Review of Safety Culture by the IAEA
Peer Review Services (IRRS)
• Question Sets for Independent Assessment of Safety Culture
for Regulatory Bodies (eg. IRRS Mission)
Independent Assessment
IAEA International Atomic Energy Agency
IAEAs Approach to
ISCA
Independent Safety Culture Assessment
within Operational Safety Review Team (OSART)
IAEA
Safety Culture Independent Assessment should: The independent assessment of safety culture should follow a similar approach as
self-assessment
• The independence and qualification of the members of the assessment team
should be considered crucial for the success of the assessment
• The team should be staffed with sufficient diversity of experience and should
include specialists in behavioural science, with knowledge of statistical
methods of analysis
• The independent assessment team should aim at identifying strengths and
areas for improvement
Safety Standard GS-G-3.5:
Assessment of Safety Culture
IAEA
The ISCA Teams Area of Expertise
• Psychology
• Cognitive science
• Sociology
• Social Psychology
• Organizational theory
• Cultural theory
• Leadership and management theory
• Human Factor / Human Factor Engineering
• Organizational Factors
• Resilience Engineering
• ITO (interaction between Individuals, Technology and
Organizations)
Basic knowledge; Nuclear technology, nuclear organizations,
regulatory framework
IAEA
Sources of information:
The IAEA ISCA Assessment Methods
Interviews
Team Findings
IAEA
Core of IAEA Safety Culture Analysis Process
e. g. independent assessment within OSART
Interview
data
Cultural facts
Cultural
themes
Survey
data
Cultural facts
Cultural
themes
Focus
group data
Cultural facts
Cultural
themes
Document
data
Cultural facts
Cultural
themes
Observation
data
Cultural facts
Cultural
themes
Overarching themes: Image(s) of culture
Final Issues;
Normative,
evaluative analysis
Team
findings
data
Cultural facts
Cultural
themes
IAEA
Example of amount of safety culture facts
• Team findings: Circa 220 facts
• 25 interviews: Circa 150 facts
• 7 observations: Circa 30 facts
• 7 focus groups: Circa 80 facts
• Survey 389 participated 97 questions
In total about 480 fact excluding the survey
material
IAEA
Reporting of the results
• Recommendations for areas in need of
attention
• Three senior managment workshops to
reach a shared understanding
• Exit meeting
• Report
IAEA
Summary
• The SCSA process involves 9 steps from
preparing the team and organization, through data
gathering and analysis, to reporting of results and
development of action plans
• Communication and engagement are critical along
the entire process
• Diversity of the team members is an asset that can
be used to avoid biases and enhance the rigour of
thinking through reflection and conversation