Pipeline Safety: Control Room Management /Human Factors “Final Rule

December 3, 2009”Pipeliners Club of Atlanta

January 11, 2010

Presentation Objectives

• Provide background information on development of the rule.

• Present elements of “new” control room management rule.

• Comment on ways to comply with rule, and go beyond compliance.

Background on Rule Development

• Pipeline and Hazardous Materials Safety Administration has been examining and evaluating and advising on Control Room operations for at least 14 years. (1996)

• NTSB has been making recommendations about “human factors” for at least 12 years. (1998)

• Pipeline Safety Improvement Act of 2002• Pipeline Inspection, Protection, Enforcement,

and Safety Act of 2006 (PIPES Act)

Safety Rec. P-05-001 (NTSB)

• Recommends that PHMSA:– Require operators of hazardous liquids

pipelines to follow the API Recommended Practice 1165 for the use of graphics on the SCADA* screens.

• NTSB had performed a safety study due to leaks going undetected after indications of a leak on the SCADA interface.

• New rule addresses this recommendation.

*SCADA = Supervisory Control and Data Acquisition

Safety Rec. P-05-002 (NTSB)

• Recommends that PHMSA:– Require pipeline companies to have a

policy for the review/audit of alarms.• New rule addresses this

recommendation.

Safety Rec. P-05-003 (NTSB)

• Recommends that PHMSA:– Require controller training to include simulator

or non-computerized simulations for controller recognition of abnormal operating conditions, in particular, leak events.

• New rule addresses this recommendation.

Safety Rec. P-05-004 (NTSB)

• Recommends that PHMSA:– Change the liquid accident reporting form

and require operators to provide data related to controller fatigue.

• This recommendation is being addressed in a change to accident reporting requirements for gas and liquids pipelines.

Safety Rec. P-99-12 (NTSB)

• Urges PHMSA, to:– “establish within two years scientifically based

hours of service regulations…– that set limits on hours of service,…– provide predictable work and rest schedules…– and consider circadian rhythms and human sleep

and rest requirements.”• New rule may or may not address this

recommendation, depending on whether you are NTSB or PHMSA.

Safety Rec. P-98-30 (NTSB)

• Urges PHMSA, to:– “assess the potential safety risks associated with

rotating pipeline controller shifts and…– establish industry guidelines for the development

and implementation of pipeline controller work schedules…

– that reduce the likelihood of accidents attributable to controller fatigue.”

• Closed May 3, 2006, due to P-99-12 being issued.

Reasons for NTSB Bulletins

• The NTSB noted that scientific research has shown that certain sleep factors can affect fatigue and performance, such as insufficient sleep, irregular schedules, and unpredictable schedules.

Reasons for NTSB Bulletins

• Recommendations resulted from the NTSB’s review of all transportation accidents reported to U.S. Department of Transportation (DOT) modal administrations over a 10 year period.

• NTSB noted that it had issued over 70 fatigue-related safety recommendations that resulted from its investigations of major accidents, special investigations, or safety studies that identified operator fatigue as a causal factor.

Reasons for NTSB Bulletins• Were there any pipeline accidents where

Control Room issues could have contributed to the accident?– Several examples cited:

• Colonial Pipeline 1996 (Controller fatigue)• Marathon Pipeline 1998 (SCADA & alarms)• Olympic Pipeline 1999 (SCADA maintenance)• Marathon-Ashland 2000 (Controller fatigue)

– NTSB study “10 of 18 hazardous liquids accidents have potential Control Room involvement”

Federal Most Wanted Transportation Safety Improvements

Federal Most Wanted Transportation Safety Improvements

Reduce Pipeline Accidents Caused by Human Fatigue

NTSB Most Wanted

• No Federal hours-of-service regulations exist for controllers of pipeline systems– Regulations exist in other transportation modes.

• Set working hour limits for pipeline controllers based on fatigue research, circadian rhythms, and sleep and rest requirements. – “Acceptable Response – Progressing Slowly”

PHMSA Bulletins

• “Pipeline Safety: Countermeasures to Prevent Human Fatigue in the Control Room” ADB-05-06

• “Potential Service Disruptions in SCADA Systems” ADB-03-09

• “Potential Service Interruptions in Supervisory Control and Data Acquisition Systems” ADB-99-03

PHMSA & Industry Activities• Controller Certification Study• Participation with industry associations

– API 1165 RP SCADA displays– API 1167 RP Alarm Mgt.(in progress)– API 1168 RP Control Room Mgt.

• Battelle Human Factors Analysis• API Control Room Forums• SGA Gas Control Framework Document• AGA White Papers

Congress• Pipeline Safety Improvement Act of 2002

– Study control room operations– Operator Qualification Requirements

• PIPES Act of 2006– “Reduce risks associated with human factors,

including fatigue, for pipeline controllers and other employees.”

– NTSB recommendations on display, alarms, and training

– Issue regulations

Notice of Proposed Rulemaking• PHMSA issued NPRM PHMSA-2007-27954 on

September 12, 2008.• Received 144 comments, mostly from liquids

and gas pipeline industry groups and companies.

• Majority of comments said “withdraw the proposed rule and start over.”

• Rule was not withdrawn, but PHMSA did take some comments into consideration before issuing the final rule.

Initial Summary Statements

• Must define the roles and responsibilities of controllers…

• And provide controllers with the necessary information, training, and processes to fulfill these responsibilities.

• …must also implement methods to prevent controller fatigue

Initial Summary (continued)

• …manage SCADA alarms• …assure control room considerations are

taken into account when changing pipeline equipment or configurations…

• …review reportable incidents or accidents to determine whether control room actions contributed to the event.

Effective Dates

• Final rule is effective February 1, 2010.• Develop control room management

procedures by August 1, 2011.• Implement the procedures by February 1,

2012.– Plenty of time for compliance?– Most companies still have questions about

what parts of the rule “really mean.”

Definitions in Parts 192 & 195

• Important for determining if rule applies:– Alarm– Controller– Control Room– Supervisory Control and Data

Acquisition System

Definitions in Parts 192 & 195

• Alarm means an audible or visible means of indicating to the controller that equipment or processes are outside operator-defined, safety-related parameters.

• Control Room means an operations center staffed by personnel charged with the responsibility for remotely monitoring and controlling a pipeline facility.

Definitions in Parts 192 & 195

• Controller means a qualified individual who remotely monitors and controls the safety-related operations of a pipeline facility via a SCADA system from a control room, and who has operational authority and accountability for the remote operational functions of the pipeline facility.

Definitions in Parts 192 & 195

• Supervisory Control and Data Acquisition (SCADA) system means a computer-based system or systems used by a controller in a control room that collects and displays information about a pipeline facility and may have the ability to send commands back to the pipeline facility.

Rule for Parts 192 & 195• If you have controller(s) in a control room

monitoring and controlling with a SCADA system;• Applies to natural gas pipelines, except

– Distribution with less than 250,000 services, or– Transmission without a compressor station

• Must still implement sections about fatigue, compliance validation, compliance and deviations.

• Applies to hazardous liquids pipelines.

Control Room Mgt. Section• Written control room management plan• Define controller roles and responsibilities:

– Normal, abnormal, and emergency operations– Authority to make decisions and take actions– Controller’s role during abnormal operating conditions– Controller’s role during emergency operating

conditions– Method of recording shift-changes and hand-over of

responsibility.• I would also define roles and responsibilities of

other jobs in relation to controllers so that every person understands roles and responsibilities.

“Provide adequate information”• Implement API 1165 (partially for gas) whenever

SCADA is changed.• Conduct point-to-point verification between SCADA

displays and field equipment whenever something is changed.

• Test and verify internal communication plan for manual operation of the pipeline.

• Test any backup SCADA systems.• Implement section 5 of API RP 1168 to establish

procedures for when different controller assumes responsibility.

• Very specific in this section, because it is easier to be specific about SCADA systems & equipment.

“Fatigue mitigation”• Establish shift lengths and schedule rotations

that…provide time…to achieve 8 hours of continuous sleep. Work time + commute time

• Educate controllers and supervisors in fatigue mitigation strategies and how off-duty activities contribute to fatigue. Regular, ongoing

• Train controllers and supervisors to recognize the effects of fatigue. Recognize signs and risks

• Establish a maximum limit on controller hours of service. Day, week, dayshift, nightshift

• Less specific in “socio-technical” system.

“Alarm Management”• Written alarm management plan• Review SCADA safety-related alarm operations using

a process that ensures alarms are accurate…• Monthly identification of alarms that are off-scan,

inhibited, false, etc. • Verify correct safety-related alarm set-point values and

alarm descriptions annually.• Review alarm management plan annually.• Monitor content and volume of general activity directed

to controller annually to ensure alarm response.• Address deficiencies in any of these items.• Specific again

“Change Management”• Establish communications between control room

representatives, management, and field personnel when planning and implementing changes. Document your plans and practices

• Require field personnel to contact control room during emergency conditions or when making field changes. How will you “require?”

• Seek control room or control room management participation in planning prior to implementation of significant pipeline hydraulic or configuration changes. What is “significant?”

• Less specific, due to more “socio-technical” elements

“Operating Experience”• Incorporate lessons learned from operating

experience into procedures and training.• Review incidents that meet reporting requirements

for contributions/deficiencies:– Controller fatigue– Field equipment– Operation of any safety device– Procedures– SCADA system configuration & performance.

• Do you have operating experience system?

“Training”• Establish, then review annually and improve

training program.• Responding to abnormal operating conditions

likely to occur.• Use of simulator or tabletop method for

training controllers to recognize abnormal operating conditions.

• Working knowledge of pipeline system…• Infrequently used setups and procedures.• Most companies have a training program.

“Compliance Validation”

• Submit procedures, when requested, to PHMSA or appropriate State agency.

“Compliance & Deviations”

• Maintain records for review:– Records that demonstrate compliance– Documentation to demonstrate that any deviations

from requirements was necessary for the safe operation of a pipeline facility.

Summary• Was this rule necessary? • Can this rule help improve pipeline operations?

– Incidents and near misses are caused by any and all of the areas in the rule.

– Most organizations are averse to learning.– Most organizations have a change management

process, but sometimes do not manage changes well.

• Is the final rule the “final” rule?– Believe SCADA and alarm management are.– Do not believe fatigue mitigation will be.