Harris Plant – Loss of Main Feed Pump Lube OilINPO ICES #305656 HNP Root Cause Evaluation 598302

Presentation Outline

Main Feedwater Pump Details

Loss of Lube Oil Event Description

Lube Oil System Overview

Related History

Event Causes

Lessons Learned

2

Main Feedwater Pump Details

Each pump provides 50% capacity. Flowserve (Ingersoll-Dresser) 16X19 CA-2 2 Stage - Barrel Type 9000 HP Motor Driven Flow 14,836 GPM Developed Head 2,216

Event Desciption

On Friday afternoon 3/29/2013 around 13:00, a member of the housekeeping staff called the control room and reported an unusual amount of oil around the base of the ‘B’ Main Feed Pump.

Operations found a substantial amount of oil in the catch basin under the pump and motor, no level indicated in the reservoir, but normal flow indicated in the bearing sight glasses.

Operations then entered into Abnormal Operating Procedure for a rapid downpower, reduced power to approximately 55%, and secured the ‘B’ MFP.

HNP declared a Unit Threat and the Outage Control Center was staffed.

MFP Lube Oil System Schematic

Aux Pump

Shaft Driven Pump

CUNO Filter

Pump Bearings (2)

V

PI

PS

2142

PS

2143

PS

2141

Sight Glass (2)

Orifice (2)Sight Glass

(2)

Motor Bearings (2)

1

4

23

5

6

7 8

1011

12

9LIC

12

1. Shaft Driven Lube Oil Pump 2. Motor Driven Lube Oil Pump – 1FW-

E014 3. CUNO Filter – 1FW-E020 4. Lube Oil Cooler – 1FW-E024 5. Pressure Relief Valve – 1LO-159 6. Main Feed Pump (MFP) – 1FW-

MFPB 7. MFP Motor – 1FW-MFPB:002 8. Motor Bearing Inlet Orifices – 1FW-

E018 9. Lube Oil Reservoir Breather 10. Pump Suction Foot Valves 11. Lube Oil Reservoir Immersion

Heater – 1FW-E026 12. Instrumentation (pressure switches

and local pressure indicator)

Lube Oil System Illustration (not including MFP)

10

Inboard Motor Bearing with Casing Disassembled

Oil Slinger Ring Arrangement

Outboard Motor Bearing Showing Connected Hoses

Related History

Neoprene stoppers installed in 1991 in MFP Motor bearing inspection ports to stop weepage and associated housekeeping problem without any formal evaluation.

Breathers were first installed on the MFP’s in 2004 to provide better filtration of contaminants. Subsequently it was determined these new breathers are subject to clogging from oil mist.

Several historical WO’s described oil leaks, which were cleaned up, with no cause of leak determined.

Mechanics began to slit the paper element in MFP breather in 2006 to eliminate oil leaks on the MFP Motors. There was no engineering evaluation or WO identified to direct this practice.

Site had grown tolerant of small amounts of leakage from motor and did not pursue cause of leakage.

Other new uses of breathers on site were also not trouble free. Type of breather changed from paper element to desiccant type about a month before event.

14

Event Causes

15

Root Cause:• Inadequate breather installed on ‘B’ MFP oil reservoir

Contributing Causes:• Specific design information for MFP lube oil sub-system not obtained.• MFP oil system pressure control valve return line dumps oil above the liquid surface in the oil

reservoir resulting in aeration.• Failure to effectively use available OE/CAP (clogged breathers can cause oil leaks).• Lack of ownership of oil reservoir breathers within the Predictive Maintenance Lube Oil

program.• Management follow-up or monitoring of activities did not identify problem.

Lessons Learned

Do not make changes to equipment without involving OEM.

Things can be more complex than they first appear.

Fully understand “what is the worst thing that can happen” before making changes.

Remember the old adage “If it ain’t broke ………………….

Aggressively pursue small signs that something is not quite right.

Don’t let time demands/pressure cause you to accept good news or explanations because they relieve you of adding another thing to your plate.

16

17