DESIGN, STARTUP, AND COMMISSIONING OF A 230 KV - 13.8 KVSUBSTATION

Copyright Material IEEEPaper No. PCIC-2006-30

Dean Ruiz, P.E.Senior Member, IEEEM S Benbow and Associates2450 Severn Ave, Ste 400Metairie, LA 70001USA

Rob StephensMember, IEEEExxonMobil500 W. St Bernard HwyChalmette, LA 70044USA

Paul GaynorMember, IEEEExxonMobil Research and Eng2800 Decker DriveBaytown, TX 77520USA

John Wilson, P.E.Member, IEEEM S Benbow and Associates2450 Severn Ave, Ste 400Metairie, LA 70001USA

Abstract - As part of an overall system reliability and capacityupgrade, facility management at a major refinery decided tocommission site engineers to design, startup, and commission agrass roots 230 kV-13.8 kV 40 MVA spot network substation.The management directive was to complete a "flawless" startupand commissioning of the substation. A flawless startup wasimportant to management because a functional failure in a mainelectrical substation was unacceptable. The costs of such afailure could easily exceed the initial costs of the substation andcause significant process equipment damage. It was deemed ascost effective by management and engineering personnel toensure that the design, construction, and testing of thesubstation was as thorough as possible. The design wascompleted by one team and reviewed by the fresh eyes of acompletely independent team. The complicated system waschecked by the vendor, checked again by the startup team, andthen completely checked for functionality at the end user'sfacility. The testing included the normal functional andsecondary injection testing. In addition primary injection wasused to test relays and CTs and actual faults to test certain relayfunctions. This paper will briefly outline the conceptual plan andengineering for the project. The primary focus of this paper willbe to detail the effort in developing a startup and commissioningplan, factory acceptance testing, and the actual startup andcommissioning. This paper will review each type of testperformed during the startup and commissioning, along with thebenefits and usefulness it provided. This paper will concludewith a discussion of the lessons learned during the testing.

Index Terms - Substation, Startup, Commissioning, Testing,Ring Bus, Spot Network, Ground Fault.

I. INTRODUCTION

In an effort to keep up with power demand and increasingreliability initiatives at a large refinery, management decided thata new 230 kV-13.8 kV electrical power substation should bedesigned, installed, tested, and commissioned. The drivingfactors behind management approval and funding for the projectwas increased capacity, increased reliability, and a "flawless"startup. For this reason, a procedural-based, layered approachto substation startup and commissioning was developed. Inorder to meet management mandated objectives, a team wasassembled that included technical specialists, facility engineers,consultants, utility representatives, and a project manager allcommitted to the total success of the project. Actually, there

were several teams involved in the project, with some membersbeing involved from start to finish, even the eventual operators.Some team members were involved only in the testing, startup,and commissioning of the substation. The startup andcommissioning will be the focus of this paper, but somebackground information will be provided to the reader. Theoriginal team members met and laid out the path forward for theproject. The path forward was set in the following order: 1)define project goals, 2) conceptualize the substation, 3) detailedengineering and design, 4) substation construction, 5)developing a testing and commissioning plan, 6) factoryacceptance test, 7) startup and commissioning, and 8) projectcloseout.

II. PROJECT GOALS

The original team members met and developed core goalsthat should be adhered to and maintained throughout the courseof the project. These goals were 1) safety, 2) reliability, 3)equipment protection, and 4) flawless startup-all issues foundand corrected prior to in-service day. These core goals wereused as benchmarks by team members throughout the courseof the project.

III. CONCEPT, ENGINEERING, AND DESIGN

A. Background

The genesis of the new substation was rooted in therealization that load creep had allowed the refinery load tosurpass the capacity of the two existing modified spot networksubstations that provide power to the 13.8 kV distributionsystem. Both of the existing substations reflected the designphilosophy of the period and showed indications of overloadingduring their service life. Circuit switchers were the primaryprotection for transformer through faults. There were no 230 kVcircuit breakers located on site. The 230 kV bus, the 13.8 kVbus, and 13.8 kV distribution breakers were outdoor designs.Analysis of the load creep incorporated projected plant growthas well as the traditional creep of system capacity. For thepurpose of the analysis, traditional load creep is defined as loadwhich is not associated with a large project and is added to thesystem every year as the unit debottlenecking optimizationprocess continues. A load duration curve modeling the 8-hour,15-minute demand on each of the existing spot networksubstations was developed. Load creep was determined using

1-4244-0559-9/06/$20.00 ©2006 IEEE

linear regression coupled with known future projects to developa 10 to 20 year load projection. This information was used todetermine the new substation size. From a reliability standpoint,the new substation had to address several issues associatedwith the two existing substations. The main issues to beaddressed were: 1) external interruptions, 2) human error, 3)wildlife intrusion, and 4) relay protection system failure.The mean time between failure (MTBF) rates experienced by

the refinery between 1995 and 2001 averaged 0.8 events peryear. Due to the absence of transmission breakers within therefinery substation, the 13.8 kV main breakers on the existingsubstations opened to clear lightning related faults that wereexternal to the plant by isolating the line section. The 230 kVtransmission line breakers reclosed automatically. The 13.8 kVmains could reclose under remote manual supervision by theutility. This resulted in extended periods of operation of a singletransformer at both substations, exposing the transformers' tooverloading and voltage sag as the system transformersautomatic load tap changers switched from parallel transformeroperation to single transformer operation.The existing substations consisted of outdoor switchgear

breakers located at grade, one substation with outdoor metal-clad switchgear and the other with outdoor circuit breakers, andopen 13.8 kV bus bars. The outdoor equipment increased therisk of failure due to animal intrusion and flying debris. Theexisting substations had experienced numerous relay failures inthe past, some causing major damage to electrical equipment.These failures resulted in significant down time. The existingsubstations did not have redundant relaying and the backuprelaying was at the transmission level located several miles way.A single relay failure exposed the substation to equipmentdamage and outage. Relay and relay trip circuit failuresresulted in faults not being cleared, exposing equipment to shortcircuit related damage. Due to the location of the facility, it wasdecided that a prefabricated substation building should beinstalled at an elevated level. The design would also specifyindoor switchgear with metal enclosed buses to reduce animalintrusion risk and prevent flying debris from causing outages.

B. Substation

The development of the design parameters for the newsubstation followed the company's design practices coupledwith the concept of no single point of failure and the ability tosafely follow NFPA 70E [4] recommendations for maintenanceof the substation equipment without power interruption to theusers. This guidance suggested a 230 kV breaker half ring bus(see Fig.1) was necessary as well as multiple breakers and abuilding with remote circuit breaker operation. The local utilitywas engaged early in the process as they would own andoperate the 230 kV bus and breakers. The utility and companyengineers recommended a four-breaker ring bus to achieve ahigh level of reliability and functionality. A reliability analysis,using a Monte Carlo simulation program, was performed todetermine overall reliability and cost of the different breaker

/ 230kV

Vtv N ~\ Xv M3.8kV

Fig. 1 - 230 kV Breaker Half Ring Bus

minimum five year mean time between downing events(MTBDE). Modeling of the 4-breaker ring bus yielded anMTBDE of 19.8 years while the 2-breaker arrangement yieldeda 5.2-year MTBDE. The 2-breaker design MTBDE was belowthe 4-breaker ring bus reliability, but was significantly higherthan the existing 1.25 MTBDE (MTBDE = 1/ MTBF = 1/0.8). Alife cycle cost analysis was performed for the two cases. Thefuture avoided cost savings due to lost production and repairs ofthe more reliable 4-breaker ring bus did not satisfy the netpresent value (NPV) of the discounted cash flow (DCF)threshold for the 2 additional breakers. There were twoinvestment thresholds, a lower DCF threshold for "non-retrofitable" equipment and a higher DCF threshold for "retrofitable" equipment. The 2-breaker modified ring bus was acceptedwith a design that would allow the system to be retrofitted into a4-breaker ring bus with minimal risk and impact on service.

C. Relaying

In an effort to address the facility problem with protectiverelaying at the substation level, a coordination philosophy had tobe developed. Standard relay protection consists of overlappingzones of protection and backup protection for both phase faultsand ground faults for all fault locations. The relay protectionphilosophy used in this project goes beyond a single fault (seeFig 2). It is based on a simultaneous electrical fault and a relayprotection circuit fault. Such a simultaneous fault shall isolate asingle zone and not result in a complete loss of substationpower supplied to the facility. Separate zones of protection areestablished around major electrical equipment components.The protective zones are divided into feeder breakers, buses,transformers, and transmission lines. When a fault is notisolated, backup protection clears the faulted equipment bysending a second trip signal to the circuit breaker or to anupstream circuit device. Feeder units have two redundantrelays. The relays are different models with different CT inputs,but perform the same functions and send simultaneous trips tothe feeder circuit breaker.The adjacent two main buses and bus tie section zones are

protected by full bus differential relays and backed up by partialbus differential protection. The main transformer zone ofprotection includes incoming 230 kV line and secondary 13.8

"box", 4) availability of functions such as communications, selfmonitoring, and a broader range of operating characteristics notfound in electro-mechanical relays, 5) lower burdens, and 6)event reporting. The event reporting feature was usedextensively to confirm test results.

D. Switchgear

Fig. 2 - Phase Protection One-Line Diagram

kV metal enclosed buses. This zone has primary protection bydifferential relays, with primary and secondary overcurrentprotection, secondary ground fault, and secondary differentialground fault protection as backups. A breaker failure relay,armed by downstream protective relay and associated lockoutrelay operation, is activated after a time delay, removing theprimary power supply to that side of the bus.The relay design provides equipment isolation that is limited

to a single zone of protection during simultaneous occurrence ofa fault and a single relay failure. All breaker trip coil circuitshave voltage monitors with alarms. Simultaneous loss of afeeder breaker trip coil during a fault would result in tripping theupstream zone of protection main breaker and both bus tiebreakers. Past failures of main breaker or bus tie breakersfailing to operate during faults resulted in damaged equipmentand long outages for material replacement. When the mainbreaker trips due to a fault, the upstream circuit switcher is alsotripped. When either of the two bus tie breakers receives a tripsignal both breakers trip. Tripping the additional breakersmaintains plant power supply through the remaining bus andassures primary and backup fault isolation.The project design incorporated the use of microprocessor

relays. The reasons the project decided to utilizemicroprocessor based relays include: 1) greater accuracy andmore repeatability, 2) equivalent protection functions can beobtained at lower cost, 3) multiple protective functions in one

The existing dual incoming utility power supplies are operatedin parallel through a normally closed bus tie breaker called aspot network. Past failures of the bus tie breakers and bus faultsresulted in loss of the entire spot network bus. Control wire fire,resulting from ground potential rise due to lightning, also causedthe loss of the entire spot network buses. A bus fault adjacent tothe bus tie breaker and a main breaker trip circuit failureresulted in the fault not being immediately cleared. The lack ofbackup transformer primary protection resulted in an extendedfault duration, damaging the bus section and destroying thetransformer. A long term outage to repair half the spot networkbus and to replace major equipment was required. The facilitywas able to operate on the remaining utility supply with greatlyreduced reliability. Given this history, the final design includeddual bus tie breakers designed so that both trip simultaneously.This had the additional benefit of allowing complete isolation ofthe tie breaker bus for maintenance by shutting down one mainbus and both bus tie breakers.

In order to minimize interruption to the facility, the use of auto-reclosure on the 13.8 kV main circuit breakers at the newsubstation was investigated. The existing substation 13.8 kVmain breakers are part of the 230 kV transmission line reclosersystems. During lightning activity, the 230 kV breakers isolatethe faulted line and reclose. To prevent back feed of the powersupply through the 13.8 kV spot network from one 230 kV linesection to the isolated 230 kV line section, sensitive reversepower (32-device) relays trip the 13.8 kV main breaker.Opening of the 13.8 kV main is necessary for complete isolationof the lightning affected transmission line. The utility 230 kVbreakers will automatically reclose, but the 13.8 kV mains aremanually reclosed, remotely, by the utility. Extended operationon one utility line continues until the utility first inspects theirequipment to verify its condition and manually recloses the mainbreaker. It was determined that this was not acceptable in thedesign of the new substation. The new substation would utilizedirectional current monitoring to open the main breaker fortransmission line faults similar to ones experienced by therefinery's existing two substations. To avoid the delayedmanually supervised breaker reclose, an automatic reclosecontrol was installed. Upon detection of a line side problem, thedirectional current (67-device) trips the main breaker via adedicated directional current lockout relay. The lockout relay issupervised by the upstream circuit switcher position and themain breaker position. If the appropriate criteria are met, thelockout relay electrically resets. The main breaker automaticreclosure is additionally supervised by a synchronism checkrelay (25-device) and other trip conditions. The automaticreclose is a one-shot reclose. If it trips again, a second reclosurewill not occur.

Arc flash is an issue that is ever present and requires theutmost respect and attention. There are methods in switchgeardesign to assist with arc flash. Switchgear can be designed tore-direct the force of the arc flash or even intentionally place athree-phase fault on the bus to increase fault current, thereby

decreasing clearing time. For this substation, neither of thoseapproaches was utilized. Rather, the company determined thatarc flash management was the preferred philosophy. To do this,the company determined that removing the employee from thefront of the switchgear was an effective practice to protect theemployee. All of the circuit breaker controls were moved to aremote location (see Fia. 3). Remote feeder breaker controls

Fig. 3 - Remote Breaker Control Panel

reduced operator exposure to arc flash hazards by eliminatingwork on or near energized equipment during normal equipmentoperation and maintenance activities. Remote manual breakercontrols located beyond the arc flash exposure zone within thesubstation building were installed. Remote breaker rack-in andrack-out control systems were also installed. The dual tieconfiguration also 'allowed complete power supply isolation formaintenance access into the bus-tie cubicle with one of the twobus sections de-energized.

IV. STARTUP AND COMMISSIONING

This section contains the major purpose for writing this paper:to inform and educate the reader on how the new substationand its associated equipment were tested and commissioned fora flawless startup. The important factors were to develop projectspecific testing procedures, plans, and forms prior to performingany testing. The approach had to be structured and detailed,while being flexible enough to be modified as conditionsdictated. The testing plan had to be developed, reviewed, andrevised prior to the start of the factory acceptance tests to allowscheduling to make sure the time and equipment to conduct thetests were available. A well thought out plan will be the baselineto ensure that the necessary testing gets done even asconditions change.

Prior to any tests or checks being performed, anorganizational chart of the startup and commissioning team wasdeveloped to clearly define all roles and responsibilities,including the selection of the decision maker. This person hasthe control to keep team members working on the plan and notdeviating from the plan, unless it was warranted. The startupteam was comprised largely of members not involved in theoriginal planning and design of the project. This approachprovided an opportunity for completing a "cold eyes" review of

the project prior to proceeding with the factory testing andstartup.

Detailed procedures were developed for checking out allaspects of the substation. The procedures were developedusing the manufacturer-furnished drawings in addition to clientstandard checkout procedures and other industry-acceptedstandards. Detailed procedures for each test were prepared andreviewed prior to the start of testing. The procedures listed thetest equipment required, the connections that were to be made,and the configuration of the substation equipment for the test.

Calculations were included that predicted the current andvoltage levels to expect during each test. The calculations forexpected test results are much more accurate when done in acontrolled environment such as an office rather than developingthem "on the fly" at the work site. Check sheets were provided tocompare predicted values against actual values. These checksheets listed each meter and relay involved in each test, and theparameters that needed to be recorded. A detailed testingschedule was developed as a method of organizing the manydifferent groups involved. To maintain productivity, testing wasstaggered to sequence the work and avoid conflicts.

A. FactoryAcceptance Test

The goal of the factory acceptance test (FAT) should be totest every circuit and piece of equipment that can be tested atthat time. The personnel to identify and correct problems arethere with the sole purpose of making sure that the equipment isas specified and rectify other items that may have not beenaddressed in the original design. The cost of changes made atthe factory will be far less than those made in the field. For thissubstation, there was an FAT for the substation transformersand the 13.8 kV switchgear building. The switchgear buildingFAT included all relay outputs and permissives, remote breakeroperation (rack in, rack out, open, close), auto-reclose, and acomplete check of the building to ensure it met specification.The factory acceptance test team was assigned individual

tasks in an effort to maximize the amount of checks performedduring the 5-day test period of the switchgear building. It shouldalso be noted that there were periodic meetings with themanufacturers to discuss projects, address issues, and keep theproject on track. The schedule for the switchgear FAT wasdeveloped and presented to the manufacturer on the first day ofthe testing. Unfortunately, the manufacturer was not prepared tostart at the same point as the testing team and had equipmentset up for a different sequence of test. This would have resultedin lost time waiting for the manufacturer to properly align theequipment if the testing plan had not been flexible.Lesson Learned #1: project team and factory personnel need

to jointly work on the testing schedule prior to testing.The schedule was modified each day to account for that day's

testing and planning for the remaining testing. A punch list wasdeveloped and presented to the manufacturer each day. Themanufacturer would then assign a crew to work on those itemsovernight so that they were corrected prior to the next day oftesting. After the initial scheduling difficulties, the switchgearFAT was completed to the manufacturer and testing team'ssatisfaction.

B. Site Checkout And Commissioning

The goal of this part of the checkout was to improve on theplan developed during the FAT. As good as any plan can be, itmust be flexible enough to adapt and improve. Armed with afirst hand knowledge of how the equipment was constructed, thestartup and commissioning team attacked the substation withthe project core goals in mind, particularly safety and flawlessstartup. For this reason, the organization chart (see Fig. 4) ofthe startup team was modified to include a startup lead person.This person was a site electrical

StartupManager

SubstationStartup

Org Char

Fig. 4 - Substation Startup Org Chart

specialist who was familiar with every aspect of the project. Hispurpose was to ensure that all facility safety procedures wereadhered to and that the testing met facility requirements fromboth a maintenance and an operation standpoint. The startupand testing schedule was difficult to develop and track. Therewere several groups whose efforts had to be coordinated andprioritized. Due to facility startup requirements, there were partsof the substation still under construction while testing was beingperformed. This was being done in an effort to meetmanagement requirements for a startup date. A grass rootsprocess unit was being constructed simultaneously within thefacility battery limits whose startup depended directly on thecommissioning of the new substation. There would be significantfinancial and operation repercussions if the target dates werenot met. The varying groups to be coordinated were the localutility, construction contractor, sub-contractors, consultants,testing firms, and manufacturers. This schedule was critical tohaving the appropriate group in the appropriate place at theappropriate time. The startup team and the other groups met ona daily basis first thing in the morning to discuss safety, reviewprogress, and discuss the day's tasks and the potential impacton other areas of the substation.

It was important to build on the testing and checks that wereperformed at the FAT. The relay systems, the most complicatedsub system in the building, were a possible source of failure.The relays could be programmed incorrectly, the relayphilosophy might be flawed, or equipment failure might cause anundesired operation. For these reasons, it was decided to notonly apply the standard tests to the substation checkout, but tointentionally apply single-phase faults to the system at specified

points with anticipated results predetermined. The followingsections will discuss the key tests performed during startup andcommissioning.

1) Circulating Current (vars): The purpose of this testwas to circulate phase current of a significant magnitude in aneffort to check for open CTs, verify CT polarities, verifydirectional relay operations, and verify power meter operation.With two transformers, equipped with load tap changers,connected in parallel on the primary and secondary, full loadcurrent could be obtained by raising one transformer tap tomax and lowering the other (see Table 1).Lesson Leamed #2: this would have been a good time to

infrared scan the bus for high resistance connections.By coordinating the tap changes, the voltage was kept at

nominal (raise one, lower the other). All current readings onthe meters and relays were checked. With the voltagespresent, the direction of the current was detected andmonitored using the relays and meters, allowing thedirectional relays to be checked. Additional checks included

TABLE 1CIRCULATING CURRENT - SAMPLE CALCULATIONS

.................................................................

Transformer LTC taps are 5/8%; 5/8% x (13800 / 13) = 49.8 V /tapTransformer %Z = 1 0, MVA = 40; Voltage = 230 kV - 13.8 kVZbase (Zb) = kV / MVA = 13.82/40 = 4.76 )Transformer Zt at base = %Z / 100 x Zb - 10 / 100 x 4.76 = 0.476 )Loop Z of two transformers (ZI) = 2 x Zt 2 x 0.476 = 0. 952 Q*

*does not include bus Z;Circulating current induced by 1 tap change

Icirc- Vpertap/Zl -=49.8V/0.952 - 52.3AAt R16 on one transformer and L16 on the other

32 taps

the verification that the three phases to each overcurrentmicroprocessor relay were wired correctly. If the three phasecurrents didn't cancel each other out, the 51 N in themicroprocessor relay would send a trip signal. The next checkincluded the verification that the directional relaying (67-device) picked up and activated a trip signal when the varcurrent flowed out of its incoming breaker toward the utility. Itwas verified that the directional relay on the opposite mainbreaker blocked a trip because the var current flowed fromthe transformer through the incomer into the bus. The testwas performed in both directions. The final check was fullyloading the transformer to check for any unforeseenproblems. The circulation checks found that one of thetransformer's primary side phase CTs was wired backwardsto the relay. The relay's 51 N tripped while the bus was loadingup. The phase CT wiring was reversed to correct thisproblem, carefully documented, and rechecked to confirm theproper operation. This relay circuit would have provided afuture unwanted trip, or lack of trip, with the process andbusiness interruption problems that went along with it. Plan,procedures, and testing were justified and paid for with thisone error correction.Lesson Leamed #3: directional relays need to be checked

under actual conditions.

2) Primary Current Injection: A 700 A current injectionset was used for primary current injection. The capacity of this

set was marginal. The 700 A rating was for use as a circuitbreaker test set with very low resistance. The resistance ofthe substation bus and the connection cables limited thecurrent available to 400 A. With current injected on one phaseof the primary buses, all of the meters and relays on that buswere read and checked off. From this test, it was determinedthat the CT ratios were correct, the meter and relay scalingwas correct, and the proper phases were connected.

Lesson Leamed #4: primary injection is the only sure wayto be certain that all factors in a microprocessor based systemare correct.One wiring issue was found regarding the wiring of the

partial differential relaying system. When it was possible, thetest current was used to trip the relays. The low currentcapacity of the test set limited these tests.Lesson Learned #5: use a larger current test set than you

think you need, preferably one that is sized to supplyadequate levels of short circuit current. A rental generatormay be required to supply enough power to the test set.

3) Primary Voltage Injection: A 480 V variac was used toinject 480V into the primary of the 230 kV - 13.8 kVtransformer with the transformer secondary shorted. Theimpedance of the transformer and the bus limited the primary

TABLE 2PRIMARY VOLTAGE INJECTION SAMPLE

CALCULATIONS

Transformer %Z = 10, MVA = 40; Voltage = 230 kV - 13.8 KvBy definition: With 10% of primary voltage applied and transformersecondary shorted, transformer FLA will be developed

10% of primary voltage - 230000 x 0.10 = 23000With a 480 V applied to the transformer primary

Isec = (480 /23000) x 1674 = 35 Acurrent to 3 A and the secondary current to 35 A (see Table2). This current was enough to indicate on the primary (300:5ratio) and secondary relays (3000:5 ratio) and determine thatthe ratios and the phase connections on the CTs located inthe transformer were correct. The use of a boostertransformer to get a higher voltage input would have madethe test connections more difficult, but would have providedbetter readings.Lesson Leamed #6: use a high enough voltage for voltage

injection to get significant current readings.

Fig. 5- Ground Protection One-line Diagram

temporary hard wire connection to each phase of four specificfeeders to ground. These intentional faults were (see Fig. 6)applied multiple times to each feeder, removing a layer ofprotection each time. The length of time that these intentionalsystem faults were allowed to stay on the system neverexceeded any equipment rating. As a precaution, personnelwere placed at circuit breaker control switches to isolate thefault if the protective relays did not operate as designed.Portable cooling fans were placed on the resistors to keepthem within their temperature ratings and an infraredthermometer was used to track resistor temperatures. Thiswas done to ensure that the resistor had ample time to coolprior to another ground fault test. Extensive ground faulttesting was required due to the

4) Bolted Ground Fault: This test was performed in aneffort to emulate real world conditions and confirm that theground fault relaying worked as intended. The new substationwas designed as a low resistance grounded system. Theresistors in the neutral circuit were designed to limit theground fault current to 400 A on each transformer or 800 A ona feeder when the transformers are in parallel. Thetransformer's full load amp rating is 1674 A, the bus is ratedfor 3000 A, and the resistors are rated at 400 A for 10 s.There were numerous levels of ground protection at thesubstation (see Fig. 5). This test involved installing a

closed. The relays operated and tripped the breaker. Therelay data was recovered and analyzed to determine if thecorrect relay tripped. That trip was disabled with the testswitch and the test was repeated. The temperature of theresistors was monitored and the tests were not repeated untilthe resistors cooled to a specific temperature. This protectedthe resistors and kept the current fairly constant, as hotresistors draw less current and prolong tripping times.Lesson Learned #7: resistor temperatures need to be kept

constant at the start of each test, as it will significantly affecttrip times due to current decrement.The tests continued until the entire ground fault relay

system was tested and compared. The tests were performedwith both the tie breaker closed (two sources, 800 A groundfault current) and the tie breaker opened (one source, 400 Aground fault current) conditions. The results were effective inlocating several CT wiring errors, largely in polarity. Theseerrors were corrected, documented, and retested to confirmthe proper operation. Once the tests were complete, a table ofthe operating times was developed (Table 3). During thetesting, the restricted ground fault

TABLE 3GROUND FAULT TESTING VALUES

Relay

Fig. 6- Bolted Ground Fault Connection

resistance grounded system. All ground faults, regardless oflocation, produce essentially the same ground fault currentback to the transformer. Some of the in-facility faults will becleared by fuses and some by field ground fault relays. Theclearing time between main substation feeder relays, partialbus differential relays, restricted ground fault relays, and mainbreaker ground fault relays is critical and needs to functionproperly for individual transformer operation as well as paralleltransformer operation. Having a ground fault on a feeder tripthe substation mains would not be acceptable and definitelynot meet the project's goals. This test verified that there wasproper coordination between the following relays: 1) feederground (50GS, 51 N)-trips the feeder breaker, 2) bus partialdifferential (51 N)-trips the main and tie breaker, 3)transformer (51 G2)-trips the tie breaker, 4) transformer(51G1)-trips the main incoming breaker and the circuitswitcher located on the high side of the transformer, 5) mainbreaker (51 G)-trips the main and the circuit switcher locatedon the high side of the transformer, and 6) transformer (87G)-trips the main and the circuit switcher located on the highside of the transformer if the fault is within the protection zone.A table of predicted operating times based on relay settingswas developed prior to the test. A large ground jumper wasconnected between one phase of a feeder breaker output lugand the ground bus. This is a definitive example of a boltedground fault. With the appropriate personnel in the properlocations, the appropriate feeder circuit breaker was then

Trips Gnd OpAmps Time

(. )22 (s

Tie Open, 400 A ground current, fault on Bus 5Feeder Feeder ? 2.843G2 Tie ? 4.116Gl Main ? 5.109PB Main & Tie 372 7.845

a6...............9...........................................................

Main Main 372 8.569

Bus 5ResTemp(OF)

198307307456456

Bus 6ResTemp(OF)

N/AN/AN/A

-N/AN/A

Tie closed, 400 A ground current seen by Main, Gl, G2; 800A groundcurrent seen by Feeder and PBFeeder Feeder ? 1.725 169 130PB Main &Tie 804 2.008 ?G2-Bus 5 Tie 390 4.013 293 192Gl -Bus 5 Main 390 4.983 293 192G2 - Bus 6 Tie 411 3.917 293 192G1-Bus 6 Main 411 Pu 293 192

----'''''' '' T-----------------------------------------------------------. ...................................................... 7-----------------

Main Main 378 7.845 390 200

differential relays (87G) protecting the transformers trippedwhile the fault was outside the protection zone (protectionzone is the transformer low side winding and bus duct to themain breaker). The CT connections were reversed and therelays functioned correctly. These relays were to be testedlater by reversing the CT leads. Again, the testing paid foritself by preventing a future process interruption as the relaywould have tripped on a feeder problem and taken out both ofthe transformers.

5) Loading The Substation: After the substation had beenchecked and tested, it was time to add load. Due tomanagement required deadlines, the load had to be added tothe substation without taking a unit outage and minimizingfacility exposure. Detailed switching and check procedureswere developed, reviewed, and utilized to transfer existingloads to the new substation live. The operation was

completed without incident and was transparent to thecustomers (process units).

V. PROJECT CLOSEOUT

The company, all the way from management to project teampersonnel, displayed a commitment to doing the job correctly.This resulted in a completely tested, well documented, andreliable product being delivered. It required more up front costs,but was necessary to prevent the higher costs that would resultfrom failures after the in service date. Considering some of theissues that were found during the intensive testing, and thedown time they could have caused, the up front cost was welljustified. It gave the site personnel confidence that thesubstation operation during steady state and transientconditions would be as designed. As part of the project closeout,a lessons learned meeting was held to determine what thingscould be been designed differently or tested in a different way toimprove the quality of the substation. Some of the key itemsdiscussed were: 1) the use of 3d party checks to verify thesystem performed as desired, particularly with complicatedrelaying systems, 2) documentation is always an issue. Makingsure that all of the changes make their way onto the appropriatedrawings is a challenge. Keep one set as the master. Make allmarkups on that set and use it as the as built. Keep it up to datedaily. Make one person responsible. It is worth the time andmoney to keep the documentation correct.Lesson Leamed #8: keep as-built drawings as-built.Resolve the questions as soon as possible while everyone's

memory is fresh not days later when the equipment is energizedand unavailableLesson Learned #9: complete as-builts as soon as possible.

3) Originally, all design errors were being funneled through aprocess with the original design consultant and the turnaroundtime associated for the corrections took too long. The startuplead determined that future errors would be addressed by thefield startup team.Lesson Learned #10: correct design errors in the field,

document them well, and review them concurrently, not serially.4) There was a timing problem in programming and testing therelays. Relay setting changes were being made by one firm,relay programming was being performed by another firm, andyet another firm was testing the relays, all at the same time. Thisproved to be very inefficient and could have been better if all ofthe relaying items had been addressed prior to the testing firmperforming their tasks. The testing firm could have alsodeveloped a lot of their test calculations prior to arriving onsite.Lesson Leamed #11: relay settings need to be finished and

reviewed before field testing.

A. Operating Procedures

Once the substation was commissioned and served facilityloads, it was important to communicate to those who were notpart of the startup team how to operate the substation. Theoperating procedures should detail the step-by-step process tooperate the equipment as it is designed to operate. Theseprocedures need to incorporate all of the thought that went intothe design to provide personnel a background on the intent ofthe design and a structured format to safely and reliably operatethe substation. The members of the startup team wroteprocedures on how to operate every piece of equipment in the

substation, utilizing illustrations and pictures to communicate thepoint. The procedures were specific to each piece of equipmentin addition to being combined into a complete procedure on howto energize and de-energize the substation. It included relaylogic diagrams, relay settings, drawings, and step by stepinstructions. Some of the specific procedures included circuitbreaker lockout and removal, circuit breaker racking, use of the"dummy" circuit breaker, and how to attain event reports fromthe relays. After the procedures were complete, a training classwas held to instruct individuals on how to use the books. Thetraining class was provided to end user engineers andtechnicians to ensure they were familiar with the equipment,where it was located, how it was designed to operate, andwhere the information on the equipment can be found (and howto use it).Lesson Leamed #12: the design and startup team should

participate in the preparation and review of the operatingprocedures.

B. Keys to Success

There were several keys to having a successful and "flawless"startup. A commitment from company management to supportthe effort and a startup team committed to delivering a qualityproduct resulted in success. The personalities on the startupteam led to balanced discussion when trying to resolveuncertain items. To summarize the keys to success: 1)balanced discussions, 2) document all errors and field changes,3) testing must be thorough, reasonable, and test the desiredfunction. This found several errors that could have causednuisance trips in the future, not to mention being difficult totroubleshoot, 4) QAQC-everything that was done wasreviewed to ensure that it met the intent of design philosophy,and 5) manufacturer input-they know their equipment.

VI. KATRINA UPDATE

After Hurricane Katrina devastated the area, the originalstartup team was assembled and tasked with starting up thefacility. The two older substations experienced significantdamage. The water in the substations was 50" in depth andcaused the outdoor, grade level equipment to sustain majordamage. The new, indoor substation on the other hand,sustained minimal damage because it was elevated above thehigh water. More importantly, all of the procedures that weredeveloped for the original startup, commissioning, and operationwere pulled off the shelf and used to re-commission thesubstation. Once again, this resulted in a flawless startup.

VIl. CONCLUSION

Thorough checkout, testing, and documentation are worth theexpenditure. One problem discovered during checkout canprevent costly interruptions later and avoid embarrassingexplanations later. Make sure the testing budget on the projectis adequate to do complete on site testing and do not let therush to complete and close the project keep the as builtdrawings from being complete. Enough time and money shouldbe budgeted to write operating procedures. Too many times thesafety features that are designed into the system are not evidentto or understood by operators and are inadvertently disabledwith disastrous results. No matter how brilliant, innovative, andwell-thought out the designs are, they are only good if the

people who use the equipment understand the equipment andtools available to them. Management decided that it was moreimportant to spend "up front" money in an effort to minimize"back end" money. Using an old cliche.. you can pay me now oryou can pay me more later.

Lesson Learned #13: budget the funds and engineering timefor adequate testing, true as-builts, and operating procedures.

Vil. ACKNOWLEDGEMENTS

The authors wish to thank all of the many people who helpedeither directly or otherwise, in designing and building thesubstation as well as preparing the material that appears in thispaper. Special thanks to Stephen Cutrufello, the projectmanager, who tirelessly put up with our numerous engineeringchanges designed to "get it right" and Thomas J. Watson, thereal brains behind the project. The automatic breaker rack in/outdesign was just one of Tommy's many ideas.

IX REFERENCES

[1] Electrical Machinery Fundamentals, Stephen J.Chapman

[2] Acceptance Testing Specifications for Electrical PowerDistribution Equipment and Systems (ATS-2003),International Electrical Testing Association, Inc.

[3] Maintenance Testing Specifications for Electrical PowerDistribution Equipment and Systems (MTS-2001),International Electrical Testing Association, Inc.

[4] Standard for Electrical Safety in the Workplace (NFPA70E-2004) National Fire Protection Association

[5] National Electrical Code (NFPA 70-2002), National FireProtection Association

[6] IEEE Recommended Practice for Protection andCoordination of Industrial and Commercial PowerSystems, IEEE Std 242-2001 (Buff Book)

[7] Reliability Statistics, Robert a. Dovich, ASQ QualityPress

through. He is an IEEE member and is presently employed byExxonMobil Corporation. Rob's email address isrobert. r.stephens@exxonmobil .com.

Paul A. Gaynor graduated from Manhattan College MagnaCom Laude with a Bachelor of Electrical Engineering degree in1990. He later graduated from Rensselaer Polytechnic Institutewith a Masters of Electric Power Engineering degree in 1991,where he also worked as a research assistant. He is presentlyemployed at ExxonMobil Research & Engineering at theBaytown Area Engineering Office. He has recently authoredand presented technical papers at Saudi Aramco and at severalprevious PCIC conferences. He is an active IEEE member.Paul's email address is paul.a.gaynor@exxonmobil.com.

John L. Wilson graduated from Tulane University in 1964with a BSEE. He retired after 30 years holding various positionsin engineering and supervision with Monsanto Company. He ispresident of John L. Wilson, PE Inc. and a Senior Engineer withMS Benbow & Associates. John is an Adjunct AssistantProfessor of Electrical Engineering and Computer Science atTulane University and is a member of IEEE, ISA, NSPE, LSPEand LES. John's email address is jlwilson@msbenbow.com.

X VITA

Dean C. Ruiz graduated from the University of New Orleanswith a BGS in Marketing and Sales of Engineering in 1994 anda BS in Electrical Engineering in 1996. He has worked withcolleges on High School Engineering Career Day and otherengineering activities promoting the advancement ofengineering. Dean is an active contributor and volunteer toLouisiana Special Olympics. He also served on the hostcommittee for IEEE PCIC 2002 in New Orleans, Louisiana. Heis presently employed as the Electrical Engineering DepartmentTechnical Manager for M S Benbow and Associates located inMetairie, Louisiana. Dean is an active member of IEEE, ISA,NFPA, NETA, NSPE, LSPE, and LES. Dean's email address isdcruiz@msbenbow.com.

Rob Stephens graduated from Ohio University, Athens, Ohioin 1975 with a Bachelor of Science in Electrical Engineering. In1986, he graduated from NOVA University, Fort Lauderdale,Florida, with a Masters Degree in Business Administration. Hehas worked with local grade schools promoting women inengineering for the Women Engineering Career week. He hasco-authored technical papers on the topic of voltage sag ride