Upload
elijah-stevens
View
220
Download
2
Tags:
Embed Size (px)
Citation preview
1
The National Finance Center and
Hurricane Katrina
John White, Deputy Director
June 2006
2
USDA, Office of the Chief Financial Officer
Chief Financial Officer’s View - USDA Characteristics:
- 110,000 employees @ 14,000 offices and field locations- 300+ national programs - $77B in annual spending- $128B in assets- $100B in loans
Program Management Responsibility Includes:- Financial leadership across the enterprise; roughly
equivalent in size to 6th largest private sector firm in U.S.- Financial policy and planning- Financial systems and operations- Government-wide service delivery through the National
Finance Center (NFC)
3
Scope of USDA CFO Operations inNew Orleans (co-located)
National Finance Center (NFC)- Government-wide payroll/personnel for 585,000 employees- Government-wide Human Resource services- Government-wide Health Benefits Programs- Disbursements and Collections- Data Center Services- At time of Katrina, over 1300 employees
CFO Controller Operations Division- Provides financial services to USDA agencies, including administrative payments
(2.5 million annually), interagency payments, reconciliations, and vendor file- Assembles USDA financial reports, including FACTS I & II, and annual consolidated
financial statements- At time of Katrina, almost 300 employees
CFO Financial Systems Division- Operates the corporate consolidated financial system for all USDA agencies- Operates administrative systems, such as travel and property, for all USDA agencies- Creates and reconciles data extracts for consolidated audit and financial statements- At time of Katrina, over 70 employees
4
Prolog: Accountability – Guiding Principles
Embrace that you are in charge and responsible for your organization, its finances, and people.
Be prepared to go it alone, do what is necessary to get the job done and care for your staff. In a major situation, there may be no one else to consult about how to proceed.
Never forget that during a crisis, people demand leadership, decisions must be made--often in a vacuum, stick to the plan but be flexible.
5
Preface: Risk Management – Guiding Actions
Managing to deliver the expected business results by considering risks in doing what we do.
Focusing on most significant risks associated with the nature of the business:- Economic – controls & testing based on what is at stake- Operational – environmental, internal, natural, compliance,
technical, organizational, and human risks
Keys to successful risk management:- Understand the risk profile of the business- Leadership sets the tone- Integrate risk management into day-to-day practices and
decisions- Evaluate on clarity, transparency, integrity, & performance
6
Preface: Business ContinuityApproach – Prepare!
Business Impact Assessment Disaster Recovery Plan Business Continuity Plan Uninterruptible Power Supply Emergency Power Supply Two annual NFC drills USDA and Government wide drills Three historical near misses
7
8
Hurricane Katrina
Activated DR on Friday
evening Deployed Advanced Teams
on Saturday morning Completed payroll late
Saturday evening Shutdown data center and
facility early Sunday Made disaster declaration
Sunday evening Began implementation of
COOP on Monday
Initial Timeline:
9
COOP Concept
Subscription service for
data center, workstations, and bulk print and mail
Philadelphia site used for data center and 101 data center staff seats
Grand Prairie site used for 150 business operations seats
Plans all geared to loss of New Orleans facilities
Concept of operations:
10
11
Realities of Katrina
Implications were larger
than anyone imagined This would be a long-term
situation Immediate focus was:
- Service to the customers- Accountability and
caring for employees- Fiscal accountability
Balance delivery and personal needs
Initial reactions:
12
Oversight Priorities
Incorporating local efforts
into Departmental and Government-wide efforts
Keeping long-term, big picture in focus
Finding ways to add value while allowing those on the line to keep things moving: - Policies- Resources- Contracting- Financial- Liaison - Monitoring and Reporting
View from the top:
13
Locating and Deploying Staff
Local telecommunications
infrastructure mostly inoperable
Pre-planned out-of-town contact numbers functioning
No single data base for locating people in shelters
Travel impacted by roads and gas shortages
Family issues had to be addressed
First 14 days:
14
Expanding Capabilities to Meet Long-Term Deployment
Duration of COOP means
more seats needed- Trailers acquired in Atlanta- USDA sites used in D.C.,
Kansas City, Roslyn, and Alexandria
- Customer sites in Birmingham and Fairfax leveraged
Secure connectivity and postal mail were two key infrastructure challengesto achieve
First 14 days:
15
First Job, First
Critical data center resources are recovered
Essential services are restored
Payroll is made timely while migrating 60,000 new payees (565,000 total)
Financial systems returned online
First 14 days:
16
Settling In for the Long Haul
Subscription service
requires sharing of resources during 6-week recovery period
Essential services require dedicated resources
Impact of Katrina implies 6 months not 6 weeks
Customer dedicated connectivity key issue
Decision made to buildout in “cold site”
First 14 days:
17
Focus on Service Delivery
Resuming all services
requires that all resources (human, technical, and other) be available
Service delivery focus isfor Agency customers, payees, vendors, and employees
Give employees the challenge and the toolsand stay out of their way
First 14 days:
18
New Orleans Critical to a Timely Return to Full Service
Returning to full service
required a full staff Deployed status had limited
seat availability, some security concerns, was costly, and came at a personal sacrifice to many
The New Orleans facility has the infrastructure to address business needs and support the additional population
Next 60 Days:
19
Reconstitution in New Orleans
Housing, family issues, telecommunications,postal services, and local infrastructure had to be addressed- Security- Logistics- Habitation necessities- Health care
Family & friends, FEMA trailers, and cruise ships address most housingneeds
Next 60 Days:
20
It’s Nice to Have Friends
USDA Secretary Johanns
and entire USDA family strongly supported COOP needs and the employees
Tremendous outreach by Federal colleagues, local Government, local organizations, and peoplein deployed locations
Food, clothing, education opportunities, friendship, and other assistance
Keys to success:
21
It’s Nice to Be Prepared
Managers and involved
staff built the plan Same managers and staff
practiced the plan Live and desktop drills
under varying scenarios honed the mental and technical readiness
We know our customers rely on our mission delivery in order to sustain theirs
Keys to success:
22
Lessons Learned – Governmentwide
Consolidated information
source needed to support decision-making
Central clearinghouse for needs and resources
Designation of essential services and associated rights is needed
Handbook to guide Federal Agencies through extraordinary authorities
Single coordination point for Federal efforts andissues
Learned:
23
Lessons Learned - NFC
Communications are
always the challenge; planned communications channels are a real aid
Having a well-drilled plan allows for the basics to be accomplished with little intervention and allows management to focus on exceptions and surprises
Learned:
24
Subscription service not
an ideal model for essential service provider
Administrative support a key aspect of support for long-term deployed staff
For long-term deployment, coordination of outreach efforts targeting employees becomes an essential task
Lessons Learned - NFCLearned:
25
Some organizations
benefit from close proximity with one another
Must revalidate plan assumptions, plan contents, and business requirements annually
You cannot assume infrastructure items will be there for you
Lessons Learned - NFCLearned:
26
Working to Mitigate Future Risks
Data Center equipment will
remain at subscription service location until new primary facility is selected
New AWS alternatives being explored
All business operations are being reassessed based on experiences
Lessons learned are being incorporated into revised DR and COOP plans
27
People, Planning, & Practice
People who understand &
believe in DR and COOP make it happen
Plans have to account for your threats, the business requirements for COOP, and the people side of things
Practice is the best teacher. Practice on a regular schedule (top priority) and against changing scenarios. Capture the valuable lessons from each exercise.
28
Epilog
Facing the most devastating
natural disaster to impact the United States, the employees of NFC set aside their personal concerns and focused on delivering for their customers.
Given such an event, the best came out of people, far and wide, and we are all better because of it.
29
One-Year Review of Key Indicators
Housing rehabilitation and demolition are well underway while the housing market tightens- Pace of demolitions has accelerated in the last six months- Number of permits issued for rehab has nearly doubled- Rent prices have increased by 39% and home sale prices
have spiked in suburban parishes
Public services and infrastructure remain thin and slow to rebound- Approximately half of the bus and streetcar routes running
with only 17% of buses in use- Gas service reaching 41%; electricity reaching 60% of pre-
Katrina customer base
30
One-Year Review of Key Indicators
Labor force in the region is 30% smaller than one year agowith slow growth over the last six months- Unemployment rate remains higher than pre-Katrina- The New Orleans metro area lost 190,000 workers- In the past six months, 3.4% more jobs but much of that may reflect the rise
in new job seekers- Unemployment rate is now 7.2%, higher than last August
Over $100 billion in federal aid dedicated to serving familiesand communities impacted by hurricanes Katrina, Rita,and Wilma
Number of displaced and unemployed workers remains high- Federal government approved approximately $109 billion in federal aid to the
Gulf Coast states- Half has been dedicated to emergency and longer-term housing- Estimated 278,000 workers are still displaced- 23% remain unemployed
31
32
33
34
35
36
37
38
39
40
Questions???
Please ask us questions important to you.
“Lagniappe” is a New Orleans’ term often used to indicate getting something extra in a
transaction. Our lagniappe for you today includes additional pictures of the post-Katrina
New Orleans area.
41
LagniappeCanal Street in CBD
42
LagniappeLower 9th Ward
43
LagniappeMold
44
LagniappeThe lake is on the other side of the building
LagniappeThe lake is on the other side of the building
45
LagniappeBoat came 3 miles inland
LagniappeBoat came 3 miles inland
46
LagniappeBefore
NATIONAL FINANCE CENTER
LagniappeBefore
47
LagniappeAfter
LagniappeAfter
48
LagniappeAnyone missing a hot tub?
LagniappeAnyone missing a hot tub?
49
LagniappeA quite country road
LagniappeA quiet country road
50
LagniappeMain bridge into Eastern New Orleans
LagniappeMain bridge into Eastern New Orleans
51
LagniappeA view from the elevated expressway
LagniappeA view from the elevated expressway
52
LagniappeBourbon Street – Never so clean!
LagniappeBourbon Street – Never so clean!
53
NSAA IT Conference