2
Breakout Session # 1411
Robert F. Watts, Senior Consultant, ESI International
Date: April 16
Time: 2:40 – 3:40 PM
Buying Results Through Service Level Agreements: Best Practices in Performance- Based Acquisition
3
Performance Based Acquisition -- Still a Challenge
• Recent Acquisition Advisory Panel study determined that more than half the contracts designated as PBA in the Federal Procurement Data System-Next Generation were not performance-based
4
It’s All About Mindset…Mindset Characteristics
Process
Focused on process and activities -- how requirements need to be accomplished Often lacks confidence in contractor’s quality control practices Prefers to micromanage the company's technical processes
Personnel
Focused on personnel - who needs to accomplish requirements Often lacks confidence in contractor’s human resource and management practices Prefers to micromanage the company's personnel
Performance
Focused on results and mission accomplishment - what needs to be accomplished Has confidence in contractor’s technical, quality, human resources, and
management practices Prefers to manage outcomes and customer satisfaction and not to micromanage
the company’s processes and personnel
5
PBAIncentive
Plan
Quality Assurance
Surveillance Plan
PerformanceWork Statement
or SOO
PerformanceRequirements
Summary
6
What is a Service Level Agreement (SLA)?
• A performance-based technique where a written agreement is established between the customer and service provider that defines key service objectives, metrics, and acceptable quality levels (AQLs)– Primarily used in information technology (IT)
procurements – Expanding into other sustainment-type services
7
Example SLA OutlineService Level Agreement
(Title of Agreement)
1.0 Introduction
1.1 Statement of Need
1.2 Background
2.0 Service Objectives
4.0 Service Level Metrics/AQLs
3.0 Applicable Directives
3.1 Compliance
3.2 Reference
4.0 Constraints and Assumptions
5.0 Data Deliverables
6.0 Government Furnished Property (GFP)
Equivalent to PWS/SOO
and Performance Requirements
Summary
Equivalent to PWS/SOO
and Performance Requirements
Summary
8
Service Objectives
Service Desk
Provide, implement, operate and maintain a fully-functional, reliable Service Desk…
….for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime.
Do what?
To accomplish what?
Example
9
Service Level Metrics/AQLs
Title: Service Desk
Objective: Provide, implement, operate and maintain a fully-functional, reliable Service Desk for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime.
Metrics:
Response: Percentage of inbound calls answered within 20 seconds.
First Call Resolution: Percentage of inbound calls handled to resolution at Tier-1.
Quality: Percentage of customers surveyed expressing a “satisfactory” or better quality of service rating.
Acceptable Quality Levels:Response: 80%
First Call Resolution: 70%
Quality: 80%
10
Service Level Metrics: Possible Areas to Measure
Network Performance Hosting Application User Support
Availability: Percent of time the network is available to users (percentage)
Server Availability: Percent of time the server is available to users (percentage)
User Interactions: Percent of successful user interactions (downloads, data requests, and so forth)
Help Desk: Response time and accuracy in resolving trouble tickets
Throughput: Capacity of the backbone connections within the network’s core (bits/sec)
Administration of Servers: Acceptable response times for restoring failed servers (period of time)
Data Transfer: Acceptable time lapse between user request for data and moment data screen appears (seconds)
User Training:
Quality of user training, such as classroom, online, and so forth (customer survey)
Connection: Acceptable data losses and data latency, plus bandwidth provisioning (percent of packet delivery rates)
Data Backup and Storage: Scheduled data backups that will be conducted (percent of completion)
Ease of Use: Quality of customer’s experience in using the application (customer survey)
Needs Assessment: Quality and thoroughness of program for assessing ongoing user needs
Security: Protection of applications, data, and services in transit (levels of encryption, number of breaches, and so forth)
11
What is the QASP?
• An internal plan used by the customer that describes the performance monitoring methods they will use to evaluate the contractor’s progress and success in meeting service level metrics and AQLs contained in the SLA
12
Some Techniques to Monitor Contractor Performance
Example PerformanceMonitoringApproaches
13
Contractor Monitoring (Example)
Title: Service Desk
Objective: Provide, implement, operate and maintain a fully-functional, reliable Service Desk for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime.
Metrics:
Response: Percentage of inbound calls answered within 20 seconds.
First Call Resolution: Percentage of inbound calls handled to resolution at Tier-1.
Quality: Percentage of customers surveyed expressing a “satisfactory” or better quality of service rating.
Acceptable Quality Levels:Response: 80%
First Call Resolution: 70%
Quality: 80%
Method of Government Monitoring:
Response: Weekly audit of Service Desk reports
First Call Resolution: Weekly audit of Service Desk reports
Quality: Automated email customer satisfaction survey
Monitoring Officials:
COTR; Performance Monitors
14
Automated Email Survey How satisfied are you that the Service Desk responded quickly to your problem?
Very Dissatisfied Dissatisfied Satisfied Very Satisfied
O O O OComments: How satisfied are you that the Service Desk was professional and “customer-oriented” in resolving your problem?
Very Dissatisfied Dissatisfied Satisfied Very Satisfied
O O O OComments: How satisfied are you in the overall quality of the Service Desk’s final resolution of your problem?
Very Dissatisfied Dissatisfied Satisfied Very Satisfied
O O O OComments:
15
Incentive Plan
• Rewards and/or penalties assessed against the contractor based on their success in achieving service level metrics and AQLs in the SLA
16
Possible Incentive StructuresAward Fee
Pool of dollars awarded at specified periods to contractor based on performance of SLA
Award TermAdditional contract terms awarded to contractor based on performance of SLA
Past Performance RatingsFormally documented ratings of contractor performance of SLA
Milestone PaymentsLinkage of contractor payments to successful completion of milestones contained in SLA
OptionsLinkage of follow-on options to contractor based on successful performance of SLA
Invoice AdjustmentsAdjustments to contractor billings based on performance of SLA
17
Some Best Practices in Developing SLAs
• Keep it simple! – Look at the overall mission and develop a small set of key service level
metrics—find things that are measurable, meaningful and have real impact on the bottom line
• Stay focused on mission success and customer satisfaction– Leave measuring of activities and processes to the contractor
• Manage AQL expectations – be realistic– The closer to 100% perfection you require, then the higher the price --
be practical and realistic
• Performance standards cannot exist in a vacuum; there must be a forum for discussion and negotiation
18
Case Study
Chicago adjusts on the fly to make its contract perform