Upload
alexander-bruce
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Performance Based Studies Research Group
www.pbsrg.com
State of Oklahoma Risk State of Oklahoma Risk Management TrainingManagement Training
State of Oklahoma Strategic Plan
• Compete based on value (proven performance and expertise)
• Transfer risk and control to vendor
• Minimize client decision making, directives, and control
• Vendor writes majority of the contract
• Vendor manages contract
• Measurement, measurement, measurement
State Vendor Expectations
• Proactive and Accountable– Vendor has control– Focus on risk in the seams– Risk – Plans to minimize risk – impact – customer satisfaction – Performance measurement – uses dominate information– Performance information/risk identification disengages the
bureaucracy
• Win-Win– Vendor’s success is just as important as State’s success– Goal alignment
• “No contract” – “Vendor writes their own” – Vendor has control– Contract is a guide– If we have to go to the contract then everyone has already lost– Should never have to go the contract
Filter 1Registration:
Past Perf. Info.
Filter 2Scope
RAVA plan
Filter 4Best Value
Prioritization
Filter 5Pre-planning
Phase:WR and Scope
Filter 6Weekly
Report &Post-
Rating
Time
Filter 3Interviews
High
Low
2009 State of Oklahoma 2009 State of Oklahoma Consultant Selection and Risk Consultant Selection and Risk Management ProcessManagement Process
Aw
ard
Sh
ort
listCriteria:
RAVATech.PPIQual.
Criteria:InterviewRAVAPPIQual. Id
en
tify
BV
5
Final RatingFinal Rating
• After the project is complete, the client will evaluate the project.
• The Final Rating will be heavily impacted by: – Change orders– Project delays– Poor quality– Owner surprises– Complaints– Ability to submit accurate and timely weekly reports
• In Best-Value Procurement, the vendors goal is to get a high rating in order to stay competitive.
6
50% Modification50% Modification
NO CRITERIA Ven 1 Ven 2 Ven 3 Ven 4
1 Ability to manage the project cost 9.4 9.8 9.4 9.52 Ability to maintain project schedule 9.6 9.8 9.7 9.83 Quality of workmanship 10.0 9.8 9.7 9.94 Professionalism 9.6 9.8 9.7 9.75 Close out process 9.3 9.7 9.5 9.56 Ability to communicate 9.4 9.8 9.4 9.57 Ability to follow the users rules 9.7 9.8 9.4 9.88 Overall customer satisfaction 9.9 9.8 9.7 9.89 Total number of different jobs 19 18 18 20
10 Total number of different customers 19 17 18 17
NO CRITERIA Ven 1 Ven 2 Ven 3 Ven 4
1 Ability to manage the project cost 9.4 7.4 9.4 9.52 Ability to maintain project schedule 9.6 7.4 9.7 9.83 Quality of workmanship 10.0 7.4 9.7 9.94 Professionalism 9.6 7.4 9.7 9.75 Close out process 9.3 7.4 9.5 9.56 Ability to communicate 9.4 7.4 9.4 9.57 Ability to follow the users rules 9.7 7.4 9.4 9.88 Overall customer satisfaction 9.9 7.4 9.7 9.89 Total number of different jobs 19 18 18 20
10 Total number of different customers 19 17 18 17
NO CRITERIA Ven 2
1 Ability to manage the project cost 9.82 Ability to maintain project schedule 9.83 Quality of workmanship 9.84 Professionalism 9.85 Close out process 9.76 Ability to communicate 9.87 Ability to follow the users rules 9.88 Overall customer satisfaction 9.89 Total number of different jobs 1810 Total number of different customers 17
NO CRITERIA Ven 2Current Project
1 Ability to manage the project cost 9.8 5.02 Ability to maintain project schedule 9.8 5.03 Quality of workmanship 9.8 5.04 Professionalism 9.8 5.05 Close out process 9.7 5.06 Ability to communicate 9.8 5.07 Ability to follow the users rules 9.8 5.08 Overall customer satisfaction 9.8 5.09 Total number of different jobs 18
10 Total number of different customers 17
NO CRITERIA Ven 2Current Project
Ven 2
1 Ability to manage the project cost 9.8 5.0 7.42 Ability to maintain project schedule 9.8 5.0 7.43 Quality of workmanship 9.8 5.0 7.44 Professionalism 9.8 5.0 7.45 Close out process 9.7 5.0 7.46 Ability to communicate 9.8 5.0 7.47 Ability to follow the users rules 9.8 5.0 7.48 Overall customer satisfaction 9.8 5.0 7.49 Total number of different jobs 18 18
10 Total number of different customers 17 17
State of Oklahoma Risk Management Process
• Vendor takes into account all client requirements and constraints
• Vendor creates a baseline (cost, schedule, scope, quality)
• Vendor identifies all risks that they do not control that could impact baseline plan and shows a plan to minimize them.
• Vendor takes into account all client concerns and shows how they will be minimized.
• If any deviation occurs during the project it is documented and tracked on the weekly report.
• Vendor will be rated by the client on their deviation from the baseline.
The Weekly Report
• An excel document that tracks pre-identified risks/concerns and actual risks/concerns on a project.
– Pre-Identified Risks/Concerns: • Any concerns of the client at any point in the project• Any risks identified before the project begins
– Actual Risks/Concerns• Risks that are unforeseen• Risks that the contractor feels has a good probability of
occurring
• All information on the weekly report must be in “Dominant” fashion.
– “Dominant” means:• Simple, clear, concise, accurate…..• Easy to understand without a lot of information.
Weekly Report Schedule
• Inserted in the “Schedule&Budget” tab of the weekly
• The schedule should go from task order award till construction close-out (even if contractor was only awarded till the design/workplan phase)
• The schedule items should include the following types of activities:– Major milestones– High risk activities
• The schedule should be updated every week. – It should have incorporated impact of risks– It should accurately reflect the current status of the project
Reporting Pre-Identified Risks
• Before a project begins the vendor should have done the following:
– Talked with all client parties and documented their concerns– Identified any potential risks that the vendor does not control– Created a plan to minimize all concerns and risks– Placed all concerns/risks and minimization plans in the weekly
report in the RMP tab
• Throughout the course of a project if anyone has concerns or identifies more potential risks the vendor should add them to the RMP tab with plans to minimize them.
• If a pre-identified risk becomes a potential problem on the project it is then documented on the Risks tab as well.
Reporting Actual Risks
• Actual risks should be identified on the “RISKS” tab of the weekly report
• When an actual risk is identified the following should documented:
– How it became a risk– If it was not pre-identified, an explanation of why it was not– A dated history of actions performed to minimize the risk– Impact to cost and time– Owner rating of risk
• Until a risk is resolved the following must be updated:
– Estimated impact to time and cost– Actions performed to minimize the risk– Owner rating of the risk
• Every resolved risk with an impact, must clearly be linked to a modification on the “Schedule&Budget” tab.
Industry Structure
High
I. Price Based
II. Value Based
IV. Unstable Market
III. Negotiated-Bid
Specifications, standards and qualification based
Management & Inspection
Best Value (Performance and price measurements)
Quality control
Competition
Pe
rfo
rman
ce
Low
High
Owner selects vendor
Negotiates with vendor
Vendor performs
Best Value is best predictable
performance
Award is based on subjective minimum
which has to be enforced
High
Low
Perf
orm
an
ce
Owners
“The lowest possible quality
that I want”
Contractors
“The highest possible value that you will get”
Minimum
Problem with Priced Based Systems: proliferates decision making
High
Low
Perf
orm
an
ce
Maximum
Perf
orm
an
ce
High
Low
Ris
k
High
Low
Impact of Minimum StandardsBlind vs Visionary
Contractor 1Contractor 2Contractor 3Contractor 4
Contractor 1
Contractor 2
Contractor 3
Contractor 4
Perf
orm
an
ce
High
Low
Ris
k
High
Low
Decision making: what is the minimum standard, and do all contractors meet the minimum standards
Traditional PM vs. New PM Model
RiskRiskss
Control Don’t Control
Me & Them Us
RisksRisks
Control Don’t Control
Initial conditions
Final conditions
Accurate perception leads to less decision making and expectations
Time
Laws Laws
Best Value Selection
Contractor 1
Contractor 2
Contractor 3
Contractor 4
Perf
orm
an
ce
High
Low
Ris
k
High
Low
Contractor 1
Contractor 2
Contractor 3
Contractor 4
Perf
orm
an
ce
High
Low
Ris
k
High
Low
Industry performance and capability
Highly Trained
MediumTrained
Vendor XCustomers
OutsourcingOwner
PartneringOwner
PriceBased
MinimalExperience
Impact on Professionalism
Best Value Environment• More accountability • More expertise• More professionalism
Price Based Environment• Less accountability• Less expertise• Less professionalism• Less perceived “value added”• Movements to make designer
work for contractors due to lack of value added
High
I. Price Based
II. Value Based
IV. Unstable Market
III. Negotiated-Bid
Specifications, standards and qualification based
Management & Inspection
Best Value (Performance and price measurements)
Quality control
Competition
Pe
rfo
rma
nce
Low
High
Owner selects vendor
Negotiates with vendor
Vendor performs
Minimized decision making
Maximized decision making
Dominant information
• What does the most qualified, experienced people in a firm do?
• Get work, or do work?
• Why is this?
• It is more important in a destabilized industry to get work then to do work….
• Downside….what happens to our professionalism????
ASTM Standard Setting
ASTM Standard Setting
What Will Manufacturer ‘C’ Do?
CInitial Elongation % (break) D412 200% 100%Min.Initial Tensile Strength (mass Stress) D412 450psi 150psi Min.Final Elongation % Weathering D412 200% 100%Min.
Permeance E96. B 3.52.5 U.S. perms
MinWater Absorption (mass) D471 <1%
Delete requirement
Weathering (5000 hours) G53
No cracking No cracking
Adhesion C794 3 pli 2pli min.
Fungi Resistance G21 No growthDelete
requirement
Tear Resistance D624 30 lb/in 20 lb/inLow Temperature Flexibility D522 Passes Passes
Viscosity D2196115-130
KU35,000-
50,000cps
Volume Solids D2697 57 - 66% 57-80%
TEST PROPOSEDPHYSICAL PROPERTY
Standards motivate
manufacturers to lower quality
There is something wrong with the delivery of services…..
No one knows how bad the problem really is…..
Entire system is broken….
Requires more management….
Performance/value is decreasing….
Relationships are more important than results….
Management
….it becomes less important to be skilled, accountable, and able to minimize risk
As management, control, and direction become more important…..
Skill 1 Skill 2 Skill 3 Skill 4
“Manager’s Code” The movement of risk.....when making
decisions
Don’t Mess With It!
YES NO
YES
YOU IDIOT!
NO
Will it Blow UpIn Your Hands?
NO
Look The Other Way
Anyone ElseKnows? You’re SCREWED!
YESYES
NO
Hide It
Can You Blame Someone Else?
NO
NO PROBLEM!
Yes
Is It Working?
Did You Mess With It?
Impact on Designer’s Value and Professionalism
Best Value Environment• More accountability • More expertise• More professionalism• Dominantly clear that the
designer can add the most value
Price Based Environment• Less accountability• Less expertise• Less professionalism• Less perceived “value
added”• Movements to make
designer work for contractors due to lack of value added
High
I. Price Based
II. Value Based
IV. Unstable Market
III. Negotiated-Bid
Specifications, standards and qualification based
Management & Inspection
Best Value (Performance and price measurements)
Quality control
Competition
Pe
rfo
rma
nce
Low
High
Owner selects vendor
Negotiates with vendor
Vendor performs
Minimized decision making
Maximized decision making
What is the difference?Do designers spend more time in reacting to bureaucracy or doing a good technical job?
2
3
1
4
1
2
3
4
Traditional PM vs. New PM Model
RiskRiskss
Control Don’t Control
Me & Them Us
RisksRisks
Control Don’t Control
Evolution of Delivery Processes
Design-Bid-Design-Bid-Build Build (DBB)(DBB)
Design-Design-Build Build (DB)(DB)
ConstructioConstruction Manager n Manager
At Risk At Risk (CMAR)(CMAR)
Design BuildBid
Design
BuildBid
Design-BuildBid
Where is value added?????
Design-Bid-Design-Bid-Build Build (DBB)(DBB)
Design-Design-Build Build (DB)(DB)
ConstructioConstruction Manager n Manager
At Risk At Risk (CMAR)(CMAR)
Design BuildBid
Design
BuildBid
Design-BuildBid
Who should be the center of the Universe in construction? In design?
High PerformingVendor
Low PerformingVendor
ClientProfessional
Client
Outsourci
ng
Direction,
control, review
Design Intent
ScopeCost Est.
Baseline $
Re-Scopew/Modifications
Final Design
CMAR Best Practices
Contractor
Designer
0% 30% 60% 100%
1
2
3
4
5
Bonding capacity
Self Regulating Loop(Six Sigma DMAIC Generated)
Actions• Minimize data
flow• Minimize analysis• Minimize control Scope
Risk Assessment/Value Added Plan
Preplanning, Quality Control Plan
Measure again
50%
Identify value (PPI, RA, Interview)V
50%
Interview Key PersonnelPast PerformanceInformation
M
Requirements(DBB, DB, CMAR, DBO)
Efficient Service
M R
MM
R
R
R
= Minimize Risk
= Self Measurement
= Identify Value
M
R
V
M
M
M
M
Filter 1Registration:
Past Perf. Info.
Filter 2Scope
RAVA plan
Filter 4Best Value
Prioritization
Filter 5Pre-planning
Phase:WR and Scope
Filter 6Weekly
Report &Post-
Rating
Time
Filter 3Interviews
High
Low
2009 State of Oklahoma 2009 State of Oklahoma Consultant Selection ProcessConsultant Selection Process
Aw
ard
Sh
ort
listCriteria:
RAVATech.PPIQual.
Criteria:InterviewRAVAPPIQual. Id
en
tify
BV
Past Performance Surveys/ Past Past Performance Surveys/ Past Performance SheetPerformance Sheet
Design Firm
Past Client
State of Oklahoma
Mail survey back to Design Firm
Compiles Information
Email/Fax Survey
Call To Verify
Design Firm
compiles and sends surveys and past
performance sheet back to State
Past Performance InformationForm 254
• Design firms
• Critical sub elements
• Lead project managers
Initial conditions
Final conditions
Traditional Management
Time
Laws Laws
Risk is deviation fromexpected measurements
D
D M&C
Initial conditions
Final conditions
When do we determine the expectation so we can measure performance?
Time
Laws Laws
Risk is deviation fromexpected measurements
M
M M
M
Scope
• Will be rated in relative fashion
• Conservative plan on how the work will be done
• If other options are available, they will be considered along with the risk
• 30,000 foot level description
• Final design description
• Milestone schedule
Risk Assessment and Value Added Risk Assessment and Value Added Plan (RAVA)Plan (RAVA)
• The Risk Assessment and Value Added (RAVA) Plan consists of two piecesThe Risk Assessment and Value Added (RAVA) Plan consists of two pieces– The Risk AssessmentThe Risk Assessment– The Value AddedThe Value Added
• The Risk Assessment section is used to identify high performing vendors The Risk Assessment section is used to identify high performing vendors that can:that can:– Identify and minimize risk Identify and minimize risk beforebefore the service has started the service has started – Deliver plan to minimize risk during the life of the service.Deliver plan to minimize risk during the life of the service.
• The RA Plan should clearly address the following items:The RA Plan should clearly address the following items:
– List and prioritize major risk itemsList and prioritize major risk items (areas that may cause the service (areas that may cause the service not to meet the expectations of the Government).not to meet the expectations of the Government).
– Each vendor should Each vendor should focus on risks it does not controlfocus on risks it does not control and and
– Explain how the vendor will Explain how the vendor will minimize the riskminimize the risk..
Risk Assessment Plan Steps
• What events cause risk
• How will the design firm minimize the risk from happening
• If the risk happens, what will the designer do
• Transfers control and the opportunity to minimize risk to the design firm
RAVA Plan Rated BlindRAVA Plan Rated Blind
• The RAVA Plan must NOT exceed The RAVA Plan must NOT exceed 22 pages. pages.
• The RAVA Plan must not have The RAVA Plan must not have ANY NAMESANY NAMES (such as (such as vendor or manufacturer names, personnel names, vendor or manufacturer names, personnel names, project names, product names, or company letterhead) project names, product names, or company letterhead)
• Do NOT include brochures, marketing information, or Do NOT include brochures, marketing information, or product names!product names!
• The RAVA Plans will be evaluated “blind” in order to The RAVA Plans will be evaluated “blind” in order to minimize any bias. minimize any bias.
Value Added DifferentiationValue Added Differentiation
• Answer the questions: “What value do I bring that differentiates Answer the questions: “What value do I bring that differentiates me from my competitors?”me from my competitors?”
• The Value Added Differentiation section of the RAVA Plan is to The Value Added Differentiation section of the RAVA Plan is to allow a vendor to:allow a vendor to:– Show how it will add value (that its competitors cannot)Show how it will add value (that its competitors cannot)– The impact of that value in simple, provable termsThe impact of that value in simple, provable terms– How the value added will be measured – must have an How the value added will be measured – must have an
impact on dollars, time, and/or satisfaction of the impact on dollars, time, and/or satisfaction of the GovernmentGovernment
• Marketing material is worthless and will negatively impact Marketing material is worthless and will negatively impact scoresscores
• Value Added is where vendors provide “great ideas” Value Added is where vendors provide “great ideas”
How The Submittal Process Works
Submittal
Evaluation Members
Proposal Form(1 page)
Proposal Form & Other Documentation
Proposal Form(1 page)
RAVA and Tech. Prop.(2 Pages Each)
Average Score
ContractingOfficer
ContractingOfficer
InterviewsInterviews
• The following individual will be interviewedThe following individual will be interviewed– Project Manager/Lead DesignerProject Manager/Lead Designer
• All interviews will be individual All interviews will be individual
• The same individuals used for the past performance survey will The same individuals used for the past performance survey will be the same individuals interviewed and will be the same be the same individuals interviewed and will be the same individuals that will be assigned to the projectindividuals that will be assigned to the project
• No bait and switchNo bait and switch
Sample Interview QuestionsSample Interview Questions
1.1. Why were you selected for this project?Why were you selected for this project?
2.2. How many similar projects have you worked on?How many similar projects have you worked on?
3.3. What is different about this projects from other clients that you have worked for?What is different about this projects from other clients that you have worked for?
4.4. What are you doing differently on this project than you have done before?What are you doing differently on this project than you have done before?
5.5. Identify, prioritize, and how you will minimize the risks of this project.Identify, prioritize, and how you will minimize the risks of this project.
6.6. What risks don’t you control? How will you minimize those risks?What risks don’t you control? How will you minimize those risks?
7.7. What are your personal goals on this project?What are your personal goals on this project?
8.8. How are you going to measure your performance?How are you going to measure your performance?
9.9. What value do you bring to the project in terms of differences based on dollars, What value do you bring to the project in terms of differences based on dollars, quality, or time?quality, or time?
10.10. How will you measure performance for this project? How will you measure performance for this project?
Best Value Selection
• Past Performance
• Risk Assessment (don’t control) / Value Added
• Scope/Technical Proposal (2 Page proposal/milestone schedule)
• Qualifications
• Interview
Filter 1Registration:
Past Perf. Info.
Filter 2Selection:RAVA plan
Filter 4Best Value
Prioritization
Filter 5Pre-planning
Phase:WR and Scope
Filter 6Weekly
Report &Post-
Rating
Time
Filter 3Interviews
High
Low
Rating Scheme (1-10, 1(not qualified, Rating Scheme (1-10, 1(not qualified, 5(can’t differentiate/don’t know), 10 5(can’t differentiate/don’t know), 10
(dominantly better(dominantly better))
Aw
ard
Sh
ort
listCriteria:
RAVATech.PPIQual.
Criteria:InterviewRAVAPPIQual. Id
en
tify
BV
Unforeseen Risks
PERFORMANCE SUMMARY• Vendor Performance• Client Performance• Individual Performance• Project Performance
QUALITY ASSURANCE• Checklist of Risks• Sign and Date
Risk Management Plan Sheet (RMP)
• Pre-Identified Risk• Risk Minimization
Risk Sheet (Actual)• Risk that actually occurs• Unforeseen Risks
Weekly Report
What are the measurements of a Designer
• Scope and cost
• Plan with minimal risk
• Plan to minimize risk
• Knowing this with minimal effort
• Has the designer done this before
• Do they know what they don’t know on a future project
• Can they clearly articulate their vision
Traditional PM Model: Influence and Control (micromanagement)
• Reactive
• Focus on changing people
• Followers are the constraint
• Requires maximum amount of resources
• Relieves management from accountability of nonperformance
• Inefficient and nonperforming
New PM model: Leadership Model(measurements)
• Proactive
• Alignment (measurement)
• Requires Understanding (acceptance of measurements)
• Leader/manager and system is the constraint (blindness)
• Focus is on setting up a quality controlled system based on performance measurements, best value requirements, and dominant information
• Efficient (measurement)
Filter 1Past
PerformanceInformation
Filter 2Proposal & Risk / Value
Plan
Filter 4Prioritize (Identify
Best Value)
Filter 5Pre-Award
Phase (Pre-Plan)
Filter 6Weekly
Report &Post-
Rating
Time
Qualit
y o
f V
endors
Filter 3Interview
Aw
ard
High
Low
State of OklahomaState of OklahomaBest Value Process for Designers and ContractorsBest Value Process for Designers and Contractors
Differences in new system
• Provide past performance information (PPI) from references
• PPI on organization and critical components
• Submittals (scope and RAVA plan) will be two pages each– One additional page for milestone schedule
• Interview will be with the project manager– How do you know you can do it– How are you doing it– How are you taking control of the project and minimizing risk
and decision making of client– How did you pick your engineering subvendors
Requirements on Competing Firms
• More or less time invested on a project?
• More or less detail?
• Risk they control or do not control?
• Dominant differential or technical qualifications?
• Risk minimization done when?
• More risky approach or conservative control?
Designer should separate the requirement from the risk
• Propose the vision that they can do with minimial risk
• Propose to immediately analyze any “potentially risky options” in the beginning of the design contract
• Only pursue “risky options” if there is dominant information that justifies the “perceived risk”
• Cost should be based on “vision” and “requirements”
• Makes all other parties accountable and gives control of the design back to the design professional
Delivery of Design Services
• Control, manage, direct, and inspect
• Increase the flow of information
• Inefficient, ineffective
• Maximizes technical issues
Initial conditions
Final condition
s
Time
Laws Laws
Initial conditions
Final condition
s
Time
Laws Laws
RisRisksks
ControlDon’t
Control
Me & Them
Us
RiRiskskss
Control
Don’t Contro
l
• Align• Transfer control to the contractor• Preplanning• Quality Control• Measure• Minimize flow of information•Minimizes technical issues
Status Quo
New Design Delivery Model
M
M
M
M
MX