Clint Britt _ Six Sigma Project _ MFP_ June2016

0 | P A G E

COMCAST - MONTHLY FINANCIAL PACKAGE PROJECT

Six Sigma Black

Belt Project May 31, 2016

Clint Britt

9844 Titan Ct, Littleton, CO 80125

p. 303.513.6348

[email protected] Comcast.com

1 | P A G E


Table of Contents

I. Project Charter ....................................................... 2

II. DEFINE ................................................................ 3

III. MEASURE: ........................................................... 8

IV. ANALYZE ............................................................ 19

V. IMPROVE: .......................................................... 25

VI. CONTROL: ......................................................... 36

COMCAST MFP INTEGRITY_ PROCESS IMPROVEMENT VIA SIX SIGMA - MAY 31, 2016 2


Project Charter

Project Name: Month Financial Package (MFP) Data Integrity and Over-hall

The purpose of this charter is to establish a Process Improvement Team to rapidly address senior leadership concern surrounding the

accuracy and methodology of the Monthly Financial Package (MFP). The team will be led by Clint Britt, Finance Director with Comcast.

The team is charged with analyzing the existing monthly performance reports, examining source data, identifying inconsistencies in

reported information, developing standard control procedures, and implementing improvements to ensure consistent and reliable reporting

for all projects.

Project Sponsor: CIO, and Executive Management Office

Project Manager: Clint Britt

Email Address: [email protected]

Phone Number: 303.513.6348

Expected Start Date: January 2016

Expected Completion Date: June 2016

Expected Savings: $795,324

Expect Costs: $222,000

Green Belt Assigned: Matt Walters

Black Belt Assigned: Brian Rector

Project Goals: Design and develop a complete set of solutions to address root causes behind the impairment of actual cost data

in the Project Management System. At the highest level, this involves two data streams: Timesheets for Labor Costs and Expense Entries

for all other costs. Determine the causes behind timesheets not processed and / or processed incorrectly. Determine the causes behind

expense entries not processed and / or processed incorrectly. Determine the full extent of the problem through data analysis, interviews,

and other tests. Develop solutions for improving the processes and monitor the results of the implemented solutions.

Process Problem: Performance information related to IT Projects is not reliable and accurate as reported in the Monthly

Performance Reports. This information is extracted from the Project Management System. Processes are poorly defined, inconsistent, and

prone to high error rates each month. To address concerns with senior leadership regarding the integrity of cost data in Monthly

Performance Reports. This project will attempt to identify root causes behind distortions with actual project cost data in the Project

Management System and implement solutions to fix broken processes in capturing and reporting costs.

mailto:[email protected]



Scope: This project is limited to only those projects that are considered "developmental" since these projects must report Earned

Value Management and these projects currently rely on the Project Management System for cost data. Projects not using the Project

Management System will not be reviewed. Additionally, this project will not develop detail system requirements for the Project

Management System. The focus will be on the process itself and how the process relates to source systems for data.

Schedule

Key Milestone Start Complete

Form Project Team / Preliminary Review / Scope

01/23/16

Finalize Project Plan / Charter / Kick Off

02/12/16

Define Phase 02/28/16

Measurement Phase 03/18/16

Analysis Phase 4/19/16

Improvement Phase 5/16/16

Control Phase 6/07/16

Project Summary Report and Close Out

6/5/16

Process Owner. The Chief Information Officer owns the overall process over cost performance reporting for IT investments.

Each Project Manager must make sure they follow a set of procedures for capturing and reporting actual costs correctly. The Project

Management Office provides oversight and support for the processes.

Key Stakeholders. All personnel assigned to IT Developmental Projects, including Project Managers, Project Analyst, Project

Planners, Project Schedulers, and Budget Managers. All personnel who provide leadership support above the project level, including the

Chief Information Officer, the CIO Staff, Directors and Senior Managers within the IT Department and outside departments or agencies

that have an interest in the cost performance of IT investments within the agency.

Authorized Resources: Clint Britt, Brian Rector, Matt Walters, and Erik Jensen. In addition to the four core team members,

System Administrative support has been budgeted at one-full time FTE for the life cycle of the project. Black Belt Support will come from

the CIO Council. Project Management Office staff will provide some administrative help. Projects that are reviewed may get tasked to

help with data gathering and collection. At least two of the core team members will need network access to the Project Management

System. The project will also need programming support to extract data from two other systems: Payroll and Financial Reporting.

Sponsor Approval: Prabuling Patel - Executive Vice President

DEFINE

CTQC Background Information

Critical to Quality (CTQ) or Critical to Quality Characteristics (CTQC): Specific factors or attributes that are associated with your products,

processes, and services that customers consider extremely important. This is often identified by listening to your customers through



surveys, interviews, complaints, and other ways for fully understanding the requirements of your customers. Six Sigma professionals refer

to this as Voice of the Customer or VOC. Once you have captured VOC, you can then translate this information into CTQ's.

Translation of CTQ to Operational Definitions: This project did not rely heavily on Comcast’s "Voice of the Customer" through surveys, focus groups, and other techniques since the

customer audience (project personnel) did not provide much insights into the sources of the problems. Therefore, it was considered

necessary to create a CTQ Diagram that drilled down to the CTQ related sources.

Once each CTQ was identified per the CTQ Diagram, operational definitions were established to help indicate what is acceptable

performance. Operating definitions (acceptable performance) has been expressed in terms of three dimensions:

1. Performance Standard - Set a threshold or benchmark for measuring if performance is acceptable or not. 2. Measurement Test - Describe a procedure or task that can be performed to determine if the actual transactional data is actually

meeting the Performance Standard.

3. Measurement Results - What is the output of the measurement test?

Critical to Quality (CTQ) Diagram

Accurate Cost

Amounts

Accurate

Labor Costs

Accurate

Expenses

Accurate

Labor Rates

Accurate

Labor Hours

Vendor Invoices

Employee Expenses

Credit Card Charges

Base Rate

Fringe Benefits

Manager Approval

Charge Code

Full AccountabilityContracting Officer

Approval

Correct CodingManager Approval

Correct Coding

Full Accountability Manager Approval

Correct Coding

Timely Base Rates

Accurate Base Rates

Timely Fringe Benefits Rates

Accurate Fringe Benefits Rates

Critical to Quality Characteristics (CTQC)

CTQ per Diagram Performance Standard

Measurement Test Measurement Results Continuous or Discrete

Preliminary Results

Timely Base Rates

Updated Listing provided every 6 months

All System Administrators confirm that updates received after 6 months

Refer to Sample Procedures and Attribute Measurement System Analysis

Discrete

Working

Accurate Base Rates

Actual Base Rate of Pay for Employee is within 1% accurate of the source system

Comparison of sample data between source system and Project Management System


Discrete

U T D



Timely Fringe Benefits Rates

Updated Listing provided every 6 months

All System Administrators confirm that updates received after 6 months


Discrete

Working

Accurate Fringe Benefits Rates

Actual Fringe Benefits Rate of Pay is within 2% accurate of the source system

Comparison of sample data between source system and Project Management System


Discrete

Broken

Manager Approval of Labor Hours

All weekly timesheets approved - 99% approval processing rate

Comparison of sample timesheets at selected cut-off dates within Project Management System


Continuous

Broken

Correct Coding of Labor Hours

All weekly timesheets coded and entered correctly - 99% accuracy rate

Analyze sample timesheets for accuracy within the Project Management System


Continuous

Broken

Completeness of Labor Hours

All weekly hours have been accounted for and entered - 99% accuracy rate

Analyze sample timesheets for accuracy within the Project Management System


Continuous

Broken

Officer Approval of Vendor Invoices

All monthly vendor invoices have been approved - 95% approval rate

Analyze sample vendor invoices and test for approval in processing


Continuous

Broken

Correct Coding of Vendor Invoices

All monthly vendor invoices have been coded and entered correctly - 99% accuracy rate

Analyze sample vendor invoices and compare to expense entries in Project Management System


Continuous

Broken

Manager Approval of Employee Expenses

All monthly employee expense statements approved - 95% approval rate

Analyze sample expense statements and test for approval in processing

Do Not Measure - not material to costs per Pareto

Continuous

Correct Coding of Employee Expenses

All monthly employee expense statements coded and entered correctly - 99% accuracy rate

Analyze sample expense statements and test accuracy in the Project Management System


Continuous

Completeness of Employee Expenses

All monthly employee expense statements accounted for - 99% accuracy rate

Analyze sample expense statements and test accuracy in the Project Management System


Continuous

Manager Approval of Credit Card Expenses

All procurement credit card purchases approved and authorized - 99% approval rate

Analyze sample credit card purchases and test for approval in processing

Analyze sample credit card purchases and test for approval in processing

Continuous

Correct Coding of Credit Card Expenses

All procurement credit card purchases coded and entered correctly - 99% accuracy rate

Analyze sample credit card purchases and test for accuracy in the Project Management System


Continuous



SIPOC DIAGRAM - High Level

Functional Flow Diagram:

Functional Flow Chart No. 1: Labor Costs

Payroll Dept Project Analyst Project Team Project Manager

Run Payroll

Extracts

Payroll Extract

Report

Receive Request

Submit

Request

for Pay

Rates

Enter

Pay

Rates

Pull

Active

Records

Updated

Rate Table

Project

Management

System

Submit

Timesheets

Approves

Timesheets

Labor Cost

ReportsMonthly

Performance

Reviews

Employees Employee Setup Data Active Employee Record Project M anager and Team

Contractors Contractor Setup Data Active Contractor Record Project M anager and Team

Employees & Contractors Planning M eeting Project Schedule Project M anager and Team

Contracting Officer Statement of Work Project Schedule Project M anager and Team

Payroll Department Employee Pay Rates Rate Table System Administrator

Contractors Contractor Pay Rates Rate Table System Administrator

Employees Timesheets Labor Cost Report Project M anager and Team

Contractors (Time & M aterials) Timesheets Labor Cost Report Project M anager and Team

Contractors (Fixed Priced) Invoices Contractor Cost Report Project M anager and Team

Project M anagement System Timesheet & Invoices M onthly Performance Reports CIO, Program M anagers, VP Finance

► ► ► ►Supplier Input Process

Setup Resources

Assign Activities to Resources

Assign Rates to Resources

Enter Time Sheets

Enter Vendor Invoices

Summarize and Report Costs

Output Customer



Define Milestone Review with sponsor:

What problem is this project trying to solve? Processes related to the capturing and reporting of actual costs within the Project

Management System (PMS). These costs are not considered reliable given the current processes in place. Need to rapidly mature these

processes so that senior leadership and project managers can properly evaluate cost performance.

What is the scope of this project? Has it changed since the Project Charter was issued? All IT investments that are considered

developmental must participate in using Earned Value Management and report the results in the Monthly Performance Reviews. These

projects define the scope of this project; i.e. focus on the procedures these projects follow in capturing their costs.

No – the project scope has not changed and is not expected to change. The Project Charter is fairly specific as to scope.

What are some of the CTQ’s for this project? There are five inputs that are critical to quality: Timesheets depend upon two critical

inputs: Hours and Rates. Expense Entries depend upon three critical inputs: Vendor Invoices, Employee Expenses and Procurement

Credit Card entries. Each of these inputs can represent a CTQ, such as accurate base rates for timesheets or accounting for all hours each

week for timesheets.

Do you have some idea on what our current performance or defect rate is? Not yet, we have to sample the data to fully understand

how high the error rate is. However, our discussions with System Administrators indicates that most projects are not fully compliant with

entering all timesheets. This problem also exists with expense entries; i.e. not all vendor invoices are entered. So we suspect that the error

rate is extremely high.

What are some of the work products during this phase of the project? Thought Process Map – Outline of critical questions that need

to be answered and possible techniques for analyzing the problems. Stakeholder Analysis – How to engage each stakeholder so that this

project is successful. SIPOC Diagram – Map the overall process and input streams Critical to Quality – Define what is critical to the

quality of cost data Operating Definitions – A basic level of performance, what has to be measured, and the measurement results (to be

determined in the next phase)

What quality control reviews were conducted to ensure work products were acceptable? The project team submitted all work

products to the PMO Staff for review and approval. We also sought out guidance from the CIO Leadership Council members; especially

those who are Six Sigma Black Belts since this project is following the DMAIC Six Sigma approach.

What are some of the process areas this project will improve? Timesheet processing – making sure a control procedure is in place to

make sure timesheets are getting processed. Vendor invoices – making sure all projects have a control procedure to properly capture

vendor invoices.

When can we expect to see these improvements implemented? The current plans are to start implementing solutions in June 2016.

Since training and SOP documents will have to be distributed, you probably will not notice improvements until mid year 2017.

Is this project on schedule according to the project plan? The project is on schedule overall, but is somewhat behind in the Define

phase. This delay is due to collecting data to feed the next phase – analysis. Once we get a good sample of data, we can move quickly. The

remaining phases are expected to be on schedule since the analysis is fairly straight forward and the solutions are fairly straight forward.

The implementation and monitoring or control phases of the project can get passed over to the Project Management Office (PMO) once

the project nears completion. We will setup the control plan and implementation plan and transition these over to the PMO.



MEASURE:

Data Collection Procedures

A. Purpose – In order to ensure high quality data in this project, it will be necessary to follow a standard set of procedures. The goal is to make sure that sufficient detail cost data is captured to gain a solid understanding of root cause issues that are creating distortions in the Project Management System. Since the Project Management System is a stand-alone application, it does not represent the primary source for much of the data. Where practical, the sampling of data should “go around” the Project Management System and go directly to the related sources of the cost data.

1. Data Needed – All actual costs data incurred on IT developmental projects. 2. Data Usage – Compare this data against the actual costs that are getting reporting in the Project Management System.

Determine the accuracy level and where major errors are found, develop and implement solutions to reduce the errors. B. Types of Data

1. Labor Costs – Includes all employees of the agency that charge time directly to an IT investment. 2. Expenses – Includes all non-direct labor costs associated with an IT investment. Expense entries include:

i. Voucher Payables – Contract personnel, suppliers, and other vendors ii. Employee Expenses – Employees submit expense statements for reimbursements due to travel and training iii. Credit Card Expenses – Various projects may have procurement credit cards which allow small purchases for

supplies, food, and other items the project may need. C. Source Systems

1. Financial Reporting System (General Ledger) – All expenses will be sampled from this system since this represents the source system for payment of all expense entries.

2. Project Management System – All direct labor costs are entered directly into the Project Management System. There is no other source system involved with labor costs on a project by project basis. Additionally, the Financial Reporting System does not capture any detail labor cost records and access to the direct payroll files is restricted to System Administrative personnel.

D. Sampling Rules to Follow 1. Where practical, select all developmental projects in the IT portfolio. As of December 2016, the complete listing of

developmental project numbers is: 34332, 32221, 54443, 33443, 19834. 2. Since most cost data is captured and processed on a monthly basis, one-full month should be selected for data analysis.

The period January 2015 will be considered the baseline period for sampling data since this period falls in the current annual reporting period and all source documents should be fully filed and processed as of the review date.

i. Automated Extracts for Labor Costs – System Administrators will be used to pull labor costs from the Project Management System. They can easily design custom reports to extract and test data.

ii. Automated Extracts for Expenses – Expenses will be extracted directly from the Financial Reporting System using a query program that selects all projects listed above. A formal system request is required for this extract.

3. Since timesheets are processed once every two weeks, certain measurements must be taken during a “live” two week period when timesheets are processed. This would include:

i. Testing to see how many timesheets were submitted on time within the Project Management System. ii. Testing to see how many timesheets were submitted without errors, such as accounting for all hours and using

the correct charge codes. 4. Standard Check Sheets should be used where applicable for capturing the measurement data. A set of Check Sheets are

listed below:

Measurement Check Sheet E.1 – Base Pay Rates within 1%

Appraiser Name: * Period Covered:

Observation Pay Rate per PMS Pay Rate per Adm Source

% Variance

Is Variance more than 1%?

1

2

3

4

5

6

7



8

9

10

11

12

13

14

15

16

17

18

19

20

Comments and Notes:

* period covered must match-up for a correct comparison

Measurement Check Sheet E.2 – Fringe Benefit Rates within 1%


Observation Fringe Benefit Rate per PMS

Fringe Benefit Rate per Adm Source

% Variance

Is Variance more than 1%?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Comments and Notes:

* period covered must match-up for a correct comparison

Measurement Check Sheet E.3/E.4 – Timesheet Processing


Employee Employee Submitted TS?

If Yes, was TS complete

If Yes, did Time Sheet (TS) use correct codes?

Foot Note*

1

2

3

4



5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Comments and Notes:

* attach separate sheet of footnotes to describe coding errors where applicable.

Measurement Check Sheet E.5 – Vendor Invoice Processing Up to 10 Invoices are Sampled over entire Jan 2007 Extract


Invoice Entry Was this entry posted?

If Yes, is the amount correct?

If No, is the amount off by more than 1%?

1

2

3

4

5

6

7

8

9

10

Comments and Notes:

* period covered should be January 2007 in both systems – Financial Reporting System and Project Management System. The source documents may relate to past service periods since invoices are processed approximately one month late.

E. Operational Definitions – Seven operating definitions will be measured with the sample data: 1. Base Rates of Pay used in the Project Management System are within 1% accuracy of the source pay rates that are found

in the Monthly Payroll files as provided by System Administrators. 2. Fringe Benefit Rates used in the Project Management System are within 1% accuracy of the source fringe benefit rates

that are found in the Monthly Payroll files as provided by System Administrators. 3. Timesheets have been submitted in a timely manner by all personnel assigned to the IT Project. Testing of this sample

data may require that you test during the two week “live” period when people are required to submitted timesheets. 4. Timesheets have been submitted in an accurate manner:

i. Account for all hours during the two week period ii. Hours coded to appropriate work codes for the two week period

5. Vendor Invoices have been properly posted to the Project Management System: i. Timely posting of vendor invoices to the appropriate reporting month and year. ii. Amounts posted are within 1% of the total invoice amount per the vendor invoice documents



F. Attribute Measurement System Analysis – The operating definitions that get measured must be validated through Measurement System Analysis or MSA. Validation will be based on two factors:

1. Repeatability – The measurement observations by the observer or appraiser are the same with each measurement. 2. Reproducibility – The measurement observations across different observers or appraisers is the same with each

measurement.

A 95% confidence level be used for validation. The measurement attributes are gaged as “Yes or No” per the operating

definitions defined above. Two trials are made per each appraiser to gage repeatability. Three appraisers will be used to gage

reproducibility: Matt Walters, Brian Rector, Erik Jensen. Once everything has been measured and validated, a baseline level of

performance will be established to facilitate measurement going forward using Statistical Process Control or SPC.

Preliminary Data Analysis: Components for Pareto Chart:

AT - Agency Transfers

CC - Credit Cards

EE - Employee Expenses

JV - Journal Vouchers

VP - Voucher Payables

Sum of Amount Trans Type

Project ID AT CC EE JV VP Grand Total

D001139HT03 1,373 1,374 381,174 383,921

D001196QC02 19,546 1,540 50,000 1,159,586 1,230,672

D001893SG01 23,525 23,525

D002743LB02 16,745 53,688 21,547 1,819,912 1,911,891

D002784NC01 147,913 147,913

D002784NC02 499 8,159 5,609 347,507 361,775

D002889YL02 8,142 1,443 1,715 11,300

D002965YL02 2,789 1,660 4,449

D002977BA01 10,511 2,703 13,214

D002981BR02 3,739 2,108 12,250 18,098

D003671MT03 6,532 1,560 69,340 77,432

D006122YV02 1,895 3,736 5,631

D007566MJ03 778 1,204 1,982

D008632JS02 32,656 32,656

D008763VM01 3,957 1,229 1,253,626 1,258,811

D009441GS02 7,160 878 1,013,290 1,021,327

D018322HE01 1,204 250,734 251,938

M005912LQ02 87 1,638 1,134 150,735 153,594

M005912LQ03 6,810 1,756 237,732 246,299

Grand Total 17,331 160,669 49,056 50,000 6,879,372 7,156,428



Direct Labor Costs:

Project ID Grand Total

D001139HT03 $ 167,515

D001196QC02 $ 3,244,101

D001893SG01 $ 347,566

D002743LB02 $ 488,961

D002784NC01 $ 290,077

D002784NC02 $ 303,113

D002889YL02 $ 88,164

D002965YL02 $ 13,642

D002977BA01 $ 41,672

D002981BR02 $ 264,581

D003671MT03 $ 509,462

D006122YV02 $ 28,516

D007566MJ03 $ -

D008632JS02 $ 101,681

D008763VM01 $ 648,707

D009441GS02 $ 840,203

D018322HE01 $ 101,687

M005912LQ02 $ 96,711

M005912LQ03 $ 62,331

Total $ 7,638,690



Sigma Calculator

A. All values required to calculate Sigma level

Defects: 210 DPMO: 130,031

Units: 323 Sigma Level: 2.63

Opportunities per Unit: 5

B. Sigma calculated based on defects and number of opportunities

Defects: 210 DPMO: 130031

Overall Monetary Impact of Defects for One Month:

Estimated $ % of

Type of Defect or Error Impact of Defect Total

Vendor Invoice Not Posted to PMS 1,395,727$ 77% < Critical Defect No. 1

Timesheet not entered into PMS 296,000$ 16% < Critical Defect No. 2

Vendor Invoice Amount not Correct in PMS 38,012$ 2%

Fringe Benefit Rate is off by more than 1% 36,200$ 2%

Base Pay Rate is off by more than 1% 33,000$ 2%

Timesheet activity codes are not correct 12,400$ 1%

Timesheet hours are not correct 9,600$ 1%

Total $ Amount of Defects 1,820,939$ 100%



Number of Opportunities: 1,615 Sigma Level: 2.63

C. Enter only the known Defects Per Million Opportunities

Enter DPMO 130,031 Sigma Level: 2.63

Summarize MSA Results

Purpose

Measurement System Analysis or MSA is conducted to validate the measurements in the measurement phase of the project. The goal is to make sure that wide variations are

not introduced by the measurement process itself. This will help ensure that the control phase is measuring actual variation and not actual variation + measurement variation.

Measurements

This project will measure the number of defects associated with seven different CTQC (Critical to Quality Characteristics):

1. Are employees submitting timesheets on a timely basis within the Project Management System? 2. Are the timesheets that have been submitted accounting for all hours during the two week period? 3. Are the timesheets that have been submitted using the correct activity codes for charging projects within the Project Management System? 4. Are the base rates of pay used to cost out timesheets within 1% of the source rates provided by System Administrators? 5. Are the fringe benefit rates used to cost out timesheets within 1% of the source rates provided by System Administrators? 6. Are vendor invoices posted in a timely manner to the Project Management System? 7. Are the vendor invoices that are posted accurate within 1% of the total invoice amount?

Method

A short approach known as Attribute Gage R & R will be used to measure Repeatability (variation from the same repeating measurement process) and Reproducibility

(variation from the same measurement with different observers).

Results

Overall, the Operational Definitions seem to be well defined with little room for variation during the measurement process:

Test No.

Critical To Quality Characteristic Repeat % Agree

Repeat % Agree

Repeat % Agree

Reprod % Agree

1 Base Pay Rates within 1% 100% 100% 100% 100%

2 Fringe Benefit Rates within 1% 100% 100% 95% 95%

3 Timesheets entered into PMS on time 100% 100% 100% 100%

4 Timesheets are complete (all hours) 100% 100% 100% 90%

5 Timesheets coded correctly 95% 100% 100% 75%

6 Vendor invoices posted into PMS 100% 100% 100% 90%

7 Vendor invoice amounts correct 100% 100% 95% 90%

Trend Charts:



Histogram:

Observation Value Describe the Sample Test and Defect Measured:

1 27 Nine observations were made over a 9 month period regarding the posting of

2 22 vendor invoices into the Project Management System. Compare the invoices

3 20 that were processed and paid per the General Ledger with invoices that were

4 26 posted in the Project Management System. Count the number of invoices that

5 21 were not posted in the Project Management System.

Two

Week

Ending

Timesheets Not

Entered into PMS10/12/15 41

10/26/15 37

11/9/15 35

11/23/15 33

12/7/15 36

12/21/15 40

1/4/16 38

1/18/16 36

2/1/16 34

0

5

10

15

20

25

30

35

40

45

Timesheets Not Entered into PMS

Month /

Year

Vendor Invoices

Not EnteredJan 2015 27

Feb 2015 22

Mar 2015 20

Apr 2015 26

May 2015 21

Jun 2015 28

Jul 2015 25

Aug 2015 22

Sep 2015 28

0

5

10

15

20

25

30

Jan2015

Feb2015

Mar2015

Apr2015

May2015

Jun2015

Jul2015

Aug2015

Sep2015

Vendor Invoices Not Entered



6 28

7 25

8 22

9 28

Maximum 28.0

Minimum 20.0

Range 8.0

Number of Intervals based on Square Root of Observations > 3 5 < Minimum Intervals

Width of each interval based on Range / (No of Intervals - 1) 2

Histogram Table based on above data: Simple Side Ways Plot

Interval Freq

1 Less than or Equal to 20 1

2 Between 21 and 22 3

3 Between 23 and 24 0 Appears to be a Bi Modal Distribution



More Formal Histogram Bar Chart:

Milestone Review:

Project Definition Phase is fully complete – We have a good set of Operating Definitions to drive the Measurement Phase

Preliminary Data Review – We know what needs to be measured and analyzed going forward: Timesheets and Vendor Invoices

Baseline levels of performance are now established to move us into analysis and improvement

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5

Fre

qu

en

cy

Intervals

Frequency Distribution

Freq



What have you decided to measure? Based on the January 2015 actual costs, we will focus on two big areas: Direct Labor

Costs and Vendor Invoices. Regarding direct labor costs, we will look at two critical data sources: Rates and Hours. Regarding

vendor invoices, we will measure two things: Timely posting of the invoices into the Project Management System and correct

posting of the amounts. These are the vital few attributes that will get measured.

Breakdown of Actual Costs

Total Vendor Costs

Total Labor Costs

All Other Costs

Two Preliminary Tests: Compared January 2015 General Ledger Costs to Project Management System Compared assigned

employees to submitted timesheets in Project Management System

Labor Rates are updated, but may not be accurate

Timesheet Processing is Broken: Not all employees assigned to projects are submitting timesheets into the Project Management

System!

Vendor Invoice Processing is Broken: Invoices not entered at all on several projects!

Distortions to Actual Costs appear to be material – additional testing is required

What techniques were used to validate your measurement data?

Attribute Measurement System Analysis was applied to each Operating Definition that we measured. This is a basic Gage R & R tool to

make sure we have repeatability and reproducibility:

• Repeatable – Are we coming up with the same measurement values given each observer (three used / two trial runs)?

• Reproducibility – Are we coming up with the same measurement values across all three observers?



What are the results of your MSA (Measurement System Analysis)? Everything was acceptable except for our testing of

timesheet accuracy. We had to revise the procedure and standards so that we had a better listing of activity codes to test

against. This eliminated the bias. The results of our MSA are listed here:

Gage R & R

Observers / Appraisers

CB BR MW EJ Description of Measurement

100% 100% 100% 100% Base Pay Rates agree to Source System Data

100% 100% 95% 95% Fringe Rates agree to Source System Data

100% 100% 100% 100% Timesheets submitted in a timely manner

100% 100% 100% 90% Timesheets are complete

95% 100% 100% 75% * Timesheets are accurately coded

100% 100% 95% 90% Vendor invoices posted in a timely manner

100% 100% 95% 90% Vendor invoice amounts posted correctly

What is the current baseline level of performance?

Focus on two critical to quality defects:



Critical Defect No. 1 – Vendor Invoices not posted to PMS

Critical Defect No. 2 – Timesheets not posted to PMS

Is this project on schedule according to the project plan? The project is 8 days behind schedule as a result of team members pulling

off to do other work. We still expect to meet the overall end-date since the analysis phase and improvement phases should move rather

quickly.

ANALYZE

Simple Fishbone – Invoices

Simple Fishbone - Timesheets

Fishbone Diagram

Invoices Not Posted to PMS

Not Followed

Not Trained

No Single System

Not DocumentedNot Accounting

Oriented

Not Well Defined

Not Important to Leadership

Wrong DocumentsNo Tracking at All

Wrong MetricsNo Integration of Data Not Available

Environment PeopleProcedures

SystemsMeasurement Source Documents

Fishbone Diagram

Timesheets Not Submitted

Managers Forget to Approve

Managers Not TrainedManager Approvals is

Not Easy

Employees Not Trained

Employees Forget

No Email Notifications

No Approval Panels in PMS

Employees Not Setup in PMS

Cost Impact Not Understood

Not Important to Managers

No Exception TrackingNo Exception Reports All Copies are Electronic

Environment PeopleProcedures

SystemsMeasurement Source Documents



Cause and Effect:

Y

X1 X2

X4 X5

Simple Cause Effect Diagram

X3

Y: Actual Costs

X1: Labor Rates X2: Hours

X1-1: Base Pay Rate X1-2: Fringe Benefits Rate

X3: Vendor Invoices X4: Travel Vouchers X5: Credit Card Payments

X1-1 X1-2

Source System for

X1-1 and X1-2 is the

Payroll System

Source System for

X2 is the Project

Management System

Source System for X3, X4,

and X5 is the Financial

Reporting System

Output System for Y

is the Project

Management System

FMEA:



Failu

re Mo

des an

d E

ffects An

alysis

Process or

Product N

ame:

Prepared by: C

lint

Britt

Page 1 of 1

Responsible:

FM

EA

Date (O

rig) 2/10/2016 (Rev) _____________

Pro

cess S

tep

Ke

y Pro

cess

Inp

ut

Po

ten

tial

Fa

ilure

Mo

de

Po

ten

tial F

ailu

re

Effe

cts

Ho

w S

eve

re

to C

usto

me

r

Po

ten

tial

Ca

use

s

Fre

qu

en

cy

of E

rror

Cu

rren

t Co

ntro

lsC

urre

nt A

bility to

De

tect E

rror

RPN

Actio

ns R

eco

mm

en

de

dR

esp

.A

ction

s Ta

ken

Subm

ission of

Time S

heets

Em

ployees

entering

Timesheets

into Project

Managem

ent

System

Em

ployees fail

to enter and

submit w

eekly

timesheets for

updating the

Project

Schedule

Project C

osts are

under-stated in the

Monthly

Perform

ance

Reports and

Project S

chedules

are incorrect in

terms of tim

e spent

and progress to

date

CIO

considers

this very severe

since it impairs

the reliability of

performance

information.

RP

N F

actor = 5

No program

controls or

manual check

list controls in

place for all IT

projects. Project

Managers are

not approving

weekly

timesheet

submissions.

Weekly.

RP

N F

actor

= 4

Project M

anagers

are responsible for

making sure that

all team m

embers

have entered time

sheets into the

Project

Managem

ent

System

. Time

Sheet approval

screen provides

employee listing.

Relatively easy if

Project M

anagers

approve time sheets

every week. R

PN

Factor =

2

11P

MO

Com

munication M

emo to

be issued to all Project

Managers w

ith Procedure P

aper

/ Screen S

hots attached.

Project Team

to develop more

robust solutions.

PM

O

What are the com

pleted

actions taken with the

recalculated RP

N? B

e

sure

to in

clud

e

com

ple

tion

mo

nth

/yea

r

Subm

ission of

Vendor

Invoices

Contractor's

invoices are

received and

entered into

the Project

Managem

ent

System

Project C

ost

Analyst does

not receive and

enter vendor

invoices into

the Project

Managem

ent

Project C

osts are

under-stated in the

Monthly

Perform

ance

Reports and

Budget

Com

parisons are

CIO

considers

this very severe

since it impairs

the reliability of

performance

information.

RP

N F

actor = 5

No autom

atic

interface

between

systems. N

o

reconciliations

between

systems.

Monthly.

RP

N F

actor

= 3

Contracting

Officers w

ho

administer

contracts send

copies of vendor

invoices to Cost

Analyst for

Som

ewhat difficult

since Cost A

nalyst

must m

anually track

and monitor each

month. R

PN

Factor =

4

12

Project Team

will evaluate

current process and develop

improvem

ents

PM

O

Integrity of Costs in M

onthly

Perform

ance Reports

Project M

anagement O

ffice

(PM

O)



Scatter Plots:

Vendor Invoices:

< - - Data to Plot - ->

X Values Y

Values

1 3 3

2 4 3

3 4 4

4 2 2

5 6 5

6 5 5

7 4 2

8 6 5

9 7 7

10 6 6

11 8 7

12 4 4

13

14

15

Timesheets:

< - - Data to Plot - ->

X Values

Y Values

1 11 11

2 16 16

3 22 20

4 17 16

5 34 32

6 26 25

7 19 19

8 18 18

9 29 28

10 24 24

11 26 26

12 21 21

13

14

15

0

1

2

3

4

5

6

7

8

0 5 10

Invo

ices E

nte

red

Invoices Reconciled

Scatter Plot - Invoices

Invoices

Linear(Invoices)

0

5

10

15

20

25

30

35

0 10 20 30 40

Tim

esh

eets

En

tere

d

Timesheets Reviewed

Scatter Plot - Timesheets

Timesheets Linear (Timesheets)



z-Tests:

Critical Defect No. 1 - Vendor Invoices Posted to PMS

H0: p1 = p2 Null Hypothsis: Number of Vendor Invoices not entered is the same regardless of reconciliation control procedure

H1: p1 > p2 Alternative Hyposis: Number of Vendor Invoices not entered is higher w hen reconciliation procedure is not follow ed

n1: population of projects that have no reconciliation procedure in place

n2: population of projects that do have a reconciliation procedure in place

Size => 141 62 0.99

Observation Sample n1 Sample n2 0.01

d1 d2 2.326

1 12 2

2 11 1 You should reject the Null Hypothesis if the Test Statistic

3 8 1 is higher than the Decision Cut Off Value of: 2.326

4 9 0

5 7 0

6 11 1 1. Proportion of defects (d1 / n1) 0.55


8 9 1 3. Overall Proportion of defects (d1+d2) / (n1+n2) 0.41

Average (u) 9.625 0.750 4. Calculate the Test Statistic (z) 6.00

Variance 2.839 0.500

Std Deviation 1.685 0.707

Total Defects 77 6 Test Statistic of 6.00 is higher than 2.326

Reject the Null Hypothesis and Accept the Alternative

Critical Defect No. 2 - Timesheets Entered into PMS

H0: p1 = p2 Null Hypothsis: Number of Timesheets entered is the same regardless if Managers review and approve timesheets

H1: p1 > p2 Alternative Hyposis: Number of Timesheets not entered is higher w hen Managers do not review and approve timesheets

n1: sample timesheets on those projects w here Managers do not approve timesheets

n2: sample timesheets on those projects w here Managers do approve timesheets

0.99

Size => 436 312 0.01

Observation Sample n1 Sample n2 2.326

d1 d2

1 26 6 You should reject the Null Hypothesis if the Test Statistic

2 19 4 is higher than the Decision Cut Off Value of: 2.326

3 22 5

4 25 6



7 23 5 3. Overall Proportion of defects (d1+d2) / (n1+n2) 0.45

8 24 6 4. Calculate the Test Statistic (z) 11.98

9 22 6

10 25 5

11 21 4 Test Statistic of 11.98 is higher than 2.326

12 26 5 Reject the Null Hypothesis and Accept the Alternative

Average (u) 23.167 5.083

Variance 8.152 0.629

Std Deviation 2.855 0.793

Total Defects 278 61

Conclusion:

Confidence Level =>

Alpha or Sign Level =>

Decision Cut Off =>

Confidence Level =>

Alpha or Sign Level =>

Decision Cut Off =>

Calculate the Test Statistic:

Conclusion:

Calculate the Test Statistic:



Milestone 3 Review - Analysis Phase Complete

• Analysis is focused on two critical to quality processes:

– Processing of Time Sheets

– Processing of Vendor Invoices

– Major Analytical Steps: Identify Process Variables - Compare Impacts – All Variables

– Apply Analytical Techniques

1. Fish Bone & Five Whys

2. Scatter Plots & Regression Analysis

What variables have the biggest impact on improving the two critical processes?

Processing of Vendor Invoices (Critical Defect #1). Comparing / Reconciling to General Ledger Listing of Vendor Invoices.

Receiving all monthly copies of invoices. Manual tracking of monthly recurring contract invoices

Processing of Time Sheets (Critical Defect #2). Manager review & approval of timesheets each pay period. Employee

notifications of timesheets not entered

Describe the current sampling plan for the two critical processes. Focus on the high impact projects within the IT portfolio. Need to

sample these projects on a continuous basis as follows: Statistical sampling of timesheets every two weeks to drive control charts and

subsequent project phases (improvement and control).

What is the expected time frame for sampling? In order to ensure quality sampling, we are recommending that the team (or PMO staff)

continue to pull samples well into next year. This will also help facilitate control and improvement over the two critical processes involved.

ANOVA Test: Assess differences between various treatments:



IMPROVE:

Solutions Ration Matrix - Invoices

1.1

1.2

1.3

1.4

1.5

1.6

Solutions Ratings Matrix - Invoices

Fully automated solutions are

available

Solution must be done m

anuallySolution m

ight be automated, but

very costly

Solution can be automated, but

requires change

Automated solutions exist, som

e

changes are needed

No measurable costs or tim

e are

required

Management does not support

this solution

Limited support for this solution

Support exists, but some are

doubtful

Good buy-in from m

anagement

on this solution

Full and complete support from

managem

ent

Very high costs and time are

involved

Additional costs and time will be

required

Some m

odest costs and time are

needed

Relatively low costs and time are

involved

Very little if any support is needed

to implem

ent

No significant impact on

improving the process

Modest impact on im

proving the

process

Some gains will be realized

Very solid improvem

entsCom

plete turnaround of the

process

Very high support levels needed to

implem

ent

A permanent level of support is

needed to implem

ent

Some continuing support will be

needed to implem

ent

Only modest levels of support are

needed to implem

ent

Extremely easy, projects should

comply quickly

1 = Lowest2 = Som

ewhat Low3 = Moderate

4 = Somewhat High

5 = Highest

Very difficult to implem

ent across

projects

Difficult, some projects m

ay not

comply with the change

Somewhat difficult, but all projects

comply over tim

e

Within one year, all projects

should easily comply

Easy to Implem

ent

Easy to Maintain

Significant Impact

Minimal Cost and Tim

e

Senior Management Support

Solution can be Automated



Solutions Ration Matrix – Timesheets

Criteria Criteria Criteria Criteria Criteria Criteria Criteria Criteria TOTAL

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 RATING

3.1 3 3 5 3 5 3 22

3.2 5 4 3 4 5 2 23

3.3 5 5 2 5 5 1 23

3.4 0

3.5 0


1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 WGHT

Weight Assigned > Weight Assigned > 100% 50% 75% 50% 100% 75% RATING

3.1 3 1.5 3.75 1.5 5 2.25 0 0 17

3.2 5 2 2.25 2 5 1.5 0 0 17.75

3.3 5 2.5 1.5 2.5 5 0.75 0 0 17.25

3.4 0 0 0 0 0 0 0 0 0

3.5 0 0 0 0 0 0 0 0 0

Most Attractive

Cost = All resources to implement, such

as funding, personnel, equipment, tiime, etc.

Benefit = Both tangible and intangible benefits

such as fewer errors, reduces time, improves

customer satisfaction, lowers costs, etc.

Least Attractive

Someone is trained / assigned to enter invoices

0

0

Projects reconcile costs to the General Ledger

Projects receive copies of vendor invoices

Projects reconcile costs to the General Ledger

Projects receive copies of vendor invoices

Someone is trained / assigned to enter invoices

Lo

w <

- -

Ben

efi

ts -

- >

Hig

h

High < - - - - - - - - Costs - - - - - - - - - - > Low

Solution 3.1

Solution 3.2

Solution 3.3



1.1

1.2

1.3

1.4

1.5

1.6

Solutions R

atings Matrix - Tim

esheet Processing

Senior M

anagement S

upport

Solution can be A

utomated

Easy to Im

plement

Easy to M

aintain

Significant Im

pact

Minim

al Cost and Tim

e

Extrem

ely easy, projects should

comply quickly

1 = Lowest2 = S

omewhat Low

3 = Moderate

4 = Som

ewhat High

5 = Highest

Very difficult to im

plement

across projects

Difficult, som

e projects may not

comply w

ith the change

Som

ewhat difficult, but all

projects comply over tim

e

Within one year, all projects

should easily comply

Very little if any support is

needed to implem

ent

No significant im

pact on

improving the process

Modest im

pact on improving the

process

Som

e gains will be realized

Very solid im

provements

Com

plete turnaround of the

process

Very high support levels needed

to implem

ent

A perm

anent level of support is

needed to implem

ent

Som

e continuing support will be

needed to implem

ent

Only m

odest levels of support

are needed to implem

ent

No m

easurable costs or time

are required

Managem

ent does not support

this solution

Limited support for this solution

Support exists, but som

e are

doubtful

Good buy-in from

managem

ent

on this solution

Full and complete support from

managem

ent

Very high costs and tim

e are

involved

Additional costs and tim

e will be

required

Som

e modest costs and tim

e

are needed

Relatively low

costs and time

are involved

Fully automated solutions are

available

Solution m

ust be done manually

Solution m

ight be automated,

but very costly

Solution can be autom

ated, but

requires change

Autom

ated solutions exist,

some changes are needed



Error Proof Critical Processes:

Vendor Invoices:

1. What is the defect? Vendor invoices fail to get posted on a monthly basis within the Project Management System (PMS)

2. What is the source system involved? Expense Module within the Project Management System

3. What is the source document? - Vendor Invoices

4. What positions are responsible for running this process? Project Planners and Project Managers

5. Is it possible to automatically detect / inspect the defect? Possible matching by invoice number between General Ledger and

Project Management System


1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 RATING

3.1 5 4 5 4 4 2 24

3.2 2 5 5 4 5 4 25

3.3 5 5 2 5 4 3 24

3.4 0

3.5 0


1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 WGHT

Weight Assigned > Weight Assigned > 100% 50% 75% 50% 100% 75% RATING

3.1 5 2 3.75 2 4 1.5 0 0 18.25

3.2 2 2.5 3.75 2 5 3 0 0 18.25

3.3 5 2.5 1.5 2.5 4 2.25 0 0 17.75

3.4 0 0 0 0 0 0 0 0 0

3.5 0 0 0 0 0 0 0 0 0

Most Attractive

Cost = All resources to implement, such

as funding, personnel, equipment, tiime, etc.

Benefit = Both tangible and intangible benefits

such as fewer errors, reduces time, improves

customer satisfaction, lowers costs, etc.

Least Attractive

Managers must approve timesheets

Email notifications when timesheets not entered

Mandatory timesheet training

Lo

w <

- -

Ben

efi

ts -

- >

Hig

h

High < - - - - - - - - Costs - - - - - - - - - - > Low

Mandatory timesheet training

0

0

Managers must approve timesheets

Email notifications when timesheets not entered

Solution 3.1

Solution 3.2

Solution 3.3



6. Is it possible to issue a warning to the source person involved? Possible search and report blank entries when past monthly entries have been posted for the last consecutive six months 7. Is it possible to correct the error as part of the process loop? No - PMS is outside the Accounts Payable process within the General Ledger Timesheets:

1. What is the defect? Employees fail to submit a timesheet within the Project Management System (PMS) 2. What is the source system involved? Time & Attendance Module within PMS 3. What is the source document? Electronic Timesheet for Employee (Resource ID) 4. What positions are responsible for running this process? All employees assigned to the project 5. Is it possible to automatically detect / inspect the defect? Yes - all active resources who are setup for timesheet entry have failed to submit a timesheet for the two week pay cycle 6. Is it possible to issue a warning to the source person involved? Yes - once a due date and time have been reached, issue an email notification to those resources that are past due 7. Is it possible to correct the error as part of the process loop? Yes - warnings must be issued within 24 hours on each Friday of each two week pay cycle with a hard close the following Tuesday at 12 noon.

Corrective Action Plans – Vendor Invoices and Time Sheets



Co

rrec

tive

Ac

tion

Pla

n a

s o

f 5/1

0/1

6

Pro

ce

ss

ing

of V

en

do

r Inv

oic

es

Issue

: Pro

jects

may n

ot p

ost a

ll vendor in

voic

es in

to th

e P

roje

ct M

anagem

ent S

yste

m, th

ere

by d

isto

rting c

ost p

erfo

rmance in

form

atio

n a

nd m

ak

ing th

e m

onth

ly revie

w p

rocess s

om

ew

hat n

on-va

lue a

dded.

Ob

jectiv

e: N

eed to

imple

ment s

pecific

contro

l pro

cedure

s th

at w

ill impro

ve th

e c

urre

nt p

rocess e

nsurin

g th

at ve

ndor in

voic

es g

et p

oste

d in

PM

S.

Ref N

oD

escrib

e A

ctio

n Ite

mS

cope o

f Actio

n Ite

mC

ham

pio

nC

ore

Team

/ PO

CP

ilot?

Paper?

Tra

inin

g?

Pro

cedure

?D

ate

Date

of S

uccess

Resolu

tion

VS

.1D

eve

lop a

nd D

istrib

ute

month

ly

Genera

l Ledger R

eports

on a

pro

ject b

y p

roje

ct b

asis

to a

ll

Pro

ject M

anagers

so th

at th

ey c

an

revie

w / re

concile

vendor in

voic

es

Syste

m R

equest

require

d c

ove

ring a

ll IT

pro

jects

setu

p in

the

Genera

l Ledger

PM

O D

irecto

r -

Clin

t Britt

Six

Sig

ma T

eam

and F

RS

Pro

gra

mm

er

Yes - ru

n

seve

ral te

st o

f

pro

gra

m b

efo

re

puttin

g in

to

pro

ductio

n

No - th

is is

belo

w th

e

polic

y le

vel

Yes - d

eve

lop

train

ing s

lides

and c

onduct

live m

eetin

g

Yes -

refe

rence th

is

new

activity

in

EV

M

Pro

cedure

9

TB

D -

Depends

upon

Syste

m

Request

TB

D - 3

0

days a

fter

Syste

m

Request is

100%

com

ple

te

All p

roje

cts

are

usin

g th

e n

ew

report to

valid

ate

vendor in

voic

es in

PM

S.

TB

D - 3

month

s

afte

r

imple

menta

tion

date

VS

.2P

ayable

pro

cessin

g is

revis

ed to

inclu

de fo

rmal d

istrib

utio

n o

f invo

ice

copie

s to

all IT

Pro

ject P

lanners

Fin

ancia

l Accountin

g

Contro

l and IT

Pro

ject

Managers

PM

O D

irecto

r -

Clin

t Britt

Six

Sig

ma T

eam

and P

MO

Not a

pplic

able

No

No

No

5/1

/2016

6/1

/2016

All p

roje

cts

are

receivin

g c

opie

s o

f

vendor in

voic

es

VS

.3A

ll pro

jects

have

a P

roje

ct P

lanner

assig

ned a

nd tra

ined in

the p

ostin

g

of ve

ndor in

voic

es to

the P

roje

ct

Managem

ent S

yste

m

All IT

Pro

jects

that

should

be u

sin

g P

MS

PM

O D

irecto

r -

Clin

t Britt

Six

Sig

ma T

eam

and P

MO

Not a

pplic

able

No

Yes - u

pdate

PM

S T

rain

ing

Module

s

Yes - u

pdate

PM

S S

OP

11

5/1

/2016

6/1

/2016

All p

roje

cts

have

a

train

ed p

lanner

assig

ned

VL.1

Deve

lop a

n a

uto

mate

d in

terfa

ce

com

paris

on o

n a

n in

voic

e b

y in

voic

e

num

ber b

asis

betw

een th

e G

enera

l

Ledger a

nd th

e P

roje

ct

Managem

ent S

yste

m.

All IT

Pro

jects

setu

p

in th

e G

enera

l Ledger

Syste

m

PM

O D

irecto

r -

Clin

t Britt

Six

Sig

ma T

eam

and P

MO

Yes - ru

n

seve

ral te

st

No

Yes - w

ill need

to re

vise

train

ing

Yes - w

ill need

to u

pdate

EV

M

SO

P

TB

DTB

DA

ll pro

jects

are

usin

g th

e w

eb

based to

ol

Glo

ssary

of T

erm

s

EV

ME

arn

ed V

alu

e M

anagem

ent

FR

SF

inancia

l Reportin

g S

yste

m

ITIn

form

atio

n T

echnolo

gy

PM

OP

rogra

m M

anagem

ent O

ffice

PM

SP

roje

ct M

anagem

ent S

yste

m

PO

CP

oin

t of C

onta

ct

TB

DTo B

e D

ete

rmin

ed

Imm

ed

iate

Ac

tion

Item

s

Lo

ng

Te

rm S

olu

tion

s



Co

rrec

tive

Ac

tion

Pla

n a

s o

f 5/1

0/1

6

Pro

ce

ss

ing

of T

ime

sh

ee

tsIssu

e: E

mplo

yees m

ay n

ot s

ub

mit tim

esheets

into

the P

roje

ct M

anagem

ent S

yste

m, th

ere

by d

isto

rting c

ost p

erfo

rmance in

form

atio

n a

nd m

ak

ing th

e m

onth

ly revie

w p

rocess s

om

ew

hat n

on-va

lue a

dded.

Ob

jectiv

e: N

eed to

imple

ment s

pecific

contro

l pro

cedure

s th

at w

ill impro

ve th

e c

urre

nt p

rocess e

nsurin

g th

at e

mplo

yees e

nte

r timesheets

in P

MS

with

each p

ay c

ycle

.

Run T

est

Polic

yG

uid

e B

ook /

Std

Opera

ting

Impl

Est C

Measure

Fin

al

Ref N

oD

escrib

e A

ctio

n Ite

mS

cope o

f Actio

n Ite

mC

ham

pio

nC

ore

Team

/ PO

CP

ilot?

Paper?

Tra

inin

g?

Pro

cedure

?D

ate

Date

of S

uccess

Resolu

tion

TS

.1A

ll Pro

ject M

anagers

appro

ve

timesheets

for a

ll pers

onnel

assig

ned to

the p

roje

ct

All IT

Pro

jects

that u

se P

MS

PM

O D

irecto

r - Bria

n

Recto

r & C

hie

f

Info

rmatio

n O

fficer

Six

Sig

ma T

eam

and P

roje

ct

Managers

No

No

No

Yes - u

pdate

SO

P's

4/1

/2016

6/1

/2016

All P

roje

ct M

anagers

are

appro

ving

timesheets

TS

.2A

ll em

plo

yees a

ssig

ned to

a

pro

ject m

ust c

om

ple

te m

andato

ry

timesheet tra

inin

g w

ith tw

o w

eeks

All n

on-c

ontra

ct la

bor

pers

onnel a

ssig

ned to

an IT

pro

ject

PM

O D

irecto

r - Bria

n

Recto

r & C

hie

f

Info

rmatio

n O

fficer

Six

Sig

ma T

eam

and P

roje

ct

Managers

No

No

Yes - n

eed to

revis

e tra

inin

g

and G

uid

e

No

4/1

/2016

6/1

/2016

All a

ssig

ned

em

plo

yees a

re

ente

ring tim

esheets

TL.1

Deve

lop a

n a

uto

mate

d e

mail

notific

atio

n m

essage e

ach tim

e a

n

em

plo

yee fa

ils to

ente

r their

timesheet w

ithin

a 2

4 h

our p

erio

d

All n

on-c

ontra

ct

pers

onnel a

ssig

ned to

IT p

roje

cts

PM

O D

irecto

r - Bria

n

Recto

r

Six

Sig

ma T

eam

and P

MS

Syste

m

Adm

inis

trato

rs

Yes

No

No

Yes - u

pdate

SO

P's

once

imple

mente

d

TB

DTB

DA

ll assig

ned

em

plo

yees a

re

ente

ring tim

esheets

Imm

ed

iate

Ac

tion

Item

s

Lo

ng

Te

rm S

olu

tion

s



Training Plan

1. Revised Training on Timesheet Processing

2. New Training on Reconciling Costs

1. Revised Training on Timesheet Processing

A. Goal / Objective of Training

Update the current training on timesheet processing to include additional information on timesheet approval. This would include how

Project Managers can go online and approve timesheets within the Project Management System. The training should also include a closing

schedule of due dates for both submission and approval of timesheets. The goal is to make sure everyone submits timesheets on time and

to make sure Project Managers are approving timesheets.

B. Title of Training Program

Submitting and Approving Timesheets in the Project Management System. This training module is already in place.

C. Training Method(s)

Computer Based Training exists. Need to update certain pages in the training module. Also should update the Standard Operating

Procedure and announce the updated training program to all Project Managers.

D. Training Time Line / Dates

Revised training module should be available by end of May 2016.

E. Scope of Training

All personnel subject to submitting or approving timesheets in the Project Management System. This represents approximately 350 people

within the Office of Information and Technology.



F. Resources Needed / Administrative Help

Approximately one-week of content development from the System Administrators. Approximately two weeks of time from the Web

Master for updating pages. PMO will review and approve changes once uploaded. Some administrative help will be required to issue a

communication notice to all personnel impacted.

G. Length / Format

Current program will be expanded from approximately 2 hours to 2.5 hours in length. Same format – online web based training module

will be used.

H. Other Components (Handouts / Exercises / Evaluation)

Post and update on a regular basis the Timesheet Closing Schedule. Include a link in the training program. Continue to use the same

evaluation form currently in place for assessing the effectiveness of the training.

I. Training Sources

Screen shots and content should be developed directly from System Administrators – Project Management System.

J. Dependencies/Constraints/Limitations

Availability of System Administrators may represent a dependency. However, the extent of content development appears minimal and no

problems are anticipated. Also, web based training can be limited in its effectiveness – need to re-enforce with revised Standard Operating

Procedure, communications from leadership and desk side assistance for Project Managers. This will require some work on the part of the

PMO.

2. New Training on Reconciling Costs

A. Goal / Objective of Training

Introduce and describe a new control procedure on how to reconcile costs between the Project Management System (PMS) and the

Financial Management System. Goal is to get project teams into a monthly routine of comparing their project costs against the General

Ledger to ensure actual costs have been properly captured in PMS.

B. Title of Training Program



Reconciling Actual Costs to the General Ledger.

C. Training Method(s)

Combination of web based training module, desk side assistance, and live on-line meeting for project leads.

D. Training Time Line / Dates

Launch new web based training module in May 2016. Provide desk side assistance after May 2016 on an as needed basis.

E. Scope of Training

Project Planners, Schedulers, Managers and other personnel who are directly responsible for maintaining project cost data within the

Project Management System (PMS). This includes approximately 50 personnel within the Office of Information and Technology.

F. Resources Needed / Administrative Help

Need System Administrators to develop screen shots and certain narrative content for this training module. PMO will need to design and

develop most of the content. The PMO will be responsible for developing supporting materials, such as handouts, operating procedures,

and other documents. Some administrative help will be needed to setup live meetings, issue communication notices, and do other tasks.

G. Length / Format

Anticipated length of this new training module is 30 minutes. Training format will be similar to other web based training modules

previously developed by the PMO.

H. Other Components (Handouts / Exercises / Evaluation)

The web based training module should include an example of how to reconcile costs, showing screen shots, and providing links to report

examples. A separate handout should also be developed to assist with desk side help. This can include a Standard Operating Procedure to

document the monthly reconciliation procedure. Evaluation survey is the same as used by other training modules.



I. Training Sources

The PMO has developed the pilot procedure that is now ready to go. This includes an extract report from the Financial Management

System. Personnel within Financial Accounting are also useful in understanding the General Ledger data. It is important to map and align

this data between the two systems: Project Management System vs. Financial Management System.

J. Dependencies/Constraints/Limitations

New control procedures can be difficult to implement. Most of the project personnel have little or no understanding of how accounting

works. Project Managers, the PMO, and others will have to monitor implementation. This control procedure is time consuming and

project teams are often under high demands with other tasks.

Milestone Review - Improvement Phase Complete

Three Immediate Improvements

1. Require a reconciliation / comparison to the General Ledger

2. Require training:

1. Invoices – Entering Expense Entries into the Project Management System

2. Time Sheets – Entering Time Sheets into the Project Management System

3. Require Timesheet Approvals by Managers

Two Long Term Solutions

1. Automated comparison of invoices by invoice number between GL and PMS

2. Automated Email notification to employees who fail to submit time sheets

Monitor and Control

Final project phase will:

Establish a Control Plan to monitor defect rates per continuous sampling.

Set initial performance goals and revisit performance goals once initial goals have been met

Include Reaction Plans that map out how to deal with out of control process situations



Monitor progress using simple Trend Charts

CONTROL:

Control Plan

Control Plan for Critical Process 1: Vendor Invoices

Critical to Quality Issue:Revision Number 1 Date 6/1/2016

Organization

A. Critical to Quality Characteristics to Reduce Defects

A.1

A.2

A.3

A.4

A.5

B. Control Chart Process

B.1 What type of control charts will be used?

B.2 Chart control process champion is:

B.3 Where are the charts located at?

B.4 What measurement method will be used?

B.5 What is the control target?

C. Readiness for Control Charting

C.1 Has a measurement study been conducted?

C.2 What is the current defect rate?

C.3 Does the control process have a reaction plan?

C.4 Is the process currently stable?

D. Sample Plan

D.1 Is there a documented sample plan in place? Yes

D.2 What is the frequency of sampling? Monthly

D.3 What is the sample size? 10

D.4 Who will take the samples?

D.5 Where are the sample results located at?

Office of Information & Technology

Simple Trend Chart

Posting of Vendor Invoices to PMS

Reconcile Invoices to the General Ledger

Complete distribution of invoice copies to project control personnel

Project Management Office (PMO)

Brian Rector - PMO

Count and plot number of defects

95% of monthly vendor invoices have been posted to PMS

PMO

Brian Rector - PMO

Yes - Attribute Measurement System Analysis

20% overall based on samples of prior months data

Yes - see next tab in this workbook

No - project solutions just now getting implemented



Reaction Plan for Critical Process 1: Vendor Invoices


Organization

In the event that the Control Chart indicates an out-of-control condition, follow this plan of attack to address the condition:

Monthly sample of

vendor invoices per

the General Ledger

Does the sample meet Process is under control.

the control target of Consider calculating process

95%? Yes capability and setting a

new target and sample.

No

Identify the project id and

vendor invoices related to

the defects

Update the distribution list to

Is the defect due to Is Financial Control include the project id and point

a missing invoice copy sending copies of invoices of contact involved. Re-sample

not sent to the project? Yes to the project team? No next month to see if this problem

has been addressed.

No Yes

Train the Project Planner in

Does the project have Does the project reconciling invoices and make

a trained planner in reconcile invoices to sure the Planner gets copies

posting invoices to PMS? Yes the General Ledger? No of monthly GL Extract Reports.

No Yes

Enroll the Planner in the Notify the Planner and

new training program on Manager of the defects.

invoice processing and Re-sample this project

provide desk side help. next month.

Posting of Vendor Invoices to PMS




Control Plan for Critical Process 2: Timesheets


Organization

A. Critical to Quality Characteristics to Reduce Defects

A.1

A.2

A.3

A.4

A.5

B. Control Chart Process

B.1 What type of control charts will be used?

B.2 Chart control process champion is:

B.3 Where are the charts located at?

B.4 What measurement method will be used?

B.5 What is the control target?

C. Readiness for Control Charting

C.1 Has a measurement study been conducted?

C.2 What is the current defect rate?

C.3 Does the control process have a reaction plan?

C.4 Is the process currently stable?

D. Sample Plan

D.1 Is there a documented sample plan in place? Yes

D.2 What is the frequency of sampling? Bi Monthly

D.3 What is the sample size? 35

D.4 Who will take the samples?

D.5 Where are the sample results located at?


Simple Trend Chart

Timesheets Submitted to PMS

Timesheets are submitted and approved by Project Managers every pay cycle

Project Management Office (PMO)

Brian Rector - PMO

Count and plot number of defects

90% of timesheets are entered and approved

PMO

Brian Rector - PMO

Yes - Attribute Measurement System Analysis

21% overall based on samples of prior months data

Yes - see next tab in this workbook

No - project solutions just now getting implemented



Reaction Plan for Critical Process 2: Timesheets


Organization

In the event that the Control Chart indicates an out-of-control condition, follow this plan of attack to address the condition:

Bi Monthly Timesheets

are entered into

Project Management

System (PMS)

Does the sample meet Process is under control.

the control target of Consider calculating process

90%? Yes capability and setting a

new target and sample.

No

Identify the project id,

the employees, and the

pay period for all defects.

Make sure the Manager is

Is the defect due to Is the employee setup aware of the approval process

the Manager not and trained for entering within PMS and notify the

approving the timesheet? No timesheets into PMS? Yes Manager of all missed employees

Yes No

Make sure the Manager Have the employee

understands how to complete PMS setup

approve and process forms and related

timesheets. Followup training. Followup with

in next pay cycle. employee when done.

Resample these defects

to make sure the

proposed solutions are

implemented.

Timesheets Submitted to PMS




Trend Charts - Initial Implementation

A. Processing of Timesheets:

The Six Sigma Project pilot was launched in Feb 2016 and started to test and implement new procedures

Period Defects

1 9

2 11 Average Before 10

3 8 Average After 2

4 10

5 10

6 11

7 9

8 10

9 11

10 10

11 8 This is a period when improvements were proposed and tested

12 3 against four major projects within the Office of Information and Technology.

1 3 This is a period when the improvements were fully implemented against

2 1 the four major projects.

0

2

4

6

8

10

12

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Nu

mb

er

of

De

fects

Period

Trend Chart - Timesheets Not Entered QC02

Defects



Best Practice Log

B. Processing of Vendor Invoices:

The Six Sigma Project pilot was launched in Feb 2016 and started to test and implement new procedures

Period Defects

1 2

2 3

3 3 Average Before 3

4 1 Average After 1

5 3

6 3

7 2

8 3

9 3

10 3

11 3 This is a period when improvements were proposed and tested

12 1 against four major projects within the Office of Information and Technology.

1 1 This is a period when the improvements were fully implemented against

2 1 the four major projects.

0.5

1

1.5

2

2.5

3

3.5

4

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Nu

mb

er

of

De

fects

Period

Trend Chart - Invoices Not Entered QC02

Defects

BEST PRACTICES LOGProject Name: Month Financial Package (MFP) Data Integrity

Project Sponsor: Project Management Office

Project Manager Name: Clint Britt

Project Description:

ID Date

Identified

Point of

Contact

Source of Best Practice Describe Best Practice Recommendations & Comments Follow-Up Actions

1 06/01/16 Brian Rector Project Management System - System

Administrators Group

Report of Pending Timesheets that must be

approved w ithin Project Management System.

Communicate this practice to all projects. Include

in training and SOP documents w here

applicable.

Yes - Post Implementation Review to

see if other projects are using this

best practice

2 06/01/16 Brian Rector Financial Control Systems Group - Accounts

Payable Processing Unit

Electronic access to vendor invoices - print

copies for posting into Project Management

System.

Communicate this practice to all projects. Include

in training and SOP documents w here

applicable.

Yes - Post Implementation Review to

see if other projects are using this

best practice

Improve the quality of cost data reported in monthly performance review s



Milestone 5 Review - Control Phase Complete

• Control Phase is fairly straight forward:

1. Monitor progress on Trend Charts – Four Initial Projects Only (QC02, LB02, VM01 & GS02)

2. Implement Improvements on all remaining projects (over 12 projects subject to control)

3. Start Trending remaining projects for FY2008

4. Long Term Goal: 80% of all monthly project costs are at Sigma Level 5 by end of FY2008.

Good Progress

Continuous Improvement Practices

Desk Side Mentoring and Assistance to fully institutionalize the changes

• PMO will continue to track defects within PMS through the end of FY2016 (10/31/16)

• PMO will evaluate how to automate control procedures:

– Comparison of Invoices by Invoice Number between PMS and FMS

– Email notification and kick-out reports when timesheets are not submitted

What was this project’s overall cost performance?

– The project came in under budget since fewer resources were required to complete the project. Also, the solutions to

improve the processes were fairly easy to develop and implement. So this was a good “quick win” type project.

– Final Project Costs . . . . . . . . . . . . $ 164,000

– Project Budget . . . . . . . . . . . . . . . .$ 222,000



– Under Budget . . . . . . . . . . . . . . . . $ 58,000