2012-RenzoToledo

Effects of PMBOK Training on

Professional Individual

Performance Related to

Engineering and Construction

Projects Development

Renzo Toledo, MBA, PMP

Cosapi S.A.

Acerca del Autor

Renzo Toledo, es Gerente de

Proyectos de la Unidad de

Negocios de Ingeniera y

Proyectos EPC en Cosapi. Cuenta

con ms de 15 aos de

experiencia en proyectos de

ingeniera y construccin en los

sectores minero, energa,

hidrocarburos e industrial.

En adicin a ello es trainer y

conferencista en Gerencia de

Proyectos.

Renzo Toledo, MBA,

PMP

Antes que nada Advertencia

La ponencia refiere una tesis doctoral en curso

Al momento las mediciones no han sido realizadas

El contenido de la exposicin se enfoca en el anlisis del problema, los antecedentes en la

literatura y la propuesta del estudio a realizar

Texto en ingls Exposicin en castellano

Agenda

Introduction

Peru update

Brief story on the Research Proposal (RP)

Literature review

Doctoral Thesis Proposal (Draft)

Next steps

Conclusions (so far)

Introduction

Introduction

This paper presents an approach on the project management training effect on actual

engineering and construction project

performance.

Introduction

First part: Literature review on several studies conducted on this matter around the world,

and a detailed review of three of them.

Introduction

Second part: Displays a doctoral thesis research proposal (in progress) regarding the

above mentioned topic, specifically considering

Peruvian firms and practitioners, and the

training products related to A Guide to the

Project Management Body of Knowledge

(PMBOK Guide).

Peru Update

Peru Update

Among South American countries, Peru has a favorable combination of political stability,

represented by an civilian electoral system for

more than 30 years and a continuous economic

growth for more than 10 years.

An important investment opportunities portfolio: mining, oil and gas, energy, metals

and machining, construction, agriculture,

fishing, airports and ports, financial services,

and tourism.

PMI is a registered trade and service mark of the Project Management Institute, Inc. 2012 Permission is granted to PMI for PMI Marketplace use only.

Peru Update

In 2011, foreign investment exceeded 15 billion dollars (Banco Central de Reserva del Per

[BCRP], 2012).

Peru Video 1

Construction Industry

During the past few years, the construction industry has been one of the most important

for Peruvian economic development (Instituto

Nacional de Estadstica e Informtica, [INEI],

2012), showing a continuous growth.

The construction industry fund sources are related to new real estate initiatives, national

civil infrastructure, and private equity (local

and foreign) projects.

Peruvian Construction Industry

Peruvian Construction GDP Yearly Variation

Peru: Mining Operations Map

13.6% territory with mining concessions 1.09% territory in mining operations Peru World Mining Ranking Zinc #3 Tin #3 Lead #4 Gold #6 Copper #2 Silver #2

Mining Projects Portfolio

52 Main Projects

US$ 53.4 Billion

Brief Story on the Research

Proposal

Background (Personal Quest)

February 1997 Starting in Project Community: Engineering and Construction Industry

September 2000 MBA degree February 2002 Metals and Machinery Industry April 2007 PMP Certification June 2007 Starting as PM Trainer October 2008 Project Management VP February 2011 Doctoral Program Application:

Thesis Theme Proposal

January 2012 Back to Engineering and Construction

Starting Insights Project Management Practitioner:

Are the PMBOK Guide practices actually applied? Are they really helping me?

Project Management VP: Are the training programs worth enough? Do the training programs impact on performance?

Project Management Trainer: Is the PMBOK Guide a valid framework? How do the training programs improve practitioners

performance?

Preliminar Questions

Do the PMBOK Guide training programs actually impact on the projects practitioners performance?

What are the related variables? How to measure performance?

How to put this all together?

Literature Review

Background

Several project management best practices are compiled in the PMBOK Guide. They are recognized as the best practices used in project development, shared by different project management teams around the globe.

Background

The PMBOK Guide is not a methodology guide, but a reference for project management planning and control. The specific methodology for each project is developed by the project team assigned and the organization in charge (PMI, 2010, pp. 4, 1314, 3765)

Background

On the other hand, PMI (2007) proposed in the Project Manager Competency Development (PMCD) the professional competences required by the project management practitioners.

The competences described are classified as:

(a)knowledge

(b)professional, and

(c) personal

Background Knowledge competence is what the project

manager knows about the application of processes, tools and techniques for project activities (p. 2).

Performance competence is how the project manager applies project management knowledge to meet the project requirements (p. 2).

Personal competence is how the project manager behaves when performing activities within the project environment; their attitudes, and core personality characteristics (p. 2).

Background

Personal

Performance

Knowledge

Kirkpatrick and Phillips approach

Kirkpatrick model comprised four dimensions:

1. Satisfaction/ReactionThoughts, insights and feelings from the participants regarding the training.

2. LearningChanges in knowledge or skills. Changes in attitudes. This evaluation is handled during the training (demonstrations, presentations, tests, etc.)


3. BehaviorKnowledge, skills and attitudes (acquired in the classroom) applied on the job. Changes in behavior. This evaluation is applied 36 months post-training, while the trainee is performing the job.

4. ResultsResults or performance changes (improvement) because of attendance and participation in a training program (can be monetary, performance-based, etc.)


Adding to the Kirkpatrick model, Phillips (1996) included:

5. A return on investment (ROI) evaluation level,

in order to compare the results measured in the

fourth level of the Kirkpatrick model versus the

overall costs of training.

Approach to Training Evaluation

Kirkpatrick and Phillips Model

Paper 1: Impact of interpersonal

skills training Hunt, J. W., & Baruch, Y., (2003). Developing top

managers: The impact of interpersonal skills

training, Journal of Management Development,

22(8), 729752.

This work represents an approach to the study of general managerial training effectiveness and

personal skills evaluation.

Program: A concentrated five-day training and development workshop for top-level managers

(very different backgrounds, countries, and

industries).

Paper 1: Impact of interpersonal skills

training

The research focused on three questions about training and development:

Style theorists: Are leadership skills more relevant than technical skills?

Training and development practitioners: What is the actual value of training and how to measure

it?

Competency theorists: Are there skills or competencies that differentiate effective from less effective managers?


training

Hypotheses

H1. The impact of interpersonal skills training on subsequent skill performance will improve

the effectiveness of those skills.

H2. The impact of interpersonal skills training on subsequent skill performance will be positive

but modest.

H3. The impact of interpersonal skills training on subsequent skill performance will vary across

different skills.


training 180 evaluation approach: Only direct reports

(1,808) were asked to assess the training

participants.

Pre- and post-training program assesment

Paper questionnaire to record their level of satisfaction with their managers interpersonal skills.

29 questions, representing the skill sets already discussed above, were ranked.

Four to six questions assessed each skill set, using a 5-point scale: From (a) very satisfied to

(e) very dissatisfied


training

Results:

The statistical tests did not deny the hypotheses

The results supported the three proposed hypotheses. The questionnaire respondents

(direct reports of the program participants),

noted various skills improved after the training

program.

Paper 2: Impact of a design management

training initiative

Bibby, L., Austin, S., & Bouchlaghem, D. (2006). The impact of a design management training

initiative on project performance. Engineering,

Construction and Architectural Management 13(1),

726.

This work is quite closer to the typical project management training program approach. The

authors studied the impact of a training initiative

in the Design Management Handbook (DMH) on

individual and project performance.


training initiative

The program consisted on a design management training initiative to improve performance in a

major UK civil and building design and construction

company.

The research intended to measure the impact of:

(a) the DMH training program

(b)critical practices application, and

(c) a suite of 25 tools on design management

performance across the company.


training initiative

No hypotheses were presented:The research refers a program assessment.

The methodology for research included:

(a)a structured questionnaire

(b)a design management maturity assessment,

and

(c) semi-structured interviews, and

(d)a case study.


training initiative

Results:

The study draws several conclusions regarding the usefulness of the handbook and the

handbook application significant impact on work

performance.

Doctoral Thesis Proposal (draft)

Problem Statement

An education industry related to project management has been rising in recent years.

Construction industrys overall investment in project management training programs is

estimated at 5 million dollars annually for external

training programs, and a similar amount for

assigned internal resources.

There is not a study confirming that performance has improved due to this training.

Purpose Statement

The purpose of this quantitative research study is to determine the impact of PMI training on

Peruvian teams professional and personal performance on engineering and construction

projects.

This study also intends to measure the impact of:

(a)The firms attitudes about training;

(b)The hierarchical level of training program

attendees; and

(c) The training program mode: internally or

externally to the company.

Conceptual Framework

Dependent Variables / Definitions

Based on Changchit, Holsapple, and Maddens (1999) experiment design, we will define three variables:

Effectiveness: accuracy of decision making based in PMBOK Guide training.

Efficiency: the time used to accurate decision making based in PMBOK Guide training.

Perception: own perceptions related to the PMBOK Guide training.

Dependent Variables

Efficiency

Perception

Effectiveness

Independent Variables / Definitions

Firm Attitude: Moderator Firm commitment in training programs, associated to man-hours invested for this purpose.

Seniority Level: Moderator Experience years in project management.

Type of course: Mediator Qualification of Internal or External conducted training program.

Performance Evaluation

The individual performance to be evaluated will be, for example:

(a) technical knowledge in project management,

(b)ability to create a Work Breakdown Structure

(WBS),

(c) capacity to define a project time schedule,

(d)accuracy for cost estimating,

(e)risk evaluation and assessment knowledge,

(f) communication techniques,

(g) interpersonal skills, among others.

Performance Evaluation

Project performance, on the other hand would consider:

(a)project plan knowledge,

(b)project baseline deviations management,

(c) schedule accomplishment,

(d)budget control,

(e)risk responses, and others.

Research Questions / Hypotheses

Question 1: Is the current educational offer accurate and satisfactory for the attendees? This

means that even the important training programs

offer, the actual customer expectative is

accomplished.

H1: The PMBOK Guide training program suitable (satisfactory) for the attendees.


Question 2: Do the attendees obtain new and practical knowledge for their project management

activities? In addition to the question 1, this

question intends to assure if the attendees

acquired the formal concepts and knowledge.

H2: The attendees gain knowledge on Project Management concepts and practices after the

training program.


Question 3: Is the training attitude towards the training program an success factor for the

individual performance improvement?

H3: The competences are improved if the firm attitude (quantified as investment or manhours)

towards training is positive.


Question 4: Is the training more effective for improving new entry level workers that senior ones? The training should be more significant for less experienced attendees due to the senior practitioners should have acquired the knowledge from the job routine.

H4: The seniority level is inversely correspondent to the competences improvement after the training.


Question 5: Is the external teaching training course more effective that

the in-house program? Finally, is the

external training company condition is

relevant for attendees? Are the

external lecturers more effective than

the internal ones?

H5: The external teaching approach is more effective than the internal (in

house) training.

Next Steps

Next Steps General Overview

To define tools and techniques

To define questionnaires

To define and contact study subjects

To deploy the study

Subjects of Study

Target: 300 project practitioners

Top five Engineering and Construction firms

Training programs of 80 lesson-hours at least

How to measure (E1 & E2)

In order to measure E1 and E2, two test groups will be defined, composed of at least 50 similar project

management practitioners each. The first group

will be named CG, control group; and the second,

PMT, project management trainees.


Testing will be performed using three typical cases development (individual). The first one will be applied for both groups, previous to

the PMBOK Guide training.

The second and the third typical studies will be applied after that. The second typical case will be used as essay and experience equalizer prior to the third typical case solving.

A typical case is a written simulation of several project management situations. Each question will have four possible responses, with only one correct response.


The results for the groups will be compared among groups in order to measure the PMBOK Guide

training impact among two different practitioners

groups; and among the same group, before and

after the training.

Effectiveness and efficiency will be quantified in accordance to typical case responses to the proposed questions.


GROUP GROUP

CG PMT

TYPICAL

CASE

A X

B X X COMPARISON

C

COMPARISON

XX

How to measure (P1)

For the perception variable, a questionnaire will be given to project management practitioners with

and without PMBOK Guide training.

The results in both comparisons will be evaluated with T-student test, as well for independent

samples as for paired samples.

The study will evaluate the influence of the variables regarding firm attitude (FA), seniority

level (SL), and type of course (TC) with a MANOVA

test.

How to process the results

The results in both comparisons will be evaluated with T-student test, as well for independent

samples as for paired samples.

The study will evaluate the influence of the variables regarding firm attitude (FA), seniority

level (SL), and type of course (TC) with a MANOVA

test.

Conclusions (so far)

Conclusions

Doctoral studies on PMBOK Guide impact are limited

Peruvian training impact studies are limited

To asses training impact is complex

To measure training impact is challenging

Tools and techniques could be hard to design and deploy

But it is worth enough (vale la pena)

Peru Video 2

Preguntas y respuestas

Informacin de contacto

Nombre: Renzo Toledo, MBA, PMP

Compaa/Organizacin: Cosapi

Telfono: 211-3500

Email: [email protected]

Sesin:

Documents

2012-RenzoToledo