Upload
hoangdieu
View
221
Download
3
Embed Size (px)
Citation preview
TRAINING IMPACT ASSESSMENT REPORT 2016-2017
DIRECTORATE: PEOPLE EMPOWERMENT
DRAFT REPORT
31 March 2017
Authors:
Ilse Eigelaar-Meets
Dr Amiena Bayat
1
TABLE OF CONTENTS
1. INTRODUCTION .................................................................................................................................. 4
2. EVALUATION METHODOLOGY AND ANALYTICAL FRAMEWORK ....................................................... 5
3. FINDINGS ............................................................................................................................................ 7
3.1 PROGRESSIVE DISCIPLINE FOR MANAGERS AND SUPERVISORS .................................................... 8
3.1.1 Description of Learning Programme ..................................................................................... 8
3.1.2 Level Two Analysis................................................................................................................. 9
3.1.3 Level Three Measurement .................................................................................................. 11
3.1.4 Summary .............................................................................................................................. 15
3.2 MANAGING ABSENCE IN THE WORKPLACE ................................................................................. 15
3.2.1 Description of Learning Programme .................................................................................... 15
3.2.2 Level Two Analysis ............................................................................................................... 15
3.2.3 Level Three Measurement ................................................................................................... 19
3.2.4 Summary .............................................................................................................................. 24
3.3 MEETING SKILLS FOR CHAIRPERSONS AND FACILITATORS .......................................................... 25
3.3.1 Description of learning process ........................................................................................... 25
3.3.2 Level Two Analysis ............................................................................................................... 26
3.3.3 Level Three Measurement ................................................................................................... 30
3.3.4 Summary .............................................................................................................................. 34
3.4 PROMOTION OF ADMINISTRATIVE JUSTICE ACT (PAJA) .............................................................. 34
2
3.4.1 Description of Learning Programme .................................................................................... 34
3.4.2 Level Two Analysis ............................................................................................................... 35
3.4.3 Level Three Analysis ............................................................................................................. 39
3.4.4 Summary .............................................................................................................................. 43
3.5 PERFORMANCE MANAGEMENT FOR SUPERVISORS .................................................................... 44
3.5.1 Description of Learning Programme .................................................................................... 44
3.5.2 Level Two Analysis ............................................................................................................... 44
3.5.3 Level Three Analysis ............................................................................................................. 48
3.5.4 Summary .............................................................................................................................. 52
3.6 PUBLIC SERVICE INDUCTION ........................................................................................................ 52
3.6.1 Description of Learning Programme .................................................................................... 52
3.6.2 Level Two Analysis ................................................................................................................... 53
3.5.3 Level Three Analysis ............................................................................................................. 57
3.6.4 Summary .............................................................................................................................. 61
3.7 POLICY ANALYSIS, DEVELOPMENT, IMPLEMENTATION ............................................................... 61
3.7.1 Description of Learning Programme .................................................................................... 61
3.7.2 Level Two Analysis ............................................................................................................... 62
3.7.3 Level Three Analysis ............................................................................................................. 66
3.7.4 Summary .............................................................................................................................. 70
3.8 RECRUITMENT & SELECTION FOR LINE MANAGERS .................................................................... 70
3.8.1 Description of Learning Programme .................................................................................... 70
3.8.2 Level Two Analysis ............................................................................................................... 71
3
3.8.3 Level Three Analysis ............................................................................................................. 75
3.8.4 Summary .............................................................................................................................. 80
4 CONCLUDING COMMENTS AND RECOMMENDATIONS .................................................................. 81
4
1. INTRODUCTION
The Chief Directorate People Training and Empowerment (PTE) measures the effectiveness
and subsequent impact of training offered at the Provincial Training Institute (PTI) to
individual learners and client departments within the Western Cape Government through a
Training Impact Assessment (TIA). This happens on an annual basis.
The Kirkpatrick Evaluation Model, which evaluates the impact of learning on 4 Levels, was
identified as a suitable model for the TIA process. However, the PTE assessed the impact of
training up to Level 3 only, which refers to knowledge gained (Level 2) and possible
behavioural changes (Level 3) pertaining to the application of gained knowledge and skills in
the workplace.
The aim of this report is to present the findings from the levels 2 and 3 assessments, which
was derived from the completed pre- and post-assessment questionnaires and individual and
group interviews conducted by the PTE Skills Development Facilitators (SDFs) with
approximately 632 learners and supervisors.
The goal of the assessment was to measure the impact of training in the province in terms of
knowledge gained and possible behavioural changes for the period 1 April 2015 - 31 March
2016, in respect of the following learning programmes (LP) offered by the PTI as from 1 April
2015:
1. Progressive Discipline for Managers & Supervisors
2. Managing Absence in the Workplace
3. Meeting skills for Chairpersons and Facilitators
4. Promotion for Administrative Justice Act (PAJA)
5. Performance Management for Supervisors
6. Public Induction 1- 5 (PSI)
7. Policy Analysis, Development, and Implementation
8. Recruitment & Selection or Line Managers
These LPs were regarded as part of the core strategic training needs, identified by the
management team in the Chief Directorate PTE and were delivered by both internal and
external facilitators.
5
The data collection and data capturing processes were undertaken by the Skills Development
Facilitators (SDF) from the People Empowerment (PE) directorate, whilst the creation of the
datasets, analysis of data and formal report writing remained the responsibility of the service
provider. Strong emphasis was however placed on the capacitation of the PE team in
capturing the data in SPSS (Statistical Programme for the Social Scientist), data analysis and
report writing.
2. EVALUATION METHODOLOGY AND ANALYTICAL FRAMEWORK1
This evaluation is based on the Kirkpatric Evaluation Model, also referred to as the Four Levels
of Evaluation. This model has been identified as a suitable model for the Training Impact
Assessment. The four levels of evaluation as defined in this model are: (1) the reaction of the
learner and their thoughts about the training experience, (2) the resulting learning and
increase in knowledge from the training experience, (3) the learner’s behavioural change and
improvement after applying the skills on the job and (4) the results or effects that the
learner’s performance has on the business. Even though the PTE only measures Levels 2 and 3
as part of its TIA, all four levels of Kirkpatrick’s Model are discussed below.
LEVEL 1: REACTION
The measurement in level 1 solicits opinions of the learning experience following a training
event or training. Typical questions will measure the degree to which learners found the
experience valuable and relevant, whether they felt engaged, their assessment of the quality
of the presentation materials and the venue. This level of measurement is viewed as
important, as it aids in understanding how well the training was received by the audience. It
also assists with improving future training as well as identifying further areas or topics that
are missing in the content that needs to be included in future training.
1 Sources for information the Kirkpatrick scale:
1. Kirkpatrick’s Four-level Training Evaluation Model,
http://www.mindtools.com/pages/article/kirkpatrick.htm;
2. Four Levels of Evaluation, https://www.trainingindustry.com/wiki/entries/four-levels-of-evaluation.aspx
6
LEVEL 2: LEARNING
Level 2 measures the degree to which learners acquired the intended knowledge, skills and
attitudes during their training. At this level, the increase in their knowledge as a result of the
training is measured. Measurement on this level is informed by the training objectives and is
typically conducted as pre- and post- measures, although it can also be conducted by means
of a post test only. The objective of measurement on this level is to provide an indication of
what was learned and not learned. This level of measurement leaves the training
administrators and presenters with information on how to best restructure and/or adapt the
training to ensure that the training objectives are met in the future.
LEVEL 3: BEHAVIOUR
Level 3 measures the degree to which the learner’s behaviour has changed as a result of the
training. Behaviour in this context should be understood as the ability of the learner to apply
what he or she has learned in the training to their specific work environment. A note of
caution is necessary here, as it is important to recognise that behaviour can only change if
conditions are favourable. A lack of behavioural change does not necessarily mean that the
learner did not learn anything, but may simply be as a result of the work environment which
does not allow for the application of newly acquired knowledge. On a personal level, a learner
might also simply not have the desire to apply what he/she has been taught.
LEVEL 4: RESULTS
Measurement on level 4 seeks to determine the tangible results of the training impact based
on outcomes that the organisation important for its effective functioning, such as: improved
quality and efficiency, increased productivity, higher employee retention, and higher morale.
As indicated earlier, for the purpose of this evaluation the PTE collected data applicable to
levels two and three of the Kirkpatric Evaluation Model. For measurement on level two i.e.,
assessing the level of learning that took place, learners were requested to complete a pre-
and post course evaluation, prior to and immediately following the completion of the training.
For measurement on level three i.e., measuring the impact of the training on the behaviour of
7
the learner, both learners and supervisors were requested to complete a questionnaire
provided some time after the training was conducted. The data was subsequently captured in
SPSS, cleaned, verified and analysed.
In measuring the impact of the learning programme on the knowledge and behaviour of
participants, respondents were asked to respond to a set of statements by selecting an
appropriate response set on a five point scale commonly referred to as a Likert scale. Dr.
Rensis Likert developed a procedure for measuring attitudinal scales in 1932. The original
Likert scale used a series of questions with five response alternatives: strongly approve (1),
approve (2), undecided (3), disapprove (4), and strongly disapprove (5). He combined the
response from the series of questions to create an attitudinal measurement scale and based
his analysis on the composite score from the series of questions that represented the
attitudinal scale. Individual questions were not analysed2.
Likert scale data is categorised as ordinal scale observations. Ordinal scale observations are
ranked in some measure of magnitude in that numbers are assigned to groups expressing a
‘greater than’ or ‘smaller than’ relationship. However, how much greater or smaller is not
specified, the numbers only indicate an order from high to low or vice versa. Given the ordinal
nature of Likert scale data, descriptive statistics recommended for this measurement scale
include the establishment of a mode3 or median4 for trend and frequencies for variability5.
3. FINDINGS
This section presents the findings for the respective training evaluations. As indicated in the
above section, the evaluation measurement applied in this evaluation study is based on the
Kirkpatric Evaluation Model. Measurement in this report is, however, confined to level two
and three measurement as defined in this evaluation model.
In the discussion, research findings will first be discussed for each individual training
programme, showing (a) the impact it has had on the knowledge gained of participants (level
2 Analyzing Likert Data, Journal of Extension, April 2012, Vol. 50 (20), Article #2TOT2, www.joe.org
3 Mode refers to the response that occurs most often
4 Median refers to the average responses provided
5Analyzing Likert Data, Journal of Extension, April 2012, Vol. 50 (20), Article #2TOT2, www.joe.org
8
2) as reported by the participants themselves and, (b) to what extend the training has had an
impact on the behaviour of participants (level 3) as reported by the participants themselves as
well as their respective supervisors.
Given the ordinal nature of the Likert scale items employed here, analysis will focus on
descriptive statistics. These statistics will describe firstly; the main tendencies as observed in
the data by means of the mean and mode scores and secondly; illustrate variability in
responses by analysing the frequencies of responses within statements.
3.1 PROGRESSIVE DISCIPLINE FOR MANAGERS AND
SUPERVISORS
3.1.1 Description of Learning Programme
The target group for this learning programme include all Government Western Cape
supervisors and managers. The purpose of this learning programme was to equip
supervisors/mangers to effectively implement progressive discipline in the workplace focusing
on the following:
The Legislative framework that prescribes Progressive Discipline in the Public Service
Duties of the parties in the employment relationship
Acts of misconduct
Applying progressive discipline
Applying progressive discipline to acts of misconduct
9
3.1.2 Level Two Analysis
In testing the impact of the training programme on knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
In measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
asked to respond by rating their understanding of each core aspect of the course content as
either Poor (1), Average (2), Good (3) , Very Good (4) and Excellent (5). The core aspects
tested to which respondents had to rate their level of understanding, were:
1. Legislative Framework that Prescribes Progressive Discipline in the Public Service
2. The Principles of the Disciplinary Code and Procedure
3. Duties of Role Players in the Progressive Discipline
4. Duties and roles of the supervisor in Progressive Discipline
5. Duties and roles of the employee in Progressive Discipline
6. Duties and roles of the Trade Union in Progressive Discipline
7. Applying Progressive Discipline in the Workplace
8. What Constitute an act of Misconduct
9. Applying Progressive Discipline to Acts of Misconduct
Seventy respondents completed a pre- and post-test for this learning programme. The
analysis shows a general increase in the mean and mode scores for all nine defined core
components of the training programme (Table 1.1). Comparing the total pre- and post –test
mean scores, an increase of approximately two ‘points’ is noted, with a total pre- and post
test score of two and four respectively. The higher total post-test score confirms the
accumulation of a great deal of knowledge with a growth in the overall understanding of the
subject matter from an “Average” to a “Very Good” understanding. The same trend is
illustrated for each of the nine individual core competencies.
With regards to the variability in responses Figure 1.1 and Figure 1.2 illustrate limited variance
between statements with the frequency analysis clearly illustrating an increase in the level of
10
knowledge for all aspects measured. This outcome strongly supports that the defined training
outcomes and objectives set for this training programme were achieved.
Table 1.1:Pre- and Post-measurements - Progressive discipline for supervisors and managers
Code Statement
Pre-test Post-test
Mean Mode Mean Mode
1 Legislative framework that prescribes Progressive Discipline in the Public Sector 2.26 2 3.84 4
2 Principles of the disciplinary code and procedure 2.33 2 3.99 4
3 Duties of role players in the progressive discipline 2.31 2 3.97 4
4 Duties of Supervisor in progressive discipline 2.54 2 4.06 4
5 Duties of the employee in progressive discipline 2.50 2 4.04 4
6 Duties of the trade union in progressive discipline 2.20 2 3.89 4
7 Applying progressive discipline in the workplace 2.31 2 3.97 4
8 What constitute an act of Misconduct 2.34 2 3.96 4
9 Applying progressive discipline to acts of misconduct 2.13 2 3.93 4
Total means score 2.3 4.0 N=70
Figure 1.1: Pre-measurement - Progressive discipline for supervisors and managers
24.3 22.9 20.0 15.7 18.6 28.6
20.0 15.7 24.3
38.6 35.7 42.9 38.6 35.7
40.0 44.3 47.1
47.1
25.7 30.0 24.3
22.9 24.3
17.1 20.0 25.7 20.0
10.0 8.6 11.4 21.4 20.0 11.4 15.7 10.0 8.6
1 2 3 4 5 6 7 8 9
Pre-measurement
Poor Average Good Very good Excellent
11
Figure 1.2: Post-measurement - Progressive discipline for supervisors and managers
3.1.3 Level Three Measurement
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learned to their specific work environment. This impact was
tested on two levels, firstly; a subjective reflective measurement (self-evaluation) provided by
the learner and secondly; an objective reflective measurement provided by the supervisors of
learners (supervisor-evaluation).
Measuring the possible impact of the learning programme on the application of the learned
knowledge, both learners and their supervisors were required to respond to a similar list of
statements derived from the defined training objectives. For each statement the two
respondent groups had to indicate their response by means of the following five item Likert
Scale; (1) Skill application poorer than before, (2) Skill application on the same level as before
the training: not on required level, (3) Skill application on the same level as before the training:
on required level, (4) Skill application better than before the training and, (5) Skill application
much better than before the training. In the event where respondents opted for either option
one or two to a statement, they were asked to provide an explanation.
27.1 22.9 27.1 24.3 24.3 27.1
25.7 25.7 30.0
47.1 47.1 44.3
41.4 42.9 40.0 47.1 44.3 42.9
21.4 27.1 27.1 32.9 31.4 27.1 25.7 27.1 25.7
1 2 3 4 5 6 7 8 9
Post-measurement
Poor Average Good Very good Excellent
12
The statements to which respondents had to respond to were:
1. The incumbent is /I am able to apply the legislative framework that prescribes
Progressive Discipline in the Public Service
2. The incumbent is /I am able to apply the principles of the Disciplinary Code and
Procedure
3. The incumbent is /I am able to perform the role of the supervisor in the progressive
discipline process
4. The incumbent is /I am able to guide employees in terms of their role in the
progressive discipline process
5. The incumbent is /I am able to explain the role of the Trade Union representative in
the progressive discipline process
6. The incumbent is /I am able to apply progressive discipline in the workplace
7. The incumbent is /I am able to identify an act of misconduct in the workplace
8. The incumbent is /I am able to apply progressive discipline to act of misconduct
Forty seven course participants and thirty seven supervisors completed level three
questionnaires, measuring the impact of the training course in terms of the ability of the
course participant to apply what was learned in the work environment. Table 1.2 compares
the total and individual mean and mode scores for core proficiency in the competencies
defined for the course, as reported by both course participants and their respective
supervisors.
The general scores illustrated for both respondent groups compare well (4.09 and 4.27 for the
supervisor and participant groups respectively) with both exhibiting mode scores four (skill
application better than before the training) for all the core competencies defined. From these
scores it is clear that the learning programme did indeed have a positive impact on the ability
of training participants to perform their job functions.
Figures 1.3 and 1.4 illustrate the variance in the responses by means of the frequencies of
responses to the individual statements as scored by the two respondent groups. From these
illustrations it is clear that very little variance exist between the two groups with the majority
of responses in both groups indicating a better ability to apply all the defined competencies.
13
In the few cases where a respondent did indicate that their skills level were either poorer or
the same than before the training, the reason offered in nearly all cases (N=7) was that the
respondent experienced a lack of opportunity to apply what was learned. One respondent
indicated that the particular competency tested fell outside his/her work description.
Commenting about their ability to apply what they have learned in their respective work
environments, nine course participant respondents6 indicated a particular competency as
either outside their scope of work (N=8)7 or not possible to apply due to a lack of opportunity
to do so (N=43).
Table 1.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor evaluation
Mean Mode Mean Mode
1 Able to apply the legislative framework that prescribes progressive discipline in the public service 3.64 4 3.65 4
2 Able to apply the principles of the disciplinary code and procedure 3.62 4 3.81 4
3 Able to perform the role of the supervisor in the progressive discipline process 3.66 4 3.70 4
4 Able to guide employees in terms of their role in the progressive discipline process 3.96 4 3.89 4
5 Able to explain the role of the trade union representative in the progressive discipline process 3.70 4 3.76 4
6 Can apply progressive discipline in the workplace 3.89 4 3.86 4
7 Can identify an act of misconduct in the workplace 4.26 4 4.16 4a
8 Can apply progressive discipline to acts of misconduct 3.79 4 3.78 4
Total mean score 3.8 3.8
Self-evaluation (N=47)
Supervisor-evaluation (N=37)
a. Multiple modes exist. The smallest value is shown
6 Department of Education (N=1); Department of Environmental Affairs and Planning (N=1); Department of
Health (N=4) and Department of the Premier (N=3). 7 In this instance the N-value refers to the number of competencies
14
Figure 1.3: Self-evaluation on impact of training (N=47)
Figure 1.4: Supervisor evaluation on impact of training (N=37)
17.0 19.1 21.3 12.8 10.6 12.8
4.3 17.0
17.0 12.8 10.6 8.5
27.7 6.4
6.4
8.5
51.1 55.3 48.9
48.9
42.6
59.6
48.9
53.2
14.9 12.8 19.1 29.8
19.1 21.3 40.4
21.3
1 2 3 4 5 6 7 8
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
16.2 16.2 18.9 10.8 10.8 13.5
5.4 16.2
18.9 13.5 16.2
13.5 21.6 13.5
13.5
16.2
48.6 43.2
40.5 51.4
48.6 45.9
40.5
40.5
16.2 27.0 24.3 24.3 18.9
27.0 40.5
27.0
1 2 3 4 5 6 7 8
Supervisor- evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
15
3.1.4 Summary
The analysis above clearly shows the positive impact of the learning programme on both
measurement levels, that is the intent to (a) convey a specific set of knowledge and skills to
learners (level 2) and (b) enabling learners to apply what has been learned to their respective
work environments (level 3). The learning programme was further shown to have achieved all
respective defined training outcomes and associated competencies with an overwhelming
majority of learners indicating (i) an increase in knowledge and (ii) a stronger ability to apply
each defined outcome. The impact on the latter measurement was strongly confirmed by
both course participants and their respective supervisors.
3.2 MANAGING ABSENCE IN THE WORKPLACE
3.2.1 Description of Learning Programme
The target group for this learning programme include all Government Western Cape
employees on levels four to twelve. The purpose of this learning programme is to equip
managers and supervisors to effectively manage absenteeism in the workplace. The learning
programme focussed on the following aspects:
Leave prescriptions and categories
Absenteeism
Discipline
3.2.2 Level Two Analysis
In testing the impact of the training programme on the knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
In measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
16
asked to respond by rating their understanding of each core aspect of the course content as
either Poor (1), Average (2), Good (3) , Very Good (4) and Excellent (5). The core aspects
tested to which respondents had to rate their level of understanding, were:
1. Legislative framework and policies that regulate leave of absence in the SA Public
Service
2. Annual and normal sick leave
3. Eight week rule
4. Temporary and Permanent Incapacity leave
5. Acceptance of medical certificates
6. Incapacity due to substance abuse leave
7. Leave for occupational injuries and disease
8. Pre-natal and Maternity leave
9. Special and Family responsibility leave
10. Leave for Office Bearers or Shop Stewards of recognised employee organisations
11. Unpaid leave
12. Process for applying for leave
13. Process for Temporary incapacity leave
14. Process for administration of leave
15. Definition of Absenteeism
16. Forms of Absenteeism
17. Roles and responsibilities of supervisors
18. Roles and responsibilities of employees
19. Establishing a pattern for dealing with abseentism
20. Authorised and unauthorised absence
21. Abscondment
22. Difference between unauthorised absence and being absent without a valid reason
23. Disciplinary process for transgressions regarding absence
24. Sanctions regarding unauthorised absence and non-observance of office rules
25. Sanctions regarding substance abuse
26. Burden of proof
17
A total of thirty eight respondents completed a pre- and post-test for this learning
programme. The analysis shows a general increase in the mean and mode scores for all
twenty six defined core aspects of the training programme (Table 2.1). Comparing the total
pre- and post –test mean scores, an increase of just less than two ‘points’ is noted, with a
total pre- and post test score of 2.4 and four respectively. The higher total post-test score
confirms the accumulation of a great deal of knowledge with growth in the general
understanding of the subject matter increasing from “Average” to a “Very Good”
understanding. Considering the individual core learning areas for this learning programme,
the analysis shows an increase in knowledge for all the defined core competencies.
With regards to variability in responses, Figure 2.1 and Figure 2.2 illustrate limited variance
between statements with the frequency analysis clearly illustrating an increase in the level of
knowledge for all aspects measured. This outcome strongly supports the observation that the
defined training outcomes and objectives set for this training programme were achieved.
Table 2.1: Pre- and Post-measurements –Managing absence in the workplace
Code Statement
Pre-test Post-test
Mean Mode Mean Mode
1 Legislative framework and policies that regulate leave of absence in the SA public Service 2.13 2 3.79 4
2 Annual and normal sick leave 2.97 3 4.29 4
3 Eight week rule 2.66 3 4.26 4
4 Temporary and permanent incapacity leave 2.11 2 4.00 4
5 Acceptance of medical certificates 2.63 2 4.11 4
6 Incapacity due to substance abuse leave 1.97 1 3.76 4
7 Leave for occupational injuries and diseases 2.24 2 3.74 4
8 Pre-natal and maternity leave 2.47 2 4.11 4
9 Special and family responsibility leave 2.76 3 4.32 4
10 Leave for office bearers or shop stewards of recognised employee organisations 2.21 2 3.95 4
11 Unpaid leave 2.42 2a 4.03 4
12 Process for applying for leave 3.00 3 4.21 4
13 Process for temporary incapacity leave 2.26 2 3.79 4
14 Process for administration of leave 2.26 2 3.95 4
15 Definition of absenteeism 2.61 3 4.32 4
18
16 Forms of absenteeism 2.53 2 4.24 4
17 Roles and responsibilities of supervisors 2.61 2 4.22 4
18 Roles and responsibilities of employees 2.74 2a 4.16 4
19 Establishing a pattern for dealing with absenteeism 2.45 2 4.13 4
20 Authorised and unauthorised absence 2.53 2 4.03 4
21 Abscondment 2.24 2 4.21 4
22 Difference between unauthorised absence and being absent without a valid reason 2.26 2 4.13 4
23 Disciplinary process for transgressions regarding absence 2.11 2 4.00 4
24 Sanctions regarding unauthorised absence and non-observance of office rules 1.92 2 3.87 4
25 Sanctions regarding substance abuse 1.89 2 3.79 4
26 Burden of proof 1.95 1a 3.84 4
Total mean 2.40
4.0
N=38
a. Multiple modes exist. The smallest value is shown
Figure 2.1: Pre-measurement - Managing absence in the workplace
26.3
13.2
26.3
5.3
44.7
26.3
13.2 10.5
23.7 21.1
2.6
18.4 21.1
10.5 15.8
5.3 7.9 15.8
7.9
23.7 21.1 28.9
34.2 36.8 39.5
44.7
31.6
28.9
47.4
50.0
31.6
42.1
42.1
21.1
44.7
34.2
28.9
52.6 42.1
31.6
34.2 47.4
36.8
47.4 55.3
47.4 52.6
47.4
47.4 47.4 39.5
21.1
47.4
42.1
15.8
28.9
10.5 18.4
28.9
52.6
21.1
34.2
39.5
15.8 26.3
44.7 31.6 31.6
36.8
18.4 18.4
18.4 10.5 10.5
10.5 7.9 10.5
5.3
13.2 10.5
10.5
7.9 7.9 7.9 15.8 13.2
7.9 2.6
23.7
10.5 10.5 13.2
18.4 13.2 10.5 13.2 13.2
2.6 10.5 10.5
7.9 5.3 7.9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Pre-measurement
Poor Average Good Very good Excellent
19
Figure 2.2: Post-measure - Managing absence in the workplace
3.2.3 Level Three Measurement
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learnt in the training to their specific work environment. This
impact was tested on two levels, firstly; a subjective reflective measurement provided by the
participant and secondly; an objective reflective measurement provided by the supervisors of
learners.
In measuring the impact of the learning programme in the application of the learned
knowledge, both learning programme participants as well as their supervisors were required
to respond to a similar list of statements derived from the defined training objectives. For
each statement the two respondent groups had to indicate their response by means of the
following five item Likert Scale; (1) Skill application poorer than before, (2) Skill application on
the same level as before the training: not on required level, (3) Skill application on the same
level as before the training: on required level, (4) Skill application better than before the
training and, (5) Skill application much better than before the training. If respondents scored a
statement as either one or two, that is (1) Skill application poorer than before or (2) Skill
application on the same level as before the training: not on required level, they were
31.6
10.5 10.5
23.7 13.2
34.2 28.9
13.2 10.5
21.1 26.3
13.2
34.2
15.8
10.5 10.5
8.1 7.9 18.4 15.8
7.9 13.2
21.1 31.6 34.2 26.3
42.1
50.0 52.6
52.6
55.3
39.5 52.6
55.3
47.4
47.4 44.7
52.6
36.8
57.9
47.4 47.4
62.2 68.4 50.0
57.9
55.3 52.6
42.1
42.1 44.7
47.4
21.1
39.5 36.8
23.7 28.9
21.1 13.2
28.9
42.1
26.3 28.9 34.2
23.7 21.1
42.1 39.5 29.7
23.7 31.6
23.7 34.2 31.6 31.6
23.7 18.4 21.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Post-measure
Poor Average Good Very good Excellent
20
requested to indicate the main reason for this rating. The statements, to which respondents
had to respond, were:
1. The incumbent is /I am able to apply the legislative framework and policies that
regulate leave of absence in the Public Service
2. The incumbent is /I am able to manage the annual and normal sick leave of my team
3. The incumbent is /I am able to manage the eight week rule
4. The incumbent is /I am able to manage the temporary and permanent incapacity leave
of my team
5. The incumbent is /I am able to determine the validity of the medical certificates
6. The incumbent is /I am able to manage incapacity due to substance abuse
7. The incumbent is /I am able to ascertain when leave for occupational injuries and
diseases will apply
8. The incumbent is /I am able to manage the pre-natal an maternity leave of my team
9. The incumbent is /I am able to mange special and family responsibility leave of my
team
10. The incumbent is /I am able to ascertain when leave for Office Bearers or Shop
Stewards or recognised employee organisations will apply
11. The incumbent is /I am able to manage unpaid leave
12. The incumbent is /I am able to guide the process for applying for leave
13. The incumbent is /I am able to guide the administrative process for temporary
incapacity leave
14. The incumbent is /I am able to perform the roles and responsibilities of a supervisor
15. The incumbent is /I am able to guide employees with regard to their roles and
responsibilities
16. The incumbent is /I am able to identify an absenteeism pattern with regard to the
leave of team members
17. The incumbent is /I am able to distinguish between authorised and unauthorised
absence
18. The incumbent is /I am able to guide the Abscondment process
19. The incumbent is /I am able to differentiate between unauthorised absence and being
absent without a valid reason
21
20. The incumbent is /I am able to follow the disciplinary process for transgressions
regarding leave of absence
21. The incumbent is /I am able to implement disciplinary actions regarding unauthorised
absence and non-observance of office rules
22. The incumbent is /I am able to implement disciplinary actions regarding substance
abuse
23. The incumbent is /I am able to explain the burden of proof principle within
government
Sixteen course participants and twelve supervisors completed level three questionnaires,
evaluating the impact of the training course in terms of the ability of the course participant to
apply what was learned to the work environment. Table 2.2 compares the mean and mode
scores evaluating general proficiency as well as competencies in particular tasks related to the
subject matter, as reported by both course participants and their respective supervisors.
The average mean scores presented for both respondent groups compare well; 3.8 and 3.9 for
the participant and supervisor respondent groups respectively, with both exhibiting mode
scores of four (skill application better than before the training). These scores illustrate that in
general skill, application was better after training than before. From these scores, it is clear
that the learning programme did have a positive impact on the ability of training participants
to perform their job functions. Considering the individual competencies, responses by both
course participants and supervisors consistently indicated a higher competency level than
before the training.
Figures 2.3 and 2.4 illustrate the variance in the responses by means of the frequencies of
responses to the individual statements as scored by the two respondent groups. From these
illustrations, it is clear that very little variance exist between the two groups with the majority
of responses in both groups indicating a better ability to apply all the defined competencies.
Commenting about their ability to apply what they have learned in their respective work
environments, five course participant respondents (Department of Health and Department of
22
the Premier) indicated a particular competency as either outside their scope of work (N=14)8
or not possible to apply due to a lack of opportunity to do so (N=28).
Table 2.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor-evaluation
Mean Mode Mean Mode
1 Able to apply the legislative framework and policies that regulate leave of absence in the Public Service 3.94 4 4.00 4
2 Able to manage the annual and normal sick leave of my team 4.06 4 3.92 4
3 Able to manage the eight week rule 3.94 4 4.08 4
4 Able to manage the temporary and permanent incapacity leave of my team 3.25 4 3.67 4
5 Can determine the validity of the medical certificates 4.00 4 4.00 4
6 Can manage incapacity due to substance abuse 3.38 4 3.58 4
7 Able to ascertain when leave for occupational injuries and disease will apply 3.31 4 3.92 4
8 Able to manage the pre-natal and maternity leave of my team 3.50 4 3.50 4
9 Able to manage special and family responsibility leave of my team 3.88 4 4.08 4
10 Able to ascertain when leave for office bearers or shop stewards of recognised employee organisations will apply 3.63 4 3.67 4
11 Able to manage unpaid leave 4.00 4 3.75 4
12 Able to guide the process of applying for leave 4.25 4 4.08 4
13 Able to guide the administrative process for temporary incapacity leave 3.75 4 4.27 4
14 Able to administer the leave process 4.31 4 4.25 4
15 Able to perform the roles and responsibilities of a supervisor 4.38 4 4.00 4
16 Able to guide employees with regard to their roles and responsibilities 4.25 4
a 3.92 4
17 Able to identify an absenteeism pattern with regard to the leave of team members 4.13 4 4.25 4
18 Able to distinguish between authorised and unauthorised absence 4.31 5 4.33 4
8 In this instance the N-value refers to the number of competencies
23
19 Able to guide the abscondment process 3.50 4 3.75 4
20 Can differentiate between unauthorised absence and being absent without a valid reason 3.88 4 4.25 4
21 Able to follow the disciplinary process for transgressions regarding leave of absence 3.88 4 4.00 4
22 Can implement disciplinary actions regarding unauthorised absence and non-observance to office rules 3.81 4 4.08 4
23 I can implement disciplinary actions regarding substance abuse 3.31 4 3.42 4
24 Able to explain the burden of proof principles within government 3.56 4 3.92 4
Total mean 3.8 3.9
Self-evaluation (N=16)
Supervisor evaluation (N=12)
a. Multiple modes exist. The smallest value is shown
Figure 2.3: Self-evaluation on impact of training (N=16)
12.5 6.3
25.0 25.0
6.3 12.5
25.0
6.3 12.5
6.3 6.3 6.3 18.8
6.3 6.3 12.5
25.0 12.5 12.5
6.3 12.5
31.3
12.5
6.3
18.8
6.3
6.3
6.3
18.8
6.3
25.0
6.3 6.3 6.3 6.3 6.3
18.8
12.5 18.8 6.3
25.0
25.0
81.3
43.8
62.5
37.5
75.0 50.0
43.8
62.5
62.5 50.0
43.8
62.5
37.5
56.3 50.0
43.8 56.3
37.5
56.3
68.8 56.3 68.8
43.8 56.3
6.3
37.5
18.8 6.3
12.5 18.8
31.3 31.3 18.8 18.8
31.3 31.3 25.0
37.5 43.8 43.8
31.3
50.0
6.3 12.5
18.8 12.5
6.3 6.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
24
Figure 2.4: Supervisor-evaluation on impact of training (N=12)
3.2.4 Summary
The analysis above clearly shows the positive impact of the learning programme on both
measurement levels, that is the intent to (a) convey a specific set of knowledge and skills to
learners (level 2) and (b) enabling learners to apply what has been learnt to their respective
work environments (level 3). The learning programme was further shown to have achieved all
respective defined training outcomes and associated competencies with an overwhelming
majority of learners indicating (i) an increase in knowledge and (ii) a stronger ability to apply
each defined outcome. The impact on the latter measurement was strongly confirmed by
both course participants and their respective supervisors.
8.3 16.7
8.3 16.7
8.3 16.7
8.3 16.7
8.3 16.7 16.7
8.3 8.3 16.7
8.3 8.3 8.3 16.7
8.3
8.3
8.3 16.7
8.3
16.7 16.7 8.3
8.3
9.1 8.3
8.3
25.0
8.3
25.0
8.3
58.3 58.3 66.7
66.7
75.0
58.3
66.7
66.7
66.7
50.0 58.3
50.0 54.5 58.3
58.3 58.3
75.0 66.7
50.0
58.3
75.0 66.7
58.3
66.7
25.0 25.0 25.0
8.3 16.7
8.3 16.7
25.0 16.7 16.7
33.3 36.4 33.3 25.0 25.0 25.0
33.3
16.7
33.3
16.7 25.0
16.7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Supervisor-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
25
3.3 MEETING SKILLS FOR CHAIRPERSONS AND FACILITATORS
3.3.1 Description of learning process
The target group included all Government of the Western Cape employees on salary levels
eight to twelve, who are chairpersons and facilitators at meetings. The purpose of this
learning programme was to ensure that chairpersons and facilitators understood their roles
and responsibilities in chairing/facilitating meetings effectively to achieve set outcomes. The
programme consisted of three modules:
Module 1: Chairperson and facilitators
Introduction
What is the role of the chairperson/facilitator?
Chairperson and facilitator skills
Different approaches for different meetings
Module 2: Chairperson/facilitator rules during the meeting cycle
Introduction
Preparation for meetings
Conducting effective meetings by achieving the goals set
Conducting meetings
Follow-up actions
Module 3: General tips for chairpersons/facilitators
Introduction
Why meetings fail
List of process points
26
The outcomes of the learning programme are defined as follows:
Plan and prepare a meeting
Demonstrate an understanding of the roles of a chairperson/facilitator to lead and
manage group interactions during a meeting
Demonstrate the use of various methods to achieve meeting outcomes
The target group for this learning programme was supervisors and managers. The
purpose of this learning programme was to equip supervisors/mangers to effectively
implement progressive discipline in the workplace.
3.3.2 Level Two Analysis
In testing the impact of the training programme on knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
In measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
asked to respond by rating their understanding of each core aspect of the course content as
either Poor (1), Average (2), Good (3) , Very Good (4) and Excellent (5). The core aspects to
which respondents had to rate their level of understanding was:
1. The role of Chairperson and Facilitator
2. Skills of a Chairperson and Facilitator
3. Different approaches in different types of meetings to facilitate effective outcome
4. Planning and preparation to facilitate a meeting
5. How to set ground rules for a meeting
6. How to determine and structure the rules of procedure during a meeting
7. Facilitation of a meeting using various techniques
8. How to deal with different people in different situations
9. How to help groups reach consensus
10. How to ensure participation
27
11. How to conclude a meeting
12. The importance of follow-up action after the meeting
13. Self assessment to evaluate the performance of the Chairperson
14. Reasons contributing to poor meeting performance
15. Process points that must be followed to conduct an effective meeting
Sixty respondents completed a pre- and post-test for this learning programme. The analysis
show a general increase in the mean and mode scores for all fifteen defined core aspects of
the training programme (Table 3.1). Comparing the total pre- and post –test mean scores, an
increase of approximately two ‘points’ is noted, with a total pre- and post test score of 2.3
and 4.1 respectively. The higher total post-test score of nearly two ‘points’ confirms the
accumulation of a great deal of knowledge with an increase in the general understanding of
the subject matter from an “Average” to a “Very Good” understanding. The same trend is
illustrated for each of the nine individual core competencies.
With regards to the variability in responses Figure 3.1 and Figure 3.2 illustrate limited variance
between statements with the frequency analysis clearly illustrating an increase in the level of
knowledge for all aspects measured. This outcome strongly supports the notion that the
defined training outcomes and objectives set for this training programme were achieved.
28
Table 3.1: Pre- and Post-measurements – Meeting skills for chairpersons & facilitators
Code Statement
Pre-test Post-test
Mean Mode Mean Mode
1 The role of chairperson and facilitator 2.83 3 4.33 4
2 Skills of a chairperson and facilitator 2.45 2 4.15 4
3 Different approaches in different types of meetings to facilitate effective outcomes 2.10 2 3.97 4
4 Planning and preparation to facilitate a meeting 2.62 3 4.27 4
5 How to set ground rules for a meeting 2.53 2 4.43 5
6 How to determine and structure the rules of procedure during a meeting 2.13 2 4.10 4
7 Facilitation of a meeting using various techniques 2.00 2 3.95 4
8 How to deal with different people in different situations 2.23 2 3.93 4
9 How to help groups reach consensus 2.30 2 3.97 4
10 How to ensure participation 2.47 2 4.08 4
11 How to concluded meetings 2.45 2 3.85 4
12 The importance of follow-up action after the meeting 2.45 2 4.22 4
13 Self-assessment to evaluate the performance of the chairperson 2.02 2 3.97 4
14 Reasons contributing to poor meeting performance 2.02 2 4.17 4
15 Process points that must be followed to conduct an effective meeting 2.08 2 4.12 4
Total mean 2.31
4.10
N=60
29
Figure 3.1: Pre-measurement - Meeting skills for chairpersons & facilitators
Figure 3.2: Post-measurement - Meeting skills for chairpersons & facilitators
1.7 3.3
25.0 15.0
8.3 20.0
30.0 18.3 16.7
8.3 8.3 10.0 22.0 25.4 20.3
33.3
55.0
43.3
30.0 46.7
55.0 43.3
51.7 50.0
48.3 46.7 43.3
59.3 49.2 55.9
45.0
35.0 28.3
36.7 30.0
18.3 23.3 20.0 21.7
33.3 38.3 40.0
13.6 23.7 18.6 20.0
6.7 3.3
15.0 13.3 5.0 3.3
8.3 10.0 8.3 5.0 5.0 5.1 1.7 5.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Pre-measurement
Poor Average Good Very good Excellent
1.7 10.0 13.3
25.0 11.7 6.7
21.7 28.3 30.0
22.0 15.0
35.0
11.7 22.0
13.6 15.3
46.7
58.3
53.3
50.0
43.3
46.7
48.3 46.7 59.3 61.7
45.0
55.0
54.2
55.9 57.6
43.3
28.3 21.7
38.3 50.0
31.7 23.3 23.3 18.6 23.3 20.0
33.3 22.0
30.5 27.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Post-measurement
Poor Average Good Very good Excellent
30
3.3.3 Level Three Measurement
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learnt in the training to their specific work environment. This
impact was tested on two levels, firstly; a subjective reflective measurement provided by the
participant and secondly; an objective reflective measurement provided by the supervisors of
learners.
In measuring the impact of the learning programme in the application of the learned
knowledge, both learning programme participants as well as their supervisors were required
to respond to a similar list of statements derived from the defined training objectives. For
each statement the two respondent groups had to indicate their response by means of the
following five item Likert Scale; (1) Skill application poorer than before, (2) Skill application on
the same level as before the training: not on required level, (3) Skill application on the same
level as before the training: on required level, (4) Skill application better than before the
training and, (5) Skill application much better than before the training. If respondents scored a
statement as either one or two, that is (1) Skill application poorer than before or (2) Skill
application on the same level as before the training: not on required level, they were
requested to indicate the main reason for this rating. The statements, to which respondents
had to respond, were:
1. The incumbent is /I am able to execute/perform the role of the Chairperson/Facilitator
2. The incumbent is /I am able to apply the skills as a Chairperson and Facilitator
3. The incumbent is /I am able to apply different approaches in different types of
meetings to ensure effective outcomes of the meetings
4. The incumbent is /I am able to do proper planning and preparation to facilitate a
meeting
5. The incumbent is /I am able to set ground rules for a meeting
6. The incumbent is /I am able to determine and structure the rules pertaining to the
procedures during a meeting
7. The incumbent is /I am able to deal with different people in different situations
8. The incumbent is /I am able to assist groups to reach consensus
31
9. The incumbent is /I am able to involve all participants during a discussion at a meeting
10. The incumbent is /I am able to use various techniques to conduct a meeting
11. The incumbent is /I am able to do follow-up on actions after the meeting
12. The incumbent is /I am able to assess myself on my performance as a chairperson
13. The incumbent is /I am able to identify the reasons that contribute to a poor meeting
performance
14. The incumbent is /I am able to process points that must be followed to conduct an
effective meeting
Fifty-two course participants and forty-five supervisors completed level three questionnaires,
evaluating the impact of the training course in terms of the ability of the course participant to
apply what was learned to the work environment. Table 3.2 compares the mean and mode
scores evaluating general proficiency as well as particular competencies related to the subject
matter, as reported by both course participants and their respective supervisors.
The general scores illustrated for both respondent groups compare well (mean of 3.8 and 3.9
for the supervisor and participant groups respectively) with both exhibiting an overall mode
of four (skill application better than before the training). From these scores it is clear that the
learning programme did have a positive impact on the ability of training participants to
perform their job functions.
Figures 3.3 and 3.4 illustrate the variance in the responses by means of the frequencies of
responses to the individual statements as scored by the two respondent groups. From these
illustrations, it is clear that very little variance exist between the two groups with the majority
of responses in both groups indicating a better ability to apply all the defined competencies.
A total of six respondents (DECAS [N=1], Department of Health [N=3], Provincial Treasury
[N=1] and Department of Social Development [N=1]) commenting about the difficulty they
experience in applying what they have learned in their respective work environments. This
was mostly attributed to a competency falling outside their scope of work (N=26) 9 and a lack
of opportunity to do so (N=8).
9 In this instance the N-value refers to the number of competencies
32
Table 3.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor-evaluation
Mean Mode Mean Mode
1 Can execute/perform the role of the chairpersons/facilitator 3.96 4 3.96 4
2 Can apply the skills as a chairperson and facilitator 3.96 4 3.93 4
3
Can apply different approaches in different types of meetings to ensure effective outcomes of the meeting 3.76 4 3.89 4
4 Able to do proper planning and preparation to facilitate a meeting 4.02 4 4.02 4
5 Able to set ground rules for a meeting 4.02 4 3.96 4
6 Able to determine and structure the rules pertaining to the procedures during a meeting 3.94 4 3.87 4
7 Able to deal with different people in different situations 3.98 4 4.07 4
8 Able to assist groups to reach consensus 3.92 4 4.02 4
9 Able to involve all participants during a discussion at a meeting 3.94 4 3.91 4
10 Able to use various techniques to conduct a meeting 3.85 4 4.02 4
11 Able to do follow-up on actions after the meeting 3.90 4 4.09 4
12 Able to assess myself on my performance as a chairperson 3.75 4 3.84 4
13 Able to identify the reasons that contribute to a poor meeting performance 3.87 4 3.96 4
14 Able to apply the process points that must be followed to conduct an effective meeting 3.83 4 3.93 4
Total mean 3.9
3.7
Self-evaluation (N=52)
Supervisor-evaluation (N=45)
33
Figure 3.3: Self-evaluation on impact of training (N=52)
Figure 3.4: Supervisor-evaluation on impact of training (N=45)
11.5 15.4 29.4
11.5 19.2 25.0 19.6 19.2 13.5 21.2 21.2 26.9 19.2 21.2
57.7 55.8
52.9
57.7 53.8 50.0 56.9 57.7 61.5
61.5 48.1
51.9 57.7 63.5
23.1 23.1 13.7
25.0 25.0 23.1 21.6 19.2 19.2 13.5 25.0
15.4 17.3 11.5
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
13.3 13.3 28.9
15.2 26.7 35.6
21.7 15.6 26.7
15.6 15.6 17.8
22.2 24.4
64.4 66.7 46.7 60.9
51.1 42.2
50.0 60.0 48.9
60.0 53.3
60.0 60.0 57.8
17.8 15.6 22.2 21.7 22.2 22.2 28.3 22.2 22.2 22.2 28.9 15.6 17.8 17.8
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Supervisor-evaluation Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
34
3.3.4 Summary
The analysis above clearly shows the positive impact of the learning programme on both
measurement levels, that is the intent to (a) convey a specific set of knowledge and skills to
learners (level 2) and (b) enabling learners to apply what has been learned to their respective
work environments (level 3). The learning programme was further shown to have achieved all
respective defined training outcomes and associated competencies with an overwhelming
majority of learners indicating (i) an increase in knowledge and (ii) a stronger ability to apply
each defined outcome. The impact on the latter measurement was strongly confirmed by
both course participants and their respective supervisors.
3.4 PROMOTION OF ADMINISTRATIVE JUSTICE ACT (PAJA)
3.4.1 Description of Learning Programme
The target group for this learning programme include all Government of the Western Cape
employees, preferably those appointed on salary levels six and higher. The purpose of this
learning programme was to create awareness of the impact of the relevant Acts in the
application of administrative functions. The content of the training programme is as follows:
Promotion of Administrative Justice Act
Government
Judiciary
Executive
Administration
35
Promotion of Access to Information Act [PAIA]
Section 32 of the Constitution
National legislation
Purpose of PAJA
3.4.2 Level Two Analysis
In testing the impact of the training programme on knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
Measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
asked to respond by rating their understanding of each core aspect of the course content as
either Poor (1), Average (2), Good (3), Very Good (4) and Excellent (5). The core aspects to
which respondents had to rate their level of understanding, were:
1. Legislative framework relating to PAJA
2. The purpose of PAJA
3. Definition of administrative action
4. Definition of administrative justice
5. The decision making process
6. Preparing for a decision
7. Importance of a paper trail
8. Legal authority to make decisions
9. Delegation of powers
10. Developing a decision
11. Rules of natural justice (procedural fairness)
12. Taking action/ making the decision in terms of PAJA
13. Applying the law to the facts
14. Discretion
36
15. How to communicate the action or decision
16. The judicial review of decisions
17. The public participation process as stipulated in PAJA
Thirty eight respondents completed a pre- and post-test for this learning programme. The
analysis shows a general increase in the mean and mode scores for all nine defined core
aspects to the training programme (Table 4.1). Comparing the total pre- and post –test mean
scores, an increase of just below two ‘points’ is noted, with a total pre- and post test mean
score of 2.1 and 3.9 respectively. The higher total post-test score confirms the general
accumulation of knowledge with an illustrated growth in the general understanding of the
subject matter moving from a general “Average” to a “Very Good” understanding. The same
trend is recorded for each of the nine individual core competencies with particular strong
growth for two competencies; (1) Rules of natural justice (procedural fairness) and (2) Taking
action/making the decision in terms of PAJA. These two competencies showed above average
increase in knowledge with both showing an increase in knowledge moving from Poor (pre-
test) to Very Good (post-test).
With regards to the variability in responses Figure 4.1 and Figure 4.2 illustrate limited variance
between statements with the frequency analysis clearly illustrating an increase in the level of
knowledge for all aspects measured. This outcome strongly supports the notion that the
defined training outcomes and objectives set for this training programme were indeed
achieved.
37
Table 4.1: Pre- and Post-measurements - Promotion of Administrative Justice act (PAJA)
Code Statement
Pre-test Post-test
Mean Mode Mean Mode
1 Legislative framework relating to PAJA 1.66 2 3.73 4
2 The purpose of PAJA 2.00 2 4.08 4
3 Definition of administrative action 1.89 2 4.05 4
4 Definition of administrative justice 1.82 2 3.73 4
5 The decision making process 2.21 2 4.00 4
6 Preparing for a decision 2.29 2 3.95 4
7 Importance of a paper trail 2.58 3 4.11 5
8 Legal authority to make decisions 2.24 2 4.11 4
9 Delegation of powers 2.37 2 4.11 4
10 Developing a decision 2.11 2 3.97 4
11 Rules of natural justice (procedural fairness) 1.84 1 3.78 4
12 Taking action/making the decision in terms of PAJA 1.71 1 3.84 4
13 Applying the law to the facts 1.97 2 3.65 3
14 Discretion 2.24 2 3.57 3
15 How to communicated the action or decision 2.32 2 3.95 4
16 The judicial review of decisions 1.78 2 3.78 3a
17 The public participation process as stipulated in PAJA 1.81 1 3.84 3
Total mean 2.05
3.90
N=38
a. Multiple modes exist. The smallest value is shown
38
Figure 4.1: Pre-Measure - Promotion of Administrative Justice act (PAJA)
Figure 4.2: Post-Measure - Promotion of Administrative Justice act (PAJA)
42.1 31.6 31.6
36.8
15.8 18.4 21.1 28.9 26.3
31.6 42.1 44.7
31.6 28.9 18.9
40.5 43.2
52.6
39.5 50.0
47.4
52.6 42.1
26.3
34.2 31.6
36.8
39.5 42.1
47.4
31.6 43.2
45.9 37.8
2.6
26.3 15.8 13.2
26.3 31.6
31.6
23.7 26.3
21.1
13.2 10.5
13.2
28.9 27.0
8.1 13.5
2.6 2.6 2.6 2.6 5.3 7.9
15.8 10.5
10.5 10.5 2.6
2.6 7.9
7.9 8.1 5.4 5.4 5.3 2.6 5.3 2.6 2.6 2.7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Pre-measurement
Poor Average Good Very good Excellent
2.7 2.7 2.7 2.8 2.7 2.7 5.4 5.4 2.7
29.7
13.9 18.9
35.1
27.0 29.7
22.2 25.0 24.3 33.3 29.7
29.7
43.2 48.6
32.4
35.1 40.5
59.5
63.9 56.8
48.6
45.9 37.8
36.1 38.9 40.5
36.1
54.1 56.8
40.5 29.7
40.5 35.1 27.0
8.1
22.2 24.3 13.5
27.0 29.7 38.9 36.1 35.1
30.6
13.5 13.5 13.5 16.2 27.0 24.3
29.7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Post-measurement
Poor Average Good Very good Excellent
39
3.4.3 Level Three Analysis
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learnt in the training to their specific work environment. This
impact was tested on two levels, firstly; a subjective reflective measurement provided by the
participant and secondly; an objective reflective measurement provided by the supervisors of
learners.
In measuring the impact of the learning programme in the application of the learned
knowledge both learning programme participants as well as their supervisors were required
to respond to a similar list of statements derived from the defined training objectives. For
each statement, the two respondent groups had to indicate their response by means of the
following five item Likert Scale; (1) Skill application poorer than before, (2) Skill application on
the same level as before the training: not on required level, (3) Skill application on the same
level as before the training: on required level, (4) Skill application better than before the
training and, (5) Skill application much better than before the training. If respondents scored a
statement as either one or two, that is (1) Skill application poorer than before or (2) Skill
application on the same level as before the training: not on required level, they were
requested to indicate the main reason for this rating. The statements, to which respondents
had to respond, were:
1. The incumbent is /I am able to apply legislative framework relating to PAJA
2. The incumbent is /I am able to explain the purpose of PAJA in the working
environment
3. The incumbent is /I am able to define administrative action and how it applies in the
organisation
4. The incumbent is /I am able to define administrative justice
5. The incumbent is /I am able to follow the decision making process in the workplace
6. The incumbent is /I am able to prepare for a decision
7. The incumbent is /I am able to ensure that a proper trail is created during the decision
making process
40
8. The incumbent is /I am able to determine who has the legal authority to make
decisions
9. The incumbent is /I am able to describe delegation of powers and how it applies in the
working environment
10. The incumbent is /I am able to develop a decision and take action in terms of PAJA
11. The incumbent is /I am able to apply the rules of natural justices (procedural fairness)
in the workplace
12. The incumbent is /I am able to apply the law to the facts provided in a situation
13. The incumbent is /I am able to communicate the action or decision taken to
citizens/clients
14. The incumbent is /I am able to describe the judicial review of decisions
15. The incumbent is /I am able to describe and co-ordinate the public participation
process as stipulated in PAJA
Thirty-one course participants and twenty-five supervisors completed level three
questionnaires, evaluating the impact of the training course in terms of the ability of the
course participant to apply what was learned to the work environment. Table 4.2 compares
the mean and mode scores evaluating general proficiency as well as particular competencies
related to the subject matter, as reported by both course participants and their respective
supervisors.
The general scores illustrated for both respondent groups compare well (3.7 and 3.9 for the
supervisor and participant groups respectively) with both exhibiting a general mode score of
four (skill application better than before the training). From these scores it is clear that the
learning programme did have a positive impact on the ability of training participants to
perform their job functions.
Figures 4.3 and 4.4 illustrate the variance in the responses by means of the frequencies of
responses to the individual statements as scored by the two respondent groups. From these
illustrations, it is clear that very little variance exist between the two groups with the majority
of responses in both groups indicating a better ability to apply all the defined competencies.
41
In total, eight respondents indicated a particular competency as either outside their scope of
work or not possible to apply due to a lack of opportunity (Table 4.3). Of these, the majority
were from the Department of Social Development (N=4).
Table 4.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor-evaluation
Mean Mode Mean Mode
1 Able to apply legislative framework relating to PAJA 3.55 4 3.88 4
2 Able to explain the purpose of PAJA in working environment 3.71 4 4.00 4
3 Able to define administrative action and how it applies in my organisation 3.74 4 4.00 4
4 Able to define administrative justice 3.68 4 3.96 4
5 Able to follow the decision making process in the workplace 3.90 4 3.96 4
6 Able to prepare for a decision 3.74 4 3.92 4
7 Able to ensure that a paper trail is created during the decision making process 3.81 4 4.00 4
8 Able to determine who has the legal authority to make decisions 3.77 4 3.96 4
9 Able to describe delegation of powers and how it applies in the working environment 3.87 4 4.00 4
10 Able to develop a decision and take action in terms of PAJA 3.58 4 3.88 4
11 Able to apply the rules of natural justice in the workplace 3.65 4 3.96 4
12 Able to apply the law to the facts provided in a situation 3.74 4 3.88 4
13 Able to use discretion to make decisions in terms of PAJA 3.77 4 3.96 4
14 Able to communicate the action or decision taken to citizens/clients 3.87 4 3.84 4
15 Able to describe the judicial review of decisions 3.41 4 3.91 4
16
Able to describe and co-ordinate the public participation process as stipulated in PAJA 3.48 4 3.86 3
Total mean 3.70
3.90
Self-evaluation (N=31)
Supervisor evaluation (N=25)
42
Figure 4.3: Self-evaluation on impact of training (N=31)
Figure 4.4: Supervisor-evaluation on impact of training (N25)
9.7 9.7 6.5 3.2 3.2 3.2 3.2 6.5 6.7 9.7 12.9 3.2 6.5 3.2
13.8 10.3
29.0 16.1 22.6 32.3
19.4 29.0 25.8 19.4
10.0
29.0 19.4 29.0 22.6
19.4
34.5 37.9
58.1 67.7 61.3
58.1
61.3 58.1 58.1 64.5
73.3
54.8 58.1 58.1 58.1 64.5
48.3 44.8
3.2 6.5 9.7 6.5 16.1 9.7 12.9 9.7 10.0 6.5 9.7 9.7 12.9 12.9
3.4 6.9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
4.0 4.0 4.0
24.0 28.0 28.0 32.0 28.0 32.0 24.0 29.2 28.0 32.0
24.0 28.0
28.0 24.0 31.8 40.9
52.0 44.0 44.0 40.0 48.0 44.0
52.0 45.8 44.0 48.0
56.0 44.0 48.0 52.0 45.5 31.8
20.0 28.0 28.0 28.0 24.0 24.0 24.0 25.0 28.0
20.0 20.0 24.0 24.0 20.0 22.7 27.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Supervisor-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
43
Table 4.3: Inability to apply a particular competency by department
Course participant Supervisor
Department
No
opportunity
Not my
job Total
No
opportunity
No
resources
Not my
job Total
Community Safety 0 2 1 2 0 1 1
DCAS 0 4 1
DEADP 1 2 1
Health 0 15 1 3 0 12 1
Social Development 7 17 4 2 1 2
TPW 1 14 1 1
Total 8 40 8 3
Total are based on respondents.
a. Group
3.4.4 Summary
The analysis above clearly shows the positive impact of the learning programme on both
measurement levels, that is the intent to (a) convey a specific set of knowledge and skills to
learners (level 2) and (b) enabling learners to apply what has been learnt to their respective
work environments (level 3). The learning programme was further shown to have achieved all
respective defined training outcomes and associated competencies with an overwhelming
majority of learners indicating (i) an increase in knowledge and (ii) a stronger ability to apply
each defined outcome. The impact on the latter measurement was strongly confirmed by
both course participants and their respective supervisors.
44
3.5 PERFORMANCE MANAGEMENT FOR SUPERVISORS
3.5.1 Description of Learning Programme
The target group for this learning programme include all Government of the Western Cape
employees. The purpose of this learning programme was to provide an overview of the
performance management policies applicable in the WCG and to enhance the skills of
supervisors, managers and employees to (i) compile performance measures and standards, (ii)
conduct performance reviews and appraisals more effectively. The learning programme
content was as follows:
Overview of policy relating to the performance management process
o Performance planning, compile measures, monitoring and review
o Personal development & performance appraisal
o Incentives and corrective measures
Conducting performance reviews and appraisals
o How to prepare & what to discuss
o Follow-up after reviews
The three levels of moderation
The link between performance measures and moderation
How to deal with different personalities during the appraisal interview
3.5.2 Level Two Analysis
In testing the impact of the training programme on knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
In measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
45
asked to respond by rating their understanding of each core aspect of the course content as
either Poor (1), Average (2), Good (3), Very Good (4) and Excellent (5). The core aspects to
which respondents had to rate their level of understanding, were:
1. The regulatory framework that prescribes Performance Management in the Public
Service
2. The importance of Performance Management in the workplace
3. The integration of performance management with other organisational processes
4. The principles of the SPMS (Staff Performance Management Service)
5. The systematic approach to performance management
6. How to draft key performance areas, measurable outputs, performance measures,
norms and standards
7. Drafting a Personal Development Plan
8. Ho to conduct the review and appraisal interviews with employees
9. The various levels of moderation
10. The non-agreement process
11. Rewarding good performance with incentives
12. Development as part of the performance management process
Fifty-five respondents completed a pre- and post-test for this learning programme. The
analysis shows a general increase in the mean and mode scores for all twelve defined core
aspects to the training programme (Table 5.1). Comparing the total pre- and post –test mean
scores an increase of just more than 1.5 ‘points’ is noted, with a total pre- and post test score
of 2.5 and 4.2 respectively. The higher total post-test score confirms the accumulation of
knowledge with a growth in the general understanding of the subject matter moving from
“Average” to “Very Good” understanding. The same trend is illustrated for each of the twelve
individual core competencies.
With regards to the variability in responses, Figure 5.1 and Figure 5.2 illustrate limited
variance between statements with the frequency analysis clearly illustrating an increase in the
level of knowledge for all aspects measured. This outcome strongly supports the notion that
46
the defined training outcomes and objectives set for this training programme were indeed
achieved.
Table 5.1: Pre- and Post-measurements – Performance management for supervisors
Code Statement
Pre-test Post-test
Mean Mode Mean Mode
1
The regulatory framework that prescribes Performance Management in the Public Service 2.33 2 4.15 4
2
The importance of Performance Management in the workplace 3.18 3 4.35 5
3
The integration of performance management with other organisational processes 2.47 2 4.02 4
4
The principles of the SPMS (Staff Performance Management System) 2.66 2 4.24 4
5
The systematic approach to performance management 2.39 2 4.24 4
6
How to draft key performance areas, measurable outputs, performance measures, norms and standards 2.42 2 4.20 4
7 Drafting a Personal Development Plan
2.55 2 4.13 4
8
How to conduct the review and appraisal interviews with employees 2.45 2 4.30 4
9 The various levels of moderation
2.24 2 4.09 4
10 The non-agreement process
2.06 2 4.02 4
11 Rewarding good performance with incentives
2.40 2 4.25 4
12
Development as part of the performance management process 2.38 2 4.13 4
Total mean 2.46
4.18
N=55
47
Figure 5.1: Pre-measurement - Performance management for supervisors
Figure 5.2: Post-measure - Performance management for supervisors
12.7 1.8
10.9 9.4 14.8 10.9 9.1 5.5 18.5
27.8
12.7 14.5
54.5
23.6
43.6 37.7
42.6 47.3 41.8
61.8
50.0
46.3
41.8 47.3
20.0
40.0
34.5
34.0
33.3 32.7 36.4
16.4 22.2
18.5
40.0 23.6
12.7
23.6
9.1 15.1
7.4 7.3 10.9 14.5 7.4 7.4 3.6
14.5 10.9
1 2 3 4 5 6 7 8 9 10 11 12
Pre-measurement
Poor Average Good Very good Excellent
11.1 13.0
20.4 13.0 14.8 14.8
24.1 13.0
18.5 25.9
11.3 17.3
63.0
38.9
46.3
44.4 46.3 44.4 38.9
44.4
53.7 46.3
52.8 51.9
25.9
48.1
29.6 40.7 38.9 38.9 37.0
42.6
27.8 27.8 35.8 30.8
1 2 3 4 5 6 7 8 9 10 11 12
Post-measurement
Poor Average Good Very good Excellent
48
3.5.3 Level Three Analysis
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learnt in the training to their specific work environment. This
impact was tested on two levels, firstly; a subjective reflective measurement provided by the
participant and secondly; an objective reflective measurement provided by the supervisors of
learners.
In measuring the impact of the learning programme in the application of the learned
knowledge both learning programme participants as well as their supervisors were required
to respond to a similar list of statements derived from the defined training objectives. For
each statement the two respondent groups had to indicate their response by means of the
following five item Likert Scale; (1) Skill application poorer than before, (2) Skill application on
the same level as before the training: not on required level, (3) Skill application on the same
level as before the training: on required level, (4) Skill application better than before the
training and, (5) Skill application much better than before the training. If respondents scored a
statement as either one or two, that is (1) Skill application poorer than before or (2) Skill
application on the same level as before the training: not on required level, they were
requested to indicate the main reason for this rating. The statements, to which respondents
had to respond were:
1. The incumbent is /I am able to prescribe the regulatory framework that prescribes
performance management in the Public Service
2. The incumbent is /I am able to integrate performance management with other
organisational processes
3. The incumbent is /I am able to contextualise the Staff Performance Management
System (SPMS) in the organisation through a systematic approach to performance
management
4. The incumbent is /I am able to draft a Performance Agreement (inclusive of
measurable outputs, performance measures and norms and standards)
5. The incumbent is /I am able to draft a Personal Development Plan
6. The incumbent is /I am able to conduct the review and appraisal interviews
49
7. The incumbent is /I am able to differentiate and explain the various levels of
moderation applicable within the Staff Performance System (SPMS)
8. The incumbent is able to advise on and implement the non-agreement process
9. The incumbent is /I am able to explain how and when good performance is rewarded
by means of incentives
10. The incumbent is /I am able to use the Staff Performance System (SPMS) as a
developmental tool as part of the performance management process
Twenty course participants and twelve supervisors completed level three questionnaires,
evaluating the impact of the training course in terms of the ability of the course participant to
apply what was learned to the work environment. Table 5.2 compares the mean and mode
scores evaluating general proficiency as well as particular competencies related to the subject
matter, as reported by both course participants and their respective supervisors.
The general scores illustrated for both respondent groups compare well (4.0 and 4.2 for the
supervisor and participant groups respectively) with both exhibiting a general mode score of
four (skill application better than before the training). From these scores it is clear that the
learning programme did indeed have a positive impact on the ability of training participants
to perform their job functions.
Figures 5.3 and 5.4 illustrate the variance in the responses by means of the frequencies of
responses to the individual statements as scored by the two respondent groups. From these
illustrations, it is clear that very little variance exist between the two groups with the majority
of responses in both groups indicating a better ability to apply all the defined competencies.
One respondent from the Department of Health indicated two particular competencies as
outside his/her scope of work. This was confirmed by the supervisor of this respondent.
50
Table 5.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor-evaluation
Mean Mode Mean Mode
1
Able to apply the regulatory framework that prescribes performance management in the Public Service 4.05 4 4.25 4
2
Able to integrate performance management with other organisational processes 4.16 4 4.42 4
3
Able to contextualise the SPMS (Staff Performance Management System) in my organisation through a systematic approach to performance management 3.85 4 4.17 4
a
4
Able to draft a Performance Agreement (inclusive of measurable outputs, performance measures and norms and standards) 3.89 4 4.33 4
5 Able to draft a Personal Development Plan
3.80 4 4.17 4
6 Able to conduct the review and appraisal interviews
4.00 4 4.17 4a
7
Can differentiate and explain the various levels of moderation applicable within the SPMS (Staff Performance Management System) 4.00 4 4.25 4
a
8
Able to advise on and implement the non-agreement process 3.40 3 4.00 4
9
Able to explain how and when good performance is rewarded by means of incentives 3.85 4 4.25 4
a
10
Able to use the SPMS (Staff Performance Management System) as a developmental tool as part of the performance management process 4.05 4 4.42 4
Total mean 3.91
4.24
Self-evaluation (N=20)
Supervisor-evaluation (N=12)
a. Multiple modes exist. The smallest value is shown
51
Figure 5.3: Self-evaluation on impact of training (N=20)
Figure 5.4: Supervisor-evaluation on impact of training (N12)
5.3 5.0 5.0 10.0 5.0 5.0
20.0 15.8
20.0 15.8
25.0 20.0 25.0
55.0
30.0 20.0
40.0 52.6
45.0 57.9 55.0
45.0 50.0
30.0
55.0
55.0
35.0 31.6 25.0 21.1 15.0
30.0 25.0 10.0 15.0
25.0
1 2 3 4 5 6 7 8 9 10
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
8.3 8.3 8.3 8.3
8.3 16.7
8.3 16.7
25.0 16.7
58.3 58.3
41.7 50.0
50.0 41.7 41.7
50.0
41.7 58.3
33.3 41.7 41.7 41.7
33.3 41.7 41.7
25.0 41.7 41.7
1 2 3 4 5 6 7 8 9 10
Supervisor-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
52
3.5.4 Summary
The analysis above clearly shows the positive impact of the learning programme on both
measurement levels, that is the intent to (a) convey a specific set of knowledge and skills to
learners (level 2) and (b) enabling learners to apply what has been learned to their respective
work environments (level 3). The learning programme was further shown to have achieved all
respective defined training outcomes and associated competencies with an overwhelming
majority of learners indicating (i) an increase in knowledge and (ii) a stronger ability to apply
each defined outcome. The impact on the latter measurement was strongly confirmed by
both course participants and their respective supervisors.
3.6 PUBLIC SERVICE INDUCTION
3.6.1 Description of Learning Programme
The target group for this learning programme included all newly appointed public servants
appointed on levels one to five and appointed on probation. The purpose of this programme
was to welcome all new public servants and to ensure that all new public servants understood
the goals, structures and key government policies.
The learning programme consisted of three parts with part 1 focusing on functions of
government and its link to development. Part 2 focused on the effective implementation of
government functions and Part 3 on the rights and responsibilities of public servants. The
different parts carry the following names and sub-sections:
PART 1: WE CARE PART 2: WE SERVE PART 3: WE BELONG
The goals and key programmes
of government
The government structures
and the Public Sector
Building a better public service
53
The defined outcomes for this training programme were;
To understand the challenges government faces, the vision for development and key
programmes
To demonstrate a basic understanding of the Constitution and the three spheres of
government, intergovernmental relations and social partners
To understand Batho Pele, the values of the Public Service and key initiatives to
improve service delivery
To explain the framework for implementation management through basic strategic
planning, performance management and financial management
To understand their rights as employees as well as regulations and benefits
3.6.2 Level Two Analysis
In testing the impact of the training programme on knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
In measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
asked to respond by rating their understanding of each core aspect of the course content as
either (1) Poor, (2) Average, (3) Good, (4) Very good and (5) Excellent. The aspects measured
to which respondents had to rate their level of understanding was:
1. Define government and the role of government
2. The main challenges the country is facing
3. Government’s vision for development and programmes to address
4. Understanding of the constitution
5. Spheres of government
6. Benefits and conditions of Service
7. Batho Pele and Service Delivery Improvements
8. Building of Relationships” Co-operative Governance and Intergovernmental Relations
54
9. Government money – where it comes from and how it is spent
10. Role of Public Service and Social Partnerships
11. Corruption and unethical behaviour
12. Legislative framework – laws, policies that govern the public service
13. In determining the impact of this learning programme on the knowledge levels of
participants,
14. With regards to the variability in responses,
Sixty six respondents completed a pre- and post-test for this learning programme. The
analysis shows a general increase in the mean and mode scores for all twelve defined core
aspects to the training programme (Table 6.1). Comparing the total pre- and post –test mean
scores an increase of approximately 1.5 ‘points’ is noted, with a total pre- and post test score
of 2.7 and 4.1 respectively. The higher total post-test score confirms the accumulation of
knowledge with a growth in the general understanding of the subject matter moving in
general from “Average” to “Very Good” understanding. The same trend is illustrated for each
of the twelve individual core competencies.
With regards to the variability in responses Figure 6.1 and Figure 6.2 illustrate limited variance
between statements with the frequency analysis clearly illustrating an increase in the level of
knowledge for all aspects measured. This outcome strongly supports that the defined training
outcomes and objectives set for this training programme were achieved.
55
Table 6.1: Pre- and Post-measurements – Public service induction
Code Statement
Pre-test Post-test
Mean Mode Mean Mode
1 Define government and the role of government
2.71 2 4.20 5
2 The main challenges the country is facing
2.95 3 4.11 4
3
Governments vision for development and programmes to address 2.65 2
a 3.97 4
4 Understanding of the constitution
2.77 4 4.12 4
5 Spheres of government
2.56 2 4.17 4a
6 Benefits and conditions of service
2.85 2 4.21 5
7 Batho Pele and Service Delivery Improvements
2.97 2a 4.27 4
8
Building of Relationships: Co-operative Governance and Intergovernmental Relations 2.50 2 3.94 4
9
Government money: where it comes from and how it is spend 2.62 3 4.05 5
10 Corruption and unethical behaviour
2.68 2 4.05 4a
11 Corruption and unethical behaviour
2.76 3 4.03 4
12
Legislative framework: laws, policies that govern the public service 2.59 2 3.92 4
Total mean 2.7 4.1
N=66
a. Multiple modes exist. The smallest value is shown
56
Figure 6.1: Pre-measurement - Managing absence in the workplace
Figure 6.2: Post-measure - Managing absence in the workplace
18.2 12.1
21.2 19.7 24.2 13.6 13.6
25.8 24.2 16.7 18.2
24.2
28.8
19.7
25.8 22.7
25.8
31.8 27.3
27.3 22.7
31.8 25.8 25.8
25.8
40.9
25.8 24.2
24.2 25.8
19.7
22.7 28.8 24.2
27.3 24.2
18.2 15.2 21.2
27.3 21.2
13.6 27.3
19.7 15.2 21.2 19.7 18.2
9.1 12.1 6.1 6.1 4.5
15.2 12.1 4.5 9.1 6.1 9.1 7.6
1 2 3 4 5 6 7 8 9 10 11 12
Pre-measurement
Poor Average Good Very good Excellent
19.7 18.2
28.8 21.2 19.7 15.2
12.1
28.8 25.8 22.7 22.7 24.2
36.4 48.5
36.4
40.9 39.4 37.9 43.9
43.9
33.3 36.4 40.9 39.4
42.4 31.8 31.8 36.4 39.4 43.9 42.4
25.8 37.9 36.4 33.3 30.3
1 2 3 4 5 6 7 8 9 10 11 12
Post-measurement
Poor Average Good Very good Excellent
57
3.5.3 Level Three Analysis
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learnt in the training to their specific work environment. This
impact was tested on two levels, firstly; a subjective reflective measurement provided by the
participant and secondly; an objective reflective measurement provided by the supervisors of
learners.
In measuring the impact of the learning programme in the application of the learned
knowledge both learning programme participants as well as their supervisors were required
to respond to a similar list of statements derived from the defined training objectives. For
each statement the two respondent groups had to indicate their response by means of the
following five item Likert Scale; (1) Skill application poorer than before, (2) Skill application on
the same level as before the training: not on required level, (3) Skill application on the same
level as before the training: on required level, (4) Skill application better than before the
training and, (5) Skill application much better than before the training. If respondents scored a
statement as either one or two, that is (1) Skill application poorer than before or (2) Skill
application on the same level as before the training: not on required level, they were
requested to indicate the main reason for this rating. The statements, to which respondents
had to respond, were:
1. The incumbent is /I am able to define the role of government
2. The incumbent is /I am able to explain the main challenges the country is facing
3. The incumbent is /I am able to define and identify government’s vision for
development and programmes to address the challenges
4. The incumbent is /I am able to explain the constitution and how it applies in my work
5. The incumbent is /I am able to classify the spheres of government and explain the
spheres I work in
6. The incumbent is /I am able to explain the benefits and conditions of service
7. The incumbent is /I am able to apply Batho Pele and service delivery improvements
8. The incumbent is /I am able to describe what building relationships, co-operative
Governance and Intergovernmental Relations entails
58
9. The incumbent is /I am able to explain where government money comes from and
how it is spent
10. The incumbent is /I am able to define the roles of public service and social
partnerships
11. The incumbent is /I am able to identify corruption and unethical behaviour
12. The incumbent is /I am able to apply the legislative framework – laws and policies that
govern the public service.
13. In determining the impact of this learning programme on the ability of learners to
apply what has been learned to the work environment, the mean and mode scores of
the responses need to be considered.
Thirty-three course participants and thirty-six supervisors completed level three
questionnaires, evaluating the impact of the training course in terms of the ability of the
course participant to apply what was learned to the work environment. Table 6.2 compares
the mean and mode scores evaluating general proficiency as well as particular competencies
related to the subject matter, as reported by both course participants and their respective
supervisors.
The general scores illustrated for both respondent groups compare well (4.3 and 4.2 for the
supervisor and participant groups respectively) with both exhibiting mode scores between
four and five (skill application better than before the training and skill application much better
than before the training). From these scores, it is clear that the learning programme did have
a positive impact on the ability of participants to perform their job functions.
One respondent from the Department of Health indicated a single competency as outside
his/her scope of work with another official from the Department of Education indicating a
lack of opportunity to apply a particular competency. Responses from supervisors to this issue
were, however, somewhat more pronounced with supervisors of three departments, Cultural
Affairs, Health and Social Development, indicating the inability of course participants to apply
some of the competencies since it either (i) falls outside their scope of work (N=14), (ii) there
is a lack of resources (N=2) or (iii) due to a lack of opportunity (N=5).
59
Table 6.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor evaluation
Mean Mode Mean Mode
1 I am able to define the role of government 4.39 4 4.39 5
2 I am able to explain the main challenges the country is facing 4.27 4 4.25 4
3
I am able to define and identify government's vision for development and programmes to address the challenges 4.27 4 3.92 4
4 I am able to explain the constitution and how it applies in my work 4.15 4 4.03 4
5 I am able to classify the spheres of government and explain the sphere I work in 4.24 4 3.97 4
6 I am able to explain the benefits and conditions of service 4.39 4 4.14 5
7 I am able to apply Batho Pele and service delivery improvements 4.58 5 4.50 5
8
I am able to describe what building relationships, co-operative governance and intergovernmental relations entail 4.00 4 3.72 3
9 I am able to explain where government money is coming from and how it is spent 4.18 4 4.36 5
10 I am able to define the role of public service and social partnerships 4.30 4 4.03 5
11 I am able to identify corruption and unethical behaviour 4.55 5 4.51 5
12 I am able to apply the legislative framework e.g. laws and policies that govern the public service 4.33 5 4.29 5
Total mean 4.31
4.18
Self-evaluation (N=33)
Supervisor-evaluation (N=36)
60
Figure 6.3: Self-evaluation on impact of training (N=33)
Figure 6.4: Supervisor-evaluation on impact of training (N=36)
3.0 3.0 6.1 15.2 9.1 3.0
21.2 18.2 12.1
3.0 12.1
54.5 57.6 60.6
54.5 57.6
54.5
42.4
48.5 45.5
45.5
39.4
42.4
42.4 36.4 33.3 30.3 33.3 42.4
57.6
27.3 36.4 42.4
57.6 45.5
1 2 3 4 5 6 7 8 9 10 11 12
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
11.1 11.1 27.8
16.7 22.2 30.6
5.6
41.7
16.7 25.0
8.6 14.7
38.9 44.4
44.4 63.9 41.7 25.0
38.9
36.1
30.6
30.6
31.4
41.2
50.0 41.7 25.0 19.4
30.6 44.4
55.6
19.4
52.8 38.9
60.0 44.1
1 2 3 4 5 6 7 8 9 10 11 12
Supervisor-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
61
3.6.4 Summary
The analysis above clearly shows the positive impact of the learning programme on both
measurement levels, that is the intent to (a) convey a specific set of knowledge and skills to
learners (level 2) and (b) enabling learners to apply what has been learnt to their respective
work environments (level 3). The learning programme was further shown to have achieved all
respective defined training outcomes and associated competencies with an overwhelming
majority of learners indicating (i) an increase in knowledge and (ii) a stronger ability to apply
each defined outcome. The impact on the latter measurement was strongly confirmed by
both the majority of course participants and their respective supervisors.
3.7 POLICY ANALYSIS, DEVELOPMENT, IMPLEMENTATION
3.7.1 Description of Learning Programme
The target group for this learning programme include all Government of the Western Cape
officials on salary levels eight to twelve, who are involved in the analysis, development and
implementation of policies within departments. The purpose of this learning programme was
to equip officials with the relevant information to analyse, develop and implement policies
within departments. The learning programme consists of the following components:
Nature of public policy
What is public policy
Who makes public policy
Types and levels of policies
Policy agenda setting
Problem identification
Structuring
Prioritisation
62
Policy design and option generation
Policy goals and objectives
Generation of policy options and policy selection
Feasibility testing
Policy processes
Policy implementation
Implementation styles
Participation styles
Constraints strategies, implementation constraints
Policy monitoring, evaluation an analysis
Monitoring policy implementation
Policy analysis methods and techniques
Review of existing and previously developed organisation policies
Evaluation issues and strategies
3.7.2 Level Two Analysis
In testing the impact of the training programme on knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
In measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
asked to respond by rating their understanding of each core aspect of the course content as
either (1) Poor, (2) Average, (3) Good, (4) Very good and (5) Excellent. The aspects measured
to which respondents had to rate their level of understanding, were:
1. The context of public policy analysis, development, implementation and
evaluation in the public service
2. Define public policy and policy types within the public service
3. The application of public policy within public service
63
4. The policy life cycle and the policy process
5. The methods and techniques used for policy analysis and development
6. How policy goals and objectives are set in the public service
7. How to generate policy options and do policy selection
8. The structure of a policy and how to structure a policy for the public service
9. Define and explain policy implementation management in the public service
10. Monitoring and Evaluation of public policy used within the public service
Thirty seven respondents completed a pre- and post-test for this learning programme. The
analysis show a general increase in the mean and mode scores for all ten defined core aspects
of the training programme (Table 7.1). Comparing the total pre- and post –test mean scores
an increase of approximately two ‘points’ is noted, with a total pre- and post test score of
1.96 and 3.87 respectively. The higher total post-test score confirms the accumulation of
knowledge with a growth in the general understanding to the subject matter moving from
“Average” to a “Very Good” understanding. The same trend is illustrated for all the core
competencies, with the exception of one, How to generate policy options and do policy
selection. For this competency the increase in knowledge gained is stronger than for the rest,
moving from an initial Poor understanding to a Very Good understanding.
With regards to the variability in responses Figure 7.1 and Figure 7.2 illustrate limited variance
between statements with the frequency analysis clearly illustrating an increase in the level of
knowledge for all aspects measured. This outcome strongly supports that the defined training
outcomes and objectives set for this training programme were achieved.
64
Table 7.1: Pre- and Post-measurements – Policy analysis, development, implementation
Code Statement
Pre-test Post-test
Mean Mode Mean Mode
1 The context of public policy analysis, development, implementation and evaluation in the public service 2.22 2 3.86 4
2 Define public policy and policy types within the public service 2.08 2 3.92 4
3 The application of public policy within the public service 2.46 3 3.97 4
4 The policy life cycle and the policy process 2.00 2 3.97 4
5 The methods and techniques used for policy analysis and development 1.78 2 3.92 4
6 How policy goals and objectives are set in public service 1.86 2 3.89 4
7 How to generate policy options and do policy selection 1.68 1 3.76 4
8 The structure of a policy and how to structure a policy for the public service 1.78 2 3.81 4
9 Define and explain policy implementation management in the public service 1.78 2 3.76 4
10 Monitoring and Evaluation of public policy used within the public service 1.92 2 3.81 4
Total mean 1.96
3.87
N=37
65
Figure 7.1: Pre-measurement - Policy analysis, development, implementation
Figure 7.2: Post-measure - Policy analysis, development, implementation
27.0 27.0 18.9
30.6 40.5
32.4
48.6 40.5 37.8
32.4
35.1 40.5
24.3
44.4
45.9 51.4
37.8 43.2 48.6
48.6
32.4 29.7
51.4
19.4
10.8 13.5 10.8 13.5 10.8 13.5
1 2 3 4 5 6 7 8 9 10
Pre-measurement
Poor Average Good Very good Excellent
27.0 24.3 24.3
32.4 35.1 35.1 37.8 21.6
40.5 29.7
51.4 59.5 54.1 37.8
37.8 40.5
48.6 59.5
43.2 51.4
18.9 16.2 21.6
29.7 27.0 24.3 13.5 13.5 16.2 16.2
1 2 3 4 5 6 7 8 9 10
Post-measurement
Poor Average Good Very good Excellent
66
3.7.3 Level Three Analysis
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learnt in the training to their specific work environment. This
impact was tested on two levels, firstly; a subjective reflective measurement provided by the
participant and secondly; an objective reflective measurement provided by the supervisors of
learners.
In measuring the impact of the learning programme in the application of the learned
knowledge both learning programme participants as well as their supervisors were required
to respond to a similar list of statements derived from the defined training objectives. For
each statement the two respondent groups had to indicate their response by means of the
following five item Likert Scale; (1) Skill application poorer than before, (2) Skill application on
the same level as before the training: not on required level, (3) Skill application on the same
level as before the training: on required level, (4) Skill application better than before the
training and, (5) Skill application much better than before the training. If respondents scored a
statement as either one or two, that is (1) Skill application poorer than before or (2) Skill
application on the same level as before the training: not on required level, they were
requested to indicate the main reason for this rating. The statements, to which respondents
had to respond, were:
1. The incumbent is /I am able to contextualise public policy and differentiate between
various types of public policies used in the public sector
2. The incumbent is /I am able to apply various types of public policies in my current
occupation
3. The incumbent is /I am able to apply various policy processes in terms of the policy life
cycle in my current occupation
4. The incumbent is /I am able to apply various methods and techniques as part of the
policy analysis and development process
5. The incumbent is /I am able to set policy goals and objectives in the work context
6. The incumbent is /I am able to apply various generate and select policy options based
on organisational goals and objectives
67
7. The incumbent is /I am able to apply various structure policies for the organisation,
according to public service guidelines/principles
8. The incumbent is /I am able to manage the implementation of policies in the
organisation
9. The incumbent is /I am able to monitor and evaluation policies in the organisation
Thirteen course participants and supervisors completed level three questionnaires, evaluating
the impact of the training course in terms of the ability of the course participant to apply
what was learned to the work environment. Table 7.2 compares the mean and mode scores
evaluating general proficiency as well as particular competencies related to the subject
matter, as reported by both course participants and their respective supervisors.
The general scores illustrated for both respondent groups compare well (3.3 and 3.4 for the
supervisor and participant groups respectively) with both exhibiting mode scores of between
three and four for the respective competencies tested (Skill application the same than before
the training: on required level and Skill application better than before the training). These
overall scores are not particularly positive since a score of 3 (Skill application the same than
before the training: on required level) is really an indication that the training provided no real
added value to the working environment.
Considering the individual scores in table 7.2 and figures 7.3 and 7.4, there seems to be
disagreement between the ability to apply what has been taught by the training participants
and supervisors. Consistently, course participants rate their ability to apply what they have
learnt as either lower (in most cases) or higher than what their supervisors do.
Commenting on their ability to apply what has been learnt during the training, three
respondents, each from different departments (Department of Agriculture, Department of
Human Settlements and Department of the Premier) indicated some learning competencies
to fall outside of their scope of work [N=12] 10 with other not possible to apply due to a lack of
opportunity (Department of Agriculture [N=4]).
10
The N-value refers to the number of competencies
68
Table 7.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor-evaluation
Mean Mode Mean Mode
1
Able to contextualise public policy and differentiate between various types of public policies used in the public service 3.69 3 3.46 4
2
Able to apply contextualise public policy and differentiate between various types of public policies used in the public service 3.23 3 3.62 4
3
Able to apply various policy processes in terms of the policy life cycle in my current occupation 2.92 3 3.38 4
4
Can apply various methods and techniques as part of the policy analysis and development process 3.00 3 3.38 4
5 Able to set policy goals and objectives in my work context 3.33 4 3.54 3
6 Able to generate and select policy options based on organisational goals and objectives 3.25 3
a 3.38 4
7
Can structure policies for my organisation, according to public service guidelines/principles 2.92 3 3.15 4
8 Able to manage the implementation of policies in my organisation 3.69 4 3.23 3
9 Able to monitor and evaluate policies in my organisation 3.31 4 3.31 3
Total mean 3.26
3.38
Self-evaluation (N=13)
Supervisor-evaluation (N=13)
a. Multiple modes exist. The smallest value is shown
69
Figure 7.3: Self-evaluation on impact of training (N=13)
Figure 7.4: Supervisor-evaluation on impact of training (N=13)
7.7 7.7 8.3 8.3 7.7
7.7 7.7 16.7
8.3 15.4
46.2
61.5
69.2 61.5
41.7
41.7
66.7
30.8
38.5
38.5
30.8
15.4 23.1
50.0 41.7
16.7
69.2
46.2
15.4
1 2 3 4 5 6 7 8 9
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
7.7 15.4 15.4 15.4
7.7 15.4
30.8
7.7 15.4
38.5 15.4
30.8 30.8 46.2 30.8
23.1 61.5 46.2
53.8
61.5
53.8 53.8 30.8
53.8 46.2
30.8 30.8
7.7 15.4
7.7
1 2 3 4 5 6 7 8 9
Supervisor-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
70
3.7.4 Summary
The analysis above clearly shows the positive impact of the learning programme on
particularly the level of knowledge gained (level 2). In terms of the level three analysis, that is
the ability to apply what has been learned, the findings illustrates that training participants
are in general not confident that they are better equipped to apply what they have learned,
although the feedback from supervisors paint a more positive picture.
3.8 RECRUITMENT & SELECTION FOR LINE MANAGERS
3.8.1 Description of Learning Programme
The target group for this learning programme include all line managers employed on salary
levels seven to twelve. The purpose of this learning programme was to enhance the
recruitment and selection knowledge and skills of line managers, so as to to promote
uniformity with regards to recruitment and selection practices in the PGWC. This learning
programme consisted of the following components:
Recruitment
What does recruitment involve
Legislative and policy framework with regard to recruitment
Compiling an advertisement
Selection
Defining selection
Legislative and policy framework with regard to selection
Compiling a grid (short listing)
Roles of panel members (secretariat)
71
Appointments
Appointments
Promotions
Contract appointments
Transfers
Resettlement
Probation
Probation
Induction
Calculation of date of confirmation of appointment
3.8.2 Level Two Analysis
In testing the impact of the training programme on knowledge gained, learners were
requested to respond to a list of statements on two occasions that is, prior to the training
(pre-test) and immediately thereafter (post-test).
Measuring the extent to which learners gained the relevant knowledge as outlined in the
programme objectives, learners were asked to rate their level of understanding of a pre-
defined set of statements prior to and following the completion of the training. Learners were
asked to respond by rating their understanding of each core aspect of the course content as
either Poor (1), Average (2), Good (3), Very good (4) and Excellent (5). The aspects measured
to which respondents had to rate their level of understanding was:
1. Define and understand the recruitment process
2. The legislative and policy framework that regulates recruitment in the Public
Service
3. Compilation of a Job brief
4. Request to advertise a post
5. The role of a manager in the recruitment process
6. Defining and understanding the selection process
72
7. The legislative and policy framework that regulates selection in the Public Service
8. Compiling a short-list and participate in panel interviews
9. The steps in selection process and the role of line manager in the process
10. Competency Assessments (Western Cape Government Competency Policy, 2012)
11. Defining appointments, promotions and transfers
12. The legislative and policy framework that regulates appointments, promotions and
transfers in the Public Service
13. The legislative and policy framework that regulates probationary appointments in
the Public Service
14. The role of line manager in the appointments and probation process
15. Compiling induction programme
Twenty-three respondents completed a pre- and post-test for this learning programme. The
analysis shows a strong increase in the mean and mode scores for all fifteen defined core
aspects to the training programme (Table 8.1). Comparing the total pre- and post –test mean
scores, an increase of approximately two ‘points’ is noted, with a total pre- and post test
score of 2.34 and 4.29 respectively. The higher total post-test score confirms the
accumulation of a great deal of knowledge with growth in the general understanding of the
subject matter from an “Average” to a “Very Good” understanding.
With regards to the variability in responses, Figure 8.1 and Figure 8.2 illustrate limited
variance between statements with the frequency analysis clearly illustrating an increase in the
level of knowledge for all aspects measured with the strongest growth illustrated for the
following core components:
The legislative and policy framework that regulates appointments, promotions and
transfers in the Public Service (Poor to Excellent)
Request to advertise a post (Average to Excellent)
Defining and understanding the selection process (Average to Excellent)
The steps in selection process and the role of line manger in the process (Average to
Excellent)
Competency Assessments (Poor to Very Good)
73
The legislative and policy framework that regulates appointments, promotions and
transfers in the Public Service (Poor to Very Good)
This outcome strongly supports that the defined training outcomes and objectives set for this
training programme were indeed achieved.
Table 8.1: Pre- and Post-measurements – Recruitment & selection for line managers
Count Statement
Pre-test Post-test
Mean Mode Mean Mode
1 Define and understand the recruitment process 2.73 3 4.48 5
2 The legislative and policy framework that regulates recruitment in the Public Service 2.14 2 4.26 4
a
3 Compilation on a Job brief 2.09 1 4.13 5
4 Request to advertise a post 2.23 2 4.26 5
5 The role of a manager in the recruitment process 2.76 3 4.32 5
6 Defining and understanding the selection process 2.67 2 4.35 5
7 The legislative and policy framework that regulates selection in the Public Service 2.14 3 4.13 4
8 Compiling a short-list and participate in panel interviews 3.00 3a 4.39 5
9 The steps in selection process and the role of line manger in the process 2.55 2 5.74 5
10 Competency Assessments 1.82 1 4.00 4
11 Defining appointments, promotions and transfers 2.48 3 4.22 4
12 The legislative and policy framework that regulates appointments, promotions and transfers in the Public Service 1.95 1 4.00 4
13 The legislative and policy framework that regulates probationary appointments in the Public Sector 2.10 2 4.00 4
14 The role of line manager in the appointments and probation process 2.36 2 4.22 4
a
15 Compiling induction programme 2.14 1a 3.82 4
Total mean 2.34
4.29
N=23
a. Multiple modes exist. The smallest value is shown
74
Figure 8.1: Pre-measurement - Recruitment & selection for line managers
Figure 8.2: Post-measure - Recruitment & selection for line managers
13.6 22.7
36.4 27.3
9.5 9.5
31.8
9.1 18.2
45.5
23.8
36.4 28.6
9.1
31.8
27.3
45.5 27.3
36.4
28.6 38.1
27.3
22.7
31.8
27.3
23.8
31.8 38.1
50.0
31.8
36.4
27.3 27.3
22.7
38.1 28.6
36.4
31.8
27.3
27.3
33.3
31.8 28.6
36.4 27.3
18.2
4.5 9.1
13.6 23.8 23.8
4.5
31.8
22.7 19.0
4.8 4.5 9.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Pre-test
Poor Average Good Very good Excellent
4.3 4.3 4.3
4.3 4.3 4.3 4.3 4.3 4.3 8.7 8.7
9.1 8.7
8.7 8.7 4.5 4.3
13.0 8.7 4.3
21.7
8.7 17.4 13.0
4.3
39.1
43.5 39.1 39.1
40.9 39.1
47.8
30.4 34.8
43.5
47.8
52.2 47.8
43.5
54.5
56.5
43.5 43.5 47.8 50.0 52.2
34.8
56.5 56.5
30.4 39.1
26.1 30.4
43.5
27.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Post-test
Poor Average Good Very good Excellent
75
3.8.3 Level Three Analysis
The level three measurement tests the impact of the learning programme on the ability of the
learner to apply what has been learnt in the training to their specific work environment. This
impact was tested on two levels, firstly; a subjective reflective measurement provided by the
participant and secondly; an objective reflective measurement provided by the supervisors of
learners.
In measuring the impact of the learning programme in the application of the learnt
knowledge both learning programme participants as well as their supervisors were required
to respond to a similar list of statements derived from the defined training objectives. For
each statement the two respondent groups had to indicate their response by means of the
following five item Likert Scale; (1) Skill application poorer than before, (2) Skill application on
the same level as before the training: not on required level, (3) Skill application on the same
level as before the training: on required level, (4) Skill application better than before the
training and, (5) Skill application much better than before the training. If respondents scored a
statement as either one or two, that is (1) Skill application poorer than before or (2) Skill
application on the same level as before the training: not on required level, they were
requested to indicate the main reason for this rating. The statements, to which respondents
had to respond, were:
1. The incumbent is /I am able to perform all the steps involved in the appointments
process of an employee
2. The incumbent is /I am able to compile a job brief The incumbent is /I am able to
3. The incumbent is /I am able to write a submission to request the advertisement of a
post
4. The incumbent is /I am able to perform the role of a manager as prescribed in the
regulatory framework in the recruitment process
5. The incumbent is /I am able to perform all the steps in the selection process
6. The incumbent is /I am able to participate in panel interviews
7. The incumbent is /I am able to compile a short-list
76
8. The incumbent is /I am able to perform the role of a manager as prescribed in the
regulatory framework in the selection process
9. The incumbent is /I am able to explain the Competency Assessments (Western Cape
Government Competency Policy, 2012)
10. The incumbent is /I am able to perform all the steps involved in the appointments
process of an employee
11. The incumbent is /I am able to perform as manager in the appointment, promotion
and transfer process
12. The incumbent is /I am able to apply the regulatory framework that regulates
appointments, promotions and transfer
13. The incumbent is /I am able to perform the role of the line manager in the probation
process as prescribed by the legislative and policy framework
14. The incumbent is /I am able to compile an induction programme for an employee
Twelve course participants and nine supervisors completed level three questionnaires,
evaluating the impact of the training course in terms of the ability of the course participant to
apply what was learned to the work environment. Table 8.2 compares the mean and mode
scores evaluating general proficiency as well as particular competencies related to the subject
matter, as reported by both course participants and their respective supervisors.
The general scores illustrated for both respondent groups compare well (3.7 and 3.4 for the
supervisor and participant groups respectively) with both exhibiting general mode scores
between three and four (Skill application the same than before the training: on required level
and Skill application better than before the training). The overall mean and mode scores are
not particularly positive since a score of 3 (Skill application the same than before the training:
on required level) is an indication of no real added value to the working environment.
However, considering how the individual scores are presented in table 8.2 and figures 8.3 and
8.4 respectively, course participants do indicate a stronger ability to apply newly learned
knowledge on most of the defined competencies, with the exception of four competencies:
Ability to write a submission to request the advertisement of a post
Ability to perform as manager in the appointment, promotion and transfer process
77
Ability to explain the competency assessments
Ability to perform the role of a manager as prescribed in the regulatory framework in
the selection process
The scoring of supervisors is more conservative, illustrating a lack of added ability to apply
newly learned knowledge on most of the defined competencies, except in the case of four
competencies that were allocated a score of four:
Ability to compile a job brief
Ability to explain the Competency Assessments
Ability to perform all the steps involved in the appointment process of an employee
Ability to perform the role of the line manger in the probation process as prescribed
by the legislative and policy framework
Commenting on their ability to apply what has been learnt during the training, eight
respondents, each from different departments (Department of Cultural Affairs, Department
of Health and, Department of the Premier) indicated some learning competencies to fall
outside of their scope of work [N=28] 11 with other not possible to apply due to a lack of
opportunity [N=32]12 and finally due a lack of resources [N=1]13 (table 8.3). This was
confirmed by the supervisors from these departments who explained the limits to applying
what has been learned in the training programme as a direct consequence of a lack of
opportunity to do so.
11
This N-value refers to the total number of competencies selected by all respondents 12
This N-value refers to the total number of competencies selected by all respondents 13
This N-value refers to the total number of competencies selected by all respondents
78
Table 8.2: Self- and Supervisor evaluation of impact of training programme
Code Statement
Self-evaluation Supervisor-evaluation
Mean Mode Mean Mode
1 Able to perform all the steps in the recruitment process 3.83 4 3.33 3
2 Able to compile a job brief 3.55 4 3.56 4
3 Able to write a submission to request the advertisement of a post 3.30 3 2.78 3
4
Able to perform the role of a manager as prescribed in the regulatory framework in the recruitment process 3.55 4 3.00 3
5 Able to perform all the steps in the selection process 4.08 4 3.33 3
6 Able to participate in panel interviews 4.08 4 3.67 3
7 Able to compile a short-list 4.17 4 3.67 3a
8
Able to perform the role of a manager as prescribed in the regulatory framework in the selection process 3.42 3
a 3.00 3
9 Able to explain the Competency Assessments 3.45 3 3.63 4
10 Able to perform all the steps involved in the appointment process of an employee 3.91 4 3.38 4
11 Able to perform as manager in the appointment, promotion and transfer process 3.55 3 3.13 3
a
12
Able to apply the regulatory framework that regulates appointments, promotions and transfer 3.55 4 3.13 3
a
13
Able to perform the role of the line manger in the probation process as prescribed by the legislative and policy framework 3.58 4 3.44 4
14 Able to compile and induction programme for an employee 3.91 4 3.67 3
a
Total mean 3.71
3.34
Self-evaluation (N=12)
Supervisor-evaluation (N=9) a. Multiple modes exist. The smallest value is shown
79
Figure 8.3: Self-evaluation on impact of training (N=12)
Figure 8.4: Supervisor-evaluation on impact of training (N=9)
8.3 9.1 9.1 9.1 8.3 8.3 9.1 20.0 18.2 8.3
9.1 9.1 9.1 9.1 8.3
25.0 36.4
40.0 27.3
25.0 16.7
8.3
33.3 45.5 18.2
45.5 36.4
16.7
27.3
41.7
45.5 30.0
36.4
41.7 58.3 66.7
33.3 36.4
36.4
18.2 36.4
50.0 54.5
25.0 9.1 10.0
18.2 33.3
25.0 25.0 16.7
9.1 27.3
18.2 9.1
16.7 18.2
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Self-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
11.1
33.3 25.0
11.1 22.2
12.5
25.0 25.0 25.0
11.1 11.1
77.8
33.3
55.6
50.0
55.6
55.6 44.4
55.6
25.0
25.0
37.5 37.5
33.3 33.3
11.1
44.4
11.1
25.0
22.2
22.2 44.4
22.2
50.0
37.5
37.5 37.5
55.6
33.3
11.1 11.1 11.1 22.2
11.1 12.5 12.5 22.2
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Supervisor-evaluation
Skill application poorer than before training
Skill application on same level as before the training: Not on required level
Skill application on same level as before training: On the required level
Skill application better than before the training
Skill application much better than before the training
80
Table 8.3: Inability to apply a particular competency by department
Department No opportunity No resources Not my job Total
Cultural Affairs and Sport 3 0 0 1
Health 3 1 24 4
Premier 26 0 4 3
Total 32 1 28 8
Percentages and totals are based on respondents.
a. Group
3.8.4 Summary
The analysis above clearly shows the positive impact of the learning programme on
particularly the level of knowledge gained (level 2). In terms of the level three analysis, that is,
the ability to apply what has been learned, the responses by course participants and
supervisors do not concur, with the majority of course participants confident that they are in
general able to apply what they have learned in contrast to the responses by supervisors who
illustrate the same competency than before the training for most of the listed competencies.
This does, however, not necessarily imply that the training programme is effective, but rather
that course participants were left more confident to conduct specific tasks, although they
level of performance was always good, and thus the lower rating by supervisors.
81
4 CONCLUDING COMMENTS AND RECOMMENDATIONS
In general the evaluation on the impact of these eight learning programmes is clearly positive
with the majority of respondents, both learners and supervisors, highlighting the positive
effect of training in the work place. The measurement of Level 2 data is presented as more
pronounced illustrating all the learning programmes as effective in transferring new
knowledge to the general course participant. It is, however, the impact on Level 3, that is, the
behaviour of the learner or in this case the application of the taught skills and competencies
in the work place, where the results are somewhat more complex.
In spite of this, the data collected was able to provide a reliable account of the impact
showing that in the majority of cases, the effect of the training did work its way through to
the work place thus leaving participants more capable of conduct their daily tasks as well as
leaving them feeling more confident in the execution of their work.
The analysis of the respective learning programmes did, however, also reflect on the need to
re-think and re-consider some objectives, specifically in the inclusion of training on specific
skills and competencies that might not be relevant to the bigger participant group.