personality and motivation profiler SHORT MANUAL.pdf · of measurement. In particular, the PMP is designed to aid in the assessment, development and counselling of non-managerial

personality and motivation profiler

pmp

KCP questionnaire series

manual

by

A M Roberts

personality and motivation profiler

BSc, CPsychol, CSci, AFBPsS, FCIPD, Registered Psychologist

pmp

KCP questionnaire series

pmp

KCP questionnaire series 3

manual

Section Page

1 INTRODUCTION 4

2 THE DEVELOPMENT OF PMP 5

3 THE PMP SCALES 11

4 ADMINISTRATION 34

5 SCORING 37

6 NORMS 38

7 RELIABILITY 40

Table of contents

pmp


manual1. INTRODUCTION

Effective assessment of people at work is critical to selection, development,counselling and succession planning.

Some job-related attributes are best assessed through the use of ability tests,simulation exercises or interviews. Others lend themselves to measurement by self-report questionnaires. Generally, characteristics relating to temperament, style,preferences, typical responses, values and motivations can usefully be measuredon this self-report basis.

The Personality and Motivation Profiler has been designed to measure individualcharacteristics that are critical to the occupational setting and conducive to this formof measurement. In particular, the PMP is designed to aid in the assessment,development and counselling of non-managerial populations, including sales,customer service, administrative and graduate populations.

The PMP is derived in part from KCP’s leading and innovative instrument, TheManagerial and Professional Profiler (MAPP). By contrast with MAPP, however, thePMP focuses less on managerial behaviours and preferences, and more on theday-to-day behaviours of occupational families that have a more operational thanstrategic focus.

During the early stages of MAPP’s development, existing self-report questionnaireswere reviewed in order to identify the strengths and limitations of the instrumentsalready available to practitioners. This work enabled KCP to produce a model ofpersonality and values which incorporated the characteristics effectively measuredby other questionnaires, but also included characteristics which are highly relevantbut not assessed by existing instruments.

The PMP utilises the same model as MAPP, but the questionnaire’s scales anditems have been modified to suit a non-managerial population.

The information resulting from PMP is both far reaching, in terms of the differentdomains explored, and concise. It is a state-of-the-art instrument for psychologicalassessment, developed to maximise its practical utility and ease of interpretation.

pmp


manual2. THE DEVELOPMENT OF PMP

2.1 Market Analysis

This involved a critical evaluation of the employment market to ascertain whichoccupational groups would benefit most from using a sophisticated self-reportquestionnaire in assessment. This evaluation demonstrated that whilst there is abroad range of substantive personality questionnaires geared to high level populationssuch as graduates, managers and professionals, there are fewer effective instrumentsthat have addressed populations such as:

• Administrative personnel• Support staff (e.g. IT)• Customer service personnel• Call centre operators• Personal assistants•Graduate trainees

PMP has therefore been targeted at these occupational groups.

2.2 Literature and Existing Product Search

A number of questionnaires are currently available for use in occupationalassessments. These vary in their degrees of complexity, their structure, theirpresentation and their validity. Some of them are extremely simplistic and theuse of them for any purpose is questionable, but most of the existingquestionnaires have their own strengths and limitations. KCP have reviewedthese extensively in order to build on the strengths and avoid the limitations.Further research involved reviewing relevant literature, including validation andutility studies.

Additional information was collected by talking to practitioners about perceivedstrengths and limitations of existing instruments. However, as with MAPP, a moregeneral issue concerned the management of staff turnover. While somequestionnaires provide personality information which is relevant to performancein a given role, none of the existing instruments appeared to help in predicting jobsatisfaction and length of tenure. In order to make predictions of this kind it isnecessary to have some insight into the motivational characteristics of theindividual who is being considered for a job. More specifically, it is important toknow what the individual values about work. The objective is to match individuals,in terms of their motivations and values, to the challenges, rewards and sacrificeswhich a particular job entails.

An understanding of the individual’s work-related values also provides an opportunityto predict how well these are suited to the cultural characteristics of the organisation,or a department within an organisation.

pmp


manual2.3 Model Conception: the Structure of PMP

Having analysed market requirements, a “model” for this assessment instrumentwas formulated. The primary consideration here was to present profiled informationabout candidates in such a way as to maximise the practical usefulness of thisinformation to the assessor. The model therefore had to take account of whatpeople actually look for in a candidate and how best these data can be used.

Firstly a broad distinction was made between “outward” and “inward” dimensions.

The “outward” characteristics relate generally to the individual’s behaviour - whathe/she does and how he/she does it. Included are attributes relating to therespondent’s interpersonal style and his/her work style. Cognitive or “intellectual”style attributes are included in this “outward” category insofar as these have adirect effect on the ways in which work is performed.

“Inward” characteristics are those which are experienced by the individual, butmay not always be apparent to an outside observer. The individual’s emotionsare included, together with his/her values. “Inward” dimensions are to do withhow the individual feels about things, and often concern the respondent’sperceptions of what he/she would like or how things should be, rather thanhow they actually are. For example, an individual may value closeness to others(inward), without necessarily being particularly warm or affectionate in his/herbehaviour (outward).Within each of the “outward” and “inward” categories, there are two furthersubdivisions:

PMP

“INWARD CHARACTERISTICS”“OUTWARD CHARACTERISTICS”

INTERPERSONALORIENTATION

TASK ORIENTATION PERSONAL ORIENTATION WORK ORIENTATION

This model closely mirrors the proven effectiveness of the MAPP model.

pmp


manualInterpersonal Orientation:

Attributes relating to the respondent’s interpersonal style,with particular emphasis on relationships with others atwork.

Task Orientation:

Individual differences in approach to tasks with emphasison preferred cognitive style.

Personal Orientation:

The emotional or temperamental aspect of personality.including energies.

Work Orientation:Characteristics pertaining to work-related values - ananalysis of what the individual holds as important orsignificant.

This classification facilitates the emergence of contrasts between “outward” and“inward” attributes. It also provides a breakdown of scales into clear domains whichis useful and accessible to practitioners with limited training in psychology andpersonality theory.

pmp


manual2.4 Scale Definitions

All of the scales are bi-polar, which means that each end of each scale is characterisedby certain kinds of behaviours, feelings or values. For example, Empathetic measuresthe extent to which the respondent can put themselves into another person’s shoes;Detailed measures the extent to which the respondent is concerned to get the detailof a task right.

Full descriptions of the characteristics measured by each scale were written beforethe questionnaire items were generated. Modified versions of the scale descriptionsare presented in Section 3.

2.5 Item Writing and Qualitative Item Analysis

The items that were trialled for PMP were derived from an item bank that had beenconstructed at the time of the development of MAPP, and which therefore had provenreliability and validity, as well as additional items that were developed for the “new”scales that had not been included in the original MAPP model. Items were written inthe form of statements which respondents would rate in terms of the extent to whichthey agreed or disagreed with the statement. The items were written so that agreement(or “endorsement”) would characterise one end of the bi-polar scale, whiledisagreement would characterise the other end of the bi-polar scale.

The written items were then subjected to a qualitative item analysis. A numberof criteria were applied in order to select items for the PMP trialling versions.

The criteria included:

a. Impression ManagementThis refers to the distortion of responses towards that which is sociallyacceptable or desirable. Items for which such a response would be highly“tempting” to the candidate were excluded.

For example the item “I appear confident in most social situations”would be too susceptible to impression management, because most peoplein a selection situation would endorse the statement in order to presentthemselves in a favourable light.

A better item would be “In a group I am usually the one to start theconversation”, since shy, socially apprehensive individuals would be lesslikely to endorse this statement than the previous example.

pmp


manualb. Clarity of Language

All items were screened for ambiguities or language which may be unfamiliarto any representatives of the target groups for which the PMP was designed.In particular, words and phrases which could be unfamiliar to specific groups(such as ethnic minorities) were disallowed.

Those items that were derived from the MAPP model were scrutinised fortheir appropriateness to a non-managerial group. We had already excludedscales that were irrelevant (based on job analyses of non-managerialpopulations), so we were now concerned to ensure that individual items thatloaded onto occupationalyy insensitive dimensions were not pitched at toosophisticated a level.

c. Occupational RelevanceOnly items with a direct justifiable relevance to the occupational settingwere included.

d. Scale RelevanceItems were checked to ensure greater relevance to the scale for whichthey were written than any other RPMQ scale included in the trial version.Items which could be strongly related to more than one scale were screenedout.

e. Scale BreadthAlthough the items for a particular scale must measure a discrete attribute,a very narrow dimension results from simply rewording the same item againto generate several items per scale. Although this kind of narrow scale canbe reliable, it’s validity in predicting job performance or job satisfaction tendsto be slight. Care was therefore taken to promote scale breadth by choosingitems which utilise different ideas, scenarios and wording.

f. Scale GradationItems were selected to ensure that the range of possibilities for each scalewere represented. For each scale, then, some items are included toreference each of the two extremes of the dimension, while other includeditems represent the middle range between the two extremes.

pmp


manual2.6 Trialling of the Initial Version

The initial version of PMP was then completed by a sample of approximately 150people, consisting of job incumbents from a range of commercial and industrialorganisations. Occupations represented included administrators and call centre staff.

2.7 Item Analyses

Having collected responses to the questionnaire extensive item analyses took place.The overriding objective here was to select items for the final version which wouldcomprise reliable scales, with good ranges, low susceptibility to impressionmanagement and which would measure discreet characteristics.

Item selection took into account:

• Item means and standard deviations, to ensure the exclusion of items withexcessive endorsement rates and inclusion of items with good response range.

• Item-scale correlations, to ensure that selected items are all measuring thecharacteristic measured by the scale itself. Inclusion of items with high item-scalecorrelations produces a refined scale with high internal consistency. Exclusion ofitems with high correlations with scales other than their own reducesintercorrelations between scales.

• Item correlations with the Response Style scale. Items with lower correlationswith the Self Critical scale were selected on the basis that these were less influencedby response style factors, and, in particular, impression management. Hence, thoseitems included in the PMP are those for which individual differences in respondingare representative of actual differences in personality or values, rather than differencesin style of responding to the questionnaire.

2.8 Final Version

Item analysis resulted in the selection of 12 items per scale. Of these 12 items perscale, six were positively loading and six were negatively loading (endorsement ofpositively loading items results in scores towards the right on the profile chart;endorsement of negatively loading items results in scores towards the left on theprofile chart). Scale reliabilities (internal consistencies) were computed for the 22scales and are presented in Section 8.

pmp


manual3. THE PMP SCALES

PMP consists of 22 bi-polar scales. One of these is a Response Style measure andindicates the extent to which the respondent has been self-critical and frank in givingtheir ratings. The remaining 21 scales all measure characteristics which are importantin the occupational setting and can be categorised under the headings InterpersonalOrientation, Task Orientation, Personal Orientation and Work Orientation. Thescale titles are listed below:

RESPONSE STYLE

Self-Critical

INTERPERSONAL TASK PERSONAL WORK

Sociable Imaginative Resilient ResultsConfident Proactive Positive ChangeConvincing Systematic Energetic Self-directingTeam focused Reliable WorkFlexible Detailed Material WealthEmpathetic Competition

Recognition

Although each scale has a single title, which describes one end of the scale, all ofthe dimensions are bi-polar. It should be noted that the scale titles should not beover-interpreted. These are “labels” for the scales rather than full interpretations ofthe meaning of the scales.

Users should be cautious about using the scale titles in feedback and report writing,and should concentrate instead on describing behaviours.

Descriptions of “high scorers” and “low scorers” for each scale are presented on thefollowing pages to provide an understanding of the meaning of each dimension.

pmp


manual3.1 RESPONSE STYLE

Self-critical

This is also an indication of the individual’s style in answering the questions ratherthan their personality in the broader context.

LOW SCORER DESCRIPTION

Has tended to give responses which are socially acceptable or desirable. Hasbeen less self-critical in self-evaluations. May have oriented responses towardsperceptions of job-requirements.

HIGH SCORER DESCRIPTION

Has tended to respond in an open, frank and self-exposing manner. Hasprobably attempted to present self in a way which is true to life.

pmp


manual3.2 INTERPERSONAL ORIENTATION

Sociable


Prefers working alone. Less likely to seek out company. Avoids social activities.Happy with more solitary tasks. Content with own company.


Outgoing and friendly. Likes being with other people. Prefers to work with peoplerather than working alone.

RELATIONSHIPS WITH OTHER RPMQ SCALES:

Respondents with higher Sociable scores tend to have higher scores on:

• Convincing• Team focused• Empathetic• Imaginative• Positive• Results• Change

pmp


manualConfident


Shy and reticent, particularly in unfamiliar social settings. Unable to disguiseapprehension in difficult social situations. Reluctant to speak up. Finds socialmixing difficult. Dislikes being the centre of attention.


Projects a confident image. Finds it easy to overcome shyness. Mixes well. Eagerto speak up and contribute.


Respondents with higher Confident scores tend to have higher scores on:

• Convincing• Imaginative• Proactive• Reliable• Resilient• Positive• Energetic• Results• Change• Self-directing

pmp


manual

• Imaginative

Respondents with lower Convincing scores tend to have higher scores on:

• Flexible

Convincing


May fail to convince others, even when right. Avoids pushing ideas on others;agrees to differ. Dislikes selling, persuading or negotiating.


Enjoys trying to influence others. Is persistent and persuasive. Happy selling andnegotiating.


Respondents with higher Convincing scores tend to have higher scores on:

• Sociable • Confident

• Proactive

• Resilient • Positive • Energetic • Results • Change • Self-directing

pmp


manualTeam focused


More of an individualist than a team player. Tends not to involve self with teamactivities. Competes with others. Is more interested in personal targets andsuccesses than team.


Interested in team activities. Focuses on team success rather than personalsuccess. Prefers interpersonal co-operation to competition. Involves self withother team members.


Respondents with higher Team focused scores tend to score higher on:.

• Sociable• Flexible• Imaginative• Self-directing

pmp


manualFlexible


Stubborn; insists on getting own way. Pursues own goals without compromise.Gets very frustrated if own expectations are not met. Inflexible.


Prefers compromise to conflict. Accepts not having own way. Makesconcessions. Flexible in dealings with others.


Respondents with higher Flexible scores tend to have higher scores on:

• Convincing• Team focused• Systematic• Reliable• Detailed• Resilient

Respondents with lower Flexible scores tend to have higher scores on:

• Self-directing• Self-critical

pmp


manualEmpathetic


Less empathetic. May find it difficult to understand others. Can appear lesstolerant or sympathetic. Prefers not to involve self in others’ problems.


Patient; tolerant of others. Likely to be a good listener. Enjoys helping others.More perceptive about people and their needs.


Respondents with higher Empathetic scores tend to have higher scores on:

• Sociable • Imaginative

• Proactive

• Systematic• Reliable• Detailed• Positive• Energetic• Results• Change• Work

Respondents with lower Empathetic scores tend to have higher scores on:

• Self-critical

pmp


manual3.3 TASK ORIENTATION

Imaginative


Accepts existing systems and methods. Adapts to existing practices androutines. Does things by the book. Prefers work which is clearly defined anddoesn’t require a unique or creative approach.


Looks at problems in a fresh, unconstrained manner. Avoids preconceptions.Likes tasks for which systems/methods have not yet been established. Enjoyscreative innovation.


Respondents with higher Imaginative scores tend to have higher scores on:

• Sociable • Confident • Convincing

• Team focused

• Empathetic• Systematic• Proactive• Reliable

• Detailed• Positive• Energetic• Results• Change• Self-directing• Work

Respondents with lower Imaginative scores tend to have higher scores on:

• Self-critical

pmp


manualProactive


Waits for things to happen. Reacts to events rather than anticipating them.Doesn’t look for things that need to be done. Prefers to be told what to do.


Spots things that need doing and does them. Doesn’t wait for things to happen.Anticipates the future. Looks for work to do.


Respondents with higher Proactive scores tend to have higher scores on:

• Confident• Convincing• Empathetic• Imaginative• Systematic• Reliable• Detailed• Resilient• Positive• Energetic• Results• Change• Self-directing• Work

Respondents with lower Proactive scores tend to have higher scores on:

• Self-critical

pmp


manualSystematic


Disorderly approach to work. Avoids rigid systems. Is unstructured. Crisis driven.Tends not to plan in advance; muddles through. Less inclined to establishpriorities.


Organised, methodical approach to work. Maps out what needs to be done inadvance. Uses systems and priorities to structure work. Keeps things orderly.


Respondents with higher Systematic scores tend to have higher scores on:

• Empathetic• Flexible• Imaginative• Proactive• Reliable• Detailed• Positive• Energetic• Results• Work

Respondents with lower Systematic scores tend to have higher scores on:

• Self-critical

pmp


manualReliable


Loses interest in tasks. Leaves things unfinished. Less concerned about meetingdeadlines. Frustrated by routine. Seeks distractions.


Perseveres with tasks. Sees things through. Conscientious about deadlines.Avoids distractions.


Respondents with higher Reliable scores tend to have higher scores on:

• Empathetic• Flexible• Imaginative• Systematic• Proactive• Detailed• Positive• Energetic• Results• Change• Work

Respondents with lower Reliable scores tend to have higher scores on:

• Self-critical

pmp


manualDetailed


Tends not to check work. Overlooks mistakes and errors. May lose things. Hasan untidy approach to work.


Likes to get the details right. Checks work. Spots errors. Prefers work to be tidyand up to date.


Respondents with higher Detailed scores tend to have higher scores on:

• Empathetic• Flexible• Imaginative• Proactive• Systematic• Reliable• Positive• Energetic• Results• Work

Respondents with lower Detailed scores tend to have higher scores on:

• Self-critical

pmp


manual3.4 PERSONAL ORIENTATION

Resilient


Easily upset. Takes things personally and finds it hard to ignore criticism. Isconcerned about what others think. Is more emotionally affected by events.Strongly moved by feelings. Tends to worry.


Thick skinned. Not concerned by what others think. Able to shrug off criticism.Rarely feels strong emotions. Difficult to provoke or upset. Calm and relaxed.


Respondents with higher Resilient scores tend to have higher scores on:

• Confident• Convincing• Flexible• Proactive• Systematic• Reliable• Positive• Energetic• Results• Change• Self-directing

Respondents with lower Resilient scores tend to have higher scores on:

• Self-critical

pmp


manualPositive


Inclined to be pessimistic; expects things to go wrong. Has a negative approachto work. Tends to see obstacles more easily than opportunities.


Takes an optimistic view. Is positive. Doesn’t dwell on problems. Expects thingsto get better.


Respondents with higher Positive scores tend to have higher scores on:

• Empathetic• Proactive• Systematic• Reliable• Detailed• Resilient• Energetic• Results• Change• Self-directing• Work

Respondents with lower Positive scores tend to have higher scores on:

• Self-critical

pmp


manualEnergetic


May lack energy. Becomes tired more quickly. than others. Prefers to take thingsquietly. Generally less active.


Has energy. Likes to be active. Becomes restless with nothing to do. Keepsgoing.


Respondents with higher Energetic scores tend to have higher scores on:

• Confident• Convincing• Empathetic• Imaginative• Proactive• Systematic• Reliable• Detailed• Resilient• Positive• Results• Change• Self-directing• Work

Respondents with lower scores on Energetic tend to have higher scores on:

• Self-critical

pmp


manual3.5 WORK ORIENTATION

Results


Motivated more by process than outcome. Low need for achievement. Able to becontented, even when not working towards a particular goal.


Motivated by results. Sets goals. Is concerned about outcomes. Likes to aim attargets. Has drive.


Respondents with higherResults scores tend to have higher scores on:

• Sociable• Confident• Convincing• Empathetic• Imaginative• Proactive• Systematic• Reliable• Detailed• Change• Self-directing• Work

pmp


manualChange


Likes things to stay as they are. Prefers the familiar and the predictable. May findchange uncomfortable.


Looks for new experiences. Values novelty and change. Dislikes the predictable.Comfortable with the unfamiliar.

RELATIONSHIPS WITH OTHER RPMQ SCALES:Respondents with higher Change scores tend to have higher scores on:

• Sociable• Confident• Convincing• Empathetic• Imaginative• Proactive• Reliable• Resilient• Positive• Energetic• Results• Self-directing• Work

pmp


manualSelf-directing


Unwilling to take on burdensome responsibilities. Would rather shareaccountability for tasks. May feel for more comfortable when somebody else is incharge.


Demotivated when not given responsibility for tasks. Likes to have accountability.Takes responsibility for own work. Dislikes close supervision.


Respondents with higher Self-directing scores tend to have higher scores on:

• Confident• Convincing• Imaginative• Proactive• Resilient• Positive• Results• Change

Respondents with lower scores on Self-directing tend to have higher scores on:

• Team focused• Flexible

pmp


manualWork


More likely to see work as necessary rather than pleasurable. Prefers to havemore leisure time. Outside interests are more important than career.


Enjoys work. Committed to career. Misses work when away. Sees hard work ascharacter building. Puts career first.


Respondents with higher Work scores tend to have higher scores on:

• Empathetic• Imaginative• Systematic• Reliable• Detailed• Positive• Energetic• Results• Change

Respondents with lower Work scores tend to have higher scores on:

• Self-critical

pmp


manualMaterial wealth


Not interested in money or outward displays of wealth. Places more emphasis oninterest or enjoyment at work than on material reward. Doesn’t value the pursuit ofpossessions or comfort. Rejects materialism.


Puts emphasis on material wealth and domestic comfort. Is motivated by financialreward. May be jealous of those who are better off. Likes spending money.


Respondents with higher Material wealth scores tend to have higher scores on:

• Competition

pmp


manualCompetition


Prefers co-operation to competition. Is less concerned about making comparisonsbetween self and others. Loses gracefully. Is interested in own standards ratherthan those of others.


Motivated by competition. Compares own achievements with others. Measuresown success on a relative basis. Strongly dislikes losing. May feel envious ofothers.


Respondents with higher Competition scores tend to have higher scores on:

• Convincing•Material wealth•Results

pmp


manualRecognition


Low need for respect/recognition. Relatively indifferent to people’s opinions.Doesn’t expect thanks for good work. Motivated more by own perceptions of goodwork than other people’s.


Values respect and praise from others (including peers, subordinates, boss).Responds well to judicious praise. Likes to be seen to be good at job. Motivatedby “pats on the back”.


Respondents with higher Recognition scores tend to have lower scores on:

• Resilient

pmp


manual4. ADMINISTRATION

PreamblePrior to administering any personality questionnaire, it is important that bothyou and the candidate(s) are properly prepared. The bulk of this section focusesupon the physical process of administering the PMP, but before discussingthat, please consider these points:-

Candidate advance preparation

¨ In advance of the session, notify the candidate about what they will be doingand why.

¨ Explain how PMP fits into the procedure - selection, development orcounselling.

¨ Advise them of the duration of the preparation and administration session. ¨ Inform them of the arrangements for feedback.

Candidate preparation on the day

¨ Reiterate the key points - nature of the assessment, why you are using PMP,what it can and can’t do, and how feedback will be given.

¨ Provide assurances about the confidentiality of the data; who will have accessto it; and how the data will be stored (NB if electronically, you must complywith the Data Protection Act).

¨ Check for candidate understanding and obtain their consent to be assessed.

PMP Materials

¨ PMP questionnaire booklets and answer sheets (for candidate use).Booklets begin with a page of instructions. This is followed by the questionnaireitself.

¨ PMP scoring keys and profile charts (for assessor use) are alsoprovided. The use of these is described in Sections 5 and 6 respectively.

pmp


manual

• a questionnaire booklet• an answer sheet• 2 pencils• a rubber

ADMINISTRATION OF THE PMP

Ask the respondents to open their questionnaire booklets to theinstructions page.

Pause. Then say:

“Please follow the instructions while I read them.

The PMP is a self-report questionnaire. You are asked to rate yourself on anumber of statements which make up the questionnaire.

After reading each statement, give your answer by marking the answer sheetas shown below:

Fill in circle A if you strongly disagree with the statement.

Fill in circle B if you disagree with the statement.

Fill in circle C if you are in between or you neither agree nor disagree with thestatement.

Fill in circle D if you agree with the statement.

Fill in circle E if you strongly agree with the statement.”

Ensure that the room in which the questionnaire is to be administeredis free from undue noise and distraction, and will provide adequatelighting, ventilation/heating, seating and space. Ensure that eachrespondent has:

SETTING UP

Administration (continued)

pmp


manualSay: “Now look at the two examples which have been completed.”

Pause while respondents look at the worked examples. Then say:

“In Example 1, the respondent has indicated that they strongly disagree that thestatement, ‘I am often the person with most to say in a group’ applies to them.In Example 2, the person has indicated that they agree with the statement, ‘Rulesare made to be broken’.”

“Are there any questions about these?”

Pause. Deal with any questions raised.

“There are 264 statements in this questionnaire.”

“When giving your ratings, please remember:

• Be as honest about yourself as you can. This is NOT a test.• There are no right answers or wrong answers.• Answer as you think you really are, not how you would like to be.• The questionnaire is concerned with you at work rather than your social life.• Sometimes you might think that your answer depends on the situation.

If so, try to give the rating that is most typical of you.• Try to avoid using the “In Between” rating.• Rub out any answer you want to change.• Although there is no time limit, please work as quickly as you can.”

Pause, then say: “Please fill in your last name, first name and today’s date, in thespaces provided on the answer sheet.”

Pause, then say: “Please turn over the questionnaire booklet to page 1. Pleasecomplete ALL of the questions.”

Check that respondents are giving their ratings correctly on the answersheet. Walk around periodically to check that all is well. If a respondent isgiving his/her ratings very slowly, the administrator should ask him/her totry to speed up and to avoid pondering for too long over each question.

When respondents have finished, collect answer sheets, checking thatresponses have been given for ALL of the questions. Collect questionnairebooklets.

pmp


manual5. SCORING

There are three scoring keys which should be used to score the Answer Sheet .

5.1 Take the first of the three scoring keys and lay it on Side A of the answersheet. Align the scoring key so that the reference circles on the answersheet appear through the arrowed circles on the key.

5.2 Each group (or “string”) of six questionnaire items gives a subtotal on aparticular scale.

5.3 Read off the number below the circle which has been filled in by therespondent.

5.4 Add up the total of the six items in each group (or “string”) and write theresulting numbers in the appropriate boxes on the right hand side of theanswer sheet.

5.5 Note - items may be scored from 1-5 or 5-1, so take care to check that youhave recorded the correct score.

5.6 When the totals for all of the strings on the first scoring key have beenentered in the boxes to the right, turn over the answer sheet to Side B and,using the same scoring key, repeat the steps described above.

5.7 When the first scoring key has been used to score item strings on bothsides of the answer sheet, repeat the whole process using the secondscoring key and then the third.

5.8 After all three scoring keys have been used, subtotals should appear in allof the boxes on both sides of the answer sheet.

5.9 Scale raw scores are produced by adding the two appropriate subtotalsfor the scale, one from each side of the answer sheet.

The easiest way to add up the subtotals to produce the raw scores is to firsttranscribe all of the subtotals (from Side A and Side B of the answer sheets) intothe appropriate boxes on the profile chart. The sequence of boxes on answersheets and profile charts is the same to facilitate this process. Write the sum of thetwo subtotals in the box marked “RS” (Raw Score) on the profile chart underneaththe two subtotals.

The raw score is now ready to be normed. Usually this involves transforming raw scores into sten scores, which can either be entered in the boxes marked “ST” on the profile chart, or marked directly as profiled scores.

pmp


manual6. NORMS

Normative data for a questionnaire are produced by asking a large sample of peopleto complete the questionnaire. The sample of people is called a norm group and thepurpose of collecting normative data is to enable scores derived from subsequentadministrations of the questionnaire to be interpreted by comparing these scores withthose of a norm group. We can then conclude that an individual’s pattern of responseson a particular scale is either around the average in comparison with the norm group,or tending towards one end of the bi-polar dimension. There are a number of ways ofexpressing a normed score. The simplest index is the percentile.

6.1 Percentiles

Normative data are usually published in the form of norm tables. Percentile normtables are used to transform raw scores into percentiles. If a respondent has scored atthe 50th percentile on a particular scale, this means that his/her score is average incomparison with the norm group (the score is higher than 50% of the group and lowerthan the other 50%).

A score at the 89th percentile means that the respondent’s score is higher than 89% ofthe norm group (and lower than the remaining 11%): the score is quite well above theaverage.

Percentiles have the advantage that they are easy to understand. However, they havethe disadvantage that they are not equal units of measurement and tend to exaggeratedifferences near the mean and collapse differences at the low and high extremes ofthe score range.

6.2 Z-scores

Standard scores (such as Z-scores, T-scores and stens) are not susceptible to theproblems of the percentile scale described above. The Z-score scale is the most basicof the standard score systems.

A Z-score is computed from a raw score using the mean and standard deviationfor the norm group, and indicates how many standard deviations a score is aboveor below the mean:

Z-score = raw score - mean standard deviation

A Z-score is therefore a raw score which has been changed to standard deviationunits. A Z-score of 1.00 means that the raw score is one standard deviation abovethe mean (in comparison with the norm group). A Z-score of-2.00 means that the raw score is two standard deviations below the mean(in comparison with the norm group). A Z-score of 0 represents an average score.

pmp


manualAlthough Z-scores are the basis of all standard score scales, they are not widelyused in the interpretation of self-report questionnaires. This is partly because scoresneed to be expressed to one or two decimal places in order to be sufficientlyprecise, and all scores below the average are negative numbers. In order to avoiddecimals and negatives, the sten-score scale may be used.

6.3 Stens (“Standard Tens”)

Sten scores are calculated from Z-scores as follows:

Sten score = (Z-score x 2) + 5.5

The result is then rounded to the nearest whole number (to avoid decimals). Therange of sten scores is 1 (well below average) to 10 (well above average). Al-though it is theoretically possible for a sten score to be below 1 or above 10,scores at these extremes are usually expressed as 1 and 10 respectively in orderto conceptualise all scores in the 1-10 range.

The mean of a sten score scale is 5.5 and the standard deviation is 2. A stenscore of 5 or 6 therefore represents an average score in comparison with thenorm group. A sten of 8 is quite well above average and is roughly equivalent tothe 89th percentile. A score of 3 is quite well below average and is roughlyequivalent to the 11th percentile.

pmp


manual2012 PMP NORMS

The use of computer-based and online testing has seen the virtual disappearance ofpaper based assessment. There are pros and cons to this, but KCP has respectedclient wishes and discontinued the sale of paper versions of PMP.

The current online composite norms have the following characteristics:

N = 1, 100

Composition - administrators/senior clerical, customer service and sales (approx 33%each).

Female/male - 70%/30%

Origin; UK, US, Australia

White 80%, Black 20%

The scale means, SDs and SEms are shown in the following chapter.

pmp


manual7. RELIABILITY

No psychometric instrument can measure an individual characteristic with 100%accuracy. It is therefore important to know about the level of consistency andprecision offered by each scale incorporated in a psychological test orquestionnaire.

7.1 What is Test Reliability?

Test reliability refers to a test’s degree of precision, or accuracy of measurement.To understand this concept, we can use the analogy of measuring distance (or“length”). If we wanted to measure the length of a football pitch, we could pacefrom one end to the other counting each pace as roughly 1 metre. This wouldprovide us with an imprecise indication of the football pitch’s length.

A more reliable (or precise) method would be to use a tape measure. However, atape measure is not the most reliable instrument for measuring a length or distance:very precise devices such as the micrometer can be used to measure the nearestthousandth of a millimetre. So, “pacing”, the tape measure and the micrometerare three methods of measuring distance which differ in their reliabilities. Tomeasure a football pitch it would be absurd to use a micrometer, but “pacing” maybe too inaccurate: the level of reliability offered by the tape measure method maymake it the most appropriate option.

Like the “pacing” and tape measure methods, testing has a degree of inherentunreliability. In evaluating tests or questionnaires for use in assessment, we areconcerned to ensure that the test’s reliability is acceptable. If scores are tooinaccurate, wrong decisions will be made.

The reliability of a test or questionnaire scale is expressed as a correlationcoefficient and is called a reliability coefficient.

A reliability coefficient of 0.0 means the test or scale has zero reliability; it iscompletely inaccurate and the characteristic is being measured no more preciselythan a random guess. A reliability of 1.0 means that the test or scale is 100%reliable; it measures the characteristic with perfect precision. However, this degreeof reliability is never actually found.

A reliability coefficient is thus an index with a theoretical range of 0 to 1. In practice,reliabilities generally vary between 0.5 and 0.9.

As a rule of thumb, it is generally not advisable to use a questionnaire with scalereliabilities below 0.6.

pmp


manual7.2 Different Kinds of Reliability

TEST-RETEST RELIABILITYThe test-retest reliability coefficient is the correlation between the scores obtainedby the same person on two separate administrations of the test or questionnaire.If results are generally different on the second administration from the first, thetest-retest reliability coefficient will be low. If there is a good relationship betweenscores on the two occasions, the coefficient will be high.

ALTERNATE-FORM RELIABILITYThis is computed by comparing scores obtained by the same individuals on twoforms (or versions) of the same test. Of course, this method is limited to tests andquestionnaire scales for which more than one form exists.

INTERNAL CONSISTENCYOnly one administration of the questionnaire is required to compute the internalconsistency coefficients of the scales. These are derived by splitting the scalequestions into two and then correlating the respective totals for the two halves.Sophisticated statistical techniques can be applied to split the questions in differentways in order to take an average of the correlations resulting from each split. Thisaverage correlation is then identified as the internal consistency of the scale, andshould be high if all the questions making up a scale are measuring the samecharacteristic.

7.3 Internal Consistency Estimates for PMP

The following statistics were derived from the standardisation sample of PMP. Thestatistics are based on the 12 items per scale selected for the final version of PMP.It should be noted that these reliability coefficients are probably an underestimate ofthe true scale reliabilities as they have not been corrected for restriction in range .

pmp


manual

PMP INTERNAL CONSISTENCIES

RESPONSE STYLEMean SD Internal Consistency

(Cronbach’s Alpha)Self Critical 34.16 7.57 0.76

PEOPLE ORIENTATIONMean SD Internal Consistency

(Cronbach’s Alpha)

Sociable 41.80 6.22 0.74

Confident 41.02 8.50 0.86

Convincing 34.18 6.14 0.71

Team focused 42.65 4.62 0.64

Empathetic 40.69 6.12 0.70

Flexible 39.24 5.69 0.74

TASK ORIENTATIONMean SD Internal Consistency

(Cronbach’s Alpha)Imaginative 34.33 4.00 0.76

Proactive 39.00 4.93 0.65

Systematic 38.58 7.58 0.81

Reliable 40.24 7.34 0.84

Detailed 41.29 5.50 0.65

pmp


manualPERSONAL ORIENTATION

Mean SD Internal Consistency(Cronbach’s Alpha)

Resilient 36.76 7.78 0.78

Positive 37.25 5.49 0.62

Energetic 37.69 6.60 0.69

WORK ORIENTATIONMean SD Internal Consistency

(Cronbach’s Alpha)

Results 40.25 5.30 0.75

Change 39.93 5.78 0.68

Self-directing 38.62 5.17 0.62

Work 39.15 7.15 0.71

Material wealth 36.48 6.74 0.80

Competition 35.27 7.38 0.81

Recognition 39.07 7.21 0.83

pmp


manual8. VALIDITY

What is Test Validity?

The validity of a test or questionnaire is concerned with the extent to which it actuallymeasures what it is supposed to measure. In assessment, a questionnaire’s validityis usually a measure of its relevance to the job (or training) content. In the case ofPMP, however, we are interested in the extent to which the questionnaire predictsjob satisfaction and length of tenure, as well as the extent to which scores predictjob performance.

Different Kinds of Validity

The validity of a questionnaire can take a number of forms:

FACE VALIDITYThis is concerned with the extent to which a questionnaire appears to have relevanceto a particular job. Face validity is important since poor perceived relevance cancause resistance on the part of the respondents.

CONTENT VALIDITYThis is similar to face validity in that it concerns the extent to which questionnaireitems are representative of the attribute to be measured. Content validity can beassessed in a structured way by performing a detailed job analysis, thereby derivingdescriptions of the job content, which can then be matched with the test content. In asimple case, it may be discovered that a sales job involves persuading others to apoint of view: if a questionnaire scale incorporates items specifically exploring thiskind of behaviour its content validity would be high.

In developing psychometric instruments the process of establishing content validityis important as this enhances face validity, and provides a basis for concurrent andpredictive validity.CONCURRENT VALIDITYThis is based on the correlation between scale scores and some measure (or“criterion”) of performance or satisfaction in the job. Both the scale scores and theperformance/satisfaction data are obtained at the same time from existing jobincumbents.

PREDICTIVE VALIDITYThis is measured by obtaining scale scores (usually at the point of selection) andthen collecting performance and/or job satisfaction data at some subsequent point(e.g. 1 year after selection). This kind of validity is the most salient to the selectionsituation, in which the aim is to find out whether scores do predict subsequentperformance or job tenure.

THE VALIDITY OF RPMQ

8.1 FACE AND CONTENT VALIDITY

pmp


manualIn summary, these four kinds of validity are important in the evaluation of psychometricinstruments. Face validity and content validity are rational (rather than empirical)methods and as such do not result in a numerical validity coefficient. The latter canonly be computed using the concurrent or predictive approaches. However, theempirical methods are dependent on the availability of a reliable criterion (ofperformance or tenure). Performance measures are often based on managers’ ratingswith reliabilities as low as 0.2, which obviously undermines the validation. Concurrentand predictive validation studies are also dependent on the availability of largesamples of job incumbents.

THE VALIDITY OF PMP

8.1 FACE AND CONTENT VALIDITY

Careful design and qualitative screening of the questionnaire items in the developmentof PMP has served to maximise both the face validity and the content validity of thequestionnaire as an assessment instrument for use at non-managerial, administrative,sales and customer service levels.

All of the questionnaire scales have been defined in terms of behaviours, preferencesor values that are strictly relevant to the incumbent’s performance and/or jobsatisfaction at work. The high internal consistencies (see Section 8.3) of the PMPscales provide further evidence that the questionnaire items relate to scale definitionsin a homogenous, consistent and unambiguous fashion.

.

pmp


manual8.2.1 AT HOME AND ON SITE CALL CENTRE AGENTS

One hundred and thirty eight call centre agents were asked to participate in thisstudy by completing the PMP questionnaire. Performance ratings for a number ofdifferent aspects of the job were collected from appraisal data, as well as overallfinancial performance ratings.

The PMP scales were then correlated with overall performance ratings to producevalidation coefficients. The inter-rater reliability of the overall performance ratingswas also computed and used to correct validation coefficients for attenuation.

ON SITE RESULTS

Prediction of Overall Performance

Multiple R = 0.49Significance = 0.01

ScalesNoveltyImaginativeMaterial Wealth

Prediction of Conversion Rate


ScalesReliableProactive

Prediction of Premium Value

R = 0.30Significance = 0.01

ScalePositive

pmp


manualAT HOME RESULTS

Prediction of Overall Performance


ScalesConfidentEmpatheticFlexibleDetailedReliableResilientResultsRecognition

Prediction of Conversion Rate


ScalesResilientSelf-directingResultsMaterial Wealth

Prediction of Premium Value

R = 0.41Significance = 0.03

ScaleReliable

© 2010 Knight Chapman Psychological Ltd. All rights reserved.This Booklet may not be reproduced by any means without the written permission of the publisher.

KCP Questionnaire Series

Documents

personality and motivation profiler SHORT MANUAL.pdf · of measurement. In particular, the PMP is designed to aid in the assessment, development and counselling of non-managerial