Quantifying the Qualitative

Stephen Billett, Education and Professional Studies

Progression

Why quantify the qualitative?Considerations for research practice and

disseminationAn example ……………………Gathering quantitative with qualitativeImproving reliability Tabulation of qualitative data

Why quantify the qualitative?

Qualitative inquiry suffers from claims of:weak and descriptive analysis, and analytical procedures that limit its potential to

inform others about the phenomena under investigation.

Data can be very time and resource intensive to gather and then not able to be fully used

Risk of qualitative researchers’ audience will only be peers.

Informing policy or practice often demands greater certainty, scale and rigour of the kind promised by quantitative inquiry.

Ways forward

Consider how qualitative-based inquiry can be undertaken and its data analysed to address these concerns

This includes:potentially adding rigour and greater precisions to

quantitative analysisusing informants’, researchers’ and readers’ time

more effectively

Example: Interviews with workers to identify contributions to their learning

PurposeUnderstanding how workers initially learnt their vocational

knowledge and what role workplace experiences play in that learning.

Attempting to understand how ongoing skill development occurs through work activities.

Data gatheringInterviews to identify how workers learnt their vocational

knowledge using their experiences

AnalysisCategorisation and frequencies of responses

Section 1. Initially acquiring skillsDetermine / confirm the nature of their area of skill and inform the respondent that this will be the focus of the questions.

1a. What does it mean to be a skilled (whatever they do)?1b. How did you initially acquire the skills for that job?1c. What was useful about this way of acquiring skills?1d. What were its limitations?1e. What do you think is the ideal way of acquiring skills for your type of work?

Section 2. Continuing development of those skillsThink about the last time you had to solve a problem in your job for which you did not have the necessary skills or knowledge. 2a. How did you go about learning what you needed to know?2b. Who or what in the organisation helped?2c. Did it work, if so why?2d When has this type of learning failed to be useful?

Interview data

‘what is”

‘What should be’

What was useful about the Integrated mode of skill acquisition?

combined theoretical and practical (theory applied practically and realistically) (24)

learning from experts on the job (7)relevant to work skills/ applicable/ specific (7)on-going development of skills/ upgrading (6)apprenticeship important initially but o-j-t essential (4)grounding in basics (transferable skills) (4)hands-on experience (4)good educators/ trainers (4)appropriate background knowledge/ theoretical (2)up to date methods - first hand experience (1)practical experience more important than theory (1)

What was useful about the formal pre-employment mode of skill acquisition?

appropriate background knowledge/ theoretical (10)combined theoretical and practical (theory applied

practically and realistically) (8)relevant to work skills/ applicable/ specific (2)grounding in basics (transferable skills) (1)good educators/ trainers (1)share experiences - learn from others (1)

What was useful about on-the-job mode of skill acquisition?

hands-on experience (79)relevant to work skills/ applicable/ specific (27)learning from experts on the job (20)practical experience more important than theory

(7)up to date methods - first hand experience (4)you get paid while you learn (3)on-going development of skills/ upgrading (2)

What was useful about school followed by on-the-job mode of skill acquisition?

school basics, then o-j-t and learning from others (5)

learning from experts on the job (1)grounding in basics (transferable skills) (1)hands-on experience (1)

Analysis – (e.g. Modes of skill development (1993))

Frequencies and modes

How could the on-the-job mode of skills acquisition be improved?• longer o-j-t time and follow-up (24)• no improvement (18)• more time with expert on the job (16)• more detailed/ advanced special courses when required (13)• formalisation of o-j-t and skills development (11)• more group sharing of skills/ideas (7)• more up to date theory provided/ background knowledge (7)• text to support/ complement o-j-t (5)• better trainers - capable/qualified/ up to date (5)• on-going refresher courses for staff and trainers (4)• more specific o-j-t -including vendor training (3)• better communication between management/staff re staff's training needs (3)• more motivation and incentive to learn/improve/achieve (2)• better resources (2)• more varied work experience (2)• job training off-site (1)• improved o-j-t and personnel exchange/rotation (1)• keep up with modern technological changes (1)• standardised criteria/ expectations for trade/ profession (1)• cater for individual differences and capabilities (1)

Use a question – work based learning could be better if ….

Integrated Formal pre-employment

On-the-job School

Actual n=67 (27%) 23 (9%) 148 (59%) 8 (3.2%)

Ideal N=75 (32%) 19 (8.1%) 138 (59%) 2 (.8%)

Example – comparing question 1a and 1e

Using qualitative responses for a quantitative comparison

Incorporating quantitative procedures within qualitative method

Example of interview schedule –

Improving reliability Inter-rater reliability1. Getting someone else to independently code the data and then 2. Checking for the degree of similarity and difference between the two

raters.3. Understand difference and then each code another set of data.4. Then review amend data coding

Table 5.5 Frequency of congruence between raters on all codes

Number of items coded Agreement between researcher and independent rater (% in parentheses)

51 47 (92)

48 38 (79)

65 53 (82)

47 36 (77)

Total 211 Total 174 (83)

Tabulation of qualitative dataTable 6.4 Subject attitudes (Nunnaly, 1967)

Sub Preference for (likes) Preference against (dislikes) A1 satisfying clients - chemical and colour awkward clients and streaking A2 working with (some) people - cutting

colour way some clients treat her

F3 client contact, conversation and cutting colouring - up-fashion cuts and weddings F4 social aspects of work (clients and co-

workers) overlap of responsibilities between family and work

F5 interaction with clients - cutting perming and chemicals F6 satisfying clients - creative work unhappy clients and those who are rude on the

phone C7 helping clients - self-esteem duties of running a business C8 satisfying and working with clients unhappy clients C9 different tasks and working with clients bad days - tints on tints O10 people and clients nothing O11 meeting people styling

Table 6.5 Commonality in categorisation across settings

Settings Commonality in categories Differences in categories Salon A

Personal backgrounds and concerns about clients

A1 - client lifestyle A2 - openness/ personal confidence

Salon C Concerns about treatments and smooth running of production line

C7 - what they want to spend (just a trim) C8 - clients' mood C9 - all the same

Salon F

Occupational and interest factors associated with clients (e.g. freedom to be creative)

F3 - occupation and age F4 - personal standing/values F5 - age and openness F6 - openness and sort of demands

Salon O Characteristics of clients (age or frequency of visits)

O10 - age and client status O11 - status