Workshop on Q Workshop on Q methodologymethodology

Midwestern Educational Research AssociationMidwestern Educational Research AssociationSt. Louis, MOSt. Louis, MO8 – 9:20 AM8 – 9:20 AM

October 25, 2007October 25, 2007Sue RamloSue RamloJoe JurczykJoe Jurczyk

About the presenters:About the presenters:

Sue Ramlo, PhDSue Ramlo, PhD Physicist w/ PhD in Physicist w/ PhD in

Curricular & Instructional Curricular & Instructional StudiesStudies

Presentations of Q studiesPresentations of Q studies Published Q studiesPublished Q studies Editorial board of both Q Editorial board of both Q

journals (English) journals (English) • Operant SubjectivityOperant Subjectivity

• Human SubjectivityHuman Subjectivity Previous Q workshop Previous Q workshop

developerdeveloper

Joe JurczykJoe Jurczyk Systems Engineer w/ MBA Systems Engineer w/ MBA

& ABD in C&I& ABD in C&I Dissertation – developing & Dissertation – developing &

evaluating a versatile on-evaluating a versatile on-line Q-sort toolline Q-sort tool

Presentations of Q studiesPresentations of Q studies Previous Q workshop Previous Q workshop

presenterpresenter

Workshop OutlineWorkshop Outline Introduction to Q methodologyIntroduction to Q methodology

Sorting items onto a normal Gaussian Sorting items onto a normal Gaussian distributiondistribution

Concourse of items & the Q sampleConcourse of items & the Q sample

Analyzing the sorts using PQ Method – Analyzing the sorts using PQ Method – factors & interpretationfactors & interpretation

Why groups of people? Why groups of people?

Examples of Q studies Examples of Q studies

Introduction to Q methodology - Introduction to Q methodology - OverviewOverview

Investigate the views, or perspectives, of a Investigate the views, or perspectives, of a person or a group of people.person or a group of people.

Process involves: Process involves: Creating a concourse of items (text, sounds, Creating a concourse of items (text, sounds,

pictures). pictures).

Sorting a sample of the items into a normal Sorting a sample of the items into a normal Gaussian distribution.Gaussian distribution.

Sorts are factor analysized to group people Sorts are factor analysized to group people with similar sorts (Note: R FA groups items). with similar sorts (Note: R FA groups items).

Sorting items onto a normal Sorting items onto a normal Gaussian distributionGaussian distribution

Pre-sort into 3 pilesPre-sort into 3 piles

Distribute (& re-Distribute (& re-distribute) to fit distribute) to fit specific normal specific normal Gaussian Gaussian distribution. distribution.

Most UNlike my view(~14 statements here)

MOST like my view

(~14 statements here)

Neutral view about this statement(~14 statements here)

Now you try it!Now you try it! Condition of instruction – Because you’ve been working so Condition of instruction – Because you’ve been working so

hard, your boss is going to give you a bonus in the form of hard, your boss is going to give you a bonus in the form of a one year vehicle lease & he wants your input. In the a one year vehicle lease & he wants your input. In the envelope you have received, there are pictures of a variety envelope you have received, there are pictures of a variety of different vehicles. Rate these items on a scale of “most of different vehicles. Rate these items on a scale of “most desired” (+4) to “most undesired” (-4)….”desired” (+4) to “most undesired” (-4)….”

Most undesirable(~7 items)

MOST desirable

(~7 items)

Neutral view about this statement(~7 items)

Pre-sort into 3 pilesPre-sort into 3 piles

Distribute (& re-distribute) Distribute (& re-distribute) to fit specific normal to fit specific normal Gaussian distribution. Gaussian distribution.

1 2 2 3 4 3 2 2 1

Most undesirable neutral

Most desirable

-4 -3 -2 -1 0 1 2 3 4

Introduction to Q methodology – Introduction to Q methodology – historical backgroundhistorical background

Developed in 1935 by William Stephenson Developed in 1935 by William Stephenson

Physicist-psychologist Physicist-psychologist

Student of SpearmanStudent of Spearman

A Study of Behavior, 1955A Study of Behavior, 1955

Q for QuantumQ for Quantum

Most typically used in fields of psychology, marketing, Most typically used in fields of psychology, marketing, advertising, political science…advertising, political science…

Mixes quantitative & qualitative aspects of researchMixes quantitative & qualitative aspects of research

Why not another method to Why not another method to determine views?determine views?

Alternatives for determining perspectives are not Alternatives for determining perspectives are not as powerful as Q (McKeown, 2001). as powerful as Q (McKeown, 2001).

Likert scale evaluations and rank ordering lead Likert scale evaluations and rank ordering lead to the loss of meaning (McKeown, 2001) – to the loss of meaning (McKeown, 2001) – e.g. e.g. aggregate resultsaggregate results

Because Q measures personal opinion Because Q measures personal opinion regarding a concourse of items related to a regarding a concourse of items related to a topic, validity is not a consideration (Brown, topic, validity is not a consideration (Brown, 1999).1999).

What is meant by subjectivity?What is meant by subjectivity? What do you see? Bunny? Duck?What do you see? Bunny? Duck? Is one right & the other wrong or are they Is one right & the other wrong or are they

both just different views?both just different views?

Any Q study starts with a Any Q study starts with a concourse:concourse:

Can consist of words / statements, pictures, sounds, smells…Can consist of words / statements, pictures, sounds, smells…

Subjective Subjective Not “It’s raining” Not “It’s raining”

But can be “the rain makes me feel sad” –or- “I love to walk in the But can be “the rain makes me feel sad” –or- “I love to walk in the rain.”rain.”

Items are interpreted by participant – removes the view of the Items are interpreted by participant – removes the view of the researcher & the issue of validity / reliability.researcher & the issue of validity / reliability.

Select the Q sample from the concourseSelect the Q sample from the concourse

Try to “balance” the Q sampleTry to “balance” the Q sample

Sample needs to be sufficiently “large” {sample size here is Sample needs to be sufficiently “large” {sample size here is the number of items, not the number of people in the study}the number of items, not the number of people in the study}

Concourse of items – 3 Concourse of items – 3 possibilitiespossibilities

““Naturalistic” statements - taken from participants’ oral or written Naturalistic” statements - taken from participants’ oral or written communications.communications.

InterviewsInterviews

Focus GroupsFocus Groups

“ “Ready made” statements - from sources other than those of the Ready made” statements - from sources other than those of the participants’ communications. participants’ communications.

Likert survey itemsLikert survey items

Based on knowledge of researcher w/o interviewsBased on knowledge of researcher w/o interviews

Hybrid - combine both “naturalistic” and “ready made” items. Hybrid - combine both “naturalistic” and “ready made” items.

One is not inherently superior to the other (McKeown & Thomas, One is not inherently superior to the other (McKeown & Thomas, 1988). 1988).

Researcher selects the type best suited to the project at handResearcher selects the type best suited to the project at hand

Q sample – select items from the Q sample – select items from the concourse to use in the study.concourse to use in the study.

Example: Selection from a Q sample of 44 (chosen Example: Selection from a Q sample of 44 (chosen from a concourse of 74)from a concourse of 74)

6

I have very little control over how much I learn in this

course.

28I often think about how well I u

nderstand the topics

in this class.

7

If I can't understand something right away, I w

ill keep on

trying

29What I le

arn in this class will help me in other

classes.

8

Working with classmates inside &/or outside this class

helps me learn.

30What I le

arn in this class will help me when I get a

job in my field.

9Learning something really well takes me a long time.

31I would learn more in this class if

I spent more time

studying and doing homework.

10If I c

annot understand something quickly, it usually

means I will never understand it.

32I only do as much work as necessary to pass th

is

course.

Condition of instructionCondition of instruction

Participants sort based upon a condition of instruction (or Participants sort based upon a condition of instruction (or multiple conditions).multiple conditions).

E.g. Sort the following statements as they relate to E.g. Sort the following statements as they relate to your views about learning in this class.your views about learning in this class.

The statements are matters of subjective opinion and may The statements are matters of subjective opinion and may mean different things to different people.mean different things to different people.

Meaning is determined by sorter, not researcherMeaning is determined by sorter, not researcher

Reason why validity is not a considerationReason why validity is not a consideration

e.g. I worked hard in this class.e.g. I worked hard in this class.

Analyzing Q sortsAnalyzing Q sorts

SPSS & SAS not really designed for Q SPSS & SAS not really designed for Q sorts – you mess with weightings, etc.sorts – you mess with weightings, etc.

Need software designed for Q Need software designed for Q methodologymethodology PCQPCQ PQMethodPQMethod QUANALQUANAL

Factor AnalysisFactor Analysis Higher order correlationHigher order correlation

Used to determine patterns in a data setUsed to determine patterns in a data set

R-factor analysis groups items (people are rows, items R-factor analysis groups items (people are rows, items are in columns). Factors represent similar items. are in columns). Factors represent similar items. ObjectiveObjective..

Q-factor analysis groups people (people are in columns, Q-factor analysis groups people (people are in columns, items are in rows). The factors represent people with items are in rows). The factors represent people with similar topologies. similar topologies. ObjectiveObjective

Q methodology is not Q FA but does group people based Q methodology is not Q FA but does group people based upon their VIEWS on a subject. Factors represent similar upon their VIEWS on a subject. Factors represent similar views about a topic. views about a topic. Subjective.Subjective.

PQ Method to determine factors & PQ Method to determine factors & assist in their interpretationassist in their interpretation

PQMethodPQMethod Free download (start at Free download (start at www.qmethod.orgwww.qmethod.org) ) DOS basedDOS based Designed for handling Q sort entry and Designed for handling Q sort entry and

analysesanalyses ChoicesChoices

Centroid versus Principal Components factor Centroid versus Principal Components factor extractionextraction

Graphical hand rotation versus VarimaxGraphical hand rotation versus Varimax Start PQMethodStart PQMethod

PQMethod AnalysesPQMethod Analyses

Creates print out with:Creates print out with: Factor loadings Factor loadings Factor correlationsFactor correlations Distinguishing statementsDistinguishing statements Consensus statements, etcConsensus statements, etc Example Example

- knowledge Tech Physics sorts Ramlo 2006.lis- knowledge Tech Physics sorts Ramlo 2006.lis..

ResultsResults

Different factors represent the various Different factors represent the various views within the P-setviews within the P-set More democratic, not simply majority “wins”More democratic, not simply majority “wins” Allows further investigation (linear regression, Allows further investigation (linear regression,

etc) especially if groups not known a priorietc) especially if groups not known a priori

Consensus allows researcher to see Consensus allows researcher to see where there is agreementwhere there is agreement Organizational change theoryOrganizational change theory

For more on Q methodology:For more on Q methodology: www.qmethod.orgwww.qmethod.org

I4S – International Society for the Scientific Study of I4S – International Society for the Scientific Study of Subjectivity; Next conference in Hamilton, ON; Subjectivity; Next conference in Hamilton, ON; Sept/Oct. 2008Sept/Oct. 2008

Brown, S. R. (1980). Brown, S. R. (1980). Political subjectivity: Applications Political subjectivity: Applications of Q methodology in political scienceof Q methodology in political science. New Haven: Yale . New Haven: Yale University Press.University Press.

McKeown, B., & Thomas, D. (1988). McKeown, B., & Thomas, D. (1988). Q methodologyQ methodology. . Newbury Park, Calif.: Sage Publications.Newbury Park, Calif.: Sage Publications.

Stephenson, W. (1955). Stephenson, W. (1955). The study of behavior: Q-The study of behavior: Q-technique and its methodologytechnique and its methodology. Chicago: University of . Chicago: University of Chicago Press.Chicago Press.