Chapter 20-Newman and Ramlo_from Tashakkori I Proof [Low Res]

USING Q METHODOLOGY ANDQ FACTOR ANALYSIS IN MIXEDMETHODS RESEARCH

� Isadore Newman and Susan Ramlo

� 505

20

After reading this chapter, the reader will be able to

• describe and define Q methodology,

• describe and define Q factor analysis,

• identify the historical roots of both Q methodology and Q factoranalysis,

• differentiate between Q methodology and Q factor analysis,

• give examples of when it would be appropriate to use Q method-ology and Q factor analysis,

• give examples of the types of studies that would involve Q method-ology and those that would involve Q factor analysis, and

• identify some strengths and weaknesses for using Q methodologyand Q factor analysis as data reduction techniques.

Object ives

506–––�–––Issues Regarding Methods and Methodology

�� Introduction

The purpose of this chapter is to identify twomultivariate techniques that can be used tofacilitate the interpretation of mixed methodsresearch. These techniques can aid researchersin answering their research questions bydemonstrating how to disaggregate or aggre-gate their data. More specifically, our purposeis to introduce two multivariate techniques,Q methodology and Q factor analysis, toreaders by describing these techniques as wellas by giving examples that should assist read-ers in performing their own Q methodologyand Q factor analysis studies. Finally, we dis-cuss how researchers can take their research astep farther by answering more-sophisticatedresearch questions that include groups ofpeople. For instance, these groups, whetherderived empirically such as via Q factor analy-sis or theoretically such as psychosocial stages,can be used as variables within other types ofmultivariate analyses.

Statistical analyses frequently produceprobabilistic conclusions, not absolutetruths. The use of these quantitative tech-niques to aid in the interpretation of quali-tative findings is therefore not inconsistentwith the ontology of universal laws basedon an objective reality. The philosophy thatunderlies quantitative judgments is based onmaking statements about relationships,while acknowledging that these measuresare not free of error and may be situation orgroup specific (interactions). This is notinconsistent with the qualitative philosophythat different relationships may exist for dif-ferent situations, reflecting multiple realities.

The emphasis of this chapter is on howto use sophisticated multivariate datareduction techniques, with an emphasis onQ methodology and Q factor analysis, bothof which can be considered to be mixedmethodologies. Both Q methodology(McKeown, 2001; Ward, 1963) and Q fac-tor analysis (Burt, 1941; Cattell, 1978) rep-resent approaches for grouping peoplebased on their typologies (profiles).

Creswell (2010 [this volume]) identifies theneed for the development of new techniques

and procedures to be used in mixed methodsresearch. We strongly agree with this position,but we believe it would be helpful to begin thediscussion by considering two techniques thathave existed for decades, Q methodology andQ factor analysis. These and other multivari-ate procedures are frequently not consideredor applied to mixed methods and qualitativeresearch.

Q methodology has been discussed inqualitative annals (Brown, 2008; Watts &Stenner, 2005) but also has been designatedspecifically as a quantitative method(Block, 2008; Brown, 2008; McKeown &Thomas, 1988; Nunnally, 1978). However,Q methodology best fits the framework ofmixed methods research as described byCreswell (2010), Tashakkori and Teddlie(1998), Newman and Benz (1998),Ridenour and Newman (2008), and others,as is evidenced by the section dedicated toQ in the edited book Mixed Methods inPsychology (Stenner & Stainton-Rogers,2004). Stenner and Stainton-Rogers (2004)state that Q methodology is such a qualita-tive–quantitative hybrid that the term“mixed methodology” is not sufficient todescribe its position; they suggest the newterm “qualiquantology.” They base theirstatements on the idea that the philosophi-cal underpinnings of the Q are a mixtureof qualitative and quantitative ideas.Tashakkori and Teddlie (2009) have calledthis type of hybrid, “inherently mixed.”Brown (2008) agrees that with its focus onsubjectivity and, therefore, self-referentialmeaning and interpretation, Q methodol-ogy shares many of the focuses of qualita-tive research while utilizing the type ofstatistical analyses typically found in quan-titative studies.

INTRODUCTION TO QMETHODOLOGY ANDQ FACTOR ANALYSIS

R factor analysis and Q methodologyparallel, respectively, Newtonian physics(models that mathematically predict theforces and motion in the macroscopic real

Using Q Methodology and Q Factor Analysis–––�–––507

world) and quantum mechanics (Howard,2005; Ramlo, 2006; Stephenson, 1982b,1988). Stephenson expanded on the workof Cyril Burt (the creator of Q factoranalysis) and Burt’s and his mentor, CharlesSpearman, to develop Q methodology.Stephenson was able to blend the con-cepts of factor analysis with conceptsfrom quantum mechanics (Ramlo, 2006;Stephenson, 1982a, 1982b, 1987, 1988) inorder to address his desire to objectivelymeasure subjectivity (Stephenson, Brown,& Brenner, 1972).

Like Cyril Burt, Stephenson learnedabout factor analysis from its creator,Charles Spearman (Brown, 1998). As aPhD in both psychology and physics,Stephenson was able to blend the conceptsof factor analysis with concepts from quan-tum mechanics, a field of physics that stud-ies particles at the subatomic level whereone can never measure the exact location ofa particle but instead only can attempt topredict its “behavior.” Although connectedmathematically, quantum mechanics wasless readily accepted as Newtonian laws ofmotion. Even the great Einstein took issuewith some of the ideas of quantum mechan-ics (Howard, 2005; Sauer, 2008), not unlikesome key social science researchers whoseemingly disregarded Q methodology, atleast in part, due to its subjectivity and,potentially, its mixed methods frameworksince, as Stenner and Stainton-Rogers(2004) indicate, such a mixed methodshybrid ought to be discomforting. This dis-comfort emerges from the reorganization ofdistinct ideas which come together, crossboundaries, and form something new.

A similar discussion between two com-peting conceptions of research, such as thedebate between Newtonian and quantumphysics, is found within Mixed MethodsResearch: Exploring the InteractiveContinuum (Ridenour & Newman, 2008),in which Ridenour and Newman attempt toconnect quantitative and qualitative con-ceptualizations. As they explain, mixedmethods research does not consist of adichotomy of quantitative and qualitative,but instead represents a third research

model. Q methodology fits well into thisidea of mixed methods, although whatexactly that model consists of is still indebate (Creswell, 2010; Tashakkori &Teddlie, 2009).

It is important to realize that Q method-ology is not simply a statistical techniquebut, instead, a complete methodology(McKeown & Thomas, 1988; Stephenson,1953) where the focus is on measuring sub-jectivity (Brown, 1980, 2008; McKeown &Thomas, 1988; Stephenson, 1953). Thus, Qmethodology is a set of procedures, theory,and philosophy that focuses on the study ofsubjectivity, where subjectivity is typicallyassociated with qualitative research andobjectivity is usually associated with quanti-tative research (Brown, 2008; Stenner &Stainton-Rogers, 2004). Similarly, Q method-o logy fits into the qualitative framework ofnaturalistic contextualization of research(Stenner & Stainton-Rogers, 2004). Com -pared to typical qualitative research,though, Q methodology maintains the rela-tionship among themes within the data as itminimizes the impact of the researcher’sframe of reference (Stainton-Rogers, 1995).It does this through complex statistical analy-sis including correlation and factor analysis(Brown, 1980; Stephenson, 1953).

This sophisticated use of statistics hasbeen what has led to the designation ofQ methodology as quantitative (Brown,2008). Yet within Q methodology aspectsof these statistical analyses, especiallywithin the factor analysis, there also existboth qualitative and quantitative aspects(Brown, 2008; Stenner & Stainton-Rogers,2004). Even the issue within mixed meth-ods studies of how to collect two differentstrands of data (Creswell, 2010) is not aproblem within Q methodology. Thesemixed methodological aspects of Q will bediscussed later in this chapter.

Certainly, Q methodology was not origi-nally identified as a mixed method since,as Creswell (2010) states, mixed methodresearch began around 1988. Perhaps this iswhy, since its inception 75 years ago whenStephenson first published an article describ-ing Q methodology in Nature (Stephenson,


1935), Q methodology has held a controver-sial position in social science research that hasled to its relatively small following (Brown,1998). It is only recently that Q methodologyhas become more widely accepted in journalsin a variety of disciplines (A. Wolf and S. Ramlo, personal communication, April 29,2009), possibly due to the greater acceptanceof mixed methodology research.

Q factor analysis, which is different fromQ methodology, also fits into the conceptionof mixed methods research. Although Q fac-tor analysis also groups people, it does notinclude the sorting of items into a grid distrib-ution as a means of measuring subjectivitythe way Q methodology does. In other words,the participants’ sorting of the statements inQ methodology is what determines the factorsand groupings, which makes these cate-gories of operant subjectivity (Brown, 1998).Instead, Cattel (1978) described Q factoranalysis as a means of determining dimen-sions or profiles of people. Within Q factoranalysis the factors or groupings are based onvarious characteristics or data collected, butthey are not categories of operant subjectivitysuch as those in studies involving Q method-ology. Even though the techniques are differ-ent, sometimes the term “Q factor analysis” isused incorrectly to refer to studies that actu-ally involve Q methodology. Adding to theconfusion, the factor analysis of the Q sorts inQ methodology is often referred to as Q fac-tor analysis (McKeown & Thomas, 1988;Stephenson, 1953).

Differences between Q methodologyand Q factor analysis are discussed in detailby their respective creators elsewhere(Stephenson & Burt, 1939). Thus, both ofthese preexisting yet different researchapproaches fit the new age of social scienceresearch that is mixed methods. Specifically,these two methods, Q factor analysis and Qmethodology, use sophisticated statisticaltechniques to reduce large amounts of data.The data collection is typically based onqualitative research, as is the naming of thefactors, or groupings of people.

Stephenson presented the idea of Q metho -dology as a way of investigating people’s

views of any topic. Thus, Q methodologyallows researchers to investigate researchquestions that involve determining the var-ious views within a group about a specifictopic, as well as using those views to inves-tigate how they affect some other aspect ofthe study.

Q methodology is a measure of subjec-tivity that represents an individual’s feelings, opinions, perspectives, or prefer-ences (Brown, 1980, 2008; McKeown &Thomas, 1988; Siegesmund, 2008; Stenner& Stainton-Rogers, 2004; Stephenson,1953). This is consistent with qualitativeresearchers’ focus on the investigation ofsubjectivity (Siegesmund, 2008). Certainly,Q methodology represents a unique wayto measure subjectivity (McKeown &Thomas, 1988; Stephenson, 1953) becauseit allows participants to provide their per-spectives by sorting items, typically state-ments related to the topic, into a sortinggrid determined by the researcher. These Qsorts are then analyzed via factor analysis,which allows those of similar views to begrouped into factors. Thus, within Q,people, not items, are grouped and, there-fore, researchers must have a sufficientnumber of items to determine differencesamong the participants, not a sufficientnumber of participant to determine differ-ences among the items as is typicallyrequired within R factor analysis (Brown,1980; McKeown & Thomas, 1988;Stephenson, 1953).

PERFORMING AQ METHODOLOGY STUDY

Q studies commence with the develop-ment of a collection of items, typicallystatements or pictures, which is qualitativein design.1 This collection of items is called“the concourse within Q methodology,”and represents the communications aboutthe topic. These items are typically takenfrom interviews, focus groups, and othersources of dialogue such as newspapers(McKeown & Thomas, 1988). Often the

sramlo

Highlight

“the concourse within Q methodology,” Should not be in quotes... maybe the "concourse" within Q methodology,...


concourse is extracted via a theme analysis(Strauss & Corbin, 1990). The Q sample,which includes the items for participants tosort, is then selected from the concourse(Brown, 1980; McKeown & Thomas,1988; Stephenson, 1953). Typically, theQ sample consists of 30 to 60 items selectedas representative of the concourse (Brown,2008). It is the Q sample that is sorted bythose participating in a Q study (Brown,1980, 2008; McKeown & Thomas, 1988;Stephenson, 1953).

The sorting process is inherently subjec-tive because participants judge each Q sam-ple item relative to the others while placingthem into a distribution based on a condi-tion of instruction, both of which are pro-vided by the researcher (Brown, 1980;McKeown & Thomas, 1988; Stephenson,1953). An example would be to havestudents sort 34 items related to their viewsof learning in a first-semester collegephysics course. Each item is presented on anindividual strip of paper, which is then

placed into the grid shown in Figure 20.1(Ramlo, 2008a, 2008c). Participants maybe interviewed during the sorting process ormay be asked to make written commentsregarding their sorting selections in order tobetter inform the researcher’s interpretationof the results (Brown, 1980; McKeown &Thomas, 1988).

The analyses of Q sorts involve correla-tion, factor analysis, and the calculation offactor scores (Brown, 1980, 1986, 2008;Stephenson, 1953). Although sophisticatedmathematically, parts of this factor analysisprocess are qualitative. The first step of thefactor analysis process in Q methodology isto select the factor extraction method.Typical software used specifically for Qmethodology studies, such as the PQMethod (Brown, 2008; Schmolck, 2002),offers two choices for factor extraction:principal components and Centroid.Principal components analysis is a commonextraction method that is frequently used inR factor analysis where there are 1s in the

2 4 5 5 6 5 5 4 2

Leastlike my

view neutral

Mostlike my

view

−4 −3 −2 −1 0 1 2 3 4

Figure 20.1 Sample Sorting Grid for a Q Methodology Study


diagonal of the matrix. Conceptually, how-ever, principal components analysisassumes that an individual’s sorts areinvariant (correlated at 1.00). It is unlikely,however, that a person would sort items ina Q study identically, at two different times,even if the separation of those sorts wereonly a day or two. Thus, in Q methodology,Centroid is the extraction method ofchoice. Stephenson (1953) originally desig-nated the Centroid extraction because ofthe indeterminacy of its solution (one cor-rect solution does not exist among the infi-nite solutions possible).2

In Q methodology software packages,there are two choices for factor rotationavailable: Varimax and hand rotation. It isthe indeterminacy of the Centroid solutionthat allows the researcher to rotate the fac-tors based on theoretical considerationsusing hand rotation (Brown, 1980, 1986;Stephenson, 1953). Varimax is the pre-ferred quantitative choice in situations suchas R factor analysis because Varimaxallows the researcher to produce simplestructure, maximizing the eigenvalue foreach factor (Stevens, 2002). In this way,Varimax and other similar rotation meth-ods related to simple structure seek to reacha factor structure that is operationally inde-pendent of the researcher (Brown, 1980).However, in Q methodology, the researcheris not interested in objectivity but, instead,in subjectivity, including the subjective rota-tion of factors (Brown, 1980; Stephenson,1953).3 Thus, the use of Centroid extrac-tion in conjunction with hand rotationillustrates the strong qualitative aspect ofthe factor analytic procedures used withinQ methodology, because of its focus onsubjectivity and its involvement of theresearcher in the rotation process.

Following rotation, the researcher mustselect individuals who are represented by afactor; this is called flagging. In Table 20.1,those who are flagged on factors are indi-cated with Xs. Factor descriptions and analy-ses are determined only by those Q sorterswho are flagged on that factor (Brown, 1980;McKeown & Thomas, 1988; Schmolck,

2002; Stephenson, 1953). Flagging isdescribed further within the section on factorscores. Flagging sorters for the factors is nec-essary before the analyses produce a reportthat involves a variety of tables. Althoughdeveloped statistically, these tables assist theresearcher’s description of the various viewsdetermined from the factor scores, beyondthe types of tables that are generated in morestandard statistical packages such as SPSS.These tables will be discussed in detail withinthe sections that follow. For instance, thesorts of those who are represented by a par-ticular factor are used to create one sort thatrepresents that factor’s view, also known as arepresentative sort (Brown, 1980, 2008;McKeown & Thomas, 1988). In addition,distinguishing factor statements and consen-sus statements also are identified via theanalyses (Brown, 1980; McKeown &Thomas, 1988). Although the analyses ofthese statements include calculating statisticalsignificance, which is highly quantitative, theinterpreting and naming of the factors fallsinto a more typical qualitative framework(Stenner & Stainton-Rogers, 2004).

In order to best demonstrate the Q methodology research process, in the fol-lowing section we discuss ongoing studiesabout how students view their learning in afirst-semester physics course, referencedearlier within this chapter. More details ofthese studies are available via a variety ofpublications of this research (Ramlo,2006/2007, 2008a, 2008b, 2008c).

EXAMPLE 1: EXAMINING STUDENTEPISTEMOLOGY WITH QMETHODOLOGY

This example is primarily to describe thecreation and revision of a concourse ofstatements and the subsequent Q sample inan ongoing Q methodology study. In addi-tion, we focus on the analyses of this studyand how they are used to interpret per-spectives in a specific study, and how thatinformation can be used in additional wayssuch as correlation and linear regression.


This example begins with the desire of asmall group of faculty to investigatestudents’ views of learning and knowledge,also known as personal epistemology(Chan & Elliott, 2004; Hofer & Pintrich,2002), in a variety of classes at a large,Midwestern public university. This facultygroup also sought to compare instructors’beliefs about learning and knowledge to

those of their students in these selectcourses. Using Q methodology allowedthem to avoid the lengthy interview processtypical of qualitative epistemology studies(Duell & Schommer-Aikins, 2001; Elby &Hammer, 2001; Schraw, Bendixen, &Dunkle, 2002). In addition, Q allowedthem to determine the various epistemo-logical views of students, unlike studies

Table 20.1 Factor Matrix with an x Indicating a Defining Sort

NOTES: The Q sort ID in this table contains demographic information; the first letter represents the students’ major(C = construction, M = mechanical engineering technology, E = Electronic engineering technology, S = Surveying &Mapping), the second letter represents the students’ undergraduate level (F = freshman, S =sophomore, J = junior),the third letter represents the grade received by the student. The first numerical part of the ID represents self-reportedage, and the second set of numbers represents the students’ score on the FMCE at posttest.

Qsort ID 1 2 3 4

1 CJ24D14 0.05 0.45X −0.15 −0.24

2 MJ22C17 0.31 0.00 −0.34 −0.28

3 EF19C24 0.29 0.08 −0.75X 0.09

4 SF18A22 0.22 0.26 0.52X 0.19

5 EF19B31 0.59X 0.41 0.01 0.15

6 CJ21C28 −0.24 0.61X −0.13 0.15

7 MF19C18 0.05 −0.14 0.30X −0.12

8 ES20C44 0.53X −0.45 −0.16 0.15

9 MS22C38 −0.28 −0.08 −0.11 0.28

10 CS19A33 0.69X −0.26 0.27 −0.16

11 MF22D7 −0.09 0.62X −0.12 −0.06

12 CF19A26 0.63X 0.15 0.06 0.01

13 MS20B24 0.57X −0.05 −0.40 −0.11

14 SJ35C15 0.12 0.03 0.05 0.17

15 MS21D12 0.02 0.29 −0.12 0.45X

16 SF19B41 0.13 0.28 0.27 0.01

17 MS27A34 0.78X 0.08 0.31 0.15

18 CF20C25 0.32X −0.03 −0.13 −0.09


that have used Likert-scale surveys (Adams,Perkins, Dubson, Finkelstein, & Wieman,2005; Halloun & Hestenes, 1998; Perkins,Adams, Pollock, Finkelstein, & Wieman,2005; Schommer, 1990).

Yet the concourse of statements for theQ methodology study on student epistemol-ogy began with the popular Likert surveydeveloped by Schommer (1990), which shedeveloped from interviews. A Q sample of32 statements was selected from this 72-itemquestionnaire. Students were required to sortthe Q sample statements; analyses of theQ sorts were performed on each class. Theresults and students’ written comments fromthis initial epistemological study indicatedthat students typically sorted the Schommerstatements—not based on their personalepistemological views but, instead, based ontheir public epistemology. Lising (2005) dif-ferentiated these two views by describingpersonal epistemology as how someone per-ceives their own learning and knowledge.Alternatively, someone’s public epistemologyrepresents how they view others’ epistemol-ogy such as scientists or other authorities.

Thus, the preliminary results led Ramlo(2006/2007) to change the wording of theseinitial Q sample statements to make themmore personal. For example, “Learningsomething really well takes a long time”was changed slightly to “Learning some-thing really well takes me a long time in thiscourse.” These changes were intended tostress to students that they were to reflecton their own personal epistemology relativeto the course they were taking. Studentinterviews also were used to develop anadditional 22 statements, some replacingSchommer survey items, such that the Qsample increased from 32 to 44 statements.Although other courses and instructorsused this revised Q sample, only the resultsof the physics course’s portion of the studyare currently available as a journal article(Ramlo, 2006/2007). This aspect of thestudy allowed the instructors to investigatehow their perceptions of students’ views oflearning in their courses actually comparedto the students’ views.

However, the physics course investigatorwanted to delve more into how students’epistemological views related to their learn-ing of force and motion concepts. Thus, afollow-up study was done with a slightlyrevised Q sample to better investigatestudents’ experiences in the physics class-room and laboratory (Ramlo, 2008a,2008c). This Q sample was not used inother courses, which allowed the researcherto change the Q sample such that it targetedlearning in the first semester of physics only.This subsequent study demonstrated howQ methodology can be used to investigatecollege physics students’ views of theirlearning (Ramlo, 2008a, 2008c) and howthose views affected their learning of impor-tant physics concepts in a first-semesterphysics course (Ramlo, 2008c). In addition,this study contributed to the already largebody of literature on students’ learning offorce and motion concepts, which hasestablished students’ difficulty gainingNewtonian-based understanding of forceand motion concepts (Ramlo, 2008d;Redish, Saul, & Steinberg, 2000; Thornton& Sokoloff, 1998).

Although this study further confirmedthe connection between learning in physicsand students’ epistemologies (Halloun &Hestenes, 1998; Hammer & Elby, 2003;Lising & Elby, 2005), the focus of theremainder of this section is to furtherdescribe the sorting process and introducethe reader to the type of results producedwithin a Q methodology study. The fourbasic types of tables that are generated are(1) factor scores, (2) rank-ordered list of theQ sample statements with z-scores to createa representative sort for each factor, (3) thelist of statements that distinguish each fac-tor from the other, and (4) the list of con-sensus statements that represent agreementamong all the factors (Brown, 1980;McKeown & Thomas, 1988). Each of thesetables will be described related to theRamlo 2008 study (Ramlo, 2008a, 2008c)

In this study (Ramlo, 2008a, 2008c), 18students sorted the 44 statements into thedistribution shown in Figure 20.1. Each of


these statements was on a separate slip ofpaper for ease in sorting. The participantssorted these items based on how theyviewed their learning in this first-semesterphysics course.

Each individual sort was entered into PQMethod, one of several software packagesdesigned specifically for analysis of Q sorts(Schmolck, 2002).4 Only these types ofpackages provide the types of outputreports required to interpret the partici-pants’ view on a topic. As Bazeley (2010[this volume]) suggests, software such asPQ Method is necessary for mixed methodsresearch to effectively integrate differentdata elements and analyses. Although fac-tor analysis more often fits into the moreconventional standard statistical packagessuch as SPSS, the analysis of the Q sortsrequires software that allows the researcherto combine the qualitative and quantitativeaspects of such studies. We agree withOnwuegbuzie and Combs (2010 [this vol-ume]) that one of the difficulties of per-forming mixed methods research is that theresearcher must be competent in analyzingboth qualitative and quantitative data.However, Q methodology and Q factoranalysis actually represents the integrationof qualitative and quantitative methods. AsBazeley suggests, this integration allows theresearcher to produce findings that are ofgreater use and to better address theresearch purpose (Newman, Ridenour,Newman, & DeMarco, 2003). This think-ing is supported by the work of a variety ofmixed methods researchers (Bazeley, 2010;Creswell, 2010; Onwuegbuzie & Combs,2010; Tashakkori & Teddlie, 2009)

Within Q methodology, the analysesproduce an extensive report with a varietyof tables. Four basic types of tables are pro-duced, however. These tables are the basisof the interpretation of the Q sorts and,therefore, the participants’ views. The firsttable is a listing of the factor scores that areused to determine which participants arerepresented by which factors (views). Eachfactor represents a similar perspective orworld view of the topic. The other three

types of tables produced are specifically forinterpreting the views of those representedby each of the perspectives (factors). Thesetables are for the representative sort foreach factor, distinguishing statements foreach factor, and statements representingconsensus among the factors. Each of thesetables is described below in the context ofour example on studying students’ episte-mology in a first-semester physics course.

Table of Factor Scores

The first table, Table 20.1, is the factormatrix for the Ramlo (2008a, 2008c) studydiscussed here. This table illustrates that fourperspectives on learning in this physicscourse, each represented by a factor, werefound in this study. In qualitative research,these factors would be called typologiesbecause they group people of similar views.The table has rows for each sorter thatinclude the sorter’s identification and thatperson’s loadings (correlations) with each ofthe four factors that were retained. If a sorteris primarily represented by one factor, that isindicated by placing in X next to thatperson’s factor score for that particular factor.For instance, in Table 20.1, the sorter in Row1, CJ24D14, has an X placed next to the fac-tor score of .45 in the Factor 2 column. Thus,as one can see from Table 20.1, Typology /Factor 1 is made up of persons 5, 8, 10, 12,13, 17, and 18, and Typology / Factor 2 ismade up of participants 1, 6, and 11. Thatmeans that Factor 2 represents the view heldby CJ24D14 along with two other sorters(Row 6, CJ21C28, and Row 11, MF22D7).This is an important table in that allows us tosee that the 18 sorters can be identified asfour types via data reduction. However, thistable does not explain these types. Othertables must be examined to identify the vari-ous views determined and to name these dif-ferent perspectives. Therefore, to describethese world views, we must examine othertables produced from the analyses.

To learn more about those representedby Factor 1, for instance, their Q sort datamust be analyzed. The remaining analyses

sramlo

Callout

it


are based on the individual’s sorts selectedfor each factor, shown with the Xs.Participants who do not have an X placednext to a factor score are not represented byany of the four factors and, thus, do nothave their sorts included in the analyses. Inthis way, the factor interpretation followsevaluating the factor scores (McKeown &Thomas, 1988).

Representative Sorts for Each Factor

One sort that represents the views of thepeople on each factor is created through theanalyses. This representative sort is createdfrom the Q sorts of those who were selectedas being represented by that factor. Thisrepresentative sort is created via the listingof all the statements, in rank order oflargest positive to largest negative z-score. Itis these z-scores that represent each state-ment’s position in the sorting grid (Brown,1980; McKeown & Thomas, 1988).

The most extreme z-scores (represent-ing the positions toward the outside ofthe sorting grid) are most useful for inter-preting the factor. Table 20.2 lists the topand bottom five statements for Factor 1in this study. The Q sorts for each personwho is represented by Factor 1 were usedto create this statement list, which can beused to create a Q sort representing thatfactor. The ranking of the z-scores wasused to determine the grid position,which also is given in the table. Althoughonly the extreme ends of the grid arereported here, the researcher can use thez-scores to create a complete representa-tive sort for this factor, as already men-tioned. Similarly, the remaining threefactors in this study have statement rank-ings, based on z-scores, which can beused to create the additional Q sorts thatrepresent these three views.

Representative Q sorts, with focus onthose statements on the ends of the grid,

Table 20.2 Factor 1 Tops Five Most Like / Most Unlike Statements

No. Statement z-score Grid Position

12 I like the exactness of math-type subjects. 1.804 4

30 I enjoy solving problems. 1.372 4

10 I can tell when I understand the material in this class. 1.269 3

13 What I learn in this class will help me in other classes. 1.226 3

15 When I don’t understand something in my physics lab, Iask another student to help me understand.

1.011 3

7 Learning something really well takes me a long time inthis course.

−1.275 −3

16 If I am going to understand something in this course, itwill make sense to me right away.

−1.275 −3

5 I have very little control over how much I learn in thiscourse.

−1.310 −3

23 Sometimes I found the lab results hard to truly believe. −1.369 −4

8 In this course, if I don’t understand something quickly, itusually means I won’t understand it.

−1.477 −4


were used, in part, for the interpretationof the epistemological views of thestudents in this study. Because of the impor -tance of the Factor 1 view, we will focuson its interpretation here. From Table 20.2,the most extreme positioned statementsfor Factor 1 indicate that those repre-sented by this view were reflective, helpseeking, and enjoyed math or problemsolving. They also saw the relevance ofthis course to other courses they wouldtake. These students also did not seelearning as immediate (disagree withStatements 7, 8, and 16) but did see thatthey have control over their learning (dis-agree with Statement 5). These state-ments allowed us to consider names forthis factor or world view, such as reflec-tive-learners. However, examining thedistinguishing statements, which allowedus to differentiate this Factor 1 view fromthe remaining three, led to other possiblenames for this view. The next section

describes the use of distinguishing state-ments in this Q methodology study.

Distinguishing Statements

As previously mentioned, the extensivereport produced within Q methodologystudies, includes tables of distinguishingstatements. These tables are created foreach factor in the study. Such statementsdistinguish, here, each factor from the otherthree at a significance level of .05. Again,our primary interest here concerns Factor 1so we will only focus on those statementsthat distinguish Factor 1 from the otherthree factors found in the study. Table 20.3contains these statements for Factor 1.Although the representative sort results,such as those given in Table 20.2, are help-ful in describing a particular factor’s view,the distinguishing statement results provideadditional and often insightful informa-tion for the researcher. The statements that

Table 20.3 Distinguishing Statements for Factor 1

No Distinguishing statementF1 GridPosition

F2 GridPosition

F3 GridPosition

F4 GridPosition

20 When I don’t understand something in myphysics lab, I ask my instructor to help meunderstand.

1 0 −4 4

1 I see the ideas of force and motion ascoherent and interconnected.*

1 −4 −4 −2

29 I find it hard to learn from our textbook.* 0 4 −3 4

17 Sometimes I just have to accept answersfrom my professor even though I don’tunderstand them.

−1 2 1 −4

24 Sometimes I find I have problemsunderstanding the terms used in physics.

−2 3 1 1

16 If I am going to understand something in thiscourse, it will make sense to me right away.*

−3 2 2 3

8 In this course, if I don’t understandsomething quickly, it usually means Iwon’t understand it.*

−4 0 3 1


differentiate Factor 1 from the other threeviews are contained in Table 20.3. It is thistable that reveals that Factor 1 studentsindicated that they sought a coherent viewof force and motion (Statement 1) andbelieved that their learning would take time(Statements 8 and 16). These three state-ments distinguish this view from the othersat the .01 level. In other words, these threestatements distinguish Factor 1 from theother factors; it is more certain, therefore,that this difference is not due to chance.

It is important to note that only thoserepresented by this view agreed that theysought coherence for the force and motionconcepts and disagreed that learning neededto be immediate (Statement 16). This addi-tional information, in conjunction with therepresentative sorts from the other views,allowed the researcher to better reveal theepistemology of this Factor 1 perspective onlearning in this first-semester physics course.This additional insight allowed the researchto select the name of “Reflective learnerswho sought coherence and found it” for thisfactor, or view. Describing this view is espe-cially important, given that the Factor 1view also contains those students whoscored highest on the Force and MotionConceptual Evaluation (FMCE) posttest,thus the additional “and found it” in theview’s name. Thus the Factor 1 epistemo-logical view correlates with the only groupthat had a Newtonian view of force andmotion, as will be discussed further in a sub-sequent section. However, the consensusamong the four factors also revealed impor-tant insight into the physics students’ episte-mological views about the course.

Consensus Statements

Consensus statements do not distinguishbetween any of the factors (Brown, 1980).Thus, in addition to the tables of distinguish-ing statements for each factor, a table of con-sensus statements for the factors also isidentified in the Q methodology analyses’report. Thus in this study, agreement existedacross the four different views about learning

in the physics class. Consensus has allowedresearchers to focus on agreement among dif-ferent views, which can be used to start a dia-logue related to commonality, a key idea inorganizational change (Ramlo, 2005).

In this way, the consensus yielded addi-tional information related to students’ epis-temologies. Here, two consensus statementswere determined. One of these consensusstatements indicated that the physicsstudents agreed that they ask their peers forhelp in understanding the lab activities.Instructors’ observations substantiated thisfinding. These same students disagreed thatthey tried to combine ideas across the labactivities. It is worth noting, however, thatthe lab activities frequently refer to previousactivities from earlier labs and that theseactivities including having students com-pare their current findings to the findingsfrom earlier activities. Thus the consensusstatements here supported the importanceof peer interaction among the lab studentsbut indicated that lab materials and instruc-tors needed to focus more on havingstudents combine ideas across the variouslaboratory activities during the semester.

Correlation of the EpistemologicalViews With Conceptual Understanding

The true purpose of the Ramlo (2008c)epistemology study, however, was to inves-tigate the relationship between students’views about learning and knowledge andtheir understanding of force and motionconcepts. In other words, discovering thefour different views is not the end of ourstory. Instead, we can relate these worldviews to the learning of physics concepts.Thus, we had to ask how well this quasi-qualitative outcome, via Q methodology,relates to an important research questionwithin the physics education research com-munity that could not have easily beendetectable without Q methodology.

The understanding of force and motionconcepts was determined by using theFMCE, an instrument that has been shownto have strong estimates of both validity


and reliability (Ramlo, 2008d), at the endof the semester. Thus, the Q methodologyaspect of the study allowed the researcherto identify four different factors that repre-sented the four differing epistemologiesheld by students in the course. The correla-tions between these factors and the FMCEposttest scores are presented in Table 20.4.

These statistical results indicate that onlythe Factor 1 epistemological view had a pos-itive correlation (.46) with the FMCE posttestscores. Thus, this view is most interesting inthat only this epistemological view representsstudents who have a Newtonian understand-ing of force and motion. The remaining threeviews are negatively correlated with theposttest, and are held by students who didnot have a Newtonian understanding of forceand motion. Therefore, these correlationssuggest more about the epistemological viewsrevealed by the Q methodology aspect of thestudy. This investigation further demon-strated the strength of Q to investigate viewsin the research area of conceptual under-standing in physics. In addition, it demon-strates how the perspectives revealed within aQ study can be used to investigate how thoseviews are related to other student attributes.

The usefulness of Q methodology hasbecome apparent in other studies with dif-ferent purposes. Consensus and differingviews about the creation of a school of tech-nology at a large Midwestern universitywere determined using Q methodology that

enabled discussions about organizationalchange (Ramlo, 2005). The effectiveness ofa reading circle professional developmentexperience to help university faculty reflecton their teaching also was determinedthrough a Q methodology study. Thus,using Q methodology to reveal the differingperspectives, as well as consensus, withingroups of people can address a largenumber of research purposes.

EXAMPLE 2: CLASSIFICATION OFINJURED WORKERS IN RELATIONTO VOCATIONAL TRAINING WITHQ FACTOR ANALYSIS

Although Q factor analysis is a by-person factor analysis similar to Q method-ology, it is not the same as Q methodology.Instead, Q factor analysis only employsone aspect of Stephenson’s procedure: thegrouping of people with factor analysis.Yet this grouping is not based on partici-pants’ sorting of items as it is in Q method-ology (Stephenson, 1953; Watts & Stenner,2005). Q methodology groups peoplebased on subjective data (from the sorts)and Q factor analysis groups people usingdata that may come from a variety ofsources including interviews, observations,surveys, and demographic information.These differences and similarities of Qmethodology and Q factor analysis are

Table 20.4 Correlations Between the Factors (Views) and the FMCE Posttest Scores

NOTES: The FMCE was used for the posttest and has a maximum of 47 points possible. Only Factor 1 had apositive correlation with the posttest.

Factor Post Average Posttest score Std Dev Number of students

F1 0.463 31 7 7

F2 −0.393 16 11 3

F3 −0.171 21 3 3

F4 −0.318 12 N/A 1

sramlo

Callout

citation: (Ramlo & McConnell, 2008)


demonstrated in Figure 20.2. Specifically,for Q factor analysis, the groupings ofpeople into types are based on the shape ofthe responses to the various items, as shownin Figure 20.3. In this figure, althoughPersons 1 and 2 have similar absolute

scores overall, they do not represent thesame type or profile. Instead, Persons 1 and3 have similar shapes to their responses;therefore, the Q factor analysis would con-sider these two persons as representing thesame type or profile.

Figure 20.2 Comparison of Q Methodology and Q Factor Analysis

Q methodologyuses Q sorts

of items.

Q Factoranalysis usesdata (non-Q

sort).

Factoranalysisgroupspeople.

−3

−2

−1

0

1

2

3

Variables

Z-s

core

s

Person 1 Person 2 Person 3

Figure 20.3 Three Persons Representing Two Types / Profiles

NOTE: The plots of Z-scores, called profiles, for three different people are shown here to demonstrate thatPersons 1 and 3 have similar profiles and therefore represent one type. Person 2 has a different profile andtherefore represents a different type.


Analyses in the example of Q factoranalysis described here used QUANAL(Vantubergen, 1975) a program developedfor Q methodology.5 However, unlike astudy using Q methodology, the data sourcewas not participants’ Q sorts. Instead, thedata used within the analyses came from acomputer-based assessment and interviews.The computer-assisted vocational assessmentsystem, the Apticom Aptitude Test Battery,has established estimates of validity and reli-ability and is used for measuring the level ofphysical functioning of people with disabili-ties (Alston & Mngadi, 1992). In ourexample, scores with the Apticom instru-ment along with age, sex, and type andnumber of injuries were used to groupinjured workers into types (Waechter,Newman, & Nolte, 1998).

A number of variations on Q method-ology exist in the literature. The Q factoranalysis we discuss here involved themathematical analyses without the Q sort,thus prompting the use of the QUANALprogram for analysis with quantifieddata from the Apticom along with inter-view and demographic information. TheApticom has 10 scales that were used in astudy along with eight other variables(Waechter et al., 1998). Table 20.5,which contains a sample of the type ofdata used in the Waechter study, will bediscussed further in the context of thestudy. The use of these scales to grouppeople here distinguishes this study fromour first example where the data used togroup people came from the Q sort. Inthis current example, the patterns of the

Table 20.5 Descending Array of z-scores for Each Factor Type

Variable Type I Type II Difference

Motor coordination 0.73 −0.54 1.29

Finger dexterity 1.60 0.84 0.76

Manual dexterity 0.90 0.38 0.52

Eye–hand–foot coordination 0.70 0.30 0.40

Back injury −1.10 −1.07 −0.03

Knee injury −1.12 −1.08 −0.04

Finger or hand injury −1.12 −1.08 −0.04

Male or female −1.11 −1.08 −0.03

Number of injuries −1.10 −1.05 −0.05

Neck injury −1.12 −1.05 −0.07

Clerical perception 0.81 0.99 −0.17

Verbal 0.81 1.05 −0.24

Spatial 0.71 1.02 −0.31

Intelligent 0.91 1.31 −0.40

Numerical 0.97 1.44 −0.48

Age −0.28 0.75 −1.03

NOTE: This table is a subset of items based upon the study by Waechter and colleagues (1998). This specificexample of data is useful in demonstrating how Q factor analysis can be used to differentiate between types. Pleasenote that the data presented here is not exactly the same as that presented in Waechter and colleagues (1998).


data were used to classify people creatingtypologies, sometimes also referred to asprofiles or dimensions (Nunnally, 1978).

The purpose of the injured workersstudy was to develop a process of classify-ing people who experienced work-relatedinjuries and who were identified for voca-tional job retraining. Sixty-seven injuredworkers, 37 males and 30 females, who hadtheir injuries verified, were involved in thisstudy. The data were Q factor analyzedwith QUANAL in order to determine simi-lar patterns in the data (see Figure 20.4)that were used to identify the differenttypologies within the sample. Thus, theanalyses are grouping people based on theirsimilar patterns.

In the injured worker study, two types ofpersons were identified with 59 of the 67 par-ticipants represented by one of the two types.The patterns (shapes) for the two types found,for each of the variables, are represented inFigure 20.4, which shows the z-scores foreach variable for Types I and II plotted as atraditional X–Y graph. Table 20.5 containsthe data used to create figures 20.4 and 20.5.The patterns found were used to interpret thetwo types. This table lists the descending arrayof z-scores for each of these two types ofinjured workers by variable type. The differ-ence between the z-scores listed in the fourthcolumn was used to determine which itemsdifferentiated one type from the other and toindicate where there were similarities.

−1.5

−1

−0.5

0

0.5

1

1.5

2

Mot

or c

oord

inat

ion

Fin

ger

dext

erity

Man

ual d

exte

rity

Eye

-han

d-fo

ot c

oord

inat

ion

Bac

k in

jury

Kne

e in

jury

Fin

ger/

hand

inju

ry

Mal

e/F

emal

e

Num

ber

of in

jurie

s

Nec

k in

jury

Cle

rical

per

cept

ion

Ver

bal

Spa

tial

Inte

llige

nt

Num

eric

al

Age

Variable

Z-s

core

s

Type I Type II

Figure 20.4 Q Factor Analysis Results With Two Typologies

NOTE: Visual representation of the two types, I and II, of injured workers found in the vocational education studyusing the z-score data from Table 20.5.


Figure 20.5 is an alternative presenta-tion of the Table 20.5 data. This can becompared to the more traditional graph ofthe same data illustrated in Figure 20.4. InFigure 20.5 the data are presented using abubble plot, which is one of the dimen-sional display formats for data presenta-tion suggested by Dickinson (2010 [thisvolume]). This type of data visualizationtool provides researchers with another wayto discern patterns. Although Dickinsonpresents examples utilizing bubble plotswith frequency data, other quantitativemeasures also may be represented withinthis format. Specifically, the data displayedin Figure 20.5 use the bubble’s (circle)diameter to reflect the z-score values givenin Table 20.5 for the two different profilesor types. In other words, the bubble withthe largest diameter represents the largestz-score, and so on. Because z-scores can be

positive or negative, these differences mustbe noted by the patterns used for the bub-bles (circles). For instance, in Figure 20.5the positive z-score bubbles have a solidoutline and the negative z-score bubbleshave a dashed outline. By observing thedistinct pattern differences of the data pre-sented in Figure 20.5, the researcher candiscern the two distinct profiles that existin this study.

From Figures 20.4 and 20.5, as well asfrom Table 20.5, one can see that Age andthe Apticom Motor Coordination scale dif-ferentiated between the two types themost. Type I individuals were younger andreceived a higher score on the Motor Coordi -nation, Finger Dexterity, Motor Dexterity,and Eye–Hand–Foot Coordination scales ofthe Apticom. Individuals represented byType II were older, and had higher scores onscales used to measure intelligence, verbal,

1.31 0.75 −1.07 0.99 0.3 0.84 −1.08 −1 −1.08 0.38 −0.54 −1.05 −1.05 1.44 1.02 1.05

0.91 −0.28 −1.1 0.81 0.7 1.6 −1.12 −1.3 −1.12 0.9 0.743 −1.12 −1.1 0.97 0.71 0.81

IQ

scale

age

back

Clerical

eye_hand_ft

finger_dex

finger_hand

gender

knee

manual_dex

motor

neck

num_injury

numerical

spatial

verbal

Factor

Type_II

Type_I

Figure 20.5 Injured Workers Study

NOTE: Alternative representation of the Table 5 z-score data using this bubble plot presentation provided byDickinson (2010).


numerical, and spatial reasoning. The sub-jects’ files also were reviewed to help inter-pret these findings. This part of the studyindicated that Type II individuals werereferred to the testing by their attorneys aspart of the litigation process against theiremployers. Since they were older workers,it is not surprising that they also had longerwork histories. In addition, these workersrefused opportunities to participate inretraining programs. They also had stoppedreceiving Worker’s Compensation, whereasthe Type I workers were still receiving assis-tance. Point biserial correlations also indi-cated that Type II workers scored lower onseveral of the dexterity and coordinationscales of the Apticom than their Type Icounterparts. These results and method ofclassifying injured workers may allow voca-tional evaluations to be better informedabout the types of injured workers and mayimprove decision making about screeningapplications (Waechter et al., 1998).

The method is identical to, but thetopic is different from, an earlier Q factoranalysis study by Newman (1971) wherethe views of both Black and White bas-ketball team members were evaluated bygrouping the team members and thecoach into factors. The data used werefrom the Subjective Perception RatingScales, a semantic differential scale, and abehavioral differential scale. Profilesidentified for White and Black subjectswere then determined from the patternsof responses to these 130 variables. Basically,two profiles or typologies emerged, eachrepresenting a different type across these130 items. These two types were namedWhite Typology and Black Typologysince the most discriminating factorbetween the two factors was race. Itturned out that everyone on Type I wasBlack and everyone on Type II wasWhite, except for one person who loadedsimilarly on both Types. This person wasthe coach (Newman, 1971).

With factor analysis, factors are notstable—they are sample specific. They are

notoriously unstable. We strongly recom-mend that when doing a Q or R factoranalysis, the sample be split in half to seewhat types emerge and then to cross vali-date the types. For example, if four typesemerge in the first Q factor analysis and weget five types from the second sample, wecould correlate the types from Sample 1with the types from Sample 2. To the extentthat this results in similar types in both sam-ples, we have reason to believe that thesetypes are more stable. From these analyses,we may see, for instance, that only three ofthe types replicate between the two samples.These three may then be more likely to bemore stable and this may warrant furtherinvestigation. The other types may be sam-ple specific. Thus, depending on the researchquestion these may or may not also warrantfurther investigation. With R factor analy-sis, other methods tend to be used such asconfirmatory factor analysis or Kaisers fac-tor matching techniques (Newman,Dimitrov, & Waechter, 2000).

FUTURE RESEARCH: EXTENDINGTHE USE OF GROUPS OF PEOPLE

This chapter has demonstrated howsophisticated statistical techniques thatare employed by Q factor analysis and Q methodology can effectively reduce largeamounts of data that are frequently usedin mixed method research. Such datareduction techniques may broaden theresearchers’ ability to interpret the data in amore efficacious manner. In addition, wealso suggest that coupling the groupingswith other statistical techniques allowsresearchers the ability to extend beyond theinformation they would get by simplygrouping. For instance, we could use Chi-squares to compare the groups to investi-gate differences. We also could use thesegroupings as predictor variables in linearregression models.

To further illustrate this concept, pre-vious studies investigating conceptual


understanding have shown that numer-ous student characteristics play roles instudents’ learning of force and motion(Ates & Cataloglu, 2007; Dykstra, Boyle,& Monarch, 1992; Ramlo, 2003, 2007a,2007b; Rowlands, Graham, & Berry,1998). Thus, to further study the poten-tial impact of the epistemology of physicsstudents described in Ramlo (Ramlo,2008a, 2008b), we could control vari-ables shown to influence the learning offorce and motion concepts. Thus, wecould predict the learning of the physicsconcepts at the end of the course (FMCEposttest scores) in a linear regressionmodel that included the epistemologyviews (Q methodology factors) alongwith students’ FMCE pretest scores andprevious physics course experience. Wecould then “test” the effect of the episte-mological views by comparing, statisti-cally, the full model (the one describedabove) to the model that includes onlythe FMCE pretest scores and previousphysics course experience, without theepistemology views, to better understandthe influence of student epistemologieson their learning while “controlling” theother variables (FMCE pretest and previ-ous physics course work). It is importantto remind the reader that although sam-ple size in Q methodology is related tothe number of statements sorted, studiesusing statistical techniques such as linearregression require the researcher to havea sufficiently large sample of people inorder to have sufficient statistical power(Cohen, 1988). Thus a larger studentsample would be very desirable to con-duct the investigation suggested here.However, the current Q methodologystudy and its suggested further investiga-tion offers the potential of gaining greaterinsight into physics students’ thinkingabout their learning and knowledge andits potential relationship to student learn-ing, beyond other epistemology studies inphysics education that have used Likert-type scales (Adams et al., 2005; Gire,

Price, & Jones, 2007; Halloun & Hestenes,1998; Perkins et al., 2005; Perkins, Gratny,Adams, Finkelstein, & Wieman, 2006).

Q factor analysis offers researchers away to create profiles or groupings ofpeople based on patterns of data. Theseprofiles can be used to identify the underly-ing constructs that can assist in classifyingpeople in a meaningful way. Such profilesalso can be used for evaluation purposes,such as the example given about vocationaltraining (Waechter et al., 1998). In thatexample, further research also may haveincluded using the profiles within linearregression analyses. The researchers couldhave extended their study by identifyingimplications drawn from their data thatcould suggest further direction for theirwork, based on their analyses. In such astudy, a full linear regression model couldhave the completion of vocational trainingpredicted by each of the worker types(Type 1 or Type 2) and the intervention.The restricted model could “test” the inter-vention by removing that variable andusing only the injured worker type to pre-dict completing vocational training. Thisnew study would then have fewer variablesin the models because only the injuredworker type is included, omitting all of thevariables used to determine these two types.Thus, the number of variables in the mod-els is reduced, which increases the statisticalpower (Cohen, 1988) and potentiallyincreases the conceptual understanding ofthe models.

Profiles and perspectives determinedwith Q factor analysis or Q methodology,respectively, can be used in studies that havepurposes beyond simply classifying peopleinto groups. It makes sense to use the types(concepts, constructs, factors) instead of allthe individual variables that make up thetype, when developing the linear regressionmodels, or when using other statistical tech-niques. In addition, these profiles could beused to disaggregate data so thatresearchers could further study a particulargroup, such as the Type II workers in the


vocational education study. More data,such as interview responses, could be col-lected on others who fit that profile. Thiscould potentially enhance the insight andunderstanding of the initial findings(Waechter et al., 1998).

Thus, investigations that use Q factoranalysis or Q methodology allow researchersto better study the stakeholders from differ-ent perspectives. This is frequently impor-tant in program evaluation, where there isoften value in addressing the various stake-holder groups differently to ascertain theirneeds, in an attempt to improve the effec-tiveness of the program (McNeil, Newman,& Steinhauser, 2005). To the extent that theevaluation identifies and communicateswith the relevant stakeholder, the morelikely it is to be useful and the recommen-dations implemented. Too often, though,stakeholder groupings are simply based ondemographic characteristics such as ethnic-ity or socioeconomic status. We are sug-gesting that such variables may not be themost appropriate way to group stake-holders. Instead, Q factor analysis and Q methodology can be used to grouppeople more effectively using profiles thatgo beyond such surface characteristics.Therefore, these types of profile analysescan provide the evaluators with additionalinsight into the participants, as demon-strated by Ramlo and others (Newman &Benz, 1987; Ramlo, 2005; Ramlo &McConnell, 2008; Ramlo, McConnell,Duan, & Moore, 2008). In addition, thistype of profile analysis can improve com-munication with the various groups or sub-groups. By identifying the specific needs ofthe various stakeholder groups, evaluatorscan tailor-fit their services and recommen-dations, as suggested by McNeil and col-leagues (2005). This is likely to makeevaluation services more effective and moremeaningful to the stakeholders. The bene-fits of integrating qualitative and quantita-tive methods in evaluation is a pragmaticapproach that is not new and is gainingmore widespread use and acceptance(Creswell, 2010; McNeil et al., 2005;

Newman & Benz, 1987; Onwuegbuzie &Combs, 2010; Tashakkori & Teddlie, 2009).

�� Use of Other MultivariateTechniques to FacilitateInterpretation

Certainly, data for grouping can be eitherquantitative or qualitative. Whichever isused, there always will be some similaritycoefficient (such as correlations, distance,z-scores, variability estimates, density func-tions, etc.) to decide how to group, but onehas to understand the strengths and weak-nesses of such techniques. There is reallyno right or wrong approach but it is impor-tant to understand the approach that ischosen. Different procedures for groupingpeople are based in different assumptions.In other words, one has to determine if theprinciple used for grouping makes logicalsense and is aligned with the purpose of thestudy and the research question. Thus, aresearcher would not use height andweight as characteristics to investigatestudent motivation. Instead, there must bea logical, theoretical link between the char-acteristics of the individuals studied andthe purpose of the study.

This chapter has focused on two specifictypes of multivariate analyses that we believedemonstrate ways to effectively reduce thehuge amount of qualitative variables intoprofiles that can be used to better investigateresearch questions. When using multivariatetechniques for data reduction, one has to besensitive to how data are being aggregated ordisaggregated to better understand the out-comes. For example, factor analysis can beused to aggregate data that was initially qual-itative, to facilitate the interpretation concep-tually. Let’s assume there are a number ofsubjects that have been interviewed and thatthe data from each subject has been initiallycoded, and let’s further assume that four ini-tial codes have emerged. That is, all subjectsinterviewed talk about clarity, flow, persua-siveness, or resources. As one can see from


Figure 20.6, the coding of the interview ofSubject 1 indicates that this subject identifiedclarity, flow, and persuasiveness, but notresources, yielding a matrix code of 1 1 1 0.The coding of the interview of Subject 2 iden-tified persuasive and resources, yielding a

matrix code of 0 0 1 1. All the subjects can bequalitatively coded in a similar manner; thesequalitative codings can then be quantitativelycoded and placed into a matrix made up of 1sand 0s. The following demonstrates how thiscould be done.

...

...1000

...1001

...1100

...0111

.........

...resourcessubj4

...resourcesclarity subj3

...resourcespersuasivesubj2

...persuasiveflowclaritysubj1

...DCBA

- - -

- - -

- - -

- - -

- - -- - -

- - - - - -

Figure 20.6 Qualitative Coding “Matrix” From Faculty Interviews Related to Writing forPeople “subj1” to “subj4”

NOTE: Resulting matrix from turning the codes into 1s and 0s based on whether the subject spoke of, forinstance, clarity (yes = 1, no = 0).

From this initial coding, qualitativeresearchers would typically identify differ-ent concepts or categories that occur basedon how the initial codes logically grouptogether; and the logical groupings of theseitems is frequently referred to as thethemes. This is demonstrated in Figure20.6 where we have taken the text-codesfrom Figure 20.6 (A through D), which canbe thought of as items, and used them toconstruct the themes, which can be con-ceptually thought of as factors. Additionally,we can take these text-codes and convertthem into the 1–0 matrix similar to the oneshown on the right in Figure 20.6. This isvery much what a qualitative researchermight do during the coding process and iswhere they may end with the developmentof the emerging themes. This type of tablelooks similar to a truth table, which anelectrical engineer might use to addresscomputer logic or a sociologist might useto investigate complex human phenome-non. However it is more similar to Qualita -tive Comparative Analysis, known as QCA

(Berg-Schlosser, De Meur, Rihoux, & Ragin,2009; Ragin, 1987).

We are suggesting that researcher can goback to the 1–0 matrix shown at the rightside of Figure 20.6, and factor analyze it todetermine what factors emerge from thisnumerical coding, as shown in Figure 20.7.

...

100

010

010

001

... ... ...

Theme 3...- - -D

C

...Theme 2- - -

...Theme 2- - -

B

- - -- - -Theme 1A

Figure 20.7 The Theme AnalysisDetermined Three Themes Fromour Simulated Qualitative Data

NOTE: These themes can be represented as a matrix,with different items (A, B, C, etc.) forming Themes 1, 2,and 3.


The factor analysis can then be com-pared to the qualitative theme-analysis. Ifthe factor analysis results do not supportthese findings, it may suggest a need for theresearcher to look at the data in new waysor that the data should be investigated fur-ther. Thus, further analysis of qualitativedata using quantitative methods canenhance the qualitative research findings.

Finally, we can use similar techniques,once the theme or factors have emerged,to determine which types differentiatebetween the themes or factors. Forexample, a discriminate analysis could beperformed to better understand how thesethemes are related to the types found viaQ factor analysis. It is important, however,to note that some would take issue withour violating various assumptions by usingmultivariate statistics as a way to grouppeople and for data reduction. Therefore, italso is important to note that we are notmaking statistical inferences from samplesto populations. Instead, we are using thesetechniques to help us interpret our data byslicing it in different ways, which allows usto have different perspectives from whichthe data can be viewed. This may facilitatea better understanding and interpretationof the data.

�� Conclusions

In this chapter, we have discussed alterna-tive methods to effectively reduce the hugeamount of qualitative variables into group-ings to better investigate research ques-tions. The suggested techniques discussed,including Q methodology and Q factoranalysis, fit into the conception of mixedmethods. Specifically, we have focused onthe strengths of using multiple quantitativeand qualitative techniques to better under-stand data. We have demonstrated that thequalitative philosophy that different rela-tionships may exist for different situations,reflecting multiple realities, is not inconsis-tent with using multiple methods.

When these aspects of quantitative andqualitative procedures merge into a mixedmethods research, we can enhance our abilityto address a variety of new research purposesand questions. As researchers, whether weconsider ourselves qualitative, quantitative,or mixed, our ultimate goal is to effectivelyaddress our research purpose (Newmanet al., 2003) and answer our research ques-tions. The methods discussed here introduceways to do this that are not typically seen inthe research literature at this time.

Research Questions and Exercises

1. Discuss data reduction techniques; include some of the advantages and disadvantagesof using them.

2. Discuss univariate and multivariate analysis as data reduction techniques.

3. Describe how Q methodology and Q factor analysis are related and how they are different.

4. Discuss how Q factor analysis and Q methodology may be used to facilitate the inter-pretation of qualitative research.

�� Notes

1. A variety of publications provide moredetailed explanations of Q methodology (Brown,

1980, 2008; McKeown & Thomas, 1988; Ramlo,2008a; Stephenson, 1953) and how to performstudies using Q methodology (Brown, 1980, 1986;McKeown & Thomas, 1988; Ramlo, 2008a;Watts & Stenner, 2005).


2. See Brown (1980) for a greater explana-tion of different extraction methods. Suffice it tosay here that Centroid extraction is the preferredmethod for Q methodology based on philosoph-ical considerations related to the type of rotationpromoted by Stephenson (Brown, 1980;Stephenson, 1953).

3. Hand rotation, via a graphical interface,is preferred in cases where it is important toensure a specific participant is represented by afactor such as an instructor or leader within agroup (McKeown & Thomas, 1988). Brown(1986) explained that Centroid extraction of thefactors, followed by hand rotation, allows theinvestigator the opportunity to rotate based uponhunches and to examine the data from a theoret-ical standpoint. See Brown (1980) for a detailedexplanation of hand-rotation procedures.

4. This software is available for free down-load at http://www.lrz-muenchen.de/~schmolck/qmethod/downpqx.htm

5. This program was previously availableonly for mainframe computers. An alternativefor QUANAL is PQ Method, mentioned previ-ously within this chapter.

�� References

Adams, W. K., Perkins, K. K., Dubson, M.,Finkelstein, N. D., & Wieman, C. E. (2005).The design and validation of the Coloradolearning attitudes about science survey. AIPConference Proceedings, 790(1), 45–48.

Alston, R. J., & Mngadi, P. S. (1992). A study ofAPTICOM’s effectiveness in assessinglevel of physical functioning. Journal ofRehabilitation, 58(3), 35.

Ates, S., & Cataloglu, E. (2007). The effects ofstudents’ cognitive styles on conceptualunderstandings and problem-solving skills inintroductory mechanics. Research in Science& Technological Education, 25(2), 167–178.

Berg-Schlosser, D., De Meur, G., Rihoux, B., &Ragin, C. C. (2009). Qualitative compara-tive analysis (QCA) as an approach. InB. Rihoux, & C. C. Ragin (Eds.), Configu -rational comparative methods: Qualitativecomparative analysis (QCA) and related

techniques (pp. 1–18). Thousand Oaks,CA: Sage.

Block, J. (2008). The Q-sort in characterappraisal: Encoding subjective impressionsof persons quantitatively. Washington, DC:American Psychological Association.

Brown, S. R. (1980). Political subjectivity:Applications of Q methodology in politicalscience. New Haven, CT: Yale UniversityPress.

Brown, S. R. (1986). Q technique and method:Principles and procedures. In W. D. Berry& M. S. Lewis-Beck (Eds.), New tools forsocial scientist: Advances and applicationsin research methods (pp. 57–76). BeverlyHills, CA: Sage.

Brown, S. R. (1998). The history and principlesof Q methodology in psychology and thesocial sciences. Unpublished manuscript.Retrieved May 5, 2009, from http://facstaff.uww.edu/cottlec/QArchive/Bps.htm

Brown, S. R. (2008). Q methodology. InL. M. Given (Ed.), The Sage encyclopediaof qualitative research methods (pp. 700–704).Thousand Oaks, CA: Sage.

Burt, C. L. (1941). The factors of the mind: Anintroduction to factor-analysis in psychol-ogy. New York: Macmillan.

Cattell, R. B. (1978). The scientific use of factoranalysis in behavioral and life sciences.New York: Plenum.

Chan, K., & Elliott, R. G. (2004). Relationalanalysis of personal epistemology and con-ceptions about teaching and learning.Teaching and Teacher Education, 20(8),817–831.

Cohen, J. (1988). Statistical power analysis forthe behavioral sciences (2nd ed.). Hillsdale,NJ: Erlbaum.

Duell, O. K., & Schommer-Aikins, M. (2001).Measures of people’s beliefs about knowl-edge and learning. Educational PsychologyReview, 13(4), 419–449.

Dykstra, D. I., Boyle, C. F., & Monarch, I. A.(1992). Studying conceptual change inlearning physics. Science Education, 76(6),615–652.

Elby, A., & Hammer, D. (2001). On the sub-stance of a sophisticated epistemology.Science Education, 85(5), 554–567.


Gire, E., Price, E., & Jones, B. (2007). Charac -terizing the epistemologicai development ofphysics majors. AIP Conference Proceedings,883(1), 65–68.

Halloun, I., & Hestenes, D. (1998). InterpretingVASS dimensions and profiles for physicsstudents. Science and Education, 7(6),553–577.

Hammer, D., & Elby, A. (2003). Tapping epis-temological resources for learning physics.Journal of the Learning Sciences, 12(1),53–90.

Hofer, B. K., & Pintrich, P. R. (2002). Personalepistemology: The psychology of beliefsabout knowledge and knowing. Mahwah,NJ: Erlbaum.

Howard, D. A. (2005). Albert Einstein as aphilosopher of science. Physics Today,58(12), 34–40.

Lising, L. (2005) The impact of epistemology onlearning: A case study from introductoryphysics. American Journal of Physics,73(4), 372–382.

Lising, L., & Elby, A. (2005). The impact of epis-temology on learning: A case study fromintroductory physics. American Journal ofPhysics, 73(4), 372–382.

McKeown, B. (2001). Loss of meaning inLikert scaling: A note on the Q method-ological alternative. Operant Subjectivity,24, 201–206.

McKeown, B., & Thomas, D. (1988).Q method-ology. Newbury Park, CA: Sage.

McNeil, K. A., Newman, I., & Steinhauser, J.(2005).How to be involved in program eval-uation: What every administrator needs toknow. Lanham, MD: Scarecrow Education.

Newman, I. (1971). A multivariate approach tothe construction of an attitude battery.Unpublished doctoral dissertation, SouthernIllinois University at Carbondale.

Newman, I., & Benz, C. R. (1987). Multivariateevaluation design: A suggested technique formental health systems. Unpublished manu-script. Paper presented to the Ohio Academyof Science, Malone College, Canton, OH.

Newman, I., & Benz, C. R. (1998). Qualitative-quantitative research methodology: Explo -ring the interactive continuum. Carbondale:Southern Illinois University Press.

Newman, I., Dimitrov, D., & Waechter, D.(2000). Factor structure of perceived indi-vidualization of instruction: Argument formultiple perspective. Educational ResearchQuarterly, 24, 20–29.

Newman, I., Ridenour, C. S., Newman, C., &DeMarco, G. M. P. Jr. (2003). A typologyof research purposes and its relationshipto mixed methods. In A. Tashakkori &C. Teddlie (Eds.), Handbook of mixedmethods in social & behavioral research(pp. 167–188). Thousand Oaks, CA:SAGE.

Nunnally, J. C. (1978). Psychometric theory(2nd ed.). New York: McGraw-Hill.

Perkins, K. K., Adams, W. K., Pollock, S. J.,Finkelstein, N. D., & Wieman, C. E. (2005).Correlating student beliefs with studentlearning using the Colorado learning atti-tudes about science survey. AIP ConferenceProceedings, 790(1), 61–64.

Perkins, K. K., Gratny, M. M., Adams, W. K.,Finkelstein, N. D., & Wieman, C. E.(2006). Towards characterizing the rela-tionship between students’ interest in andtheir beliefs about physics. AIP ConferenceProceedings, 818(1), 137–140.

Ragin, C. C. (1987). The comparative method:Moving beyond qualitative and quantitativestrategies. Berkeley: University of CaliforniaPress.

Ramlo, S. (2003). A multivariate assessment ofthe effect of the laboratory homework com-ponent of a microcomputer-based labora-tory for a college freshman physics course.Unpublished doctoral dissertation, TheUniversity of Akron, Akron, OH.

Ramlo, S. (2005). An application of Q method-ology: Determining college faculty perspec-tives and consensus regarding the creationof a school of technology. Journal ofResearch in Education, 15(1), 52–69.

Ramlo, S. (2006). A physicist’s reflection on Q methodology, quantum mechanics &Stephenson. Operant Subjectivity, 29(2),81–86.

Ramlo, S. (2006/2007). Student views of learn-ing in an introductory college physicscourse: A study using Q methodology.Operant Subjectivity, 30(1 / 2), 52–63.


Ramlo, S. (2007a). Critical thinking and the learn-ing of force and motion concepts. Paper pre-sented at theAmerican Association of PhysicsTeachers, Summer Meeting, Greensboro, NC.

Ramlo, S. (2007b). Physics lab renovation 101.Physics Teacher, 45(4), 228–231.

Ramlo, S. (2008a). Determining the various per-spectives and consensus within a classroomusing Q methodology. Physics EducationResearch Conference Proceedings, 1064(1),179–182.

Ramlo, S. (2008b). Student perspectives onlearning physics and their relationship withlearning force and motion concepts: A studyusing Q methodology. Paper presented atthe Paper Presented at the InternationalSociety for the Scientific Study of Subjectivity/Q Methodology Conference, Hamilton,Ontario, Canada.

Ramlo, S. (2008c). Student perspectives onlearning physics and their relationship withlearning force and motion concepts: Astudy using Q methodology. HumanSubjectivity, 2(1), 73–90.

Ramlo, S. (2008d). Validity and reliability of theforce and motion conceptual evaluation.American Journal of Physics, 76(9), 882–886.

Ramlo, S., & McConnell, D. (2008). Perspectivesof university faculty regarding faculty read-ing circles: A study using Q methodology.Journal of Faculty Development, 22(1)

Ramlo, S., McConnell, D., Duan, Z., & Moore, F.(2008). Evaluating an inquiry-based bioin-formatics course using Q methodology.Journal of Science Education andTechnology, 17(3), 219–225.

Redish, E. F., Saul, J. M., & Steinberg, R. N.(2000). On the effectiveness of active-engage-ment microcomputer-based laboratories.American Journal of Physics, 65, 45–54.

Ridenour, C. S., & Newman, I. (2008). Mixedmethods research: Exploring the interactivecontinuum. Carbondale: Southern IllinoisUniversity Press.

Rowlands, S., Graham, T., & Berry, J. (1998).Identifying stumbling blocks in the devel-opment of student understanding ofmoments of forces. International Journalof Mathematical Education in Science &Technology, 29(4), 511.

Sauer, T. (2008). Einstein’s struggles withquantum theory. Physics Today, 61(5),56–57.

Schmolck, P. (2002). PQMethod manual mirror.Unpublished manuscript. Retrieved April 29,2004, from http://www.rz.unibw-muenchen.de/~p41bsmk/qmethod/

Schommer, M. (1990). Effects of beliefs aboutthe nature of knowledge on comprehen-sion. Journal of Educational Psychology,82, 498–504.

Schraw, G., Bendixen, L. D., & Dunkle, M. E.(2002). Development and validation of theepistemic belief inventory (EBI). In B. K. Hofer& P. R. Pintrich (Eds.), Personal epistemology:The psychology of beliefs about knowledgeand knowing (pp. 261–277). Mahwah, NJ:Erlbaum.

Siegesmund, R. (2008). Subjectivity. In L. M. Given(Ed.), The Sage encyclopedia of qualitativeresearch methods (pp. 843–845). ThousandOaks, CA: Sage.

Stainton-Rogers, R. (1995). Q methodology. InJ. A. Smith, R. Harré & L. van Langenhove(Eds.), Rethinking methods in psychology(pp. 178–192). London; Thousand Oaks,CA: Sage.

Stenner, P., & Stainton-Rogers, R. (2004). Q methodology and qualiquantology: Theexample of discriminating between emotions.In Z. Todd, B. Nerlich, S. McKeown, &D. D. Clarke (Eds.), Mixing methods in psy-chology (pp 101–120). Hove, UK; New York:Psychology Press.

Stephenson, W. (1935). Technique of factoranalysis. Nature, 136, 297.

Stephenson, W. (1953). The study of behavior:Q-technique and its methodology. Chicago:University of Chicago Press.

Stephenson, W. (1982a). Newton’s fifth ruleand Q methodology: Application to psy-choanalysis. Operant Subjectivity, 5(4),127–147.

Stephenson, W. (1982b). Q-methodology, inter-behavioral psychology, and quantum theory.Psychological Record, 32(2), 235–248.

Stephenson, W. (1987). Q methodology: Inter -behavioral and quantum theoretical con-nections in clinical psychology. New York;England: Greenwood.


Stephenson, W. (1988). Quantum theory ofsubjectivity. Integrative Psychiatry, 6(3),180–187.

Stephenson, W., Brown, S. R., & Brenner, D. J.(1972). Science, psychology, and com-munication: Essays honoring WilliamStephenson. New York: Teachers CollegePress.

Stephenson, W., & Burt, C. (1939). Alternativeviews on correlations between persons.Psychometrika, 4, 269–281.

Stevens, J. (2002). Applied multivariate statisticsfor the social sciences (4th ed.). Mahwah,NJ: Erlbaum.

Strauss, A. L., & Corbin, J. M. (1990). Basics ofqualitative research: Grounded theory pro-cedures and techniques. Newbury Park,CA: Sage.

Tashakkori, A., & Teddlie, C. (1998). Mixedmethodology: Combining qualitative andquantitative approaches. Thousand Oaks,CA: Sage.

Tashakkori, A., & Teddlie, C. (2009). Foundationsof mixed methods research: Integrating quan-titative and qualitative approaches in the

social and behavioral sciences. ThousandOaks, CA: Sage.

Thornton, R. K., & Sokoloff, D. R. (1998).Assessing student learning of Newton’slaws: The force and motion conceptual eval-uation and the evaluation of active learninglaboratory and lecture curricula. AmericanJournal of Physics, 66(4), 338–352.

Vantubergen, N. (1975). QUANAL user’s guide(computer program manual). Lexington:Department of Communication, Universityof Kentucky.

Waechter, D., Newman, I., & Nolte, D. (1998). Q-factor analysis: The first step in developingtypology classifications based upon theApticom. Vocational Evaluation and WorkAdjustment Journal (Fall and Winter), 61–66.

Ward, J. H. (1963). Hierarchical grouping to opti-mize an objective function. Journal of theAmerican Statistical Association, 58(301),236–244.

Watts, S., & Stenner, P. (2005). Doing Q method-ology: Theory, method and interpretation.Qualitative Research in Psychology, 2(1),67–91.

Documents

Chapter 20-Newman and Ramlo_from Tashakkori I Proof [Low Res]