Measuring Variables2

8/10/2019 Measuring Variables2

1/24

Defining and Measuring

Variables


2/24

1. An overview of measurement

two aspects of measurement are particularlyimportant in planning a research study orreading a research report:

often there is not a one-to-one relationship betweenthe variable measured and the measurementobtained (knowledge, performance and exam grade)

there are usually several different options formeasuring any particular variable (types of exams

and questions on exams) Direct measurement (height, weight) vs indirectmeasurement (motivation, knowledge, memory,marital satisfaction)


3/24

2. Constructs and operationaldefinitions

Theories summarize our observations, explainmechanisms underlying a particular behavior andmake predictions about the behavior.

many research variables, particularly variables ofinterest to behavioral scientists, are hypotheticalattributes or mechanisms explaining and predictingsome behavior in a theory are called constructs

externalstimulus construct behaviorfactor

reward motivation performance


4/24

constructs can not be directly observed or measured however, researchers can measure external, observable

events as an indirect method of measuring the constructitself operational definition

is a procedure for measuring and defining a construct, indirectmethod of measuring something that can not be measured directly

an operational definition specifies a measurement procedure formeasuring an external, observable behavior and uses the resultingmeasurements as a definition and a measurement of thehypothetical construct

e.g. IQ test is an operational definition for the construct intelligence

- provide and example of a theoretical construct and itsoperational definition

You dont always have to come up with your ownoperational definition of the construct, you can use someconventional measurement procedure from previousstudies


5/24


6/24

Validity of measurement

Validity of measurement concerns the truth of the measurement

it is the degree to which the measurementprocess measures the variable it claims tomeasure

Is the IQ score truly measuring intelligence?What about size of the brain and bumps onthe scull?


7/24

Different kinds of validity

face validity

the simplest and least scientific definition of validity it is demonstrated when a measure superficially appears to

measure what it claims to measure

Based on subjective judgment and difficult to quantify e.g. intelligence and reasoning questions on the IQ test Problem - participants can use the face validity to change

their answers

concurrent validity (criterion validity)

is demonstrated when scores obtained from a new measureare directly related to scores obtained from a moreestablished measure of the same variable

e.g. new IQ test correlates with an older IQ test


8/24

Different kinds of validity (cont.)

Different kinds of validity predictive validity

when scores obtained from a measure accurately predict

behavior according to a theory e.g. high scores on need for achievement test predict

competitive behavior in children (ring toss game)

construct validity is demonstrated when scores obtained from a measure are

directly related to the variable itself Reflects how close the measure relates to the construct

(height and weight example)

in one sense, construct validity is achieved by repeatedlydemonstrating every other type of validity


9/24

Different kinds of validity (cont.)

Different kinds of validity convergent validity

is demonstrated by a strong relationship between the scoresobtained from two different methods of measuring the same

construct e.g. an experimenter observing aggressive behavior in childrencorrelated with teachers ratings of their behavior

divergent validity is demonstrated by using two different methods to measure

two different constructs convergent validity must be shown for each of the twoconstructs and little or no relationship exists between thescores obtained from the two different constructs when theyare measured by the same method

e.g. aggressive behavior and general activity level in children


10/24


11/24

Reliability of measurement

Reliability of measurement a measurement procedure is said to be reliable if

repeated measurements of the same individual underthe same conditions produce identical (or nearly

identical) values reliability is the stability or the consistency of

measurement

measured score = true score + error

IQ score = true IQ score + mood, fatigue etc.


12/24

Reliability and error ofmeasurement

Inconsistency (lack of reliability) of measurement comesfrom error

The higher the error the more unreliable themeasurement

Sources of error observer error

the individual who makes the measurements can introduce simplehuman error into the measurement process

environmental changes small changes in the environment from one measurement to another

(e.g. time of the day, distraction in the room, lighting) participant changes

participants change between measurements (mood, hunger,motivation)


13/24

Types and measures of reliability

successive measurements Obtaining scores from two successive measurements and calculating

a correlation between them the same group, the same measurement at two different times test-retest reliability

simultaneous measurements obtained by direct observation of behaviors (two or more separateobservers at the same time), consistency across raters

inter-rater reliability

internal consistency degree of consistency of scores from separate items on a test or

questionnaire consisting of multiple items you want all the items or groups of items tapping the sameprocesses

researchers commonly split the set of items in half, compute aseparate score of each half, and then evaluate the degree ofagreement between the two scores

split-half reliability


14/24

The relationship between reliability

and validity

they are partially related and partiallyindependent

reliability is a prerequisite for validity

(measurement procedure can not be validunless it is reliablee.g. IQ, huge variance ofrepeated measurements is impossible if weare truly measuring intelligence)

it is not necessary for a measurement to bevalid for it to be reliable (e.g. height as ameasure of intelligence)


15/24

4. Scales of measurement

Scales define the type categories we use in measurementand the selection of a scale has direct impact on ourability to describe relationships between variables

the nominal scale

simply represents qualitative difference in the variable measured can only tell us that a difference exists without the possibility

telling the direction or magnitude of the difference

e.g. majors in college, race, gender, occupation

the ordinal scale the categories that make up an ordinal scale form an ordered

sequence

can tell us the direction of the difference but not the magnitude

e.g. coffee cup sizes, socioeconomic class, T-shirt sizes, foodpreferences


16/24

Scales of measurement (cont.)

the interval scale categories on an interval scale are organizedsequentially, and all categories are the same size

we can determine the direction and the magnitude ofa difference

May have an arbitrary zero (convenient point ofreference)

e.g. temperature in Farenheit, time in seconds

the ratio scale

consists of equal, ordered categories anchored by azero point that is not arbitrary but meaningful(representing absence of a variable

allows us to determine the direction, the magnitude,and the ratio of the difference

e.g. reaction time, number of errors on a test


17/24

5. Modalities of measurement

One can measure a construct by selectinga measure from three main categories

There are three basic modalities ofmeasurement: self-report

physiological measurement

behavioral measurement

behavioral observation content analysis and archival research


18/24

Self-report measures

you ask a participant to describe his behavior,to express his opinion or characterize hisexperience in an interview or by using aquestionnaire with ratings

Positive aspects Only the individual has direct access to informationabout his state of mind

More direct measure

Negative aspects Participants may distort the responses to create abetter self-image or to please the experimenter

The response can also be influenced by wording ofthe questions and other aspects of the situation


19/24

Physiological measures

Physiological manifestations of the underlyingconstruct

e.g. EEG, EKG, galvanic skin response,perspiration, PET, fMRI

advantages provides accurate, reliable, and well-defined

measurements that are not dependent on subjectiveinterpretation

disadvantages equipment is usually expensive or unavailable Presence of monitoring devices may create unnatural

situation

question: Are these procedures a valid measure of theconstruct (e.g. increase in heart rate to fear, arousal)


20/24

Behavioral measures

behaviors that can be observed and measured (e.g.reaction time, reading speed, focus of attention,disruptive behavior, number of words recalled on a

memory test) How to select the right behavioral measure? Depends on the purpose of the study

In clinical setting the same disorder can reveal itself throughdifferent symptoms

In studying memory we want to have the same measure for allsubjects to be able to compare them

Beware of situational changes in behavior (e.g.disruptive behavior in school vs when observed) anddifferent behavioral indicators of a construct


21/24

6. Other aspects of measurement

multiple measures sometimes you can use two (or more) different

procedures to measure the same variable (e.g. heartrate and questionnaire as a measure of fear)

problems (the two variables may not behave in thesame way)

e.g. a specific therapy for treating fear may have large effecton behavior but no effect on heart rate

the lack of agreement between two measures is calleddesynchrony

One measure can be more sensitive than other Different measures may indicate different dimensions of the

variable and change at different times during the treatment


22/24

Sensitivity and range effects

are the measures sensitive enough to respond to thetype and magnitude of the changes that areexpected? (e.g. seconds vs. milliseconds, difficult oreasy exams)

range effects a ceiling effect (the clustering of scores at the high end of a

measurement scale, allowing little or no possibility ofincreases in value, e.g. test that is too easy)

a floor effect (the clustering of scores at the low end of a

measurement scale, allowing little or no possibility ofdecreases in value, e.g. test that is too difficult)

Range effects are usually a consequence of using a measurethat is inappropriate for a particular group (e.g. 4-grade testfor college students)


23/24

Participant reactivity andexperimenter bias

participant reactivity is the way how participant reacts tothe experimental situation (e.g. overly cooperative, overlydefensive, or hostile) To avoid these problems one can try to disguise the true purpose of

the experiment or observe individuals without their awareness

(beware ethical issues) experimenter bias is the way experimenter influences

results (e.g. by being warm and friendly with one group ofparticipants vs. cold and stern with other group)

to avoid participant reactivity and experimenter bias we

use: standardized procedures (e.g. instructions recorded on a tape) a research study is single blind if the researcher does not know the

predicted outcome a research study is double blind if both the researcher and the

participants are unaware of the predicted outcome


24/24

Participant reactivity andexperimenter bias

to avoid participant reactivity andexperimenter bias we use blind experiments

a research study is single blind if the researcherdoes not know the predicted outcome

a research study is double blind if both theresearcher and the participants are unaware of thepredicted outcome

Documents

Measuring Variables2