COM 633: Content Analysis Measurement; Validity Kimberly A. Neuendorf, Ph.D. Cleveland State...

COM 633: Content AnalysisMeasurement; Validity

Kimberly A. Neuendorf, Ph.D.

Cleveland State UniversityFall 2010

Measurement “Assignment of numerals to objects or events according to rules” (Stevens, 1951)

Operationalization (Measurement)

Categories that are exhaustive and mutually exclusiveHigh and appropriate level of measurement

RatioIntervalOrdinalNominal

Evaluating Measures:Comparing Reliability,

Accuracy (lack of Bias), and Precision

Targets…see p. 114 of book.See also:

Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Newbury Park, CA: Sage Publications.

ValidityInternal

Conceptual definition =matches= OperationalizationThe standard of having a “good” measure—”Are we hitting the target we want?”

ExternalGeneralizability

SampleEcological validityReplicability

Validity“Face”Criterion

Relate to established standard or behaviorConcurrentPredictivee.g., Oliver & Kalyanaraman (2002) study of sexual and violent content in trailers and films. . . Criterion is the MPAA rating

ValidityContent

Construct

ValidityContent

ValidityContent

Construct = ???

ValidityContent

High reliability. . .

ValidityConstruct

Related to other measures (constructs) in ways consistent with theory

Convergent—relationships as expectedDiscriminant—lack of relationships, as expected

Construct ValiditySmith volume (1992) on “thematic content analysis”—psychographic applications—has examples

Trust-mistrust coding system for TAT stories construct validated against:

Other thematic coding systems (e.g., Affiliation; Intimacy)Demographics (e.g., Mothers’ feeding habits during infancy (more or less principled), moderated by gender & age)Self-reported behaviors (e.g., Reported positive relationships with others; Divorce, family fights in last year; Drug use)Personality measures (e.g., Impulsivity; Locus of Control)

Validity Assessment & CATA

Short et al. (2010) provides a model for validity and CATA, using tests of a new dictionary for “entrepreneurial orientation”

Content ValidityDeductiveInductive

External ValidityReliabilityDimensionality (a type of Construct Validity--Discriminant)Predictive Validity (seems to really be Construct Validity—Convergent)