Introduction to Research Methods

Sampling & External Validity1KNR 497Research MethodsSamplingSlide 1Chapter 2 part 2

21KNR 497Research Methods:SamplingSlide 2The 65, 95, 99 Percent Rule

KNR 497Research Methods:SamplingSlide 3Estimating the Population Using a Sampling Distribution21

1The rest of the slidesTypes of samplingProbability based Non-probability based

KNR 497Research Methods:SamplingSlide 4

2

31KNR 497Research Methods:SamplingSlide 5Probability SamplingUtilizes some form of random selectionAll units in the population have equal probability of being chosenNomenclature:N = number of cases in the sampling framen = number of cases in the sampleNCn = number of combinations of n from Nf = n/N and is the sampling fraction

2

4

KNR 497Research Methods:SamplingSlide 6Probability SamplingSimple random samplingStratified random samplingSystematic random samplingCluster (area) random samplingMultistage sampling1

KNR 497Research Methods:SamplingSlide 7Probability Sampling:Simple Random SamplingObjective: To select n units out of N such that each NCn has an equal chance of being selectedProcedure: Use a table of random numbers or computer random-number generatorExample: N = 1000n (desired) = 100f = n/N = 100/1000 = .10 or 10%Randomly select 100 units (10%)Generalizable; may not be representative of subgroups312

KNR 497Research Methods:SamplingSlide 8Probability Sampling:Stratified Random SamplingObjective: To select n units out of N such that key subgroups of n are representative of subgroups of NProcedure: Divide the population into nonoverlapping groups (strata) N1, N2, N3Ni, such that N1 + N2 + N3 + Ni = N. Then do simple random sample of f = n/N in each strataDisproportionate stratified random sampling can be used to oversample small groups.3124

KNR 497Research Methods:SamplingSlide 9Probability Sampling:Systematic Random SamplingObjective: To systematically select n units out of N such that n is a random sample of NProcedure: Number units in the population from 1 to N (NOTE: Units must be randomly ordered)Decide on the n that you needCalculate k = N/n = the interval sizeRandomly select an integer between 1 and kTake every kth unit(diagram on next slide illustrates this)12

KNR 497Research Methods:SamplingSlide 10Probability Sampling:Systematic Random Sampling1

KNR 497Research Methods:SamplingSlide 11Probability Sampling:Cluster (Area) Random SamplingObjective: To obtain a representative sample from N when N is spread out over a large geographic areaProcedure: Divide the population into clustersRandomly sample clustersMeasure all units within sampled clusters

Clusters are usually divided along geographical boundaries.312

KNR 497Research Methods:SamplingSlide 12Probability Sampling:Multistage SamplingObjective: To obtain a representative sample from N by combining several sampling techniques to create a more efficient or effective sample than the use of any one sampling type can achieve on its ownExample: 1. National sample of school districts stratified by economics2. Simple random selection of schools within districts3. Simple random selection of classes within schools312

KNR 497Research Methods:SamplingSlide 13Nonprobability SamplingDoes not involve random selectionMay be representative but cannot depend upon the rationale of probability theoryUsed when it is not feasible, practical, or theoretically sensible to use random samplingAccidental versus purposive3124

Nonprobability Sampling:Accidental, Haphazard, or Convenience SamplingOne of the most common methods of samplingMan on the streetVolunteers or subjects who are conveniently availableNo evidence that sample is representativeKNR 497Research Methods:SamplingSlide 1412

KNR 497Research Methods:SamplingSlide 15Nonprobability Sampling:Purposive SamplingSampling with a purpose in mindUseful in reaching a targeted sample quicklyTarget population is reached but with over-representation of subgroups that are more readily accessibleTypes:Modal instanceExpertQuotaHeterogeneitySnowball312

KNR 497Research Methods:SamplingSlide 16Nonprobability Sampling:Purposive SamplingModal InstanceSampling the most frequent case or typical caseDifficult to define what a typical case isProbably only useful for informal sampling contexts (or perhaps even more dangerous for those)12

3

KNR 497Research Methods:SamplingSlide 17Nonprobability Sampling:Purposive SamplingExpert SamplingAssembling of a sample of persons with known or demonstrable expertise in some areaPanel of expertsMay be useful for providing evidence as to the validity of another sampling approach you have chosen12

KNR 497Research Methods:SamplingSlide 18Nonprobability Sampling:Purposive SamplingQuota SamplingSample selected nonrandomly according to some fixed quotaProportional quota sampling used to represent the major characteristics of the population of interest by sampling a proportional amount of eachNonproportional quota sampling used to supply a minimum number of units in each category but not concerned with proportions12

KNR 497Research Methods:SamplingSlide 19Nonprobability Sampling:Purposive SamplingHeterogeneity SamplingUsed to provide a sample that will include all the view or opinions without regard to proportional representationSampling for diversityCan be thought of as the opposite of modal instance sampling12

KNR 497Research Methods:SamplingSlide 20Nonprobability Sampling:Purposive SamplingSnowball SamplingPeople meeting the criteria for inclusion in the sample are identified and then they recommend others they know who meet the criteriaUseful when trying to reach inaccessible or hard to find populationsExamples may include the homeless, drug users, etc.12

Threats to External ValidityInteraction of selection and treatmentInteraction of setting and treatmentInteraction of history and treatmentMaybe it is just these people.Maybe it is just these places.Maybe it is just these times.Guiding Questions for Critiquing the External Validity of ResearchWhat are the main results of the study (e.g., positive or negative relationship, group differences, effectiveness of the intervention or treatment)? Do the researchers explicitly state or imply that similar results would hold for other: (a) people, (b) places or situations, and/or (c) times? If so, what is the population/place/time they are attempting to generalize to?If the researchers are generalizing their results, how reasonable are these conclusions given the sample, sampling procedures, and settings used? [This is the key External Validity question]What specifically might lead you to question these conclusions? In other words, if they did suggest the results were generalizable, why might you think otherwise? [The more convincing of a rationale you can generate, the more you should question the external validity] Use the guiding questions to evaluate the external validity of the following studyPrior research has found that (a) intercollegiate athletes are especially at-risk for excessive alcohol consumption (e.g., Nelson & Wechsler, 2001), and (b) sport-type differences exist among college athletes in terms of yearly drinking prevalence rates (National Collegiate Athletic Association, 2001). No studies, however, have examined sport-type differences on more specific measures of alcohol consumption (i.e., drinks per week). In the present study, data were analyzed on 298 intercollegiate athletes from two different NCAA Division III universities. Results indicated significant sport type differences on alcohol consumption variables, with athletes from the sports of swimming and diving and wrestling reporting the highest levels of alcohol consumption (M = 5.20, SD = 4.00) and soccer and football reporting the lowest (M = 4.02, SD = 3.25). Results suggest college athletes participating in individual sports are at-risk for future alcohol abuse.