Sampling Methods, Sample Size, andSampling Methods, Sample Size, andStudy PowerStudy Power

ObjectivesObjectives

• Describe common sampling methodsDescribe common sampling methods

• Review basic types of samplesReview basic types of samples

• Discuss the basic information required to Discuss the basic information required to calculate a sample sizecalculate a sample size

• Review the concept of study powerReview the concept of study power

Definition of TermsDefinition of Terms

• Population =Population = a collection of itemsa collection of items

• CensusCensus = = a complete canvass of a a complete canvass of a populationpopulation

• Sample = Sample = units selected as representative units selected as representative of a population and from of a population and from

which which inferences about the inferences about the population population will be madewill be made

• UnitUnit = = an individual item in the an individual item in the populationpopulation

Relationship Between Samples and ValidityRelationship Between Samples and Validity

• Sampling has the largest effect on which Sampling has the largest effect on which of the following?of the following?* a. random assignmenta. random assignment

* b. internal validityb. internal validity

* c. construct validityc. construct validity

* d. external validityd. external validity

* e. both a and de. both a and d

Basic Questions of SamplingBasic Questions of Sampling

• Sample or Census?Sample or Census?

• If sample, what procedure?If sample, what procedure?

• If sample, what size?If sample, what size?

Census or Sample?Census or Sample?Depends on....Depends on....

• Size of populationSize of population

• CostCost

• Need for precisionNeed for precision

• Destruction of sampling unitsDestruction of sampling units

• Need for in depth studyNeed for in depth study

• Constancy of characteristics being Constancy of characteristics being studiedstudied

Sample if the ...Sample if the ...

• Population is largePopulation is large

• Cost of census is largeCost of census is large

• Need for absolute accuracy is Need for absolute accuracy is not not criticalcritical

• There is a need for in-depth studyThere is a need for in-depth study

• Observed units will be destroyedObserved units will be destroyed

• Characteristic being studied is relatively Characteristic being studied is relatively constantconstant

Sampling ProceduresSampling Procedures• Non-probability samplesNon-probability samples

* Unit selection is influenced by personal Unit selection is influenced by personal judgementjudgement

* Objectivity and randomness is absentObjectivity and randomness is absent

• ProbabilityProbability* Objective rules for selectionObjective rules for selection* Randomness allows for inferences about a Randomness allows for inferences about a

population based on measurement of sampling population based on measurement of sampling errorerror

Non-probability SamplesNon-probability Samples

• Convenience sampleConvenience sample* Select units on the basis of convenienceSelect units on the basis of convenience

• e.g., a mall intercept e.g., a mall intercept

• Judgement sampleJudgement sample* Units selected based on the assumption that the Units selected based on the assumption that the

researcher can identify subjects that serve the researcher can identify subjects that serve the research purposeresearch purpose• e.g., an expert panele.g., an expert panel

Non-probability SamplesNon-probability Samples

• Quota sampleQuota sample* Select units to make the sample contain the same Select units to make the sample contain the same

proportion of an important characteristic as found in the proportion of an important characteristic as found in the populationpopulation• e.g., public and private hospitals, bed sizee.g., public and private hospitals, bed size

• Snowball sampleSnowball sample* Obtain a few units and use those units to obtain referralsObtain a few units and use those units to obtain referrals

* Used when the study topic is a rare event, accidental, Used when the study topic is a rare event, accidental, controversial, or confidentialcontroversial, or confidential

Probability SamplingProbability Sampling

Based on two principlesBased on two principles

• Unbiased selection procedureUnbiased selection procedure

• All units in a population have a All units in a population have a known and non-zero probability of known and non-zero probability of selection selection

The value of probability samplingThe value of probability sampling

• Standard error = Standard error = / / nn* A measure of precisionA measure of precision

• For any given standard deviation of a For any given standard deviation of a mean a sample of 100 is as precise for a mean a sample of 100 is as precise for a population of 200,000 as it is for 20,000population of 200,000 as it is for 20,000

Probability SamplesProbability Samples• Simple random sampleSimple random sample

* Each population unit has a known and equal Each population unit has a known and equal probability of selectionprobability of selection

* Each possible sample of n units from a Each possible sample of n units from a population of N units has an equal probability of population of N units has an equal probability of selectionselection

* Assumes replacement of units to populationAssumes replacement of units to population* Sampling from finite populations requires an Sampling from finite populations requires an

adjustmentadjustment* Use of a sampling frame limits generalizations to Use of a sampling frame limits generalizations to

the sampling framethe sampling frame

Probability SamplingProbability Sampling

• Stratified SamplingStratified Sampling* Divide the population into mutually exclusive and Divide the population into mutually exclusive and

exhaustive categoriesexhaustive categories

* Take a simple random sample from each strataTake a simple random sample from each strata

Take a simple random Take a simple random sample from each stratasample from each strata

Stratified SampleStratified Sample• Use when simple random sample will not Use when simple random sample will not

capture enough units of a stratacapture enough units of a strata• Units may be taken proportionate to the size of Units may be taken proportionate to the size of

the strata or to the variancethe strata or to the variance* Size procedure is more commonSize procedure is more common

• Want units to be Want units to be similar within stratasimilar within strata and and different between stratadifferent between strata (e.g., maximize intra- (e.g., maximize intra-strata homogeneity and inter-strata strata homogeneity and inter-strata heterogeneityheterogeneity

Probability SamplingProbability Sampling

• Cluster samplingCluster sampling* Divide the population into mutually exclusive and Divide the population into mutually exclusive and

exhaustive categoriesexhaustive categories

* Take a simple random sample of clustersTake a simple random sample of clusters

Take a simple random Take a simple random sample of clusterssample of clusters

Cluster SampleCluster Sample• Can do multi-stage cluster sampleCan do multi-stage cluster sample

• Main advantage is that you do not need a Main advantage is that you do not need a sampling framesampling frame* Census map exampleCensus map example

• Two common approaches within clustersTwo common approaches within clusters* Area samplingArea sampling

* Systematic samplingSystematic sampling

Steps for Drawing a SampleSteps for Drawing a Sample

• Define the populationDefine the population

• Identify the sampling frameIdentify the sampling frame

• Select a sampling procedureSelect a sampling procedure

• Determine the sample sizeDetermine the sample size

• Select the sample elementsSelect the sample elements

Importance of Sample SizeImportance of Sample Size

• n must be large enough to detect n must be large enough to detect differences worth detectingdifferences worth detecting

• If n is too large, it is wasteful, potentially If n is too large, it is wasteful, potentially costly, and risky for sampling unitscostly, and risky for sampling units

• If n is too small, effects will be missedIf n is too small, effects will be missed

Basic Information Needed Basic Information Needed to Calculate Sample Sizeto Calculate Sample Size

• Purpose Purpose

* Point estimate (e.g., population mean)Point estimate (e.g., population mean)

* Hypothesis testingHypothesis testing

• Precision desired (acceptable error)Precision desired (acceptable error)

• Variability (dispersion in the population)Variability (dispersion in the population)

• ConfidenceConfidence

Sample size for a point estimateSample size for a point estimate

• Set level of acceptable error (e)Set level of acceptable error (e)

• Set the confidence level (Set the confidence level ( expressed as Z) expressed as Z)

• Calculate (estimate) the standard deviationCalculate (estimate) the standard deviation

• Calculate sample size: n = [Z * SD / e]Calculate sample size: n = [Z * SD / e]22

Sample size for a point estimateSample size for a point estimate

Example: ER visits/year for asthma ptsExample: ER visits/year for asthma pts

n = [Z * SD / e]n = [Z * SD / e]22

ZZ = 1.96 (alpha = 0.05)= 1.96 (alpha = 0.05)

SDSD = 100= 100

ee = +/- 4= +/- 4

n = [1.96 * 100 / 4]n = [1.96 * 100 / 4]22 == 2,401 2,401

• How can we reduce sample size required?How can we reduce sample size required?

Sample size for a hypothesis testSample size for a hypothesis test

• Set values for Type I (Set values for Type I () and Type II () and Type II () errors ) errors and express them as Z valuesand express them as Z values

• Calculate (estimate) the standard deviationCalculate (estimate) the standard deviation

• Calculate sample size: n = Calculate sample size: n = ((ZZ + Z+ Z) SD ) SD 22

mm11 – m – m2 2

• Where n = number of subject per groupWhere n = number of subject per group

Sample Size Example for Paired t-testSample Size Example for Paired t-test

• Quality adjusted life yearsQuality adjusted life years* Drug A - 9.8 QALYsDrug A - 9.8 QALYs

* Drug B - 11.2 QALYsDrug B - 11.2 QALYs

• Standard deviationStandard deviation* Range 1.3 to 18.5Range 1.3 to 18.5

= 0.05 (z=1.64, one tailed)= 0.05 (z=1.64, one tailed) = 0.20 (z=0.84); power = 1 - = 0.20 (z=0.84); power = 1 - BetaBeta

Sample size for a paired t-testSample size for a paired t-test

• n = n = ((ZZ + Z+ Z) SD ) SD 22

mm11 – m – m2 2

• n = n = (1.64(1.64 + 0.84) 4.3 + 0.84) 4.3 22

11.2 – 9.811.2 – 9.8

• n = ??? (per group)n = ??? (per group)

Independent Sample t-testIndependent Sample t-test

• Multiply the results of the paired test Multiply the results of the paired test formula by 2formula by 2

• This is the number of subjects per groupThis is the number of subjects per group

• Multiply by 2 again for the total number of Multiply by 2 again for the total number of subjectssubjects

Sample Size CaveatsSample Size Caveats

• Response ratesResponse rates

• Refusal to participateRefusal to participate

• AttritionAttrition

• Quality of sample is as important as Quality of sample is as important as sizesize

• Sampling error is controlled by Sampling error is controlled by increasing sample sizeincreasing sample size

Power of a statistical testPower of a statistical test

• Power = 1 - Power = 1 - • Power = probability of rejecting the Power = probability of rejecting the

null when it is false null when it is false• Power is a function of:Power is a function of:

* Sample sizeSample size* Effect size Effect size * Alpha level (e.g., odds the result is due to Alpha level (e.g., odds the result is due to

chance)chance)

Power of a statistical testPower of a statistical test

• Usually try to estimate sample size needed Usually try to estimate sample size needed to achieve a desired power of statistical testto achieve a desired power of statistical test* A prioriA priori power calculation power calculation

• Cannot know power of the test until the Cannot know power of the test until the study is complete because effect size is an study is complete because effect size is an estimateestimate* Post hocPost hoc power calculation power calculation

Power of a statistical testPower of a statistical test

• Quick calculation for:Quick calculation for:* alphaalpha = 0.05= 0.05* betabeta = 0.20= 0.20* power power = 0.80= 0.80

• n >= 15.68 (C/R)n >= 15.68 (C/R)22 wherewhere* C = coefficient of variation (std dev as a % of mean)C = coefficient of variation (std dev as a % of mean)* R = relative precision level (as a % of the mean)R = relative precision level (as a % of the mean)

• Sometimes rounded to n >= 20 (C/R)Sometimes rounded to n >= 20 (C/R)22

SummarySummary

• We always prefer probability to non-prob. We always prefer probability to non-prob. samplessamples

• Studies with low sample size often have low Studies with low sample size often have low power and therefore have difficulty power and therefore have difficulty detecting effect sizesdetecting effect sizes

• Sampling is efficientSampling is efficient