9
This article was downloaded by: [University of Nebraska, Lincoln] On: 24 October 2014, At: 18:50 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK American Journal of Health Education Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ujhe20 Power Analysis in Survey Research: Importance and Use for Health Educators James H. Price PhD, MPH a , Joseph A. Daek PhD, MPH b , Judy Murnan MPH a , Jaime Dimmig MPH a & Sutoidem Akpanudo MBBS, MPH a a Department of Public Health , University of Toledo , Toledo , OH , 43606 , USA b Division of Health , Wayne State University , Detroit , MI , 48202 , USA Published online: 22 Feb 2013. To cite this article: James H. Price PhD, MPH , Joseph A. Daek PhD, MPH , Judy Murnan MPH , Jaime Dimmig MPH & Sutoidem Akpanudo MBBS, MPH (2005) Power Analysis in Survey Research: Importance and Use for Health Educators, American Journal of Health Education, 36:4, 202-209, DOI: 10.1080/19325037.2005.10608185 To link to this article: http://dx.doi.org/10.1080/19325037.2005.10608185 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Power Analysis in Survey Research: Importance and Use for Health Educators

Embed Size (px)

Citation preview

Page 1: Power Analysis in Survey Research: Importance and Use for Health Educators

This article was downloaded by: [University of Nebraska, Lincoln]On: 24 October 2014, At: 18:50Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

American Journal of Health EducationPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/ujhe20

Power Analysis in Survey Research: Importance and Usefor Health EducatorsJames H. Price PhD, MPH a , Joseph A. Daek PhD, MPH b , Judy Murnan MPH a , Jaime DimmigMPH a & Sutoidem Akpanudo MBBS, MPH aa Department of Public Health , University of Toledo , Toledo , OH , 43606 , USAb Division of Health , Wayne State University , Detroit , MI , 48202 , USAPublished online: 22 Feb 2013.

To cite this article: James H. Price PhD, MPH , Joseph A. Daek PhD, MPH , Judy Murnan MPH , Jaime Dimmig MPH & SutoidemAkpanudo MBBS, MPH (2005) Power Analysis in Survey Research: Importance and Use for Health Educators, American Journalof Health Education, 36:4, 202-209, DOI: 10.1080/19325037.2005.10608185

To link to this article: http://dx.doi.org/10.1080/19325037.2005.10608185

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Page 2: Power Analysis in Survey Research: Importance and Use for Health Educators

202 American Journal of Health Education — July/August 2005, Volume 36, No. 4

INTRODUCTIONThere are several purposes to this article,

the first of which is an overview of poweranalysis: what it is, why it is important, andhow to calculate it. The second purpose isthe relative importance of power analysisto adequate survey return rates. While thesetwo issues could be learned elsewhere (e.g.,various research methods texts and journalarticles), this article provides those readerswho are less familiar with power analysis asummary of the key points as they relate tohealth education survey research. The thirdpurpose of this article is to assess the use ofpower analysis in seven leading health edu-cation journals. This article is directed atreaders unfamiliar with power analysis, aswell as those who are better versed in its use,with the intent being to increase the appro-priate use of power analysis in health edu-cation survey research.

Theory of Power AnalysisAnytime a researcher conducts a quan-

titative study, it is essential that the re-searcher calculate the statistical power of astudy before any data are collected, with thepossible exception of pilot studies. In fact,grant proposals to some federal agenciesrequire that a power analysis be conductedbefore the proposal is submitted. A statisti-cal power assessment tells us how likely it isthat a statistical significance test (e.g., t-test,ANOVA, chi-square) will detect a signifi-cant difference between two or moregroups, given that a difference actually ex-ists. In other words, statistical tests attemptto disprove the null hypothesis that there isno difference or no association between oramong various samples.1 Rejection of a nullhypothesis means that a difference or anassociation may be inferred from the studysample to the population.

Using statistical significance tests to as-sess data from a study can result in severaldifferent outcomes (Figure 1). In the firstcell (A), we see that the null hypothesis is

Power Analysis in Survey Research:Importance and Use for Health Educators

James H. Price, Joseph A. Dake, Judy Murnan, Jaime Dimmig, and Sutoidem Akpanudo

James H. Price, PhD, MPH, is professor ofhealth education, Department of Public Health,University of Toledo, Toledo, OH 43606; E-mail: [email protected]. Joseph A.Dake, PhD, MPH, is assistant professor ofhealth education, Division of Health, WayneState University, Detroit, MI 48202. JudyMurnan, MPH, is a doctoral student at the De-partment of Public Health, University of To-ledo, Toledo, OH 43606. Jaime Dimmig, MPH,is a doctoral student at the Department of Pub-lic Health, University of Toledo, Toledo, OH43606. Sutoidem Akpanudo, MBBS, MPH, isa doctoral student at the Department of PublicHealth, University of Toledo, Toledo, OH 43606.

ABSTRACT

This article has three purposes: to explain the two different uses of power analysis that can be used in health educa-

tion research; to examine the extent to which power analysis is being used in published health education research;

and to explain the implications of not using power analysis in research studies. Articles in seven leading healtheducation journals (American Journal of Health Behavior, American Journal of Health Education, American

Journal of Health Promotion, Health Education &Behavior, Health Education Research, Journal of American

College Health, and Journal of School Health) were analyzed for the years 2000–2003. For four of the sevenjournals, less than 5% of their research articles reported a power analysis. Only two journals (American Journal of

Health Behavior and Health Education Research) had a modest number of research articles (14–35%) that re-

ported power analysis. This is the first reported examination of power analysis in health education journals. Thefindings indicate a potential problem with the quality of health education research being reported.

Continuing Education

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014

Page 3: Power Analysis in Survey Research: Importance and Use for Health Educators

James H. Price, Joseph A. Dake, Judy Murnan, Jaime Dimmig, and Sutoidem Akpanudo

American Journal of Health Education — July/August 2005, Volume 36, No. 4 203

false in the population and if our study re-sults find the null hypothesis to be false, weobtained a correct outcome. In this case, wefind support for a hypothesis that says thereis/are difference(s) between/among groupsor an association between the variable(s)under study.1

The second cell (B) indicates the nullhypothesis in the population is true but ourstudy findings reject the null hypothesis,identifying the hypothesis as false. This iscalled a Type I error, wrongly rejecting a realnull hypothesis. The probability of commit-ting a Type I error is set by researchers whenthey establish the level of statistical signifi-cance or the p-value, also known as the al-pha (α) level. By convention, researchersusually use a p-value of 0.05, indicating theyhave a 5% chance of committing a Type Ierror.1 Thus, the example study findingshave incorrectly led to a rejection of the nullhypothesis. Researchers can reduce thechance of committing a Type I error by in-creasing the level of significance, as an ex-ample, from 0.05 to 0.01. In so doing, theresearcher has reduced the statistical powerof the test (the ability to find a differenceshould it exist) and increased the chance ofmaking a Type II error.

In the third cell (C), the null hypothesisfor the population is false but the study find-ings indicate it is true (Figure 1). In otherwords, a difference exists but the study didnot detect the difference, which is known asa Type II error. The probability of making a

Type II error is usually denoted as beta (β).1

The example study results are incorrect.In contrast, statistical power is usually

denoted as 1-β, or the chance of not mak-ing a Type II error when the population nullhypothesis is false (when a true differencedoes exist). By convention, statistical poweris usually set at 0.80, meaning that four outof five times (80%) a false null hypothesiswill be correctly rejected. A higher power(e.g., 0.85, 0.90) would always be preferred,if possible.2 Both statistical significance andstatistical power are influenced by the size ofa sample. Under-powered studies (e.g., toosmall sample size) are frequently the reasonfor not detecting differences between/among groups in a study. It is also possibleto have the power of a study so high that veryminor differences are detected as statisticallysignificantly different, but in which the dif-ferences have no practical implications.3

In the fourth cell (D), the example studyresults correctly support the population nullhypothesis. Thus, there are two potentiallycorrect, but different, outcomes when con-ducting a study (Figure 1): correct rejectionor correct acceptance of the null hypothesis.

Most studies in the health educationarena are more likely to be under-powered,rather than over-powered.4 In other words,because of time and costs, more health edu-cation researchers will use smaller samples(i.e., a few hundred subjects) rather thenvery large samples (i.e., 3,000 to 10,000subjects). It should be noted that a case has

been made in the professional literature tosuggest that under-powered studies areunethical.5 This is, in part, due to researchsubjects being inadequately informed aboutthe potentially limited value of being partof a study in which the research may notbe able to detect important statistically sig-nificant effects.

Forms of Power AnalysisStatistical power is influenced by four

factors: the level of statistical significance(α); the effect size—the magnitude of thedifference between the two sample groupsbeing examined on a specific outcome vari-able; the variance of the responses to theoutcome variable; and the size of thesample.6,7 The only factor that logically canbe modified at the beginning of a study isthe size of the sample. Thus, researchersneed to focus their attention on sample sizeto ensure adequate statistical power for theanalysis of their data.

The first and most common use ofpower analysis seeks to determine whatsample size is needed to be able to reject anull hypothesis at a particular p-value (e.g.,0.05). The second component, effect size(ES), is not known but needs to be esti-mated. Effect size often can be estimatedfrom a review of the published literature, apilot study can give an estimate, and onecan use a “guesstimate” by using generaleffect sizes proposed by well known research-ers in this field (e.g., Jacob Cohen).7,8 It isrecommended that collaboration with a

Population Score(Null hypothesis is:)

Correct Rejection Type I Error(1-β) (α)

(A) (B)

Type II Error Correct Acceptance(β)

(C) (D)

False True

Figure 1. Hypothesis Testing Using Statistical Significance Testing and Power Analysis

False

True

Study Score(Null hypothesis is:)

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014

Page 4: Power Analysis in Survey Research: Importance and Use for Health Educators

James H. Price, Joseph A. Dake, Judy Murnan, Jaime Dimmig, and Sutoidem Akpanudo

204 American Journal of Health Education — July/August 2005, Volume 36, No. 4

statistician with the technical skills to con-duct such an analysis take place. For thosemore comfortable with statistics, there is anincreasing amount of software for determin-ing sample size, including nQuery Advisor,PASS, UnifyPow, and Power and Precision.

The second form of power analysis iswhen a researcher wants to be able togeneralize the results of his/her sample tothe population from which the samplewas drawn. To determine this sample size,researchers need to know the following:how much sampling error they will accept;the size (n) of the population; how muchvariation there is in the population withrespect to the outcome variable being stud-ied; and the smallest subsample in thesample for which sample size estimates areneeded. Table 1 provides sample sizes nec-essary to be able to generalize the sampleresults to the population given a variety ofsampling errors, population sizes, andvariation in the variable under study. Forexample, if one wanted to survey a commu-nity regarding firearm control and the re-searcher knew that the population had

evenly split (50/50) perceptions regardingsupport for a ban on the sale of handgunsto the general public, and the populationof the community was 50,000 people, andone wanted the responses to the survey tohave only a +/- 3% sampling error, then onewould need a sample of 1,045 completedsurveys. However, if the researcher was will-ing to have a larger sampling error, for ex-ample 5%, then one would need only 381completed surveys. In other words, usingthe 5% sampling error column (and the50,000 population row), this would meanthat if the gun control survey found that63% of the population supported eliminat-ing the sale of handguns to the public, thenone could be sure 95% of the time that, witha random sample of 381 individuals, theentire 50,000 adults believe the same resultswithin a +/- 5% range (58% to 68%).

From Table 1, it can be seen that in verylarge populations (e.g., 100,000 or more)the samples needed are about the same sizeregardless of the size of the population.However, when a researcher is examining apopulation of 5,000 or less, then the sample

size needed is a much larger portion of thetotal population. Also, it should be notedthat the more diverse the beliefs in a popu-lation, the larger the sample size needed.

Power Analysis VersusSurvey Return Rates

The use of power analysis for determin-ing sample size is needed for calculating sta-tistical analyses and for appropriate gener-alization to the population. The latter ofthese, generalizing to the population (ex-ternal validity), requires an additional con-sideration: the survey return rate.10 Whenthe concern is the ability to generalize to thepopulation, power analysis is important asan initial step to determine the number ofcompleted and usable surveys needed. Thisneeds to be taken a step further, however.

Suppose that power analysis was con-ducted to determine the number of usablesurveys needed to be returned to general-ize to a population of 5,000 (with 95% con-fidence, 50/50 split, and plus or minus 3%error). The number of completed surveysneeded in this example is 880. If Survey Awere sent to a sampling frame of 3,000 (ofthe 5,000) and 880 were returned, theneeded number of surveys was achievedbut with a return rate of 29.3% (880/3,000).In another example, Survey B was sent toa sampling frame of 1,500 (of the 5,000)and 880 were returned for a rate of 58.7%(880/1,500). Which situation is better?The answer depends on two issues: poten-tial for sampling bias and potential forresponse bias.

Sampling bias occurs when the sampleis obtained in such a manner that thesample is different from the population re-garding characteristics important to thestudy. Sampling bias can be investigated ifdata are available from the population re-lated to the subject matter being studied.In most cases in the health education arena,it may not be possible to have this informa-tion. Thus, the investigation of samplingbias is assessed based on the quality of themethods used to obtain a representativesample of the population. The quality ofthese sampling methods can vary fromvery good (random sample of the entire

Table 1. Sample Sizes For Three Levels of SamplingError at the 95 Percent Confidence Level

+ 1% + 3% + 5% Sample error Sample error Sample error

50/50 80/20 50/50 80/20 50/50 80/20split split split split split split

100 99 98 92 87 80 71250 244 240 203 183 152 124500 475 462 341 289 217 165750 696 669 441 358 254 1851,000 906 860 516 406 278 1982,500 1,984 1,777 748 537 333 2245,000 3,288 2,757 880 601 357 23410,000 4,899 3,807 964 639 370 24025,000 6,939 4,934 1,023 665 378 24350,000 8,057 5,474 1,045 674 381 245100,000 8,763 5,791 1,056 678 383 2451,000,000 9,513 6,109 1,066 682 384 246100,000,000 9,603 6,146 1,067 683 384 246

Source: Data were generated from Questa Research Associates.

Sample size standard calculator.9

Note: Sampling error numbers refer to completed questionnaires returned.

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014

Page 5: Power Analysis in Survey Research: Importance and Use for Health Educators

James H. Price, Joseph A. Dake, Judy Murnan, Jaime Dimmig, and Sutoidem Akpanudo

American Journal of Health Education — July/August 2005, Volume 36, No. 4 205

population) to very poor (volunteers, con-venience samples, etc).

Response bias occurs when the peopleresponding to the survey are different fromthose not responding to the survey in re-gards to the subject of interest. In our pre-vious handgun example, this could be asituation where members of the NationalRifle Association (NRA), a conservative gunownership support group, responded to thequestionnaire more often than people whoare not members of the NRA. This can beinvestigated by seeking out a sample of non-respondents and trying to collect the infor-mation originally sought. The extent towhich those who responded were differentfrom those who did not respond representsthe magnitude of the response bias.

In the aforementioned examples, if bothSurvey A and Survey B were free from sam-pling bias and response bias, then the ex-ternal validity of the responses of Survey Awould be equal to the external validity ofthe responses of Survey B. Thus, the differ-ence in the survey return rates would notbe important when generalizing the resultsto the population (e.g., both have good ex-ternal validity).

If both surveys contained sampling biasbut were free from response bias, then Sur-vey A would be better than Survey B. Thisis because the sampling frame of Survey Acontained a larger portion of the entirepopulation [3,000/5,000 (60%)] than Sur-vey B [1,500/5,000 (30%)]. A larger por-tion of the population included in the sam-pling frame increases the probability thatthe varied perceptions in the populationare included in the responses of the sample.Because response bias does not exist in ei-ther survey in this example, the smallersampling frame in Survey B is more likelyto negatively impact the generalizability ofthe responses of the sample.

If both surveys were free from samplingbias (e.g., both were randomly selected) butthey each had a response bias, then SurveyB would be better than Survey A. Withoutsampling bias, the sampling frames for eachsurvey were likely to be representative of thepopulation. Thus, the ability to generalize

the responses of the sample varies based onhow well the people who respond to thesurvey represent the potential responses ofthe subjects composing the sampling frame.While both surveys have response bias, themagnitude of the impact from the responsebias is greater in Survey A because two-thirds of the sampling frame did not re-spond. This is in contrast to Survey B whereonly one-third of the sampling frame didnot respond. Thus, in this example, the sur-vey return rate plays an important role inthe ability to generalize the sample resultsto the population.

The importance of survey return ratesalready has been examined.10 However, ofequal importance in assessing the qualityof survey research is understanding the ap-propriate use of the size of samples (poweranalysis). Thus, another purpose of thismanuscript is to examine the use of poweranalysis in health education research.

METHODS

JournalsSeven leading journals in the field of

health education were studied to assess thereporting of power analysis. Criteria forjournal selection included: health educationorientation, a general nature instead oftopic-specific (e.g., Journal of Drug Educa-tion), and availability in at least 25% of col-lege and university libraries.11 The sevenjournals included in the sample were (inalphabetical order): American Journal ofHealth Behavior, American Journal of HealthEducation, American Journal of Health Pro-motion, Health Education & Behavior,Health Education Research, Journal ofAmerican College Health, and Journal ofSchool Health. Power analysis deficiencies inarticles in these journals potentially wouldhave a major impact on health educationresearch. Data were collected from the jour-nals for the years 2000 through 2003, rep-resenting a span of four years.

InstrumentThe selected journals were reviewed for

articles meeting the criteria of a quantita-tive research article. These articles included

Likert-type surveys, tallies, and other sur-veys containing data that could containquantitative statistical analyses. Excludedarticles included qualitative articles, reviewarticles, editorials, and column articles thatwere not main articles (i.e., book reviews,letters from the editor, etc.).

The reviewers examined the methodssections of the selected articles, which werethen recorded on a simple scoring sheetdeveloped specifically for this project. Thedata recorded included: journal name andyear, total number of main articles, totalnumber of quantitative articles, and per-centage of quantitative articles in which apower analysis was performed. Power analy-sis included any author self-reports of apriori power analysis to detect a statisticaldifference or to generalize the study find-ings to the population. In the event that theauthor of an article did not state that apower analysis was performed, the review-ers instead searched for key words andphrases indicating the potential use of apower analysis. These words included“sample size calculation,” “Cohen’s effectsize,” and formulas and diagrams withpower calculations. If the author of the ar-ticle did not perform a power analysis priorto the study, but mentioned it in the limita-tions section, the article was not counted ascontaining a power analysis.

AnalysisAnalysis of the data consisted of descrip-

tive data, namely, frequencies, percents, andmeans. To assess accuracy of identifyingreported survey return rates, a sample oftwo different journals was used and a Kappacoefficient was calculated to assess inter-rater reliability among the three journal re-viewers. The Kappa coefficient was used tocompensate for chance agreement of the“yes” or “no” assessments. The mean Kappacoefficient was 0.905.

RESULTSPower analyses were rare in the seven

health education journals (Table 2). Overthe years 2000 through 2003, the averagepower analysis ranged from a high of 25%

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014

Page 6: Power Analysis in Survey Research: Importance and Use for Health Educators

James H. Price, Joseph A. Dake, Judy Murnan, Jaime Dimmig, and Sutoidem Akpanudo

206 American Journal of Health Education — July/August 2005, Volume 36, No. 4

of the quantitative research articles in theAmerican Journal of Health Behavior to a lowof 1% in the Journal of American CollegeHealth. Four (American Journal of Health

Promotion, Health Education & Behavior,Journal of American College Health, andJournal of School Health) of the seven jour-nals had power analyses of less than 5% of

Table 2. Power Analysis Assessment of Research Articles in Leading Health Education Journals, 2000–2003

Journal Year Total Articles Quantitative Articles Power Analysis N (%)

American Journal of Health Behavior2000 44 28 6 (21.4)2001 53 29 10 (34.5)2002 45 28 8 (28.5)2003 55 50 10 (20)Total 197 135 34 (25)

American Journal of Health Education2000 45 22 3 (13.6)2001 39 19 2 (10.5)2002 44 24 1 (4.2)2003 47 20 4 (20)Total 175 85 10 (12)

American Journal of Health Promotion

2000 39 90 0 (0)2001 32 23 0 (0)2002 25 20 1 (5)2003 41 24 1 (4.2)Total 137 67 2 (3)

Health Education & Behavior2000 45 29 0 (0)2001 40 23 1 (4.3)2002 40 27 2 (7.4)2003 39 26 1 (3.8)Total 164 105 4 (4)

Health Education Research2000 58 32 10 (31.3)2001 52 31 2 (6.5)2002 56 28 4 (14.3)2003 57 36 8 (22.2)Total 223 127 24 (19)

Journal of American College Health2000 28 19 1 (5.2)2001 27 21 0 (0)2002 22 20 0 (0)2003 22 21 0 (0)Total 99 81 1 (1)

Journal of School Health2000 58 27 1 (3.7)2001 52 33 2 (6.1)2002 53 40 0 (0)2003 52 38 1 (2.6)Total 215 138 4 (3)

their quantitative research articles.

DISCUSSIONThe current study has confirmed in

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014

Page 7: Power Analysis in Survey Research: Importance and Use for Health Educators

James H. Price, Joseph A. Dake, Judy Murnan, Jaime Dimmig, and Sutoidem Akpanudo

American Journal of Health Education — July/August 2005, Volume 36, No. 4 207

health education what has been found inother research fields, such as nursing andhealth psychology12,13: that few researchersare using a priori statistical power analysis.While it is not evident from this study whyhealth education researchers, manuscriptreviewers, and journal editors continue todiscount this important attribute of qual-ity research, it is likely that there are mul-tiple reasons. One reason may be that manyresearchers are unfamiliar with the impor-tance and appropriate use of power analy-sis in survey research. This would indicatea lack of training in health education pro-grams pertaining to power analysis. Gradu-ate programs in health education could helpto remedy this issue by including units onpower analysis into their research methodscourses. Most health education researchersengage in research for altruistic reasons,such as to advance the field of health edu-cation and/or to advance the skills of gradu-ate students. Thus, it is critically importantto the quality of health education researchthat both graduate students (our future re-searchers) and our peers be better informedabout power analysis.

Another reason for the lack of poweranalyses done in health education researchcould be that sample sizes based on appro-priate power analysis would sometimes re-quire larger samples than are seen in pub-lished health education research. Thiswould require greater financial investmentand/or time investment. These researchersmay not consider power analysis to be es-sential when compared to tradeoffs for timeand financial investment due to largersample sizes. However, not to use poweranalysis can result in important hypothesesnot being supported by underpowered re-search. For example, suppose a health edu-cation researcher investigated the effective-ness of a curriculum to increase the physicalactivity of students. In the evaluation, theresearcher surveyed 150 students when 250students would have been required, basedon an appropriate power analysis calcula-tion. The results of the evaluation concludethat there were no statistically significantdifferences between the intervention and

control group. Because a power analysis wasnot conducted, one would be less confidentin the findings. Due to the greater possibil-ity of a Type II error, the curriculum mayindeed be effective at increasing physicalactivity. By not conducting a power analy-sis and using the appropriate sample size,the evaluator/researcher may have wastedlimited resources on an evaluation that haslittle to offer. Furthermore, the evaluatormay be reporting a curriculum as ineffec-tive when, in fact, it may have been veryeffective. In other words, underpoweredstudies can result in important researchfindings not being found. Effective inter-ventions overlooked due to underpoweredassessments could result in a serious prob-lem for the health education field. To helpreduce this problem in health education, re-searchers need to calculate power analysisbefore conducting studies or evaluationsand then include the information on howsample size decisions were made when theyreport their findings.

Finally, the limitations of this studyshould be explored before accepting the re-sults. First, it may have been that more pub-lished research studies than found in thecurrent study actually were based on a prioripower analysis, but the authors of the stud-ies failed to report the analysis. Second, theauthors of some studies may intuitivelyhave used large enough samples such thatpower analysis would not have changed thesample size. However, guessing at adequatesize samples could have led to overpoweredstudies and statistically significant trivialresults. Third, the current analysis of sta-tistical power simply examined whether apower analysis was reported; it did notattempt to assess if the power analysis wasadequately conducted. Fourth, it may bethat health education research published injournals with higher-impact factors may bereporting power analyses. Even if this wereso, it would not appear to justify the lim-ited use of power analysis in the majorityof health education journals.

REFERENCES1. Fox N, Mathers N. Empowering research:

statistical power in general practice research.

Fam Pract. 1997; 14: 324–329.

2. Lenth RV. Some practical guidelines for

effective sample size determination. Am Stat.

2001; 55: 187–193.

3. Torabi MR. How to estimate practical sig-

nificance in health education research. J Sch

Health. 1986; 56: 232–234.

4. Cuijpers P. Examining the effects of pre-

vention programs on the incidence of new cases

of mental disorders: the lack of statistical power.

Amer J Psychiatry. 2003; 160: 1385–1391.

5. Halpern SD, Karlawish JHT, Berlin JA. The

continuing unethical conduct of underpowered

clinical trials. JAMA. 2002; 288: 358–362.

6. Kelly K, Maxwell SE, Rausch JR. Obtain-

ing power or obtaining precision: delineating

methods of sample size planning. Eval Health

Prof. 2003; 26: 258–287.

7. Hallahan M, Rosenthal R. Statistical

power: concepts, procedures, and applications.

Behavioral Research Ther. 1996; 34: 489–499.

8. Cohen J. Statistical Power Analysis for the

Behavioral Sciences. 2nd ed. New York: Academic

Press; 1988.

9. Questa Research Associates. Sample Size

Standard Calculator. Available at http://

www.questaresearch.com/calc_ss_adv.php. Ac-

cessed November 9, 2004.

10. Price JH, Murnan J, Dake JA, Dimmig J,

Hayes M. Mail survey return rates published in

health education journals: an issue of external

validity. American Journal of Health Education.

2004; 35: 19–23.

11. Laflin MT, Horowitz SM, Nims JK,

Morrell LJ. Availability of health education jour-

nals in academic libraries. Amer J Health Behav.

2000; 24: 193–200.

12. Maddock JE, Rossi JS. Statistical power

of articles published in three health psychology-

related journals. Health Psychol. 2001; 20:

76–78.

13. Polit DF, Sherman RE. Statistical power

in nursing research. Nurs Res. 1990; 39: 365–369.

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014

Page 8: Power Analysis in Survey Research: Importance and Use for Health Educators

208 American Journal of Health Education — July/August 2005, Volume 36, No. 4

1. Approximately ______of high school stu-dents self-report having participated insome form of academic dishonesty.

a. 10–20%b. 20–40%c. 30–45%d. 50–75%

2. Participants in the study were 700 stu-dents in grades 6–12 from a single

a. Northern school district.b. Southern school district.c. Western school district.d. Eastern school district.

3. Data were collected using

a. online testing.b. a self-report questionnaire.c. qualitative interviewing techniques.d. observation of students.

4. The Hare Self-esteem Scale provides anarea-specific measure of self-esteem in theareas of

a. peer.b. school.c. home.d. all of the above.

5. Two religiousity items focused onritualistic and ______

a. denomination.b. verification of church membership.c. experiential.d. number of church leaders.

6. The results indicated that _____of stu-dents had told lies to stay out of troublein the past year.

a. 64%b. 74%c. 84%d. 94%

7. Gender was not independent of

a. having told lies to get someone else introuble.

b. having stolen items from stores.

c. having stolen items from individuals.d. all of the above.

8. Frequency of attendance at worshipservices was not independent of stealingfrom an individual.

a. trueb. false

9. Results of the univariate analyses indi-cated that _____ and home self-esteem andself-efficacy were significantly related toparticipation in all six character behaviors.

a. churchb. schoolc. community groupsd. sports teams

10. This study supports the claims ofBensley and others that believing that some-thing is wrong does not necessarily lead tobehaviors that represent those beliefs.

a. trueb. false

Continuing Education Questions

Event Code: 7-2005For Continuing Education Contact Hours Article,

Liars, Cheaters, and Thieves: Correlates of Undesirable Character Behaviors in AdolescentsCongruent to Area VII: Communicating Health and Health Education Needs, Concerns, and Resources

Self-study Questions for Continuing Education HoursAAHE, as a multiple event provider through The National Commission for Health Education Credentialing, Inc., provides this self-study opportunity in the American Journal of Health Education. Category 1

continuing education contact hours (CECH) are awarded for each article. After completion of six articles, a certificate is sent to the Certified Health Education Specialist. Each article is worth 1 CECH.Please keep a copy for your records. There will be no notification from AAHE until you complete six CECH or if you fail to score 80% on each article.

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Name: _______________________________________________________________________________________________________________

Address: ___________________________________________________________________________________________________________________________________________________________________ Email: _________________________________________

1 CECH PER ARTICLEMEMBER NON-MEMBER Ches #:

CHES #:__________________ AAHE Membership #: _______________

1 CECH PER ARTICLE 6 CECH for $12; 12 for $24

Make checks payable to AAHE.

6 CECH for $25; 12 for $50

Make checks payable to AAHE.

AMERICAN ASSOCIATION FOR HEALTH EDUCATION Registration Form

AAHE • 1900 Association Drive • Reston, VA 20191-1599 • NCHEC Provider Number: VA0008

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014

Page 9: Power Analysis in Survey Research: Importance and Use for Health Educators

American Journal of Health Education — July/August 2005, Volume 36, No. 4 209

Event Code: 8-2005For Continuing Education Contact Hours Article

Power Analysis in Survey Research: Importance and Use for Health EducationCongruent to Area VII: Communicating Health and Health Education Needs, Concerns, and Resources

1. A _______assessment tells us how likelyit is that a statistical significance test willdetect a significant difference betweentwo or more groups given that a differ-ence actually exists.

a. frequencyb. descriptivec. statistical powerd. regression

2. Statistical power is influenced by______factor(s).

a. 1b. 2c. 3d. 4

3. The first and most common use ofpower analysis seeks to determine what______ is/and needed to be able to rejecta null hypotheses at a particular p-value.

a. sample sizeb. confidence intervalsc. software packaged. demographics

4. _______occurs when the sample is ob-tained in such a manner that the sampleis different from the population regard-ing characteristics important to the study.

a. Sampling frameworkb. Internal validityc. Sampling biasd. External validity

5. _______occurs when the people re-sponding to the survey are different thanthose not responding to the survey in re-gards to the subject of interest.

a. Sampling biasb. Response biasc. Halo effectd. External validity

6. Data were collected from the journalsfor the years 2000 through ______.

a. 2001b. 2002c. 2003d. 2004

7. The selected journals were reviewed forarticles meeting the criteria of a

a. quantitative research article.b. qualitative research article.c. classic experimental design article.d. non-parametric design article.

8. Power analyses were rare in the sevenhealth education journals.

a. trueb. false

9. The current study has confirmed thatin health education which has been foundin other research fields, such as ______ andhealth psychology.

a. sociologyb. social workc. nursingd. dentistry

10. Guessing at adequate size samplescould have led to over-powered studiesand trivial results.

a. trueb. false

Self-study Questions for Continuing Education HoursAAHE, as a multiple event provider through The National Commission for Health Education Credentialing, Inc., provides this self-study opportunity in the American Journal of Health Education. Category 1

continuing education contact hours (CECH) are awarded for each article. After completion of six articles, a certificate is sent to the Certified Health Education Specialist. Each article is worth 1 CECH.Please keep a copy for your records. There will be no notification from AAHE until you complete six CECH or if you fail to score 80% on each article.

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Name: _______________________________________________________________________________________________________________

Address: ___________________________________________________________________________________________________________________________________________________________________ Email: _________________________________________

1 CECH PER ARTICLEMEMBER NON-MEMBER Ches #:

CHES #:__________________ AAHE Membership #: _______________

1 CECH PER ARTICLE 6 CECH for $12; 12 for $24

Make checks payable to AAHE.

6 CECH for $25; 12 for $50

Make checks payable to AAHE.

AMERICAN ASSOCIATION FOR HEALTH EDUCATION Registration Form

AAHE • 1900 Association Drive • Reston, VA 20191-1599 • NCHEC Provider Number: VA0008

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

18:

50 2

4 O

ctob

er 2

014