ByPatricia Pinto - NJMS Class of 2009,

Marielos L. Vega, BSN, RN, Jeanne Ferrante, MD,

Daniel M. Rosenblum, PhD, Stanley H. Weiss, MDSeptember 8, 2005

This study was supported by: NJMS -Office of Research and Sponsored Programs- 2005 Summer Student Research Program

This study was supported by: NJMS -Office of Research and Sponsored Programs- 2005 Summer Student Research Program

"An Evaluation of Cancer Educational Materials with a Focus on those Currently Distributed by

Health Departments in New Jersey"

Background

Health Literacy: “The degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions”. (Ratzan and Parker, 2000)

40 million Americans are functionally illiterate. (Horner, 2000)

Health educational materials should be written below the 6th grade level. (Cotunga, 2005)

About 90 million (47 percent) U.S. adults cannot accurately and consistently locate, match, and integrate information from newspapers, advertisements, or forms. (Kirsch et al., 1993)

Study Purpose

This study sought to determine publication sources, types of cancers addressed, readability and cultural sensitivity of cancer education materials [English & Spanish] that are currently distributed by local health departments in New Jersey.

A secondary objective was to compare the results obtained from different reading scales.

Methods

Procurement of materials (brochures, booklets, fact sheets)

Contacted local health departments (LHDs) in NJ via postal mail and e-mail (NJDHSS, 2005)

Asked to submit copies of cancer education materials that they distributed

Materials were also obtained directly from publishers so that Spanish and English versions were available for all materials analyzed

To assess the reading level of English materials the validated SMOG scale was used (McLaughlin, 1969)

Methods (continued)

The Fry Graph method was used to assess English and Spanish materials, as it is validated as an instrument for Spanish materials (Gilliam, 1980)

To assess the cultural sensitivity of the materials, the Cultural Sensitivity Assessment Tool (CSAT) was used

- This tool rates the format, written and visual messages of educational materials on a scale of 1 to 4, with 4 being the best possible score for each section and 12 being the highest total score for cultural sensitivity. Elements such as print size, font, familiarity of expressions and cultural sensitivity of graphics were rated.

Results

Although 382 items were obtained, 148 items were available in English only and therefore not included in the study;

No item was available only in Spanish

295 materials were received from LHDs 54 (46%) of the 116 LHDs in NJ sent materials

87 pieces of materials were obtained directly from publishers For 53 of these pieces, only the English version is being

distributed by LHDs. We obtained the Spanish counterpart. 34 were pieces not being used by the LHDs (17 paired sets).

Results

There were 117 paired sets of English and Spanish documents which are

further analyzed below

17

(both from publishers)

47

(both English and Spanish from LHDs)

53

(English from LHD; Spanish counterparts from publishers

Where do the paired pieces in analysis come from?

117 paired sets for analysis

Results

Of the 234 items analyzed, 59 (25%) were being distributed by more than one health department; of these items, 41 were in English and 18 were in Spanish

Only 17 health departments distributed any paired English and Spanish set

6% of the materials were published by a LHD; these were all flyers

26% were published by for-profit publishers

68% were published by non-profit agencies

Publishers - Results

Main PublishersAmerican Cancer Society

(n=64, 27%) National Cancer Institute (n=42,18%)

Krames, Inc (n=36, 15%)

Cancer Research Foundation of America (n=18, 7.7%)

Figure 1: Publishers of education materials

27%

6%

18%

26%

23% American CancerSociety

Health Department

National CancerInstitute

Profit Organizations

Other Non ProfitOrganizations

Figure 1: Publishers of education materials

27%

6%

18%

26%

23% American CancerSociety

Health Department

National CancerInstitute

Profit Organizations

Other Non ProfitOrganizations

Content - Results

Cancer Education Materials Received by Type of Cancer

Other Cancers23%

Breast33%

Cervical11%Colorectal

22%

Prostate 11%

Reading Level - Results

92% of English materials were written above the recommended 5th grade reading level as assessed by either scale – mean reading levels for Fry: mean= 9.3, 95% CI=9.3±0.43 &SMOG: mean=8.7, 95% CI=8.7±0.49.

93% of the Spanish materials were written above the recommended 5th grade reading level – mean reading levels for Fry: mean=8.0,95% CI=8.0±0.42.

Reading Level - Results

Materials written in English for cancer patients had only a slightly higher mean reading level (Fry: mean=9.2, 95%CI=9.2±0.89; SMOG: mean=9.5, 95%CI=9.5±0.89) than materials written for the general public(Fry: mean=8.5, 95%CI=8.5±0.58; SMOG: mean=8.9, 95%CI=8.9±0.58)

Materials written in Spanish for cancer patients had a significantly higher mean reading level (Fry: mean=9.4, 95%CI=9.4±0.83) than those written for the general public (Fry: mean=7.5, 95%CI=7.5±0.44) (p<0.001, Student’s t-test)

All of the Spanish versions were translations of pre-existing English materials, as ascertained by copyright dates

Cultural Sensitivity - Results

48% of the Spanish materials had cultural sensitivity scores at or below the acceptable value of 2.5 in one or more of the written, format and visual categories

23% of English materials scored at or below the acceptable value in one or more of the above categories

English materials had significantly higher average written (3.7) (p<0.01) and visual scores (2.7) (p<0.001) than their Spanish counterparts (3.6 written, 2.2 visual)

Cultural Sensitivity - Results

Materials in both languages scored highest (best) on format of content and lowest (worst) in visual message

Materials published by for-profit publishers had significantly higher total cultural sensitivity scores, indicating a higher level of cultural sensitivity, than materials published by non-profit publishers (p<0.05)  

Figure 3: Average total cultural sensitivity scores by publisher

0

2

4

6

8

10

12

Total score

Publ

isher

Spanish

English

Figure 3: Average total cultural sensitivity scores by publisher

0

2

4

6

8

10

12

Total score

Publ

isher

Spanish

English

Conclusions

Cancer education materials can be very useful in informing the general public about risks of developing cancer and can give cancer patients information on treatment options.

If these materials are written at reading levels above their capability or if they lack cultural sensitivity, then they will have limited utility for target audiences.

Conclusions

No LHD had any materials on oral cancer AND most LHDs did not cover a majority of the NJ-CCCP priority cancers* in either language.

No item existed solely in Spanish (although, many documents not analyzed as part of this report existed in English).

*NJ-CCCP priority cancers: breast, cervical, colorectal, lung, melanoma, oral, & prostate

Conclusions

We strongly recommend that LHDs utilize census data concerning linguistic abilities and educational levels in their communities when choosing materials to disseminate.

Demographic information may also be useful in helping LHDs select appropriate cancer-related health materials based on community needs.

Conclusions

Further validation of reading scales would be of value to ascertain whether differences observed were due to intrinsic vs. extrinsic language and culture factors.

Special consideration must be given to the selection of Spanish education materials, since the 2 languages have different expressions and syntax; thus, direct translations of materials may result in the use of expressions that are unfamiliar to the Hispanic audience.

Conclusions

If materials are not originally intended for a Hispanic audience, graphics may be culturally inappropriate.

Many items were translated but the graphics were left intact rather than being culturally adapted for the new target audience.

Both Spanish and English materials assessed showed a wide degree of variation (Fry: Range=2 to 15, SD=2.3; Range=1 to 15, SD=2.3, respectively) in reading levels.

In 59 (50%) sets, the English and Spanish versions differed by 2 or more reading levels indicating a great variation among the translated materials in relationship to the original English version.

Next Steps

To assist local health departments that have collaborated in this project in choosing materials, we plan to distribute to them an electronic database which includes the scores and sources for all materials provided to us

Disseminate finding to scientific community – Journal of Cancer Education

References

Cotunga, N, et al. 2005. Evaluation of literacy level of patient education pages in health-related journals. Journal of Community Health, 30: 213-219.Department of Health and Senior Services. 2005. Directory of Local Health Departments in New Jersey.Doak, L, et al. 1995. Strategies to improve cancer education materials. Oncology Nursing Forum, 23: 1305-1313.Fondo de Cultura Económica. Accessed at: http://www.fce.com.ar/fsfce.asp?p=http://www.fce.com.ar/categorias.asp?SCAT=2401.Gilliam, B., et al. 1980. The Fry graph applied to Spanish readability. The Reading Teacher, 33:426-430.Horner, S, et al. 2000. Improving Readability of Patient Education Materials. Journal of Community Health Nursing, 17: 15-23. McLaughlin, G. 1969. SMOG grading: A new readability formula. Journal of Reading, 12: 639-646.