12/4/20041 UNC ECHO, HBCU Health Promotion Alliance, NC A&T School of Agriculture Workshop on Grantwriting for Health Disparities Research Victor J. Schoenbach,

12/4/2004 1

UNC ECHO, HBCU Health Promotion Alliance, NC A&T School of Agriculture Workshop on Grantwriting for Health Disparities Research

Victor J. Schoenbach, Ph.D.Director, Minority Health Project

Department of EpidemiologyUNC-Chapel Hill School of Public Health

www.unc.edu/~vschoenb/

This presentation can be downloaded from:www.minority.unc.edu/present/2004/

An honest politician

“That lowdown scoundrel deserves to be

kicked to death by a jackass, and I'm just the

one to do it” – Congressional candidate in

Texas.

3

Portions drawn from

“A Grant Writing Primer” by Hinda Zlotnik, Ph.D.MARC/MBRS Program, NIGMS

brin.hpcf.upr.edu/symposium/How%20to%20write%20successful%20proposals.ppt

“Writing a grant proposal”, by Malcolm Gordon, Ph.D., National Institute of Mental Health, 1999

www.charityadvantage.com/iaswr/images/grantproposals.pdfBoth downloaded 11/26/2004

11/29/2004 PHS 398 Instructions 4

Proposal components• Abstract• Budget, Biographical Sketches, Resources• Research Plan

– A. Specific Aims (and hypotheses)– B. Background and Significance– C. Preliminary Studies / Progress Report (and

background of investigators)– D. Research Design and Methods– E. Human Subjects Research– G. Literature cited


Follow instructions

“Read and follow the instructions carefully to avoid delays, misunderstandings and possible return of the application. Adherence to font and margin requirements is necessary for several reasons.…”

“Use English and avoid jargon.”


Don’t believe everything

“The PHS estimates that it will take approximately 40 hours to complete this application for a regular research project grant.”



– A. Specific Aims (and hypotheses) [1 page]– B. Background and Significance– C. Preliminary Studies / Progress Report (and



a. Specific Aims

(outline labeling is inconsistent)

The framework for the entire proposal, including the research plan.

Hinda Zlotnik, Ph.D. 9

Requirements for a Good Specific Aims Section

• Brief introduction including the long range goal of the project

• The goals of this particular application

• The central hypothesis to be examined

• Rationale for the project

• Specific aims

• Anticipated results


Rationale

• This is the underlying reason for the studies you propose.

• The rationale must be relevant to the problem that has been presented in the introduction.

A dubious rationale

“It isn't pollution that's harming the environment. It's the impurities in our air and water that are doing it.” – Al Gore, Vice President

From Hinda Zlotnik, Ph.D. 12

Specific Aims

• Statements of (measurable) end results, not means to an end.

• Brief, focused, and limited in scope.

• Each aim flows logically into the next aim.

• Each should be briefly expanded upon.

• Be realistic: do not overestimate your capabilities for completing the work in the time requested.



– A. Specific Aims (and hypotheses)– B. Background and Significance [2-3 pages]– C. Preliminary Studies / Progress Report (and


11/29/2004 Adapted from the PHS 398 Instructions

14

b. Background and Significance

• Sketch the background, critically evaluate existing knowledge, and specifically identify gaps.

• State concisely the importance and health relevance by relating the specific aims to broad, long-term objectives.

• State how scientific knowledge or clinical practice will be advanced. Describe the effect on the concepts, methods, technologies, treatments, services or preventative interventions that drive this field.

11/30/2004 V. Schoenbach 15

The Review Process

• Lynchpin of the NIH review process is the initial review group (IRG).

• Helpful to know the psychology and sociology of review committees – many agendas.

• Helpful to know who is on the review committee, what is their expertise, what they have published.


The Review Process - 2

Purchasing metaphor• Want to buy, have money to spend• Want to be an informed consumer• Don’t want to be duped• Want value for money


The Review Process - 2

Marketing metaphor:• Stimulate interest, be in fashion• Convey information• Build positive regard• Avoid unrealistic expectations• Package attractively

July 1999 Malcolm Gordon, Ph.D. 18

Proposals not scored or downgraded

• “Fatally flawed” methodology or infeasible

• “Incomplete” – missing key detail

• Serious human subjects concern

• Likely not to provide any new or important substantive knowledge

Would you fund this advance?

“If somebody has a bad heart, they can plug this jack in at night as they go to bed and it will monitor their heart throughout the night. And the next morning, when they wake up dead, there'll be a record.” – Mark S. Fowler, FCC Chairman




background of investigators) [8-10 pgs]– D. Research Design and Methods– E. Human Subjects Research– G. Literature cited


c. Preliminary Studies/Progress Report

“Preliminary Studies. For new applications, use this section to provide an account of the principal investigator/program director’s preliminary studies pertinent to this application, including his/her preliminary experience with and outreach to the proposed racial/ethnic group members. This information will also help to establish the experience and competence of the investigator to pursue the proposed project.” [pilot studies]

Malcolm Gordon, Ph.D. 22

Preliminary Studies

“… results obtained from prior research that are relevant to the proposed research, including results that bear on the feasibility of the proposed research, on the adequacy of sample sizes you will have access to, and on the likelihood that the major hypotheses of the study will be supported.”

From Paul Godley, M.D., Ph.D. 23

Preliminary Studies – Example

“A recently published examination of racial differences in prostate cancer treatment outcomes sets the stage for successful completion of the proposed project. The study was conducted by the investigators, who have extensive experience in population based case-control studies of cancer etiology and in case-control studies of cancer screening.

“Godley PA, Schenck AP, Amamoo MA, Schoenbach VJ, Peacock S, Manning M, et al. Racial differences in mortality among Medicare recipients after treatment for localized prostate cancer. J Natl Cancer Inst 2003;95(22):1702-10.

We investigated whether racial disparities in outcomes of clinically localized prostate cancer vary by treatment. Merged SEER and Medicare files provided data (age, race, stage, grade, census tract socioeconomic status, and date of death) on 5,747 black and 38,242 white patients with clinically localized prostate cancer, aged 65 to 84, first diagnosed between 1986 and 1996 in five SEER sites. . . .”


Preliminary Studies

• Describe published studies in limited detail and include the most important figures and/or tables.

• Describe unpublished studies in more complete detail, including newer data.

• Do not duplicate the preliminary studies with the proposed studies.


Preliminary Studies

• Include the results of your recent work that have direct relevance to the studies proposed in your grant application.

• Exclude any studies in which the relationship to your proposed study is not relevant.


Editorial Considerations for Preliminary Studies

• Supporting data should be placed as close as possible to where it is referenced in the text of the application.

• All figures/graphs and tables should be clear and legible.

• Methodology should be placed in the figure/table legends, not in the text.


Technical Considerations for Preliminary Studies

• Graphs should be uncomplicated ... the simpler, the better.

• Each table or figure should be designed to convey a single point or idea.

• Extraneous or irrelevant data should be avoided.




background of investigators)– D. Research Design and Methods [8-14 pgs]– E. Human Subjects Research– G. Literature cited

11/26/2004 from PHS 398 instructions 29

d. Research Design and Methods

As described in the PHS 398 instructions:

• Conceptual or clinical framework, procedures, and analyses to accomplish the specific aims

• How data will be collected, analyzed, interpreted, and shared

• Novel concepts, approaches, methodologies, tools, or technologies

11/26/2004 from PHS 398 instructions 30

d. Research Design and Methods(more from the PHS 398 instructions)

• Discuss potential difficulties and limitations and alternative approaches

• Tentative sequence or timetable

• Hazards to personnel and precautions to be exercised


THE RESEARCH DESIGNThe “heart and soul” of the application.Should state precisely:

• What you propose to do

• How you plan to do it

• What the result will mean in terms of the overall project

• What pitfalls you might encounter

• Alternative approaches to cope with anticipated problems


Research Design: Organizational Structure

Restate each specific aim and for

each, provide:1. Introduction

2. Methods of approach

3. Anticipated findings or results

4. Potential pitfalls/alternative approaches


Research Design (continued-1)

1. INTRODUCTION

Each section of the research design should restate the hypothesis to be tested, the rationale for the study, overall approaches to be taken, and the anticipated results.



2. METHODS OF APPROACH

• Separate sections based on the specific aims should be used to develop each of the planned set of studies.

• Avoid emphasis on routine methods

• Use detailed methods only for unfamiliar technology



3. ANTICIPATED RESULTS

• Summarize your results.

• Emphasize only the most important results.

• Do not overinflate or overinterpret the results.

• If limitations exist, they should not be ignored, but should be discussed in a positive manner.



4. POTENTIAL PROBLEMS

• Anticipate potential problems, and discuss them but do not overemphasize them.

• Offer alternative strategies.

• Reconcile the results of differing approaches.


Research Design – Example“The cases for the proposed study will be males with a diagnosis of prostate cancer who have died in the years 1997 to 2000. A merged SEER/Medicare dataset, Patient Entitlement and Diagnosis Summary File (PEDSF) for years 1997 to 2000, will be used to identify the cases. The PSA screening exposure will be documented for each case from 1988, when PSA screening became widely available, to the date of diagnosis of the case. Since prostate cancer can be a relatively slow growing cancer, by limiting the study to most recent cases available in the SEER/Medicare dataset we are allowing as much as 12 years for patients who were not screened or patients who were unsuccessfully screened to become fatal cases. Ideally, cases in a case-control study of screening efficacy should be those who have suffered an adverse outcome of the condition that screening is intended to identify or prevent (41). Five SEER regions (Atlanta, Connecticut, Detroit, San Francisco and Seattle) were selected because they provided a substantial number of African American cases for analyses and included data collected since 1986.”


Research Design – Example (cont’d)

“The following potential cases will be excluded from the dataset:• Subjects with a diagnosis of a non-prostate cancer before or after the prostate cancer diagnosis• Subjects with prostatic intraepithelial neoplasia as the only prostate cancer diagnosis• Subjects with a missing month of diagnosis• Subject who were diagnosed with prostate cancer at death• Subjects without Medicare coverage during the study period • No age restrictions will be used.An analysis of the 1996 PEDSF file reveals that 924 deaths among prostate cancer cases meet these criteria. Approximately 4,000 cases can be expected over the proposed 4-year study period. Of the 924 deaths among prostate cancer patients, 739 were Caucasian and 185 were African American.”


Research Design – Example (cont’d)“Exposure AssessmentThe proposed study will compare the screening exposure of deceased cases to the screening exposure of living controls. This approach has been used previously in case-control studies of cancer screening, and the use of deceased cases not be a source of addition bias (55, 56). Screening PSA determinations were not reimbursed by Medicare until 2000 (57). However, PSA tests were frequently charged to Medicare prior to 2000 and appear in Medicare administrative data files. Diagnostic PSA testing was covered by Medicare so some of the PSA tests appearing in Medicare files may be from screening and not diagnostic patient encounters. The high prevalence of PSA testing supports the probability that PSA tests were predominantly screening and not diagnostic in intent. Freeman et al. (57) found that 37% of men in 1998 SEER/Medicare files had a PSA test. Among 1996 controls matched to the 924 cases described above, 65% of men had PSA tests billed to Medicare between 1988 and 1996. It is unlikely that the majority of men ages 65 years and over had PSA testing solely for diagnostic purposes.”


Research Design – Example (cont’d)“DATA ANALYSISThe analysis of the primary specific aim will examine exposure to PSA testing only, since PSA screening has been widely used since 1988, is the most sensitive screening test for prostate cancer, and has been documented in Medicare claims files since 1996. Our analysis will use conditional logistic regression, with case-control status as the dependent variable and PSA testing as the independent variables. We will assess the impact of the following potential confounding variables: race, age, SEER site, income and education (57). In this context a confounding variable is a determinant (or surrogate for a determinant) of prostate cancer death and is also associated with the probability of undergoing prostate cancer screening.Since the analysis will evaluate fatal prostate cancer as the endpoint, comparison of screening between cases and controls is only appropriate for the period prior to the diagnosis of prostate cancer. Therefore, using an approach similar to that of Friedman (19), we will estimate odds ratios for “screened with PSA since 1988”. Since this approach is only appropriate for cases diagnosed after 1988, any case diagnosed prior to that time will be excluded. We will also examine shorter intervals that may correspond to estimates of the detectable preclinical phase, as described by Weiss et al. (58).”



In general a review committee will be concerned with four aspects of the proposal:

1. its importance

2. its feasibility

3. the technical merits of the methodology

4. its innovativeness



Regarding technical merit, the review committee considers:

(1) conceptual framework guiding the research

(2) measurement of key variables

(3) sampling plan of the research

(4) procedures for data collection

(5) data analysis plan for the research.


Most Common Reasons for Failure

• Lack of a good original idea.

• Unimportant or unresponsive problem.

• Unacceptable rationale.

• Lack of expertise, experience, or resources.

• Superficial or unfocused research design.

Why you want to write with precision

"Your food stamps will be stopped effective March 1992 because we received notice that you passed away. May God bless you. You may reapply if there is a change in your circumstances." – Department of Social Services, Greenville, South Carolina


Most Common Reasons for Failure(more)

• Unrealistic amount of work proposed.

• Uncertain outcomes and/or lack of future directions.

• Disorganized/poorly written application.

• Lack of progress.


Practical suggestions for how to proceed

• Serve as a reviewer.

• Form a team, identify a leader, set a schedule

• Find out who will be reviewing the proposal and with what criteria

• Find a model proposal to emulate

• Make an outline of the proposal in a word processing document, add notes, write text

• Review and revise, review and revise, …


Practical suggestions - 2

Outline of Research Design and Methods section:• Overview• Design• Subjects (eligibility, recruitment, consent)• Data collection (measures, procedures, lab analyses)• Pretesting, piloting• Data analysis (variables, approach), sample size• Quality control and data management• Timetable and project management


The Research Plan: Summary of General Considerations - 1

• Read and follow instructions carefully.• Specific aims are statements of end results. They

are measurable statements, not means to an end.• Make the logic very clear and write a focused

research plan.• Don’t ramble--give sufficient background to

make the significance of the proposed research very clear.

• Frame questions in terms of testable hypotheses.



• Clearly delineate the problem and the purpose of the research.

• Emphasize the specific advantages of doing the research as you propose.

• Consider alternative strategies – there is concern when only a single approach is given or a single uncertain result serves as the basis for the project.

• Include preliminary data if possible.• Don’t overinterpret or misinterpret data.



• Don’t detail routine methods, but include comments that can convey your expertise to the reader.

• List relevant intellectual and physical resources available to you.

• Consider and discuss all variables that will impact the outcome. That is, which ones need to be controlled, which ones don’t, and how this will affect the outcome.



• Discuss the limitations of your conclusions, if any.• Prioritize the tasks proposed and provide time

estimates.• Use shorter words rather than longer ones unless

they are technical.• Use short sentences and avoid jargon.• Make sure the proposal flows logically from

section to section, i.e. methods proposed = logical extension of the specific aims.


Resources

• Succinct overview: “Proposal Writing: The Business of Science”, by Wendy Sanders. The Whitaker Foundation, 2004, www.whitaker.org/sanders.html

• Quick Guide for Grant Applications (deainfo.nci.nih.gov/extra/extdocs/gntapp.htm)

• Guide: “Writing a grant proposal”, Malcolm Gordon, Ph.D., National Institute of Mental Health, July 1999 www.charityadvantage.com/iaswr/images/grantproposals.pdf


Resources

• www.training.nih.gov/careers/careercenter/grants.html

• “Getting started – useful links” (grants.nih.gov/grants/useful_links.htm)

• Current NIH application forms (grants.nih.gov/grants/forms.htm)

• fdncenter.org/learn/faqs/html/proposal_writing.html• “Proposal writing short course”

(www.fdncenter.org/learn/shortcourse/prop1.html) – available in Spanish and French


ResourcesBooks:

• Designing clinical research: an epidemiologic perspective, Stephen B. Hulley and Steven R. Cummings and others. Baltimore, Williams and Wilkins, 1988

• How to Write a Successful Research Grant Application: A Guide for Social and Behavioral Scientists, by Willo Pequegnat, Ellen Stover. Plenum Publishing Corporation (September 1, 1995, Paperback: 252 pages, ISBN: 030644965X)


Resources• AAPM&R-RPC: Resident’s Research Packet

American Academy of Physical Medicine and Rehabilitation. Finding a Mentor, Funding, Searching the Literature, Research Design, Research Proposals, The IRB, Statistics, Presenting Your Work

www.aapmr.org/resident/resrchpac/respacka.htm

• Sigma Xi Grant Writing Course Series, 5-day intensive course presented by the Grantsmanship Center, Inc. (TGCI) for Sigma Xi [tuition: $1,000]

www.sigmaxi.org/meetings/grant/index.shtml [email protected]


Resources

My home page www.unc.edu/~vschoenb/ has these links:

• epid160.epidemiology.us – Introduction to epidemiology (includes on-line recorded lectures)

• www.epidemiolog.net – Epidemiology learning resources and “The Evolving Text” (“el Texto en Desarrollo”)

• www.minority.unc.edu – Minority Health Project, includes slide presentations, webcasts, readings

“If we don't succeed, we run the risk

of failure.” – Bill Clinton

Documents

12/4/20041 UNC ECHO, HBCU Health Promotion Alliance, NC A&T School of Agriculture Workshop on Grantwriting for Health Disparities Research Victor J. Schoenbach,