Review Process

How is the fate of your proposal decided?

Checklist

• Cover page• Table of content• Summary• Project description• References cited• Biographical sketches• Budget and budget justification• Current and pending support• Facilities and equipment

Put all these documents together in the same sequence in a single file, paginate it and convert it into a pdf file. Email me one pdf file with your last name as file name by November 4th 2004

Now, you have submitted the proposal via Fastlane through your SRO

Who looks at it and how deep?

• Administrative assistants• Format compliance• Complete set of documents• Assign to area/manager as you requested• Prepare list of individuals that are in conflict with you

• Program manager (also a Professor at some U)– Reads title and abstract– Confirms the program to which your proposal is

assigned or shifts to other programs– Decides upon the panel manager

Panel manager

• Who is the panel manager? (Roll of dice)– Well-established, successful professor at some

renowned University– Works for one-two years in a row– Is happy with his/her own career, well connected– Has vision for the future research in the topic– Reads ALL abstracts and first pages of ALL

proposals.

• Decides in consultation with program manager who could be panel members and ad hoc reviewers

Panel members

• Based on the topics of proposals submitted to a program, 10-12 panel members are invited.

• Some are the same as last year’s panel• Mostly they do not compete for funding in the

same year or have to leave discussion if any direct conflict of interest exists.

• Primary, secondary reviewers and readers• Each reads about 10-12 proposals at depth and

prepares reports for panel.• Gets input from ad hoc reviewers

Ad hoc reviewers• About 6-10 specialists are

invited to review your proposal.

• They should have no direct conflict of interest.

• Done electronically via fastlane

• Use two major criteria– Intellectual merit– Broader impact

• They rate the proposal

Next step…

• Not every ad hoc reviewer responds on time (~2 months)• May have personal bias against or for you or too busy• May be harsh or too mild (where are they on the ladder?)• They may be jealous of your success• All such reviews first go to the primary reviewer who

makes up his/her mind about fate of your proposal• Defends or destroys your proposal in the panel meeting• Secondary panel members keep a check on primary

member and reader takes notes of panel discussions• Open process on hidden agendas!

After about 3 months of proposal submission

• Panel meets at Washington D.C. for a week• Each proposal gets less than ~15 minutes of

discussion.• Someone has to passionately defend you!• Each proposal gets a rating

– Excellent, Very good, Good, fair, poor

• If it stays on the table and your name is on a sticky note, your chances of success are high

• Next day or so, panel members go back read proposals again and change ratings if necessary

Horror stories

• Simple methodology omitted and proposal tanked.

• Panel members had bad experiences with the PI• Co-PI almost lost a grant but saved at the last

minute by his fax declaring change of job• Supporting evidence submitted before the panel

meeting did save a dying proposal• Catch-22 situation! First, asked for preliminary

data and then asked why you need more money?

Program manager

• Decides the final ranking of proposals• Has some freedom to move within ranks• Decides how much money can be given• Calls or communicates with the PI• Negotiates what needs to be done and for

how much support?• Sends declination letters and reviews• Answers your questions

What if you get a grant?

• Do Party but not forever!• You are among top 10% researchers in your

field• Hire people and deliver the goods promised on

time• Publish profusely in high quality journals• Write more grants! Why?

– funding does not last for ever– 10% success rate– distribution of wealth principle

What if you do not get a grant?

• don't cry (OK, cry a little if you feel better)• pick up the pieces of your failed proposal

and restart your “grants writing” engine• get reviewers comments, read and get

angry then keep them in a drawer away from your view for a while..

• come back and read reviews again• talk to PM and your mentor/well wishers• resubmit until you succeed

How will YOUR proposals be reviewed for this FW5850 class?

• You have already submitted a single pdf file of your proposal to me by email (Today is the deadline).

• I will put them on web ASAP and email you the location of all proposals from your group.

• You will review all proposals from your group except your own.

• You will write one page review for each proposal along with rating and email me by Nov 18th.

• Your advisors, Valorie and I may review too!• All reviews will be anonymous

Evaluation Criteria

• Intellectual merit– How important this proposal is for advancement of knowledge?– Qualification of PI and quality of proposal?– Creative and original concepts? – How well conceived and organized is this activity?– Sufficient resources available for this research?

• Broader impacts– Advance discovery and understanding– Can promote teaching and research integration– Diversity (gender, ethnicity, disability, geographical), if any– Infrastructure development– Dissemination of information obtained – What is the benefit to society?

ALL CRITERIA MAY NOT APPLY FOR EACH PROPOSAL!Focus more on the contents (summary, description) than the format!

Panel meeting on December 2, 2004

• I will forward all reviews of each proposal to one of your group members (not you).

• That person (the panel member) will prepare panel summary with general topic, positive and negative aspects and final rating of the proposal.

• On December 2, we will meet in the atrium area outside G002.

• There will be seven tables, one per group• Each proposal will get seven minutes discussion (a bell

will ring every seven minutes)• You will go to other table when your own proposal is

being discussed. • The report will go in my file and you will select a leader.

PHYSICISTS

ENV I

ENGINEERS

MGB

ENV II

CPJ

FRES II

FRES I

Timeline for your 2004 FW5850 proposals

• Final proposals submitted on or before November 4th• Your reviews of other people’s proposals are due on or

before Nov 18th (send me an email)– Reviews should be critical and anonymous– Rate each proposal

• Outstanding, Very good, Good, fair, poor, not competitive

• Remember there is a thanksgiving break week in between Nov 20th-28th

• Summary for one proposal prepared by December 2nd • Panel meeting on December 2nd • Final presentations December 9th

Panel presentation

• You will select one leader per group who will read all reports in December 9th class

• Each proposal will get one-two minutes

• Each report will have a specific rating.

• We will have a pizza and drinks at the end!

• Enjoy your Xmas vacation!

Any questions?

Tidbits

• Only about 5 students had perfect budget worksheets in excel formats– 50K and 80K included full salaries– Used too high stipend for MS and Ph. D– Forgot to put fees for GRAs– Too high project cost and travel cost– Did not type 56% twice in the budget sheet F & A

costs– Did not include 5% raise in salaries per year which is

Ok for this mock proposal

Tidbits continued

• While evaluating proposals, focus on real summary and 15 pages of description only than other practice stuff

• Write constructive criticism, no personal attacks please

• Do not discuss your evaluation with others• Send me your reviews by email• Should we go for December 4th proposal

Jamboree?• Next class bring No. 2 pencil. You will evaluate

this class in last 15 minutes!