FUNDAMENTAL RESEARCH GRANT APPLICATION & EVALUATION Muhammad Fauzi Mohd Zain Fakulti Kejuruteraan dan Alam Bina, UKM 18 Feb 2014

FUNDAMENTAL RESEARCH GRANT

APPLICATION & EVALUATION

Muhammad Fauzi Mohd ZainFakulti Kejuruteraan dan Alam Bina, UKM18 Feb 2014

1 Introduction

1 Introduction

Program Penyelidikan Fundamental

2013

Single Disciplinary Project

-permohonan projek penyelidikan melibatkan satu (1) bidang penyelidikan.-siling peruntukan adalah RM250,000.00 sepanjang tempoh penyelidikan.

Trans Disciplinary Project

- permohonan projek penyelidikan melibatkan sekurang-kurangnya tiga (3) bidang penyelidikan (trans disciplinary) melangkaui jabatan / fakulti dari institusi yang sama.- sasaran hasil penyelidikan yang sama.- siling peruntukan adalah RM1,500,000.00 sepanjang tempoh penyelidikan.

Research Acculturation Grant Scheme (RAGS)

-dana tunas yang bertujuan untuk membudayakan penyelidikan di kalangan penyelidik muda di IPTA bukan RU sebagai persediaan untuk membangunkan prestasi penyelidikan supaya dapat berdaya saing di peringkat kebangsaan dan antarabangsa.

Research Acculturation Collaborative Effort (RACE)

Membantu Non-RU meningkatkan budaya penyelidikan dan seterusnya dapat meningkatkan output hasil penyelidikan. Usaha ini dapat mempercepatkan Non-RU untuk mencapai tahap setanding dengan RU yang lain.

1 Introduction

BIL PERKARA

FRGS

LRGS ERGS PRGSSingle Disciplinary

Project

Trans Disciplinary

ProjectRAGS

1 Siling Permohonan RM250,000 RM1,500,000 RM50,000 –

RM80,000RM3

juta/tahun RM300,000 RM500,000

2 Tempoh Penyelidikan

1 hingga 3 tahun

1 hingga 3 tahun

1 hingga 2 tahun

3 hingga 5 tahun 3 tahun 2 tahun

3 KPI • 1 PhD• 3 papers in index link journal (2 years)

• 10 PhD (3 years) • 50 papers (3 years)• 3 IP (per program) - number of researchers with Citation Index of 100)

• 1 PhD• 3 papers in index journal• 1 IP (filed)

•1 IP/project

1 Introduction

Penyelidik

RMC IPT

Panel Penilai KPM

Jawatankuasa Induk FRGS

Peringkat IPT

Peringkat Nasional

2 ahli panel membuat penaziran

RMC membuat proses saringan terperinci dengan bantuan pakar dalaman/luaran

60 pakar

Meluluskan polisi, halatuju dan dana

Panel Penilai IPT

Jawatankuasa Penilaian FRGS KPM

1 IntroductionCARTA ALIR PROSES KERJA RMC DI PERINGKAT CARTA ALIR PROSES KERJA RMC DI PERINGKAT

UNIVERSITI UNIVERSITI

1 IntroductionCARTA ALIR PROSES KERJA RMC DI PERINGKAT CARTA ALIR PROSES KERJA RMC DI PERINGKAT

UNIVERSITI UNIVERSITI

1 IntroductionSENARAI DOKUMEN YANG PERLU DISEDIAKAN SENARAI DOKUMEN YANG PERLU DISEDIAKAN

1 Introduction

Fundamental Research (FRGS) Basic research Pure research

Fundamental research generates new knowledge and technologies to deal with unresolved problems.

1 Introduction

Exploratory Research (ERGS) Relies on secondary research To gather preliminary information

Research intended only to provide greater familiarity with the phenomena that researcher wants to investigate so that he can formulate more precise research questions and perhaps develop hypotheses. Such studies can be essential when researcher is investigating new phenomena or phenomena that have not been studied before.

1 Introduction Most common reasons for grant writers

(GWs) not receiving funds1. Not new or lack of original ideas2. Diffuse, superficial or unfocused research

plan3. Lack of knowledge of published relevant work4. Lack of experience in the essential

methodology5. Uncertainty concerning the future directions6. Questionable reasoning in experimental

approach7. Absence of acceptable scientific rationale8. Unrealistically large amount of work9. Insufficient experimental detail10.Uncritical approach

1 Introduction

Quality of the Proposal The measures for a good quality proposal

are:

Informative title; Convincing executive summary; Clear problem statement; Scientific background and rationale; Good selection of research methods; Ethical considerations; and Realistic budget and schedule.

1 Introduction

A document that is neat, well organized and easy to read;

Responsiveness to the program need, with specific references showing how the proposed project will achieve program goals and objectives;

Fresh insight into an important problem; Writing that communicates the enthusiasm and

commitment of the researcher; Evidence that the PI knows the field; Convincing preliminary data; and A feasible work plan that is supported by an

appropriate budget.

Characteristics of a good proposal:

2 Evaluation: Evaluator

Internal evaluation Department/Faculty/Institute level University level

Internal evaluators improve the quality of grant submissions

How to “improve”

2 Evaluation: Evaluator

External evaluation Sponsor level Many constraints

Number of proposals, amount of money available, etc.

To find mistakes/weaknesses, etc.

To find ways on…

How to “reject”

2 Evaluation: Evaluator Selecting An Evaluator

When selecting an evaluator for research grant project, the following criteria may be useful to consider:

Credentials/Reputation- To what extent has the individual evaluated research grants, particularly fundamental focused researches? Is the evaluator affiliated with an academic institution?

Education- To what extent the individual trained in evaluation? Does the evaluator have a certificate, or doctoral degree related to evaluation?

Experience- To what extent does the individual have formal or informal experience with evaluation in fundamental research grant? Can the evaluator provide samples of evaluation reports that he or she has completed?

Sensitivity- To what extent does the evaluator have experience working with the target population?

Integrity- To what extent is the organization familiar with the evaluator? Does the evaluator present any conflicts of interest with evaluating the project?

2 Evaluation: Evaluator Selecting An Evaluator

When selecting an evaluator for research grant project, the following criteria may be useful to consider:

Communication Skills- To what extent is the evaluator able to explain technical concepts in understandable language and demonstrate clear verbal and written expression?

Availability- To what extent is the evaluator available to meet with the project timelines and be flexible if timelines need to be modified?

Cost- Are the proposed cost for the evaluation reasonable for the required tasks?

Contract/Scope of Service- Is the evaluator willing to design a scope of service/contract or agreement outlining his/her responsibilities along with timelines?

2 Evaluation: Evaluator Pros and Cons of using External Evaluators

Pros Cons

Less work for your organization

Evaluators professional expertise

Evaluators bring objectivity

Evaluation results may have more credibility

Less control over the process

Staff may have more complete understanding on the program

Less opportunity to develop internal evaluation capacity

Expensive

2 Evaluation: Evaluator Conflict of Interest

Evaluators are required to declare any personal interests according to the following criteria. Evaluators must disqualify themselves if they can in any way benefit from the approval or rejection of the proposal. They must also disqualify themselves in the following circumstances: Evaluators have close collaboration with the GW (e.g. have co-authored and published an article with the GW during the past three years, have been involved in the preparation of the application, or are involved in the publication or application of the results) Evaluators have been a superior, subordinate or instructor of the GW during the past three years Evaluators are currently applying for the same post as the GW Evaluators are currently applying for funding from the same funding instrument on the same research area.The GW is a close person to evaluator.

3 Evaluation: Process Research Proposal VS Research

evaluationModule of FRGS/ERGS Proposal

TitleDetails of ResearcherResearch InformationExecutive SummaryResearch Background

Problem StatementHypothesesLiterature Review

Research ObjectivesMethodology/Research DesignTimeline/ScheduleExpected ResultsFacilities and Special ResourcesBudgetResume/Brief CVAppendices

What Ext. Evaluators Look For:

Title (1)Details of ResearcherResearch InformationExecutive Summary (2)Research Background

Problem Statement (3)HypothesesLiterature Review

Research Objectives (4)Methodology/Research Design (5)Timeline/ScheduleExpected Results (6)Facilities and Special ResourcesBudget (7)Resume/Brief CVAppendices

Fundamental Criteria of Evaluation

CARE: Are GWs tackling an important problem? If they can make progress on it, will anyone care?

NOW: Why now? If this problem is so important, why has it not been addressed before?

IDEAS: Do GWs have concrete ideas for starting an attack on the problem and a vision for proceeding further? Is initial progress likely and subsequent progress possible?

RESULTS: Do GWs have some preliminary results? Do they demonstrate a good understanding of the problem and the methods needed attack it further?

PLAN: Do GWs have sensible plans and methods (e.g., concrete steps and ways of decoupling risks)?

CAN-DO: Why these GWs? Why are their qualifications and infrastructure appropriate?

LEGAL: Have GWs followed the rules of the solicitation (e.g., compelling broader impacts for FRGS/ERGS)?

.

3 Evaluation: Process

Main Criteria of Evaluation All proposals must be evaluated based on the criteria outlined below.

Relevance Degree to which the proposal was related to KPM (F/E/L/PRGS) Degree to which the proposed research results would be

relevant to policy-makers Extent and appropriateness of dissemination plans Degree to which the proposed research would facilitate the goals

of KPM Degree to which the proposed research represented an innovative

approach and develop new knowledge in the field of engineering or other disciplines

Research Team Quality of the research team and their research The applicant and the research team are among the leading in their

field. The publications are at a remarkable international level. The articles are published in the best peer-reviewed journals, or proceedings, which are indexed in the leading databases of the field.

The impact of the applicant (number of citation; the scientific level of the journals, where the articles are published) is, in the respective field, at a remarkable international level)


Main Criteria of Evaluation

All proposals must be evaluated based on the criteria outlined below. Proposal Quality

Degree to which the research approach provided results that can be generalized beyond the unit of analysis and were applicable to multiple settings or groups

Degree to which proposal demonstrated general criteria of excellence

Innovation Quality of the research objectives and linkage to

literature review Clarity of the research questions Appropriateness of methodology Quality/clarity and detail of proposed research method etc


Main Criteria of Evaluation (detail)Evaluating what worked and what didn't will be crucial for grant sponsor and for GWs. What impact do evaluators expect to achieve and how will they evaluate it? Before evaluators design their evaluation, Carlson and O'Neal-McElrath, authors of Winning Grants, Step by Step, suggest that evaluations can accomplish these 6 purposes: To find out if the hypothesis was right. Did GWs actually do what they set out to do? To determine if the methods specified were used, and if the objectives were met. To find out if an impact was made on the identified need. To obtain feedback from the people served and other members of the community. To maintain control over the project (evaluations are done at various points in the project). To make changes in the program mid-stream, if necessary, to insure the program's success.


Determine if GWs will use quantitative or qualitative methods for their data collection, or what combination of the two types they will use. Develop a good description of these methods and their rationale for the grantor.

Make sure the evaluation component of GWs proposal connects with the proposal's objectives and methods. If those objectives and methods are measurable and time-specific, the evaluation will be easier to design.

Evaluators must ask these questions as they develop the evaluation section of GWs proposal: What is the evaluation's purpose? How will GWs use the findings? What will GWs know after the evaluation that they

didn't know before? What will GWs do as a result of the evaluation that

they couldn't do before because they lacked the relevant information?

How will the clients and community served be better as a result of the program?


Main Criteria of Evaluation (detail)The following criteria will be used to evaluate proposals:Quality of the proposal (i.e., significance, relevance of methodology, importance of findings, value to larger academic community, etc). Proposals should be framed in a manner easily understood by someone outside your discipline. Proposals must meet high professional standards in presentation, including spelling, grammar, and proper budgeting.Potential for peer-review publication, presentation, exhibition, etc.Budgetary appropriateness.Potential for disseminating research to broader academic community.Value for professional development of faculty member.


Main Criteria of Evaluation (detail)Review questions include, but are not limited to:

Does the proposal communicate the importance of the work?Is the importance of the project within its field made clear?Will successful completion of the project have an impact upon the field?Is the project significant to the development of a program of scholarly activity by the lead GW?Are the objectives clearly defined, and is the basic question to be answered clearly identified?Are all necessary facilities available?


Main Criteria of Evaluation (detail)Review questions include, but are not limited to:

Does the lead GW have the necessary background and expertise?Are project activities well planned, and do they realistically fall within an appropriate timetable?Have all items requested been justified?Is the amount requested reasonable and consistent with the total funding available to this grant program?If equipment is requested, has the possibility that it is already available elsewhere on campus been addressed?If support for students is requested, is it clear that their activities are essential to the research program?


External evaluation Sponsor level Evaluators focus on the Four Cs

Clarity. How GWs do cross-reference current literature in laying out their premises.

Content. How GWs organize their ideas around aims linked to their hypothesis.

Coherence of concepts. How GWs present coherent set of ideas predicated by previous work.

Cutting edge. Are GWs ready to take legitimate risks.


4 Evaluation: New Approach

5 Evaluation: Title

A good title should:

Indicate the type of study. Address the main problem. Be concise, short, and descriptive. Convey to the evaluator the main focus of the research. Use the correct terms in the title.Should be intelligible to non-specialists.Limit the title to a single sentence.Relevant in 2 years time?

5 Evaluation: Title

Selection of research topic should be based on…..

Magnitude of the problem and its impactUrgency of the need for a solution.Relevance to the aim of the funding agency.Amenability of the problem to investigation.Feasibility of the approach.The proposed research topic is very timely and relevant both internationally and locally. The prospective results make a substantial contribution to the development of science,

technology, and/or society

6 Evaluation: Executive Summary

An informative abstract, giving evaluators the chance to grasp the essentials of the proposal without having to read the details

GWs must present their project ConciselyState significance ClearlyState Hypotheses, Research Problem, SolutionMethods and RationaleExpected output.

7 Evaluation: Research Background

Research Background (RB)

1.Title2.Problem statements3.Objectives

Flows naturally from Title, Problem Statement to Research Objectives


The RB serves several important functions: Evaluators must ensure that…

GWs are not "reinventing the wheel". GWs demonstrate their knowledge of the research problem. GWs demonstrate their understanding of the theoretical and research issues related to their research question. GWs show their ability to critically evaluate relevant literature information. GWs indicate their ability to integrate and synthesize the existing literature. GWs provide new theoretical insights or develops a new model as the conceptual framework for their research. The proposed research will make a significant and substantial contribution to the literature (i.e., resolving an important theoretical issue or filling a major gap in the literature).


Problem Statements The most important aspect of a research proposal is

the clarity of the research problem The problem statement is the focal point of the

research Evaluators must ensure that…

GWs give a short summary of the research problem that have been identified.

The research proposal may not acceptable or credible if GWs not clearly identify the problem.

GWs present the persuasive arguments as to why the problem is important enough to study or include the opinions of others (politicians, futurists, other professionals)


References-Most resent

Up-to-dateHighly relevant with the problemOriginal source

First Order : High Impact Journals and Books

Second Order : Indexed Proceeding Publications

Third Order : Reputable Technical Report

8 Evaluation: Objectives

Objectives specify the outcome of the project, the end product(s). GWs must state the objectives clearly and keep them “S-M-A-R-T” or “S-I-M-P-L-E.“ Specific -what GWs intend to change through their project.Immediate -time frame during which a current problem will be addressed.Measurable -what GWs would accept as proof of project

success.Practical -how each objective is a real solution to a real

problem.Logical -how each objective systematically contributes to achieving GWs overall goal(s).Evaluable -how much change has to occur for the project to be effective.

9 Evaluation: Methodology GWs state it explicitly GWs give an overall summary of the research design

and methodological approach. GWs provide the methodology for each specific

objective. GWs describe

the specific design (what will they do and how, number of replicates, etc.),

the materials and techniques that will be used, and

the feasibility of these techniques.use literature to support design, materials &

techniques

Walk the reader through GW projectDescribe the activities as they relate to the

objectivesDevelop a time line and/or and organizational

chartHow will the activities be conducted?When?How long?Who?Where?What facilities?

Basically, GWs must provide answers to the following questions: What activities need to take place in order to meet the objectives? What are the start and finish dates for the activities? Who has responsibility for completing each activity? How will participants be selected? What factors determine the suitability of your methodology? Does this project build on models already in existence? if not, how is it superior? What facilities and equipment will be required to conduct the activities?

9 Evaluation: Methodology Milestones

The milestones are the results which The milestones are the results which the project seeks to achieve. the project seeks to achieve. The milestones should, as much as The milestones should, as much as possible possible relate to ‘tangible productsrelate to ‘tangible products’ ’ (quantifiable, qualitative or verifiable) (quantifiable, qualitative or verifiable) from conduct of the research. from conduct of the research. They They indicate viable achievementsindicate viable achievements..

9 Evaluation: Methodology

Gantt’s Chart /Flow Chart

GWs must clearly show the research activities and milestones (•/M)Reflection of the project objectives,

methodologies, outputs, etc. Very important

10 Evaluation: Expect Output

Scientific OutcomesNew methodologyPatentsPublications (High Impact Journal)Discoveries

Socio-Economic outcomes/impactBetterment of SocietyDevelopment of the EconomyImproving Livelihood of People

What are the expected outcomes and what do GWs wish to achieve, e.g.:A new theoryA prototypeA new modelAn artefactA new plant processA solution to a practical problemA specific aid to practitioners in a particular fieldAn instrument of use in the manufacturing industry, etc.What contribution will this research make to the body of knowledge in the particular field of study?The F/EGRS Evaluation Guidelines specifically require that the expected outcomes be clearly defined, as well as the likelihood that the research will achieve the expected results within the stated timeframe.

11 Evaluation: Track Record

Track Record (CV) Experience, Qualifications and Availability of Research Team

This section should begin with the principal investigator, and then provide similar information on all individuals involved with the project. Two elements are critical:Professional research competence (relevant research experience, the highest academic degree held, and technical societies).Relevant management experience (if any).

12 Evaluation: Quality of Proposal

Style:Use most recent formFollow guidelines (font, size, margins, etc.)Spell check, correct grammarHighlight signposts (italic, bold, underlining)One main idea per paragraphUse topic sentencesUse transitions (e.g., in contrast, however, likewise, etc)Use graphics in methodology and needs sectionsEnd paragraph with closing sentence

Evaluator-friendly applicationGWs must give evaluators enough time !

12 Evaluation: Quality of Proposal

Novelty, Cutting Edge, High Impact

Does the research use novel techniques, tools, and procedures?Is new data required?Is data gathered in a new way?Is existing data utilised in a new way?Can an existing application be used in a new way?Is the proposed research potentially patentable and publishable?

13 Evaluation: Policy and Support

Budget (Grants Fail…)

Problem: The budget exceeds the available amount The budgeted items are not reasonable for the

work proposed The cost of the program is greater than benefit The budget justification is not included There is a mismatch between activities and

budget The budget justification does not clearly explain

the budget item Inappropriate use of funds

Solution: Outline budget items carefully and use standard amounts for expenses


Budget: GWs must ensure that…

They present the budget based on the sponsor requests. (Read Guideline)The budget must be reasonable, acceptable, and appropriate (GWs must not inflate…)They must follow strictly the GuidelinesDetail justifications on each item must be provided (Vote 35000)

Itemized BudgetBudget Narrative


Infrastructure/Facilities

Use whatever available in campus (related to proposed project)Reduce to a minimum any call upon outside facilities and expertiseThe requirements of infra will vary from study to study. GWs must carefully list the relevant facilities and resources that will be used. The costs for such facility use should be

detailed in GWs budget.

14 ConclusionMake Life Easy for Evaluators

Evaluators are knowledgeable, experienced scientists, but they can’t know everything.

Problem: evaluators may not get the significance of the proposed research.

Solution: GWs write a compelling argument. Problem: evaluators may not be familiar with all the

research methods. Solution: GWs write to the non-expert in the field. Problem: evaluators may not be familiar with the research

lab. Solution: GWs show to evaluators that they can do the

job. Problem: evaluators may get worn out by having to read

10 to 20 applications in detail. Solution: GWs write clearly and concisely, and make sure

the application is neat, well organized, and visually appealing.

14 Conclusion

Ethical Statement

Researchers undertaking any form of fundamental research using animals or people have to submit a proposal to either the animal ethics committee or the human ethics committee for approval before the data gathering can begin.

14 Conclusion Joel Orosz, of the W.K. Kellogg Foundation,

has sagely observed that there are really only four kinds of proposals

Bad idea, bad proposalBad idea, good proposalGood idea, bad proposalGood idea, good proposal

14 Conclusion

Good Proposal Grant Proposal (Failed/Success)-Sample

Good Ideas Good GrantsmanshipGood Presentation Good ReviewGood Luck

15 Bibliography

Kevin C. Chung, MD, Melissa J. ShauverCheryl Anne Boyce, Ph.DXander HT Wehrens, M.D. Ph.DGitlin, L. N., Lyons, K. J.Simon Peyton JonesBaharuddin Salleh

16 Ketua Panel Kluster Geran KPT1. Sains Tulin - Prof. Emeritus Dato' Dr. Muhamad bin Yahaya, UKM2. Sains Gunaan - Prof Emeritus Dato' Dr Md Ikram Bin Mohd Said,

UKM3. Sains Sosial - Prof. Dr. Samsudin bin A. Rahim, UKM4. Sains Tabii dan Warisan Negara - Prof. Dato’ Dr. Nik Muhamad bin

Nik Ab. Majid, UPM5. Sains Kesihatan dan Klinikal - Prof Dato‘ Dr. Amin bin Jalaluddin, UM6. Sastera dan Sastera Iktisas - Dato‘ Prof. Salleh bin Yaapar, USM7. ICT - Prof Dr. Ku Ruhana binti Ku Mahamud, UUM8. Teknologi dan Kejuruteraan - Prof Dr Muhammad Fauzi Mohd Zain,

UKM

Thank you

Documents

FUNDAMENTAL RESEARCH GRANT APPLICATION & EVALUATION Muhammad Fauzi Mohd Zain Fakulti Kejuruteraan dan Alam Bina, UKM 18 Feb 2014