Upload
ishmael-callahan
View
32
Download
1
Embed Size (px)
DESCRIPTION
Presentation for the Advisory Committee for Business & Operations. Effective Practices Research Overview For Merit Review. Arlington, VA March 31, 2004. This document is confidential and is intended solely for the use and information of the client to whom it is addressed. - PowerPoint PPT Presentation
Citation preview
Arlington, VAMarch 31, 2004
Presentation for theAdvisory Committee for Business & Operations
Effective Practices Research Overview For Merit Review
This document is confidential and is intended solely for the use and information of the client to whom it is addressed.
NSF BUSINESS ANALYSIS
2
NSF BUSINESS ANALYSIS
Today, we will provide…
Brief update on the status of the NSF Business Analysis
Summary of the key findings from the Effective Practices research for Merit Review
3
NSF BUSINESS ANALYSIS
NSF Business Analysis has recently competed the following activities and is currently moving into the design phase of the project
Completed as of 3/31/04Completed as of 3/31/04
Completed researching effective practices for the Merit Review and Award Management & Oversight processes
Completed workload study that assessed the amount and distribution of workload across NSF
Completed first full draft of the Target Enterprise Architecture for Applications, Data, Network and Security
Completed researching effective practices for the Merit Review and Award Management & Oversight processes
Completed workload study that assessed the amount and distribution of workload across NSF
Completed first full draft of the Target Enterprise Architecture for Applications, Data, Network and Security
“Works in Progress”“Works in Progress”
Developing process scenarios and preliminary business cases for the Merit Review and Award Management & Oversight processes
Conducting an eJacket study to determine how the electronic-jacket technology tool impacts human capital issues
Developing an IT Technology Governance Framework and IT Implementation Plan that considers the 2-7 year transition plan necessary to move to the Target Enterprise Architecture
Developing process scenarios and preliminary business cases for the Merit Review and Award Management & Oversight processes
Conducting an eJacket study to determine how the electronic-jacket technology tool impacts human capital issues
Developing an IT Technology Governance Framework and IT Implementation Plan that considers the 2-7 year transition plan necessary to move to the Target Enterprise Architecture
Quick Snapshot of Project Status
4
NSF BUSINESS ANALYSIS
For the “effective” practices research, the team researched the practices of grant-making organizations in order to identify potential alternative practices
Research Objectives:
To understand how other federal and non-federal organizations review proposals and manage awards, and the relative emphasis that is placed on either activity
To provide NSF with insights into how other organizations address challenges similar to those faced by the Foundation in the proposal review and award management processes
To identify practices or principles that could be leveraged or modified by NSF in the redesign of its core business processes
5
NSF BUSINESS ANALYSIS
The team interviewed representatives of 14 federal and non-federal organizations
Organizations Interviewed
6
NSF BUSINESS ANALYSIS
Important to note is that few effective practices were identified in the areas of performance measurement, “working across the organization,” and technology usage
Few organizations apply performance metrics to their review or award management practices
– Customer satisfaction surveys are performed ‘ad hoc’
– Processing times tend to be the same or longer than NSF
– Few agencies have monitoring plans in place to address the degree of award oversight that is applied
Few organizations had a significant need for ‘working across the organization’
Many agencies were either “developing” or “progressing” in their eBusiness capabilities with availability of resources cited as the greatest barrier to development efforts
7
NSF BUSINESS ANALYSIS
Effective practices emerged in each of the following areas
Process for how an organization receives, processes, and streamlines activities associated with proposal intake, review, and decision-making
Reviewer Community
Management
Proposal Workflow
Management
Review Methodologies
Description
Various practices an organization applies to review proposals and make funding decisions with limited resources
Management practices that an organization applies to attract, recruit, and reward its reviewers, as well as how the organization maintains a relationship with its community
Merit Review Areas
8
NSF BUSINESS ANALYSIS
Reviewer Community
Management
Proposal Workflow Management
Review Methodologies
Strategy Description
Apply “filters” to reduce the number of proposals that receive reviews and to manage future workload requirements
Leveraging Letters of Intent (LOI), pre-applications or other filters that enable organizations to better manage workload
Identifying early in the process proposals that are noncompetitive or inappropriate, and begin panel coordination earlier in the review process
Coordinate the scheduling of program deadlines to spread the workload more evenly across the fiscal year
For organizations with multiple programs, centrally coordinating the deadlines for programs to reduce the burden on both internal resources and the reviewer community
Employ temporary staff during peak periods or outsource some administrative functions to mitigate workloads
Using temporary employees to provide staff with administrative relief during periods of peak workload. Services range from minor administrative support (i.e., jacket assembly) to outsourcing all proposal support (i.e., panel coordination, soliciting reviewers, conflicts of interest, etc.)
Strategy Description
Apply “filters” to reduce the number of proposals that receive reviews and to manage future workload requirements
Leveraging Letters of Intent (LOI), pre-applications or other filters that enable organizations to better manage workload
Identifying early in the process proposals that are noncompetitive or inappropriate, and begin panel coordination earlier in the review process
Coordinate the scheduling of program deadlines to spread the workload more evenly across the fiscal year
For organizations with multiple programs, centrally coordinating the deadlines for programs to reduce the burden on both internal resources and the reviewer community
Employ temporary staff during peak periods or outsource some administrative functions to mitigate workloads
Using temporary employees to provide staff with administrative relief during periods of peak workload. Services range from minor administrative support (i.e., jacket assembly) to outsourcing all proposal support (i.e., panel coordination, soliciting reviewers, conflicts of interest, etc.)
Many organizations employ different strategies to mitigate workload demands
9
NSF BUSINESS ANALYSIS
Reviewer Community
Management
Proposal Workflow Management
Review Methodologies
Strategy Description
Structure decision-focused review process; allow reviewers to spend more time on competitive proposals
Posting reviews prior to panel or pre-ranking proposals based on reviewer feedback, then focusing panels on those proposals that require additional feedback upon which to base a decision (e.g., proposals “on the fence”)
Reduce degree of “budget” negotiations that must occur throughout the post-review process
Employing strategies to mitigate the degree of post-review budget discussions include:
– Embedding “cost realism” as a review criteria
– Adjusting budgets and issuing ‘conditional awards’
– Employing policies of non-negotiable awards
Enable reviewers to participate remotely in panels through the use of technology
Providing tools that: facilitate involvement of panelists who are unable to physically be present through remote participation; allow NSF to utilize expert reviewers for the appropriate panels; and, broaden the accessible reviewer pool
Strategy Description
Structure decision-focused review process; allow reviewers to spend more time on competitive proposals
Posting reviews prior to panel or pre-ranking proposals based on reviewer feedback, then focusing panels on those proposals that require additional feedback upon which to base a decision (e.g., proposals “on the fence”)
Reduce degree of “budget” negotiations that must occur throughout the post-review process
Employing strategies to mitigate the degree of post-review budget discussions include:
– Embedding “cost realism” as a review criteria
– Adjusting budgets and issuing ‘conditional awards’
– Employing policies of non-negotiable awards
Enable reviewers to participate remotely in panels through the use of technology
Providing tools that: facilitate involvement of panelists who are unable to physically be present through remote participation; allow NSF to utilize expert reviewers for the appropriate panels; and, broaden the accessible reviewer pool
Most organizations structure the review process to optimize the time of reviewers
10
NSF BUSINESS ANALYSIS
Reviewer Community
Management
Proposal Workflow Management
Review Methodologies
Most organizations treat reviewers as “customers” throughout the process
Strategy Description
Treat reviewers as “customers” throughout the review process
Coordinating all panel logistics including travel arrangements
Providing social/networking opportunities while on-site
Pay them well Offering attractive honorariums, paying ‘ad hoc’ reviews
Educate reviewers on their duties and responsibilities
Providing training to new panelists. Training could include on-line tutorials, mentoring, or “mock reviews”
Structure “terms of service” to ensure continuity
Requiring panelists to serve for a period of time -- generally for a 3-year term and not necessarily consecutive years
Reach out to the full ‘pool’ of possible reviewer candidates
Providing targeted marketing and outreach efforts to potential reviewers (i.e., soliciting universities, societies, etc.)
Integrating of databases for applicants, reviewers, and awardees
Accepting on-line applications
Strategy Description
Treat reviewers as “customers” throughout the review process
Coordinating all panel logistics including travel arrangements
Providing social/networking opportunities while on-site
Pay them well Offering attractive honorariums, paying ‘ad hoc’ reviews
Educate reviewers on their duties and responsibilities
Providing training to new panelists. Training could include on-line tutorials, mentoring, or “mock reviews”
Structure “terms of service” to ensure continuity
Requiring panelists to serve for a period of time -- generally for a 3-year term and not necessarily consecutive years
Reach out to the full ‘pool’ of possible reviewer candidates
Providing targeted marketing and outreach efforts to potential reviewers (i.e., soliciting universities, societies, etc.)
Integrating of databases for applicants, reviewers, and awardees
Accepting on-line applications
11
NSF BUSINESS ANALYSIS
Based on the findings in the effective practices research, NSF should consider the following in the scenario design
Broadening the use of Letters of Intent (LOIs) for determining program fit and begin planning and coordinating panels earlier in the process
More pro-actively coordinating program deadlines taking into account the needs of the research community
Developing piloting opportunities to broaden the use of temporary help during peak proposal times or outsourcing certain process functions
Incorporating process changes that reduce the degree of proposal budget rework that occurs throughout the Merit Review process
Compensating off-site reviews, broadening the concept of panel “terms of service”, and providing training for how best to perform a review
Developing strategies that “reach out” to a greater pool of reviewers (e.g., integrating databases for applicants, awardees, reviewers; enabling better remote access; accepting on-line reviewer applications, etc.)
12
NSF BUSINESS ANALYSIS
Questions & Answers
For additional questions, please contact:
– Abe Zwany, [email protected], 703.902.5342
– Tim Koch, [email protected], 703.377.0389