Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Office of the Science AdvisorOffice of the Science Advisor
Peer Review2006 Peer Review Policy and
3rd edition of the PR Handbook
Barbara KlieforthOffice of the Science Advisor, ORD
2
Long History of Peer Review at EPA
Long History of Peer Review at EPA
1993 Peer Review Policy1998 1st Guidance Handbook2003 epa.gov/si Science Inventory2006 New Policy and 3rd edition Handbook;
http://epa.gov/osa/spc/peerreview/
3
2004 OMB Peer Review BulletinKey Features
• Highly influential scientific assessments, influential scientific information
• Peer review expectations• Public comment on peer review plans• Annual reports to OMB
4
Scientific Information
Factual inputs, data, models, analyses, technical information, or scientific assessments
5
Influential Scientific Information (ISI)
Will have or does have a clear and substantial impact on important public policies or private sector decisions
Same definition as used in the Information Quality Guidelines• Major per the old Peer Review Handbook• Supports top Agency actions (including economically significant
actions)
6
Scientific AssessmentScientific AssessmentEvaluation of a body of scientific or technical
knowledge, which typically synthesizes multiple factual inputs, data, models, assumptions, and/or applies best professional judgment to bridge uncertainties.Includes: state-of-the-science reports, technology assessments, weight-of-evidence analyses; meta-analyses; health, safety, or ecological risk assessments; toxicological characterizations of substances; integrated assessment models; hazard determinations; or exposure assessments
7
Highly Influential Scientific Assessment (HISA)
Highly Influential Scientific Assessment (HISA)
• Could have a potential impact of more than $500M in any year
• Is novel, controversial, or precedent-setting, or has significant interagency interestOverlaps with definition of old “major”
8
All EPA work products vs those considered ISIs or HISAs
All EPA work products vs those considered ISIs or HISAs
All work products EPA may generate or use
Influential Scientific Information
Highly Influential Scientific Assessments
9
Peer Review Expectations for Other Information
Peer Review Expectations for Other Information
• Most products in Science Inventory are not ISI or HISA
• Should determine if product would benefit from peer review and, if so, what type of review to conduct
• Should also consider peer input
ID work product & put in SI
Planning a Peer ReviewPlanning a Peer ReviewPlanning a Peer Review
Is it ISI or HISA?
Would it still warrant PR?
Is PR exemptor waived?
Does it consist only of science previously PR’ed?
Peer Review needed:•ID charge•ID key staff•Create PR record•Ensure funding•Determine time frame
PR Not Needed
no
yes
no
yes
no
no
Conducting a Peer ReviewConducting a Peer Review
Complete PR & Obtain comments
Work ProductReady for review
Develop Charge
Select Mechanism
Set Timelines
Select Peer Reviewers
Send Materials &Conduct Review
(Add all to PR Record)
Document how used &Put in PR record
Receive Comments from Peer Reviewers
Completing a Peer ReviewCompleting a Peer Review
Brief Decision-Maker, if appropriate
Are PR comments usedin Work Product?
Document why not &Put in PR record
Brief Decision-Maker on approach to handling
comments
Finalize Work Product
Update PR record
DONE!
yes
no
13
Science Inventory/Peer Reviews
Science Inventory/Peer Reviews
• Science Inventory• 1995: Approximately 100 products• 2005: >5,000 activities and products
• Peer Reviews in 2004• Over 2,000 (includes journal articles)• Estimated annual cost: $10M
14
Contents of Peer Review PlansContents of Peer Review Plans• Title/subject/purpose, EPA contact, if HISA or ISI• Timing of the review (including deferrals)• If panel or letter review (or through alternative procedure)• If and how public will be asked to comment on work
product• If EPA will provide public comments to peer reviewers• # of reviewers (3 or fewer, 4-10, >10)• Description of primary disciplines or expertise needed• Whether reviewers will be selected by EPA or outside
organization• If the public (including scientific/professional societies) will
be asked to nominate reviewers
15
Roles/ResponsibilitiesRoles/Responsibilities• AAs/RAs• Science Policy Council• ORD/OSP• Peer Review Coordinators• Peer Review Leaders
16
AA/RAAA/RA
• Is ultimately responsible for implementing policy• Determines ISI/HISA• Identifies other products needing peer review• Makes peer review decisions• Allocates resources for peer review
• Ensures that her/his office’s science activities and products are captured in the Science Inventory
17
Science Policy Council (SPC)Science Policy Council (SPC)• Develops EPA science policy and provides
oversight
• Peer Review Advisory Group• Assists the SPC with implementing policy
Agency-wide• Responsible for revising Peer Review Handbook
18
Peer Review CoordinatorPeer Review Coordinator• Serves as the main peer review contact for
his/her organization• Provides training on peer review process and
roles/responsibilities• Coordinates the entry and update of information in the
Science Inventory• Serves as a liaison with ORD and SPC• Assists with preparing Annual Report to OMB
19
Peer Review LeaderPeer Review LeaderOrganizes/oversees peer review for a specific work
product• Selects peer review mechanism• Develops charge• Considers public comments if HISA/ISI• Documents the peer review process and results• Responds to findings• Maintains the peer review records• Works with the Coordinator on the Science
Inventory
20
In SummaryISI – What is new?
In SummaryISI – What is new?
• Expanded definition of peer review products• Offer and consider public comment on peer review
plans• Consider asking public to nominate potential peer
reviewers• Preparing Peer Review Report and making it (and
related materials) available on the internet• Discussion in preamble and certification in the
administrative record
21
Major Products(per old Peer Review Handbook)
Major Products(per old Peer Review Handbook)
• Establishes a significant precedent, model or methodology
• Addresses significant controversial issues• Focuses on significant emerging issues• Has significant cross-Agency/inter-agency implications• Involves significant investment of Agency resources• Considers an innovative approach for a previously
defined problem/process/methodology• Satisfies a statutory or other legal mandate for peer
review
22
HISA- What is new?HISA- What is new?In addition to those listed for ISI:• When feasible, soliciting public comment on
work product and allowing public to make presentation to peer reviewers
• Providing peer reviewers with significant public comments that pertain to science issues
• Preparing and posting Agency response to peer review report