Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
OFFICE OF ADOLESCENT HEALTH (OAH) TEEN PREGNANCY PREVENTION (TPP)
PROGRAM
SIX-MONTH PROGRESS REPORT GUIDANCE
Last Updated November 2016
Six-Month Progress Report Due: February 1st
2
OAH TPP PROGRAM SIX-MONTH PROGRESS REPORT GUIDANCE
Table of Contents
PART ONE: GENERAL INSTRUCTIONS 3
PART TWO: PROGRESS REPORT SUBMISSION 3
PART THREE: PROGRESS REPORT CONTENT 3 I. Six-Month Progress Update 4 II. Performance Measures 6 III. Appendices 6 IV. Evaluation Reporting (for ALL TPP Grantees except Tier 2B) 6 V. Evaluation Reporting (Tier 2B Grantees Only) 6
PART FOUR: ADDITIONAL INFORMATION AND SAMPLE TEMPLATES 8
EXHIBIT A: Six-Month Progress Update Templates 9
EXHIBIT B: Description of OAH Grant Expectations 19
EXHIBIT C: MAX Filing Guidance for TPP Grantees – Year Two 24
EXHIBIT D: CONSORT Diagram Instructions, Templates, and an 27 Example (TPP Tier 2B Grantees Only)
3
OAH TPP PROGRAM SIX-MONTH PROGRESS REPORT GUIDANCE
PART ONE: GENERAL INSTRUCTIONS
The purpose of the six-month progress report is to report on the progress of the OAH TPP grant project during the six month project period (July 1 – December 31), including the status of programmatic activities, evaluation activities, and submission of performance measure data.
The OAH TPP Program Six-Month Progress Report Guidance describes the content and submission procedures for completing the six-month progress report. Six-month progress reports will be reviewed by the OAH Project Officer and the Office of Grants Management (OGM) Grants Management Specialist.
PART TWO: PROGRESS REPORT SUBMISSION
The six-month progress report and all supporting documents must be received no later than 11 p.m. Eastern Standard Time on February 1st.
Electronic Submission (required)
The six-month progress report, including all required documents, should be submitted to OAH and OGM electronically through Grantsolutions.gov. All required reporting documents should be submitted and uploaded within the Grant Notes section of “My Grants” for access by the assigned OAH Project Officer and OGM Grants Management Specialist. Grantees should include the grant number on all submissions. All materials must be submitted by February 1st at 11:00 p.m. Eastern Standard Time.
PART THREE: PROGRESS REPORT CONTENT
The six-month progress report should include: • Six-month progress update for the reporting period (July 1 – Dec 31) • Performance measure data for the reporting period • Additional materials in the appendices • Evaluation update for the reporting period
The contents of the report should be properly labeled and numbered. Content should be concise, complete and written in 12-point font.
4
I. SIX-MONTH PROGRESS UPDATE
The six-month progress update should include (see Exhibit A for the Six-Month Progress Update Template):
• An update on the status (i.e., met, unmet, in progress) and a narrative description of the progress in completing each of the OAH expectations for the respective grant program. (See Table 1 on page 5 for a summary of OAH grant expectations and Exhibit B on page 19 for a detailed description of the OAH grant expectations).
For each OAH expectation, you should describe the key activities that were implemented to address the expectation, as well as any accomplishments and barriers encountered. If the expectation has not been met, you should describe what is currently being done to meet the expectation and the timeline for when it will be met.
• An update on any other significant project activities that occurred during the six-month project period that don’t fall under one of the OAH expectations.
• A report on any significant barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome them.
• A description of key accomplishments during the six-month reporting period.
• Verification that all relevant documents from the reporting period (July 1 – Dec 31) have been saved in your Grantee Folder in Max.gov (see Exhibit C for the Max.gov Filing Guidance).
See Exhibit A for the Six-Month Progress Update Template for each TPP grant program. See Exhibit B for a Description of OAH Grant Expectations for each TPP grant program. See Exhibit C for the Max.gov Filing Guidance for TPP Grantees.
5
Table 1 – Overall OAH Grant Expectations for each TPP Grant Program
Capacity Building to Support Replication of EBPs (1a)
1. Project Management 2. Community Mobilization and Sustainability 3. Recruit and Retain CBA Recipients 4. Provide CBA to CBA Recipients 5. Safe and Supportive Environments 6. Youth-Friendly Healthcare Linkages and
Referrals 7. Dissemination and Communication 8. Evaluation 9. Continuous Quality Improvement
Replication of EBPs to Scale in Communities with Greatest Need (1b)
1. Project Management 2. Community Mobilization and Sustainability 3. Implementation of EBPs 4. Safe and Supportive Environments 5. Youth-Friendly Healthcare Linkages and
Referrals 6. Dissemination and Communication 7. Evaluation 8. Continuous Quality Improvement
Supporting and Enabling Early Innovation (2a)
1. Project Management 2. Sustainability 3. Strategy to Foster Innovation 4. Innovator Selection 5. Innovator Progress and Materials 6. Innovator Supports 7. Dissemination and Communication 8. Evaluation 9. Continuous Quality Improvement
Rigorous Evaluation of New and Innovative Approaches (2b)
1. Project Management 2. Sustainability 3. Engage Target Population 4. Intervention Development and Packaging 5. Implementation 6. Safe and Supportive Environments 7. Youth-Friendly Healthcare Linkages and
Referrals 8. Dissemination and Communication 9. Evaluation 10. Continuous Quality Improvement
Providing Capacity Building Assistance to OAH TPP Grantees (National CBA Providers)
1. Project Management 2. Coordination and Collaboration 3. Assess CBA Needs of TPP Grantees 4. Develop Coordinated CBA Plan 5. Provide CBA in Priority Area 6. Dissemination and Communication 7. Evaluation 8. Continuous Quality Improvement
6
II. TPP PERFORMANCE MEASURE REPORTING
All TPP grantees are required to submit their performance measure data for the period from July 1st through December 31st by February1st through the OAH TPP Performance Measure website (https://tpp.rti.org).
A summary of all required performance measures is available on the OAH TPP Performance Measure website (https://tpp. rti.org). All TPP grantees are expected to submit all required performance measure data for all youth served during the reporting period. In addition, all TPP grantees are expected to submit performance measure data on partners, training, and dissemination for the reporting period.
A Help Desk is also available if additional assistance is needed. To contact the Help Desk, click on the Help Desk tab at the top of the TPP Performance Measures Website (after logging on), and you will be able to contact our webmaster regarding your issue. When reporting your problem, please be as descriptive as possible by including the page on which the problem was encountered as well as steps that could be used to replicate the issue. In addition, please provide the name of your grantee organization along with your name, email, and telephone number.
III. APPENDICES
Supporting documents that add value or clarity to the information presented in the progress report should be included in the appendices. Materials included in the appendices should present information clearly and succinctly and add depth to your report.
IV. EVALUATION REPORTING (for ALL TPP Grantees except Tier 2B)
In addition to the update on evaluation and CQI activities provided in your progress report template, all TPP grantees should report any results to-date of their evaluation activities, including pilot results, lessons learned, and any results from implementation and/or outcome evaluation activities. Grantees should also report specifically on how they are using the data gathered through performance measures and evaluation activities to inform their decision-making and make continuous improvements to the grant project.
V. EVALUATION REPORTING (TPP Tier 2B Grantees Only)
All TPP Tier 2B grantees should submit an (1) evaluation abstract and (2) a CONSORT diagram as a part of their six-month progress report. The templates needed to complete these submissions are available at Max.gov on the Evaluation Information Page and as Exhibits at the end of this guidance.
The completed documents must be uploaded into Grantsolutions as a part of your six-month progress report. In addition, we ask that you also submit these evaluation reporting documents
7
via email to [email protected] and copy your eval TA liaison and your OAH project officer.
Evaluation Abstract
The evaluation abstract should be a short (~2 page) structured summary of the intervention, evaluation design, sample, research questions and planned analyses. A template for this abstract is available here on the Max.gov website, along with an example of a completed evaluation abstract. See here for abstracts from the previous cohort. TPP Tier 2B grantees are asked to use the following convention to save the evaluation abstract: [Grantee Name]_Abstract_#.docx.
The TPP Eval team will provide feedback on your abstract before OAH formally approves it. Grantees should expect two rounds of review prior to approval. The Eval TA team will review submissions on a rolling basis to enable an efficient review process.
CONSORT diagrams
All TPP Tier 2b grantees need to provide information sample flow in a CONSORT diagram. A CONSORT diagram is a visual representation of the flow of sample members through an impact evaluation, including sample sizes from the time of random assignment through each of the data collection periods (e.g. baseline, first, and second [and if applicable, third] follow-up surveys).1 Frequently CONSORT diagrams are used in randomized controlled trials, but they are a valuable tool in understanding the sample flow and analytic sample in evaluations of any design.
If your study is using a cluster design, you will need to submit both an individual and a cluster CONSORT diagram. If your study is an individual design, you only need to complete an individual CONSORT diagram. Templates for the CONSORT diagrams are available here. The file should be named using the following convention: [GranteeName]_CONSORT_Feb 2017.docx.
Monitoring this aspect of your evaluation is important for understanding whether your implemented evaluation is maintaining the rigor of the original design. See the Attrition TA Brief for details on how non-response (attrition) can produce bias in an impact estimate. Understanding levels of and reasons for sample loss (attrition) provides information that can be used to target resources towards maximizing consent rates and response rates, either overall, by study condition, or for key subgroups. It is important to examine sample intake and follow-up data collection sample sizes throughout the study for two reasons: 1) to assess whether you are meeting the target sample size on which you based your power calculations, and 2) to assess the likelihood that the final analytic sample for key follow-up time periods will have rates of overall or differential attrition that exceed the HHS evidence standard threshold.2
1 It may also be useful to track non-consent that occurs after random assignment as a step in the CONSORT
diagram process for some designs.
8
We recognize that many grantees are in the early stages of enrollment; therefore, sample sizes for the CONSORT diagram will likely be small and not representative of the final sample that will contribute to the evaluation report. However, completing the CONSORT diagram at this early stage of the evaluation will provide some preliminary information about sample flow, and will serve as useful practice in this activity that we will continue throughout the grant period.
EXHIBIT D includes a bulleted list of the information that the evaluator will need to complete the CONSORT diagram for either individual or cluster designs and an example of an individual level design and associated CONSORT diagram (this example is also available at Max.gov here).
The completed documents must be uploaded into Grantsolutions as a part of your six-month progress report. In addition, we ask that you also submit these evaluation reporting documents via email to [email protected] and copy your eval TA liaison and your OAH project officer.
PART FOUR: ADDITIONAL INFORMATION AND SAMPLE TEMPLATES
• EXHIBIT A: SIX-MONTH PROGRESS REPORT UPDATE TEMPLATES
• EXHIBIT B: DESCRIPTION OF OAH GRANT EXPECTATIONS
• EXHIBIT C: MAX FILING GUIDANCE FOR TPP GRANTEES – YEAR TWO
• EXHIBIT D: CONSORT DIAGRAM INSTRUCTIONS, TEMPLATEs, and EXAMPLE (Tier 2B Grantees Only)
9
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1A Grantees
Grantee Name Grant #:
July 1, 20xx – December 31, 20xx
OAH Grant Expectation STATUS In Progress, Met, Unmet
NARRATIVE DESCRIPTION Provide a description of the status of meeting each of the OAH expectations. Describe key activities implemented, related accomplishments, and any barriers encountered. If the expectation has not been met, describe what is currently being done to meet the expectation and the timeline for when it will be met.
Project Management
Community Mobilization and Sustainability
Recruit and Retain CBA Recipients
Provide CBA to CBA Recipients
Safe and Supportive Environments
Youth-Friendly Healthcare Linkages and Referrals
Dissemination and Communication
Evaluation
Continuous Quality Improvement
10
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1A Grantees – p. 2
Update on Organizations Receiving CBA
For each organization receiving CBA, provide an update on the type of CBA received and progress in providing CBA
Organization #1 –
Organization #2 –
Organization #3 –
Additional Project Activities
Report on any other significant project activities that occurred during the first six-months of the project period.
Barriers, Challenges, and Solutions
Report on any additional barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome them.
Key Accomplishments and Lessons Learned for Six-Month Project Period
Provide a description of the key accomplishments of the grant during the six-month project period. Provide a description of your lessons learned during this project period and how it will impact the next project period.
11
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1B Grantees
Grantee Name Grant #:
July 1, 20xx – December 31, 20xx
OAH Grant Expectation STATUS In Progress, Met, Unmet
NARRATIVE DESCRIPTION Provide a description of the status of meeting each of the OAH expectations. Describe key activities implemented, related accomplishments, and any barriers encountered. If the expectation has not been met, describe what is currently being done to meet the expectation and the timeline for when it will be met.
Project Management
Community Mobilization and Sustainability
Implementation of Evidence-Based Programs
Safe and Supportive Environments
Youth-Friendly Healthcare Linkages and Referrals
Dissemination and Communication
Evaluation
Continuous Quality Improvement
12
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 1B Grantees – p. 2
Additional Project Activities
Report on any other significant project activities that occurred during the first six-months of the project period.
Barriers, Challenges, and Solutions
Report on any additional barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome them.
Key Accomplishments and Lessons Learned for Six-Month Project Period
Provide a description of the key accomplishments of the grant during the six-month project period. Provide a description of your lessons learned during this project period and how it will impact the next project period.
13
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2A Grantees
Grantee Name Grant #:
July 1, 20xx – December 31, 20xx
OAH Grant Expectation STATUS In Progress, Met, Unmet
NARRATIVE DESCRIPTION Provide a description of the status of meeting each of the OAH expectations. Describe key activities implemented, related accomplishments, and any barriers encountered. If the expectation has not been met, describe what is currently being done to meet the expectation and the timeline for when it will be met.
Project Management
Sustainability
Strategy to Foster Innovation
Innovator Selection
Innovator Supports
Innovator Progress and Materials
Dissemination and Communication
Evaluation
Continuous Quality Improvement
14
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2A Grantees – p. 2
Update on Selected Innovators
For each innovator receiving funding and support, provide an update on the innovator, the team, the interventions, and other relevant information.
Innovator #1 – update on innovators, the team, the intervention, other relevant information
Innovator #2 – update on innovators, the team, the intervention, other relevant information
Innovator #3 – update on innovators, the team, the intervention, other relevant information
Innovator #4 – update on innovators, the team, the intervention, other relevant information
Innovator #5 – update on innovators, the team, the intervention, other relevant information
Innovator #6 – update on innovators, the team, the intervention, other relevant information
Innovator #7 – update on innovators, the team, the intervention, other relevant information
Additional Project Activities
Report on any other significant project activities that occurred during the first six-months of the project period.
Barriers, Challenges, and Solutions
Report on any additional barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome.
Key Accomplishments and Lessons Learned for Six-Month Project Period
Provide a description of the key accomplishments of the grant during the six-month project period. Provide a description of your lessons learned during this project period and how it will impact the next project period.
15
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2B Grantees
Grantee Name Grant #:
July 1, 20xx – December 31, 20xx
OAH Grant Expectation STATUS In Progress, Met, Unmet
NARRATIVE DESCRIPTION Provide a description of the status of meeting each of the OAH expectations. Describe key activities implemented, related accomplishments, and any barriers encountered. If the expectation has not been met, describe what is currently being done to meet the expectation and the timeline for when it will be met.
Project Management
Sustainability
Engage Target Population
Intervention Development and Packaging
Implementation
Safe and Supportive Environments
Youth-Friendly Healthcare Linkages and Referrals
Dissemination and Communication
Evaluation
Continuous Quality Improvement
16
EXHIBIT A – Six-Month Progress Update Template for TPP Tier 2B Grantees – p. 2
Additional Project Activities
Report on any other significant project activities that occurred during the first six-months of the project period.
Barriers, Challenges, and Solutions
Report on any additional barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome them.
Key Accomplishments and Lessons Learned for Six-Month Project Period
Provide a description of the key accomplishments of the grant during the six-month project period. Provide a description of your lessons learned during this project period and how it will impact the next project period.
17
EXHIBIT A – Six-Month Progress Update Template for TPP National CBA Provider Grantees
Grantee Name Grant #:
July 1, 20xx – December 31, 20xx
OAH Grant Expectation STATUS In Progress, Met, Unmet
NARRATIVE DESCRIPTION Provide a description of the status of meeting each of the OAH expectations. Describe key activities implemented, related accomplishments, and any barriers encountered. If the expectation has not been met, describe what is currently being done to meet the expectation and the timeline for when it will be met.
Project Management
Coordination and Collaboration
Assess CBA Needs of TPP Grantees
Develop Coordinated CBA Plan
Provide CBA in Priority Area
Dissemination and Communication
Evaluation
Continuous Quality Improvement
18
EXHIBIT A – Six-Month Progress Update Template for TPP National CBA Provider Grantees – p. 2
Additional Project Activities
Report on any other significant project activities that occurred during the first six-months of the project period.
Barriers, Challenges, and Solutions
Report on any additional barriers and/or challenges encountered, as well as the solutions, or proposed solutions, for how to overcome them.
Key Accomplishments and Lessons Learned for Six-Month Project Period
Provide a description of the key accomplishments of the grant during the six-month project period. Provide a description of your lessons learned during this project period and how it will impact the next project period.
19
EXHIBIT B: DESCRIPTION OF OAH GRANT EXPECTATIONS Included below is a list of expectations for each TPP grant program and a description of the specific grantee activities that fall within that grant expectation.
Capacity Building to Support Replication of EBPs (TPP Tier 1a)
OAH Grant Expectation
Description of Activities that Fall with the Expectation
Project Management
Recruit and retain staff; professional development for staff and partners; build, enhance, and sustain partnerships; monitor partners/ contractors to assess progress in meeting objectives; assess community needs
Community Mobilization and Sustainability
Assist CBA recipients in community mobilization; sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)
Recruit and Retain CBA Recipients
Recruit (if applicable) and retain youth-serving organizations to receive capacity building assistance
Provide CBA to CBA Recipients
Assess needs of youth-serving organizations; develop a plan for providing CBA to meet organization needs; provide CBA; revise, update, and implement activities with CBA recipients; monitor CBA recipients to assess progress in meeting objectives and to ensure implementation of EBPs
Safe and Supportive Environments
Annually complete OAH assessment tools on (1) integrating a trauma-informed approach into TPP programs, (2) assessing LGBTQ inclusivity of TPP programs, and (3) putting Positive Youth Development characteristics in action; identify strategies for ensuring programs are implemented in safe and supportive environments for youth and their families
Youth-Friendly Healthcare Linkages and Referrals
Identify and recruit youth-friendly healthcare providers; assess providers to ensure services are youth-friendly; develop protocols and procedures for making referrals; develop and disseminate provider referral guide; identify and train staff on referral protocols and procedures
Dissemination and Communication
Develop, maintain, and implement a dissemination and communications plan, inclusive of project website and brand; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences
Evaluation Collect, report, and use performance measure data; evaluate the implementation and success of CBA provided to youth-serving organizations; evaluate the success of each CBA recipient’s efforts to implement EBPs
Continuous Quality Improvement
Continually assess progress toward outcomes and continuously use data from evaluation to inform stakeholders and make necessary adjustments and changes to improve quality of your project
20
Replication of EBPs to Scale in Communities with Greatest Need (TPP Tier 1b)
OAH Grant Expectation
Description of What is Included Within Expectation
Project Management
Recruit and retain staff; professional development for staff and partners; monitor partners/contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships; continue to assess community needs; use Getting to Outcomes® to plan, implement, and evaluate the project
Community Mobilization and Sustainability
Mobilize the community to develop, implement, and continuously improve the plan to prevent teen pregnancy and promote positive youth development in their community; develop and continue to engage the Community Advisory Board in each community; develop and continue to engage the Youth Leadership Council in each community; sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)
Implementation of EBPs
Implement EBPs to scale in all settings (at least 3) in each community served; recruit, retain, and engage participants; monitor implementation of programs with fidelity and quality
Safe and Supportive Environments
Annually complete OAH assessment tools on (1) integrating a trauma-informed approach into TPP programs, (2) assessing LGBTQ inclusivity of TPP programs, and (3) putting Positive Youth Development characteristics in action; identify strategies for ensuring programs are implemented in safe and supportive environments for youth and their families (i.e., at least 3 actions for TIA, at least 1 action for each of the six constructs for inclusivity, and at least 3 actions for PYD)
Youth-Friendly Healthcare Linkages and Referrals
Identify and recruit youth-friendly healthcare providers; assess providers to ensure services are youth-friendly; develop protocols and procedures for making referrals; develop and disseminate provider referral guide; identify and train staff on referral protocols and procedures
Dissemination and Communication
Develop, maintain, and implement a dissemination and communications plan, inclusive of project website and brand; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences
Evaluation Collect, report, and use performance measure data; conduct implementation evaluation to document the process for developing and implementing the program and to identify key successes, challenges, and lessons learned; conduct outcome evaluation to compare outcomes in the community(ies) served to outcomes in a similar community that did not receive the program
Continuous Quality Improvement
Continually assess progress toward outcomes and continuously use data from evaluation to inform stakeholders and make necessary adjustments and changes to improve quality of your project
21
Supporting and Enabling Early Innovation (TPP Tier 2a)
OAH Grant Expectation Description of What is Included Within Expectation
Project Management Recruit and retain staff; professional development for staff and partners; monitor implementation partners/contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships
Sustainability Sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)
Strategy to Foster Innovation
Implement a promising strategy to foster innovation using a framework, theory of change, or other strategic approach
Innovator Selection Convene an open, national competition that results in selecting, funding, and retaining innovators
Innovator Supports Provide support to innovators, both individual and group-based
Innovator Progress and Materials
Monitor innovator progress; ensure that innovator materials are medically accurate, age-appropriate, culturally appropriate, LGBTQ inclusive, and trauma-informed
Dissemination and Communication
Develop, maintain, and implement a dissemination and communications plan, inclusive of project website and brand; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences.
Evaluation Collect, report, and use performance measure data; monitor fidelity for the project; evaluate the implementation and success of the project; evaluate the success of each innovator
Continuous Quality Improvement
Continually assess progress toward outcomes and continuously use data from evaluation to inform stakeholders and make necessary adjustments and changes to improve quality of your project
22
Rigorous Evaluation of New and Innovative Approaches (TPP Tier 2b)
OAH Grant Expectation Description of What is Included Within Expectation
Project Management Recruit and retain staff; professional development for staff and partners; monitor implementation partners/contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships
Sustainability Sustainability activities aligned with 8 OAH Factors of Sustainability (i.e., create an action plan, assess the environment, be adaptable, secure community support, integrate programs and services, build a team of leaders, create strategic partnerships, and secure diverse financial opportunities)
Engage Target Population Involve target population in the development, implementation, and evaluation of the grant project
Intervention Development and Packaging
Develop, pilot, and refine the intervention; package the intervention to be implementation ready by the end of the grant using the OAH TPP Packaging and Dissemination Guidance
Implementation Implement the intervention; recruit, retain, and engage participants; monitor implementation of intervention and control with fidelity and quality
Safe and Supportive Environments
Complete OAH assessment tools on (1) integrating a trauma-informed approach into TPP programs, (2) assessing LGBTQ inclusivity of TPP programs, and (3) putting Positive Youth Development characteristics in action; use the results to identify strategies for ensuring programs are implemented in a safe and supportive environment for youth and their families
Youth-Friendly Healthcare Linkages and Referrals
Identify and recruit youth-friendly healthcare providers; assess providers to ensure services are youth-friendly; develop protocols and procedures for making referrals; develop and disseminate provider referral guide; identify and train staff on referral protocols and procedures
Dissemination and Communication
Develop, maintain, and implement a dissemination and communications plan; market and raise awareness of grant project; communicate project successes; disseminate lessons learned, successes, and results to local, state, and national audiences.
Evaluation Collect, report, and use performance measure data; conduct rigorous evaluation of program implementation and program outcomes
Continuous Quality Improvement
Continually assess progress toward outcomes and continuously use data from evaluation to inform stakeholders and make necessary adjustments and changes to improve quality of your project
23
Providing Capacity Building Assistance to OAH TPP Grantees (National CBA Providers)
OAH Grant Expectation
Description of What is Included Within Expectation
Project Management
Recruit and retain staff; training and professional development for staff and partners; monitor partners and/or contractors to assess progress in meeting objectives; build, enhance, and sustain partnerships
Coordination and Collaboration
Coordinate and collaborate with the other national CBA providers and with OAH to regularly assess grantee needs, develop an annual coordinated CBA plan, provide effective CBA designed to meet grantee needs, and eliminate duplication and streamline services for individual TPP grantees
Assess CBA Needs of TPP Grantees
Assess the TPP grantee needs for CBA in the five key priority areas
Develop Coordinated CBA Plan
Use the results from the assessment of TPP grantee needs to develop an annual coordinated CBA plan that will guide the development and provision of CBA for the year
Provide CBA in Priority Area
Provide CBA that is tailored to grantee needs and uses diverse approaches; provide three levels of CBA to TPP grantees (low, medium, and high intensity), ensuring that all TPP grantees receive some level of CBA from each CBA provider annually; ensure CBA materials are medically accurate, age appropriate, culturally and linguistically appropriate, and inclusive of LGBTQ youth
Dissemination and Communication
Develop, maintain, and implement a dissemination and communications plan; disseminate and communicate information about project activities, lessons learned, successes, and evaluation results
Evaluation Collect, report, and use performance measure data; evaluate the implementation and success of the CBA provided to TPP grantees
Continuous Quality Improvement
Continually assess progress toward outcomes and continuously use data from evaluation to inform stakeholders and make necessary adjustments and changes to improve quality of your project
24
EXHIBIT C: MAX Filing Guidance for TPP Grantees – YEAR TWO
The OAH Project Management and Collaboration Community (Max.gov) is used to: 1. Track and submit requests related to your grant project in MAX Forms; and 2. Store important documents for your grant project in MAX Folders to create a shared,
historical record of the project.
For MAX Forms, each grantee has a secure space to submit requests and track information. MAX Forms is where you are expected to submit:
• grantee profile updates, • medical accuracy requests (note - medical accuracy review documentation must also be
submitted in the MAX folders); • major adaptation requests, and • minor adaptations notices.
For MAX Folders, each grantee has a separate grantee folder, also called the grantee’s individual page. The grantee and OAH Project Officer should work collaboratively to ensure the grantee folder is up-to-date and includes all relevant files, in accordance with the filing guidance below. Documents should be uploaded as an attachment to specific folders as indicated below. Grantees must consult with their Project Officers before creating any additional subfolders within their MAX folder.
MAX Folders for Tier 1A Grantees
MAX Folder Pages What to Upload Dissemination and Communications
All press, photos, communication plans, etc.
Evaluation and CQI All documents related to evaluation and continuous quality improvement. Medical Accuracy Review
All documents for the internal medical accuracy review, OAH’s medical accuracy review, program materials, and insert pages/modifications. Please note grantees must also submit the MAX Form when submitting a medical accuracy review request.
MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents from Year 1 – e.g. Materials Review, CBA Plan, etc.
Documents should be uploaded to sub-folders as appropriate; otherwise, attach them to the “Year 1” page.
Year 2 All other documents from Year 2 including those related to: 1. Project Management 2. Community Mobilization and Sustainability 3. Recruiting and Retaining CBA Recipients 4. Providing CBA to CBA Recipients 5. Safe and Supportive Environments 6. Youth-Friendly Healthcare Linkages and Referrals
25
EXHIBIT C: MAX Filing Guidance for TPP Grantees – YEAR TWO (cont. p. 2)
MAX Folders for Tier 1B Grantees
MAX Folder Pages What to Upload Dissemination and Communications
All press, photos, communication plans, etc.
Evaluation and CQI All documents related to evaluation and continuous quality improvement. Medical Accuracy Review
All documents for the internal medical accuracy review, OAH’s medical accuracy review, program materials, and insert pages/modifications. Please note grantees must also submit the MAX Form when submitting a medical accuracy review request.
MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents from Year 1 – e.g. Materials Review, Pilot Results, etc.
Documents should be uploaded to any sub-folders as appropriate; otherwise, attach them to the “Year 1” page.
Year 2 All other documents from Year 2 including those related to: 1. Project Management 2. Community Mobilization and Sustainability 3. Implementation of EBPs 4. Safe and Supportive Environments 5. Youth-Friendly Healthcare Linkages and Referrals
MAX Folders for Tier 2A Grantees
MAX Folder Pages What to Upload Dissemination and Communications
All press, photos, communication plans, etc.
Evaluation and CQI All documents related to evaluation and continuous quality improvement Medical Accuracy Review
All documents for the internal medical accuracy review, OAH’s medical accuracy review, program materials, and insert pages/modifications. Please note grantees must also submit the MAX Form when submitting a medical accuracy review request.
MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents from Year 1. Documents should be uploaded to any sub-
folders as appropriate; otherwise, attach them to the “Year 1” page. Year 2 All other documents from Year 2 including those related to:
• Selection • Strategy • Support • Innovations
26
EXHIBIT C: MAX Filing Guidance for TPP Grantees – YEAR TWO (cont. p. 3)
MAX Folders for Tier 2B Grantees
MAX Folder Pages What to Upload Dissemination and Communications
All press, photos, communication plans, etc.
Evaluation Technical Assistance and CQI
All documentation related to: • Abstract • Impact analysis plan • Implementation analysis plan • Monthly call notes • IRB • Continuous quality improvement
Upload documents to the sub-folders as appropriate; otherwise, attach them to the “Evaluation Technical Assistance & CQI” page.
Medical Accuracy Review
All documents for the internal medical accuracy review, OAH’s medical accuracy review, program materials, and insert pages/modifications. Please note grantees must also submit the MAX Form when submitting a medical accuracy review request.
MOUs All MOUs signed with partners to complete the project. Year 1 Overall All documents related to Year 1 – e.g. Materials Review, Pilot Results,
etc. Documents should be uploaded to any sub-folders as appropriate; otherwise, attach them to the “Year 1” page.
Year 2 All other documents from Year 2 including those related to: 1. Project Management 2. Sustainability 3. Engage Target Population 4. Intervention Development and Packaging 5. Implementation 6. Safe and Supportive Environments 7. Youth-Friendly Healthcare Linkages and Referrals
27
EXHIBIT D CONSORT Diagram Instructions, Templates, and an Example (Tier 2B Grantees Only)
Instructions
CONSORT DIAGRAM documentation
All CONSORT diagrams requires two pieces of information as context for interpretation:
1. A “time stamp” for the CONSORT diagram. The CONSORT should provide information about the number of clusters (or individuals) randomized and the number of clusters (or individuals) participating (i.e., that have not dropped out) as of this particular point in time. Because clusters or individuals may be at different stages of the study at any point in time due to rolling enrollment or multiple cohorts of implementation, certain clusters or individuals may not have an opportunity to contribute data to all data collection points (see more on this below).
2. A brief paragraph (~3-4 sentences) describing the design and data collection, including full enrollment period (start/end dates) and program start/end dates.
CONSORT diagrams that track clusters as the unit of interest For clustered random assignment designs (e.g., random assignment of clinics, community-based organizations, teachers, or schools) and quasi-experimental designs.
The following pieces of information are also necessary as documentation of sample flow in cluster-level CONSORT diagrams:
• The number of clusters (randomly) assigned to each condition (i.e., treatment and comparison).
• The number of clusters still participating (i.e. retained), by study condition, at each data collection time point. A cluster is defined as “participating” if at least one individual in the cluster completed the data collection event.
o At any given time point, some clusters may not have had the opportunity to contribute data for a particular data collection event. For example, clusters with youth that have not completed the program would not have had the opportunity to contribute follow-up data. Therefore, it is important to document the number of clusters that are eligible (i.e. the number of clusters that could have had youth contribute data) at a given time point, in addition to providing the number of clusters that actually did have youth provide data (i.e. the number of cluster retained).
• In addition to tracking cluster flow, it is necessary to complete a CONSORT diagram for the youth in participating clusters. The HHS evidence standards focus on attrition among those clusters who participate in each data collection event; therefore, the randomized sample in the individual- level CONSORT diagram should only include youth in the clusters that are still participating in the evaluation. More on this below.
28
EXHIBIT D (cont. p. 2)
CONSORT diagrams that track individuals as the unit of interest For both individual level and cluster level designs.
The following pieces of information are also necessary as documentation of sample flow in individual- level CONSORT diagrams:
• The number of youth (randomly) assigned in total, the number assigned to each condition, and the start and end dates of assignment.
o If the evaluation uses a cluster design, then the number of youth in each condition at any time point should reflect the number of youth in participating clusters at that time point.
• The number of youth that have provided data, by study condition, at each data collection time point (baseline and subsequent follow-ups).
o At a given time point, a subset of individuals may not have had the opportunity to contribute data for a particular data collection event. For example, individuals that have not completed the program would not have had the opportunity to contribute follow-up data. Therefore, it will be important to document the number of youth that are eligible (i.e. the number of youth that could have contributed data) at a given time point, in addition to providing the number of youth that actually did provide data.
Templates Templates for the CONSORT diagrams are available on the Max.gov website here.
CONSORT DIAGRAM example
Individual level design This example describes sample flow in an individual RCT, where youth are randomly assigned to treatment and control conditions on an on-going basis. The study began on January 2, 2016, with random assignment continuing through the end of December 2017. The data collection periods for this study are baseline, immediate post-intervention, and 6-months post-intervention. The CONSORT diagram represents sample sizes through July 15, 2016.
As of July 15, 2016, 1,000 youth had been randomly assigned to condition at some point since the study began on January 2, 2016. In total, 500 youth were assigned to the treatment condition and 500 were assigned to the control condition. All randomly assigned youth completed the baseline survey. Only a subset of the youth had been in the study long enough to be eligible to complete this first follow-up. Specifically, 450 youth in each condition. The other 50 youth in each condition, who were recently randomly assigned to condition, had not completed the program (and thus, were not eligible to complete the first follow-up survey). Among the 900 youth (450 in each condition) who had the opportunity to complete the first follow-up, 445 members of the treatment group and 442 members of the control group completed the survey. Finally, a very small subset of the sample (only 50 youth in each condition) had been in the study long enough to have the opportunity to complete the 6-month follow-up survey. Of these 100 youth, 45 members of the treatment group and 48 members of the control group completed the survey. The majority of the study participants (450 youth in each condition) had not yet been in the study long enough to be eligible for the 6-month follow-up survey.
29
EXHIBIT D (cont. p. 3)
EXAMPLE - CONSORT Diagram for Individual-Level Sample Flow
Data included in report are current as of:
July 15, 2016
This evaluation is an individual-level RCT with youth assigned to treatment or control. They study will collect data baseline, immediate post-intervention, and 6-month follow-up data. Youth are enrolled and randomly assigned on a rolling basis between January 2016 and December 2017. Programming will take place over the same period, with all programming ending in January 2018.