expansion of existing program - Indiana · 2020-01-29 · • Subtract 1 point if abstract exceeds two pages; subtract 2 points if abstract exceeds three pages (and note this in Reviewer

2018–Cohort 9 RFP: 21st Century Community Learning Centers Reviewer Scoring Rubric

1

Reviewer ID# 55

Applicants Name AYS, Inc. (Keystone Ave, Indianapolis)

I. PROJECT ABSTRACT (Up to 5 POINTS)

At a minimum, applicant describes five elements: (a) student needs; (b) participants to be served; (c) proposed activities; (d) intended outcomes; and (e) key people involved.

• Subtract 1 point if abstract exceeds two pages; subtract 2 points if abstract exceeds three pages (and note this in Reviewer Comments).

IF application is for expansion of existing program (with continued funding), must describe how additional funds will be used for new programming, i.e., will not replace current/past 21st CCLC funding.

• Subtract 2 points if applicable and not addressed (and so note in Reviewer Comments).

0 points Abstract not provided or

does not address any required elements (i.e.,

student needs; participants to be served; activities;

outcomes; or key personnel)

1-2 point range Only includes 1-2 required elements (i.e., student needs; participants to be served; activities; outcomes; or key personnel)

3-4 point range Includes 3-4 required elements (i.e., student needs; participants to be served; activities; outcomes; or key personnel). Points reduced if exceeds two pages.

5 points Includes all 5 required elements (i.e., student needs; participants to be served; activities; outcomes; or key personnel). Points reduced if exceeds two pages.

Reviewer Comments: Abstract w/in page limit requirement and addresses all five required elements. See pages 51-52.

Score: 5


2

II. COMPETITIVE PRIORITY POINTS (up to 10 POINTS) A. Required Descriptions (2 Points) Applicants describe

• How they meet application priority (i.e., students served must attend a school with at least 40% poverty; schools rated D or F; or school/s that are rural and low-income; and

• The origin of the partnership between the school/district receiving Title I funds and the community-based public or private organization/s submitting the jointly proposed project.

0 points Descriptions not provided

1 point Just one of the two required descriptions provided (how application priority is met, OR origin of partnership)

2 points Both descriptions provided (how priority is met, and origin of partnership)

Reviewer Comments: Poverty rates among the 5 proposed school sites range from 73% to 91%-- far exceeding required 40% threshold. All five are also Title I schools, with 2 of 5 receiving D/F ratings. Pre-existing partnerships with IPS (since 1980) and Decatur (2010/2014), via former AYS traditional programs and 21st CCLC programming. SENSE Charter School also previously served and continues to be a high need school (91% poverty; 13% EL).

Score: 2

B. Organizational Priority Points (4 Points) Four (4) Points awarded to applications that meet ONE of the following criteria:

• New applicant who has never received 21st CCLC funds • Identified as a Rural and Low Income Applicant (SRSA or RLIS eligible districts) • Identified as a Targeted or Comprehensive School; or a school demonstrating 90% or

greater free/reduced lunch student participation • Serving high school students • 25% (or more) program staff has completed the Child & Youth Care Credential • 95% (or greater) direct-services program staff are CPR certified (100%, See p. 54) • At least one (1) youth participant serves on program’s Advisory Board. This individual

should be representative of the age range and population served by the 21st CCLC program • Early Learning for 3-5 year olds: Must establish partnership with an elementary school, align school-

day activities and use Indiana’s Standards Tool for Alternate Reporting of Kindergarten Readiness (ISTAR-KR).

0 points Does not meet criteria

4 points Applicant meets criteria

Score: 4 C. Programming Priority Points (4 Points)


3

Four (4) points awarded to applications that provide hands-on programming, as demonstrated in the design and activity plan, in ONE of the following areas.

• STEM (Science, Technology, Engineering, and Math) • CCR (College and Career Readiness for high school programs) • Literacy (strong focus on English/Language Arts) • Family Engagement (minimally hosts 5 events annually, excluding parent courses; employs engagement

strategies, such as home visits, interviews, surveys, newsletters, or family involvement curriculum) Priority programming area identified by applicant must be implemented throughout the four-year grant period for a minimum of 3 hours per week. Priority programming area must be listed in Section V (Goals, Objectives, Activities, & Performance Measures). If priority programming area is NOT listed in Section V, points cannot be awarded.

0 points Does not meet criteria

4 points Meets criteria & area listed in Section V Goals & Objectives

Reviewer Comments -- if points not awarded: p. 54: Strong literacy focus, via 45-minute rotations of academic support M-Th: (Rotation 1 = LitART using award-winning children’s books & hands-on activities; Rotation 2 = School-day software work, e.g., Moby Max, using games, etc. to encourage skill building in ELA & math; Rotation 3 = tutoring & homework assistance.) Student choice-driven enrichment clubs demonstrating real world application of ELA concepts also offered daily. Club lessons employ research-based lesson planning tool (DoS PPT) developed by PEAR Institute/Harvard, incorporating Indiana Academic Standards. All literacy instruction supported by PD (e.g., 3 hrs of lesson planning training and 7.5 hours of literacy-specific PD annually. P. 55 Section V Goals, Objectives, Activities, Measures (p 62) directly aligned to Literacy activities/measures described above.

Score: 4


4

III. NEED FOR PROJECT (5 POINTS) A. Data Evidence Demonstrating Need (3 Points)

Analyzed student data required in THREE areas: • Achievement (e.g., State or local assessment scores; students below grade level, etc.) • Demographics (e.g., measures of poverty, student mobility, student ethnicity, etc.) • Behavioral (e.g., attendance rates, dropout rates, discipline data, rates of juvenile crime,

etc.) Data must be shown for EACH school to be served. (See Attachment B: List of Schools to Be Served). Data demonstrates high need in both poverty level and academic achievement.

0 points

Data evidence not presented

1 point Data not provided for all three areas (i.e., achievement, demographics and behavioral)

2 points All three areas addressed (i.e., achievement, demographics & behavioral) and presented for EACH school to be served (Attachment B)

3 points Achievement, demographic & behavioral data shown for EACH school (Attachment B) and demonstrates high need -- in both poverty levels and academic achievement.

Reviewer Comments: P 55 charts detailed needs for each of the five proposed school sites, including academic, demographic and behavioral. Other useful statistics also provided, e.g., neighborhood crimes against persons in last six months). All submitted evidence points to high student needs.

Score: 3

B. Demonstrate Expanded Out-of-School Time Programming (1 Point) Applicant provides CHART showing how 21st CCLC expands out-of-school time programming for EACH served school and addresses gaps in current afterschool opportunities (i.e., program is in addition to currently available services to students.

0 points: Chart/graphic not provided 1 point: Chart/graphic provided showing increased time that addresses gaps for each school

Score: 1 C. Describe Process for Assessing Needs/Services (1 Point) The process is clearly articulated and describes who was involved – including how partners, parents & youth were involved – in assessing community needs/services

0 points: Process and/or partner involvement not described

1 point: Process and partners involved are clearly described

Score: 1


5

IV. PARTNERSHIPS/COLLABORATIONS (5 POINTS) A. Describe Collaboration with Other Agencies/Funding Streams (1 point) Describes collaboration with other agencies: federal (e.g., Title I, Child Nutrition, Temporary Assistance for Needy Families); State & local programs to achieve goals (e.g., In-Kind contributions; the provision of staff development, transportation, facilities, equipment, etc.).

0 points: Not addressed or too vague to award point 1 point: Applicant demonstrates collaboration with other agencies, e.g., Title I, Child Nutrition, TANF,

State/local programs

Score: 1

B. Describe How Each Partner’s Contribution Supports Program (1 point) Applicant completed Attachment F, listing each partner and its commitment to provide services as either: “In-Kind” services; or “Contracted” services. Each partner provides authorizing signature and contact information.

0 points: Attachment F not submitted 1 point: Applicant completed and submitted Attachment F

Score: 1

C. Memorandum of Understanding Executed by Applicant and Key Partners (3 points) A Memorandum of Understanding (MOU) executed by the Applicant and partner is provided in the proposal’s APPENDIX. An MOU is completed for each key partner providing service.

The MOU details agreed upon commitments and each partner’s role, e.g. how resources will be shared (e.g., instructional space, materials, equipment); responsibilities for management/oversight; how students are chosen for program; linkages between school day and program; the provision of curriculum, PD and staffing; how/when data/surveys will be collected, compiled & shared. NOTE: This is in addition to the applicant’s submission of Attachment F (above).

0 points MOU/s detailing partner roles & responsibilities not provided. NOTE: This is in addition to Attachment F.

1 point At least one MOU provided in Appendix, but does not fully articulate roles & responsibilities between applicant & partner

2 points MOU/s provided in Appendix for all key partners offering basic info relevant to applicant/partner roles

3 points MOU/s provided in Appendix for all key partners providing clearly-articulated expectations for applicant and for partner

Reviewer Comments: MOUs provided for all key partners with basic info. These are in addition to applicant’s submission of Attachment F. Basic info relevant to partners’ roles provided.

Score: 2


6

V. PROGRAM DESIGN AND IMPLEMENTATION (30 points) A. Goals, Objectives, Performance Measures, Activities and Assessments (8 points) Applicant provides a Table overviewing the Objectives, Activities, Performance Measures and Assessment Strategies for each proposed 21st CCLC Program Goal. Three (3) goals required (minimally) – with at least two objectives per goal –along with related activities, performance measures and assessment strategies for each objective. The performance measures must be measurable, specific and challenging, yet achievable.

1. Academic Goal: Students meet/exceed State/local achievement standards in ELA and in

Mathematics. • State assessments (ISTEP, ILEARN) cannot be the only performance measure (e.g.,

include report card grades, survey data, or local assessments) • If requesting priority points for CCR, STEM or Literacy -- must include goals specific

to priority point area.

2. Student Behavioral Goal: Students demonstrate improvement in areas such as classroom attendance or performance; or decreased disciplinary actions/other adverse behaviors.

3. Family Involvement Goal: Strategies to increase involvement that supports their child’s success; or to decrease barriers to parent/guardian involvement. • If applicant requested priority points for Family Involvement, must minimally host 5

events annually, excluding parent courses; employs engagement strategies, such as home visits, interviews, surveys, newsletters, or family involvement curriculum

Additional goals required, if program serves HS or pre-school students; or offers summer programs. 4. High School Goal: Strategies to increase program participants’ accelerated course work

(dual credit, AP, IB, etc.), OR increase program participants pursuing a technical track (vocational, CTE, etc.). • Must also show x/% of regular participants in 4th year of HS that will graduate within six

months of their “grade-level cohort.” 5. Pre-school Goal: Strategies that support early learning and kindergarten readiness (ISTAR-

KR) 6. Summer Program Goals: Include up to three (3) measures relevant to either: participation

rates; maintain/improve ELA/Math performance from spring to fall; discipline, character development or service projects; career exploration; health & safety; parent engagement; STEM interest/awareness.

Objectives, activities & measures may differ for elementary, middle and high schools if all are served under the same grant. Programs may choose to develop one Table for the entire program or separate Tables for specific program sites (e.g., elementary and middle/high schools). If more than one table is presented, each must include all required goals.


7

0-2 point range Table overviewing Goals,

Objectives, Performance Measures, Activities & Assessments includes less than all three of the required

goals, i.e., (1) student achievement, (2) behavioral, & (3) family

involvement

3-6 point range Includes all three required goals, i.e., achievement, behavioral and family involvement -- as well as HS, pre-K, or summer goals, if applicable. At least two objectives provided per goal. Activities are aligned with each objective; performance measures include numerical targets and are each connected to a specific measurement strategy

7-8 point range Includes all three required goals, i.e., achievement, behavioral and family involvement -- as well as HS, pre-K, or summer goals, if applicable. At least two objectives provided per goal. Highly engaging activities are aligned with objectives; challenging performance measures include numerical targets and are each connected to a specific measurement strategy

To obtain a score of 5 points or higher: • State assessments cannot be only performance measure, i.e., also include such things as report card grades, local

assessments, survey data • Must include goals specifically related to priority points requested in Section II (CCR, STEM, Literacy or Family

Involvement) Reviewer Comments: P 62+: Includes all 3 goals (achievement, behavioral & family involvement) with more than two objectives per goal (exceeds expectation). Additional goal (Improve Literacy Engagement) also included, with relevant objectives, activities & measures. Local assessments, grades, survey results and IN-QPSA assessments included in addition to State Assessments (ELA & Math). Applicant has included goals specifically related to priority points requested (Literacy focus). Activities appear highly engaging (LitART Learn and Litamatics; gaming software platforms; enrichment clubs) and promote positive behavior and school attendance (Scholar Rewards) and Friday Open Mic events. Routinely scheduled Community Connection Nights (once per month/10 months) provide info via community partner presentations to support and involve parents. Bi-monthly Scholar Updates to parents report child’s progress.

Score: 7

B. Evidence of Previous Success (2 points)


8

Applicant must provide evidence of previous success in operating out-of-school programs targeting the youth populations to be served by the proposed grant. Evidence (provided in the Applicant’s APPENDIX) should include (a) successful student recruitment and retention efforts; and (b) successful attainment of academic outcomes for student participants. Applicants that have provided 21st CCLC programming previously are required to present the following evidence of success: (1) Rates of 30+ and 60+ attendance rates for the most recent three years of 21st CCLC

programming; and; (2) ISTEP+ scores of multiple-year attendees, Indiana Growth Model data, or local assessment

performance (e.g., DIBELS, NWEA) that demonstrate increased academic performance. If the applicant has not operated out-of-school programs in the past, the applicant must describe specific strategies that will be used to: (1) Recruit students and encourage high rates of regular program attendance, (2) Ensure students receive academic support needed to demonstrate improved academic

achievement.

0 points Information not provided in APPENDIX.

1 point If previous grantee: Some description of previous attendance rates and program benefits.

If new grantee: Limited information on supporting student retention; and general strategies for providing academic assistance.

2 points If previous grantee: Clearly documented quantitative evidence of past 30+ and 60+ attendance rates and academic outcomes (e.g., ISTEP+, DIBELS, NWEA) showing increased performance.

If new grantee: Specific activities provided to support student recruitment and attendance and to provide academic assistance.

Reviewer Comments: Page 117: Four (out of five schools now proposed) previously served, with more than 200 students attending 60 or more days. ISTEP/grades of those served 2 years/for 60 days or more, showed greatest gains. Data for four schools combined, i.e., not disaggregated by service site.

Score: 1

C. Design Requirements (20 total points for Items 1-8) Applicants must address the following Design Requirements (Narrative) C-1. Requirements of GEPA 427 (1 point) Applicant response submitted as an APPENDIX item. Describes the steps applicant will take to ensure equitable access & participation for students with special needs. Broad discretion is allowed, ensuring applicants’ ability to address barriers unique to their program. Examples include: (1) applicant proposing an adult literacy project serving LEP adults (among others) might describe how it intends to distribute a brochure about the program in the language parents/families understand; (2) applicant might describe how it will make materials available on audio tape or in Braille for students who are blind; (3) applicant might indicate how it intends to conduct “outreach” efforts to encourage middle and high school females to enroll in a model science program that has typically served mostly male students.


9

0 points Information not provided in the APPENDIX or within proposal narrative.

1 point Specific equitability issue identified and addressed (either in Appendix or proposal narrative) to reduce program barrier

Score: 1 (located on p 120)

C-2. Targeted Students and Their Families (3 points) Applicants must:

a. Provide a list of Title I and Non-Title I eligible schools to be served by the 21st CCLC program (complete Form 2 entitled List of Schools to be Served by 21st CCLC, Attachment B);

b. Describe the criteria and processes for recruiting targeted students and their families to be served from the selected school(s); and

c. If applicable, provide justification for the eligibility of school with less than 40% poverty. Provide relevant community data demonstrating the need for out-of-school programming. This can include such things as drop-out rates, criminal or delinquency rates, literacy rates, or school improvement status (comprehensive/targeted).

1 point

Only partial information provided (i.e., only Attachment B List of Schools submitted; OR only narrative supporting criteria & process to recruit students provided).If List of Schools/Attachment B not submitted, zero points.

2 point Identifies Title 1 and non-Title 1 schools (Attachment B); and describes (in narrative) general strategies for recruiting students. Justifies inclusion of any schools with less than 40% poverty (if applicable).

3 points Submits Attachment B (identifying schools). Narrative describes specific strategies for recruiting students; and justifies inclusion of schools with less than 40% poverty (if applicable). Majority of served schools demonstrate HIGH NEED (e.g., D/F schools; poverty rates greater than 50%)

Reviewer Comments: Attachment B submitted. All schools greatly exceed 40% threshold (i.e., 74-91% poverty ranges). All are Title I schools. While two of the five schools are currently D/F schools, all have high poverty levels and experience high crime rates within their respective neighborhoods. Targeted students for AYP Scholars (to be 21st CCLC funded) = ESL, eligible F/R lunch; C or below in literacy or math, not progressing at grade level on state assessment/local school assessment; exhibiting behavior concerns during school day. Other students may participate in traditional AYS program (fee-based – with the option of using CCDF or the AYP Financial Assistance Fund).

Score: 3 C-3. Dissemination of Information (2 points) Applicant describes how it will disseminate understandable and accessible information about the proposed 21st CCLC program to community stakeholders, including: a description of the services, the program location, and how to access the program.

0 points Information not provided

1 point Outlines general steps the applicant will take to disseminate general program information.

2 points Provides specific steps to disseminate detailed program information including: service description, program location, and how to access the program.

Reviewer Comments: Letters to parents of eligible students (details attendance requirements, enrollment procedures & general


10

AYP policies. AYS provides flyers to each site to advertise program, location, hours, contact info, activities, and eligibility criteria for 21st CCLC, snacks, staff, days of operation and details how to enroll. AYS also provides info for school websites/newsletters.

Score: 1 C-4. Communication with Schools (3 Points) Applicant describes its communication plans with schools that students regularly attend and regular-day teachers by addressing four key areas:

a) Equitably serving non-public school students and their families, if those students are

within the target population of the applicant’s 21st CCLC program; b) Accessing necessary student academic records to monitor objectives and provide

statewide evaluation data. • In order to ensure the confidentiality of student records, the LEA is responsible for

gathering achievement data and securing parental permission for use of data. • If the applicant is not an LEA, a Memorandum of Understanding (MOU), signed by

the authorized representative of the public/private organization and the school corporation Superintendent, or the Charter School Administrator, must be submitted with this application and the data gathering provision mentioned above must be included among the other commitments made by the LEA to the program. The MOU must be attached as an Appendix item.

c) Sharing information on student progress in the 21st CCLC program with : regular-day school staff; families of participating students; community stakeholders.

d) Alignment of in-school and out-of-school-time efforts to support student success

1 point Less than all four topics are addressed (nonpublic students; accessing academic records; sharing student progress; and alignment of in-school and out-of-school-time efforts). Zero points if none of 4 topics.

2 points All four topics are addressed (nonpublic students; accessing academic records; sharing student progress; and alignment of in-school and out-of-school-time efforts)

3 points All four topics addressed; and applicant demonstrates its strong understanding and commitment to appropriately obtain & use student data to inform efforts (e.g., specifies strategies for sharing information with teachers & parents; detailed MOU included in Appendix -- if applicant is not an LEA).

Reviewer Comments: P 35: 79 nonpublic schools invited; no response from any with intent to participate. AYS is lead partner (i.e., not LEA). AYP will access student academic, behavior and attendance records via each school’s designated Academic Liaison (stipend paid via grant funds) to coordinate sharing of data with AYS. Parents sign appropriate consent form during registration process. Classroom teachers provided with bi-weekly Scholar Update for each student in program (offering two successes and one challenge for each student, as identified by AYP staff). Teacher can provide feedback/comments to assist AYS. Parents also receive this update. Annual AYS Scholars advisory council meetings. Individual school site goals are school-specific (vary somewhat from school-to-school, based on needs/prior performance) to ensure continuous improvement. Site coordinators work with school personnel to determine needs of their students. Academic Liaison provides observation and feedback to AYS staff on program improvement and will help make connections


11

to faculty. Also, each school will designate one teacher to provide tutoring… thus another connection between program and school staff. Students will also have access to school day learning software through computer labs and/or school issued technology.

Score: 3

C-5. Parental Involvement, Family Literacy, and Related Family Educational Attainment (3 points)

The applicant describes how it will promote parental involvement, family literacy, and related family educational attainment activities for families. Key elements include: • Demonstration that family engagement is not a one-time event, but rather a set of day-to-day

practices, attitudes, beliefs and interactions that support learning both in- and out-of-school. • An evaluation of the community needs and resources for the community learning center. • Comprehensive, but achievable strategies, such as: family literacy initiatives, GED courses

or workshops that help prepare parents to support their child’s academic achievement. • Strategies that also support the needs of working families.

NOTE: If applicant’s priority points are based on Family Engagement: applicant also must minimally host 5 events annually, excluding parent courses; employ engagement strategies, such as home visits, interviews, surveys, newsletters, or family involvement curriculum)

0 points Information not provided.

1 point Plan describes at least one, solid activity to engage parents in the program.

2 points Evaluation of community needs/resources conducted; and multiple activities planned to engage parents

3 points Evaluation of needs/resources conducted; and multiple activities specified to engage parents; and needs of working parents considered.

Reviewer Comments: Fall parent survey identifies school-specific needs to guide resources highlighted by community service center presentations at weekly Community Connection Nights (Friday evenings). Second Helpings provide family dinners/program updates and offer student incentives for participation. United Way ReadUP training also provided, so parents can partner in AYS efforts to improve children’s ELA skills.

Score: 3

C-6. USDA Approved Snacks/Meals for 21st CCLC Participants (2 points) Applicants are encouraged (not required) to provide snacks and/or meals to all participating students. Applicants opting to do so are eligible for up to 2 points if: • Applicant clearly describes how snacks and/or meals will be acquired and distributed to

sites for participants; and • Applicant specifies that meals/snacks served will meet requirements of the US Department

of Agriculture (USDA) and the IDOE Office of School and Community Nutrition.

0 points Information not provided – or Applicant does not offer (optional) snacks/meals to program participants

1 point Only one of two required elements provided (i.e., how snacks/meals will be acquired & distributed to sites; OR specification that snacks/meals meet USDA and IDOE guidelines

2 points Both required elements included: how snacks/meals will be acquired & distributed; and that snacks/meals meet USDA and IDOE guidelines


12

Reviewer Comments: Both required elements addressed.

Score: 2

C-7. Weekly Schedule (5 points) The applicant must provide a tentative weekly schedule of activities proposed for the participating students and their families for EACH program site location (unless program the same at all sites). Key elements should include: • Schedule includes the total number of hours dedicated to student activities (and, as

appropriate, parent engagement) – and complies with the required minimum operational hours:

o 12 hours per week, 4 days per week for Elementary sites o 10 hours per week, 4 days per week for Middle School sites o 8 hours per week for High School sites

• Days/hours may be offered before school (1 hr.), afterschool (at least 2 hrs.), both before & after school (1+2 = 3 hrs.); non-school weekdays, e.g., Saturday (at least 4 hrs.)

• Elementary and middle school schedules should reflect activities that support academic, behavioral and recreational/enrichment opportunities.

• A separate schedule must be provided for summer or extended-break operation (e.g., spring break; intersession; etc.) – if center plans to operate during these times. Summer programs must operate at least 4 hours per day for 4 days per week (for a minimum of 4 weeks and not more than 8 weeks).

0 points Information not provided.

1-3 point range General weekly schedule provided that meets minimum hours of operation requirements for grade levels served. Applicant intends to also operate during summer OR extended-breaks, but did not submit separate weekly schedule.

4-5 point range Detailed weekly schedule provided for EACH site that meets minimum hours of operation requirements; Elem & MS schedules reflect diverse and engaging activities (academic, behavioral, enrichment/recreational); Separate schedules are provided for summer and extended breaks (if applicable).

Reviewer Comments: Mon-Fri before (6:30—8:00am for independent, child-directed stations/homework help, snack, recreation) & after school offered (3-6pm for computer labs/academic support/LitART/math & enrichment clubs & independent time). Friday (4:30-6PM) offer Open Mic and 5-6pm Community Connection night (dinners, Scholar Updates & presentations via community partners). Program operates 35 weeks, with hours exceeding minimum required for elementary out-of-school time programs (at 7.5/wk before and 15/wk after). Blend of independent & directed academic support, recreation and parental involvement opportunities. See p. 73+

Score: 5

C-8. 21st CCLC Learning Center Messaging (1 point) All applicants are required to refer to themselves as a 21st Century Community Learning Center and use the 21st CCLC logo on all program materials. Applicant describes how new terminology and logos will be incorporated into their program.

0 points No description for meeting the requirement

1 point Applicant describes how it will meet the requirement

Score: 1


13

VI. PROFESSIONAL DEVELOPMENT (5 POINTS) Applicant describes PD that is specific to all levels of program staff (i.e., director, coordinator, and direct-service staff), based on a needs assessment, and designed to enhance program quality and help the center reach its goals and objectives. Specifically, the applicant describes how: • PD needs of various staff members will be assessed. • Staff PD needs will be met. • PD will enhance program quality and align to the applicant’s goals and objectives. Directors and site coordinators are required to attend IDOE annual trainings and regional workshops (and at least one USDOE Summer Institute meeting within the four-year grant period). Program leaders and direct service staff also must receive PD aligned to their specific needs (e.g., cultural inclusion; STEM; safe & healthy youth; literacy; behavior modification, First Aid; family engagement strategies).


1-2 points range Includes one-dimensional description and plan for providing PD (e.g., focus is solely on staff attendance at State and national meetings or conferences – but no PD plan is articulated to support specific needs of center’s staff, aligned to its program goals & objectives)

3-4 point range Includes detailed plan for providing PD; connects PD to program quality and goals of project; PD strategies center around State/national workshops and trainings, but also include anticipated trainings (e.g., First Aid, vendor-provided trainings to support staff use of software instructional programs). May include a detailed chart of planned PD activities.

5 points Needs of program staff assessed and PD is a tiered-approach, addressing needs of specific staff roles (i.e., leadership vs. instructional needs). Multiple approaches will support needs (State & national workshops/conferences; and ongoing trainings to support locally-identified needs). Plan addresses initial kick-off, turn-over and ongoing training for new and veteran staff; connects PD to program quality and goals of the project; includes detailed chart of planned PD activities.

Reviewer Comments: All AYS staff trained on safety, behavioral, nutritional and core curriculum components. Tiered approach described (p 75) for administrative (director/site coordinators) vs instructional/support staff. Both pre-service (July) and mid-service trainings (January) offered annually to address possible staff turnover & maintain high skill levels among existing staff. 21st CCLC program would benefit from established practices in place via traditional AYS programming. All staff members will work toward Child & Youth Care Certification (with three already set to start the process in spring 2018. Individual PD needs identified through monthly check-ins between site coordinator and site staff and bi-weekly check-ins between program director and site coordinators. In addition to project mandated trainings, individualized training modules provided online by IAN Academy. Additional in-person training provided regularly throughout year through AYS’ Professional Development Center (wide variety of workshops in partnership with national, state and local organizations. Beyond State conferences, other anticipated program-specific academic trainings (p 76) include: LitART, Peace Learning Center/positive behaviors; PEAR Institute (lesson planning/enrichment clubs); Read UP/United Way for effective group tutoring/ability grouping), etc.


14

Score: 5


15

VII. EVALUATION (15 POINTS) A. Identification of Local Evaluator (3 points) Applicant identifies the individual and/or organization that will serve as its local evaluator for the program and describes their relevant qualifications. • Local evaluator must be an individual who is external to the 21st CCLC program and/or

partners. • Local evaluators generally possess advanced degrees and have previous knowledge or

experience in evaluation and research principals, including data collection, survey construction and research design. Strong analytical skills are needed, as well as demonstrated ability to write clearly and persuasively. Experience with out-of-school time learning a plus.

1 point Applicant intends to hire local evaluator, but entity not yet selected

2 points Local evaluator identified (external to the program) with evaluation experience

3 points Selected local evaluator with demonstrated expertise in data analyses, report writing, and afterschool program knowledge

Reviewer Comments: Limelight Analytics, supervised by Mindy Hightower King, Ph.D. Experienced evaluator (18 years), with primary work on after-school programs, school wellness, initiatives, workforce development and early childhood development. Has worked with over 250 sites in IN, KY, and OH. Also served 6 years a project director of the state evaluation of 21st CCLC in Indiana & Kentucky.

Score: 3

B. Evaluation Design (10 points) The description of the evaluation design should include: data to be collected; when it will be collected; what instruments will be used for data collection; and what steps will be taken to use evaluation data to drive program improvement. Key elements of design should include:

• Evidence of partnership between 21st CCLC program and its local evaluator (e.g., monitoring

observations conducted at program site/s; recommendations for improving program delivery; data meetings with program leaders; etc.).

• Identification of data to be evaluated annually; must minimally include the performance measures and assessments reflected in Section V (table of Goals, Objectives, Program Activities, Performance Measures, and Assessments) of the applicant’s proposal.

• Plan should specify who is responsible for gathering data for achievement, behavioral and parent involvement measures.

• Annual timeframe for local evaluation efforts, e.g., when site observations will occur; when assessments and surveys will be administered; when local evaluation report will be completed.

• How local evaluation findings will be shared among stakeholders (e.g., program and LEA staff; parents and youth) and used to inform adjustments needed to improve the program


16

0-2 point range Plan is not provided -- or of insufficient detail to convey understanding of local evaluation expectations

3-5 point range Some key elements are included in local evaluation design plan, but several descriptions are missing or vaguely presented

6-8 point range Plan demonstrates

understanding of expectations – with some key elements

better articulated than others. Applicant must address all

Section V performance measures & assessments to score in this

range (or higher).

9-10 point range Plan clearly articulated. Includes evaluator’s roles; addresses collection/analyses of all Section V performance measures & assessments; details eval implementation timeframes; and specifies how findings are shared and used to improve program

Reviewer Comments: P78+: Program staff will gather participant-level data (demographics, attendance, grades, assessments, behavior incidents, family attendance) and survey data (teacher, parent and student). Evaluator will conduct annual site visits to each program site and prepare annual evaluation report. Results will be presented twice annually (following site visits at mid-year; and at the end of each project year).

• Mid-year reports by evaluator to program director and site coordinators will identify high-quality program practices and opportunities for improvement.

• Outcome data (including performance measures) will be presented to program stakeholders at end of each project year in a written report & group presentation.

• Limelight Analytics will prepare and submit a detailed annual report to IDOE in July of each year (all required elements). At end of fourth year, final report to IDOE will aggregate data across all four years of the grant.

AYS has data sharing agreements with IPS, MSDDT and SENSE that involve a timeline of data collection and verification activities. Parents grant AYS permission to receive child’s info from school (during the enrollment process). Each school’s Academic Liaison oversees survey distribution/collection.

Score: 9 C. Annual Reporting (2 points) Applicant addresses its obligation to submit annual report/data collection for State evaluation and for federal reporting purposes:

At the end of each year of the program, the external local program evaluator is required to prepare and submit to IDOE a detailed report that includes the following information:

• Evidence of program quality (using Indiana’s After School Standards and Indiana Academic Standards);

• Student attendance trends; and • Progress toward each of its performance measures included in Section V.

All grantees must complete the Indiana Quality Program Self-Assessment (IN-QPSA) annually. The IN-QPSA is an online self-assessment tool that enables the out-of-school-time program to rate its performance based on the Indiana Afterschool Standards. (NOT reported to IDOE/US DOE) For State evaluation and federal reporting purposes, programs must submit student information such as grades, State assessment test scores (ISTEP+ or ILEARN), credits earned (high school students only) and teacher survey data (completed by the student’s regular classroom teacher). Grantees use IDOE’s data collection system (EZ Reports) to report these data and other information required by the US Department of Education (attendance, program activities, etc.).


17

0 points Information not provided. Applicant does not address its obligation to submit reports/data for both State and federal reporting

1 point Applicant adequately addresses at least one key annual reporting obligation, e.g., local program evaluator’s report submitted to IDOE at end of each program year (showing program quality evidence, attendance trends and progress toward performance measures)

2 points Applicant understands its obligation to submit reports/data to the IDOE (i.e., annual local program evaluator’s report with program quality evidence, attendance trends and progress toward performance measures; and data required in EZ reports). Grantee also uses IN-QPSA online self-assessment, to locally rate its performance.

Reviewer Comments: Annual site visits made to each program site to compare observed practices to quality-based IN Afterschool Standards. Interviews/site observations & self-assessment IN-QPSA data will inform external evaluator’s report to determine quality of programming. Outcome assessment will focus on extent to which program meets performance measures (Section V). Page 70 charts annual schedule of data collection elements and methods of reporting, including use of EZ Reports.

Score: 2


18

VIII. SUPPORT FOR STRATEGIC PRIORITIES (5 POINTS) Describe how the proposed project will address the Indiana Academic Standards, including English/Language Arts and mathematics achievement. Applicants have flexibility in their response. Some possible descriptive strategies might include:

• Proposed program is aligned with the school’s curriculum in the core subject areas of ELA and mathematics, as evidenced through routine collaboration with regular classroom teachers to inform academic focus during extended-learning-time.

• Proposed program is tied to the (specific) school improvement plan. • Program staff will participate with regular classroom instructors in PD aligned to the

school or district’s instructional strategies, to ensure coordinated efforts centered around attainment of Indiana Academic Standards.

• Proposed program using evidenced-based materials/software aligned to Indiana Academic Standards to support students’ academic improvement.

0 points Information

not provided.

1-2 points Applicant affirms that its program will align with Indiana Academic Standards but does not adequately convey how that will occur

3-4 points Applicant provides concrete examples of how its program will align to Indiana Academic Standards (e.g., collaborative planning between regular classroom teachers and extended-learning-time staff; evidenced-based software used for literacy support)

5 points Strong evidence (multiple strategies) provided supporting extended-learning-time program’s alignment with Indiana Academic Standards via routine coordination of planning, PD and academic efforts between program and school/district staff where students attend

Reviewer Comments: Examples of alignment to Indiana Academic Standards (IAS): Applicant notes that core programing component align with IAS. Students have access to school day ELA and math learning platforms, aligned with IAS. Tutoring provided by school teachers; homework assistance & skill-building provided by AYS staff. Staff training to use PEAR lesson planning tools, aligned with IAS. AYS staff communicates with school teachers via bi-weekly Scholar Updates.

Score: 3


19

IX. SUSTAINABILITY PLAN (5 POINTS) Applicant describes how 21st CCLC activities will be sustained, once grant funds are no longer available, to ensure continuation of services. This should include:

• Efforts to increase local capacity; • Specific future funding sources (e.g., general funds, Title I funds; plans to expand or

develop additional community partnerships). • Established goal for year one programming to increase capacity, sustainability and/or

available program resources (time, talent and treasure). 0 points

Information not provided.

1 point Outlines existing partnerships and a general plan for sustaining program levels beyond the grant.

3 points Outlines existing partnerships and potential partnerships; and identifies potential future funding sources (e.g., general funds/Title I)

5 points Outlines existing partnerships, expanding partnerships & potential partnerships; provides a well-conceived plan for sustaining program levels through increased local capacity and/or future funding sources. Establishes sustainability goal for Year One programming.

Reviewer Comments: Strategy to increase revenue to support programming, via minimal fee & sliding fee scale for families unable to afford ($5) fee begins in Year 1 and continues across entire grant period. Will work with some enrichment partners to offer special activities at a reduced cost/no cost. PD investments will ensure high-quality curriculum/activity implementation beyond grant period. AYS will open enrollment at 21st CCLC sites to students who do not qualify for program, but need high-quality out-of-school time support. They pay regular fees/qualify for financial assistance or CCDF vouchers. Sustain funding stream via CCDF, School Age Child Care grants, AYS’ Financial Assistance Fund (via United Way and fundraising initiatives including an annual event). Participate in CACFP to subsidize snacks. Continue to use already purchased curricula.

Score: 5


20

X. SAFETY AND TRANSPORTATION (5 POINTS) Applicant addresses safety issues, such as: • Required criminal background checks conducted for all 21st CCLC staff (retained on file and

kept confidential) • How the safety of children will be maintained on-site (e.g., requiring parent sign- out,

checking identification) and during off-site activities (if applicable) • How personnel hired to work at the center will meet the minimum requirements set forth by

the district or agency and that the personnel will have all required and current licenses and certifications, where applicable

• How a safe facility will be maintained through use of Indiana Afterschool Network Top Ten standards on Safety, Health and Nutrition.

• Programs located in facilities other than school buildings must demonstrate that the program will be at least as available and accessible as if the program were located in a school building. Such programs should include a Memorandum of Understanding related to facility including classrooms, cafeteria, gymnasium, computer labs and audio-visual equipment usage, etc.

Applicant addresses transportation issues, such as: • Describes the location(s) of the 21st CCLC and its activities and how students in the

program will travel safely to and from the center and home. • Describes how the program will meet the schedule and transportation needs of working

families. • Ensures that transportation is not a barrier to students’ participation.


1-2 point range Provides some general staffing requirements (e.g., criminal background checks) and commits to providing students’ transportation home after program

3-4 point range Demonstrates detailed program safety plan (background checks on file/confidential); district/agency staffing requirements met; required parent sign-in/out; MOU provided (if facility not located in school); and safe transportation provided to/from center and home that meets needs of working families

5 points Demonstrates detailed program safety plan (background checks on file/confidential); district/agency staffing requirements met; required parent sign-in/out; MOU provided (if facility not located in school); and safe transportation provided to/from center and home that meets needs of working families; and addresses use of IAN Safety Standards

Reviewer Comments: Rigorous screenings (criminal background check, sex offender registry & drug screen) required prior to employment with AYS. Staff trained on CPR/First Aid/Universal Precautions, Child Abuse and Neglect Detection, Health & Safety, Behavior Management/Conflict Resolution, Bullying Prevention & Reporting, etc. AYP’s policies follow the Indiana Afterschool Network Safety, Health & Nutrition standards.

1:15 ratios. Daily attendance; building lock-down/entry door bells. Parental/guardian sign in/out daily. Photo ID/written parental permission for those with whom staff is not familiar. Systems to maintain confidentiality in place (consent forms; health or special needs). AYS always operates on a model of parent drop off in mornings & parent pick up in the afternoon. IF transportation becomes a barrier for qualified students at any of the five school site, AYS will work with


21

school to use after-school activity buses/other methods of transportation. While peer reviewer appreciates long-held policies/practices of AYS’ model of parent drop off/pick up, 21st CCLC programs recognize that in high-poverty school attendance areas --serving students with high levels of educational need -- transportation is nearly always a barrier to students’ participation. No grant funds have been allocated in applicant’s budget to support transportation needs; unclear why school activity buses are not used as a general practice (unless those activity buses run considerably earlier than 6 pm).

Score: 3


22

XI. BUDGET FORM/NARRATIVE, DETAILS & SUMMARY (5 POINTS) Applicant must submit the entire Budget Workbook, comprised of: Instructions (Tab 1); Budget Summary (Tab 2); Budget Form/Narrative (Tab 3); and Details (Tab 4).

A. Budget Form (Tab 3 of Budget Workbook): This document, also known as the Budget Narrative, is where applicants describe their projected expenditure of funds. A breakdown of each line item with specific item detail is required on this form, including costs for: staffing; PD (IDOE/federal meetings & conferences, and local training initiatives; subcontractor services; transportation costs; evaluation (up to 6% of each annual grant award); data collection fee for IDOE ($800 or more); equipment & supplies; and optional indirect costs (restricted indirect cost rate, or the default rate of 8%).

• Expenditures described in budget narrative (Tab 3) must MATCH expenditures on Budget Summary (Tab 2).

• Budgets exclude in-kind donations which are shown in a separate attached document.

B. Details: Provides further breakdown of expenditures. The primary purpose of this document is to describe how the line item costs are reasonable in relation to the number of persons to be served and to the anticipated results.

C. Grant Budget Summary (Tab 2): This document automatically populates based on fields from the Budget Form (i.e., each line item’s total transfers to the same line item on the Grant Budget Summary form).

All costs should be reasonable and allocable.

• Examples of unallowable expenses include: entertainment (field trip without IDOE-approved academic support); preparation of proposal; purchase of facilities or vehicles; land acquisition; capital improvements/permanent renovations; refreshments/snacks (food purchases okay IF considered a “supply” for program cooking class); supplanting federal, State or local funds; membership dues.

• Examples of allowable expenses—with pre-approval by IDOE include: purchase of equipment (e.g., computers, laptops, DVD players, projectors; printers, scanners, phones, TVs, digital cameras, etc.); promotional/marketing items with 21st CCLC logo; staff events (e.g., retreats, lock-ins, etc.); out-of-state or overnight field trips with approved academic support.

• FYI to PEER REVIEWERS: Note any “unallowable” or “allowable expenses—with pre-approval by IDOE” in Reviewer Comments.


23

0 points Budget Form (Budget Narrative) not completed by applicant.

1-2 point range Some budget narrative pieces completed, but not all. Examples: (a) key anticipated costs not reflected in budget (e.g., evaluation and PD costs missing); OR (b) budget includes cost items not substantiated in proposal narratives; OR (c) excessive line items for equipment costs (without solid justification and intent to obtain IDOE pre-approval).

3-4 point range Budget narrative includes all anticipated line items (e.g., staffing, PD, evaluation, contracted services; transportation). Narratives adequately explain costs that are aligned to activities described in proposed RFP. Costs appear reasonable and permissible (and some items may require pre-approval by IDOE). Budget Summary is completed correctly and matches costs in Budget Form/Narrative.

5 points Exemplary budget narrative clearly articulates all anticipated line items (e.g., staffing, PD, evaluation, contracted services; transportation). Narratives summarize costs that are clearly-aligned to activities in the proposed RFP. All costs appear reasonable and permissible. No errors on Budget Summary; costs match those in Budget Form/Narrative.

Reviewer Comments: Includes all anticipated line items and matches costs described on page 31+ (Budget Summary Form). Key quantity descriptions (personnel/contractual tutoring) not shown in budget narrative, but referenced back to page 31 where descriptions appeared. These costs represent two-thirds+ of overall budget and should be adequately detailed. Note to IDOE: Confused by contractual tutoring costs ($8,750). This is for 2 hours/per week/per site. Do not find MOU to further clarify. This cost is separate from other coordinators/site staff shown in personnel budget. Is this for the tutoring provided by a regular classroom teacher at each program site? $1,500 equipment cost is for five rolling book carts (one for each program site). This is not a recurring budget item across the full grant period. No monies budgeted for transportation of students before/after program. No established indirect cost rate. Applicant used IDOE’s established default 8%.

Score: 4


24

XII. GRANT PROPOSAL ORGANIZATION (5 POINTS) Grant is organized and follows RFP directions; all materials requested are provided and in order. • Abstract no more than 2 pages • Program Narrative (excluding Abstract, Goals, Objectives & Performance Measures tables;

Evidence of Previous Success, Budget Workbook) cannot exceed 35 pages (benefit of doubt) • Proposal double-spaced, using 12-pt Times Roman font (tables/charts single-spaced/10pt font) •

0 points Not organized in prescribed format. Program Narrative section far exceeded 30-page maximum (i.e., 35 or more pages)

1-2 point range Grant materials are provided, but not in the sequence requested. Abstract exceeds 2 pages/Program Narrative section exceeds 35 pages; Did not double-space/use 12-point font.

3-4 point range Grant materials provided in sequence requested. Abstract and Program Narratives do not exceed maximum (2 pages/35 pages). Proposal double-space/12-pt font; and pages numbered with identifying headers on each page.

5 points Exceptionally well organized with materials provided in sequence requested. Abstract and Program Narratives do not exceed maximum (2 pages/35 pages). Proposal double-space/12-pt font; and pages numbered with identifying headers on each page.

Reviewer Comments: Exceptionally well-organized with materials provided in sequence requested. Abstract and narratives in 12-point font, double-spaced, pages numbered/headers on each page – and within established required page length. Easy for reviewer to locate all required Attachments, anticipated Narratives and required Appendices.

Score: 5


25

SUMMARY of PEER REVIEWER POINTS

I. Project Abstract (5 points) 5

II. Competitive Priority Points (10 points) 10

III. Need for Project (5 points) 5

IV. Partnerships/Collaboration (5 points) 4

V. Program Design and Implementation (30 points) 27

VI. Professional Development Plan (5 points) 5

VII. Evaluation Plan (15 points) 14

VIII. Support for Strategic Priorities (5 points) 3

IX. Sustainability Plan (5 points) 5

X. Safety and Transportation (5 points) 3

XI. Budget Narrative (5 points) 4

XII. Proposal Organization (5 POINTS) 5

TOTAL POINTS AWARDED (100 Points Possible) 90

Documents

expansion of existing program - Indiana · 2020-01-29 · • Subtract 1 point if abstract exceeds two pages; subtract 2 points if abstract exceeds three pages (and note this in Reviewer