Guidance for Completing Interim Report I

Evaluation Webinar Series3 Dec 2013

Housekeeping

Online Tool Available Now!• http://www.sogosurvey.com/k/SsQUUVTsSsPsPsP

Webinar Format• Review Questions / Provide Guidance

Tips & Additional Assistance• Town Halls (Open Q & A Forum)• Webinars (Evaluators & TX21st)• ACE Help Desk

Texas ACE Evaluation ConsultantMary Ann Spracher

Component 2:Implementation Practices

Resources Implementation Outputs - Activities Outputs - Participation Intermediate Outcomes Impact HUMAN (All staff and others that work in your program - include names of key personnel) Include: # of students and families you propose to serve SUPPORT (Partners, vendors, stakeholders, funders, and so forth) CURRICULUM (Be specific: Name the curriculum) OTHER? Special kinds of facilities, field trips that are related to program activities and outcomes

School Program Alignment(How is your program specifically aligned with the regular school day.Recruiting and Retaining (right students, right mix of students)(How do you recruit & retain students/families and how do you determine which to recruit? What systems, strategies, processes, people, and tools do you currently use?)Integrating Student & Family Voice(How do you include student / family voice & choice at the macro (center) level and at the micro (activity) level?)Ongoing Monitoring (data use and observation)(How do you currently monitor your program? What tools do you use? How often)Professional Development(What kinds of professional development do you and your staff participate in & how often?)

Academic Support Name of activity - brief description of what you are working on. EnrichmentName of activity - brief description of what you are working on. Family EngagementName of activity - brief description of what you are working on. College and CareerName of activity - brief description of what you are working on.

Activity name, targeted number of students, & instructor (if certified teacher or other please indicate here). List the total number of days & hours for the semester for each activity (i.e. Homework Help - 233 students, 70 days, 70 hours)

Increased Attendance Increased Academic Performance Decrease Problem Behavior Increase Promotion Rates Increase Graduation Rates Anything else you currently measure would go here as well EXAMPLES:Increased student engagement, family engagement, increased focus of control, school bonding, family bonding and so forth depending upon what you are wanting to achieve.

ALL STUDENTS GRADUATE READY FOR COLLEGE AND CAREER

Evidence

At a minimum, these Implementation Practices include evidence of the following 5 essential components:

• School Program Alignment• Recruiting and Retaining participants (Students

and their families are recruited based on at-risk profile – and we know how and why we are recruiting the “right” kids and “right mix” of kids) • Integrating student and family voice and choice• Ongoing Monitoring (data use and observation)• Professional Development for program personnel

Your ACE ProgramThe intention of this this exercise is to provide you with a current snapshot of your implementation practices. Once thoroughly complete, it will provide you with insight that will enable you to address gaps in your program.

• By responding YES to any question in this report, you are saying that you can produce clear evidence that these practices are currently in use today.

• Answering NO to a question does not mean that your program is in jeopardy, but rather that there are elements of your current implementation practices that are evolving.

Completing the Report

The best way to complete this report, is to refer to column 2 of your center logic model.

Remember, the information is only relevant to your program if the practices you have in place in your prime blueprint are actualized into the everyday practices at the center.

School Program AlignmentThe questions of this essential component of Implementation Practices are asking you to consider how the out-of-school-time program is aligned with the regular school day.

School Program Alignment

Question 1Do leadership and staff of TX ACE and the regular school day leadership team have a shared understanding about student needs?

Question 2Is there a shared understanding of the direction that will be taken to address identified students’ needs between the regular school day and ACE program?

Question 3Do your activity selections reinforce students’ mastery of academic learning objectives (TEKS)?

School Program Alignment

Question 4Do ACE program personnel and the regular school day leadership and/or other school personnel engage in communication about the ACE program on a regular basis?

Question 5Do ACE program personnel and the regular school day personnel participate in shared learning experiences?

Question 6Is there shared ownership of problems and solutions (i.e. regular day, ACE and community)?

Recruiting and Retaining Program Participants

The questions of this essential component of Implementation Practices are asking you to consider the data, processes, and strategies you use and develop to recruit and then retain students into theACE program so that you are able to effectively meet your targeted number of students.

Recruiting and Retaining Program Participants

Question 1Do you use data regularly to guide decision-making about recruiting students into the ACE program?

Question 2Do you use data regularly to guide decision-making about retaining students in the ACE program?

Question 3Do you currently use processes, and strategies to recruit and retain students into the ACE program?

Recruiting and Retaining Program Participants

Question 4Do ACE program personnel, regular school day personnel, partners, and community members, each have a specific role related to recruiting and retaining students in the ACE program?

Question 5Are you currently serving the targeted population of students and families, what I am referring to are students most in need of academic assistance?

Question 6Do you aim to recruit and retain a diverse group of students into the ACE program?

Integrating Student & Family Voice and Choice

The question of this essential component of Implementation Practices is asking you to consider how you currently integrate the opinions and desires of the students and their families about what they should or want to be doing in the ACE program.

Integrating Student & Family Voice and Choice

Question 1Do you integrate student and family voice and choice into the ACE program?

Examples are:– A Student survey asking their opinions about various aspects of

their experience in the program.– A Family survey and follow up report explaining how you used

their feedback to improve the program– Student and family focus groups about how they feel and what

they think about the ACE program.– Students have opportunities to express choice and share their

voice in regard to what and how they are participating.

Ongoing MonitoringThe questions of this essential component of Implementation Practices is asking you to consider how you monitor your program in on ongoingway throughout the entire academic year. What tools and resources do you currently use tomonitor your program both at the “macro” (across all sites and overall site monitoring) and at the“micro” (individual program activities and staff) levels.

Ongoing Monitoring

Question 1Do you currently use data to regularly to drive decision-making about program and activity development?

Question 2Do you currently use observation protocols or checklists to assure fidelity of program and activity implementation?

Question 3 Do you currently have a strategy to share program data regularly with students, families, regular school day personnel such as the school leadership and or principal, teachers, school board, other relevant, and community members?

Professional Development

The questions of this essential component of Implementation Practices is asking you

to consider how you incorporate and derive benefit from the ongoing professional

development of all the people who work directly and indirectly with the students &

families that you serve.

Professional Development

Question 1Do you use data regularly to guide decision-making about program staff skill development?

Question 2Do you currently have processes in place for responding to the skill development needs of program staff?

Question 3 Is or was professional development and training effective in improving program staff skills?