Principal’s Guide to the Quality Review€¦ · Principal’s Guide to the Quality Review 2 Table of Contents Section Page Introduction to the Quality Review 3 Quality Review Rubric

Office of School Quality

Division of Teaching and Learning

Principal’s Guide

to the

Quality Review

2014-15

Carmen Fariña, Chancellor

Phil Weinberg, Deputy Chancellor for Teaching and Learning

Carolyn Yaffe, Deputy Senior Executive Director, Office of Policy and Evaluation

[email protected]

mailto:[email protected]

Principal’s Guide to the Quality Review 2

Table of Contents

Section Page

Introduction to the Quality Review 3

Quality Review Rubric 4

Ladder of Inference 5

Code of Conduct for Reviewers 6

2014-15 Quality Review School Selection Criteria 6

2014-15 Quality Review Process 7

Stages of the Quality Review 8

Stage 1: Pre-Review Work 9

Stage 2: School Site Visit 12

Stage 3: The Quality Review Report 17

Stage 4: Verification 18

Stages 5: Appeals Process 20

Appendix A–Indicator Look-Fors 22

Appendix B–School Self-Evaluation Form (SSEF) 28

Appendix C–Sample Letter to Principal 36

Appendix D–Sample School Review Schedules 37

Appendix E–Classroom Visitation Tool 40

Appendix F–Reviewer School Meetings 42

Appendix G–Quality Review Criteria for the 2014-15 Formally Assessed Quality Indicators

43

Appendix H–Quality Review Reporting Guidelines 44

Appendix I–Verification Form 45

Appendix J–Appeal Request Form 46


Introduction to the Quality Review

The Quality Review examines how the work in a school community impacts the quality of the instructional core across classrooms to prepare students for the next level; it aligns with the Department of Education’s articulated focus on preparing students at each level for college and career readiness. The Quality Review provides an opportunity for a school community to reflect on its improvement planning processes and self-evaluate how well, and systematically, its educators make instructional decisions in service of student learning. The 2014-15 Quality Review:

Deepens the work of improving the instructional core across classrooms

Incorporates language and expectations connected to the Citywide Instructional Expectations, including the integration of Common Core-aligned curricula and the use of the Danielson Framework for Teaching for frequent cycles of teacher feedback

Continues to focus on the quality of teacher team work around collaborative inquiry and how the analysis of student work is used to inform the design of tasks and pedagogy

Formally assesses five of the ten Quality Indicators across three Quality Categories (1.1, 1.2, 2.2, 3.4, 4.2)

Quality Review Framework

The framework for the QR rubric continues to align with the diagram above. The instructional core is the relationship between the student, teacher and content (e.g., academic tasks). For the instructional core to improve or maintain a high standard across classrooms within a school, the school’s culture and systems for improvement must facilitate efforts to increase and sustain quality.

*The 2014-15 Quality Review Report will formally report on these five Quality Indicators.


Quality Review Rubric

The Quality Review (QR) rubric assesses school systems, structures, and practices across ten indicators within three Quality Categories. The three categories and ten indicators are below:

I. Instructional Core across Classrooms

Curriculum (1.1)*

Pedagogy (1.2)*

Assessment (2.2)*

II. School Culture

Positive learning environment (1.4)

High expectations (3.4)*

III. Systems for Improvement

Leveraging resources (1.3)

Teacher support and supervision (4.1)

Goals and action plans (3.1)

Teacher teams and leadership development (4.2)*

Monitoring and revising systems (5.1)

Please see Appendix A for “Look-fors” for Well Developed practice across the five formally assessed indicators.

Focus on Instructional and Organizational Coherence

The Quality Review is an examination and evaluation of how all school systems and structures work together to improve student learning, as defined by the instructional core. See below for the key questions the reviewer seeks to understand:

What are the key decisions the school has made to impact the quality of what is taught and how it is taught across classrooms?

What led to those key decisions (what did the school community look at in the way of data and student work to arrive at those decisions?)

What does the school envision as the success of these initiatives? How will success be measured along the way? What has been the impact to date?


The Reviewers

Reviewers are drawn from a pool of educators including community and high school superintendents, Quality Review Directors, School Achievement Technology Integration Facilitators (SATIFs), instructional leaders in the DOE, and retired educational leaders. Each reviewer has a background in school improvement and receives training that equips him or her to effectively review and evaluate schools.

Reviewer Trainings

Trainings for reviewers around the Quality Review occur every other month. During trainings, reviewers use the QR rubric to collectively examine school documents and reflect on evaluation criteria across rating categories. City-wide Instructional Expectations have been and will continue to be integrated into trainings around the Quality Review, promoting a shared vision of school quality.

During a Quality Review, evidence is collected in the form of “data statements” or “low-inference observations.” In Instructional Rounds (2009)1, the authors assert that description must come before analysis, analysis before prediction, and prediction before evaluation. The Ladder of Inference diagram2, below, has been shared with all stakeholders, including reviewers and Network Leaders. To promote a shared understanding of a school’s evaluation across stakeholders, reviewers are intentionally trained to remain low on the Ladder of Inference when citing supporting evidence for any conclusion.

The Ladder of Inference

------------ 1

Senge, Peter M. The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday/Currency, 1990.

2 City, Elizabeth A., Richard F. Elmore, Sarah E. Fiarman, and Lee Teitel. Instructional Rounds in Education: A Network Approach to

Improving Teaching and Learning. Cambridge: Harvard Education Press, 2009.

I make Assumptions based on the

meanings I added

I add Meanings

(cultural and personal)

I select “Data”

from what I observe

Observable “data” and experiences (as a videotape recorder might capture it);

data statements

I draw Conclusion

s

I adopt Beliefs

about the world

I take Actions

based on my beliefs


Code of Conduct for Reviewers

All reviewers are committed to a Code of Conduct that guides their work; they are expected to adhere to the Code of Conduct during the review process. Principals should contact the Deputy Executive Director of the Office of School Quality if they feel the Code of Conduct has been violated.

The Code of Conduct requires that each reviewer:

Prepares thoroughly for site visits

Communicates clearly with the school ahead of time to set site visit schedules and reduce anxiety in a timely manner

Works with integrity, treating everyone with courtesy and respect

Minimizes stress, not over-observe staff or demand unreasonable amounts of paperwork or time

Undertakes training and development as required

Acts with the best interests and well-being of students and staff

Evaluates objectively and impartially, using low-inference observations

Consistently shares emerging issues with principals and other members of the school during site visits

Reports honestly and fairly, ensuring that evidence and conclusions accurately and reliably reflect the school’s practices

Accepts and complies with the monitoring and quality assurance policy

Respects the confidentiality of information

Submits all report drafts in a timely manner, taking into account constructive feedback from readers

Communicates clearly, frankly, and sensitively

2014-15 Quality Review Selection Criteria

This year, a subset of schools will receive a Quality Review based on transparent selection criteria. Schools that meet at least one of the following criteria will receive a formal Quality Review during the 2014-15 school year:

schools whose most recent Quality Review was conducted in 2011-12 or prior

schools whose most recent Quality Review was conducted in 2012-13, AND received a rating of Proficient, Developing or Underdeveloped on that review

schools whose most recent Quality Review was conducted in 2013-14, AND received a rating of Developing or Underdeveloped on that review

schools designated as Priority or Focus

schools in their first, second, or third year of existence

Please note: Schools that meet any of the criteria listed above and have an Interim Acting (IA) Principal

can receive a QR at any point throughout the school year, though reviewers will consider the time of the year that the visit takes place..

All principals will continue to receive two supervisory visits for their Principal Performance Review, as required by Education Law 3012c. As in 2013-14, both QRs and PPOs are supervisory visits.


2014-15 Quality Review Process

For the 2014-15 school year, although reviewers will simultaneously gather evidence for all ten Quality Indicators throughout the review process, reviewers will formally evaluate the key aspects of the school's work using criteria linked to how well school practices are aligned with five indicators of the Quality Review rubric (see Appendix A for “Look-fors” for Well Developed practices across the five formally assessed indicators). These five Quality Indicators will be written about in the school’s published report.

The Quality Review will be a one day process conducted by one reviewer in schools with fewer than 1,500 students. In large schools (1,500 + Students, multi-site or large comprehensive High Schools), the one day process will be conducted by two reviewers. D75 schools may experience a two day Quality Review with one reviewer.

The Quality Review process will not culminate in an overall rating of W, P, D or UD. This is

in alignment to the Department’s vision of de-emphasizing focus on overall categories of performance and pointing instead to the various aspects of school practices that support student learning and teacher development.

At the end of the one-day process, schools will receive preliminary ratings and verbal

feedback, including an Area of Celebration and an Area of Focus, for the five indicators listed below. Final, written, detailed reports will continue to be delivered to schools 6-8 weeks following the Review.

The following five indicators will be formally assessed in the 2014-15 Quality Review Process:

1.1 CURRICULUM: Ensure engaging, rigorous, and coherent curricula in all subjects,

accessible for a variety of learners and aligned to Common Core Learning Standards and/or

content standards

1.2 PEDAGOGY: Develop teacher pedagogy from a coherent set of beliefs about how

students learn best that is informed by the instructional shifts and Danielson Framework for

Teaching, aligned to the curricula, engaging, and meets the needs of all learners so that all

students produce meaningful work products

2.2 ASSESSMENT: Align assessments to curricula, use on-going assessment and grading

practices, and analyze information on student learning outcomes to adjust instructional

decisions at the team and classroom levels

3.4 HIGH EXPECTATIONS: Establish a culture for learning that communicates high

expectations to staff, students, and families, and provide supports to achieve those

expectations

4.2 PROFESSIONAL COLLABORATIONS: Engage in structured professional

collaborations on teams using an inquiry approach that promotes shared leadership and

focuses on improved student learning


Stages of The Quality Review

The 2014-15 Quality Review is conducted by one reviewer in one day. In schools with 1,500 or more students, or in District 75 schools, an additional reviewer or an additional day will be assigned. The following stages of the Quality Review apply to all schools, regardless of size:

Stage 1: Pre-Review Work

Stage 2: School Site Visit

Stage 3: The Quality Review Report

Stage 4: Report Verification

Stage 5: Appeal Process (if applicable)

Notification of a Quality Review date is sent to the school at least two weeks in advance of the visit. Included in the notification are several informational documents including Reviewer biographies, the Principal’s Guide and School Self-Evaluation Form (SSEF). Principals provide completed SSEFs to the reviewer within seven to ten days prior to the date of the review. The reviewer uses this information to provide a contextual background for the school visit. These observations will be recorded in the appropriate areas in the Reviewer’s Record Book and serve as reference points (data trails) during the review. In addition, reviewers can access a copy of the previous Quality Review report, the outcomes of the School Survey, a school data set, school demographics, Human Resources data, and the most recent student achievement data. Prior to the review, the reviewer, in consultation with the principal, drafts a schedule for the review. The schedule may be adjusted by either the school or reviewer during the initial meeting on the day of the review and can remain flexible as long as all essential components of the review are included and completed. During the school visit, the reviewer collects first-hand evidence in a Record Book, which contains documentation, notes, analysis, and concrete examples of evidence, and the main findings. At the Feedback Conference at the end of the review, the reviewer provides preliminary verbal feedback and a written form that provides a preliminary rating for each of the five formally assessed Quality Indicators and lists an Area of Celebration (AoC), an Area of Focus (AoF), and Additional Findings. Following the visit, the reviewer is responsible for producing a full written report which includes the Area of Celebration, an Area of Focus, and Additional Findings. Evidence to support findings will be included for each of the five formally assessed indicators. The reviewer will provide the rating for each indicator as aligned to the language of the Quality Review rubric. Where there are two reviewers, the lead reviewer is responsible for the written report.


Stage 1: Pre-Review Work

Each principal will receive an email from a member of the program team in the Office of School Quality at least 2 weeks prior to the review. That email will outline next steps.

The School Self-Evaluation Form (SSEF)

Each principal is responsible for emailing his/her completed SSEF to the relevant Office of

School Quality program associate/manager, copying the reviewer(s) in that email, no later

than ten (10) calendar days prior to the review day. (See Appendix B for a copy of the

SSEF).

Use evaluative language and focus your responses on how indicated practices

impact student outcomes and improve teacher practice

Include specific references to where evidence of the self-evaluation can be found in your

school community

When possible, use bullet points to list multiple evaluative points

Refer explicitly to the indicators of the Quality Review rubric

Logistics connected to SSEF:

Please submit the SSEF a minimum of ten (10) days prior to the Quality Review by emailing

it to the Office of School Quality program associate/manager who sent your initial review

notice. Please copy the reviewer(s) on the email.

Please limit the amount of information placed in the SSEF; it is not intended to be a

comprehensive analysis of every aspect of your school community, but rather, a starting

point to understanding the key practices, decisions, and goals that your school is focusing

on to improve outcomes for all students.

A highly effective SSEF will:

Draw on a wide evidence base and take the views of staff, students and parents into account

Be reflective and analytical, explaining the basis for actions and the resulting outcomes

Be evaluative, using selective examples to support the self-evaluation and clearly link cause and effect

Explain succinctly how the school has tackled the areas for improvement and the impact of these actions on teaching, learning and student progress

Provide an accurate current picture of your school, allowing the reviewer to see evidence of the work you have been doing to ensure instructional and organizational coherence

Data Provided to Reviewers Prior to the Review

The Office of School Quality will provide the reviewer with information from the school’s most recent Comprehensive Educational Plan (CEP) about demographics and performance at the school. The reviewer will also receive a talent profile with hiring and tenure information for all staff. Along with information from the SSEF, reviewers may use previous Progress Reports, Human Resources data, School Survey results, and Quality Review report(s) to develop questions that will be used during the review to gain a deeper understanding of the school’s practices. Reviewers may also access ARIS to further examine school data.


Connecting with the Reviewer

The Quality Review process creates an opportunity for an ongoing conversation between the principal and the reviewer. The reviewer will contact the principal at least ten (10) days before the review to preliminarily plan the visit and answer any questions related to the process. The reviewer has the option of contacting the principal via email or telephone. The following elements should be addressed during communications between the principal and reviewer before the review:

1. Introduction: Reviewer introduces him/herself, providing the principal with a brief summary of his/her pedagogical/reviewer experience

2. Clarification: Reviewer asks clarifying questions regarding the content of the school’s

SSEF; reviewer can request additional information such as the organizational chart and class/prep schedule

3. Scheduling: Reviewer and principal establish site visit schedule, not including selection of

specific classes or students, etc.; principal commits to emailing reviewer a final version of the schedule and class/prep schedule. The schedule may be adjusted by either the school or reviewer during the initial meeting on the day of the review and can remain flexible as long as all essential components of the review are included and completed.

4. Further clarification: Reviewer answers principal’s questions regarding process and

protocols, and refers the principal to the Quality Review Principal’s Guide

See Appendix C for a sample email correspondence.

Creating the Visit Schedule

The principal, in collaboration with the reviewer, will develop a proposed schedule for the review during initial conversations prior to the school visit. To facilitate scheduling, principals should provide the reviewer with a bell and school schedule that clearly indicate classes with subject area, room numbers, and both start and end times for each class period. See Appendix D for sample schedules. On the morning of the day of the review (in schools where there is one reviewer), the reviewer will communicate the four-six (4-6) classes that s/he will visit in the first round of visits for the day, and both reviewer and principal make any necessary adjustments to the proposed schedule. A conversation between the reviewer and the principal informs the principal’s selection of classes for the second round of classroom visits that take place during the day. In schools with over 1,500 students and multi-site D75 schools, there will be two reviewers, and the lead reviewer will communicate the six – eight (6-8) classes that they will visit in the first round of visits for the day, and the reviewers and the principal may make any necessary adjustments to the proposed schedule. A conversation between the reviewers and the principal informs the selection of classes for the additional classes visited during the second round. FAQ: Pre-review Work

Question: Can a Superintendent meet with the school leadership before the review? Response: The Superintendent can meet with the school community as long as the purpose is not QR “preparation” but clarification and/or part of on-going school improvement support. Consequently, these meetings should be scheduled and planned in collaboration with the school’s network team.


Question: What if a school does not submit its SSEF 10 days before the review? Response: The SSEF is an opportunity for the school community to frame its work and help the reviewer understand strengths. School leadership should make every effort to get the SSEF to the reviewer in a timely fashion. Question: What if the reviewer does not contact the principal 10 days before the review? Response: If the school has not heard from the reviewer 10 days before the visit, the QR team should be notified at [email protected], with a copy to the network leader. The reviewer will be contacted immediately. Question: Will reviewers have access to the school’s ARIS data? Response: Reviewers will have access to your school’s data in My Students and Reports. Reviewers will not have access to any of your school’s private communities in ARIS Connect, unless you provide them with access. Question: Can reviewers review schools they have previously reviewed? Response: Yes, this is a possibility. Question: Can reviewers take different approaches to the first phone call/email contact with the principal? Response: All reviewers should contact the principal 10 days before the review. Essential information is discussed and requested during the first communication (see below for expectations and sample). There may be some variability following the first contact because reviewers can continue to connect with the principal through email or phone calls with questions to prepare for the review. Question: Do reviewers always visit schools alone? Response: No, there are occasions for reviewers to be accompanied by shadow reviewers, mentors, or associate reviewers. Each of these roles is defined in the initial email to a principal which will also include a brief biography of the Lead Reviewer and any person (s) accompanying the Lead Reviewer. In addition, large high schools with more than 1,500 students as well as multi-site D75 schools will experience Quality Reviews facilitated by two reviewers.

mailto:[email protected]


Stage 2: School Site Visit (One or two reviewer)

Classroom Visits

The reviewer (s) and principal will visit classrooms together to look for evidence of instruction, student engagement, student work, and the use of data to inform instruction. The reviewer (s) will ultimately be responsible for selecting most of the classrooms s/he/they will observe during the classroom visits and should be accompanied by the school leader(s). During classroom visits, the reviewer (s) will record low inference observations on the Classroom Visitation Tool (see Appendix E). See below for guidelines around classroom visits:

The reviewer will visit 7-9 classes spending 15-20 minutes in each classroom. In the case of schools with 1,500 students or more and District 75 schools, there will be two reviewers who may visit 12-14 classes.

The reviewer (s) and the principal may schedule opportunities for conversation between classroom teachers and reviewer prior to the classroom visit. This exchange can provide context for the visit and allow teachers to articulate what the reviewer should expect to see.

The reviewer (s) may speak with students regarding the lesson and their work during the classroom visit.

Reviewers, through the principal, may request to view lesson plans from classroom visitations conducted during the Quality Review.

The reviewer and school leader will debrief classroom visits.

The reviewer will look for evidence of the school’s common beliefs about teaching and learning, as well as established instructional priorities.

Reviewers will use a classroom visitation tool that focuses on instruction, student engagement, student work, and the use of data to inform instruction.

Please see Appendix E for the Classroom Visitation Tool that will be used during the review.

Meetings

The reviewer will meet with the principal, parents, teacher teams, and students (in a large and/or small group setting). During these meetings, the reviewer will listen for evidence that the various stakeholders embody the mission of the school. The principal is expected to work cooperatively with the reviewer (s) to arrange the meetings explained below as part of the Quality Review visit schedule.

Meeting with the Principal

At the start of the visit, the reviewer (s) holds an initial meeting with the principal and (where appropriate), assistant principal (s), and/or other school leaders at the discretion of the principal. During this meeting, the reviewer (s) discusses the school’s self-evaluation with the principal and begins to gather evidence for each of the Quality Indicators using the questions in the Record Book as a guide.


Meetings with Students

In a review with one reviewer, the reviewer will select four to six (4-6) students, and share the list of identified students with the principal on the day of the review. Necessary adjustments will be discussed at that time. During the student meeting, students may be asked general questions related to Quality Indicator 3.4. In addition, students may also be asked to talk about specific work products, teacher feedback, how rubrics are used, how goals are used, and how they know what their next learning steps are in their classes (2.2). The principal should ensure that students come to the meeting prepared with work from current units and projects. In a review with two reviewers, there may be a small group and a large group student meeting. The focus of the large student group meeting may be Quality Indicator 3.4. In the small student group meeting, one of two reviewers may meet with four to six (4-6) students, and may have a discussion with them about their portfolios, notebooks, or other student work. In this meeting, students may be asked to talk about specific work products, teacher feedback, how rubrics are used, how goals are used, and how they know what their next learning steps are in their classes.

Meeting with Teachers

The reviewer (s) will meet with two teacher teams. Where possible, one of the meetings should involve a team reviewing student work and related teacher work, though reviewer questions may focus on the connection between student work and data as well as resulting teacher actions and/or curricular modifications. The reviewer (s), in collaboration with the principal, will select the teacher teams that will engage in a dialogue around the impact of collaborative inquiry on practice, sharing of evidence and implications for student learning. Teachers should be able to discuss how they use data to adjust instructional practices and strategies, plan for meeting student needs, and track student progress. Where possible, the reviewer (s) will observe the first teacher team meeting for part of the time allotted for the meeting, and then may pose questions to the team regarding what s/he observed and/or ask questions connected to other evidence requiring triangulation. Contingent upon the school’s in-house calendar, the reviewer (s) will opt for one of the following choices in order to minimally disrupt student learning:

In the case that teacher teams are typically meeting during the site visit, the reviewer(s) will observe each teacher team engaged in a collaborative inquiry process, and ask questions as needed. Time can also be allotted towards the end of this meeting for questions and responses.

In the case that teacher team meetings are not slated to occur as per the school’s internal schedule, the reviewer(s) and principal can schedule a large group teacher meeting, or two smaller teacher group meetings, or one of each. The purpose will remain capturing evidence regarding the effectiveness of teacher teams engaged in collaborative inquiry at the site.

Please see these resources regarding teacher teams with additional questions.

http://schools.nyc.gov/Academics/CommonCoreLibrary/ProfessionalLearning/ResourcesforTeacherTeams/default.htm


Meeting with Parents

The principal selects eight to ten (8-10) parents who represent the school’s diverse population and grade levels to participate in a discussion with the reviewer (s). The principal should facilitate the inclusion of the president of the Parent Teacher Association or Parent Association and one additional SLT parent member as participants in this group. The Parent Coordinator (PC) should not expect to participate in the Parent Meeting. If the PC is needed for translation or the parents request the presence of the PC, the PC can join the meeting as support; the focus of the dialogue will remain between reviewer and parents.

Meeting with the UFT Representative

If the UFT Representative was not present at one of the Teacher Team meetings, the reviewer will hold a brief meeting with him or her to discuss the quality of the systems and coherence of the school. The reviewer does not have a special set of questions for the union representative and the meeting should last approximately 15 minutes. Review of Curricular and Other School Documentation

The Quality Review rubric has no stance on what curriculum a school has selected or developed; whether a school has purchased curriculum or is developing its own. The work of the 2014-15 school year is ensuring that curricular materials are situated in a thoughtful trajectory of learning over the course of the year and are meeting the needs of students. The assessment of Quality Indicator 1.1 will focus on purposeful decision-making regarding a school’s curriculum implementation process, the effectiveness of planning to meet students’ needs, and the impact on student learning.

During the 2014-15 school year, for the purposes of evaluating Quality Review indicator 1.1, reviewers and evaluators may request the following instructional/curricular documents:

Lesson plans from classroom visitations conducted during the Quality Review and unit plans and culminating tasks that situate the lessons viewed during classroom visits

Student work that is yielded from lesson plans

Prior unit plans, culminating tasks and student work*

* Reviewers can ask for unit plans/tasks implemented to date and will take the time of the year that the visit takes place and the work underway in each school into consideration.

Feedback Conference Presentation/Protocol

The reviewer, principal, network leader (or other network team representative), and two to three (2-3) other key cabinet/leadership members can be present at this meeting. The meeting can remain between just the reviewer, principal, and network leader at the discretion of the principal. If the reviewer and/or principal anticipate that the Feedback Conference will be a difficult conversation, it is suggested that the reviewer, principal, and network leader gather briefly before the Final Feedback Conference to discuss keeping the meeting between just the three of them. The reviewer, as facilitator of this meeting, begins the session by thanking the community and leadership and gaining agreement on the norms for this exchange:

Maintain respectful dialogue

Focus on evidence and avoid assumptions


Build collaborative understanding

Avoid aggressive or defensive language The reviewer then describes the feedback protocol. Please note that there will be no written script to read aloud. (~3 minutes)

Feedback Conference Protocol

Step 1: Starting with the AoC, the reviewer reads the description of the Quality Indicator exactly as it appears in the QR rubric. Then s/he shares observed trends and supporting evidence to substantiate the rating. Next, the reviewer shares the rating s/he has determined for this indicator. (Ratings for individual sub-indicators that helped determine that indicator’s ratings are not shared.) (~3 minutes) After the AoC, the reviewer repeats this process for each of the three indicators that fall under Additional Findings, and ends with the AoF. (~10 minutes) Step 2: The reviewer provides the participants with a copy of the Preliminary Ratings Form which contains the provisional rating of each indicator and shows the chosen AoC and AoF. (~2 minutes) Step 3: The principal responds to the supporting evidence and ratings and selected Area of Celebration and Area of Focus. (~6 minutes) Step 4: The reviewer invites the Network Leader (or representative) and other participants to respond if they desire. (~2 minutes) Step 5: The reviewer may then invite the network leader (or representative) and other participants to offer evidence they believe was not taken into consideration in the various indicator ratings. The reviewer states that the expectation is that the current evaluation will stand, and that all feedback will be documented in the Record Book for quality assurance purposes. (~2 minutes) Step 6: The reviewer ends the meeting; the reviewer acknowledges commentary regarding school and network feedback, and then departs. (~2 minutes) Reviewers should provide verbal feedback prior to distributing the Preliminary Ratings Form. Reviewers should use the Feedback Conference section in the Record Book to record the main points about the feedback conference, including any notable agreement or disagreement among stakeholders. If there are any anticipated concerns or questions about the preliminary feedback session, reviewers should contact the Office of School Quality.

FAQ: School Site Visit

Question: Is there flexibility in the number of classroom visits? Response: The reviewer and principal should schedule at least 7-9 classroom visits (12-14 for a large comprehensive high schools or multi-site D75 schools). Question: How are classes selected? Response: Reviewers will select a range of classes to visit based on school goals and data analyzed before the review (in the SSEF, the CEP demographics, student outcome data available in ARIS and the Data Set, etc). Prior to the school visit, the reviewer (s) chooses all classrooms to be visited during the first round of visits and there is a conversation between the reviewer (s) and principal around the principal’s selection of classes for the second round of class visits. The reviewer will inform the principal of the classes to be visited during the first round on the day of the


review. Generally, the final selection of classes should include a variety of subjects, a range of grades and multiple categories, i.e. general education, ESL, special education, etc. Additionally, the reviewer (s) will work to ensure that a range of teacher experience is observed (that is, a diversity of novice and experienced teachers). Question: Can the parent coordinator participate in the Parent Meeting? Response: The parent coordinator (PC) should not expect to participate in the Parent Meeting. If the PC is needed for translation or the parents request the presence of the PC, the PC can join the meeting as support; the focus of the dialogue will remain between reviewer and parents. Question: What is the focus of teacher team meetings? Response: One of the meetings should consist of an observation of a team involved in inquiry that addresses the connection between student work and data and resulting teacher actions including pedagogical or curricular modifications. The other team meeting will be an interview of teachers representing at least two teams. Questions asked in the interview will be focused on teacher teams impact on teaching and learning.

Question: How are students selected for the student meeting? What’s the focus? Response: In a one reviewer review, the group of approximately 4-6 students will be selected by the reviewer based on the data provided for the review, in ARIS, and in discussion with the principal. The finalized selection can occur on the day of the site visit, and should represent a strategic range of learners. In this meeting, the focus will be primarily on Quality Indicators 2.2 (Assessment) and 3.4 (High Expectations), as well as a review of student work and a discussion of how feedback and instructional supports impact their learning. In a two reviewer review, additional 10-12 students will be selected for a large student group meeting, and each reviewer may meet with one student group. In this case, students should come prepared to answer similar questions regarding Quality Indicators 2.2 and 3.4, as well as broader questions pertaining to school culture.

Question: Do reviewers contact the school after the visit? Response: A reviewer may contact the school with clarifying questions, if necessary.


Stage 3: The Quality Review Report

The Quality Review Report is organized into five sections:

1) School Context - provides demographic data

2) School Quality Criteria - provides the ratings for the five formally assessed indicators and identifies an Area of Celebration (AoC) and an Area of Focus (AoF)

3) Area of Celebration - provides the findings, impacts and three to five bullets of supporting evidence which highlight an area in which the school does well to support student learning and achievement

4) Area of Focus - provides the findings, impacts and three to five bullets of supporting evidence which highlight an area the school should work on to support student learning and achievement

5) Additional School Quality Findings - provides the findings, impacts and three to five bullets of supporting evidence for each of the three remaining Quality Indicators

Within sections 3-5, each page shows the Quality Indicator and its rating, followed by Findings, Impact and Supporting Evidence (FISE) for the given rating.

The final report will include information about the school, indicator ratings, and bullets that articulate “Findings”, “Impact”, and “Supporting Evidence” for at least two of the three sub-indicators for each of the five indicators reviewed and rated. For each of the five indicators reviewed, the reviewer will include evidence of, or lack of evidence for the practice. The final section includes a page showing the rubric criteria and rating for each of the five quality indicators formally assessed.

Quality Review Reporting Guidelines

Reviewers evaluate the key aspects of the school's work using criteria linked to how well the school’s practices align with the 5 formally assessed indicators of the Quality Review rubric across the three quality categories (see Appendix G). Each school receives individual ratings (see Appendix H) for each of the 5 formally assessed indicators. In addition, one indicator is rated as an Area of Celebration (AoC) and one indicator as an Area of Focus (AoF).


Stage 4: Report Verification

Quality Reviews will be conducted according to the protocols included in this document. Principals may raise any concerns regarding the process or protocols with the reviewers so that they may be resolved as quickly as possible during the review. The Quality Review report will be e-mailed to the principal as a PDF file approximately eight (8) weeks after the last day of the school’s Quality Review. The principal has a maximum of ten (10) business days to reply to the Office of School Quality program associate/manager, either to confirm that the report is accurate or make any factual corrections. Once the ten business days have passed, the Quality Review report will be uploaded to the school's web page on the NYCDOE website unless an appeal has been filed. The appeal process is described later in this document.

Report Verification Review

Upon receiving the School Draft Quality Review Report, the principal is asked to read the report to check for any factual inaccuracies before the document is made public. This verification process is separate from the appeal process. We urge principals to read the written report carefully to check for any inaccuracies or items that may need editing regarding the factual information provided about the school. Some examples of these inaccuracies may include incorrect names of programs, factual and/or statistical school data, and/or language usage. The principal has until 5:00PM on the tenth (10th) business day after the report was sent to submit any requests for changes or modifications using the Report Verification Form (see Appendix I). Because the report will be made public after ten (10) business days, this deadline to make any corrections is non-negotiable. Please follow the instructions listed below when verifying information in order to ensure an expedient and thorough response from the Quality Review team:

Provide the page, paragraph, and text in need of correction

Provide any factual information required to amend the error

Email this document to the program associate/manager who sent the initial draft


Sample Report Verification Form

School Name: Sample School DBN: 01M111

Principal: A. Principal Date of Review: October 27, 2015

Page, Paragraph

Text in need of correction Factual information required to amend the inaccuracy

page 2, 1st paragraph

Boys account for 46% of the students enrolled and girls account for 54%. The average attendance rate for the school year 2009 - 2010 was 79.9%.

Boys account for 42% and girls are 58% Our 2013-14 attendance was 84.7%

page 4, 6th paragraph

The school purposefully partners with Trinity Hospital to provide the school with a full time in house counselor that works with the social-emotional needs of staff, students and families.

The counselor is a pediatric psychiatrist that works with students and families on a part-time basis.

Upon receipt of the Report Verification Form, the report will be amended within ten (10) working days. The final report will be sent via e-mail to the principal.

Please note that verification of data presented in the Quality Review report does not include appealing the ratings within the report. Information regarding appeals is provided in the section below.


Stage 5: Appeal process (if applicable)

If the principal wishes to contend any part of the Quality Review Report, s/he must submit a completed Appeal Request Form to the respective support staff by 5:00PM on the tenth (10th) working day of receiving a copy of the School Draft Quality Review Report. See Appendix J for the Appeal Request form. As there is no overall score, ratings for specific indicators must be appealed. For example, a school can appeal the developing rating they received for indicator 2.2 by providing rubric-aligned evidence and demonstrated impact. Evidence supporting the proposed rating change must be labeled with the appropriate indicator and each of the three sub-indicators. Evidence submitted should not include evidence that was already presented during a school site visit. Please follow the directions below to ensure an expedient and thorough response from the Quality Review Team.

The request for an appeal must come from the principal.

Complete the Appeal Request Form by 5:00PM on the tenth (10th) working day following receipt of the Quality Review School Draft Report.

Cite the specific indicator(s) being appealed.

Include the current rating found in the Quality Review School Draft Report and the proposed rating change.

Provide supporting practices that substantiate a change in the rating for the indicator(s) being appealed. These practices must appropriately align to the 2014-15 Quality Review Rubric and must address all 3 sub-indicators for any indicator included in the appeal.

Provide the evidence of impact. The evidence of impact should address how the actions taken by the school impact the outcomes in the school community. Principals should note that in order to provide evidence towards a rating of Proficient or Well Developed, evidence of impact on the instructional core is essential.

Documents submitted as evidence of practices and evidence of impact must be labeled to show the sub-indicator(s) they support. For example, if a school is appealing a rating of Developing for indicator 2.2, the school must provide labeled evidence that supports the proposed rating change (to Proficient) by demonstrating that each of the three sub-indicators (2.2a, 2.2b, and 2.2c) is Proficient, as reflected in the language of the Quality Review rubric.

A Quality Review Director will reach out to the principal and acknowledge receipt of the appeal within five (5) business days.

The Quality Review team will conduct a full investigation, contacting the lead reviewer, and evaluating all relevant documents (in particular the Record Book).

If the investigation requires a Director to make a visit to the school in order to observe additional data/facts, the principal will be contacted by the Quality Review team to set up an appointment.


Upon completion of the investigation, a written response will be sent to the principal along with the final Quality Review Report within 25 working days (longer if the QR team determines a school visit is necessary).

The network leader will be copied on this final correspondence to the principal.


Appendix A: Indicator Look-fors

2014-15 Quality Review School Quality Expectations

Throughout the year, Quality Review Directors collect “look fors” from reviewers and vet them to create a set of expectations for what a rating of Well Developed entails for a given Quality Indicator. The goal of these expectations is to foster a common understanding of practices aligned to that rating. Below are examples of expectations aligned to a rating of well developed on selected Quality Indicators. Quality Indicator 1.1 To be Well Developed, it is expected that:

School leaders and teachers can articulate how they ensure curricula are aligned to the CCLS and other content area standards; they can also articulate a chosen strategy for integrating the instructional shifts. These strategies have resulted in coherence across grades and subjects. Coherence is defined as a fluid connection and coordination between the topics students study in each subject within a grade and as they advance through the grades. (a)

The school has clearly defined criteria for what it means to exit a grade level and to attain the enduring understandings and key skills that ensure success in college and career. (a)

School leaders and teachers integrate the instructional shifts by making purposeful connections between the shifts and the topics in each subject—both within a grade and as students advance through the grades—so as to promote college and career readiness. (a)

School leaders and teachers can articulate how curricula, across and within grade levels, are aligned to the CCLS and scaffold student success to promote college and career readiness for all students. (a)

Rigorous habits and higher order skills—such as those that require students to create their own meaning, integrate skills into processes, and use what they have learned to solve real world problems—are identified, defined, and embedded within curricula and academic tasks coherently across grades and subjects. (b)

Curricula and academic tasks require students, including English Language Learners and students with disabilities, to think accurately and with clarity, identify and consider multiple meanings and interpretations, take and support positions, resist impulsivity and engage in disciplined inquiry and thought, use and adapt what they know, deal with ambiguity, and demonstrate their thinking in new learning situations. (b)

Office of School Quality Division of Teaching and Learning


Teachers across grades and subjects use student work and data to plan and refine curricula and academic tasks in order to cognitively engage all students, including lowest and highest achieving students. (c)

Habits, as follows, are explicitly embedded in classroom instruction and academic tasks: persisting, managing impulsivity, listening with understanding and empathy, thinking flexibly, metacognition, questioning and problem posing, applying past knowledge to new situations, thinking and communicating with clarity and precision, creating, imagining, and innovating, taking responsible risks, thinking interdependently, and remaining open to continuous learning. (b)

Curricula and tasks, across grades and subjects, challenge all students, including English Language Learners and students with disabilities, to think critically; instruction provides scaffolds to ensure students can demonstrate their thinking through the work products they are asked to create. (b)

School leaders and teachers provide a data-based rationale that identifies areas of growth or achievement gaps for all students, including English Language Learners and students with disabilities and other subgroups, and explain how curricula and academic tasks are planned and refined accordingly so that all students access curricula and tasks and are cognitively engaged at a level consistent with the academic expectations for that grade level or beyond. (c)

Curricula and academic tasks are designed to engage students, advance them through the content, and assess their understanding as evidenced by their work products. (c)

Quality Indicator 1.2 To be Well Developed, it is expected that:

Across a preponderance of classrooms, teacher practices consistently reflect and support school wide beliefs about how students learn best; teachers and administrators can articulate how those beliefs are informed by the Danielson Framework for Teaching, aligned to curricula, and shaped by teacher team and faculty input. (a)

Instruction, outcomes, strategies, and learning activities are derived from standards-based curricula and reflect school leadership’s espoused beliefs about optimal student learning situations; beliefs are influenced by the priorities of the Danielson Framework for Teaching and CCLS instructional shifts. (a)

Instructional student groups are organized thoughtfully and are varied as appropriate; they build on student strengths and incorporate student choice as appropriate to maximize learning. Plans for lessons or units are well-structured with appropriate pacing and time allocations. (b)

Lessons and teaching documents represent deep content knowledge, understanding of diverse students’ linguistic differences and other needs, and available resources (including technology) resulting in a series of learning activities that engage students in high level cognitive activity. The lesson and unit structure is clear and allows for different pathways to understanding according to diverse student needs. (b)


Teachers can explain how particular teaching strategies and instructional tasks address the needs of individual students and sub-groups (English Language Learners and students with disabilities, lowest third, and highest performers) by articulating how the task is designed and/or identifying examples of ways student learning is supported or extended. (b)

Teaching practices leverage strategies such as inquiry, project-based and collaborative learning, questioning, and discussions that promote high levels of thinking. Strategic use of scaffolding techniques (e.g., modeling, needs-based grouping, activating prior knowledge, effective use of graphic organizers, visuals, imagery, and technology, building academic vocabulary—all of which may be in the student’s native language or in English) provides multiple entry points to lessons and tasks for all learners including English Language Learners and students with disabilities. (b)

Across classrooms, teachers strategically use scaffolds, questioning, opportunities for choice, and other teaching practices to create a variety of ways for students to access the content, learning project, or task, and be supported in learning or extend it to different possible endpoints so that all students show mastery of the learning objectives and corresponding standards. (b)

Teachers across classrooms provide students with challenging learning tasks that require them to use critical thinking, analysis, and problem solving; tasks encourage inquiry, collaboration, and ownership among students. (b, c)

Teachers use a variety or series of questions or prompts to challenge students cognitively, advance high level thinking and discourse, and promote metacognition. These high quality questions encourage students to make connections among concepts or events previously believed to be unrelated and arrive at new understandings of complex material. Students formulate many questions, initiate topics, and make unsolicited contributions. Students themselves ensure that all voices are heard in the discussion. (b, c)

Students, across classrooms, produce work and engage in discussions that reflect critical thinking, creativity, innovation, and problem solving, as well as student ownership of the learning process. (c)

Ample student-to-student dialogue, using academic vocabulary and evidence-based accountable talk, is built into the lesson. Students can articulate what they are working towards, why it is important, and how they help determine the direction of lessons. (c)


Teachers and administrators articulate coherent reasons for assessment choices; assessments are aligned to CCLS and/or content standards in the curriculum. These choices deliver a range of data, some daily, some monthly, and some quarterly, to sustain collaborative inquiry and continuously improve instruction. (a)

Teachers collaborate on designing and/or modifying common grade-wide, curriculum-aligned assessments, rubrics, and grading policies that are customized to address data-defined student and sub-group needs. These tools are used by teachers and administrators to track progress towards goals across grades and subject areas and make instructional decisions. (a, b)


A variety of feedback to students, from both teachers and peers, is accurate, specific, and timely -advancing learning. (a, c)

Teachers in teams determine important topics to assess with common formative assessments. Teachers effectively “unpack” the standards and analyze the instructional shifts for those topics to pinpoint concepts and skills students need to know and be able to do. The validity and reliability of school level assessments are ensured through the consistent, collaborative structures for norming and interpretation of evidence used to evaluate student performance. (b)

Teacher teams agree on learning goals and benchmark performances for units, tasks, and courses prior to designing or using formative assessments to measure student mastery of the goals. (b)

Teachers and teams effectively analyze data to glean information about students’ progress and learning needs relative to the learning goals. (b)

Teachers accurately identify specific instructional responses to the data, which might include re-teaching content, changing instructional approaches to meet the needs of all students, and/or developing more challenging tasks/units. Adjustments to lessons/tasks are effective and teachers can explicitly cite the impact of their instructional responses/adjustments. (b, c)

Assessment criteria are written clearly, students are aware of and able to articulate them, and there is evidence that students have helped establish the assessment criteria according to teacher-specified learning objectives. (b, c)

All learning outcomes have a method for assessment; assessment types match learning expectations and are authentic with real-world applications as appropriate. Plans indicate student choice in assessments, student participation in the design of assessments for their own work, and modified assessments for some students as needed. (c)

Students are actively involved in collecting information from assessments and provide input. (c)

Teacher monitoring of student understanding during lessons is visibly active and continuous: the teacher is constantly “taking the pulse” of the class and makes frequent use of strategies (e.g., cold call, questioning for explanation, stop and jot prompts, parking lot, double entry journals, exit slips) to elicit information about individual student understanding and trends. (c)

Students consistently self/peer-assess against the assessment criteria (rubrics) and monitor their own understanding and progress either by taking initiative or as a result of tasks set by the teacher. Students are aware of their next learning steps. (c)


School leaders create elevated level of expectations for all staff, which is evidenced throughout the community through verbal and written structures (e.g., new teacher orientations, ongoing


workshops, staff handbook, school website) that emulate a culture where accountability is reciprocal between all constituents. (a)

The school has clearly defined standards for professional development (including professional development plans that incorporate staff input) and classroom practices embedding elements of the Danielson Framework for Teaching to ensure that learning for all stakeholders consistently reflects high expectations. (a)

School leaders and other staff members work as a team in study groups, planning sessions, and other professional development modes, establishing a culture of professionalism that results in a high level of success in teaching and learning across the school. (a)

Staff members implement effective strategies for communicating high expectations (e.g., tasks encouraging inquiry, collaboration, and ownership) that are clearly connected to college and career readiness so that all students are challenged to meet or exceed those expectations. (b)

The school orchestrates ongoing events and creates multiple opportunities to partner with and engage families in learning, fostering their participation in a culture of high expectations connected to college and career readiness, and offering them feedback on their children’s progress towards meeting those expectations. (b)

The school provides ongoing, clear lines of verbal and written communication (e.g., online progress reports, parent/teacher conferences, parent informational sessions and workshops, parent handbook, student handbook, student led conferences) with families to deepen their understanding of college and career readiness expectations for their children and empower them to help support their children in meeting or exceeding those expectations. (b)

Teachers and other staff have a set of clear, systematic structures (e.g., advisory, guidance, college counseling) for articulating high expectations and sharing information with students, leading to student progress towards mastery of CCLS and college and career readiness expectations. (c)

Staff members have instituted a culture for learning that provides all students, especially those in high-need subgroups, with focused, effective feedback including clear next steps that determine student accountability for learning goals and expectations to prepare them for their next grade while ensuring their ownership of the learning process. (c)


The vast majority of teachers collaborate in professional teams where they develop and implement school wide instructional practices, embedding the CCLS and instructional shifts to continuously promote improved achievement for all learners. (a)

Teacher teams clearly articulate how they implement structured professional collaborations using protocols (e.g., Looking at Student Work, Tuning, Notices and Wonderings) and other structures to strengthen teacher capacity as they create, revise, and /or adopt curricula to ensure effective integration of the CCLS and instructional shifts into instruction across grades and content areas. (a,b)


School leaders and teachers have built a culture of professional collaboration (e.g., team-initiated inter-visitations, lesson study) in which they share insights relative to the coherence of teacher pedagogy, thus fostering improvement of outcomes for all learners. (a, b)

Teacher teams effectively implement systems to monitor a variety of student data and classroom practices that inform instruction leading to the achievement of goals for individual as well as groups of students. (b)

Teacher teams provide a data-based rationale and analysis of student work that inform their decisions to adjust teacher practice and create strategic goals for groups of students. (b)

School leaders and teachers offer specific and clear examples of teacher leadership that illustrate how teachers and teacher leaders play a vital role in school level decision-making. (c)

Administrators and teacher leaders (e.g., team leaders, coaches, mentors, cabinet members, instructional leaders, department chairs) are able to identify distributed leadership structures that are deeply-rooted in the school’s day to day operations and articulate how they serve as a conduit for teacher input in strategic decisions that affect student achievement. (c)

Sources:

Ainsworth, L., & Viegut, D. (2006). Common formative assessments: How to connect standards-based instruction and assessment.

Schmidt, W., Hsing C., & McKnight, C. (2005). Curriculum coherence: an examination of US mathematics and science content standards from an international perspective.

Stiggins, R. J., Arter, J., Chappuis J., & Chappuis, S. (2004). Classroom assessment for student learning: Doing it right, using it well.


Appendix B: School Self-Evaluation Form

Quality Review

School Self-Evaluation Form 2014-15

Name of principal:

Name/number of school:

School address:

School telephone number:

Principal’s direct phone number:

Principal’s email:

Number of years as principal of this

school:

Quality Review Conceptual Framework 2014–2015

The 2014–15 framework for the Quality Review (QR) rubric aligns with the diagram below. The

instructional core is the relationship between the student, teacher and content (e.g., academic tasks).

For the instructional core to improve or maintain a high standard across classrooms within a school, the

school’s culture and systems for improvement must facilitate efforts to increase and sustain quality.

* The 2014-15 Quality Review Report will formally report on these five Quality Indicators

Office of School Quality Division of Teaching and Learning


Dear Principal:

The School Self-Evaluation Form (SSEF) is an important part of your review experience. It is

designed to help you, your school community and your Quality Reviewer focus on how your school

systematically organizes around improving teacher practice and student achievement in order to

ensure our system’s goals of college and career readiness for all students.

The Quality Review rubric has been streamlined to focus specifically on the ways in which all the

work in your school community ultimately serves the goal of moving students towards college and

career readiness. As such, three quality categories have been identified:

1. The instructional core across classrooms

2. School systems for improvement

3. School culture

A successful integration of these categories is evidenced in the instructional core, as illustrated in

the graphic on the previous page. The instructional core is the ultimate substance of the Quality

Review process and should be considered your primary lens for completing this form.

Guidance on completing the form:

Use evaluative language and focus your responses on how indicated practices impact student outcomes and improve teacher practice

Include specific references to where evidence of the self-evaluation can be found in your school community

When possible, use bullet points to list multiple evaluative points

Refer explicitly to the indicators of the Quality Review rubric

Draw on a wide evidence base and take the views of staff, students, and parents into account

Be reflective and analytical, explaining the basis for actions and the resulting outcomes

Logistics connected to the SSEF:

Please submit the SSEF a minimum of ten (10) days prior to the Quality Review by emailing it to the Office of School Quality program associate/manager who sent your initial review notice. Please copy the reviewer(s) on the email.

Please limit the amount of information placed in the SSEF; it is not intended to be a comprehensive analysis of every aspect of your school community, but rather, a starting point to understanding the key practices, decisions, and goals that your school is focusing on to improve outcomes for all students.

With these points in mind, please answer the questions below. Thank you in advance for taking the time to thoughtfully complete this form.

Respectfully, The Office of School Quality


I. Development of the School Self-Evaluation Form Briefly describe how this SSEF was created, considering the following:

What process did you use to collect multiple perspectives?

Who was involved and what were their roles?

II. School Goals

Describe your school wide goals and action plans that convey the efforts undertaken to impact student achievement. Consider the following:

What are your school’s CEP goals?

What additional instructional goals, if any, are prioritized this year?

Why did you choose these goals?

What are the intended outcomes of these goals?


III. Instructional Core (1.1–Curricula)

Evaluate how your current school community aligns curricula to the Common Core Learning Standards (CCLS) and/or content standards including the instructional shifts, and the impact of this work to date. Consider the following:

Steps taken to align the curricula to the Common Core Learning Standards and/or

content standards and instructional shifts

Promotion of higher order thinking skills

Refinement of academic tasks to provide all students with access to the CCLS

IV. Instructional Core (1.2–Pedagogy)

Evaluate how your current school community incorporates the Danielson Framework for Teaching and CCLS instructional shifts into teacher practice, and the impact of this work to date. Consider the following:

The school’s instructional focus and use of the Danielson Framework for Teaching

Teaching practices that provide multiple entry points, supports and extensions to all

students

Instructional practices that promote high levels of student thinking and participation


V. Instructional Core (2.2–Assessment)

Evaluate how your current school community aligns assessments to curricula and analyzes data to improve student learning, and the impact of this work to date. Consider the following:

Alignment of assessments to curricula

Using assessment data to adjust curricula and instruction

Use of ongoing checks for understanding and student self-assessment

VI. School Culture (3.4–High Expectations)

Evaluate the current initiatives the school community is undertaking to promote a culture of high expectations for all students and the impact of this work to date. Consider the following:

Efforts to communicate high expectations to all of the school’s constituents

Establishing partnerships with families to support students’ progress towards college and

career readiness

Feedback to students regarding college and career readiness


VII. Structures for Improvement (4.2–Teacher Teams)

Evaluate the current structures and activities that inform professional collaborations, and indicate how the school ensures the effectiveness of teacher teams as well as the impact of this work to date. Consider the following:

Teacher teams promoting the implementation of the CCLS and the instructional shifts

Use of data within teacher teams to improve instruction

Structures to support distributive leadership

VIII. Highlights and Special/Unique Features of your Academic Program If there are any highlights and promising practices that are not captured fully thus far, please describe them here, stating how they currently contribute to organizational and instructional quality and coherence within your school. If there are specific features unique to your academic program, including site specific terms or definitions of practices, please add them here.


IX. Optional

Please share any other information you believe is critical to understanding your school

community’s context, if it has not been referenced in any of the preceding sections.


X. Description of Multi-site and District 75 Schools

If you are a principal of a Multi-site or District 75 school, please complete the form below.

Site Locations

Site Contact

Service Categories

Formative Assessment(s)

Summative Assessment(s)

Instructional Programs Specific to Site

Main Site

Annex 1

Annex 2

Annex 3

Annex 4

Annex 5


Appendix C: Sample Letter to Principal

Dear Principal _______________, I will be conducting the Quality Review at your school on ________. I am very much looking forward to visiting your school. I am reaching out to you now to give us plenty of time to prepare. Thank you for submitting your SSEF. Please develop an initial schedule in preparation for the review and e-mail it to me by _______. It is likely that adjustments will be made to the schedule and I would like to give us a chance to discuss them. I will also need a copy of your bell schedule, organizational sheet and preparation schedule. When developing the initial schedule, please include the following events:

A meeting with you, for about 60-90 minutes at the start of the day Two teacher team meetings, one 30-45 minute meeting to observe and interview a teacher team, and

a second 20-30 minute meeting to facilitate a Q & A with teachers from at least two teams. The UFT Chapter Leader should be invited to the Teacher Team meeting that is a Q & A, unless the UFT Chapter Leader is a member of the Teacher Team that is being observed.

A meeting with approximately 4-6 students to review their best work. I will provide you the names of these students on the day of the meeting.

A meeting with 8 to 10 parents of your choice, including your Parent Association President and at least one other SLT parent if possible, for approximately 30 minutes.

7-9 class visits for 15-20 minutes per visit. I will choose the morning classes and you will choose the afternoon classes to visit. During classroom visits I would like to observe a full range of grades and subjects, across both tenured and untenured staff.

One half hour for an document review for the reviewer One hour for reviewer reflection before the feedback meeting The remainder of the time will be used for further discussions with you and others to cover all aspects

of the review. I recommend that you visit the Quality Review website for any additional resources you may need. You can access the Quality Review website by clicking here. I have also attached a copy of the Principal’s Guide for your reference. I have included a sample schedule below for the visit. It will need to be adjusted to fit your school schedule and the availability of school community members for the required meetings. The order of events is just an example and not a required flow of the day.

[Letter will include the appropriate review schedule. Please see Appendix D.]

After you have sent the documents requested above, I would like to find a time to speak with you early next week, perhaps sometime on _________ morning or afternoon. Please let me know your availability as well as a number where you can be reached. If you feel that you need additional information regarding the review or if you have any questions regarding the process, please feel free to contact me.

I look forward to working with you. Sincerely,

Reviewer



http://schools.nyc.gov/Accountability/tools/review/default.htm


Appendix D: Sample School Review Schedule

Planning the Visit Planning and scheduling in advance of a Quality Review helps ensure that the review is conducted in accordance with established protocols and procedures and that the review causes minimal interruption to the school community. Knowing the review schedule in advance also allows the principal to make arrangements for parents, teachers, and students to be available to meet with the reviewer. The protocol for determining the review schedule is outlined in greater detail in the Principal’s Guide to the Quality Review. The chart below outlines the required and optional elements of a Quality Review with a time frame for each activity. It is up to the principal to ensure that the final schedule matches the school’s start and end time and takes into account the bell schedule. Following that chart are some sample schedules to provide ideas on how the site visit may be structured.

Required Events for All Reviews Optional Events

At reviewer’s discretion, in consultation with principal

Meetings with principal

Two teacher team meetings

Student group meeting

Parent meeting, exclusive of Parent Coordinator

Classroom Visits* 7-9 in schools with <1,500 students 12-14 in D75 schools and schools with

1,500+ students

Document review

Reviewer reflection time

Two principal debrief sessions, mid-day and end of day

Feedback conference

Site tour

Observation of the end of the school day (dismissal)

Observation of extended day/after school activities

Additional teacher team meetings

*The reviewer and principal may schedule opportunities for conversations to occur between classroom teachers and the reviewer prior to the actual classroom visits. The pre-visit teacher/reviewer exchange is to provide context for the classroom visits. It is expected that reviewers will visit classrooms with a school leader and debrief each visit for a few minutes. The first set of classrooms (4-6) is selected by the reviewer and the second set (2-3) is selected by the principal.




The following chart provides guidance around timing for Quality Review events:

Sample Schedules

Please note that each school’s schedule may be different based on bell schedules and site-specific programs. See below for sample schedules for three types of schools:

I. Sample Quality Review schedule for a school with fewer than 1,500 students

The actual sequence of events will be determined by the reviewer and the principal during the Pre-QR phone conference. The time allocations noted above are recommendations, but can be negotiated based on the school’s schedule.

*The UFT Chapter Leader should be invited to the Teacher Team meeting that is a Q & A, unless the UFT Chapter Leader is a member of the Teacher Team that is being observed. As an alternative, the reviewer, in consultation with the principal, may schedule a meeting with the UFT Chapter leader for approximately 15 minutes.

Event Approximate Time Frame

Meeting with principal 60 – 90 minutes

Classroom visits 15 – 20 minutes per classroom

Teacher team meetings 30 – 45 minutes / 20 – 30 minutes

Student meeting 30 – 45 minutes

Parent meeting 30 – 45 minutes

Reviewer reflection 75 – 105 minutes

Debrief with leadership 30 minutes

Feedback conference 20 – 30 minutes

Time Schedule Notes

60 – 90 minutes Meeting with leadership May include members of leadership cabinet

60 – 90 minutes Classroom visits (4-6) AP of Instruction may join,

Reviewer selected

30 – 45 minutes Teacher Team Meeting:

Observation

Teacher Team Meeting observed for 15-20 minutes followed by a Q & A

If the school cannot accommodate a team for logistical reasons, this meeting can be used for a Q & A with teachers representing at

least two teams.

30 – 45 minutes Reviewer reflection and document review (lunch)

15 minutes Mid-day debrief with leadership Document request

Discuss 2nd

round classroom visits

30 – 45 minutes Meeting with students

(4-6 students) Discussion and review of work products

Two students selected by school

20 – 30 minutes Teacher Team Meeting: Q & A Q & A with teachers representing at least two teams

selected in collaboration with principal. UFT Chapter Leader should be invited*

60 – 90 minutes Classroom visits (3) School selected

30 – 45 minutes Parent meeting (8-10 parents)

15 minutes Final debrief with leadership Share final noticings from the day

45 – 60 minutes Reviewer reflection and

document review

Rate the five Quality Indicators Select Area of Celebration (AoC) and Area of Focus (AoF)

20 – 30 minutes Feedback conference Principal may invite key stakeholders


II. Sample Schedule for a school with more than 1,500 students

Schools with more than 1,500 students will experience Quality Reviews facilitated by two reviewers.



Time Lead Reviewer (LR) Co-Reviewer (CR) Notes

60 – 90 minutes

Meeting with leadership May include

members of leadership cabinet

30 – 45 minutes

Classroom visits (2) (w/ Principal, AP and both reviewers) Initial joint observation & debrief are

done for norming, Reviewer selected 5 – 10

minutes Class visit debrief norming

(w/ Principal, AP and both reviewers)

45 – 60 minutes

Classroom visits (3) w/ Principal

Classroom visits (3) w/ AP

Reviewer selected

10 minutes Classroom visit debrief Classroom visit debrief

45 – 60 minutes

Teacher Team Meeting: Observation

Document review


If the school cannot accommodate a team for logistical reasons, this meeting can be used

for a Q & A with teachers representing at least two teams.

30 minutes Reviewer reflection and document review (lunch)

30 minutes Mid-day debrief with leadership


Document Request. Discuss 2

nd round classroom visits.

30 – 45 minutes

Meeting with students: Small group

(4-6 students)

Meeting with students: Large group

(10-12 students)

Can be merged into one student meeting of 6-8 students,

two from each group selected by school

30 minutes Teacher Team Meeting:

Q & A Document review

Q & A with teachers representing at least two teams

selected in collaboration with principal. UFT Chapter Leader should be invited*

30 – 45 minutes

Classroom visits (3) w/ Principal

(school selected)

Classroom visits (3) w/ AP

(reviewer selected)

10 – 15 minutes

Classroom visit debrief Classroom visit debrief

30 – 45 minutes

Parent meeting (8-10 parents)

Document review

15 minutes Final debrief with leadership


30 minutes Reviewer reflection and

document review

Rate the five Quality Indicators Select Area of Celebration (AoC) and

Area of Focus (AoF)

20 – 30 minutes

Feedback conference Principal may invite key stakeholders


III. Sample Schedule for District 75 multi-site schools

DAY 1

DAY 2



Time Schedule Notes



Reviewer selected


(4-6 students) Discussion and review of work products

Two selected by school


15 minutes Mid-day debrief with leadership Document request

Discuss 2nd


30 – 45 minutes Teacher Team Meeting:

Observation


If the school cannot accommodate a team for logistical reasons, this meeting can be used for a Q & A with teachers representing at

least two teams.


30 – 45 minutes Parent meeting (8-10 parents)


document review

15 minutes Debrief with leadership Share noticings from the day

30 minutes Transition time Time to be embedded in the schedule to allow for

transitioning from one site to another

Time Schedule Notes



Reviewer selected

20 – 30 minutes Teacher Team Meeting: Q & A Q & A with teachers representing at least two teams

selected in collaboration with principal UFT Chapter Leader should be invited*


15 minutes Mid-day debrief with leadership Document Request,

Discuss 3rd



(4-6 students)

Discussion and review of work products Two selected by school


15 minutes Final debrief with leadership Share final noticings from the day


document review

Rate the 5 Quality Indicators Select AoC and AoF

20 – 30 minutes Feedback conference Principal may invite key stakeholders

30 minutes Transition time Time to be embedded in the schedule to allow for

transitioning from one site to another


Appendix E: Classroom Visitation Tool

Anecdotal Evidence: Low-Inference Observations

What is the teacher doing?

What are the students doing?

Consider the Following:

1. How do the instructional practices align to the school’s beliefs about how students learn best and to the Danielson Framework for Teaching? (1.2 and 4.1)

2. How is each student being engaged and challenged? (1.1 and 1.2)

3. How are teachers checking for understanding and adjusting instruction accordingly? (2.2 and 1.2)

Classroom Visitation Tool

Date: Teacher:

Grade (Circle) P-K K 1 2 3 4 5 6 7 8 9 10 11 12

Subject, time, # students Subject: Time: # of students:

Type of class ( ) Gen Ed. ( ) Spec. Ed. ( ) CTT ( ) ELL/ESL ( ) Other:

Lesson portion viewed (Circle)

Beginning Middle End

Teaching Experience This is the teacher’s __________ year of teaching.


Highlights and Promising Practices observed, including the arts and technology:

Notes from debrief with administrator: You may pose and consider the following questions during the debrief: What is the evidence of school-wide beliefs?

What are the strengths and areas of improvement for the lesson? What feedback would you give this teacher? How have you been supporting this teacher?

Summary Notes and Questions:

Other Classroom Noticings (i.e., student work, teacher/student comments, etc.)


Appendix F: Reviewer School Meetings

Meeting Participants selected by

Focus/Evidence Principal participates in meeting

Duration Participants

Teacher Teams

Reviewer

The reviewer is looking and listening for evidence of the:

Collaborative inquiry work resulting in adjustments to instruction, curriculum, and assessment;

Teacher leadership; and

Coherence of pedagogical practices.

No

30–45 / 20-30 minute periods

Teacher Team: Observation can be composed of a group of members of one team.

Teacher Team: Q&A can be composed of a group of members from at least two teams.

Student meeting

Reviewer and Principal

The reviewer is listening for evidence of the:

Students’ articulation of a culture of high expectations of the school;

Students’ use actionable feedback to implement next steps in their work product; and

Students’ awareness of their academic standing.

No 20–30 minutes

4–6 students from a variety of grades

Parent meeting Principal

The reviewer is listening for evidence of the:

Parents’ awareness of the school’s expectations for college and career readiness; and

Parents’ awareness of their child’s academic standing.

No 20–30 minutes

8–10 parents (Exclusive of Parent Coordinator)

Principal’s meeting

Principal

The reviewer is looking and listening for evidence of the:

Principal’s clear understanding and explanation of organizational and instructional coherence, as they translate into academic achievement for all students as well as the impact of these systems and decisions

Yes as per schedule

The principal and key members of the cabinet

Debrief (2) Principal

The reviewer has a conversation with the principal about: 1 – What has been observed; 2 – What s/he needs to observe in the rest of the day; and 3 – Suggestions for additional documents and evidence.

Yes

Two (2) at 15 minutes each

The principal and one or two key members of the cabinet

Feedback conference

Principal The reviewer shares the preliminary ratings for the five formally assessed indicators.

Yes 20–30 minutes

The principal, one or two key cabinet members, and a network representative


Appendix G: Quality Review Criteria for the 2014–15 Formally Assessed Quality Indicators

School Quality Criteria Instructional Core

To what extent does the school… Area of: Rating:

1.1 Ensure engaging rigorous, and coherent curricula in all

subjects, accessible for a variety of learners and aligned to

Common Core Learning Standards and/or content

standards

1.2 Develop teacher pedagogy from a coherent set of beliefs

about how students learn best that is informed by the

instructional shifts and Danielson Framework for Teaching,

aligned to the curricula, engaging and meets the needs of

all learners so that all students produce meaningful work

products

2.2 Align assessments to curricula, use on-going assessment and grading practices, and analyze information on student learning outcomes to adjust instructional decisions at the team and classroom levels

School Culture


3.4 Establish a culture for learning that communicates high expectations to staff, students and families, and provide supports to achieve those expectations

Systems for Improvement


4.2 Engage structured professional collaborations on teams using an inquiry approach that promotes shared leadership and focuses on improved student learning


Appendix H: Quality Review Reporting Guidelines

Reporting The report is organized into five sections:

1) School Context - provides demographic data

2) School Quality Criteria - provides the ratings for the five formally assessed indicators and identifies an Area of Celebration (AoC) and an Area of Focus (AoF)

3) Area of Celebration - provides the findings, impacts and three to five bullets of supporting evidence which highlight an area in which the school does well to support student learning and achievement

4) Area of Focus - provides the findings, impacts and three to five bullets of supporting evidence which highlight an area the school should work on to support student learning and achievement

5) Additional School Quality Findings - provides the findings, impacts and three to five bullets of supporting evidence for each of the three remaining Quality Indicators

Within sections 3-5, each page shows the Quality Indicator and its rating, followed by Findings, Impact and Supporting Evidence (FISE) for the given rating.


Appendix I: Report Verification Form

Quality Review: Report Verification Form 2014–15

School Name: DBN:

Principal: Date of Review:

Page, Paragraph

Text in need of correction Factual information required to amend the inaccuracy




Appendix J: 2014-2015 School Quality Review Appeal Request Form

2014–2015 School Quality Review Appeal Request Form The Quality Assurance process is designed to ensure that Quality Review reports are rooted in the rubric, factually accurate, and reflect the evidence gathered during the review, with fidelity. Since each report undergoes a rigorous, multi-step process, it is expected that appeals will be rare. Once initiated, however, each appeal will be carefully considered by the Office of School Quality. A principal may appeal the rating of any Quality Indicator included in the School Quality Review draft report. An appeal is opened when a principal submits this Appeal Request form. Appeal requests should be submitted within ten (10) business days of the date of receipt of the School Quality Review draft report, along with evidence that supports the appeal, to the Office of School Quality program staff member who sent the Quality Review draft report to the principal. Evidence submitted should not include evidence that was already presented to the reviewer during the school site visit. Please refer to the Principal’s Guide for more details on the School Quality Review appeal process.

School Name: DBN:

Principal: Date of Review:

Reviewer: Date of Appeal:




Indicator(s) and Rating(s) Supporting Practices Evidence of Impact Documents

List the appealed indicator, the current indicator rating, and the proposed indicator rating.

Indicate the practices for all three sub-indicators, aligned to the 2014-2015 QR Rubric, that support the proposed rating. Practices are specific actions your school engages in to improve achievement.

Describe the impact of the supporting practices on student performance and/or professional practice.

List each document submitted to support the proposed rating. Be sure to identify the sub-indicator(s) to which each document is aligned. Bear in mind that there must be evidence for each of the three sub-indicators linked to the appealed indicator.

Indicator: ___________ Current Rating: ______ Proposed Rating: ____

a. a.

a.

b.

b.

b.

c. c.

c.


Indicator(s) and Rating(s) Supporting Practices Evidence of Impact Documents

List the appealed indicator(s), the current indicator rating, and the proposed indicator rating.

Indicate the practices for all three sub-indicators, aligned to the 2014-2015 QR Rubric, that support the proposed rating. Practices are specific actions your school engages in to improve achievement.

Describe the impact of the supporting practices on student performance and/or professional practice.

List each document submitted to support the proposed rating. Be sure to identify the sub-indicator(s) to which each document is aligned. Bear in mind that there must be evidence for each of the three sub-indicators linked to the appealed indicator.

Indicator: ___________ Current Rating: ______ Proposed Rating: ____

a.

a.

a.

b.

b.

b.

c.

c.

c.