Embed Size (px)
Citation preview
WORKING TOGETHER TO IMPROVE SCIENCE EDUCATION
PRESENTED BY GIBSON & ASSOCIATES
A CALIFORNIA MATH AND SCIENCE PARTNERSHIP RESEARCH GRANT
WISE II Evaluation Presentation2008-09
Presentation Overview
General information about the WISE II program
Discussion of evaluation methods
Challenges and how we addressed them
Our preliminary data results.
WISE II Partners
San Francisco Unified School District
San Francisco State University
Cal Academy of Sciences
City College of San Francisco
UC Museum of Paleontology
WISE II Program Description
Based on the WISE I program, WISE II is a three-year research grant for teachers in grades 3-5 to improve their science content knowledge, pedagogical content knowledge, and pedagogy.
Teachers attend a summer workshop, 3 release days, and a Saturday session. Teachers receive in-classroom coaching and lesson planning support 1 to 4 times a month and meet collaboratively with their teams to plan curricula, discuss student achievement, and develop lessons.
Challenging Courses and Curricula for Teachers and Students
CaMSP defines challenging courses and curricula asaligned with state content standards and
frameworks,resulting in a greater number of students
participatingand succeeding in advanced courses.
How do you evaluate challenging courses and curricula for elementary students and teachers?
Teacher Data Collection Tools
Teacher Data Collection Content Knowledge Assessment- Nationally Derived
Tool Additional Content Assessments – Locally Derived
Tools Pedagogical Questionnaires – Pre, Mid-Year, and Post Professional Development Feedback Forms -
Ongoing Classroom Observations –twice a year Student Notebooks Collected- twice a year Use of a Control Group
Evaluation MethodsTeacher Data Collection
National Content Knowledge Assessments
Local Content Assessments
Pedagogical Questionnaires
Formative Professional Development Feedback
Pre-Post Test content knowledge test Assessing Teacher Learning About Sc
ience Teaching ATLAST by Horizons Research.
Strength- piloted and tested and provides useful information about teacher practice and knowledge.
Challenge-not CA standards, not directly aligned to PD, and it’s difficult – so teachers do not enjoy taking it.
Evaluation MethodsTeacher Data Collection
National Content Knowledge Assessments
Local Content Assessments
Pedagogical Questionnaires
Formative Professional Development Feedback
Complement ATLAST with other content knowledge instruments:
Standards-based pre-post twice a year.
Faculty developed pre-post instruments given the day of the treatment. Teachers receive immediate feedback on what they have learned that day.
Evaluation MethodsTeacher Data Collection
National Content Knowledge Assessments
Local Content Assessments
Pedagogical Questionnaires
Formative Professional Development Feedback
QuestionnairesPre-Post & Formative Evaluations of
PD
Pedagogical questionnaires examinefrequency of practice & confidence in knowledge and instruction. They look at growth by individual teacher and area. We compare it to our control group.
After every PD session, we provide feedback forms that include questions on relevancy to standards and to instruction.
Control GroupNot receive any similar treatment
Willing to be in program for 3 years
Willing to take additional student assessments
Their own confidence in science teaching and instruction
Stipend for participation
Control teachers for WISE II Not receive any WISE II
“treatment” Complete ATLAST pre-post tests. Complete pedagogical
questionnaires. Two or more teachers at site
participating in the program. Students complete the pre-post
tests. Teacher commits to 3 years. Likely to be in district and at grade
level for three or more years- and their school willing to participate.
Control Group
Easy:
a. Have not participated in the program
Hard:
a. Have similar characteristics of those in the program.
ChallengeCompeting programs and
reformsControl group more
experiencedHigher baseline of performance.
Resolution View data in terms of growth. Growth in student scores and
growth in competencies, confidence, and practice.
Student Data Collection Tools
Student Data Collection Pre-Post Standards Based Content
Knowledge Assessment Additional Content Assessments –used
locally by teachers California Standards Test – review both
ELA and Science Classroom Observations – conducted twice
a year Student Notebooks Collected twice a year Control Group to compare growth
Evaluation MethodsStudent Data Collection
Pre-Post TestsCST Data Student
NotebooksClassroom
Observations
STUDENT Pre-post tests in science. Look at growth in student
outcomes on California Standards Test.
Use rubric to analyze student science notebooks content, structure, and understanding.
Observe engagement and learning process through classroom observations using a rubric.
Biggest challenge- lack of 3rd, 4th grade CST in science.
A SAMPLING OF PRELIMINARY DATA FOR
WISE II
RESULTS
CHANGES IN CST PERFORMANCECHANGES IN PRE-POST TEST
REVIEWED IN CONTEXT OF GRANT
Student Outcomes
WISE teachers improved student science outcomesAs compared to Control Group and the district classrooms, WISE II teachers experienced greater growth in science than their peers between 2008 and 2009.
In particular, more WISE II teachers had fewer students in below basic bands on the CST and more students were proficient than the previous year.
Student outcomes in Science on the CST WISE II teachers showed an increase on the 5th grade Science CST from the baseline 2008 year to 2009 year.
Pre-Post Test ResultsReview of student growth and average growth in Earth, Life, and Physical Science at each grade level and as compared to control group.
Other student outcomes Treatment teachers had
average double digit gains on pre-post science tests in every strand (earth, life, physical).
Student results on Pre-Post tests analyzed by growth and by WISE program offerings and curricular practices.
Example of Pre-Post Analysis
3rd grade areas of greatest change on the pre-post test of WISE II students
Standards Learned
80% of 3rd grade WISE II areas with greatest growth – pre-to-post test were addressed in WISE.
These topics were also seen in student observations and drawings in student notebooks.
ES.4.b Students know the way in which the Moon's appearance changes during the four-week lunar cycle.
ES.4.d Students know that Earth is one of several planets that orbit the Sun and that the Moon orbits Earth.
PS.1.a Students know energy comes from the Sun to Earth in the form of light.
PS.1.e Students know matter has three forms: solid, liquid, and gas.
CONTENT KNOWLEDGECONFIDENCE
PRACTICE
Teacher Changes
Teacher outcome resultsWISE II teachers improved content knowledge based on pre-post assessments.
Control Group teachers declined in content knowledge.
WISE II Teacher Content knowledge
WISE II teachers made growth based on the local –standards-based assessment tool.
Treatment teachers increased quantity of instruction.
93% of treatment teachers reported teaching a minimum of 46 minutes per week, as opposed to 78% of the same teachers the previous year.
Treatment teachers increased use of effective science teaching strategies.
Note-few changes for control group in these areas.
WISE II increased teacher confidence
Based on baseline and post-treatment surveys, WISE II teachers were more confident in their use of the following instructional strategies:
Teaching Comprehension
Vocabulary
Sketching and Drawing
Increased confidence and application
WISE II teachers showed an average growth in frequency and confidence of application of effective instructional strategies in every area analyzed.
WISE II improved more than control group teachers.
WISE II teachers improved their confidence and competence at a greater rate than the control group teachers in nearly every area.
WISE II – Other Data Collected and Analyzed
Observation Data Student Science NotebooksFeedback forms from EvaluationsTeacher Comments Faculty Input-Changes
WISE II – Challenging Courses and Curricula
For more information, Contact Nada DjordjevichGibson & [email protected] ext. 207
