1
Milwaukee Mathematics Partnership
Program EvaluationYear 6 Results
Carl HanssenHanssen Consulting, LLC
Cindy WalkerUniversity of Wisconsin-Milwaukee
November 2009 MTL Meeting
2
Where are we?
MMP efforts have been ongoing since 2003
This is the 6th year we have been conducting the evaluation
We anticipate continuing effortfor several more years
3
MMP Support
Original funding from NSF Started in 2003-04 Currently in 7th year
MMP Phase II funding from NSF Awarded last fall for 3 years Focus on research and evaluation
State of Wisconsin Provides funding for released MTL positions
4
Evaluation Goals
Help the MMP better serve its constituents and improve its effectiveness
Serve the broader mathematics education community through documentation and dissemination of MMP success factors
5
Key Evaluation Question
What are the critical factors or conditions promoted by the MMP that are related to student achievement?
6
Agenda
1. MMP Involvement
2. Social Network Analysis
3. Learning Team and Math Meeting Observations
4. Mathematical Knowledge for Teaching
5. Conclusions
6. Next Steps
7
Thank You!
142 schools provided data for the MMP Online Survey producing a record response rate!
114 schools provided social network analysis data!
25 schools allowed us to observe meetings!
Many others have provided data for a variety of other measures!
8
1. MMP Involvement
Metrics:
MMP Involvement Attendance at MTL Meetings MMP Courses
WKCE Student Achievement Growth from 2005-2008 2008 Achievement
9
MMP Involvement & GrowthSchools that have been more involved over time demonstratedhigher student achievement growth from 2005 to 2008:
Ave
rag
e %
Gro
wth
Number of Schools: high (n=34), medium (n=91), low (n=26), none (n=27)
10
MMP Involvement & 2008 Achievement
Schools that are more involved demonstrated higherstudent achievement in 2008:
Ave
rag
e %
Pro
fici
ent
Number of Schools: high (n=34), medium (n=97), low (n=31), none (n=36)
11
2. Social Network Analysis
We asked you to list individuals with whom you communicated about mathematics education in the past several months.
For each individual:1. Do they work at your school?2. Their role3. Frequency of communication4. Extent of collaboration
12
Key
Maps identify
MTL or MTS or Teachers Principal Literacy Coach Others in school Others outside school
Statistics
Network density In-School density
MTL In-Degree MTL Betweeness MTS IN-Degree
Benefit—a graphical AND statistical description ofschool-based networks.
13
Hi Density
School 09G
08 Students 7608 Passing 5508 Percent 72.4%5 year trend 20.1%SNA Respondents 16SNA Total Named 34Network Density 17.3%In School Density 35.8%MTL In Degree 23.48MTL Betweeness 5.50MTS In Degree 1.13
14
Lo Density
School 09F
08 Students 13008 Passing 4608 Percent 35.4%5 year trend -11.2%SNA Respondents 10SNA Total Named 35Network Density 3.4%In School Density 4.8%MTL In Degree 20.00MTL Betweeness 0.00MTS In Degree 2.35
15
Hi MTL In-Degree
School 09E
08 Students 14608 Passing 6908 Percent 47.3%5 year trend -31.8%SNA Respondents 23SNA Total Named 37Network Density 13.8%In School Density 24.2%MTL In Degree 49.44MTL Betweeness 40.73MTS In Degree 1.66
16
Lo MTL In-Degree
School 09I
08 Students 41808 Passing 22408 Percent 53.6%5 year trend 2.1%SNA Respondents 19SNA Total Named 57Network Density 4.6%In School Density 6.5%MTL In Degree 6.07MTL Betweeness 10.32MTS In Degree
17
Hi MTL Betweeness
School 09J
08 Students 33508 Passing 16408 Percent 49.0%5 year trend 6.2%SNA Respondents 32SNA Total Named 54Network Density 5.1%In School Density 7.5%MTL In Degree 38.05MTL Betweeness 49.22MTS In Degree 0.63
18
Hi MTS In-Degree
School 09C
08 Students 20208 Passing 7308 Percent 36.1%5 year trend -0.4%SNA Respondents 22SNA Total Named 34Network Density 16.4%In School Density 24.3%MTL In Degree 55.15MTL Betweeness 37.41MTS In Degree 24.24
19
So what?
A combination of factors…all are necessary but nosingle factor is sufficient
Math Focus
LT Quality
NetworkDensity
MTL In-Degree
MTLBetweeness
Predicts
StudentSuccess
Discriminant function analysis Indicates that this combination of factors predicts achievement quartiles
20
3. Meeting Observations
Overall, LT meetings tended to focus more on ‘administration’ than ‘learning’ though higher performing teams retain more of an emphasis on learning
Math meetings more focused, yet less team oriented
Common observations from math meetings: Real work is typically accomplished (e.g., scoring CR items)
Substantive mathematical discussions take place
Implications for classroom practice are rarely discussed as a basis for wrapping up
21
4. Mathematical Knowledge for Teaching
Two groups of teachers took the MKT assessment in 2008-09
Math Teacher Leaders Self-selecting Math Teachers
22
MTL MKT—2008-09 ResultsGrade 8-9 MTLs consistently score higher than the K-7 MTLs on both pre-tests and post-tests…especially in Geometry and Prob & Stat.
0.170.24
0.36
0.47
0.24
0.35
-0.17
0.29
0.420.47
0.71
0.52
0.90
0.46
-0.40
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
Pre Post Pre Post Pre Post Pre Post
Number Algebra Geometry Prob & Stat
K-7 Mean 8-9 Mean
*
*
23
MTL Compared with Classroom TeachersMTLs consistently score higher than classroom teachers in all areas of the MKT assessment.
-0.33
0.19
-0.08
-0.25
0.29
0.540.50
0.01
-0.40
-0.30
-0.20
-0.10
0.00
0.10
0.20
0.30
0.40
0.50
0.60
Ab
ilit
y E
stim
ate
Teachers MTLs Number Algebra Geometry Prob & Stat
24
5. Overall Conclusions
Schools that have more heavily participated in MMP-sponsored activities are demonstrating greater student achievement growth.
There appears to be a set of necessary conditions for improving student achievement—no single factor is sufficient.
25
Overall Conclusions
Productive work is taking place at math-focus meetings in the context of grade-level meetings, collaborative planning time, or other special sessions.
A shortcoming of these sessions is often not considering implications for classroom practice based on the good work being done.
26
Overall Conclusions
MTLs have regularly demonstrated math content knowledge gains during the academic year, and there is some evidence to suggest these gains are being sustained over time.
MPS Math Teachers MKT results are generally lower than that of the MTLs. Slight improvements over time have been observed.
27
6. Evaluation 2009-10
MMP Online survey in May 2010 Continue to focus on 25 case study schools for the next 2 years Math meeting observations SNA School honorarium
Implement SNA in mostschools across the district
28
Small Group Discussion
1. What implications does the information presented have for MTLs as they promote PD efforts in their building?
2. What insights have you gained about the work of the MTLs? How will that information impact your work in your building?
3. What are some areas you can enhance or strengthen in your work with teachers that will make an impact on student achievement?
4. According to M. Fullan, “The role of the leader is to ensure that the organization develops relationships that help produce desirable results.”
As you reflect on Fullan’s work about building relationships, what connections can you make to your work and the information just shared?