Embed Size (px)
Citation preview
Evaluating Professional Development
Evaluating Professional Development
Debbie Junk,Coordinator for Mathematics Initiatives
Mathematics Project Directors’ Meeting Tuesday, October 9th
Austin, Texas
Debbie Junk,Coordinator for Mathematics Initiatives
Mathematics Project Directors’ Meeting Tuesday, October 9th
Austin, Texas
True professional development should be a learning experience for all who are involved…professional development is a purposeful and intentional process designed to enhance the knowledge and skills of educators so that they might, in turn,improve the learning of all students.” (Guskey, 2001 p. 121)
True professional development should be a learning experience for all who are involved…professional development is a purposeful and intentional process designed to enhance the knowledge and skills of educators so that they might, in turn,improve the learning of all students.” (Guskey, 2001 p. 121)
What counts as professional development?
Folding ActivityFolding Activity
Why do we evaluate?Why do we evaluate?
Effectiveness of professional development in mathematics need to be evaluated to assist in-PlanningFormative assessment and,Summative assessment
To provide evidence of effectiveness so that good programs get better, great programs spread and ineffective programs get eliminated or significantly reworked.
Effectiveness of professional development in mathematics need to be evaluated to assist in-PlanningFormative assessment and,Summative assessment
To provide evidence of effectiveness so that good programs get better, great programs spread and ineffective programs get eliminated or significantly reworked.
Whose Fault is it?Whose Fault is it?
Evidence versus proof
The complex nature of teaching and learning
Evidence versus proof
The complex nature of teaching and learning
Effectiveness indicators show:Effectiveness
indicators show:Increased student achievement
Increased teacher content knowledge
Evidence that shows the two are linked in some way.
Increased student achievement
Increased teacher content knowledge
Evidence that shows the two are linked in some way.
What Counts as Mathematics Knowledge for Teaching (MKT)?
What Counts as Mathematics Knowledge for Teaching (MKT)?
Mathematics teachers need to know the math they teach in a very specialized way-
Children’s thinkingTask posingAssessmentExpectationsDecision-making
Mathematics teachers need to know the math they teach in a very specialized way-
Children’s thinkingTask posingAssessmentExpectationsDecision-making
ResearchResearch
Teachers who score high on knowledge assessments designed to test this specialized knowledge have students who learn more in mathematics in their classrooms.
Ball, et al, 2004
Students who are in classrooms with teachers who understand and can predict their thinking in mathematics score higher on achievement tests.
Carpenter, et al 1998
Teachers who score high on knowledge assessments designed to test this specialized knowledge have students who learn more in mathematics in their classrooms.
Ball, et al, 2004
Students who are in classrooms with teachers who understand and can predict their thinking in mathematics score higher on achievement tests.
Carpenter, et al 1998
What kinds of evaluation
should be given?
What kinds of evaluation
should be given?Levels of evaluation
Participants’ reactionsParticipants’ learningOrganization support and changeParticipants’ use of new knowledge and skills
Student learning outcomes
From Guskey, 2001 pp 79-81
Levels of evaluationParticipants’ reactionsParticipants’ learningOrganization support and changeParticipants’ use of new knowledge and skills
Student learning outcomes
From Guskey, 2001 pp 79-81
You get what you ask for:
You get what you ask for:
After a 2-day Workshop on Cooperative Learning, one participant responded, “The ideas were fine, but they had us working in groups too much”.
One teacher responded to a workshop titled, Tactics for Thinking: “this is all very interesting but I feel it requires students to think too much!”
From Gusky, 2001
After a 2-day Workshop on Cooperative Learning, one participant responded, “The ideas were fine, but they had us working in groups too much”.
One teacher responded to a workshop titled, Tactics for Thinking: “this is all very interesting but I feel it requires students to think too much!”
From Gusky, 2001
Designing and/or choosing the instrument
Designing and/or choosing the instrument
Understand the purpose of your assessment-
Understand the intervention you are assessing-
The questions should vary in scope so that you gain understanding in all areas of Math Knowledge for Teaching.
Understand the purpose of your assessment-
Understand the intervention you are assessing-
The questions should vary in scope so that you gain understanding in all areas of Math Knowledge for Teaching.
How much is enough?How much is enough?
Practical considerationsLength of the evaluation toolTime, when and wherePre and post dataParticipant numbersWorkshop numbers
Practical considerationsLength of the evaluation toolTime, when and wherePre and post dataParticipant numbersWorkshop numbers
What do you do with ALL THAT DATA?
What do you do with ALL THAT DATA?
Ethical considerationsPermission to use/collect dataAnonymous vs. named
Paper assessmentsElectronic assessmentsSummarizing and analyzing assessment data
Ethical considerationsPermission to use/collect dataAnonymous vs. named
Paper assessmentsElectronic assessmentsSummarizing and analyzing assessment data
Qualitative or Quantitative…Qualitative or Quantitative…
Qualitative evaluations of program effectiveness can occur at any level in many formats.
interviewsopen-ended responsesobservationsjournals
Summarizing these data usually takes more time and consistently reporting the results can be difficult.
Information gained from qualitative data can yield a more personal and deeper understanding of the participants’ experiences.
Qualitative evaluations of program effectiveness can occur at any level in many formats.
interviewsopen-ended responsesobservationsjournals
Summarizing these data usually takes more time and consistently reporting the results can be difficult.
Information gained from qualitative data can yield a more personal and deeper understanding of the participants’ experiences.
Quantitative DataQuantitative Data
Easy to collectEasier to interpret without bias*Limited in scope
Likert scalesMultiple choice
*bias is still present, especially within the assessment design
Easy to collectEasier to interpret without bias*Limited in scope
Likert scalesMultiple choice
*bias is still present, especially within the assessment design
Both?Both?
Typical professional development evaluation has both elements.
e.g. “Were your expectations met today?” (a numeric score is assigned)
Tell what you liked best about this workshop--(open-ended)
Typical professional development evaluation has both elements.
e.g. “Were your expectations met today?” (a numeric score is assigned)
Tell what you liked best about this workshop--(open-ended)
A Whole Different Animal!
A Whole Different Animal!
Assessment of learning requires a documentation of change over time.For students, we can compare year to year test data.
Assessing teachers knowledge is more complex.
Assessment of learning requires a documentation of change over time.For students, we can compare year to year test data.
Assessing teachers knowledge is more complex.
Mathematics and Science Partnership
Grants (MSP)
Mathematics and Science Partnership
Grants (MSP)ExpectationsExpectations
Texas Regional CollaborativesTexas Regional Collaboratives
Plan of Action:07-08 provide instruments for use in pre-post format (e.g. Geometry, K-8 geosciences)
08-09 form advisory committee to explore evaluation and assessment
Plan of Action:07-08 provide instruments for use in pre-post format (e.g. Geometry, K-8 geosciences)
08-09 form advisory committee to explore evaluation and assessment
HOMEWORKHOMEWORK
Evaluating Professional Development
By Thomas Guskey
Corwin Press, 2001
Evaluating Professional Development
By Thomas Guskey
Corwin Press, 2001
