Embed Size (px)
Citation preview
A Partnership to Promote School Readiness
Maryland and Ohio Departments of Education
Dr. Rolf Grafwallner
Maryland State Department of Education
Dr. Stephanie Siddens
Ohio Department of Education
Dr. Jacqueline Nunn
Johns Hopkins University
Dr. Stanley Rabinowitz
WestEd
In collaboration with the Johns Hopkins University and WestEd
Overview
Early Childhood Comprehensive
Assessment System
Online
Professional
Development
Student
Progress
Monitoring
Selected
Assessment Items
Electronically
Delivered to
Child
Online Data
Capture and
Reporting
Formative
Assessments
Kindergarten
Entry Assessment
EC-CAS Domains of School
Readiness
Social Foundations
Physical Well-Being and Motor Development
Language and Literacy
Mathematics
Science
Social Studies
The Arts
Purpose of the KEA
Inform prior education and care stakeholders
Identify individual children’s needs and provide necessary supports to children and teachers
Assist teachers in data-driven instructional decision making at the child and classroom level
Provide families with information about their children’s learning and development
To support and advance children’s early learning and academic achievement. The data collected will be used to:
The Formative Assessments
Will be based on research supported
learning progressions for children ages 36
through 72 months, and will equip
caregivers and teachers to:
Track individual children’s learning
trajectories
Individualize learning opportunities and
plan for intervention
Ensure that children are on the path for
kindergarten readiness
Alignments
Birth to K Entry Standards in MD and OH
Head Start Early Learning Framework
K – 12 Standards
Early Child Outcomes required for IDEA reporting
EL Standards (WIDA and CCSSO)
Stakeholder and Expert Input
National Technical Advisory Committee (facilitated by
CCSSO)
State and Local Advisory Councils
Stakeholder and Expert AdHoc Committees
Multi-partner Leadership Team
Research and
Evaluation
Validity, Reliability and Fairness
Measured Across 3 Stages
Design and Development
Implementation
Ongoing Evaluation & Research
Quantitative and Qualitative
Cognitive Interviews
Pilot Testing
Field Testing
Post-Administration Analyses
Validity, Reliability and Fairness, cont’d
Frequent involvement of:
State stakeholders
Early childhood experts
Students with disabilities and English learners’ experts
Monitor adherence to Universal Design principles and review accessibility requirements
Oversee development of test security guidelines
Evaluate degree to which system components work together as
intended (i.e., use of multiple measures to assess a specific
indicator)
Evaluate the degree to which technology-supported items and
traditional items perform to ensure and maintain comparability
Conduct utilization, impact, and/or cost benefit analyses to
determine support for continued use of system components
Monitor changes to state and federal policies as well as
emerging research to support suggested changes to system
components
Frequent and regular involvement of stakeholders in the
monitoring of the assessment system to assess consequences of
testing and evaluate system effectiveness
Psychometrics
Psychometrics, cont’d
Monitor assessment scale characteristics over time
Monitor classification accuracy and consistency over time
Monitor and evaluate emerging research that may impact
rubrics or scoring/rating protocols -Conduct validity and
reliability analyses at the sub-group level
Monitor impact of performance standards on all student
populations over time
Evaluate overall system security and integrity.
Psychometrics, cont’d Frequent involvement of:
o State stakeholders
o Early childhood experts
o English language learners and students with
disabilities experts
Monitor adherence to Universal Design principles and
review accessibility requirements for all items and
tasks, including APIP item tagging
Oversee development of test security guidelines
KEA Design
KEA Design
DOMAINS 36
mo.
42
mo.
48
mo.
54
mo.
60
mo.
66 mo. 72
mo.
Social-Foundations
Formative Assessment KEA
Language and Literacy
Mathematics
Physical Well-Being and Motor Development
Science
Social Studies
The Arts
Formative assessment :Development represents a continuum of changing behaviors
Summative “snapshot” of readiness
Two Types of Assessment Processes
KEA Design
The KEA features several item types
including:
Guided recorded observation
Performance tasks
Developmental checklists
Selected response items
Multiple Measures
KEA Blueprint OverviewDomain Strands Learning Progressions
1. Physical Well-Beingand Motor Development
• Physical Education• Health
• Coordination – Large Motor• Coordination – Small Motor
2. Mathematics Counting and Cardinality
Operations and Algebraic
Thinking
Measurement and Data
Geometry
• Number sense• Number operations• Classification• Measurement• Shapes
3. Language and Literacy
Reading
Speaking and Listening
Writing
Language
• Story/Text Comprehension• Phonological Awareness• Phonics and Letter Recognition• Communication• Emergent Writing• Grammar• Vocabulary
KEA Blueprint Overview cont.Domain Strands Learning Progressions
4. Science • Skills and Processes/Life Science
• Inquiry and Observation
5. Social Studies Government
History
• Responsible Behavior• Events in the Context of Time
6. Social Foundations Social Emotional
Approaches to Learning
Executive Functioning
• Awareness and Expression of Emotion
• Relationship with Adults• Conflict Resolution• Self-Control• Persistence• Working Memory• Problem Solving• Initiative• Cooperation with Peers
7. The Arts – Under
Development
KEA Blueprint by DomainDomain Strand KEA Code Standard/Essential Skills and Knowledge SR PT CL POINTS
Learning Progressio
n
Physical Developme
nt(PD)
Physical Education
(1)
PD.1.1 Demonstrate the ability to use large muscles to perform a variety of physical skills.
Coordination–Large
Motor
PD.1.1.AShow fundamental movement by identifying body parts and demonstrating a variety of ways they can move, and demonstrating spatial concepts in movement patterns.
1 2
PD.1.1.BDemonstrate locomotor skills with control, coordination, and balance during active play (e.g., running, hopping, skipping).
1 2
PD.1.1.CDemonstrate coordination in using objects during active play (e.g., throwing, catching, kicking balls, riding tricycle).
PD.1.1.DUse non-locomotor skills with control, balance, and coordination during active play (e.g., bending, stretching, and twisting).
PD.1.2Demonstrate the ability to use small muscles to perform fine motor skills in play and learning situations.
Coordination–Small
Motor
PD.1.2.ACoordinate the use of hands, fingers, and wrists to manipulate objects and perform tasks requiring precise movements.
PD.1.2.BUse classroom and household tools independently with eye-hand coordination to carry out activities.
1 2
PD.1.2.CUse a three-finger grasp of dominant hand to hold a writing tool.
1 2
Health(2)
PD.2.1 Demonstrate the ability to apply prevention and intervention knowledge, skills, and processes to promote safe living, in the home, school, and community.
Safety and Injury
Prevention
PD.2.1.A With modeling and support, identify and follow basic safety rules.
1 2
PD.2.1.B Identify ways adults help to keep us safe. 1 2
PD.2.1.CWith modeling and support, identify the consequences of unsafe behavior.
PD.2.1.DWith modeling and support, demonstrate ability to follow transportation and pedestrian safety rules.
PD.2.2 Demonstrate personal health and hygiene practices.
Personal Care Tasks
PD.2.2.AIndependently complete personal care tasks (e.g., washing hands before eating and after toileting).
1 2
PD.2.2.BFollow basic health practices (e.g., covering mouth/nose when coughing/sneezing).
KEA Development Plan
2011 2015
2012/2013
CONCEPTUAL DEVELOPMENT
2013
DEVELOPMENT AND
PILOT TESTING
2013/2014
FIELD TESTING
(State Baseline OH)
2014/2015
FULL IMPLEMENTATION
EC-CAS Timeline
3-Step KEA Development Process
Pre-Pilot (2011/2012) External/expert review by content area
Focus groups with teachers
Cognitive Labs with students
Feedback and assessment revision
Bias review
Accommodations validated
Formal Pilot (2012/2013) Testing pilot items in classrooms
Feedback and assessment revision
KEA Field Test (Fall 2013) Testing of items with larger sample
Item statistics, internal validity analysis
Item calibration and external validity analysis
Enhancements to v. 1.0, New Development (2013/2014)
Additional formative items and functionality
Enhanced accommodations
Expanded use of child-friendly technology interface
Leveling based on performance and predictive intelligence
v. 2.0 Formal Pilot (2014/2015)
Pilot testing version 2.0 technical enhancements in classrooms
Feedback and assessment revisions
v. 2.0 Field Test (2015/2016)
Testing of items with larger sample
Item statistics, internal validity analysis
Item calibration and external validity analysis
Development Process for KEA version 2.0
Professional Development
Professional Development Content
Pre-Administration
Purpose of the assessments
Data security and integrity
Communication with parents/stakeholders
Administration
Use of assessment system for data collection and reporting
Understanding of supports and accommodations
Importance of fidelity of implementation
Post-Administration
Analysis and use of the data for program
and instructional decision-making
Communication of results to stakeholders
Monitoring student progress (v. 2.0)
Audiences: child care providers, trainers, teachers, school and district administrators
Professional Development Delivery
Train the trainer model
PD Quest web-based learning modules
Online coaching and technical
assistance
Professional community of practice
Virtual simulator for validation
Web-based resources and portal
Procedural facilitators (v. 2.0)
Evidence-based and technology supported
Technology
Approach
Technology Use in the KEA
An Online Reporting System (ORS) provides secure access to teacher dashboards, live data collection, and downloadable resources
The Virtual Performance Assessment (VPA) interface provides child-friendly, student access to direct-assessment
Data import, export, and reporting tools; linkages to longitudinal data systems
Support for attaching portfolio artifacts to student profiles over time
Online, just-in-time Professional Development
Components of the KEA Technology System
Technology Use in the KEA
An Online Reporting System (ORS) for data collection: The educator’s interface - accessible via browser on a computer, laptop, or tablet - for entering the results of real-world observations or scores from any performance item. The ORS also supports:
o Browsing KEA completion status by class or student
o Downloading supporting resources
o Locating just-in-time, contextualized PD
o Entering assessment results directly into the ORS
o Attaching student artifacts to a longitudinal profile
Online Reporting System (ORS)
Technology Use in the KEA
Virtual Performance Assessment (VPA): A child-friendly student
interface comprised of 2 types of assessment item designs:
1. Point and Touch – Single touch (or click) selection items
2. Interactives – Items featuring fun and engaging opportunities for
children to interact and receive instructional feedback (primarily
in formative assessments)
The user touches the
matching quantity
Virtual Performance Assessment (VPA)
The child matches food
items with shapes as a
conveyor belt moves
along
Technology Use in the KEA
Dashboards and Reports support state, district, school, child, and classroom-level data
reporting and analysis. Customizable views and reports can be created for parents,
teachers, and building, district, and state administrators, including:
Data Import, Export, Reporting, and Longitudinal Tracking
Assessment Completion Rate - # of items for a KEA or formative period completed at a child (Parent/Teacher) or class level (Teacher/Building Admin).
KEA Readiness Performance – Child level (teacher/parent), Class-level (teacher/building administrator), District-level (District/State admin) and State-level (State admin) performance by domain to inform broad readiness monitoring.
Formative Learning Progression Levels – Child level (teacher/parent), Class-level(teacher/building administrator)
Specific Formative Item Performance – child level (teacher/parent) and class-level (teacher) to inform instructional decision making
Technology Use in the KEA
Embedded PD recommendations,
including direct links to specific,
contextual
guidance to support KEA assessment
delivery and classroom instruction
Comprehensive series of online
PD modules & resources
Use of simulation software to
familiarize teachers with the KEA
protocols & technology
Professional Development
Technology Use in the KEA
o Web-based (ORS)
o Tablet-optimized (VPA)
o Cloud hosted
o Trans-media
o Secure and scalable
o Nightly data feeds
Technical Features
Project Contacts
Dr. Rolf Grafwallner, Maryland State Department of
Education [email protected]
Dr. Stephanie Siddens, Ohio Department of Education
The Early Childhood CAS Partnership between Maryland and
Ohio was supported by grants from Race to the Top-Early
Learning Challenge from the U.S. Department of Education and
U.S. Department of Health and Human Services (CFDA
84.412A) and Race to the Top from the U.S. Department of
Education (CFDA 84.395) .
CCSSO
June 21, 2013
