Workforce Development Under Colorado’s Race to the Top Early … · 2018-01-22 · Contents Tables and Figures 4 Overview 6 Introduction 7 Foundations of Workforce Development Efforts

WORKFORCE DEVELOPMENT UNDER COLORADO’S RACE TO THE TOP EARLY

LEARNING CHALLENGE GRANT: A PROCESS EVALUATION

Submitted to: Nancie Linville, Director, Office of Early Learning and School Readiness

Colorado Department of Education

By: Jean Williams, Office of Early Learning and School Readiness

Educator Evaluation Design Specialist Colorado Department of Education

August 2017

Colorado’s Race to the Top Early Learning Challenge Grant: Process Evaluation 2

Contents

Tables and Figures 4

Overview 6

Introduction 7

Foundations of Workforce Development Efforts Under Colorado’s Race to the Top/Early Learning Challenge Grant 10 Organizational Factors 10

Interagency Communication, Collaboration and Partnerships 10

Colorado’s Early Learning Professional Development System Plan 13

Innovations 13 Goals and Recommendations 14 Guiding Principles 14

Conclusions 17

Colorado’s Competencies for Ear ly Childhood Educators and Administrators 18 Institutions of Higher Education 19

Colorado State Model Educator Effectiveness System 21

Demonstrated Competencies 23

Conclusions and Recommendations 23

Early Childhood Professional Credential 2.0 27 Respondents and PDIS Users 30

Points Earned 31

Credential Levels 33

Demonstrated Competencies 33

Guiding Principles and Fairness 35


Professional Development Information System 42 Training and Support to Use the System 42

PDIS Trainers and Trainings 44

Characteristics of Professionals Using PDIS 45

Technology Platform 46

System Outputs 46



Summary 49

Glossary* 51

References 53


Tables and Figures

Page Table 1. Goals for Early Learning Challenge Grant Workforce Development Activities

8

Table 2. Standing and Ad Hoc Committees and Task Forces Created to Support Workforce Development Efforts

12

Table 3. Percentage of Respondents Rating Early Childhood Professional Development System Guiding Principles at Each Level of a 4-Point Scale

15

Table 4. Comparison of Rank Order of Ratings of Early Childhood Professional Development System Guiding Principles Across Four Surveys

16

Table 5. Utilization of Colorado’s Competencies for Early Childhood Educators and Administrators

20

Figure 1. Minimum Courses Required to become an Early Childhood Teacher in a Colorado Preschool or Community-based Setting

21

Table 6. Key Accomplishments of the 2010 Early Learning Professional Development System Plan

24

Figure 2. Points Available for Each Credential Pathway

27

Figure 3. Roles of Respondents in 2014 and 2016 Credential 2.0 Surveys and from PDIS 29

Figure 4. Total Points Earned in 2014 and 2016 and Through PDIS 30

Figure 5a. Method of Earning Points on 2016 Survey with Area of Study Points Expiring after 25 years

31

Figure 5b. Method of Earning Points on 2016 Survey with No Expiration of Area of Study Points

31

Figure 5c. Method of Earning Points in PDIS with No Expiration of Area of Study Points 31

Figure 6. Distribution of Respondents by Early Childhood Professional Designation (Rating Level) fo14 and 2016 Survey Respondents and PDIS Users

31

Figure 7. Tools Users Would Select to Use for Demonstrated Competencies 33

Figure 8. Will Demonstrated Competency Points Make a Difference in Credential Levels: 2016 Survey Responses

33

Figure 9. Method of Method of Earning Points on 2016 Survey with No Expiration of Area of Study Points 34

Figure 10. Will Demonstrated Competency Points and Pathways Make a Difference in Credential Levels: 2016 Survey Responses

34

Figure 11. Ratings of Credential 2.0 System Adherence to Guiding Principles in 2014 and 2016

35

Table 7. Changes in Ratings of Adherence to Guiding Principles Between 2014 and 2016 36


Figure 12. Fairness of Credential 2.0 Model 37

Table 8. Early Childhood Professional Credential 2.0/Child Care Licensing Center Rules Personnel Qualifications Equivalency

40

Table 9. CDE/CDHS Collaborative Communication and Training Activities Regarding Workforce Development

42

Figure 13. Framework for Phased in Approach to Training 43

Figure 14. Connecting Learning Objectives to Activities 44


Overview

“Taken together, the activities of the Workforce High-Quality Plan instill greater measurement, accountability, visibility, support and encouragement into a system spanning a collective team of 22,000 professionals who are

caring for, guiding and instructing the most impressionable and wonder-filled members of our society.”

Colorado’s Race to the Top – Early Learning Challenge Grant Application

Colorado’s Race to the Top-Early Learning Challenge Grant (RTT-ELC) application included a proposal for a State Plan to will help ensure kindergarten readiness for all children in the State of Colorado, including Children with High Needs. This plan set rigorous goals for Improving Program Quality, Improving Outcomes for Children with High Needs and Closing the Readiness Gap. To accomplish these goals, the state plan addressed major influences affecting the education of young children, particularly development of an understanding of what it means to serve Children with High Needs. Colorado embraced the focus of the RTT-ELC as consistent with the moral demands of community, the value in which children are held, and the commitment to achieve positive outcomes for Children with High Needs.i This study examines Colorado’s efforts to improve the quality of the state’s early childhood education workforce.

`

A child’s development is influenced dramatically by the people and

environments he interacts with most frequently and intensely. Children will not develop optimally if their most sustained contacts are with people or environments that are struggling or under stress. To best

foster the healthy development of Children with High Needs, then, we must also target quality improvement efforts

to the people, programs, and places (communities) through which children

grow and learn.

Race to the Top- Early Learning Challenge Application for Initial Funding

October 19, 2011


Introduction In October 2011, the State of Colorado proposed a grand vision for improving its ability to educate the state’s young children. This vision provided the foundation for the Race to the Top Early Learning Challenge Grant that was ultimately funded to support efforts to achieve the vision. The Early Childhood Colorado Framework established a vision that “all children are valued, healthy, and thriving.” This vision along with Colorado’s Early Learning Professional Development System Plan was a driving force behind Colorado’s Race to the Top-Early Learning Challenge state plan. The theory of change presented in Colorado’s Early Learning Challenge Grant application focused on six distinct yet integrated project areas:

• Aligning and coordinating Early Learning and Development Programs across the state,

• Implementing a common, statewide Tiered Quality Rating and Improvement System, • Developing and using statewide high-quality Early Learning and Development Guidelines, • Supporting effective uses of Comprehensive Assessment Systems, • Supporting early childhood educators, and • Implementing a statewide Kindergarten Entry Assessment.

The combined work of these project areas is intended to establish a credible path toward achieving improved school readiness for Children with High Needs. Effective implementation of these project areas involves a broad cross-section of state agencies, each with its own set of roles, responsibilities, desired outcomes and “official deliverables.” For the grant objectives to be achieved, the various agencies need to meet their individual expectations. The Colorado Department of Education (CDE) was charged with leading workforce development efforts resulting in a statewide cadre of ECE professionals credentialed within an aligned system. The plan for workforce development was built on and accelerated by implementation of key recommendations in Colorado’s Early Learning Professional Development System Plan. Workforce development goals were proposed to be accomplished in two phases as outlined in Table 1. Colorado’s Race to the Top Early Learning Challenge Grant (RTT-ELC) application specified the following set of activities designed to achieve the goals articulated in Table 1:

• Implementation of a Learning Management System (Professional Development Information System or PDIS), integrated with other appropriate state systems, to: o help the Workforce understand credential requirements, track individual and aggregate progress, o plan improvement, o take online courses, and o find additional professional development opportunities.

Our vision for the State Plan is that children with High Needs join all children in experiencing the best opportunities for

healthy child development and school readiness. We will achieve this vision by

strategically supporting the people, programs and places with the highest

needs in order to improve quality, access and equity. This vision sits within and

complements the Early Childhood Colorado Framework vision: all children

are valued, healthy, and thriving.

Race to the Top - Early Learning Challenge Application for Initial Funding

October 19, 2011


Phase 1 Phase 2 Complete the Workforce Knowledge and Competency Framework and the aligned progression of Credentials and Degrees and Integrate use of the Framework into all Professional Development Education

Expand access to effective Professional Development Opportunities and Promote Professional Improvement and Advancement

Complete a performance and research based Workforce Knowledge and Competency Framework (Competency Framework), including culturally relevant measurements and alignment among existing: 1. Competency Frameworks for Early Childhood Educator

(ECE) professionals; 2. Tiered Quality Rating & Improvement System (TQRIS); 3. Educator Effectiveness Standards P-12; and 4. Colorado’s Early Learning and Development Guidelines. 5. An aligned, targeted progression of credentials and

related degrees; all existing credentials and degrees are aligned to the revised Competency Framework.

6. Alignment of all institutions and professional development providers, and resulting credentials and degrees by 2015.

7. Professional development programs for Early Childhood Educators apply the Framework statewide.

8. Integration of the Competency Framework into culturally responsive coaching, mentoring, practicums and career planning and advising.

9. Early Childhood Educators and Early Learning and Development Programs actively use the Framework and credentialing in recruiting, hiring, training, advancement and retention decisions.

1. Training in the Early Learning and Development Guidelines and all core competencies for Early Childhood Educators is available statewide, including targeted modules for serving Children with High Needs.

2. Early Childhood Educators can readily access information to track progress on their learning and career plan through a web-based Learning Management System.

3. Professional development opportunities encourage and support adult learners who reflect the ethnic and linguistic diversity of the children and families served.

4. Policies and incentives are in place and effective in accelerating professional improvement with targeted incentives for educators serving Children with High Needs and those who support the advancement of such educators.

5. Training is available for families with young children and Family, Friend and Neighbor (FFN) providers supporting Children with High Needs.

6. Across all roles and levels, measurable increases are achieved in the percentage of qualified Early Childhood Educators who reflect the ethnic and linguistic diversity of the children and families served.

7. By 2015, public reporting on the Workforce’s development, advancement and retention will be widely available.

8. By December 2015, 65% of Early Childhood Educators will be tracked in the credentialing system and 15% will be in the top two credential tiers.

Table 1. Goals for Early Learning Challenge Grant Workforce Development Activitiesii

• Rolling out new professional learning opportunities to: o Fill Current gaps in the system, including professional development modules aligned with the

Competency Framework that target serving Children with High Needs; online coaching and training opportunities; and itinerant training programs.

• Provide training on Colorado’s quality initiatives, including Early Learning and Development Guidelines, QRIS and other professional development opportunities to early childhood professionals, families and

Family, Friend and Neighbor (FFN) providers supporting Children with High Needs. • Creating professional development opportunities that address and support the needs of ethnically and

linguistically diverse learners, including: o encouraging more diverse trainers and higher education faculty and o providing mentorships and culturally relevant practicums.

• Expanding policies and incentives (scholarships, wage subsidies, tax credits) that accelerate professional learning and career advancement.


• Expanding and further integrating the coaching network. • Producing public reports for all stakeholders on the aggregate quality of educators and their

development, advancement and retention.

While all goals and activities of the RTT-ELC are important to the advancement of the state’s agenda to improve the quality of the educational experiences and outcomes for the youngest and most impressionable learners, this report has as its focus and organizing structure the creation of multiple tools, materials and processes to improve the quality of the state’s early childhood workforce. This work includes but is not limited to:

1. The creation and broad buy-in of a Competency Framework for Early Childhood Educators and Administrators that unifies the parallel systems of workforce competencies and credentialing and lays out the skills and knowledge early childhood educators need to adequately prepare young children for the future.

2. The development and implementation of a credentialing system through which early childhood education professionals may apply to have CDE staff members review their transcripts and certificates to determine the credential level for which they qualify.

3. Supporting, expanding and enhancing this work are the Competency Framework and the Professional Development Information System (PDIS), which serves as: a. a communication conduit and information repository regarding educator qualifications, b. a warehouse providing an organized structure for collecting and later accessing individuals’

information about their experience, formal education, professional development, and demonstrated competencies,

c. a professional development source to enable participants to receive credential credit for targeted professional development aligned with Colorado’s Competencies for Early Childhood Educators and Administrators,

d. a process for users to assess their own skills and development knowledge, and e. a guide for creating individualized professional plans containing suggestions for improving skills and

knowledge aligned with Colorado’s Competencies for Early Childhood Educators and Administrators.


Foundations of Workforce Development Efforts Under Colorado’s Race to the Top/Early Learning Challenge Grant Colorado’s Race to the Top/Early Learning Challenge Grant (RTT-ELC) application described foundational policies, partnerships, processes, and products the state would bring to the table to kick start and guide the work toward achieving the stated goals of the grant. This section describes those foundational pieces and how the Early Childhood Professional Development team has used the grant funds to support their implementation and institutionalization.

Organizational Factors Colorado’s Race to the Top Early Learning Challenge Grant application proposed a multi-agency approach to management and established clear and focused roles and responsibilities for each agency. Collectively, these agencies were charged with meeting rigorous and challenging goals to accelerate in quality, depth and pace the agenda focusing on the coordinated build out of key pieces of the underlying system infrastructure:

• state governance; • the adoption and integration of culturally

relevant Early Childhood Development Guidelines;

• broader and richer use of Comprehensive Assessment Systems;

• a kindergarten specific readiness assessment;

• the implementation of a rated licensed TQRIS with complementary policies and practices to promote its use;

• a Workforce Knowledge and Competency Framework with related progression of credentials and degrees; and

• expanded professional development opportunities and supports.iii

Interagency Communication, Collaboration and Partnerships One of the cornerstones of Colorado’s Race to the Top grant application was the positive long-term relationship between the Colorado Department of Human Services (CDHS) and the Colorado Department of Education (CDE). The historically sound and beneficial relationship that has been formalized by interagency agreements for many years allowed the two agencies to expand and enhance their collaborative efforts when the RTT-ELC grant was awarded. The operationalization of workforce development activities such as the Workforce Knowledge and Competency Framework, progression of credentials and degrees and expanded professional development opportunities and supports required collaboration on the part of agencies and organizations across the state as well as practitioners who work directly with children. These efforts (Table 2) initially involved broad-based communications with early childhood educators across the state to inform them of the upcoming work and to get their feedback on specifics grant activities. Such efforts have evolved over the life of the grant into partnerships that involve planning aligned activities to benefit all agencies and organizations involved in educating children and the adults who are responsible for their well-being. Such efforts have involved both formal and informal partnerships between and among CDE and other organizations and individuals. As Table 2 indicates, collaboration activities have been both broad and deep. What began as efforts to address the immediate RTT-ELC goals have progressed toward mutually beneficial activities designed to provide a support network for not only grant activities but for all involved organizations. A good example of this is the work of the

In order to advance the profession, that profession must be defined. The absence of a

unified and agreed upon scope of practice, competencies, career pathways and

nomenclature that define early childhood professionals across states and settings has limited our collective impact for too long, stifling the case for increased professional

recognition and compensation parity. . . Our collective history reminds us that success

comes when we share a message and a commitment to working from the same

blueprint so we can build it better, together.

Build It Better, 2016


Early Childhood Professional Credential and Appeals Evaluation Committee, which was organized to review and determine the merits of grievance and concerns about the Early Childhood Credentialing system. As the work of this group has progressed, it has grown to include considerations of how team members and their organizations can better understand the problems faced by local programs and offer assistance when possible. For example, one of Colorado’s local early childhood councils questioned the Early Childhood Professional Credential 2.0 scoring system and suggested that it was biased toward educators who earned degrees in early childhood education and against those who earned degrees in other areas but had honed their skills over time through professional development and college coursework. On its face, the case made by the program seemed to be about more than the stated problem and the committee decided not to adjust the scoring system. To further their understanding of the problem, CDE staff members visited the council staff and learned more about the community’s ongoing and worsening teacher shortage. The council had reached out to the committee to try to find a way to improve the credential levels of current teachers as a motivator for them to continue in their positions despite other more troubling problems such as low wages. Several document exchanges and discussions have followed the site visit as team members and program staff collaboratively search for solutions to the overarching problems facing the community, the catalyst for the letter to the committee.

Committee/ Task Force Membership* Tasks Early Childhood Professional Development Advisory

Boulder Journey School Clayton Early Learning CDE/Early Childhood Professional

Development CDE/Educator Effectiveness Coaching Consortium CO Assn. for the Education of Young

Children CO Community College System CO Department of Higher Education CO Dept. of Human Services CO Mountain College Early Childhood Council Leadership

Alliance Early Childhood Education Assn. of CO Early Connections Learning Centers Family Development Center of

Steamboat Springs Metropolitan State University Mile High Montessori Mile High United Way National Assoc. of Family Child Care Pikes Peak Community College Qualistar CO Salida Schools University of CO, Colorado Springs University of CO, Denver University of Northern CO

• Provide oversight and guidance regarding workforce development initiatives to the RTT-ELC Team

• Make recommendations to the Early Childhood Leadership Council

Organized and convened almost immediately following grant award, the PD Advisory has served as a steering, or oversight, committee to guide workforce development activities from initiation through institutionalization. Their input on development of each of the project’s components as well as their feedback throughout the pilot test, rollout, and early implementation have proven to be invaluable to building an effective, scalable and sustainable workforce development program. Current members’ terms ended in August 2017. A new membership will be seated with approval from the Program Quality and Alignment committee of the Early Childhood Leadership Commission. The focus of the newly seated ECPD Advisory Working Group will be to guide implementation of the Colorado Early Childhood Workforce 2020 plan.


Committee/ Task Force Membership* Tasks

Early Childhood Competency Framework Development

Early Childhood Professional Development Advisory Group Competencies Sub-Committee ECLC Program Quality, Coordination and Alignment Committee

Oversee the development of framework of performance-based competencies for early childhood educators and administrators. Work was completed under contract by Oldham Innovative Research.

Developed the framework for performance-based competencies for early childhood educators and administrators. Enlisted the support of experts in the field to provide advice, guidance, and feedback regarding the various iterations of the framework.

Early Childhood Alignment Group and Faculty Leads

EC Faculty Chairs from 4-year institutions Elected Representatives from 2-year institutions

Review and discuss alignment Review individual institutions’ alignment Review program and provide documentation of alignment, identifying gaps for future changes Discuss beneficial changes to statewide articulation process

• EC Competencies are being used as the foundation for rules and regulations related to the approval of teacher preparation programs at 4-year universities.

• Redesign of 13 community college early childhood programs based on the mapping of courses common to all. Redesign is focused on addressing gaps between existing programs and Competencies.

• Mapping of 4-year teacher preparation programs to EC Competencies to inform any redesign necessary to address gaps and ensure consistent delivery of course content.

• Revised articulation agreements are under development.

Colorado Shines Implementation Steering Committee

CDE CDHS Early Childhood Councils Qualistar Denver Preschool Program

Ensure that all components of the Colorado Shines system work well together

Met weekly during the first phase of RTT-ELC implementation to iron out problems within the QRIS system, including how well PDIS credential ratings, based on the Competencies, were transferring into QRIS and being used to judge program quality.

Early Childhood Professional Credential Appeals and Evaluation Committee

Clayton Early Learning CDE: ECPD, CPP, Educator Licensing CO DHS: QRIS, Child Care Licensing CO Community College System School District Early Childhood Program Qualistar Colorado University-based Researcher

Review and make decisions on formal Credential Appeal requests and conduct an extensive review of credential evaluation data to understand the nature of specific issues and make recommendations to the full ECPD Advisory on the use of the data including potential changes to the credential scoring system.

The committee has reviewed formal Credential Appeal requests from the field and offered recommendations to the ECPD Advisory regarding how best to respond.

Early Childhood Educator Evaluation Model Development Team

CDE/Early Childhood Professional Development CDE/Educator Effectiveness CDE/Exceptional Student Services Unit CO Preschool Program REL Central at Marzano Research Denver Preschool Program

Determine how to evaluate early childhood teachers using validated instruments appropriate for measuring performance against not only the Teacher Quality Standards but also the Early Childhood Credentials for Educators and Administrators (the Competencies) Identify appropriate Student Learning Outcomes and how best to measure them to evaluate early childhood educators

The Early Childhood Educator Evaluation Model Development Team was created in response to concerns about the Colorado State Model Educator Evaluation System’s (State Model System) appropriateness for early childhood educators. Teachers and administrators registered their concerns about whether teachers of young children should be expected to demonstrate some of the professional practices included in the state model system’s rubric and whether the practices expected of teachers are


Committee/ Task Force Membership* Tasks developmentally appropriate for their young students. Originally planned as a development work group charged with finding ways to use other validated measures aligned with Colorado’s Teacher Quality Standards and with the Competencies, this group’s focus has changed over time. The Educator Effectiveness (EE) team gathered feedback regarding the effectiveness of their work during the five-year pilot test following approval of Senate Bill 10-191. In response to such feedback and data collected during the same period, they determined that a major overhaul of the rubric was necessary. They completed the rubric revision in early summer 2017. Because of the availability of the new rubric, its impending pilot test, and the desire to align with the efforts of the EE team, the Evaluation Model Development Team decided to table the idea of using other validated instruments for Early Childhood Educator Evaluation and instead to both support and try to gain maximum benefit from the 2017-18 pilot test of the new rubric and auxiliary materials. This approach will enable the ECPD unit to disaggregate data based on grade level taught and to determine the relevance, validity and fairness of the revised system for early childhood teachers. Throughout the pilot test, members of the team will develop materials to support the evaluation of early childhood teachers. In addition, they will continually monitor the progress of the new evaluation system to determine its appropriateness for early childhood teachers. Table 2. Standing and Ad Hoc Committees and Task Forces Created to Support Workforce Development Efforts

Colorado’s Early Learning Professional Development System Plan In 2010 Colorado developed the Early Learning Professional Development System Plan (the 2010 Plan), a blueprint for action aimed at significantly improving the effectiveness of early learning professionals in Colorado—those individuals who both support and directly provide non-parental care and education services for the state’s children from birth to age eight. This plan guided not only the development and implementation of innovative and creative initiatives to improve the state’s early childhood workforce, but also a set of guiding principles which have formed the parameters for how the state’s education leaders work together and a set of broad goals by which to gauge progress toward the establishment of a practical and sustainable professional development system. The 2010 plan served as inspiration and guidance for the state’s Race to the Top/Early Learning Challenge Grant application and as a work plan for initiating, implementing and monitoring grant activities. This section provides a brief discussion of the 2010 plan and the state’s progress toward achieving its goals and recommendations under the RTT-ELC.

Innovations

The P-3 Professional Development System Plan (“Plan”) includes several noteworthy innovations: 1. The establishment of a research-based tiered set of competencies that will serve as the foundation for all

early childhood professional preparation and ongoing professional development;

2. The development of new quality-assurance and accountability mechanisms to ensure the effectiveness of college preparation and ongoing professional development and the effectiveness of early learning professionals; and

3. A focus on building public and political support to increase public and private funding for the P-3 Professional Development System, making it possible to recruit, compensate and retain well-prepared and effective professionals for our youngest and most at-risk children.


Goals and Recommendations

The Professional Development Task Force generated a set of goals and recommendations to guide Colorado’s efforts to improve the effectiveness of the early learning workforce. Some of the goals could be achieved through better coordination and use of existing funds while others would need significant financial resources and the support of educators across the state. Key recommendations for implementing the plan reflect careful consideration of the current status of the workforce and the environment in which it exists. These recommendations are:

• Establish the research-based tiered-set of competencies for early learning professionals. This is the foundation of the professional development system and we recommend that work begin on this important goal as soon as possible.

• Establish an ongoing, high-level advisory group to inform and advise the Early Childhood Leadership Commission regarding policy and practice related to the professional development of early learning professionals. This state-level group should be authorized to make recommendations and provide guidance for necessary statutory or regulatory changes needed to ensure successful implementation of the Plan.

• Coordinate implementation of the Plan with current state initiatives associated with the implementation of the Educator Effectiveness Act and with the redesign of Colorado’s early childhood QRIS. Coordination is essential to ensure consistency, reduce duplication and improve the implementation of new requirements.

• Initiate communication about the Plan and its implementation as soon as the Plan is approved. The early childhood community of professionals and programs will need to understand the Plan’s goals and objectives, discuss the potential impacts that implementation of the Plan may have on their day-to-day work as well as have meaningful opportunities to provide input regarding the Plan’s implementation.

Guiding Principles

In addition to the goals and recommendations, the Plan laid out a set of seven guiding principles describing the manner in which professional development efforts should be implemented. CDE has taken these principles seriously throughout workforce development efforts and have continually monitored their adherence to them as well as Colorado’s perception of their adherence to them. Table 3 provides the results of several surveys through which adherence to guiding principles was measured. In 2014 and again in 2016, CDE conducted surveys of the early childhood workforce to determine their opinions about the proposed approach to developing a statewide professional development system. A modified version of the survey, with identical questions regarding guiding principles was conducted at the end of 2016. The seven guiding principles are the cornerstone of the professional development plan and serve as both parameters for the work and ultimate goals for measuring the quality of the system as it matures and is institutionalized. As Table 3 indicates, the three groups of respondents agreed that CDE and its partners mostly or definitely address all of the guiding principles.

2014 Credential 2.0 Survey % of 624*


2016 Demonstrated Competencies

Survey % of 116*

Weighted Average

Across All Surveys*

Accountability Definitely Does Not Address 0.90 2.97 0.00 2.08

Guiding Principles for Colorado’s Professional

Development System

Accountability Innovation Accessibility Inclusivity Alignment Well-financed Collaboration





Survey % of 116*

Weighted Average

Across All Surveys*

Minimally Addresses 11.94 13.89 6.90 12.79 Mostly Addresses 52.54 48.27 52.59 49.99 Definitely Addresses 34.63 34.87 40.52 35.14

Innovation Definitely Does Not Address 1.78 3.55 1.16 2.80 Minimally Addresses 9.17 12.58 2.59 10.79 Mostly Addresses 45.86 45.09 49.14 45.61 Definitely Addresses 39.34 43.02 47.41 42.05

Access Definitely Does Not Address 2.69 3.84 1.16 3.28 Minimally Addresses 13.73 10.42 12.07 11.65 Mostly Addresses 41.79 38.00 52.59 40.21 Definitely Addresses 41.79 47.77 34.48 44.90

Inclusivity Definitely Does Not Address 2.10 3.55 0.00 2.83 Minimally Addresses 12.87 9.53 4.31 10.33 Mostly Addresses 41.92 42.32 56.03 43.05 Definitely Addresses 43.11 44.57 39.66 43.77

Alignment Definitely Does Not Address 2.12 4.00 0.00 3.11 Minimally Addresses 14.55 13.90 6.90 13.68 Mostly Addresses 43.64 44.41 50.00 44.50 Definitely Addresses 39.39 37.07 43.10 38.24

Well-Financed Definitely Does Not Address 8.51 6.63 9.30 7.44 Minimally Addresses 13.98 15.03 13.79 14.60 Mostly Addresses 37.99 38.47 50.86 39.09 Definitely Addresses 39.51 39.92 28.45 39.06

Collaboration Definitely Does Not Address 3.35 2.75 0.00 2.78 Minimally Addresses 10.67 12.64 2.59 11.18 Mostly Addresses 40.55 40.84 50.86 38.20 Definitely Addresses 43.43 43.76 46.55 40.92 *Cells shaded in green indicate the highest rating for the guiding principle. Table 3. Percentage of Respondents Rating Early Childhood Professional Development System Guiding Principles at Each Level of a 4-Point Scale In addition to the three surveys discussed in Table 3, task force and committee members who were intensively engaged in workforce development efforts were asked to rate how well the professional development system is aligned with each of the guiding principles on a scale from 0 to 10, with 0 being “Not Aligned” and 10 being “Totally Aligned.” Average ratings of the guiding principles for this survey are:

Accountability – 7.65 Alignment – 7.74 Innovation – 8.26 Well-Financed – 6.30 Accessibility – 7.83 Collaboration – 8.30 Inclusivity – 7.52

All of these ratings are well above the mid-point rating of “5” and they are clustered within a two-point range from 6.30 to 8.26. Such a pattern would seem to indicate that the respondents, Colorado education leaders who have worked most closely with CDE and most intensively on the professional development system, are generally


in agreement that the workforce development efforts under RTT-ELC are aligned with the Guiding Principles of the P-3 Professional Development Plan. To compare the responses of the four sets of responses related to guiding principles, scores for individual principles were rank ordered by group, with a rank of 1 being the highest and a rank of 7 the lowest. A weighted average for all four groups was calculated to determine the overall rating for the guiding principles. This approach ensures that every response from all four data sources is given equal weight. Table 4 indicates that the highest rated principle is Inclusivity. Survey respondents indicated that CDE has worked very hard to build and implement an inclusive system representative of the work force. Their efforts have paid off as members of the workforce have indicated the system represents them well. Several comments indicated that they believe CDE and others have worked very hard to find ways to finance the system and that they have done a good job with this, but there is still much to do. As one respondent stated, “I do not believe there is enough funding to support early childhood professionals both to pay for college courses as well as compensation once they’re in the workforce (not CDE’s fault by any means). I applaud CDE for spearheading a lot of the work to implement creative solutions to address the compensation/benefits challenges.” Others were not so complimentary. As another respondent wrote, “I have not experienced the ’tie’ to increased compensation. As an administrator within a larger system, it is virtually impossible to set up policies that will guarantee increased salaries based on factors outside of the institutional policies.” These comments are representative of others in that they illustrate two important facets of the finance issue:

• the level of salaries, benefits, education stipends, and professional development offerings for early childhood educators is typically low and

• opportunities for improving the situation are few because the policies, procedures and funding streams are outside the authority of those who are most concerned about making positive changes.

Many respondents used the request to comment on guiding principles to express their concerns about the credentialing system and/or the Quality Rating and Improvement System (QRIS). The most frequently cited issue dealt with the level of financing available to support the system as a whole. While there was praise for effective use of the funds available through the Early Learning Challenge Grant, there was also concern that the funds were insufficient for supporting the complete implementation of all workforce development efforts. For example, some respondents cited their concerns that increasing qualifications are not tied to wage increases and that as far as they know there is not a plan to make that happen soon. Further, teachers aren’t necessarily reimbursed for professional development or formal education unless their employers specifically require them to participate. This makes it difficult for low-wage staff members to justify taking the steps necessary to improve their qualifications unless their long-term plans also include a move away from early childhood education into a more lucrative setting such as public school teaching or administration or higher education. One lesson to be learned from the collective comments related to funding is that early childhood programs across the state need supplementary

2014 Credential

2.0 Survey

N = 624*

2016 Credential

2.0 Survey

N = 1100*


Survey N = 116*

2017 Survey of

Committee and Task Force

Members

Weighted Average

Rank Across All Surveys

Accountability 2 5 4.5 5 5

Innovation 1 4 2 2 2

Access 5 2 6 3 4

Inclusivity 3 1 3 6 1

Alignment 6 6 4.5 4 6

Well-Financed 7 7 7 7 7

Collaboration 4 3 1 1 3 Table 4. Comparison of Rank Order of Ratings of Early Childhood Professional Development System Guiding Principles Across Four Surveys


funding to make workforce development easier for teachers and additional funds to provide a higher wage to early childhood educators to enable them to recruit and retain well-trained teachers.

Conclusions Colorado began implementation of the RTT-ELC grant activities with a strong foundation in place. Strong support for this work came from the Governor’s office, other state agencies, state and local early childhood programs, 2- and 4-year institutions of higher education, foundations and philanthropists, researchers and the broader community which benefits from high quality child care. This support has served the grant well as people across the state have volunteered their time, expertise, and in some cases financial support to help ensure that the RTT-ELC team achieved or exceeded the goals set out in the grant application. Such broad based support will also help ensure the sustainability of grant activities as they have by now become a part of the way of doing business in early childhood programs and school districts across the state. Colorado’s Early Learning Professional Development System Plan was in place when the grant was awarded and served as a guide and foundation piece for professional development efforts across the state. While some would say the items listed as innovative (a research-based tiered set of competencies, new quality-assurance and accountability mechanisms to ensure the effectiveness of college preparation and ongoing professional development and the effectiveness of early learning professionals; and a focus on building public and political support to increase public and private funding for the P-3 Professional Development System) are not truly innovative because other states and institutions have been engaged in such efforts long before the Plan was developed, few would argue about the innovative nature of the manner in which the Competencies were embedded in early childhood workforce development efforts. In a relatively short period of time, the Competencies were integrated into:

• Rules and regulations for teacher preparation programs leading to an educator license • College and university programs, including articulation agreements between and among community

colleges and their 4-year institution counterparts • Child care program licensing • Statewide professional development efforts • Training for early childhood coaches and trainers

According to all data sources (surveys, interviews, focus groups), the RTT-ELC team adhered to the guiding principles set forth in the Plan. While there were differences of opinion regarding how well each principle was addressed throughout all grant activities, there is general consensus that the team adhered to all of the principles in a consistently high quality manner, as evidenced by the fact that average ratings for all of the principles were above the mid-point of the scoring range.


Colorado’s Competencies for Early Childhood Educators and Administrators Colorado’s Competencies for Early Childhood Educators and Administrators (Competencies) were completed in July 2013. The Competencies, a research-based tiered set of professional practices, serves as the foundation for all early childhood professional preparation and ongoing professional development. This comprehensive guide is based on a set of content areas, or domains, describing knowledge and skills that early childhood educators need to work effectively with children birth through age eight and their families. Each domain is broken down into a set of categories, adding depth and breadth to the description of the domain. The Competencies are built on a framework of four levels. They begin with the basic knowledge and skills needed to enter the field and progress to advanced levels of academic preparation with a wide range of experiences. The four levels are cumulative, meaning that early childhood educators at the top level have the skills and knowledge to meet all the competencies in the lower levels. Early childhood educators progress through the levels by furthering education, participating in training opportunities, getting involved in mentoring/coaching, and utilizing reflective practices.

In support of the Competencies, CDE developed a competency self-assessment in both paper and online formats. The online version is embedded into the Professional Development Information System (PDIS) as a first step in the development of individual professional development plans. This interactive version offers important user benefits such as:

• Serving as a data-based foundation for reflection on professional practice, and

• Suggesting online or in-person courses offered through PDIS and/or institutions of higher education to help them achieve higher competency levels. Colorado’s Early Childhood Leadership Commission adopted Colorado’s Early Learning Professional Development System Plan created through the Lieutenant Governor’s Office with support from the Early Childhood Professional Development Advisory Group. A key priority within the plan was to develop a framework of performance-based competencies for early childhood educators and administrators. A Request for Proposals was announced, and Oldham Innovative Research was awarded the contract to work collaboratively with the Professional

Development Advisory Group’s Competencies Sub-Committee. The ECLC’s Program Quality, Coordination and Alignment Committee oversaw the process. Through many iterations of gaining feedback from both state and national experts, this framework outlines what early childhood educators and administrators need to know and do to provide quality care and education for Colorado’s youngest citizens.

Colorado’s Competencies for EC Educators and

Administrators Domains

• Child Growth, Development, and Learning • Health, Safety, and Nutrition • Child Observation and Assessment • Professional Development and Leadership • Family and Community Partnerships • Program Planning and Development • Guidance • Teaching Practices

Colorado’s Competencies for EC Educators and Administrators

Level 1: Demonstrates basic skills and knowledge to support quality early childhood care and education. Demonstrates the skills and knowledge of the previous level plus: Level 2: Applies skills and knowledge to implement effective early childhood environments and experiences for young children. Level 3: Designs, plans, and analyzes policies, procedures and practices that are optimal for young children. Level 4: Advances the field of early childhood education through advocacy, leadership, teaching, coaching, and mentoring.


Colorado’s Competencies are important to the state’s workforce development efforts because they:

• provide clear descriptions of what early childhood educators and administrators need to know and be able to do to provide quality care and education,

• are an essential part of a comprehensive professional development system,

• recognize the importance of diversity in early childhood settings, which include culture, linguistics, and ability, and

• support professionalism in early childhood education by serving as the foundation for practices carried out by early childhood educators in all early childhood settings.

The Competencies are designed to be used in a variety of ways depending on the setting. Ultimately, they are meant to clearly outline and communicate the specialized knowledge and skills early childhood educators need in order to provide high-quality education and care focused on children birth through age eight. Table 5 illustrates ways in which the Competencies can be utilized by early childhood professionals. While this report does not include an evaluation of the Competencies, their development or their application in practice, since two separate panels of early childhood experts reviewed them and approved their structure and content, it would be reasonable to assume that their content is high quality and on target based on current early childhood research. Further, the Professional Development Advisory’s Competency Subcommittee and the Early Childhood Leadership Council approved them as well as the RTT-ELC teams at CDE and CDHS. In addition to the expert panels, the content of the Competency Framework has been cross walked and proven to be strongly aligned with Colorado’s Teacher and Principal Quality Standards and with a set of validated early childhood teacher observation instruments that are the standards of excellence in the field. To understand the nature and level of quality for the Competencies, it is helpful to understand their importance for alignment, training and professional development, credentialing, and monitoring activities. The following information is provided to aid in understanding their role in the state’s workforce development initiatives.

Institutions of Higher Education Twenty (20) two- and four-year institutions of higher education have been working on a variety of activities designed to align their early childhood preparation programs with the Competencies. The following outlines some of their activities: • EC Competencies are being used as the foundation for rules and regulations related to the approval of teacher

preparation programs at 4-year universities. • Redesign of 13 community college early childhood programs based on the mapping of courses common to all.

Redesign is focused on addressing gaps between existing programs and Competencies. • Mapping of 4-year teacher preparation programs to EC Competencies to inform any redesign necessary to

address gaps and ensure consistent delivery of course content. • 4-year institutions with an early childhood program (licensure and non-licensure) are mapping their courses to

the competencies. This mapping contains more specific information about course/competency alignment than is required for the approval of teacher preparation programs.

• Statewide articulation agreements are under development.

Competencies are a foundational element of a professional development system.

Colorado’s Early Childhood Educator and Administrator Competencies identify what professionals providing care and education experiences need to know and be able to do

in order to facilitate child learning and development. The Competencies are

designed to take place within a larger, aligned and supported system that embodies continuous improvement and opportunities

for meaningful feedback through professional development.

Competencies At-A-Glance #1: An Overview


The list above represents major accomplishments on the part of the Early Childhood Professional Development team and the Higher Education Alignment and Articulation group, who continue to work on issues associated with ensuring aligned and articulated early childhood preparation programs across the state. The redesign of

Early Childhood Educators

• Plan for individualized professional development • Increase understanding of different levels of competence in the field of early childhood education • Promote self-assessment and self-reflection of knowledge and skills in the domain areas • Support portfolio development

Administrators

• Guide to help early childhood educators in their professional development plan • Support for development of personnel evaluation systems • Support for reflective supervision • Plan program professional development, technical assistance and/or mentoring/coaching • Guide staff recruitment and retention • Build leadership • Develop a career ladder within an early childhood organization • Provide opportunities for continuous quality improvement

Early Childhood Mentors and Coaches

• Create supports and resources in the competency domain areas • Promote discussions which lead to practices in self-reflection • Provide targeted support and modeling

Developers and Providers of Training

• Provide the foundation to guide learning goals and objectives • Create professional development opportunities that provide for meaningful articulation into other systems • Guide quality assurance specific to professional development • Organize professional development opportunities (i.e., professional development calendars, workshops at conferences)

Higher Education

• Support the foundation to create and align coursework, child care licensing requirements, and teacher licensure requirements • Guide the development of articulation and/or transfer agreements between institutions Version 5 6.29.16 • Foster dialog between students and advisors

State and Local Agencies

• Guide the development of policies related to teacher licensure, Colorado’s Quality Rating and Improvement System, articulation to higher education, professional development and supports for educators working in a variety of settings including public schools • Support program licensing credentials for early childhood educators • Support identification of specializations that could benefit from a deeper understanding of early childhood competencies • Create inventory of institutions that provide early childhood education professional development opportunities • Align standards in specializations that serve children and families birth through age eight

Early Childhood Advocates

• Increase awareness of early childhood education • Communicate knowledge and skills that early childhood educators should have in the profession • Identify supports needed for early childhood educators • Support sustainability and funding for early childhood programs • Create connections between economic impacts of early childhood (families back to work, employment of Early Childhood Educators)

Families

• Assist in the identification of early childhood program quality • Increase understanding of the early childhood profession • Advocate for their children and family needs • Increase understanding of the importance of what it means to have competent and credentialed staff • Act as a resource for Child Care Resources and Referral staff to use when families inquire about child care

Table 5. Utilization of Colorado’s Competencies for Early Childhood Educators and Administrators


community college programs not only addressed gaps between programs in existing prior to 2014 and the Competencies, but also brought together key staff from all 13 community colleges with representatives from 4-year institutions to discuss the competencies, expectations for graduates of all programs and for members of the workforce, and how best to prepare prospective early childhood educators for the workforce. Team members reported having shared such discussions at their individual institutions and brought back broad-based input for the alignment and articulation group to consider prior to make recommendations to the PD Advisory. Such discussions seem to have served the group well. During early summer of 2017, CDE surveyed both 2- and 4-year institution team members to get their input regarding minimum requirement for becoming an early childhood teacher and the components of an ideal articulation agreement (Figure 1). Their response, while not surprising, was demonstrative of the level of agreement between and among the respondents. Of the 18 respondents, 12 agreed that the child guidance strategies course should be a minimum requirement for prospective early childhood teachers. Similarly, more than half of the respondents agreed that Introduction to Early Childhood Education, Child Growth and Development, Introduction to the Early Childhood Education Lab, Parents and Families, and Health/Safety and Nutrition should also be minimum requirements. In other words, of the 14 courses suggested as minimum requirements, more than half of the respondents agreed on six that should be at the top of the list. Similar findings were found for requirements for community college early childhood certificates and director responsibilities for which more than half of the respondents agreed on eight of 15 options.

Colorado State Model Educator Effectiveness System In 2010, the Colorado legislature passed Senate Bill 10-191 requiring all educators whose positions require a state license to be evaluated annually against a set of rigorous quality standards. CDE’s Educator Effectiveness unit was charged with developing the educator evaluation system. The Early Childhood Professional Development team has collaborated with Educator Effectiveness staff members to ensure that early childhood educators are evaluated against the quality standards and that they are meeting Competency requirements as well. CDE has taken several actions to help ensure that the State Model Educator Evaluation System is aligned with the Competencies. First, early childhood staff members attended focus groups and informal meetings while rubrics were under development to discuss the need for inclusion of research based and developmentally appropriate practices for teachers of young children. When invited, they reviewed materials and offered suggestions for revising the rubrics in ways that would increase their alignment with the Competencies. They also monitored pilot test results to determine how well early childhood educators performed during the pilot test.

Figure 1. Minimum Courses Required to Become an Early Childhood Teacher in a Colorado Preschool or Community-Based Setting

0 5 10

Infant/Toddler LabAssessment (3)

Growth & Development Lab (1)Practicum 1

Observation (1)Infant/Toddler

Language & LiteracyExceptional Child

CurricululmHealth/Safety/Nutrition

Parents & FamiliesIntro to ECE Lab

Child Growth & DevelopmentIntro to ECE

Guidance Strategies

Number of Responses


When CDE became concerned about evaluators’ level of knowledge about what good early childhood teaching looks like in the classroom, early childhood staff members collaborated with Educator Effectiveness team members to develop Colorado State Model Educator Evaluation System: Practical Ideas for Evaluating Early Childhood Educators1, an implementation guide for evaluators and the teachers they evaluate that describes the features of high quality early childhood classroom as well as research based and developmentally appropriate professional practices that should be in evidence during instruction. Practices included in the guide are tightly aligned with the Competencies and the Teacher Quality Standards. Supporting the implementation guide and other early childhood educator effectiveness efforts is a detailed comprehensive crosswalk between the Teacher Quality Standards and the Competencies. For Educator Effectiveness, the primary concern was a concern that the Competencies address the Teacher Quality Standards. Early childhood educators primarily concerned about addressing the Competencies. The crosswalk indicated that the Competencies address all of the Teacher Quality Standards but the reverse is not true. This provided the early childhood team with guidance regarding necessary supplements to the State Model Evaluation System to ensure that all Competencies are being considered when early childhood educators are evaluated. In 2016, the Educator Effectiveness team engaged in a review of State Model Evaluation System pilot test and early implementation results to determine whether revisions to the system are needed and if so, what they should look like. Since CDE’s early childhood team had already been considering such issues, they invited Educator Effectiveness to join them and other members of Colorado’s early education community to determine how best to represent early childhood teachers in the revision process. Early Childhood Educator Evaluation Model Development Team members have laid out a series of activities designed to provide valuable information about not only whether the revised evaluation process is appropriate for early childhood educators but also whether they are meeting or exceeding the Competencies. During the 2017-18 school year, the early childhood team will:

• Partner with Educator Effectiveness to support the pilot test.

• Support the participation of early childhood educators and their evaluators in the pilot test through communication strategies and responding to questions or concerns from the field as appropriate.

• Review and comment on materials from Educator Effectiveness with a lens on early childhood educators.

1 http://www.cde.state.co.us/educatoreffectiveness/implementationguidance#practicalideasguides

Key Examples of Competency

Alignment

• Distributed $2 million in scholarships to support early childhood education and training programs

• Used as a part of approval process for Higher Education Programs

• Statewide articulation agreements • Revision/redefinition of community college

preparation programs • Two- and four-year institutions of higher

education aligned early childhood programs with EC competencies.

• New rules and regulations for child care programs licensed by state require professional development hours to align to Competency domains.

• CDE teacher preparation rules and regulations were changed to include the Competencies.

• Formal and informal input into the development of Colorado’s State Model Evaluation System

• Comprehensive crosswalk between Teacher Quality Standards and Competencies

• Crosswalks between Competencies and validated observation measures in common use in CO

• Colorado State Model Evaluation System: Practical Ideas for Evaluating Early Childhood Educators

• Embedded into statewide professional development offerings through a training approval process and matches professional development providers with PDIS users who need to improve competency.

• Incentives to teachers for students enrolled in early childhood programs at 2- and 4-year institutions

• Foundation for the Professional Development Information System

• Early Childhood credentialing system, Credential 2.0, is based on the Competency Framework

• Foundation for Demonstrated Competency component of Credential 2.0

• Colorado’s Early Childhood Workforce: 2020 Plan

http://www.cde.state.co.us/educatoreffectiveness/implementationguidance#practicalideasguides


• Suggest ways to ensure that data necessary to monitor early childhood educators is readily available. • Develop tools and materials, such as an update to the implementation brief, to help educators

understand best practices for evaluating early childhood educators.

Demonstrated Competencies The creation and implementation of all aspects of the Early Childhood Professional Credential 2.0 continued in 2015 as CDE engaged Clayton Early Learning to conduct a research review and synthesis to determine which instruments have good psychometric properties, are relatively inexpensive to purchase and administer, and are in broad usage in Colorado. Following a lengthy review and analysis of findings, Clayton staff members along with CDE identified a small set of instruments meeting the search criteria. It should be noted that the instruments identified through this process are also being considered for use as artifacts for early childhood educators being evaluated by the State Model Evaluation System. Please refer to the Early Childhood Professional Credential 2.0 section of this report for additional information about Demonstrated Competencies.

Conclusions and Recommendations In 2017 CDE rolled out an update to the 2010 Early Childhood Professional Development System Plan, which formed the backbone of the professional development efforts proposed for the RTT-ELC grant. Colorado’s Early Childhood Workforce 2020 Plan included the results of a policy and program scan conducted by Clayton Early Learning and the Colorado Children’s Campaign to look at funding, relevant programs and policies and examples from other states to inform implementation of the new plan. The ECPD team and Advisory reviewed progress to date on completing goals articulated in the 2010 plan. Table 6 provides a succinct summary of key accomplishments from the 2010 plan. As the table illustrates, six of the seven key accomplishments dealt with the competencies and how they have been aligned with and integrated into crucial components of the preparation, licensing, credentialing and ongoing development of the state’s education workforce. The 2010 Early Learning Professional Development System Plan included seven overarching goals. Colorado’s Race to the Top Early Learning Challenge Fund grant provided funding to create the infrastructure of the early childhood professional development system and allowed the state to achieve most of the goals set forth in the 2010 Plan. Through ongoing partnerships across the state, major accomplishments of the plan include: • Implementation of research-based Colorado’s Competencies for Early Childhood Educators and Administrators

(Competencies) serving young children birth to age eight, which provided the foundation for the early childhood professional development work.

• Development of the Colorado Shines Professional Development Information System (PDIS), an online learning management system and registry. Currently, the PDIS has over 33,000 registered early childhood professionals, who have completed over 157,000 online courses aligned with the Competencies.

• Creation of Colorado’s Early Childhood Professional Credential 2.0, which recognizes formal education, ongoing professional development, experience, and demonstrated competencies. As of June 2017, over 10,000 new credentials were awarded through the PDIS.

• Development of the Colorado Shines Professional Development Information System (PDIS), an online learning management system and registry. Currently, the PDIS has 38,900 registered early childhood professionals, who have completed 191,129 online courses aligned with the Competencies.

• Creation of Colorado’s Early Childhood Professional Credential 2.0, which recognizes Formal Education, ongoing Professional Development, Experience and Demonstrated Competencies. As of August 29, 2017, 10,635 credentials had been awarded through the PDIS.

• Incorporation of credentials, competency assessments, and professional development plans into Colorado Shines Quality Rating and Improvement system.

• Expansion of the Statewide Coaching Network, with (as of June 2017) more than 200 professionals receiving the Colorado Coaching Credential, more than 400 professionals completing the Relationship-Based Professional


The 2010 Early Learning Professional Development System Plan included seven overarching goals. Colorado’s Race to the Top Early Learning Challenge Fund grant provided funding to create the infrastructure of the early childhood professional development system and allowed the state to achieve most of the goals set forth in the 2010 Plan. Through ongoing partnerships across the state, major accomplishments of the plan include:

Development (RBPD) training, and more than 100 professionals participating in local reflective supervision groups. • Alignment of higher education early childhood programs and professional development training programs with the

Competencies. • Implementation of a $2 Million Race to the Top funded scholarship and incentive program for early childhood

professionals. Table 6. Key Accomplishments of the 2010 Early Learning Professional Development System Plan2 The EC Workforce 2020 Plan task force identified an area of work that needs greater emphasis during the coming years. Professional development efforts, in the opinion of the task force and the ECPD Advisory, have not resulted in an increase in the number of trainers and higher education faculty from diverse backgrounds nor in the number of mentorships and culturally relevant practicums for members of diverse populations. The task force, therefore, strengthened the approach to improving cultural relevance and addressing diversity issues in the EC Workforce 2020 Plan. One of the key strategies to accomplish this is the inclusion of an objective to “Actively prepare a workforce that reflects diversity of race, ethnicity, culture, language, and gender.” To achieve this objective, Colorado’s early childhood stakeholders will:

1. Review the Competencies “to confirm they completely address working with children and families with differences in ability, race, ethnicity, place, income, language and culture and family structure.”

2. Increase access to professional development for underrepresented populations. 3. Ensure professional development opportunities are available to prepare and support professionals to

work with diverse groups of children and families. In addition to this key change, the EC Workforce 2020 Plan authors wove references to improving diversity within the workforce and training early childhood teachers to include culturally reflective learning opportunities throughout the curriculum. Great strides toward developing, communicating about, and integrating the Competencies into the fabric of early childhood workforce development efforts have been made over the past three years, but there is still work to be done. For example, respondents to surveys, discussed in subsequent sections of this report, commented that they are concerned about their programs’ professional development offerings because they are not aligned with the Competencies. This may be a function of the level of maturity of the system in that statewide and online efforts were the focus of initial implementation while individual program efforts remain the responsibility of local programs which may not have the resources to realign their work to the Competencies. The work needed to drive Competency alignment to deeper levels of the system is both expensive and time consuming. Given the staffing level and budget for CDE’s Early Childhood Professional Development Team going forward, it is likely to be more difficult to address this issue in the future than in the past.

Recommendations: 1. Tap into existing networks to maintain and deepen communication channels regarding the linkage

between the Competencies and all facets of program implementation. 2. Develop and broadly distribute 1- to 2- page help sheets on topics related to system alignment and how

programs can improve their quality through making such linkages.

2 Colorado Department of Education (June 2017). Colorado’s Early Childhood Workforce: 2020 Plan. Retrieved from https://www.cde.state.co.us/early/copdplan on August 1, 2017.

https://www.cde.state.co.us/early/copdplan%20on%20August%201


3. Offer programs the opportunity to have a spot check conducted by CDE staff as time and other resources allow. This will provide staff members the opportunity to see first-hand the status of program alignment and to offer suggestions for addressing any misalignment they observe. A good practice would be to select only programs willing to share their lessons learned from the experience through word of mouth, social networking, conference sessions and serving on task forces and committees devoted to these issues.

CDE should take steps to deepen and broaden the scope of alignment of all facets of the state’s early childhood programs with the Competencies. This may prove to be difficult given current staffing levels, but it can be accomplished by:

1. Building on the goodwill they have developed over time with partners and collaborators. Engaging such networks to help with communication and education efforts will not only leverage the resources of a small staff but will also add credibility to their messages since they will be coming from entities known to local programs.

2. Using well-established communication conduits. Colorado’s early childhood community has many ways in which communication takes place, such as a variety of websites, blogs, newsletters, conference and meetings. CDE will be able to stretch communication resources by tapping into existing conduits to inform early childhood educators about new initiatives, system changes, and promising practices.

3. Developing user friendly materials on critical issues that can be distributed broadly to help users understand the full scope of and opportunities for alignment afforded by the Competencies.

4. Monitoring the degree to which the Competencies are aligned with national and state standards for early childhood educators and update them as needed. This practice will also enable users of PDIS, QRIS, and the State Model Evaluation System to maintain currency regarding the continuing relevance of their systems.

5. Continuing alignment and articulation efforts with 2- and 4-year institutions as they deepen their use of the competencies throughout their programs and implement revised articulation agreements.

One of the most important Competency alignment issues is the relationship between Teacher Quality Standards and the Competencies. All of the Teacher and Principal Quality Standards are addressed by the Competencies, but the reverse is not true. This means that use of the State Model Evaluation System will result in assurance that early childhood educators are measured against the Quality Standards and only some of the Competencies. CDE staff members from the early childhood and Educator Effectiveness units are working on this issue, such as using validated early childhood teacher observation instruments as a part of the State Model System evaluation system and perhaps revamping the ways in which early childhood educators are evaluated.

Recommendations: Closely examine and monitor the results of the 2017-18 pilot test of the revised State Model Evaluation System. Several analyses will help CDE understand whether the revised system is valid, fair and credible, such as:

1. Compare results for early childhood educators to those of teachers of older students to determine whether early childhood teachers are rated on par with other teachers.

2. Examine early childhood educators’ perceptions of system fairness and credibility and compare to previous years to note emerging and/or continuing trends.

3. Analyze ratings for teachers of young children on the State Model Evaluation System by making comparisons to other facets of their performance collected through PDIS such as how well they perform on validated tools and instruments used as Demonstrated Competencies.

4. Examine the alignment of the Competencies with new and emerging scientifically sound research and initiatives for early childhood teachers to ensure their continued quality and relevance.



Early Childhood Professional Credential 2.0

Credentialing System Legislation3 passed in 2000 established the Colorado Early Childhood Professional Credential. The legislation created a voluntary tiered system of credentials that recognizes increased knowledge and experience for those who care for and educate our youngest children. The early childhood field was actively involved in the creation of the credential system and has enthusiastically embraced it. In 2005, legislation moved the professional credentials from a pilot project to statewide availability. Credentials ensure that all professionals caring for and educating young children have the same core early childhood knowledge regardless of setting - family child care, child care centers, public schools. The Early Childhood Professional Credentials help create a system of education and training that promotes the optimal development and education of young children. The first goal outlined in the Early Learning Professional Development System Plan is, “Improve the effectiveness of early learning professionals by establishing and adopting an aligned, research based, tiered set of competencies as the basis for credentialing early learning professionals at all levels and approving professional development programs (including teacher preparation programs).” To align with these new competencies and to implement the next step of the Professional Development System Plan, the CDE’s Early Childhood Professional Development Team along with the Early Childhood Professional Development Advisory created a new framework for the Early Childhood Professional Credential (Credential 2.0). This framework provides a common system for Colorado early childhood professionals to document and quantify their professional growth and accomplishments, and by doing so to define and advance the profession. Individuals accumulate points along each of four pathways:

• Formal Education, Up to 50 points • Ongoing Professional Development, Up to 30 points • Experience, Up to 20 points • Demonstrated Competencies, Up to 10 bonus points during 2017

The point structure is based on scientifically sound research that indicates the likelihood of each pathway resulting in improved outcomes for children. As Figure 2 illustrates, formal education, professional development and experience are awarded points up to a maximum allowable for each.

3 The introduction for this section was adapted from: Williams, J. M. (2017). Early Childhood Education Credential 2.0: C ur r en t St a tu s an d C ha n ge s S inc e 201 4

Young children thrive when they have secure, positive relationships with adults

who are knowledgeable about how to support their development and learning and

responsive to their individual progress. Thus, the adults who provide for their care and

education bear a great responsibility. Indeed, the science of child development and early learning makes clear the importance and

complexity of working with young children from infancy through the early elementary

years, or birth through age 8, It also illuminates the essential need for

consistency and continuity in early care and education both over time as children develop

and across system and services.


Applicants are awarded 1 point for each year of experience up to a maximum of 20 points. One year of experience is defined as 1820 hours of work. This point structure is intended to equalize the credit given for experience regardless of full-time vs. part-time work or 9-month vs. 12-month assignments. While many of Colorado’s early childhood educators have more than 20 years of experience, only the first 20 years will count toward improving their credential level. This does not mean that additional experience is unimportant, but rather that our best evidence indicates there are also other important activities in which teachers and administrators must engage to ensure the best outcomes for all children. Those additional evidence based practices must also be considered in the determination of the quality of Colorado’s early childhood workforce. Not represented in Figure 2 is the fourth pathway, Demonstrated Competencies, which provides educators an opportunity to earn point based on the results of classroom observations using validated tools and instruments aligned with the Competencies. When fully implemented, this addition to the Credential scoring system will complete the four-pronged approach to obtaining and maintaining early childhood credentials. CDE decided not to change the Credential 2.0 scoring framework immediately after its completion at the end of 2016. They elected instead to conduct a pilot test during 2017 to determine:

• Who takes advantage of the opportunity to obtain points in this manner? • What are the demographic and professional characteristics of users who submit credentials to earn

competency points?

Figure 2: Points Available for Each Credential Pathway

[CATEGORY NAME]: 50 points Highest Degree - 20 pts. maximum Area of Study - 4 pts. for each degree in early childhood area

[CATEGORY NAME]: 20 points 1 point per year up to 20 years

[CATEGORY NAME]: 30 points Training hours: Up to 15 points CEUs: Up to 15 points 2 points for each training that

Tools and Instruments Used for

Demonstrated Competencies

• BAS (Business Administration Scale)/PAS (Program Administration Scale)*

• CLASS (Classroom Assessment Scoring System) o Infants o Toddlers o Pre-Kindergarten o Kindergarten to Third Grade

• Colorado State Model Evaluation System Professional Practices Score

• ECERS-R (Early Childhood Environment Rating Scale – Revised)

• FCCERS-R (Family Child Care Environment Rating Scale-Revised)

• ITERS (Infant and Toddler Environmental Rating Scale)

• PAS (Program Administration Scale)* • TPOT (Teaching Pyramid Observation Tool)

*BAS and PAS are under development as of August 2017


• Whether the Demonstrated Competency points make a difference in the credential levels of users who earned them? If so, what is the nature of the impact on credential level?

During the pilot test, users are able to earn 10 “bonus” points based on their classroom performance while being observed using one of the selected tools and instruments, including the Colorado State Model Evaluation System. Points earned during the pilot test will become a part of users’ PDIS records and will be valid for one year. Results of the pilot test will determine how the Demonstrated Competency pathway will be structured in the future. The framework for awarding credential points is also part of CDE’s vision of how best to ensure that Colorado’s early childhood workforce is well-prepared and capable of preparing young children to thrive in their educational endeavor. By assigning 50 points for formal education, framework developers signaled their belief in the benefits and importance of taking higher education courses and obtaining college and university degrees. In addition, formal education points may be earned by taking early childhood courses, having early childhood education as an area of study and taking courses aligned with the early childhood competencies. It should be noted that there have been many discussions related to the expiration of formal education points. While points earned through acquisition of a degree and area of study points do not expire, points awarded for coursework have an expiration date. In practice, this means that while more advanced degrees such as Master’s and Ph.D.’s are important and are awarded more points than an Associate’s or Bachelor’s, a person who earns a Bachelor’s degree with an area of study in early childhood would be awarded 14 points compared to the 15 points awarded for a Master’s degree in an area not associated with early childhood. It is hoped that this approach will incentivize prospective and practicing early childhood educators to deepen their understanding of childhood development and early childhood education through intensive study. Developers took a similar approach with professional development and training. While applicants may be awarded up to 15 of the 30 points available for professional development hours, the remaining points are reserved for intensive training and development such as maintaining credentials or obtaining new ones, or engaging in job-embedded coaching activities. This approach is not intended to lessen the importance of short-term training and professional development, but rather to emphasize research based practices with a greater likelihood of positively improving child outcomes. The credentialing framework places all of Colorado’s early childhood professionals who participate in the credentialing process into a common system through which they document their skills, knowledge and credentials to qualify for one of six Early Childhood Professional Levels through a combination of the total number of points earned from all four pathways and the number of pathways through which points were earned. The accumulated points and number of pathways required for each Early Childhood Professional Designation (level) are:

I – 10 to 20 pts. IV – 51 to 60 pts.

II – 21 to 35 pts. V – 61 to 70 pts.

III – 36 to 50 pts. VI – 71+ pts. The model uses Colorado’s Competencies for Early Childhood Educators and Administrators framework to integrate the pathways of formal education, training, experience, and specialized credentials and certifications. Individuals accumulate points along each of the four pathways. If applicants earn at least 10 points through two or more pathways, they qualify for a professional designation. The total number of points earned through all pathways determines which of six professional designations they earn.

Requires points from 2 pathways

Requires points from 3 pathways


In the spring of 2014, before the credential system was launched in 2015, CDE administered a survey regarding Credential 2.0 to determine users’ perceptions of the system and gauge how well the system was working at its earliest stages of implementation. The Early Childhood Professional Development Advisory and CDE staff members used the 2014 survey data for decision making regarding necessary system changes. In 2016, CDE conducted a modified replication of the 2014 survey. Modifications included revisions to questions to reflect the current system and to request user perceptions regarding how bonus points may be earned by demonstrating their competence on a variety of validated tools and measures and how those points may be used to improve their professional designations at higher credential levels. Both the 2014 and 2016 surveys asked respondents to rate the credential system on how well it adheres to the guiding principles established for Colorado’s professional development system and how fair the system is. As supplement to the survey analysis, the Early Childhood Professional Development Advisory requested that, where possible, validated data contained in the PDIS system be analyzed to compare to 2014 and 2016 self-report survey data. All three sources of information are used in the following discussions to show readers not only how survey respondents, some of whom are PDIS users, rated their experiences but also to show how the respondents’ self-report data compared to the data submitted to PDIS. This approach provided qualitative information reported through anonymous survey responses while also having the benefit of credential data that has been reviewed and certified as accurate by a team of well-trained early childhood educators.

Respondents and PDIS Users Eleven hundred early childhood educators responded to the survey between Labor Day and October 8, 2016 while 624 responded to the 2014 survey. As Figure 3 illustrates, the respondents represent all roles in which Colorado’s early childhood educators engage. The largest difference between 2014 and 2016 results is that ALL respondents reported their roles in 2016 while 50.48% responded to this question in 2014. In addition, 13.42% fewer 2016 respondents reported their roles as “Other” than 2014 respondents. In other words, this survey administration resulted in a clear picture of the roles in which respondents are engaged. In addition to responses from survey respondents, data for nearly 9,000 PDIS users who have earned credentials were examined. These data were analyzed using strategies consistent with those used to examine survey data so comparisons could be fairly and accurately made. One caution regarding PDIS data needs to be kept in mind: while data for credentialed PDIS users was examined, there are an additional 20,000 users who have not been credentialed. For this study, only credentialed users were included. The survey respondent profile was further illustrated by responses to a series of questions related to the four components of the credentialing system. For example:

• 73.91% reported having at least one degree awarded from and institution of higher education.

Figure 3. Roles of Respondents in 2014 and 2016 Credential 2.0 Surveys and from PDIS


• Nearly 60% of the degrees earned were not in any of the early childhood education areas for which they may receive credential points.

• 36.6% reported having participated in more than 50 hours of training related to early childhood education.

• 27.82% received the highest possible number of professional development points, primarily earned through training hours rather than the more intensive job- embedded, 6-hour trainings, series trainings, or endorsements.

• Of the 271 respondents who received no professional development over the last 3 years, 96 (35.42%) were teachers and 87 (32.1%) were directors.

• Colorado’s early childhood workforce is well-experienced with nearly 50% earning the maximum number of points (20) for experience. 34.62% of the respondents reported having more than 20 years of experience.

Points Earned Under the Credential 2.0 system, early childhood educators may accumulate 100 points through formal education (50 points), professional development (30 points) and experience (20 points). Points are accumulated and one of 6 professional designations is assigned to each early childhood educator depending on the number of points and number of pathways for which points are accrued. The distribution of total points earned (Figure 2) illustrates that the number increased steadily from zero through 50 points for survey respondents and zero through 30 for PDIS users, and then began a similarly steady decline. Fewer than 5% of respondents and users reported having more than 70 points. On all three measures, the majority of users/respondents earned 50 or fewer points indicating that the early childhood workforce has not topped out on the total points scale and that there is room for the demonstration of growth.

An examination of the methods by which credential points are earned illustrates the variations present: • Between methods of determining points earned and the framework for earning points approved for the

Credential 2.0 system • Between and among the surveys and PDIS

Figure 4: Total Points Earned in 2014 and 2016 and Through PDIS

0 1 to 5 6 to 10 11 to 20 21 to 30 31 to 40 41 to 50 51 to 60 61 to 70 71 to 80 >80PDIS 0.00% 5.29% 4.47% 23.54% 27.57% 19.60% 10.25% 6.44% 2.06% 0.69% 0.08%2014 0.00% 0.00% 6.80% 10.70% 19.70% 23.00% 24.50% 11.80% 3.50% 0.00% 0.00%2016 1.27% 1.82% 2.55% 7.73% 13.82% 16.36% 19.18% 16.91% 10.45% 4.55% 2.09%

0.00%5.00%

10.00%15.00%20.00%25.00%30.00%35.00%40.00%45.00%


Figures 5a to 5c illustrate the various measures for determining the total number of points earned. When compared to the framework, which established research based goals for developing and maintaining a high quality early childhood workforce in Colorado. Figures 3a and 3 b illustrate the impact of one system change that took place at the time the survey closed. Feedback from users as well as from CDE advisors indicated that setting an expiration deadline of 25 years for area of study points seemed to be counte r to the framework’s message regarding the importance of formal education. . It also seemed in conflict with the granting of credit for the highest degrees no matter how long ago the degree was earned, but not granting points for the area of study for the same degree. To address this concern, the scoring system was changed, eliminating the expiration of area of study area points. Figure 3a illustrates the points for degrees and areas of study prior to the change and Figure 3b illustrates the formal education points that would have been earned had the area of study points not expired. This change resulted in formal education accounting for 8% more points, or on average, 14.60 points compared to 10.15. The PDIS data indicated the lowest average number of points and the most evenly balanced method for earning them, with each pathway accounting for about a third of the total points. Neither the self-reported data or that extracted from PDIS indicates that users are earning points according to CDE’s proposed structure. For the 2016 survey, the lowest weighted component of the framework (experience) accounted for a higher percentage of the total points earned than the highest weighted component (formal education). In addition, according to the 2016 survey, professional development accounted for 40% of the points earned, compared to its 30-point weighting in the framework. A deeper analysis of these data indicated that most of the professional development points were earned through training hours rather than more intensive research based approaches. These data should be interpreted carefully because the system is still under development and this information serves as a baseline by which developers may measure progress as they implement system improvements over time. For example, it is hoped that formal education will account for 50% of the total credential points, but that is not the case. This is partially because survey data did not include points for early childhood coursework or for courses taken that align with early childhood

Figure 3a: Method of Earning Points on 2016 Survey with Area of Study Points Expiring after 25 Years

Figure 3c: Method of Earning Points in PDIS with No Expiration of Area of Study Points

[CATEGORY NAME]

Ave. .. Pts Earned:

9.11 32% of Total [CATEGOR

Y NAME] Ave. Pts. Earned:

6.79 24% of Total

[CATEGORY NAME] Ave. Pts. Earned:

12.65 42% of Total

Average No. of Points Earned: 28.55

Figure 3b: Method of Earning Points on 2016 Survey with No Expiration of Area of Study Points

Figure 5a: Method of Earning Points on 2016 Survey and with Area of Study Points Expiring after 25 Years

Figure 5b. Method of Earning Points on 2016 Survey with No Expiration of Area of Study Points

Figure 5c. Method of Earning Points in PDIS with No Expiration of Area of Study Points


competencies. While the PDIS data includes points for early childhood coursework, it does not include information regarding aligned coursework because the institutions of higher education across the state are still aligning their courses to the early childhood competencies. Until that work is completed, it will not be possible to determine the number of points earned for aligned coursework and therefore make a truly accurate assessment of the proportion of professional development points earned through each pathway. The differences between how users currently earn points and the proposed model, the ideal workforce, is partially due to the system’s newness. Baseline data reported here reflect activities prior to Credential 2.0 implementation. It is anticipated that as early childhood educators become more aware of system components and requirements of the credential process, they will seek professional growth opportunities aligned with credential requirements. The gap between the points distribution for the current workforce and the desired distribution reflected in the model can also serve as a needs assessment for the work that needs to be done to move Colorado’s workforce toward the ideal. The gap can also be characterized as the difference between the Credential 2.0 system, which is based on degree achievement and ongoing professional development, and Colorado’s minimum qualifications, which are lower than the system model. In addition, in community based programs and some district preschools, teachers are not required to have a degree, even at the Associates level. There are many factors that need to be accounted for in characterizing the differences between the status of the current workforce and the ideal:

• The level of understanding of how the scoring system works on the part of individuals who seek credential ratings.

• The magnitude of the difference between minimum qualifications to work in specific programs and those specified in the Credential 2.0 model.

• Compensation issues associated with the relative value to the individual of attaining higher credential ratings.

Credential Levels As Figure 6 illustrates, according to the PDIS, the largest group of users achieved a rating level of “II” while the largest group of survey respondents indicated their level to be “III.” It should be noted that levels “0” and “3” include respondents who accumulated more than required to move to the next level, but their points were not earned through the required number of pathways. For example, some PDIS users reporting having more than 10 years of experience but they did not report having formal education or professional development activities. Since they have points in only one pathway, they remain at the “0” level. A similar situation exists for educators who have enough points to qualify for level “IV” but have points for fewer than three pathways.

Demonstrated Competencies One of the four pathways to obtaining points, Demonstrated Competencies, allows professionals to demonstrate their depth of knowledge of early childhood and their ability to support children’s growth and development. While the goal of Formal Education, Training and Professional Development and Experience is to increase competence, the goal of the Demonstrated Competencies pathway is to incorporate current research-based measures of

Figure 6: Distribution of Respondents by Early Childhood Professional Designation (Rating Level) for 2014 and 2016 Survey Respondents and PDIS Users

0 I II III IV V VIPDIS 6.61% 21.18% 39.31% 25.84% 8.75% 3.42% 1.58%2014 12.30% 13.20% 19.20% 26.30% 26.80% 1.10% 0.01%2016 8.61% 7.27% 20.55% 31.18% 15.09% 10.45% 6.45%

0.00%5.00%

10.00%15.00%20.00%25.00%30.00%35.00%40.00%

PDIS 2014 2016


EC professionals’ competencies and best practices into the credentialing system. Beginning in 2014, the Early Learning Challenge Grant provided funding to research and adopt measures of the EC Competencies Framework and develop scoring for this section. Validated tools and instruments that are aligned with Early Childhood Competencies, in broad usage across the state and accessible for most educators have now been identified and a preliminary scoring system developed. Initially, Demonstrated Competency points will be considered as “bonus” points with no credit given for the additional pathway. In other words, a person who is within 10 points of qualifying for the next credential level may qualify to move up a level with the additional Demonstrated Competency points. However, if that person also needs an additional pathway to qualify for the next higher level, he or she will not qualify for the higher level. As a part of an optional training for submitting credentials to obtain Demonstrated Competency points, CDE provided webinar participants an opportunity to respond to a survey related to this new credential pathway option. Participants earned one professional development clock hour for their participation. Survey results indicated there is general agreement among users and potential users that this approach would be well-received. For example, when asked which instruments they would recommend to be used for Demonstrated Competencies, their first and second choices (Figure 8) were those used throughout the state, which were also selected by Clayton Early Learning and CDE staff. When the survey respondents were asked if the Demonstrated Competency option would increase their credential level, 74.1% indicated they thought it would. When asked if they agreed with this approach to awarding points, 91% responded in the affirmative. In other words, 25% of the respondents believe this is a good approach even though they believe it would not help them personally. This compared to data currently available may help answer the question regarding how many educators may benefit from the additional 10 points available to them through Demonstrated Competencies. The answer to this question (Figure 8) depends on the credential level. About ¼ of one percent of

Figure 8. Will Demonstrated Competency Points Make a Difference in Credential Levels: 2016 Survey Responses

0 I II III IV V VIYes 0.27% 6.82% 14.18% 14.64% 15.09% 10.45% 0.00%No 8.36% 0.45% 6.36% 16.55% 0.00% 0.00% 6.64%

0.00%5.00%

10.00%15.00%20.00%25.00%30.00%35.00%40.00%

Figure 7. Tools Users Would Select to Use For Demonstrated Competencies.

BAS 4%

St Mod Sys 9%

ECERS-R 17%

FCERS-R 6%

ITERS 9%

PAS 6%

TPOT 6%

Other 43%

Figure 9. Method of earning points on 2016 survey with no expiration of area of study points.


0.00%5.00%

10.00%15.00%20.00%25.00%30.00%35.00%40.00%


the 2016 survey respondents who did not qualify for Level I would benefit from additional points. This is primarily because they have points in two pathways. Many of the respondents reported having earned no points, so the additional 10 would not qualify them for Level I, which requires having points in two pathways. This compares to Levels IV and V for which all respondents would be able to increase their level if they had the 10 extra points. Again, this is because educators at these two levels have already earned points in three pathways and would only need points to move up a level, not additional pathways, to move up one level. Overall, 61.45% of the 2016 survey respondents would qualify for a level increase if they earned all or part of the 10 additional Demonstrated Competency points. An analysis of PDIS information (Figure 9) yielded similar results, although users tended to have lower levels overall as demonstrated by the higher number of users who received ratings at levels I and II when compared to 2016 survey respondents. It must be pointed out that the survey respondents and PDIS users are not compared because they are mutually exclusive groups but rather the comparison is based on two different methods of reporting the data. The Credential 2.0 framework calls for four pathways, including Demonstrated Competencies. The pilot test includes an option for users to earn points only, with no credit for the additional pathway. A different view of the Demonstrated Competencies option emerges when giving credit for the pathway is considered. As Figure 10 illustrates, 7.00% of the respondents to the 2016 survey who were originally at the “0” rating level would earn a higher rating if both points and the additional pathway are counted. This compared to Figure 8, which indicates that 0.27% of the same group of respondents would earn an increased rating, a difference of 6.73%. This represents 74 educators who would not qualify for Level I without credit for Demonstrated Competency points and pathways and who would now be rated at least at professional credential Level I. The net effect of awarding Demonstrated Competency points only would be 676 (61.45%) of the 1100 survey respondents rated at least one credential level higher compared to 835 (75.91%) with a higher level if both points and the additional pathway are taken into consideration. An important feature of the credential scoring system is highlighted by these data: the level of rigor imposed by the requirement for earning points through multiple pathways. For both survey respondents and PDIS users, two levels seemed to “catch” credential applicants and slow their movement up the scale: movement from level “0” to “I” and from level “III” to level “IV.” A deeper analysis indicated that most of the people who did not move up a level as a result of the additional 10 points failed to do so because they had not met the pathways requirements of 2 for levels “I – III” and 3 for levels “IV – VI”.

Guiding Principles and Fairness The state’s plan for professional development identified seven principles that should guide the work of improving the early childhood workforce. To determine whether the Credential 2.0 approach reflects those principles, respondents were asked to rate how well the approach addresses each principle. The results (Figure 11) illustrate that the respondents from both years rated the level of alignment very similarly. Alignment ratings increased

Figure 10. Will Demonstrated Competency points and pathways make a difference in Credential Levels: 2016 Survey Responses


0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%


steadily as one moves along the scale from “Not At All” to “Completely,” indicating a perception that CDE’s credential approach is adhering to the guiding principles. This is true for all principles. Of interest to CDE and the ECPD Advisory are the differences between the ratings of guiding principles ratings between 2014 and 2016, when the Credential 2.0 surveys were administered (Table 7). The most noticeable feature of this chart is the number of respondents who did not respond to the questions about alignment between guiding principles and the Credential 2.0 system decreased by about 30% across the board between 2014 and 2016. In other words, about 30% more respondents were willing to express their opinions about how well CDE is doing with respect to aligning the principles to Credential 2.0 efforts. For the accountability principle, there was a 2.00% gain in the number of respondents who gave a rating of “Not at All”. Similarly, 21.79% more respondents gave a “Sort of” rating in 2016 than in 2014. This compares to the 12.07% for “Pretty Close” and 10.51% for “Completely” for same principle. This indicates that respondents viewed the system’s alignment with the accountability principle more negative in 2016 (23.79%) than in 2014 (22.58%). Similarly, innovation was rated lower in 2016.


The ratings changes in the negative direction are balanced by the positive ones. The largest positive increase (18.95%) was for “Accessible,” indicating that respondents were more able in 2016 than in 2014 to gain access to the system. The 18.95% increase in respondents who completely agree combined with the 10.48% who rated “Accessible” as pretty close, represent nearly 1/3 of the sample of 1100. A similar number rated “Inclusive” more positively in 2016. Respondents also seemed to have noticed the work on “Aligned,” “Well-Financed,” and “Collaborative” as just over 25% rated each of these more positively in 2016 than in 2014.

Figure 11: Ratings of Credential 2.0 System Adherance to Guiding Principles in 2014 and 2016 Note: Acct.=Accountable Innov. = Innovate Access. = Accessible Inc. = Inclusive Align = Aligned Finance = Well-Financed Collab. = Collaborative

Acct. Innov. Access. Inc. Align. Finance Collab.2016 2.97% 3.55% 3.84% 3.55% 4.00% 6.63% 2.75%2014 0.90% 1.78% 2.69% 2.10% 2.12% 8.51% 3.35%

0.00%

20.00%

40.00%

60.00%

Not At All


0.00%10.00%20.00%30.00%40.00%50.00%60.00%

Sort Of


0.00%10.00%20.00%30.00%40.00%50.00%60.00%

Pretty Close


0.00%10.00%20.00%30.00%40.00%50.00%60.00% Completely

Not At All Sort Of

Pretty Close Completely

No Opinion/Missing

Accountable 2.00% 21.79% 12.07% 10.51% -29.76%

Innovative 2.10% 19.87% 19.87% 10.54% -32.06%

Accessible 1.89% 15.06% 15.06% 18.95% -32.98%

Inclusive 1.92% 15.54% 15.54% 15.09% -32.09%

Aligned 2.35% 15.38% 15.38% 11.90% -33.94%

Well-Financed 1.22% 12.66% 12.66% 13.57% -33.49%

Collaborative 0.60% 15.71% 15.71% 13.67% -33.25% Table 7. Changes in Ratings of Adherance to Guiding Principles Between 2014 and 2016


Fairness of the Credential 2.0 system is critical to its success, as users who view it as less than fair are less likely to find their rating levels credible and to use the results as they were intended: to improve performance in the classroom and enhance the likelihood that their students will achieve higher outcomes. Perhaps the most important reason to consider fairness at this stage of system development is that fairness is an overriding, foundational concern in developing systems such as this. Common principles of fairness need to be applied in responding to user characteristics that could interfere with the validity of test score interpretation.4, In other words, if the system isn’t fair, it is likely that it will not be valid. Users’ perceptions of fairness are as important to the development and implementation of a system such as this is equally important as lack of bias and validity of the measures used to arrive at credential levels. Survey respondents provided similar ratings of the fairness issue in 2014 and 2016 (Figure 12) with less than two percentage points difference between the ratings at any level of the rating scale. The smallest number of respondents (2.56% in 2014 and 4.36% in 2016) rated the model “Not Fair At All,” while 32.98% in 2014 and 60.55% in 2016 rated it “Somewhat Fair,” “Fair,” or “Extremely Fair.” When the results of the two surveys are plotted on the same axes, the opinions of the two groups of respondents (2014 and 2016) look similar. More than 35% of both groups rated the system as “Fair” and the difference between them on “Extremely Fair” is less than 1%. Fewer than 5% rated the system as “Not Fair at All” on either survey.

Conclusions and Recommendations Several ongoing themes emerged during the analysis of respondent comments. These are briefly discussed below, along with preliminary recommendations for addressing problematic issues.

1. The methods by which educators earn credential points is still somewhat contentious with opinions regarding adjustments to the scoring framework depending on the personal and professional backgrounds and roles of the educators involved in discussions. For example, Family and Childcare Providers have cited the importance of experience over formal education, stating that their years of experience have helped them understand what children need and how to establish a loving and caring environment for the children in their care. Educators who entered the field having earned non-ECE focused degrees have been more likely to cite the importance of coursework in ECE over obtaining an ECE degree. In the presence of such varying backgrounds and opinions, it can be difficult to determine whether changes need to be made and if so, what they should look like.

Recommendation: For the immediate future, CDE would be well served to maintain the scoring framework as currently constructed. The system is still in the earliest implementation phase so there

4 Joint Committee on the Standards for Educational and Psychological Testing of the American Educational Research Association, American Psychological Association and National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association, p. 49.

Figure 12: Fairness of the Credential 2.0 Model

Not Fair AtAll

SomewhatFair Fair Extremely

FairNo

Response

2016 4.36% 19.55% 35.55% 5.45% 35.09%

2014 2.56% 20.67% 35.90% 6.41% 34.46%

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

45.00%


hasn’t been enough time for impactful change across the state to be measured. In addition, the framework is based on scientifically sound research that holds promise for bringing about such changes given time. Further, there is not enough qualitative or quantitative data at this time to justify changes.

A better strategy is for CDE to monitor the proportion of credential points earned through each pathway to provide a longitudinal snapshot of changes to workforce development habits over time and to identify gaps that must be closed to meet the established system goals. Staff should examine the data underlying these figures to gain a clearer picture of how early childhood educators earn credential points and to identify the actions needed to improve the likelihood that course-taking habits will change and the number of people earning ECE degrees will increase. The point structures for educators relying on professional development to improve or maintain their credential levels should also be monitored to determine whether there is movement away from less effective professional development strategies to more intensive, long-term and job-embedded strategies.

2. Credential applicants have raised concerns about variations in the amount of time between earning points

and when they expire. The most frequently cited concern on this issue is that points earned for degrees never expire, but those earned for coursework expire after 15 years. CDE conducted a high-level literature scan to determine the standard for the field and determined that institutions of higher education typically require degree-seeking students to re-take courses if the credits were earned 10 or more years in the past. Some colleges and universities set their expiration periods at as few as seven years.

Recommendation: This issue is like the one above in that developers of the credential system used their best judgment and current research to design a rigorous, credible and relevant system. They should give the system enough time to determine whether the 15-year expiration period serves as a disincentive for educators to enter or stay in early childhood programs, whether it seems to be biased against any group of educators, and whether there are other negative impacts caused by course expirations. Should either of those be the case, the expiration period should be reconsidered.

3. Demonstrated Competencies are the newest addition to

the opportunities for early childhood educators to earn credential points. The pilot test allows educators to earn up to 10 “bonus points” for demonstrating proficiency on validated tools and instruments that measure teacher’s performance through classroom observations. The pilot test does not offer credit for the additional pathway represented by demonstrated competencies. This conflicts the Colorado Early Childhood Professional Credential 2.0 Initial Scoring System, which discusses having four pathways for earning points with Demonstrated Competencies as one of the four.

Recommendation: Awarding bonus points without giving credit for the additional pathway seems to be a reasonable approach for the pilot test, but not for the long term. At a minimum, the discrepancy between the scoring framework which discusses four pathways and the practice of awarding only points must be resolved and the resolution communicated to early childhood educators.

“I am a fan of the credentialing system in general. It does well to

support people who are new to the field and outlines for them what skills and knowledge they should exhibit in

order to grow in their professionalism. I have concerns that the system does

not well accommodate or acknowledge individuals who clearly

exhibit a commitment to the field and who engage in best practice on a

daily basis. I have concerns that the PDIS may not serve to incentivize

people to stay and grow in the field.”


Note: The operationalization of the Demonstrated Competencies pathway is important because it may reduce concerns about the fairness of the Credential system based on three pathways. Through this option for earning points, educators can earn credential points based solely on their classroom performance without concern for their experience level, degrees held, or how well they have improved their skills and knowledge through professional development. In other words, this is their opportunity to show how well they have internalized the Competencies and how well they apply the lessons learned through experience, education and training by establishing nurturing classroom environments in which students excel. Their demonstration of proficiency in the classroom may help them earn a higher credential level in spite of expiring points or lack of a formal degree.

4. Some early childhood educators who are using PDIS have expressed concerns about the meaning of the rating levels: I through VI. Explanations of the distinctions between and meaning of the individual levels have proven to be difficult because the rating scale is a continuum from Level I for entry level child care professionals to experts in the field (Level VI). This has proven to be frustrating for some PDIS users who are looking for ways to describe and therefore to understand where they fit along the continuum. One of the reasons CDE has not provided more information about distinctions between the levels is that the “scores” (Levels I through VI) have not been subjected to a validation study that would examine their relationship to other validated instruments such as CLASS and ECERS. Explanations such as this would help users to anchor the levels to something that already has meaning for them.

Credential levels are used in program rating levels 3 through 5 of the Colorado Shines Quality Rating and Improvement System (QRIS) as a way of confirming the quality of a program’s workforce, one of 5 categories used to determine overall program quality. QRIS points for the quality of a program’s workforce are based on the credential levels of the Director, Early Childhood Teachers and Assistant Early Childhood Teachers.

Colorado developed a Credential/Licensing chart (Table 8) to provide a high-level explanation of the relationship between credential

levels and personnel requirements within childcare licensing rules and regulations. Meeting these requirements is Level 1 in the Quality Rating and Improvement System (QRIS). Levels 3 – 5 of QRIS means that a program is going above and beyond minimum licensing requirements. Programs with QRIS levels of 3 – 5 earn more points when staff have higher credential levels.

Categories Used to Determine Program Ratings in Colorado

Shines Quality Rating and Improvement System

• Workforce and Professional Development • Family Partnerships • Leadership, Management and

Administration • Learning Environment • Child Health

Early Childhood Professional Credential 2.0

Child Care Licensing Center Rules: Personnel

Qualifications Equivalency Level I Assistant Early Childhood Teacher

Level III Toddler Program Staff Early Childhood Teacher Kindergarten Teacher Small Center Director

Level III with the completion of ECE 111/EQIT and 1 year experience working with infants and/or toddlers

Infant Nursery Supervisor

Level IV Large Center Director

Table 8. Early Childhood Professional Credential 2.0/Child Care Licensing Center Rules Personnel Qualifications Equivalency


Recommendations: At only two years since the initial statewide rollout, The Colorado Shines PDIS/QRIS and the Early Childhood Professional Development Credential 2.0 systems are still in their infancy. As such, there has not been time for system modifications and just-in-time corrections to have stabilized to the point that they are ready to be subjected to comprehensive validation and effectiveness studies. That point is nearing, however, and at a minimum the individual ratings and program ratings should be validated for the uses for which they are intended. In other words, a determination need to be made regarding whether educator performance improves as the move up the continuum. The research question of interest in such a study is whether the credential system is measuring what it is supposed to be measuring and whether the resulting scores are appropriate for the intended uses.5 A determination of the appropriateness of using credential ratings for the purpose of rating programs also needs to be confirmed. In theory, this seems to be a reasonable approach for looking at one facet of program quality, but the underlying theory needs to be proven. This is an important validity question for both the individuals and programs and it needs to be answered if the system is to be deemed valid, credible and fair.

5 Child Trends conducted a validation study of QRIS in which they concluded that Levels 3 – 5 indicate a higher level of program quality than Levels 1 and 2.


Professional Development Information System

Colorado’s Early Learning Professional Development System Plan was approved by the Early Childhood Leadership Commission in 2010. That plan includes aligning training for early childhood professionals with EC workforce competencies, development of an interactive web-based clearinghouse to advise early learning professionals, provision for an early childhood workforce registry system, development of a data system linked to other important early childhood data systems, and the ability to develop reports for key stakeholders and policy makers. According to Colorado’s application for RTT-ELC funds, the PDIS will:

• Allow Colorado to gain information about the early childhood workforce to inform program support, system-level professional development planning, and delivery of resources.

• Provide a connected and organized professional development resource for all early childhood professionals in the State of Colorado.

• Assure a well-coordinated system of credentialing for the early childhood workforce. • Track trends over time to evaluate early childhood workforce development and its role in impacting

outcomes for children birth to 8. Understanding and supporting the workforce who cares for and educates young children will improve Colorado’s efforts to assure a strong start for young children as they enter school as well as support the goal of reading by third grade. The PDIS was made available to the entire Early Childhood Workforce on April 15, 2015. The Early Childhood Competencies Self-Assessment is available online through the PDIS. Once the self-assessment is completed the PDIS generates an initial Individual Professional Development Plan with recommendations. The system was developed with Colorado’s Competencies for Early Childhood Educators and Administrators as the foundation and all professional development offerings within the system are aligned with these competencies. A wide range of professional development opportunities are available to users in a variety of formats for both individual and group use. The PDIS allows early childhood professionals to manage their own career and professional growth using an initial Individual Professional Development Plan which includes professional experience, education, and training as well as individually constructed growth plans. As of August 29, 2017, 48 online self-paced courses were available through PDIS. These ranged from how the system was developed to ensure that the Competencies are embedded throughout to how to use the system to courses designed to prepare educators for specific in classroom practices to help ensure positive outcomes for all children, particularly those with high needs. Since PDIS was launched in April 2015, 41,420 users have completed 208,591 courses.6 In addition, 10,877 credentials have been awarded and 51 online self-paced courses are available 24/7 for early childhood educators.

Training and Support to Use the System

6 Data obtained from PDIS on October 13, 2017.

The Colorado Professional

Development Information System (PDIS) is “an online data system and workforce registry designed to track

the training and education of all professionals in Colorado’s early

childhood workforce. The PDIS also provides online professional

development in the form of self-assessments and e-learning, and

assists early childhood professionals in attaining additional training and

education to help them meet their professional goals.”


CDE has provided a variety of trainings and other types of support for PDIS users and potential users. Early on CDE staff supported the early childhood councils by giving them the tools and resources needed to conduct trainings at the local level while CDE staff members focused on statewide and regional activities. One of the most important of these efforts was support for improving the equity of access to necessary technology by gaining support of resource and referral agencies, public and school libraries, and computer laboratories willing to open their doors to the councils and early childhood educators who needed computer access to enter their credential information into PDIS. Many of the councils used their own facilities for computer labs and help sessions. CDE also provided hands-on laboratory sessions at regional and state conferences, taking computers, scanners and other equipment to the conference site and showing potential users how to import their materials into PDIS. Table 9 provides a brief list of PDIS communication and training activities conducted by CDE. Since PDIS courses are required for programs to earn a Level II QRIS rating, face-to-face training options were arranged for people who weren’t ready to make the jump to a technology based professional development system. As one CDE staff member stated, they provided high-tech support in a high-touch manner to make everyone feel at ease in using the new and still unfamiliar system and to help as many educators as possible qualify for a credential. In addition, CDE provided a fully staffed help desk and resources in schools to help novice users.

Additional Resources Related to PDIS

• Top 10 Reasons the PDIS

Benefits EC Professionals • Key Messages About the

PDIS • PDIS Fact Sheet: PDIS FAQs

Getting ready for QRIS/PDIS Webinar

• Paper version of Early

Childhood Competencies Self-Assessment (Also available through the PDIS)

• Paper version of

the Individual Professional Development Plan (Also available through the PDIS)

• PDIS 101 - An introductory

level presentation about the PDIS with script notes

Timeline Activity Description

Summer and Fall 2013

Regional Listening Tour Two-hour meetings conducted in all regions of the state to provide a high-level and broad overview of the Early Childhood Educator and Administrator Competency work as well as the establishment of the new process for awarding early childhood professional credentials that would follow soon after approval of the competencies.

Summer and Fall 2013

PDIS Launch Sessions Regional sessions with early childhood councils designed to help child care resource and referral agency staff and early childhood council members understand how the new credentialing system and PDIS would work. These sessions presented information to council members about resources, licensing specifications, coaching, and how PDIS would generate the professional credentials.

2014

Quality Rating Improvement System (QRIS)/PDIS Information Sessions Co-Hosted by Local Councils These informational sessions provided participants an advance look at QRIS and PDIS as they were developed at the time. It was designed to gain council buy-in and give council members an idea of how they would be able to support the system once it was rolled out.

Early 2015 Colorado Shines Kick-Off Events

Spring and Summer 2015

Colorado Early Childhood Leadership Councils Session content was focused specifically on what councils should do to support workforce development efforts.

Annually

Presentations at Professional Organizations such as the Association for the Education of Young Children, the Colorado Association for the Education of Young Children and the Rocky Mountain Early Childhood Conference. These presentations included information sessions regarding the status of grant activities, next steps, questions and answers about the workforce development program, and opportunities for participants to interact with staff members and try out new tools and technology.

TABLE 9: CDE/CDHS Collaborative Communication and Training Activities Regarding Workforce Development

https://www.cde.state.co.us/early/toptenpdis

https://www.cde.state.co.us/early/toptenpdis

https://www.cde.state.co.us/early/keymessagespdis

https://www.cde.state.co.us/early/keymessagespdis

https://www.cde.state.co.us/early/pdisfactsheet

https://www.cde.state.co.us/early/pdisfaq

https://www.youtube.com/watch?v=qpShPYCfIQM

https://www.youtube.com/watch?v=qpShPYCfIQM

https://www.cde.state.co.us/early/eccompetenciesselfassessment



https://www.cde.state.co.us/early/individualpdplantemplate

https://www.cde.state.co.us/early/individualpdplantemplate

https://www.cde.state.co.us/early/anintroductorylevelpresentationpdis


These efforts exceed the goals and activities articulated in the RTT-ELC application. It is evident that as PDIS was developed and has continued to evolve, staff members have found ways to support the parallel evolution of users as they have learned to access and use the system. One staff member stated, “We could never have imagined getting as far as we have back in 2013 when we got started.” And yet, there is still much to be done to improve equitable access and effective system use. Because feedback regarding the system is almost always solicited through online methods, it is likely that such feedback has a certain amount of bias against users who aren’t able to or don’t want to access the system through the use of computers. Strategies for collecting feedback through non-technology based methods should be implemented to avoid such bias. In addition, all face-to-face training sessions should include a segment on PDIS, how to access it, how to use it, and, most importantly, how to access the Help Desk and other supports should they need it while learning to use the system.

PDIS Trainers and Trainings In 2015, CDE developed a framework (Figure 13) for the EC Training Approval system. The goal of the system is to increase the availability of and access to quality trainings aligned with the EC Competencies. CDE provided an Adult Learners Course and a Training Design Course to support the efforts of prospective trainers who wanted to develop courses for inclusion in PDIS. As a part of the application process, CDE supported applicants by helping them to develop key ideas, gather what was needed to align their work with the Competencies, and develop instructional design approaches appropriate for both the content and the learners. A 360-degree feedback process provided prospective trainers with ideas for improving their trainings including ways to strengthen the alignment with Competencies and adaptations for different types of trainings and stakeholders. To receive approval of trainings, developers are required to have in place a training design or plan that supports quality training that adheres to adult learning principles and is in alignment with the EC Competencies. In the absence of these characteristics, trainings would not be approved for use in PDIS. In 2016, to more clearly describe the department’s focus on alignment of trainings with the Competencies and to avoid confusion with approved training vendors for licensing, “Training Approval” was changed to “Training Alignment and Approval” in the framework.

Goal: Increase the availability and access to quality trainings aligned to the EC Competencies.

Trainer Credential• Recognize Expertise• PD System Delivery• Support Best Practices• Identifies Trainers• Access to Resources and

Supports

Training Approval• Identifies Trainings• Increases Access• Aligns Trainings to

Competencies• Supports Quality Training

Design

Colorado Core Competencies for Trainers

Adult Learners Course Training Design Course

Phased in Approach

Figure 13: Framework for Phased in Approach to Training Source: EC Training Approval System Presentation, October 2015


Tight connections between goals, competencies, learning objectives, and the training activities are at the core of all PDIS trainings. Figure 14, taken from the training design course, illustrates the process trainers are encouraged to use to ensure such connections. The training design model provides a clear line of sight from the goal the training is designed to address, to the competencies, learning objectives, and finally the activities in which learners are expected to engage. CDE set a goal of having one training per Early Childhood Council by the end of 2015. To attain that goal, CDE supported the Councils with incentives to help them with training and development capacity. CDE also supported them by assisting with the application and review process, and providing training and technical assistance as the courses were being developed. In light of the lessons learned during the pilot test of the revised Training Alignment and Approval process, CDE developed an interactive Training Wizard housed within PDIS that assists trainers and prospective trainers with their course development and competency alignment work.

Characteristics of Professionals Using PDIS Surveys and other data collection efforts have been conducted to monitor system use and guide decision makers as they have made mid-course corrections and decisions about additional supports needed to improve the system. Child Trends conducted a survey focused on PDIS in 2015 as PDIS and the Early Childhood Professional Credential 2.0 system were being integrated into Colorado Shines. In addition, two surveys focused on Credential 2.0 were conducted in 2014 and 2016. To supplement the 2016 survey, PDIS data were examined to help explain who uses the system, how it is being used and to what effect. Child Trends administered a surveyiv, of the 12,505 PDIS users who had registered with the system in October 2015. Respondents were primarily females in the 25 to 54 age group and evenly divided across the 1 to 15, 6 to 10 and 11 to 20 years of experience groupings. One-fourth had completed some college course work but had not received a degree while 19% had earned a Bachelor’s degree, with 16% having a major in early care and education or a related field. Nearly half of the respondents worked in a child care center with the remaining 53% working in settings such as family child care, Head Start programs, and public school settings. About 31% were lead teachers and approximately half reported holding either a Level I (20%) or Level II (30%) credential. One of the reasons for encouraging people to access the system early on was simply to make them aware of its existence. Child Trends’ survey indicated that 61% of the 3,479 users who responded to their survey “heard about the system from their program director or administrator.”v This is consistent with findings from the Credential 2.0 survey conducted in 2014. Soon after system rollout, users were encouraged to set up their PDIS profiles to enhance the likelihood that their programs would be highly rated in Colorado Shines. Fifty-nine percent of the PDIS respondents reported their primary reason for participating in PDIS as “My program wanted to participate in Colorado Shines,” with the secondary reason as, “My program wanted to improve its Colorado Shines rating level.”vi While the Colorado Shines rating level remains a priority for administrators and users, communication about the system seems to have broadened and potential users report hearing from other sources that they should register in PDIS. Many respondents reported they heard about PDIS from their early childhood councils and that council members had encouraged them to set up their profiles. This is good evidence that the work of CDE to build council capacity and collaborate with the councils resulted in improved communication about PDIS to potential users. In addition, as documented by the 2016 Credential 2.0 survey, users were more likely to report hearing about the system from colleagues who had used it and recommended it to them. One of the key reasons users for continued usage is the ability to maintain all experience, professional development, training, formal education,

Figure 14: Connecting Learning Objectives To Activities Source: Training Design Course


demonstrated competency information and credential renewal requirements in one place. While they reported that setting up their profiles the first time took more time than expected, they believe the time commitment going forward will be significantly less. They also reported that they intend to continue using the system as a repository for the professional activities and a way to make it easier to apply for a professional credential.

Technology Platform This is the first time many PDIS users have had the opportunity to experience online learning, an online data collection system, and a method for assembling their credential information in an easily accessible location. According to a survey conducted by Child Trends, 64% of users reported that using the system was “somewhat easy” or “very easy,” and 40% reported experiencing technical challenges.vii The PDIS vendor implemented an overhaul of the user interface following receipt of the survey report. This overhaul resulted in a more intuitive interface that some users have reported as being easier to navigate. In spite of the interface overhaul, CDE’s 2016 user survey indicated that some users are still having difficulty in navigating the system. As one respondent stated, “The system is confusing to use, which deters me from continually updating my information. I’ve only taken one training and I thought it was difficult to access. I felt like I had to click too many links to access the information. When I needed to finish the training, I had to contact PDIS to figure out where I needed to go to complete.”viii Users also reported having issues with uploading files into the system and with linking to QRIS. By far the most frequently cited suggestion for improving the user experience was to improve the quality, quantity, and accessibility of training on how to use the system. Reports of learning through trial and error or personal practice were typical and some users reported making frequent calls to the help desk for advice as well as seeking help from colleagues who had experienced more success in using the system to meet their needs. The system has been enhanced since these results were shared. Such enhancements include the creation of multiple support documents and training videos to assist users. Since the Child Trends and Credential 2.0 surveys were completed, CDE has worked closely with the PDIS technology vendor to improve the user interface. They have also added help desk staff members who have successfully eliminated the backlog of applications for credential updates. These changes have resulted in less lag time in getting feedback regarding credential submissions and more positive feedback from users regarding ease of use.

System Outputs PDIS has a straightforward method for capturing information with a small set of automatically generated reports and the opportunity to request customized reports created using filters and file downloads in a variety of formats. Automatically generated reports are easy to access, but depending on the amount and complexity of information requested, the system takes a great deal of time to prepare the files for download, sometimes requiring several hours to process a request. Advice from the vendor regarding this issue was to request reports just before leaving work for the day and plan to have them available at the beginning of the next business day. Waiting so long for reports serves as a serious barrier to broad usage of the report feature in PDIS as few field-based staff members will have the time to request reports, wait several hours for the system to generate them, and then do the necessary analyses to arrive at conclusions regarding a program’s status. Additionally, this makes it impossible to gather information on a just-in-time basis to make timely decisions or respond to questions quickly.

The PDIS is hard to navigate! I try to spend time in it frequently so that I am familiar

with where to find things, but for new staff with no experience in it, it is confusing and

overwhelming. Typically, people are only accessing the PDIS to take specific modules

or to upload their recent trainings taken outside of the PDIS. Having to click in 4 different places, ending on a small “click here” feels counter intuitive. It should be

easier to get to.


The combined set of standard, non-customized, reports as a group contain all of the demographic and professional information submitted by individual users. They also have a unique identification field which makes it possible to merge the downloaded reports to conduct more sophisticated analyses to obtain answer to research questions or monitor progress in ways that are not used by other programs. The ability to conduct such analyses provides greater system flexibility for programs who have staff and other resources needed to do so.

Conclusions and Recommendations 1. Many comments from survey respondents related to the intersection of financing and accessibility: where

does the money come from to support educators living in those areas of the state where coursework and professional development are not available who want to continue their formal education and improve their skills? The inaccessibility of higher level training leads to questions of system credibility and seems to exacerbate feelings of isolation particularly on the part of educators in the most remote areas of the state.

Recommendation: The power of PDIS for providing professional development and training is in its infancy. As more offerings are made available, consideration should be paid to opportunities that will count toward formal degrees as well as those using research based strategies such as job embedded work, intensive training, and multi-day trainings. In addition, CDE should reach out to institutions of higher education to identify and develop easily accessible opportunities for obtaining college level coursework.

2. One potential source of bias in the process of obtaining user feedback is the use of technology as the prime

survey method. Online survey programs are more efficient than face-to-face or paper and pencil methodologies, but they limit feedback to those users who have access to and are comfortable using a variety of software for such purposes. Some early childhood programs have limited access to computers for teacher usage and many rural areas do not have reliable Internet connectivity. This makes it difficult to obtain feedback from a representative sample of users.

Recommendation: As resources permit, solicit user feedback through low-tech as well as high-tech methods to help avoid bias against people who do not have access to or aren’t proficient users of technology and compare responses of the two groups. Should the differences between them be small or not statistically significant, such a practice may be discontinued. However, if there are major differences between the two groups, continued data gathering using low tech methodologies would be called for.

3. PDIS users have consistently reported having difficulty learning how to use the system and understanding the steps involved in uploading documents and linking to QRIS. This serves as a barrier to broad usage of the system and could lead some users to curtail their use.

Recommendation: Ensure that all training sessions conducted across the state include a segment on PDIS, its importance, how to use it and how to link with QRIS.

4. PDIS is a very good data collection, training, and professional development tool that, by measures

established in the RTT-ELC application, has met its goals. As users have learned to use the system and become comfortable with uploading their credentials, they report having a greater understanding of its benefits and report that they continue to use it as a location for warehousing their professional records for the foreseeable future. The system is limited, however by its outputs. If the information gathered by the system is to achieve its highest and best use, the reporting process needs to be improved.

Recommendations:


1. Currently, programs have few options for reporting that compares their performance to that of others. Standardized reports need to be developed that will provide programs a baseline through which to compare their performance to: a. That of all programs in the state on key research based factors related to improving early learning. b. Programs comparable to theirs in terms of size, location (rural, urban, suburban), clientele and past

performance factors. c. Their own performance over time. This will make it easier for programs to identify growth trajectories

and pinpoint barriers to continued progress. 2. Increase the speed with which reports are generated. It is doubtful that users will spend the time

necessary to download large reports if the process continues to be so time consuming. 3. Make it easier to customize existing reports and develop new user-friendly reports that may be used for

decision making, planning, research, and monitoring progress. As a final note, readers should understand that the 2014 and 2016 survey information contained in this study is reported by system users. While self-reported data such as this provides an opportunity to gather rich and informative information to help understand issues at a deep level, it also is susceptible to variability in the ways in which similar respondents may answer the same question. Data extracted from the PDIS system are also included to provide an alternative look at these issues based on data that have been carefully reviewed by trained and objective CDE staff members. The combination of the two types/sources of data provided a well-rounded picture of the Credential 2.0 system, user perceptions of its strengths and weaknesses, and how it is currently being used.


Summary

Data collected by researchers and evaluators both internal and external to CDE indicate that the goals established for Colorado’s Early Childhood Race to The Top Grant have been met and in some cases exceeded. Colorado’s Competencies for Early Childhood Educators and Administrators were completed and approved for use across the state in 2013. Since their approval, they have become a part of the fabric of the state’s early childhood system. To name a few applications for the Competencies, they are:

• A part of the foundation for state licensing of early childhood educators. • Embedded in the Early Childhood Professional Credential 2.0. • Totally aligned with the Professional Development Information System to the extent that the self-

assessment for early educators is competency based and options for the individualized improvement plans generated by the system are designed to guide users to education and professional development opportunities that will help them move across the Competency continuum to higher levels of achievement.

• Required for use in trainings incorporated into the PDIS. Now nearly 5 years old, the competencies should be reviewed soon with dual lenses focused on maintaining currency of best practices identified through high-quality research studies and diversity issues identified as a need task force that developed Colorado’s Early Childhood Workforce: 2020 Plan. Needed changes identified by the review process will of necessity guide the changes in tools, materials and supports in place to embed the competencies in the state’s early childhood education system. The Early Childhood Professional Credential 2.0 system is in place an in use across the state. Currently, three of the four pathways (professional development, formal education and experience) are being used to determine credential levels. The demonstrated competencies pathway is being pilot tested to determine whether it will work as currently envisioned as a pathway. When the pilot test is completed, CDE will need to decide whether and how to incorporate the new pathway into the scoring system to make it consistent with the Credential 2.0 Scoring Framework. The Professional Development Information System is in place and linked to the Colorado Shines Quality Rating and Improvement System. Credential ratings awarded through PDIS are automatically transferred to QRIS so early childhood programs participating in the voluntary system can track the progress of their employees toward credentials used for program ratings. Users have indicated that the system is useful as a warehouse for their professional information such as experience, professional development, formal education, and credentials and certifications. Still in its infancy, the system is just beginning to have enough data to support research and evaluation efforts as well as program monitoring. The data download function is in need of an upgrade to make it more user friendly and therefore more likely to be used by individual program staff for determining program improvement and professional development needs. Another critical area needing additional immediate, intensive and ongoing attention is compensation. Education stakeholders generally agree that early childhood educator and administrator compensation is well below the level necessary to recruit and retain a high-quality workforce. Compensation levels have been linked to a number of problematic issues such as early childhood professionals who leave the field in order to earn a higher wage. Many early childhood educators have expressed that they cannot afford to participate in professional development activities or to return to higher education to obtain a degree even if they can do so free of charge because of the costs associated with their attendance: transportation to and from classes, care for their own children while they are in class, occasional meals out, class materials and lost opportunities to earn additional money through part-time jobs. This is perhaps Colorado’s biggest barrier to recruiting and retaining a high quality workforce.


Perhaps the greatest accomplishment for Colorado’s RTT-ELC team is the collection of professional networks that were brought to bear on these efforts. Early childhood stakeholders from across the state contributed their time, resources and, most importantly, their expertise to achieve the goals set forth in the 2010 Plan. They continue to collaborate with each other to ensure their work is expanded, enhanced and sustained into the future.


Glossary*

Accessibility -- An accessible professional development system offers affordable and convenient options for education, training and coaching, acknowledging multiple avenues towards teacher effectiveness including college courses, prior experience and alternative classroom- and field-based training. Accountability -- An accountable professional development system collects and evaluates data to demonstrate effectiveness, tracks child progress, plans improvements and assures quality through a continuous improvement process. Advisory Structure -- The coordination mechanism for an integrated early childhood professional development system, which should be freestanding and have some authority or direct link to authority in the state’s governance structure. State policy should require the creation of an advisory structure to examine needs and provide policy recommendations to the entity or combined entities funding the professional development system. Its composition should include representatives from the diverse settings, auspices and roles of the early childhood field and professional development system supports. Alignment -- An aligned professional development system provides a coherent, easily navigable and coordinated system for guiding early learning professionals’ college preparation, recruitment and ongoing professional development activities across all early learning settings. Articulation -- The transfer of professional development credentials, courses, credits, degrees and student performance based competencies from one program or institution to another, ideally without a loss of credits. States should require colleges and universities to make articulation agreements that assist early childhood professionals in moving seamlessly through and across undergraduate and graduate degree programs. Grants or specific directions for resource allocations should be attached to such policy requirements; colleges and universities will need fiscal support to change or augment long-standing, institutionalized processes. Career Pathways -- Routes of continuous progress for early childhood professionals, leading towards awareness and achievement of increased qualifications, professional possibilities and appropriate compensation. Policies should institutionalize pathways in all sectors and for all roles including both direct service (those individuals working with young children and their families) and non-direct service (those working on behalf of children and families in training, resource and other administrative roles). Collaboration -- A collaborative professional development system is developed by and built upon authentic partnerships among the Department of Education, higher education, Department of Human Services, Head Start, local early childhood councils, resource and referral, the Office of Professional Development and recipients of professional development services. Colorado’s Competencies for Early Childhood Educators and Administrators – Provide clear descriptions of what educators need to know and be able to do to provide quality care and education in a variety of settings. Data -- To gauge impacts and systems change, as well as to inform planning, evaluation, quality assurance and accountability. State policies should require the collection of specific data and mandate cross-sector data collection, sharing and alignment as well as non-duplication of efforts. Policies also should require comprehensive workforce studies conducted at regular intervals as well as ongoing collection and reporting of professional development utilization and improvement indicators. Additionally, policies should include specific requirements for disaggregating data by type of setting, demographics and primary financing source(s).


Early Childhood Educators – Those who work with children birth through third grade and are responsible and accountable for planning and implementing developmentally appropriate experiences that advance their cognitive, physical, and social-emotional development. Also included are those who provide support and/or leadership for those activities. Early Childhood Professional Credential – A voluntary, tiered system of credentials that recognizes increased knowledge, experience and expertise for those who care for and educate young children birth through third grade. The credential provides a common system for all Colorado early childhood professionals to document and quantify their professional growth and accomplishments. Individuals accumulate points along each of four pathways – Formal Education, Ongoing Professional Development, Experience, and Demonstrated Competencies. Early Childhood Professional Development Advisory – A working group under the Program Quality Alignment Subcommittee of the Early Childhood Leadership Commission charged with coordinating the implementation of the Early Childhood Workforce 2020 Plan and providing guidance around the Early Childhood Professional Development System. Early Childhood Workforce – Those who provide direct or indirect services to support young children birth through third grade and their families. This may include, but is not limited to, early childhood educators, home visitors, early intervention specialists, special education teachers, early childhood mental health providers, preschool teachers, kindergarten through third grade teachers, infant-toddler teachers, and family child care professionals, and family, friend, and neighbor caregivers. Financing/Well-Financed -- The funding that all professional development system need in order to operate. State policies should support the financing of integrated professional development systems in the specific areas of needs-based support for early childhood professionals to obtain education and ongoing development, support for programs/workplaces that facilitate professional development, explicit rewards and compensation parity for attainment of additional education and development as well as financing of the professional development system. A well-financed professional development system provides equitable opportunities for early learning professionals to achieve higher levels of competency through college education and ongoing professional development that is tied to increased compensation and benefits. Inclusiveness -- An inclusive professional development system promotes culturally-sensitive practices, engages diverse stakeholders, embraces individuals of all abilities and actively seeks to diversify the early learning profession. Innovation -- An innovative professional development system is flexible, incorporates relevant research and research-based practices and experiments with new approaches for preparing, credentialing and rewarding early learning professionals. Professional Standards -- The content of professional preparation and ongoing development. State policies should specify qualifications and ongoing development required for all early care and education professionals- from teacher assistants to trainers and higher education faculty, family child care providers, licensors, resource and referral staff as well as program, school, district and agency administrators. These specifications should address levels and content of education as well as ongoing development. *To ensure consistency across documents regarding CDE’s RTT-ELC activities, where possible, definitions were taken directly from documents previously approved by the ECPD Advisory and/or the Early Childhood Leadership Commission. These definitions were adhered to in the preparation of this report.


References

Clayton Early Learning (2017). Early Childhood Workforce Policy and Program Scan. Denver, CO: Author. Colorado Department of Education (2010). Colorado’s Early Learning Professional Development System Plan. Denver, CO: Author. Colorado Department of Education (2013). Colorado’s Competencies for Early Childhood Educators and Administrators. Denver, CO: Author. Colorado Department of Education (2015). Colorado State Model Educator Evaluation System: Practical Ideas for Evaluating Early Childhood Educators. Denver, CO: Author Colorado Department of Education (2017). Colorado’s Early Childhood Workforce: 2020 Plan. Denver, CO: Author. Retrieved from https://www.cde.state.co.us/early/copdplan on August 1, 2017. Colorado Department of Human Services – Office of Early Learning (2015). Race to the Top – Early Learning Challenge 2014 Annual Performance Report. Denver, CO: Author. Colorado Department of Human Services – Office of Early Learning (2016). Race to the Top – Early Learning Challenge 2015 Annual Performance Report. Denver, CO: Author. DeVries, K. (2017). ECE Alignment and Articulation Survey Results. Denver, CO: Colorado Department of Education. Joint Committee on the Standards for Educational and Psychological Testing of the American Educational Research Association, American Psychological Association and National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.

https://www.cde.state.co.us/early/copdplan%20on%20August%201


Endnotes i State of Colorado. (2011, October 19). Race to the Top – Early Learning Challenge Application for Initial Funding: CFDA Number 84.412. Submitted to United States Education Department, Washington, DC, p. 59. ii State of Colorado, (2011, October 19), pp. 180-181, 186-187. iii State of Colorado (2011, October 19), p. 70. iv Moodie, S., Daily, S., Bultinck, E., & Abrams, J. (2016). Pp. 3-4. v Moodie, S., Daily, S., Bultinck, E., & Abrams, J. (2016). Pp. 5. vi Moodie, S., Daily, S., Bultinck, E., & Abrams, J. (2016). Pp. 5. vii Moodie, S., Daily, S., Bultinck, E., & Abrams, J. (2016). Pp. 5. viii Williams, J. M. (2016). Credential 2.0 Survey Comments Organized by Guiding Principles and Other Topics. Denver, CO: Colorado Department of Education, p. 6.

Documents

Workforce Development Under Colorado’s Race to the Top Early … · 2018-01-22 · Contents Tables and Figures 4 Overview 6 Introduction 7 Foundations of Workforce Development Efforts