Comments on CA RTTT Application

8/9/2019 Comments on CA RTTT Application

1/37

Technical Review Page 1 of 10Race to the Top

Technical Review Form - Tier 1California App lication #1400CA -1

A. State Success Factors

(A)(1) Articulating State's education reform agenda and LEA's participation in itAvailableier 1655(i) Articulating comprehen sive, coherent reform agenda 5(ii) Securing LEA commitment 45 j2

(iii) Translating LEA participation into statewide imp act 15 8(A)(1) Reviewer Comments:The State presented a comprehensive reform agenda aimed at increasing achievement throughimproved instructional models designed to ad dress the diverse learning needs of its students. Toaccomp lish its goals, the State is calling for a new local-state relationship focusing on achieving m utualgoals. The State has made critical linkages between teacher quality and student achievement andrecognized the need to ma ke major investments in data systems to measure teacher effectiveness andstudent growth. The plan supports both the educ ator workforce and the children and yo uth enrolled inthe State's elementary and second ary schools. California's plan strengthens academ ic standards,provides professional development in crafting curriculum framew orks and helping teachers teach to thenew standards, and aligns assessm ents with the new con tent standards. California has securedparticipation from 804 districts representing almost one-half of all districts and 56 percent of scho ols.While this does not represent universal participation, California's size wa s takeninto account in

awarding points. Implemen ting a reform effort of this magnitude w ould be challenging for any S tate. Asthe application points out, California has districts with larger enrollments than some states. By focusingon about one-half of its districts, the State has a reasonable chan ce to impleme nt reforms more dee plyand authentically in these selected districts rather than spread resources too thinly across the State Ifall of the participating districts honor their comm itment, the Race to the To p reforms could reach 58percent of all students and 61 percent of all students living in poverty which could ma ke a significantimpact on raising achievemen t for the State's students most in need. Approximately 54 percen t ofstudents in the participating districts are low incom e, compa red to 52 p ercent of districts statewide.California's application is further strengthened by not allowing districts to opt out of any of the Race tothe Top provisions outlined in the required Memo s of Understanding (MOU). The S tate has alsoencouraged interdistrict partnerships in implem enting key elements of Race to the Top w hich werearticulated in the MOUs. Co llaboration among districts will help leverage funds and resou rces to buildcapacity to implement the Race to the Top reforms. How ever, within the 56 percent of LEAs that haveagreed to participate in Race to the Top, only 2 6 percent of district union leaders signed the MOUssupporting the initiative. Of the ten largest districts, six did not provide signatures of union leade rsincluding Los An geles Unified, the nation's second largest school district. Most districts in Californiarequire some form of collective bargaining on issues related to teacher hiring, compensation, and/orassignment. Furthermore, C alifornia sets the tone early on in its application that it is the districts thatwill be the driving force in implementing the Rac e to the Top reforms w ith the State settingaccountability standards and supporting LEA efforts. Unions are critical stakeholders in many ofCalifornia's school districts. The application did not discuss the reasons for the low level of unionengagem ent. It is not clear if unions are opposed to the R ace to the Top initiative or if the State did notappropriately engage u nions in the discussions surrounding district participation. The lack of union buy

2/18/2010


2/37

Page 2 of 10Technical Review

-in at this stage raises serious concerns about the ability of the State to impleme nt the Race to the To p Ireforms, especially the critical hum an cap ital provisions. Significant points for sections (A)(1)(ii) and (iii)were deducted for the S tate's lack of union buy-in.(A)(2) Building strong statewide capacity to implement, scale up, and sustainproposed plans

(i) Ensuring the capacity to implement(ii) Using broad stakeholder support

(A)(2) Reviewer Comments:California has made a comm itment to using this infusion of Federal funds to m ake short-terminvestment to build its educational infrastructure to support long-term reforms. As an indicator of this .comm itment, the State is proposing to retain only 2% of its requested $500 m illion for the state portionfor salaries and fringe benefits at the SEA . In ensuring capacity to implemen t its proposed plan,alifornia identifies three current entities that will support reform: the C DE, 11 regional consortia ofregional offices, and the California Comprehensive C enter. The application suggests that these threeentities will be responsible for supporting districts in implementing the R ace to the Top R eforms. TheState plans to mak e significant financial investments about 1/3 of the state portion-- in the 11 regionaloffices to support LE A efforts. It is not clear ho w this structure is significantly different from the currentstructure or how this expan ded layer of bu reaucracy will mo re effectively or efficiently deliver servicesto LEAs and the classroom. The State indicates that it will shift greater funds to the regional offices inreturn for greater accountability. Howeve r, the application does not go in to enough detail to evaluatethe degree of accoun tability or oversight the SEA will have over the regional centers. Further, it isunclear how the S tate intends to sustain the level of funding to the regional centers after the Race tothe Top grant ends. The State will add a fourth partnerthe Regional Charter Innovation Center tosupport the developme nt of high-quality charter schoollorograms. The a pplication does not describehow this organization will work in concert with the other en tities to ensure high standards of programquality and accoun tability cut across all instructional providers. A ma jor weakness of the application isthat the State did not provide a clear ma nagem ent plan or organizational structure to oversee theactivities and expen ditures of funds: The applicant did not describe its managem ent team or jobdescriptions and/or roles and responsibilities of key staff. It was not evident how the grant would be

fiscally man aged or audited. A m anagem ent plan with clear lines of authority is critical given thepotential magnitude of this award and the large am ount of funds the State is proposing to besuballocated. The application described several innovations to support teaching and learning thatmake it poised to provide significant support to statewide and reg ional professional learningcomm unities. Brokers of Expertise holds promise to deliver research-based strategies developed byteachers for teachers. The State appears to be c omm itted to evaluating its Race to the Top effortsthrough a legislatively-mandated evaluation. The application describes increased attention to datadriven decision making; however, it is not clear how the performance m easures developed by LE As,regional offices, and the CDE w ill be coordinated or linked. The application clearly demonstrates thatthe State has the support of the higher educa tion comm unity, parents, foundations, businessorganizations, and researchers. However, it again is disconcerting that teacher organizations are notcommitted stakeholders for the Race to the Top effort.(A)(3) Demonstrating significant progress in raising achievement and closing0gaps(i) Making progress in each reform area(ii) Improving student outcomes(A)(3) Reviewer Comm ents:California's application is somewha t uneven in addressing the State's past achievements in the fourRace to the To p reform areas. The S tate demonstrated that it has a long history in setting academ icstandards and aligning assessments to those standards. It also was an early ado pter of professional

2/18/2010


3/37

Total


standards for teachers and leaders and has been a leader in providing support for new teachers.California appears to be lagging in developing the necessary IT infrastructure to support the Race tothe Top reforms. The State does not have a system in place with the required teacher/students links;the governor signed legislation to remove these barriers only in January 2010. The State indicated thatits support to struggling schools has been met with mixed success. The application did not addresshow ARRA funds have supported the Race to the Top reforms or how other Federal and State dollars Ihave been leveraged in the four key areas. California's success in improving student outcomes hasbeen mixed. The State's application clearly shows that California's students as a whole are makinggains across subject and grades on the state assessments. However, gains in NAEP reading andmathematics have stalled and there continue to be significant achievement gaps between racialgroups across assessments. There is a disconnect between the outcomes reported on the state andNAEP assessments, although the application states the NOES found the state's proficiency standardsto be rigorous compared to NAEP. The State is facing continuing declines in high school graduationrates. While there is a generic statement linking systemic reforms to improved student achievement,the application does not clearly link specific reforms or instructional strategies to student outcomes.

125-1 62

B. Standards and AssessmentsAvailableier 1(B)(1)Developing and adopting common standards 4040(i) Participating in consortium developing high-quality standards 200(k) Adopting standards 200(B)(1) Reviewer Comments:California provided evidence that it i6 participating in CCSSO-NGA's Common Core StandardsConsortium to develop common standards in mathematics and English-language arts that areinternationally benchmarked and build toward college and career readiness. Fifty-one states and

territoriesa clear majority of statesare participating in the initiative. Evidence was submitted thatthe State is actively moving forward in adopting these standards. California passed legislation Iindicating its intent to adopt the common core standards by August 2, 2010. The State has establishedan Academic Content Standards Committee to oversee this process. The legislation further specifiesthat at least 85 percent of the State's standards be composed of the common core standards; theState was unable to commit to this requirement in the original MOU.[R2) :(B)(2) Developing and implementing comm on, high-quality assessments 0 j, 10(B)(2) Reviewer Comments:California appears to be committed to moving forward in developing high-quality assessments. TheState recently passed legislation to allow assessments that measure student growth. The State hassigned non-binding MOUs with three multi-state assessment consortia. Two of the three consortia

include over one-half of the states. All of the consortia are focused on assessments designed tomeasure the Common Core Standards.(B)(3) Supporting the transition to enhanced standa rds and high-qua lity 04assessments(B)(3) Reviewer Comments:California provided an ambitious schedule for transitioning to the new standards and appears to be ontrack in meeting these milestones. California has a process in place to review curriculum frameworksand adopt instructional materials in grades K-8; the State will use these processes to integrate the newstandards. The State is in the process of soliciting names for members of the committee to examinethe mathematics curriculum framework. The State is lagging behind in developing processes for

2/18/2010


4/37

I Available24


adopting instructional materials for grades 9-12. The State plans to use the planned professionallearning communities and Brokers of Expertise portal as avenues for providing professionaldevelopment to educators around the new standards. Having proven processes in place to reviewcurriculum standards and adopt instructional materials coupled with early plans for professionaldevelopment suggests that California will be able to meet its targets for transitioning to the newCommon Core Standards. While the State is on track on transitioning to new standards, it may have agreater challenge in adopting new assessments. The plans and timeline for transitioning to the newassessments are not as detailed as those for implementing new standards. It is not clear how the Stateintends to support districts in developing local formative assessments. The CDE is under a legislativemandate to explore ways to focus on science and mathematics and measure postsecondary andcareer readiness in the calculations of the State's Academic Performance Index, but it is not clearwhere the State is in the process.Tota l

C. Data Systems to S upport Instruction

(C)(1) Fully implementing a statewide longitudinal data system(C)(1) Reviewer Comments:California reported that it currently has 3 of the 12 of the America COMPETES Act elements in place inits longitudinal data system.(C)(2) Accessing and using State data(C)(2) Reviewer Comments:The State only recently passed a law to coordinate the use of student data for research purposes. The .State does not discuss the process or timeline for accomplishing this task and it is unknown at thistime what authority the State ultimately will have in releasing student data or which organization willserve as an IRB. Continuing the development of the longitudinal data system appears to be dependent 1

on the State securing an IES grant. The application states that if the State does not receive this grant, IRace to the Top funds will be used to build the system; the application does not indicate how fundswould be shifted from other Race to the Top areas to support the development of the longitudinal datasystem. Given that California is in the beginning stages of developing its longitudinal data system,there is major concern about the State's ability to have the system in place to meet its reform goalswithin the grant period. The plan for a "one-stop shop" for accessing the State's wealth of educationdata will provide accessible information to a broad-range of stakeholders. However, it is not clear howthe State will ensure full reporting by LEAs. The application indicates that LEAs will want to submitdata once they see others submitting. This might not be the best approach or incentive to ensure thataccurate, complete, and timely data are submitted through the portal.(C)(3) Using data to improve instruction(C)(3) Reviewer Comments:Given that formative assessments are conducted at the LEA level and school districts across the Stateselect their own assessments, the State is requiring (through the MOU) LEAs to develop formative 2assessments aligned with state standards and to develop local instructional improvement systems:The application does not detail how many districts currently are lacking these systems or the burdenon LEAs or the level of technical assistance required to put these systems in place by the 2011-12school year, so it is not clear whether this is a reasonable target. Furthermore, the State is targetingonly 33 percent of teachers to use data in their practice to change instruction by school year 2011-12and 66 percent by the end of school year 2013-14. These targets seem low given the importance ofdata in shaping teachers' practice and improving student learning at the classroom level. Again, there Iis concern about the State's ability to have its data systems in place to meet its reform goals within the

2/18/2010


5/37

Availableier 1Page 5 of 10Technical Review

grant period. It is critical that the State have the means to calculate a growth measure to provide therobust data teachers need to monitor student progress and adapt professional practice.Total i7

D. Great Teachers and Leaders

(D)(1) Providing high-quality pathways for aspiring teachers and principals(D)(1) Reviewer Comments:California provided sufficient documentation that it has created alternative pathways for aspiringteachers and principals that meet all five elements listed in the Race to the Top definition of analternative route. The application further documented that the alternative routes for teachers areactively in use and have grown over 400 percent since the 1995-96 school year. The routes forprincipals, however, appear to be less well-established. While the State has alternative routes forprincipals in statute, the majority of principals (75 percent of Level I admistrative certificates and all ofLevel II) complete their preparation through traditonal routes. California enjoys the support of theCenter for the Future of Teaching and Learning which conducts an independent review of the teachingworking force; the Center publishes annual statistics on teacher shortages in the State's hardest tostaff schools. The State provided additional evidence that the CDE and CTC are monitoring shortageareas on an annual basis. The results are impressive; the State has reduced the number ofunderprepared teachers from 20 percent of the workforce to 3 percent over the last decade. The dataon principals, however, is less compelling. The application notes that the State has not systematicallymonitored, evaluated, or identified shortage areas for principals. The application indicated that thereare several new programs offered by the UC system to prepare principals in high-need schools. It isunclear what data were available to support the creation of these programs to ensure they wereserving the areas with the greatest needs.(D)(2) Improving teacher and principal effectiveness based on performan ce582

(i) Measuring student growth(ii) Developing evaluation systems(iii) Conducting annual evaluations(iv) Using evaluations to inform key decisions

(D)(2) Reviewer Comments:California is in the early stages of developing an approach to measure student growth and measure itfor each student. The application provided a timeline for developing the model. It is not clear whichentity (the CDE, the state assessment contractor, or an independent contractor) was going to beresponsible for developing the model. The schedule is fairly ambitious given that the State legislatureonly recently authorized the use of a student growth model. In terms of designing and implementingrigorous, transparent, and fair evaluation systems for teachers and . principals, the State is proposing todevelop voluntary state models that LEAs could choose to adopt. LEAs could also elect to developtheir own system. It is not clear how strong the State role will be in determining the rigor of the LEAevaluation systems or what oversight it will have in ensuring the evaluations are conducted asplanned. Districts may require these systems to be included.in collective bargaining agreements; thefact that both the State and LEA systems require educator input may help achieve buy-in. The Stateintends to secure the services of a contractor to ensure that the evaluation instruments developed bythe State are valid and reliable; it is not clear how validity and reliability will be assured for the LEA-developed instruments. The State is committed to providing training to LEAs on how to conductclassroom observations which is critical to ensuring they are conducted in an objective and fairmanner. Student growth must factor into educator evaluation but the plan does not address how

4

2/18/2010


6/37


teachers in non-tested grades or subjects will be evaluated The draft guidelines issued by the Stateencom pass those elements contained in the Rac e to the Top guidelines including annual evaluation,multiple rating categories including student growth, and mech anisms to provide constructive feedback.The State provided targets on the uses the evaluations to inform the key Race to the T op activities.The S tate does not currently collect data on mo st of these activities including developing,compe nsating, promoting, granting tenure to, and removing e ducators. Exhibit 15 provides a footnotethat indicates that all of these activities are subject to collective bargaining agreem ents. Given that theapplication did not have the supp ort of the majority of union leaders (See S ection A) and that the S tateis in the beginning stages of developing its growth model, the targets may be unreaso nable to achieveeven by schoo l year 2014-15. The application does not provide strategies on how the State mightsecure educator buy -in to accomplish these activities, or if LEAs have had success in the past innegotiating these activities in c ollective bargaining agreements........ . ,(D)(3) Ensuring equitable distribution of effective teachers and principals(i) Ensuring equitable distribution in high-poverty or high-m inority schools(ii) Ensuring equ itable distribution in hard-to-staff subjects and specialty areas(D)(3) Reviewer Comments:A m ajor concern with the S tate's plan for equitable distribution of teachers and principals is the lack ofdata to determine w here the State is in the process of ensuring the e quitable distribution of effective

educators. The State does not have a definition--or the data systems--in place to measure educatoreffectiveness. Furthermore, the majority of LEAs w ill not have their evaluation systems in place until2012-13. The ap plicant did not provide baseline data or set targets in meeting the eq uity provisions ofRace to the T op. On the p ositive side, the State has an infrastructure in place to provide technicalassistance and mon itoring to ensure the equitable distribution of educators through the QE IA andCMIS (recently renamed D ELTA) systems. The State has already required that over one-half Of LEAshave plans in place to address the equitable distribution of teachers and principals. The State iscurrently working on the data system for LEAs to report out-of-field assignments by school, subject,and specialty area to the public. Once the indicators have been developed to m easure the distributionof effective educators, the State should be able to provide su pport to and hold LE As accou ntability forensuring the equitable distribution of educators in high po verty/high minority schools in a timelymanne r. LEAs have several tools at their disposal to address equity. Principals in low-performingschools for example, are no longer required to accept teachers based solely on seniority. The Statealso has a loan forgiveness program, incentives for NBPTS c ertified teachers, and mentoringprograms designe d specifically to recruit and retain teachers to teach in high-needs schools. For themost part, these programs appe ar to be targeted to teachers, not principals.

(D)(4) Improving the effectiveness of teacher and principal preparation programs I4(D)(4) Reviewer Comments:The higher education community appears to be committed to implementing the Race to the Topprovisions related to improving the effectiveness of educator preparation program s. The State is in theprocess of linking the growth in student achievem ent back to the'program w here he or she wasprepared and recom mend ed for a credential. California is ahead of the curve in measuring theeffectiveness of teacher preparation programs linked to student growth in large part because o f agovernm ent-funded pilot currently under way at C alifornia State University (CSU ). The State has buy-in from the higher education comm unity to expand the pilot to all teacher preparation programs. CS Uhas already collected preliminary data and the State has plans to scale-up and to implem ent theaccountability system by School Year 2012. Furthermore IHEs have signed MOUs agreeing to"expand, modify, or close programs based on student achievement and student growth data"suggesting there is authentic buy-in from the higher educa tion commu nity not only on reportingmeasures of qu ality, but in taking action to improve or close failing programs. The S tate appears to beon track with this ambitious plan. The State also has plans to measure the effectiveness of programsthat prepare adm inistrators, but these will lag behind efforts for teachers.2/18/2010


7/37


(D)(5) Providing effective support to teachers and principals 203(D)(5) Reviewer Comments:California documented its long-term commitment to providing quality professional development tonovice teachers. The BTSA program, in particular, has provided data driven induction support to newteachers across the State. Less clear is California's commitment to providing on-going, job embeddedprofessional development throughout a teacher's career. The application cites a report calling onreforms in professional development that appear not to have been implemented. The applicationfurther states that it will use the Race to the Top initiative to target existing federal and state funds toimplement job-embedded professional development tied to performance evaluations. It is not clearwhat role the professional learning communities will play in retooling the State's professionaldevelopment system. The application states that California has not invested the same level of supportfor new principals as it has for new teachers. The State is committing over $20 million in Race to theTop funds for LEAs to establish principal induction programs. It is also dedicating a smaller portion of lfunds to support a cadre of turnaround leaders to support the activities described in Section E. TheState is estimating that it will train 90 principals through this route during the grant period. However,this target seems low given that the State estimates that over 780 schools will be identified forturnaround intervention over the next four years. As with other sections of the application, Californiamay encounter challenges developing a data-driven professional development system because itcurrently lacks the IT infrastructure to develop measures of effectiveness. Given the lack of baselinedata, for these performance measures, it is not possible to get a sense of the challenge the State facesin meeting the (D)(5) requirements.

E. Turning Around the Lowest-Achieving SchoolsAvailable Tier 1

(E)(1) Intervening in the lowest-achieving schools and LEA s 100(E)(1) Reviewer Comments:The application states that the State has the authority to intervene in struggling schools and LEAs.Copies of the authorizing legislation were provided in the appendix.(E)(2) Turning around the lowest-achieving schools

(i) Identifying the persistently lowest-achieving schools405

1 325

27ii) Turning around the persistently lowest-achieving schools 35(E)(2) Reviewer Comments:The application detailed the process for identifying the State's persistently lowest-achieving schoolsrequiring intervention. The process identified 187 schools that must implement one of the fourintervention models in the first year of the Race to the Top grant. It is not clear if the State can levy

appropriate sanctions against LEAs for schools that continue to struggle. The application states that apersistently low-achieving school that does not demonstrate progress within 3 years of implementingthe intervention model (after a planning year) must implement one of the remaining models. The Statedoes not indicate if the school has another 3 years to demonstrate progress or if the school must takemore rapid action to improve student achievement. Points were deducted for this lack of clarity. TheState articulated a plan to offer support to LEAs in assisting turnaround schools that includeseducating LEAs about the components of the models and establishing guidelines to assist LEAs inselecting turn around partners. As mentioned in Section D, the State will support the TurnaroundPrincipal Institute to prepare a cadre of leaders to effectively turn around low-performing schools.Total 502

1-Total

2/18/2010


8/37

(F)(1) Making education funding a priorityAvailable

6

-Page 8 of 10Technical Review

F. General

(F)(1) Reviewer Comm ents:California increased the percentage of revenues a llocated to public education from 46 percen t in FY08to 47.4 percent in FY09. W hile this is not a large increase in the percentage of reven ues devoted topublic education (and total expenditures have decreased) the State's comm itment to education iscomm endable considering the severe econom ic conditions the State is facing. California facessignificant challenges in funding public education given that Proposition 13 significantly erodedproperty tax revenues that are available to fund public education. The S tate has attempted to equalizeeducation funding through the General Fund, although the formula seem s to favor funding adequacyrather than equity. The current formula is heavily weighted on district growth, type and size and notpoverty. Points were deducted because of the formula's focus on adequa te funding. However,Economic Im pact Aid and other categorical programs . provide significant funding to attempt to equalizefunding between high and low pove rty LEAs. In light of the current economic conditions, LEAs havebeen granted greater flexibility in how these funds are spent which could pu t equity funding in furtherjeopardy. The applicant did not describe how LEAs ensure equ ity between high and low povertyschools.

(F)(2) Ensuring successful conditions for high-performing charter schools and 101other innovative schools(F)(2) Reviewer Com ments:Since 1998 C alifornia has capped its numbe r of charter schools not to exceed 100 new ly authorizedcharters per year. Unused charters may be rolled over into the next year. W hile there is a cap, it is a"high cap". In Section A, the S tate reported that there were 10,225 schools operating statewide. Inschool year 2009-10 the S tate authorized 1,350 charter schools, which is 13 percent of all schools.The State provided c lear evidence that it has processes in place for LEAs to m onitor the effectivenessof charter schools and close them if necessary. O nce approved, cha rters do not operate indefinitely;they must be renew ed after their first 5 years of operation. The applicant provided data showing thatLEAs hav e revoked 139 charters since the 2005-06 school year. The State did not thoroughly addresshow charters are encouraged to serve high-need students. Howev er, the State has plans to have itsnew partnerthe Regional Charter School Innovation Centerwork with LEAs to increase support forthe creation of charters to serve high-need populations. State laws require funding mec hanisms thatensure charters receive equitable funding com pared with traditional schools and the State has ma designificant investments in securing and im proving facilities for charter school ope rations. As evidence,the applicant noted that recent bond measures have authorized $850 m illion for charter schoolconstruction projects. It is clear from the legislation provided in the appendix and the discussion on thebonds approve d by Ca lifornia voters that there is considerable public support for charter schools in theState. The applicant did not provide evidence that it enables LEAs to operate innovative, autonom ouspublic schools (such as open e nrollment programs) other than charter schools.(F)(3) Demonstrating other significant reform conditions(F)(3) Reviewer Comm ents:The application provided com pelling evidence that it is committed to additional systemic reformsrelated to empo wering parents, creating a culture of innov ation and flexibility to support reforms at thedistrict level, and improving and e xpanding early childhood ed ucation. The State did not address ho wthese reforms will improve student outcomes.

2/18/2010


9/37

Competitive Preference Priority 2: Emphasis on STEMCompetitive Reviewer Comments:Throughout the application the State demonstrated its need to educate more students in STEM areas,

strengthen STEM standards and curriulum, and increase the number of effective teachers in theseareas. The State provided evidence of a strong PK-20 partnership to improve STE M instruction. TheState also has docum ented partnerships with the technology industry designed to support the State'slong-term reform efforts in STEM education, as well as the initiatives detailed in its Race to the Topapplication. However, the State's approach to STEM does not meet the standard of a high-quality planThere are no detailed action plans or timeframe s to meet the STEM goals and the STEM activities donot appear to be coordinated across the agency. Also lacking is a cohesive discussion on how theState plans to offer a rigorous course of study across the STE M-related subject areas.

Available3 15

Total

Absolute Priority - Comprehensive Approach to Education ReformAvailable

Technical Review Page 9 of 10Com petitive Preference Priority 2: Emphasis on STEMAbsolute Priority - C omprehensive Approach to Education Reform

Absolute R eviewer Comm ents:California has done a comm endable job articulating its commitmen t to undertaking the reforms in thefour Race to the Top areas. The State has mad e critical linkages between teacher quality and studentachievement and recognized the need to make major investments in data systems to measure teachereffectiveness and student growth. The plan supports both the edu cator workforce and the children andyouth enrolled in the State's eleme ntary and secondary sc hools. California's plan strengthensacadem ic standards, provides professional development in crafting curriculum frameworks and helpingteachers teach to the new standards, and aligns assessments with the new content standards. TheState intends to coordinate Race to the Top funds with existing state and local dollars to expandsystems of supports to LEA s through the 11 regional offices to fully implement the Race to the TopReforms. The State also is leveraging state and federal funds to refocus an array of professionaldevelopment efforts to support this initiative.

Total 0Grand Total2/18/2010


10/37

Technical Review Page 1 of 7

Race to the TopTechnical Review Form - Tier 1

California Application #1400CA-2A. State Success Factors

r Available--(A)(1)Articulating State's education reform agenda and LEA's participation in it -55(i) Articulating comprehen sive, coherent reform agenda(10 Securing LEA commitment(Hi) Translating LE A participation into statewide impact

(A)(1) Reviewer Comments:California's application clearly sets forth a comprehensive and coherent reform agenda w hichaddresses all four of the key reform areas. Wh ile explicit goals for each of the key reform areas are notprovided, a wide range o f strategies and goals are articulated which do address all four areas.California created an MOU w hich closely follows the exam ple provided in the application but whichincludes the addition of an opt-out provision for districts and schools when M OU elem ents would be inconflict with local bargaining agreements. The opt-out provision and its potential impact on meaningfulschool district participation is not discussed in the application narrative. California required all districtsto agree to implement all MOU e lements in order to be a RTTT participating district and committed toproviding funds to all districts in the state. Approximately 56% of districts in the state agreed to beparticipating districts. This is a substantial level of participation but is unde rcut by the M OU opt-outprovision which is likely to limit the engagem ent of districts on some of the sensitive RTTT eleme ntssuch as the teacher evaluation and comp ensation provisions. Points were withheld in A(1)0i0becauseit would have been possible to have an even higher percentage of LE As participating.

(i) Ensuring the capacity to implemen tL___(ii) Using broad stakeholder support

(A)(2) Reviewer Comments:California has described a comprehensive and coh erent approach for supporting its RTTTimplemen tation using a network which includes the California Department of Education, the CaliforniaCom prehensive Center, the Regional Ch arter Innovation Center, 11 regional offices, and all countyoffices of education. RTTT funds will be blended with Title I, Title II, and state funds to suppo rt theplan. Approximately 96% of the grant funds will ultimately go to LE As where the b ulk of the reformwork will be do ne. The pass-through of funds to the district level is so aggressive that there is somequestion about wh ether sufficient funds will be retained to provide strategic state-level support for theinitiatives. According to the budget information in the appe ndix, California plans to provide $16,100,000annually to LEAs to provide "High Quality Alternative Pathways to Teaching in Shortage A reas." LEAswould be required to provide m atching funds. Given the California financial crisis, it is doubtful theyhave the capa city to do so. California described extensive outreach efforts aimed at getting input onthe development of their plan and holding research and practice forums examining key aspe cts of theRUT program. They also provided MOUs for state higher education institutions and letters of support

t(A)(2) Building strong statewide capacity to implement, scale up, and sustain0-proposed plans

2/18/2010


11/37


from a broad rang e of stakeholders though no letter of support or MO U was provided for the stateteachers or administrators association, a conspicuous omission.(A)(3) Demonstrating significant progress in raising achievement and closing0gaps

(i) Making progress in each reform area(ii) Improving student outcomes

18

10

(A)(3) Reviewer Comments:California did provide substantial detail on its efforts in the four key reform areas, som e of which go a sfar as 10 years back. C alifornia provided evidence of making m odest progress for all subgroups on thereading and mathem atics assessments for the California Standards Tests and the NAE Passessments. For 2009, just 50% of California tested students showed proficiency on the state EnglishLanguage A rts assessments while only 46% showed proficiency in mathem atics. California providedevidence that it has made very slight progress in closing subgroup achieveme nt gaps. The applicationincluded very little information on the actions that ma y have con tributed to these outcomes.1 Total 1258B. Standards and Assessments

r(B)(1) Developing and adopting common standards(i) Participating in consortium developing h igh-quality standards(ii) Adopting standards

; Availableier 14020(B)(1) Reviewer Comments:California provided docum entation that it is part of the National Governors Association Co nsortium forthe Developm ent of Comm on Core S tandards. The consortium includes 48 states, the District ofColum bia, and two territories. The consortium will produce a set of curriculum standards for Englishlanguage arts and mathem atics for grades K-12 as well as college and career readiness standards.California provided a detailed plan to consider adoption of the Comm on Core S tandards by August 2,2010.(B)(2) Developing and implementing common, high-quality assessments(B)(2) Reviewer Comments:California submitted MO Us regarding non-binding participation in three consortia ranging in size from23 to 36 states. The application also included a statement doc umenting the intent to apply separatelyfor Race to the Top assessment program fund s.(B)(3) Supporting the transition to enhanced standards and high-qualityassessments(B)(3) Reviewer Comments:California provided a high-quality plan for supp orting statewide transition to the im plementation of thenew standards and assessments and included key activities, a timeline, and parties responsible. Therewas a m ore coherent plan for the K-8 range than for high schools.Total

2/18/2010


12/37


C. Data Systems to Support Instruction

(C)(1) Fully implementing a statewide longitudinal data systemAvailableier 1

24(C)(1) Reviewer Comments:California currently has just three of the twelve America Competes Act elements for state data systems

in place, including the yearly test records of individual students, information on students not tested bygrade and subject, and a teacher identifier system that can match teachers to students.14,.4, , , ;(C)(2) Accessing and using State data

TOT ........5j(C)(2) Reviewer Comments:California, in the appendices, provided a high-quality plan for accessing and using state data whichincluded activities, a responsible party,timeline elements, and performance measures. The applicationdescribed a utilization focused approach to planning which will promote buy-in.(C)(3) Using data to improve instruction 18 I2

(C)(3) Reviewer Comments:California, in the appendices, provided a high-quality plan for using state data to improve instructionwhich included activities, a responsible party, timeline elements, and performance measures.California adequately addressed how it will support districts in selecting instructional improvementsystems. Six points total were withheld because California's decentralized approach to selection ofinstructional improvement systems is likely to result in compatibility and variablity issues that createserious problems for the provision of support in using the systems and in making data available toresearchers.

Total 473D. Great Teachers and Leaders

, Available (D)(1) Providing high-quality pathways for a spiring teachers and principals219(D)(1) Reviewer Comments:California submitted evidence that it provides multiple pathways for teacher and principal credentialing.Most of these pathways include all required elements but a few do not. California has one of the mostextensive systems in the nation for providing alternate pathways to credentialing and does provide anption independent of higher education. In addition to higher education, certification processes canalso be provided by community-based organizations and nongovernmental organizations. Californiasubmitted evidence that it has a track record of monitoring and addressing areas of teacher shortageand described a partnership with the Center for the Future of Teaching and Learning to continue thisactivity and extend it to include principals. Two points were withheld because of limited attention to thealternate pathways for principals.jD)(2) Improving teacher and principal effectiveness based on performance 1 58 56

(i) Measuring student growth 5 5(ii) Developing evaluation systems 1 15 i 15(iii) Conducting annual evaluations 1, ,

,10 i0

1"---------"(iv) Using evaluations to inform key decisions 1 28 i, 26

2/18/2010


13/37

Technical Review Page 4 of 7(D)(2) Reviewer Comments:California submitted a high quality plan including activities, responsible parties, and timeline forachieving the required grant elemen ts for teacher and principal evaluation. California described plansby which it will transition from the current student achievem ent status model to a growth m odel.California law provides for use of student achievemen t data in teacher and principal evaluations.California's teacher evaluation system is much more fully developed and doc umented then it's principalevaluation system but Ca lifornia described plans for assuring that both evaluation types meet R TTTrequirements. The related performance targets are ambitious but achievable.

1(D)(3) Ensuring equitable distribution of effective teachers and principals 255(i) Ensuring equitable distribution in high-poverty or high-minority schools 15(ii) Ensuring equ itable distribution in hard-to-staff subjects and spe cialty areas 10L(D)(3) Reviewer Comments:California provided evidence that it has been w orking to improve equitable distribution of effectiveteachers and principals for several years and has several program s and processes to suppo rt thatgoal. California provided a limited plan which has the m inimal characteristics of a high-quality plan inthat it addresses activities, responsible parties, and provides a timeline. California reported that it

currently does not have baseline data for the performance m easures and declined to establish annualtargets for both subareas. California did provide definitions of high poverty an d high m inority schools.(D)(4) Improving the effectiveness of teacher and principal preparation programs !4(D)(4) Reviewer Comments:California provided the MO U used for its institutions of higher education showing a strong com mitmentto the use of student assessment information for the evaluation of training programs and set a targetfor full implem entation by the end of the 2012 -2013 school year. Ca lifornia has already implem ented a 1pilot study for such a system.(D)(5) Providing effective support to teachers and principals(D)(5) Reviewer Comments:California provided information on its beginning teacher supporting assessment program w hich hasbeen in place since 20 05. California provided a plan with high-quality elements for building on thisinitial teacher support system and expanding it to include support for schools and teachers linkingprofessional development to teacher eva luation status and identified needs. Professional developm entefforts will also focus on training turnaround specialists for placement in turnaround scho ols. The plansare ambitious, appropriate, and thoughtfully integrated but do not include a ny information on theamount or source of funds that will be invested in these efforts.Total

E. Turning Around the Lowest -Achieving SchoolsAvailable

(E)(1) Intervening in the lowest-achieving schools and LEAs 10 Tier 110(E)(1) Reviewer Comm ents:California provided docum entation showing that it has been given authority by the legislature tointervene with both districts and schools requiring them to im plement one of the four interventions forturning around the lowest achieving schoo ls and in fact parents can petition to request such action.(E)(2) Turning around the lowest-achieving schools 40 302/18/2010


14/37


15/37

Competitive Preference Priority 2: Emphasis on STEM; Available

15

Grand Total

Technical Review Page 6 of 7Com petitive Preference Priority 2: Emphas is on STEMCompetitive Reviewer Comments:California provided evidence of wide ranging activities and processes to suppo rt science, technology,

engineering, and mathem atics. The application narrative included past STEM ac tivities and plannedemphases aligned with RTTT. The TechNet partnership and the Higher Education Science andMathem atics Initiates are just two examples of the good things California is doing with STEM. Theinformation submitted did not show a p lan for addressing STEM that is sufficiently systematic,compreh ensive, and coherent to be considered of high quality.Total

Absolute Priority - Comprehensive Approach to Education Reform

Absolute Priority - Comprehensive Approach to Education ReformAbsolute Reviewer Comments:California's application presented a comprehensive approach to supporting the four key educationreform areas integrating a wide range of processes an d resources to support reform efforts of localschool districts. California's successes w ith improvemen t efforts and the level of participation of LEAs

should provide a basis for a successful RUT implementation.

2/18/2010


16/37

23


Race to the TopTechnical Review Form - Tier 1

California Application #1400CA-3

A. State Success FactorsAvailable Tier 1

(A)(1) Articulating State's education reform agenda and LEA's participation in it 65 44(i) Articulating comprehen sive, coherent reform agenda 5 4(H) Securing LEA commitment 45 30(ill) Translating LEA participation into statewide impact 15 10

(A)(1) Reviewer Comments:The state's plan clearly articulates a reform agenda, set of goals, and implem entation plan that isinternally consistent and meets the c riteria for reform in the notice. The participating LEAs, w hichnumber ov er 800, represent 58% o f the state's students and 61% of the students in poverty, indicatinga significant level of commitment. Sup port from 78% of the local school boards is also good. The stateboldly chose not to allow participating LEAs to opt out of the specific strategies. The proposal does,however, raise a num ber of concerns. The conc ept of "a new local-state relationship" may result ininefficient implemen tation of the program and m ay represent an attempt by the state to garner supportfor the application -- especially given the am ount and percentage of grant funds being allocated to theLEAs in the proposal -- without garnering commitment to its components. Only 26% of the localteachers union leaders signed on, and many o f the large district union leaders did not. Overall, it islikely the implemented reforms w ill have a broad statewide impact, despite these particular concerns.(A)(2) Building strong statewide capacity to implement, scale up, and sustain0proposed plans(i) Ensuring the capacity to implemen t(H) Using broad stakeholder support(A)(2) Reviewer Comments:The state's empha sis on a regional system of support is sensible given the state's size, geography,and diversity. Further, the incorporation of the CA C omprehen sive Center and Regional C harterInnovation Center, and the establishment of Professional Learning Comm unities, improve the state'scapacity to implemen t its reforms succcessfully. The state has an app ropriate grant oversight budgetand a reasonable p lan for on-going evaluation and funding wha t works, as well as leveraging existing

funding streams by planning to use the RTTT funding for infrastructure-type initiatives. The distributednature of the impleme ntation plan, however, creates a risk of unclear accountability for the overallsuccess of the reform agenda as well as for the specific initiatives. The state does demonstrate verystrong and broad stakeholder suppo rt (with the notable exception of statewide education associations),in many cases at an advanced stage of planning.(A)(3) Demonstrating significant progress in raising achievement and closinggaps

(i) Making progress in each reform area

2/18/2010


17/37

(ii) Improving student outcomes

Total

(B)(1) Developing and adopting common standards(i) Participating in consortium d eveloping high-quality standards

(B)(1) Reviewer Comments:The state is participating in a consortium that is working towa rd timely developme nt and adoption ofcomm on standards, and which includes a significant number of states. The state is appropriatelyauthorized by recent legislation to adopt those standards.(B)(2) Developing and implementing common, high-quality assessments(B)(2) Reviewer Comments:The state is participating in several consortia (including SMA RTER and Achieve) that are developingand impleme nting comm on, high-quality assessments, and which include a significant number ofstates.(B)(3) Supporting the transition to enhanced standards and high-qualityassessments(B)(3) Reviewer Comments:The state has already demonstrated its ability to transition to enha nced standards an d it articulates asolid understanding of the issues surrounding high-quality aligned assessm ents. The state placesparticular emphasis on curriculum and instructional materials, but its proposed reliance on LEAs todrive developmen t of formative assessments may be less efficient and effective.Total 701 68

- Available24


(A)(3) Reviewer Comments:The state has m ade substantial progress initiating reforms in the past few years that a re consistentwith the RTTT reform agend a. The state has demonstrated some ability to improve student outcomesand reduce the achievem ent gap on both the rigorous California Standards Tests and on the NAEP(the achievement ga p shows a slight, but not substantial, reduction). The state has not, however,succeeded in improving its high school graduation rate.

B. Standards and AssessmentsAvailable i Tier 1

C. Data Systems to Support Instruction

(C)(1) Fully implementing a statewide longitudinal data systemTier 1

(C) (1) Reviewer Comments:Credit given for elements 6-8.(C)(2) Accessing and using State data(C)(2) Reviewer Comments:

2/18/2010


18/37

(D)(2) Improving teacher and principal effectiveness based on performance(i) Measuring student growth(H) Developing evaluation systems(Hi) Conducting annual e valuations(iv) Using evaluations to inform key decisions

137

15


The state proposes a plan that builds on its track record o f collecting data, making it publicly available,and offering guidance around its use. The state proposes to solicit stakeholder input into the expandeddata collection plan, and continue to provide good a ccessibility. The January 20 10 legislationauthorizing the ODE to coordinate data for research purposes underscores the state's comm itment tothis criterion. The plan, however, does n ot address the "how" of implem entation, e.g., technologyissues, at all.(C)(3) Using data to improve instruction(C)(3) Reviewer Comments:The state's plan is reasonab le and app ropriately addresses each individual criterion. Its concept of aConsum er Reports-like guide to local instructional improvem ent systems is novel but passive. Theconcept of an institutional review board to screen research requests may be overly restrictive andburdensom e. The plan does not address how the data will enable research with respect to subgroup,remedial and gifted students. The target for 100% im plementation of instructional improvementsystems by 2011-21 is am bitious but may not be achievable; the target of 66% of teachers using datain their practice to change instruction is achievable but may not be am bitious (unless it is restricted totested subjects).Total 479

D. Great Teachers and LeadersI Availableier 1(D)(1) Providing high-quality pathways for aspiring teachers and principals(D)(1) Reviewer Comments:The state provides a multitude of alternative certification pathways for teach ers and principals that arebeing used, as evidence d in the appendix. W ith respect to criterion (iii), the state has a good processfor teachers but mu st rely on some third party non-profit organizations for non-systemic analysis of theprincipal market. Nevertheless, the state has sponsored a program to train principals to serve in high

needs schools. O verall, the state's approach to alternative certification and addressing areas ofshortage is strong and its $64 million project budget focused o n this criterion is highly supportive of itsplan.

218(D)(2) Reviewer Comments:The state's plan addressess all of the requirements of the criteria adequately. The state's approach todeveloping evaluation systems is robust, with flexibility of LEAs to custom ize and collaborate subject tothe stated requirements. But there is signficant risk the Evaluation Ad visory Group (with no cleargovernance) will be too unw ieldy or politicized and fail to produce the required timely output. The planalso risks having recommendations that get watered down or never imp lemented. The planinsufficiently addresses part of criterion (iv) regarding the use of the evaluations for compe nsation,promotion, tenure, and rem oval; this lack of specificity is inconsistent with its aggressive targets to gofrom 10% to 85% of LEAs that use the new evaluation systems for these decisions.

ID)(3)Ensuring equitable distribution of effective teachers and principals 2532/18/2010


19/37


( .0 Ensuring e quitable distribution in high-poverty or high-m inority schools(ii) Ensuring equitable distribution in hard-to-staff subjects and specialty areas

(D)(3) Reviewer Comments:The state demo nstrates a clear understanding of the requirement, has dem onstrated a legislative trackrecord (including the Q EIA and the right of principals to refuse teacher transfers based on seniority) insupport of addressing inequity, and p resents a plan and structure that are likely to support successfulimplementation. The state's explanation for not including targets is reasonable.

(D)(4) Improving the effectiveness of teacher and principal preparation programs ;4(D)(4) Reviewer Comments:The state is comm itted to use its systems to imp rove the effectivess of these programs and to m akepublic the related data. The state demonstrates strong buy-in from C SU -- including the expansion o f apilot program that is highly consistent with this criterion -- and an ex plicit commitment b y participatingIHEs (in the form of MOUs) to expand succ essful programs based on performance da ta. Its target for100% of program s for which the public can access achievemen t and growth data is clearly ambitious,but achievable based on the extraordinary comm itment to this reform that is demonstrated.(D)(5) Providing effective support to teachers and principals(D)(5) Reviewer Comments:The state has a solid plan to provide support for the initiatives to be develope d under Se ction D. Theflexibility to align som e existing professional deve lopment funding to these initiatives, the incorporationof a plan to focus on turnaround spec ialist training and mentoring, and the intention to monitor the

validity of program evaluations are all excellent components of the plan. The novice principal induction .program -- with a $20 million project budget -- is innovative and appropriate. The state's plan to baseits support and professional development on pe rformance evaluation results is also strong andunderscores the state's comm itment to this criterion. Its targets are ambitious and achieva ble, and theexplanation of certain targets that have not been provided is appropriate.Total 13810

E. Turning Around the Lowest-Achieving Schools

(E)(1) Intervening in the lowest-achieving schools and LEAs(E)(1) Reviewer Comments:The state meets the criterion for being able to intervene in low-performing schools and LE As.(E)(2) Turning around the lowest-achieving schools

(i) Identifying the persistently lowest-achieving sc hools(i) Turning around the pe rsistently lowest-achieving schools

(E)(2) Reviewer Comments:The state demonstrates a comm itment to trying to turnaround low achieving schools and a strongmethodology for identifying these schools. The state proposes a we ll-developed, high-quality plan tosupport LEAs in turning around the lowest-achieving schools, with reasonable Oa's -- strongaccountability measures, extensive on-going supports for LEA s, additional resources, continuousimprovem ent mechanism s -- and clear strategies. Small details like providing a model MO U forengaging third-party managem ent organizations reflect the state's high degree of sophistication withrespect to the complexity of turnarounds. The state's target of approximately 200 sch ools per year is

Available i Tier 1100

2/18/2010


20/37

505Available

10

15

Total


very ambitious and may n ot be achievable, but it is not unreasonable based on thecompreh ensiveness of the state's support initiatives. There is no mechan ism to ensure that not morethan 50% of the schools employ the transformation m odel, as specified in the criterion. The projectbudgets for these criteria are reasonable and appropriate.Total

F. General

(F)(1) Making education funding a priority(F)(1) Reviewer Comm ents:The state increased the percentage of revenues used to support education in FY09. The state's schoolfinance formula results in equalized funding across LEA s and schools, with the flexibility thatcategorical funding can be funded flexibly based on local priorities. This formula m eets the criterion (ii)for equitable funding.(F)(2) Ensuring successful conditions for high-performing charter schools and01other innovative schools(F)(2) Reviewer Comm ents:The state has a "high" cap on the numb er of charter schools, multiple authorizers, favorable laws andregulations, and comparable funding. There is no facility funding stream but there are severalprograms that serve to help charter schools solve the facility challenge. (Note it is not clear whether orhow m any charter schools actually benefit from the requirements of school districts to provide spaceunder Proposition 39.) The proposal does not address criterion (v) with respect to other innovativeschools.(F)(3) Demonstrating other significant reform conditions(F)(3) Reviewer Comm ents:

In addition to providing som e financial flexibility and ce rtain early childhood education initiatives, inJanuary 2010 the state enacted legislation that creates conditions favorable to education reform andthe reform mod els included in the application. The state does not, however, provide evidence thatthese conditions have yet resulted in positive outcomes.

Com petitive Preference Priority 2: Emphas is on STEMAvailableier 1

Competitive Preference Priority 2: Emphasis on STEMCompetitive Reviewer Comments:The state's proposal meets the criterion for emphasis on S TEM. It appropriately references othersections of its proposal where it integrates its STEM empha sis. The state has previously demonstrateda comm itment to promote STE M education, and its proposal will likely increase the quality and rigor ofSTE M instruction through professional development and a lternative certification, involve externalstakeholders (e.g., TechNet), and proliferate STE M program s in low-achieving schools and forunderrepresented students through the LE A challenge grants.

2/18/2010


21/37

Available Tier 1Yes

Grand Total 50087Page 6 of 7echnical Review

Absolute Priority - Comp rehensive Approach to Education Reform

Absolute Priority - Comprehensive Approach to Education ReformAbsolute R eviewer Comm ents:The state's proposal is co mprehensive, coh erently addresses all of the criteria, and has sufficiently

broad support to have statewide impact. The state has previously demonstrated its commitment to thereform agenda a rticulated in the notice, and presents a high-quality plan that is focused on improvingstudent achievement and ou tcomes. The mo st troubling features of the plan are the low level ofteachers union support for the ambitious reform agend a, and the risk that the $700 million of fundsrequested by the state for the LEAs (exc luding the amounts in the specific project budgets) is deployedas supplementary operating funding and not used explicity to advance the Race to the Top reforms.

2/18/2010


22/37

Technical Review Page 1 of 10Race to the TopTechnical Review Form - Tier 1California Application #1400CA-4

A. State Success FactorsAvailable Tier 1

(A)(1) Articulating State's education reform agenda and LEA's participation in it 65 19(i) Articulating comprehensive, coherent reform agenda 5 2(H) Securing LEA commitment 45 12(Hi) Translating LEA participation into statewide impact 15 5

(A)(1) Reviewer Comments:California presents a "comprehensive an d coherent reform agen da, and clearly articulates its goals forimplemen ting reforms in the four education areas described in the ARRA and improving studentoutcomes statewide" RA )(1)(0]. Standards and assessmen ts, data systems, hum an capital, and schoolturn around are h ighlighted as linked elements o f the state's RTTT initiative. California does well toframe the RTTT investment as a way to "create conditions for success" and the State clearly lists fivemajor strategies that will "support new wa ys of doing business." Notable amon g these five strategiesare the them es of encourag ing local flexibility and innovation, strengthening local-state partnerships,developing and implem enting metric based accountability systems, and increasing parent choice andempow erment. California also is wise to underscore the RTTT grant as a one-time infusion of cash thatmust be applied to building structures and deve loping capacities that ideally will not require recurringinfusions of Federal assistance. California is to be comm ended for being hone st and evidence-basedin noting areas requiring m ost improvem ent. Unfortunately, California falls short in "establishing a clearand credible path to achieving these go als" [(A)(1)(0]. The link between the reform agend a's goals andactions is not strong. W ithin each of the compelling five strategies (i.e., support local flexibility tostimulate innovation, develop new local-state partnership, etc.), tangible actions are not well explained.For instance, "professional learning com munities" are featured as a major action, but not in way thatindicates ultimate value in schoo ls and classrooms. S imilarly, the "Brokers of Expertise portal," while abrilliant concept, is not shown to be a mecha nism that will make a m easurable difference in principaland teacher p ractice. California is most weak relative to this criteria in its marginal success at "securingLEA commitment" [(A)(1 )(ii)] and lack of evidence that the level of "LEA participation would translateinto statewide impact." To C alifornia's credit, the facts of the situation are clearly shared. Un fortunately,too many doub ts are raised by only 46.5 % of LEAs signing the M0Us. The participation rate is betterfor schools, students overall and students in pov erty (56, 58 and 61%, respectively), but that stillleaves roughly 40 percen t of schools and students untouched by RTTT reform. It is difficult to considerthis as "statewide impact." In addition, California does not exp lain the geographic distribution ofparticipating LEAs and to wh at extent the concentration or dispersion of participating LEAs m ay behelpful to the cause. For instance, if the participating districts are well dispersed, RUT-promptedchanges in a participating LEA m ay foster changes in neighboring, non-participating districts. Californiadid not achieve good results with MOU signatures. Local teacher union p articipation of only 26

ercentat least as conveyed by M OU signatures suggests that deep reform in participating LEAs,not to me ntion statewide, will be difficult to achieve. S imilarly, it is curious that 22 pe rcent of localschool board p residents did not sign the M OUs. O verall, California falls short relative to the reasonabletheory of change that a shared comm itment to reform among superintendents, board presidents and2/18/2010


23/37

Technical Review Page 2 of 10union leadersno m atter how difficult to achieveis essential to major changes in educationalpractices and student learning.(A)(2) Building strong statewide capacity to implement, scale up, and sustainproposed plans 30 9(i) Ensuring the capacity to implement 20 6(H) Using broad stakeholder support 10 3

(A)(2) Reviewer Comments:California's treatment of the criteria for "building strong statewide ca pacity to implem ent, scale up andsustain proposed plans" [(A)(2)] is good conce ptually but weak p ractically. As noted earlier, the"professional learning com munities" and "Brokers of Exp ertise portal" are compe lling concepts, butthere is not good explanation for how these ne tworks (human a nd virtual) will lead to improvedpractices in schools and classrooms. Plans call for forging a "new contract" between the state andLEAs, buts "for instance"as to how it lead to localhereo good explanationven as a willchanges in practice. The language is strong regarding performance measures, data systems andanalysis, and infrastructure elements, but the link to school an d classroom leve l action is not clearlydrawn. California handles well plans for resource acquisition and deploym ent, at least at a generallevel. The state does well to highlight the RTTT investmen t as one-time and thus nece ssary to bedirected at infrastructure and hum an capacity building which ideally would not require sustainedsupport at the unique level represented by RTTT. California's proposed budgetas exp lained in thebody of the proposal and in the a ttachmentsis im pressive for its focus on LEA s, with only fourpercent of RTTT fund s staying solely at the state level. The narrative is clear on the purpose of theproposal and the general app lication of funds. The narrative also explains well how the state-localrelationship around funds will operate conceptuallyrang ing from performance expec tations being setby the state and specific plans and mo nitoring being managed by LE As. Howeve r, the large numb er ofdistinct project budgets (26 by this reviewer's coun t) is not woven together in the narrative to show howthey advance the proposals goals and action plans. The m ultiple projects can be group ed to line-upwith the RTTT's four m ajor reform elements, but the resulting actions are not clearly explained. Lastly,California does not explain well how work proposed in the proposal will be simultaneously advanc ed byRTTT fund s and other Federal, state and private resources. Similarly, the proposal does not explain ina strong way how the work started with RTTT fund s will be continued. As noted earlier, Californiawisely intends to focus RTTT funds on "one-time expenses," but even those a reas as generally listedwill require future investments if deep reform is to oc cur [(A)(2)(i)]. California shows sophistication in itsmix of partnersindicating plans to draw on philanthropy, higher education, parents and business.Again, howe ver, the proposal stops short of explaining ac tions that will follow from California'sexcellent understanding of the nature and limits of RTTT fund s and the importance of involving multiplesectors in reform [(A)(2)(ii)].

(A)(3) Demonstrating significant progress in raising achievement and closinggaps

30 11

(i) Making progress in each reform area 5 4(ii) Improving student outcomes 25 7

(A)(3) Reviewer Comments:As conveye d in the proposal, California is working comprehensively to "dem onstrate significantprogress in raising achievement and c losing gaps" in student learning [(A)(3)]. However, as theproposal shows, the state's record is decidedly mixed. C alifornia has good evidence of imp ortant workin each of the four education reform areas requ ired by RTTT [(A)(3)(i)]. The state would build on effortsand learning with RTT T funding. The state's pace-setting role in standards is well documented in theproposal. The state's creation of data systems also is impressive, but the value-added at the schooland classroom leve l is not clear. Human capital developm ent is another California forte, with highlyregarded professional standards for teachers and principals, multiple alternative routes to licensure

2/18/2010


24/37

125

Page 3 of 10echnical Review

and credentialing, and an imp ressive drop in the proportion of "unprepared" teachers from 20% to 3%in recent years. Turning around low performing sch ools has proven a ch allenge for California, but thestate has learned from these struggles, recognizing now that low performing schools must be dea ltwith in the context of, and in combination with, their LEAs. Ca lifornia's results in changing studentoutcomes are mixed. A strict reading of RTTT's c riteria in this area [(A)(3)(ii)] result in a low score forCalifornia. Progress on the CST has been good , as is documented in the proposal. Similar to otherstates, however, there is a marked discrepancy w ith NAEP results, which show only mo dest studentachieveme nt and growth. Well acknowledge d by California is the difficulty in narrowing achievementgaps among and between subgroups, both on the NAEP and the CST. Lastly, the state's high schoolgraduation rates have dropped o ver the past five years, falling back to a leve l just below the 81%figure in 1996. No explanation is provided in the proposal for this drop and how it may vary am ongsubgroups.

Total

B. Standards and AssessmentsAvailable Tier 1

(B)(1) Developing and adopting common standards 40 40(i) Participating in consortium d eveloping high-quality standards 20 20(H) Adopting standards 20 20

(B)(1) Reviewer Comments:"Developing and ado pting commo n standards" [(B)(1)] is a standout area for California's RTTTproposal. California is actively participating in the NGA/CC SSO Consortium, which includes 48 statesand three territories. Californiaa pione er in the standards movement now appears to be fullycomm itted to the need for comm on core standards across as many states as possible. California willonly adopt the new standards if the "meet or exceed their own" standards. To this end, the state hasbeen central to the Consortium's work by attending national meetings and having top state educationalofficials and senior staff serve on the Co nsortium's three com mittees [(B)(1)(i)]. The state has passedand instituted laws driving commo n core standards, and by the requ isite dates for RTTT funding. Topromote broad ow nership, especially among teachers, the legislature established the AcademicContent Standards Commission, which will include 21 members appointed by the Governor andlegislature, at least half of whom are classroom teachers [(B)(1 )(ii)]. Ultimately, California scores highwith this criterion because of its proven com mitment to being a leader in developing mu lti-state corestandards and adopting them back at hom e.

(B)(2) Developing and implementing common, high-quality assessments 10(B)(2) Reviewer Comments:California is striving to have core standards operate "ha nd-in-glove" with high-quality assessments."Developing and implementing com mon, high-quality assessments" [(B)(2)] clearly is a priority for

California. Impressively, the state is part of at least three major consortia, and the number ofparticipating states and deadlines for design and im plementation mee t RTTT's requirem ents. Californiascores high on this criterion, even thou gh the proposal leav es unclear the state's particular role withthe various consortia and the ultimate quality checks that will be applied to the assessments beforeadoption and implemen tation.(B)(3) Supporting the transition to enhanced standards and high-qualityassessments

20 15

(B)(3) Reviewer Comments:California's history as documented in this proposal conveys the c onundrum of top flight standards notbeing matched by top flight student outcomes. This section conveys strongly how high quality will be

2/18/2010


25/37

Technical Review Page 4 of 10ensured during adoption and implemen tation. As a result, California earns high ratings relative to thecriterion for "supporting the transition to enhanced standards and high-qua lity assessments" [(B)(3)].Most impressive is the logical flow to the state's strategies for how standards and assessm ents will bethe lead piece of a continuous improvemen t cycle, ideally ensuring improved practices and studentachieveme nt and growth. The sequence runs neatly from curriculum framew orks to instructionalmaterials to professional development to assessmen ts to interim assessm ents to formativeassessments to revision of accountability systems. Despite the logic of this model, the proposal doesnot show how California will ensure school and classroom level adoption on a wide scale. In addition,this section is inconsistent in its attention to levels of sch ooling: while giving attention to K-8 grades,there is little provided on what will occur for grades 9-12.

Total [ 703C. Data Systems to Support Instruction Available Tier 1(C)(1) Fully implementing a statewide longitudinal data system 24 6(C)(1) Reviewer Comments:California makes clear statements in the proposal abo ut what is completed, and w hat is missing, as itmoves to "fully impleme nting a statewide longitudinal data system" [(C)(1)]. The continuousimprovem ent cycle featured in this section of the proposal is excellent. More fundamentally, the moveto reduce barriers to connecting student data to teacher history is essential to the RTT T process.Howeve r, at this point only three of the 12 Ame rica COM PETE S Act eleme nts are in place inCalifornia. Thus, the score on this criterion is at the low end of the scale.(C)(2) Accessing and using State data I 3(C)(2) Reviewer Comments:California has two good strategies proposed to adva nce its work on "accessing and using state

data" [(C)(2)]. First, California wisely intends to use RTTT fund s to expand the data elem ents in thestate's longitudinal data system. Second, in a mo ve smartly taking advantage of advanced techno logy,the state intends to create a "on e-stop shop" portal for all state data. Assuming the incredible rangeand depth of state data can be clearly organized and presented in a single portal, the idea should easethe access of L EAs a nd researchers to essential information. California argues that the av ailability ofeasily accessible, timely and accurate data will create user mom entum--that is, the more LEAs se e thedata portal as useful, the more likely they will be to contribute and use the state's "one-stop data shop."Not well explained in this section, however, is what mec hanisms and incentives will be instituted ifLEAs fail to take full advantage of the portal to improve instruction and ultimately student achievem entand growth.

(C)(3) Using data to improve instruction I8 6(C)(3) Reviewer Comments:

"Using data to improve instruction" [(C)(3)] is the final measure of the merits of any data system .California does not adequately explain how its robust data system will make a difference in schoolsand classrooms. California's MOU requires that LEAs use formative assessments and create localinstructional improvem ent systems to a dvance d ata use in instruction [(C)(3)(i)]. California proposes atleast three major mecha nisms to like data to improving instruction: professional learning comm unitieson data use, professional developme nt modules, and ex pansion of the Brokers of Ex pertise portal [(C)(3)(H)]. California also says that it will, "to the extent allowed under law," mak e public education d atafully available to researchers and other inquiry proce sses. To ensure qu ality, availability will be basedon applications to a state "institutional review board" and pa rallel application processes at the LEAlevel [(C)(3)(iii)]. Each of these m echanisms, and the comm itment to data transparency and ac cess, isimpressive. Still, the proposal does not con vey strongly how these m echanisms w ill be fully utilized to

2/18/2010


26/37

Technical Review Page 5 of 10improve instruction. The catch is the extent to which teachers will adopt and im plement the know ledgeand information conveyed in the professional learning commun ities, professional developmentmodules and Brokers of Expertise portal.Total 475D. Great Teachers and Leaders

1

Available Tier 1(D)(1) Providing high-quality pathways for aspiring teachers and principals 21 16(D)(1) Reviewer Comments:California presents strong evidence in the proposal that it excels in "providing high-quality pathways foraspiring teachers and principals" [(D)(1)]. RTTT' s requisite legal, statutory and regulatory provisions foralternative routes are definite and date back to 1967 [(D)(1)(i)]. Since then, as the proposal conve ys,

California has continued to enhance the legal and regulatory backbone for alternative paths to theeducation profession. Multiple paths are av ailable, incorporating instituational resources well beyondhigher education[(D)(1)00]. Monitoring is in place, especially for teachers, giving Ca lifornia the capa cityto track the status of educator labor markets and "actively use information to address inequitabledistribution and help fill shortages." The current systems for tracking the labor m arket for principals arenot well-built and require attention[(D)(1)(iii)]. California needs to improve how it ensures that teach ersand principals coming to the profession through m ultiple pathways are consistently of high caliber intheir effects on student achievement and growth.

(D)(2) Improving teacher and principal effectiveness based on performance 58 23(i) Measuring student growth(ii) Developing evaluation systems 15 7(iii) Con ducting annual ev aluations 10 5(iv) Using evaluations to inform key decisions 28 7

(D)(2) Reviewer Comments:California addresses head-on its weakness at "imp roving teacher and principal effectiveness based onperformance" [(D)(2)]. A major strength of the proposal is that the state will link student achieveme ntand growth to evaluations of educator qu ality as well as the institutions that feed the educational labormarket. The plan impressively links studen t results to the alternative certification pathways [(D)(2)(i)].California is smart to create an Evaluation Advisory G roup to identify best practices for LEAs, givingthe work necessary focus and energy. Peer A ssistance and Review is another good quality toCalifornia's teacher evaluation work. Howe ver, principal quality has not been systematically evaluated,a challenge given how essential school leadership is to educational reform and student achievem ent[(D)(2)(ii)]. Unfortunately, the proposal falls well short at the implemen tation level. Details are not wellstated for how the plan's conce pts will be transformed into practice. For instance, the challenge ofshifting to a growth model will require more implem entation insight than the proposal offers with theproposed "Student G rowth Technical Work G roup." The lack of clear implementation plans isparticularly apparent when the proposal turns to "conducting an nual evaluations" [(D)(2)(iii)].Contractor designed tools and resources for teachers, as we ll as major professional development, arehighlighted as strategies to advance evaluation processes and related decision m aking, but not withcareful detail. Detail also is lacking regarding the sub-criterion, "using evaluations to inform keydecisions" [(D)(2)(iv)]. Good attention is given to d eveloping teachers an d principals through theevaluation process, but evidence can not be found of solid plans for using performance ev aluationsaround m atters of compensation, tenure, and termination. Especially troubling at this point in theproposal is the lack of teacher union owne rship of the RTTT initiative, which will create a major barrierto California's plans regarding personnel evaluation and decisions. The proposal lists teacher unions

2/18/2010


27/37

Technical Review Page 6 of 10as important to the improvement of performance, but does not give evidence for how they will bebrought into evaluation and decision processes that will impact many of their mem bers.(D)(3) Ensuring equitable distribution of effective teachers and principals 25(i) Ensuring equitable distribution in high-poverty or high-minority schools 15 10(ii) Ensuring equitable distribution in hard-to-staff subjects and specialty areas 10 7

(D)(3) Reviewer Comments:California clearly addresses its history and current policies and practices in "ensuring equitabledistribution of effective teachers and principals" [(D)(3)]. Openness is a valued aspect o f the section,with the state expressing how it must do better to mak e sure all students, regardless of LEA or socio-econom ic standing, benefit from high q uality teachers and principals. Fortunately, California has muchstrength in its systems and efforts to track the distribution of educators. Impressively, the state hasbeen pursuing the issue through multiple approachese nsuring that the process is not dependent onone program or mechanism. California will use RUT to advance new strategies designed to improveteacher and p rincipal distribution. Together, the strategies are coherente stablish metrics anddevelop a system for monitoring; provide technical assistance to LEAs to m ake use of the metrics andsystem; and b etter define categories for effective teachers and principals. The lingering doubt in thissection is how the state will follow-on if these strategies are not well implemented by LEAs.

(D)(4) Improving the effectiveness of teacher and principal preparation programs 14 10(D)(4) Reviewer Comments:California has an excellent mode l and system in place for "improving the e ffectiveness of teacher and

principal preparation programs" [(D)(4)]. Most impressive is the direct link with C alifornia StateUniversity (CSU) in this aspect of the proposed R TTT initiative, since CS U is the institution preparingmore than half of the state's teachers annua lly. Specifically, the state will draw lessons from a largepilot CSU's C enter for Teacher Qua lity is conducting in five of the largest urban school districts to linkstudent achievemen t and growth data to C SU gradu ates [(D)(4)(i)]. California's intention to bringCSU 's work to scale by e ncouraging its adoption in other higher education institutions is wise. Ideallythe other institutions wou ld be drawn to the new policies and practices without state intervention, butCalifornia is smart to try to use RUT support to break down institutional silos and barriers. The statehas set three strategies to encourage wider adoption of CSU 's worko nline reporting of programassessments; incorporating results from effective programs into ac creditation requirements; and,supporting expansion or revision of p rograms based on performan ce results [(D)(4)(ii)]. Unfortunately,the proposal does not provide eno ugh attention to principal preparation programs, pushing the scoredown to the "medium" range.

(D)(5) Providing effective support to teachers and principals 200(D)(5) Reviewer Comments:California presents well its history of "providing e ffective support to teachers an d principals" [(D)(5)].The state has a notable record of wo rrying about the induction of new teachers, me ntoring them andproviding professional development. This history establishes a good foundation for the proposed R TTTinitiatives. California by strategy label and category hits all key areas: induction for teachers andprincipals; professional developmen t on instructional materials; professional development based onperformance eva luation; and regionalized assistance to LEAs to develop solid support systems. Inaddition, the state's new attention to developing "turnaround leaders and sp ecialists" is creative andapt [(D)(5)(i)]. Not as clear, and thus the reason for the score in the mid-range, is the e xplanation forhow these strategies and plans will make a difference at the school and classroom leve l. Californiaaddresses the evaluation of support programs for teachers and principals [(D)(5)00] by proposing toexpand the data system (i.e., CALTIDE S) used to track quality. While a good idea, the plan for datasystem expansion is not accomp anied with explanations for how the results would be applied toimprove professional development programs.

2/18/2010


28/37

13 8 6Technical Review Page 7 of 10E. Turning Around the Lowest-Achieving Schools

Available Tier 1(E)(1) Intervening in the lowest-achieving schools and LEAs 10 10(E)(1) Reviewer Comments:California has good systems and structures for "intervening in the lowest-achieving schools andLEAs" [(E)(1)]. The state learned from working primarily with individual schools. The state hasbroadened its reach to include the LEAs within which the struggling schools reside. Legal, statutoryand regu latory authority is in place. Californi

Documents

Comments on CA RTTT Application