21st Century Community Learning Centers 2008-09 APR and

21st Century Community Learning Centers 2008-09 APR and Cross-Year Analysis of Performance

Data

September 1, 2010

North Carolina Department of Public Instruction Division of Program Monitoring/ Support Services Section

2

ACKNOWLEDGEMENTS

The North Carolina Department of Public Instruction/ Division of ProgramMonitoring/SupportServicesSectionacknowledges the inputand feedbackof thesuperintendents, school and central office personnel, and the University of NorthCarolina at Pembroke Office of Academic Affairs for their dedication to thecompletionofthisreport.

i

TABLEOFCONTENTS

Table of Contents……………………………………………………………………… i

List of Tables……………………………………………………………....................... ii

List of Figures…………………………………………………………………………. iii

Introduction……………………………………………………………………………. 1

I. Grantee Profiles and APR Data……………………………………………………... 3

Grantees, Centers and Partners…………………………………………………… 3 Types of Organizations…………………………………………………………... 6 Times and Hours of Operation…………………………………………………… 8 Subject Areas and Center Operation……………………………………………... 10 Parental Involvement…………………………………………………………….. 12 II. Attendees, Characteristics and Outcomes for Attendees…………….…………….. 14 Attendance Statistics……………………………………………………………... 14 Regular Attendees………………………………………………………………... 15 Specific Outcomes for Regular Attendees……………………………………….. 19

III. APR Data on progress for cohorts 3, 4, & 5………………………………………. 23

SubjectAreas:Math,Science,andReading………………………………………………… 23 Attendancestatistics(bycenterandgrantee)……………………………………………. 28 Costperstudent……………………………………………………………………………………….. 31 Regularattendees…………………………………………………………………………………….. 32 Specificoutcomesforregularattendees…………………………………………………….. 38

IV. Conclusion………………………………………………………………………… 42

ii

LISTOFTABLESTable Page

1. 21stCCLCGrantsAwardedinNorthCarolinaandFundingLevel………….. 2

2. CountsforGrantees,CentersandPartnersbyCohort………………………….. 6

3. NumberandPercentofGranteesbyTypesofOrganizations………………... 7

4. NumberofCentersbyTypesofOrganizationsinEachCohort……………… 7

5. NumberofAdultFamilyMembersServedbyCentersin2006‐07,2007‐08and2008‐09APR………………………………………………………………… 13

6. StudentAttendanceDatafromthe2007,2008and2009APR…………….. 15

7. CharacteristicsofRegularAttendeesbyAPRSchoolYear……………………. 17

8. AttendanceDataforCohorts3Through5…………………………………………… 29

9. CostPerStudentbyCohortandAPRYear…………………………………………… 32

10. CharacteristicsofRegularAttendeesbyCohortandAPRYear……………... 33

11. TheProportionofStudentsinEachProficiencyLevelforReadingandMathbyCohortandAPRYear…………………………………………………………….. 35

iii

LISTOFFIGURES

Figure Page

1. GranteeProfiles‐TotalNumberofGrants,NumberofCentersandNumberofPartnersbyCohort……………………………………………………………. 4

2. NumberofCentersandHoursofOperation…………………………………………. 9

3. ProportionofCentersCharacterizedbyTypicalHoursofOperationPerWeekDuringtheSchoolYear……………………………………………………………… 10

4. ProportionofCentersbySubjectAreasProvidedDuringtheSchoolYearfor2006‐07,2007‐08,and2008‐09…………………………………………….. 11

5. AverageNumbersPerWeekTypicallyProvidedDuringtheSchoolYearbySubjectAreafor2006‐07,2007‐08,and2008‐09……………………………. 12

6. ProportionofRegularAttendeesinEachCohortbyGradeLevel,2006‐07,2007‐08,and2008‐09………………………………………………………….. 18

7.PercentofRegularAttendeesbyChangesinPerformanceLevelsForReading/LanguageArtsandMathby2006‐07,2007‐08,and2008‐09APR…………………………………………………………………………………………………….

19

8. PercentofRegularAttendeesbyGradeChangesinReading/LanguageArtsandMath,2006‐07,2007‐08,and2009APR………………………………… 21

9.PercentofRegularAttendeeswithSchoolYearImprovementsinBehaviorFederalTeacherSurveyfor2006‐07,2007‐08,and2008‐09APR……………………………………………………………………………………………………

22

10. ProportionofCentersbySubjectAreasProvidedforCohort3……………… 2311. ProportionofCentersbySubjectAreasProvidedforCohort4……………… 2412. ProportionofCentersbySubjectAreasProvidedforCohort5……………… 2513. Cohort3Through2008‐09APR………………………………………………………….. 2614. Cohort4Through2008‐09APR………………………………………………………….. 2715. Cohort5Through2008‐09APR………………………………………………………….. 2816. TotalStudentsbyCohortAcrossYears………………………………………………... 3017. RegularAttendeesbyCohortAcrossYearsofGrant…………………………….. 3118. Cohort3,4and5ProportionofRegularAttendeesbyGradeLevel………. 37

19. PercentofRegularAttendeesbyGradeChangesinReading/LanguageArtsandMath,Cohort3……………………………………………………………………… 38



22. ChangesinStudentBehavior,Cohort3………………………………………………... 4123. ChangesinStudentBehavior,Cohort4………………………………………………... 4124. ChangesinStudentBehavior,Cohort5………………………………………………... 42

1

INTRODUCTIONThe Harvard Family Research Project (HFRP) has reported consistently thatincreased investments in out‐of‐school time programs result in a “substantialevidence base about [their] academic, social, health and other benefits.” The 21stCentury Community Learning Centers (CCLC), have, for the past ten years,encouraged schools to partnerwith community‐ and faith‐based organizations todevelopwell‐designedandimplementedprogramsthatcouldshowtohavepositiveeffects on academic outcomes, socio‐emotional development, crime, drug anddelinquency prevention, and promotion of health and nutrition. Referred to asexpandedlearningopportunities(ELO),afterschoolandsummerlearningprogramsthat work with schools to support student success, have become importantadditionalpathwaystolearning.121st CCLC is a federally funded program that is administered by the individualstates’ Department of Public Instruction (DPI). Funds are requested by localgranteeswho then plan and implement the programs once they are approved byDPI.AccordingtoNorthCarolina’s21stCCLCwebpage,grantfundsareavailableto“assist schools, local education agencies (LEAs), and faith‐or community‐basedorganizationstoprovidebeforeschool,afterschoolandsummerservices.”Programsservestudentsinallgradelevelsfrompre‐kindergartenthrough12thgrade.The purpose of the 21st CCLC grants is to “provide students with academicenrichment opportunities along with activities designed to complement thestudents’ regular academic program.” Thus grantees are encouraged to scheduleacademic,cultural,andrecreationalactivitiesthataredesignedtoattractandretainstudents for the duration of the program year. Centers should offer participantsneeded youth development and support programs. Critical factors for achievingpositive21stCCLCoutcomes includesustainedparticipation,qualityprogramming,well‐preparedstaff,andstrongpartnershipsamongprograms,parents,schoolsandcommunity institutions. 21st CCLC grantees are also required to offer familymembersofparticipatingstudentsopportunitiesforeducationaldevelopment.2InNorthCarolina,DPIhasawardedoperatinggrants for21stCCLCprogramsfrom2002to2009asreported inTable1.Asof the2009grantawards,NorthCarolinahasmade $44,983,824 available. Grants typically range from three to four years,withanadditionalyearavailabletothosegranteesthatqualifyforexpansiongrants.GranteesinCohort5throughCohort8areactivecurrently.

1SupportingStudentOutcomesThroughExpandedLearningOpportunities,HarvardFamilyResearchProject,January2009.221stCenturyCommunityLearningCenters.http://www.ncpublicschools.org/21cclc/

2

Table1.21stCCLCGrantsAwardedinNorthCarolinaandFundingLevel

AwardYear CohortNumberofGrants3 TotalAllocation*

2002 1 16 $5,900,4142003 2 14 $4,981,1582004 3 33 $10,840,0912005 4 29 $7,491,0152006 5 7 $1,375,4622007 6 12 $1,375,4622008 7 19 $3,632,334

Spring2009 7B 12 $2,075,000Fall2009 8 31 $7,312,888Totals $44,983,824

NorthCarolina21stCCLCgranteessubmit“hardcopy”reportsandarerequiredtocomplete an online report through a data collection system created by LearningPoint Associates. Learning Point Associates was contracted by the Federalgovernment to create the Profile and Performance Information Collection System(PPICS), to allow grantees to enter and edit their data. One of the conditions forreceivingagrantisthatawardeesagreetosubmitquarterlyandannualreportsoftheiractivitiestobothstateandFederalofficials.PPICS collects data in two components, the Grantee Profile and the AnnualPerformanceReport(APR).TheGranteeProfileincludesinformationtypicallyfoundin a grant proposal and includes information on what the grantee expects toaccomplish.TheAPR,bycontrast,iscompletedattheendoftheprogramyearandencompasses informationaboutwhat thegranteeactuallyaccomplished, includingdata about student achievement. DPI contracted with the University of NorthCarolina‐Pembroke to assist in the production of program overviews on theprogressbeingmadebygranteesandpreparationofanAnnualEvaluationReportfor2008‐2009.Thereport that follows isbasedonthecompleteddata for theschoolyears2006‐2007,2007‐2008, and2008‐2009. It includesa longitudinal cross‐yearanalysisofthesethreeschoolyearsforactivegranteesaswellasanevaluationoftheprogressofCohorts3,4,and5acrossyearstoshowhowmuchtheseCohortshaveimprovedtheir performance each successive year in the grant. The grantees that receivedawards in 2009will submit their annual data for the first time at the end of the3Thenumberofactivegrantsinacohortcanvarybetweenawarddateandexpectedendingdate.ThedatainthisandsubsequenttableshavebeenverifiedwithDPIasactive.

3

2009‐2010schoolyear.For thisreport,GranteeProfilesandAPRdata forcohortsthree througheightwereexamined.Thisyear’sannual reportalsocontainscross‐year analyses of performance data where comparisons are possible to look atprogressovertime.The findingspresented in thisreportcamefrombothdisaggregateddata thatwasdownloaded from the “Export Data” module on the Learning Point website andaggregateddatafromtheReportssectionoftheLearningPointwebsite.Becausethedata collection system is subject to revision and/or addition of data, totals in theaggregateddataoftendiffer from those in the totals computed from the rawdataexportedfromtheLearningPointwebsite.Wherethisisthecase,footnotesareusedto clarify the discrepancy. There are also data elements, like gender andrace/ethnicity, for which grantees were not able to enter all the informationrequested. For purposes of clarity, the number and percent of grantees reportingeachdataelementwillbepresentedthroughoutthisreport.Lastly, there are several grantees recorded inmore than one cohort for receivingtwoawards.Thesegranteesusedthefundstoformnewcenters.Theanalysesinthisreportarebasedonthenumberofgranteesandcentersineachcohort,sincetheyrepresentseparatelyfundedinitiatives.I.GranteeProfilesandAPRDataIn this section we discuss the characteristics of grantees and centers, includingtotals, types of organization, times and hours of operation, subject areas, andparentalinvolvement.TheGranteeProfiledatacontaininformationon94grantees.In PPICS, the Grantee Profile section describes the characteristics of these 94grantees,whileAPRdataareavailable forall cohortswhohave completedoneormoreawardyearsandsubmitteddata.Grantees,CentersandPartnersFigure1 shows the total number of grants awarded, projectednumber of centersandprojectednumberofpartnersby cohortyearbeginningwith the2004cohortfrom the Grantee Profile data. Although most of the grantees from 2004 cohort(Cohort3)completedtheirfinalyearoffundingin2008,thereweresomegranteesthatreceivedaone‐yearextensiongrant.Thesameistrueforcertaingranteesinthe2005 cohort. It should be kept in mind that data from the Grantee Profiles areprojected figures reportedbygranteesat thebeginningof theprogrammingyear.These figuresoftendiffer from the totals reportedat theendof theprogrammingyear.

4

Figure1.GranteeProfilesTotalNumberofGrants,NumberofCentersand

NumberofPartnersbyCohort Source: Learning Point. 2009. “Grantee Profile Downloads: excel_grantees excel_partners, and excel_centers.” 21st CCLC Profile and Performance Information Collection System. http://ppics.learningpt.org/ Centers are the number of separate physical locations where after school, grant‐funded programs are located and services are provided to participating studentsand adults. Each center requires a part time Site Coordinator as well as staff forhomeworkhelp,teaching,andculturalandrecreationalactivities.A21stCCLCgrantmustfundatleastonecenter.The 21st CCLC program encourages partnerships between diverse communitystakeholders and LEAs. Partners are organizations or individuals that provide inkind or financial support for activities. Partners include such assistance as localbusinesses that provide snacks, public safety officials teaching fire and personalsafety lessons, volunteers who teach art or music, or citizens that serve on theAdvisoryBoard.Encouragingpartnershipsbetweenschoolsandotherorganizationsisanimportantcomponentofthe21stCCLCprogram.Inadditiontotheexamplesofpartnerslisted,subcontractorsarealsoconsideredtobeasubsetofpartners.The lengthof thebars indicatesan increase innumberofactivegranteesbetweencohorts2006and2008, followedbyasignificant increase in the2009cohort.Thenumberofcenterspergranteeandthenumberofpartnerspergranteehasvaried

5

over thehistoryofNorthCarolina’s21stCCLCs.Howeveras this figureshows, thetotal numbers of centers and partners have increased from the 2006 to 2009cohorts. Since 2007, the number of partners and centers has risen significantlyamongactivegranteesineachofthethreecohorts.Overall,basedonGranteeProfiledataofactivegrantees, theminimumnumberofcenters and partners was 9 and 38, respectively, in the 2006 cohort and themaximumwas 128 centers and 254 partners in the 2009 cohort. Since partnerscontribute inavarietyofwaystocenterprogramming,mostnotablybyprovidingprogrammingandactivity‐relatedservices,centerswithmorepartnersarelikelytohave more resources available to sustain their programming. Developing anintentionalpartnershipstrategywithcommunityagencies isa strategy todevelopcommunitysupportfortheprogramswhenFederalfundsareexhausted.Thevarioustypesofcommunityandsubcontractorpartnersofferdiverseresourcesandservicesfor21stCCLCprograms.AccordingtoPPICS,themostpopularpartnercontributionwasprovidingprogrammingandactivity‐relatedservices,followedbyprovision of goods for the program. The least popular type of contribution bycommunityandsubcontractorpartnerswasraisingfundsfortheprogram.4Table2reportsthedistributionsofpartnersandcenterscomparedtothenumberofgranteesforeachcohortfundedsince2004.AccordingtoGranteeProfiledatatherewere385centersand1,040partnersprojectedfor2008‐09amongactivegrantees.Includingthosegranteesfromcohorts2004and2005thatwereactive,theaveragenumber of centers per grantee is approximately four. The 2004 cohort had thelargest projected numbers of centers per grantee while the 2009 cohort had thelargestprojectednumberofcenterspergranteefortheperiodof2006to2009.The2006cohorthadthesmallestprojectednumberofcenters.Thetablealsoshowsthattheaveragenumberofpartnerspergranteedecreasedbycohortwitha slight increasebeginning in2008. Although theaveragenumberofcenterspergranteealsodecreasedsince2004,the2007cohortcontributedtoariseintheprojectednumberofcentersandaveragenumberofcenterspergrantee.

4Ibid.

6

Table2.CountsforGrantees,CentersandPartnersbyCohort

CohortNumberofGrantees1

ProjectedNumberofCenters

AverageNumberofCentersperGrantee

ProjectedNumberofPartners

AverageNumberofPartnersperGrantee

2004 5 15 3 347 69.402005 12 52 4.33 212 17.672006 5 9 1.80 38 7.602007 10 32 3.20 58 5.802008 19 38 2.00 131 6.89

2009 43 128 2.98 254 5.91Total

CurrentlyActive

94 274 2.91 1,040 11.06

Source:LearningPoint.2009.“GranteeProfileDownloads:excel_granteesandexcel_centers.”21stCCLCProfileandPerformanceInformationCollectionSystem.http://ppics.learningpt.org/(1)CountsincludeseveralLEA/countieswithmorethanonegrantee.Somegranteesalsoarerepresentedinmorethanonecohort.Thenumberofactivegrantsinacohortcanvarybetweenawarddateandexpectedendingdate.ThedatainthisandsubsequenttableshavebeenverifiedwithDPIasactive.TypesofOrganizationThereareatleast12possibletypesoforganizationsthatcharacterizebothgranteesand centers. According to Table 3, Grantee Profile data indicate nearly 32% ofgrantees in Cohorts 3 through 8 were school districts. This is a contrast fromprevious reporting years,whereat least half of all granteeswere school districts.The table shows that community‐based organizations (CBO) and faith‐basedorganizations (FBO) followed school districts at approximately 29% and 23%,respectively.Charterschoolswereadistantfourthat3.2%.Thesefiguresrepresenta continuing shift from the earliest years of the program when a much higherproportionofthe21stCCLCgranteeswereschooldistricts.Conversely, thenumberofcentersbyorganizationtypeof the2006through2009cohorts,asshowninTable4,reveals thatoverhalfareschools.Community‐basedand faith‐based centersmade up approximately 13% and 15%, respectively. Thedatarevealthatalthoughgranteesareequallydistributedamongthetopthreetypes(schooldistricts,faith‐based,andcommunity‐basedorganizations),schoolsassitesfor centers remain the practical choice for grantees, whether school district orotherwise.

7

Table3.NumberandPercentofGranteesbyTypesofOrganizations

200809APR(Cohorts38)

N %

SchoolDistrict 30 31.9%Community‐BasedOrganizations 27 28.7%Faith‐BasedOrganizations 22 23.4%CharterSchools 3 3.2%YWCA/YMCA 3 3.2%OtherUnitofCity/CountyGovernment 2 2.1%NationallyAffiliatedNon‐ProfitAgency 2 2.1%For‐ProfitEntity 2 2.1%BureauofIndianAffairs 1 1.1%CollegeorUniversity 1 1.1%

Total* 94 100.0%

Source: Learning Point. 2010. “Grantee Profile Downloads: excel_grantees.” 21st CCLC Profile and Performance Information Collection System. http://ppics.learningpt.org/ *Totals for 2008-09 APR include all active grantees from 2004, 2005, 2006, 2007, 2008 & 2009

Table4.NumberofCentersbyTypesofOrganizationsinEachCohort

2006Cohort

2007Cohort

2008Cohort

2009Cohort

Total %

SchoolDistrict 7 21 22 79 129 62.32%

Community‐BasedOrganizations 0 2 10 15 27 13.04%

Faith‐BasedOrganizations 2 9 3 17 31 14.98%

CharterSchools 0 0 1 1 2 0.97%

Park/RecreationDistrict 0 0 0 5 5 2.42%

OtherUnitofCity/CountyGovernment 0 0 1 3 4 1.93%

NationallyAffiliatedNon‐ProfitAgency 0 0 0 1 1 0.48%

Other 0 0 1 7 8 3.86%

Total 9 32 38 128 207 100.0%Source: Learning Point. 2009. “Grantee Profile Downloads: excel_centers.” 21st CCLC Profile and Performance Information Collection System. http://ppics.learningpt.org/

8

TimesandHoursofOperationAmajorfactorinthesuccessfulimplementationof21stCCLCafterschoolprogramsistheamountoftimethatstudentsareengagedandtheavailabilityofprogrammingduring the week. 21st CCLC centers in North Carolina are required to operate aminimumof12hoursperweek.Centershave theoption toprovideprogrammingbefore school hours, after school hours, during weekends, and/or during thesummer. Thus a center could operate two hours after school and two hours onweekendstomeetits12‐hourperweekrequirement.Figure2showstheactualtimesofoperationforthecurrentlyactivecohortsbasedon APR data for the 2006‐2007, 2007‐2008, and 2008‐2009 program years. It isimportanttonotethatsummeractivityisrecordedintheAPRdataforthefollowingschoolyear.Therefore,wecanonlymakecomparisonsacross cohorts fordataonsummeroperations.Accordingto2007‐2008data,centersinNorthCarolinawereopen,onaverage,for9.48hoursand2.74daysperweek.Duringthe2006‐2007schoolyearcenterswereopen for 11.11 hours and 2.96 days perweek.While in the 2005‐06 school yearcenterswereopen,onaverage,for11.48hoursand2.92daysperweek.Thustherehasbeenasteadydecline inweeklyhourlyoperationsince the2005‐06reportingperiod.

9

Figure2.NumberofCentersandHoursofOperation

10

Figure3showstheproportionofcenterscategorizedbythetypicalnumberofhoursofoperationperweekforschoolyears2006‐2007to2008‐2009.Thelengthsofthebars indicatetheproportionofcentersthatoperatedagivennumberofhoursperweek during the 2006‐07, 2007‐08, and 2008‐09 school years. The majority ofcenters (approximately 74%) operated between 11 and 15 hours per week.AccordingtoPPICS,alargenumberofcenterswereopenMondaythroughThursdayduringtheschoolyear.ThosecentersthatwereopenonFridaytypicallyusedthatday for cultural and recreational activities, or other non‐subject area specifiedinstruction.Centersoperatingforonetofivehoursperweekwereproportionatelyfewerduringthe 2006‐07 and 2007‐08 school years than the 2008‐09 school year. Many 21stCCLC program directors assigned the duty of reporting center‐level data to sitecoordinators/representatives at the individual centers.Therehavebeen instancesin the past where errors have been made in the reporting of data, which couldcontributetothereportingofhoursofoperationbelow11hoursduringtheweek.However, without actual site monitoring it is impossible to know for certainwhetherthesegranteesmadeerrorsornot.

Figure3.ProportionofCentersCharacterizedbyTypicalHoursofOperationPerWeekDuringtheSchoolYear

SubjectAreasandCenterOperationThecoreacademicfocusfor21stCCLCsisimprovingthestudents’scoresonEnd‐of‐Gradetests(EOGs)inreadingandmathematics.Inadditiontoprogrammingtocore

11

areas of reading and math, centers offer a wide range of activities and subjects.Figure4showsthatReading,Math,Science,andArtofferingshavebeenprominentamongthevastmajorityofcenters.Othertopactivitiesorservicesofferedbymostcenters include Academic Enrichment Learning Programs, Tutoring, HomeworkHelp,RecreationalActivities,andParentalInvolvementactivities.ReadingandMathwereprovidedatnearlyallcenters(approximately100%)fortheschool years 2006‐07 to 2008‐09. Science and Art activities follow these subjectareas.Althoughschoolyears2006‐07and2007‐08showaconsistenteffortbyover93% of centers to provide Science and Art, the 2008‐09 school years show adecrease in the percentage of centers. The figure also indicates that during theschoolyearsof2006‐07and2007‐08CulturalandHealthactivitieswereprovidedbythemajorityofcenters(atleast88%).Thosesamesubjectareaswereprovidedin2008‐09byaslightlylesspercentage(approximately85%)ofcenters.However,during the 2008‐09 school year, more centers (93%) provided Technological‐relatedactivitiesandservicesthanduringtheprevioustwoschoolyears.

Figure4.ProportionofCentersbySubjectAreasProvidedDuringtheSchool

Yearfor200607,200708,and200809Figure5providesadditionalinformationaboutcentersandthetimespentofferingspecific subjectareas.ReadingandMathwereprovidedat thehighest rateduringthe school yearby centers. In2006‐07, centers spent an averageof fivehours onReadingandfourhoursonMath.Boththe2007‐08and2008‐09schoolyearsshow

12

fewerhoursspentonthesesubjectsbutnotasignificantdecrease(slightlylessthanfourhourspersubject).Technologyisthethirdhighestsubjectareabytimespent.For all three years it averaged at 2.5 hours per week. Science was behindTechnology at approximately 2.3 hours perweek. This table also reveals that theamountof timespentonHealthactivitieswasgreater thantimespentonCulturalactivities. Other programming received more attention during the 2006‐07 and2007‐08 school years than in school year 2008‐09. One component of otherprogramming is physical education, which is gaining more attention to reducechildhoodobesityandaddressassociatedhealthissues.

Source:LearningPointAssociates.2009.“APRReports:APRandServicesProvided‐AggregatedReportingApproach.”21stCCLCProfileandPerformanceInformationCollectionSystem.http://ppics.learningpt.org/Figure5.AverageNumbersPerWeekTypicallyProvidedDuringtheSchool

YearbySubjectAreafor200607,200708,and200809ParentalInvolvementNumerous research studies note the importance of parental involvement inchildren’s academic achievement. Serving parents, consequently, is a keycomponentofthe21stCCLCinitiativeand21stCCLCsiteshavebeenencouragedbyNCDPIleadershiptoinvolveparentstohelpsustainimprovementsamongstudentparticipants.

13

Since the 2005‐2006 school year, the percentage of centers providing programsinvolving parents and promoting family literacy has increased. According to the2009GranteeProfile,57%ofall centersassigned“high”priority toprogramsthatpromoteparentalinvolvementandfamilyliteracy.Table5 indicates that,overall, fewerparentshavebeenservedover the last threeAPRyears.However,centersareservingroughlythesameamountineachAPRyear.Roughly20%ofcentersprovidenoadultfamilyinvolvementactivitiesbutthereissignificantvariationintheamountofparentsservedbetweencentersforthosethatdo offer services. The proportion of centers serving between 1 and 25 parentsincreasedbynearly12% from2006‐2007 to theschoolyearsending in2008and2009.Thistablealsoshowsthatfewcentersservemorethan50parents,ingeneral.

Table5.NumberofAdultFamilyMembersServedbyCentersin200607,200708and200809APR

PercentofCentersNumberofParents

Served 200607(N=234)

200708(N=205)

200809(N=190)

0 22.6% 20% 22.6%125 29.1% 42% 41.6%2650 23.1% 21% 16.8%5175 9% 9.3% 7.4%76100 6.4% 2.9% 3.2%101125 4.3% 1% 3.7%126+ 5.6% 3.9% 4.7%

TotalParentsServed 9,014 6,186 5,757

Mean 38.5 30.2 30.3Minimum 0 0 0Maximum 250 221 210Standarddeviation 46.1 38.8 39.7Source:LearningPoint.2010.ExportData,APRDownloads,ExportAPRAttendanceData(Year).21stCCLCProfileandPerformanceInformationCollectionSystem.http://ppics.learningpt.org/

Inthissectionseveralaspectsof21stCCLCgranteesandthecentersthatdelivertheprogrammingwerehighlighted.Datasuggest that the totalnumberof centersandpartnershasincreasedsteadilyforeachcohortfrom2006to2009.Onegoalofthe21st CCLC program is to establish viable relations between grantees and partnersthat could help make centers stronger. Although the numbers of prospectivepartners have increased over four cohorts (2006 through 2009), the averagenumberofpartnerspergranteehasfluctuatedtoreducedlevelsovertime.

14

Schooldistricts,CBOsandFBOsmakeupthebulkof21stCCLCgrantees.WhilesomeCBOsandFBOsdoserveassitesforcenters,mostformpartnershipswithschoolssoastoutilizethemascentersfortheirprograms.Mostcentersoperatedbetween11and 15 hours per week. Centers that operated more than 15 hours per weekincludedthoseprogramsthatwereopenonweekendsandbeforeschool.Reading andMath, followed by Technology and Science topped the subject areasoffered by centers during the week. In addition to serving students, the vastmajorityofcentersreportprovidingservicestobetween1and50parents.Lessthana quarter reported not serving any parents. In the next section, we look at thedistributionandcharacteristicsof21stCCLCattendees.II.Attendance,CharacteristicsandOutcomesforAttendeesSection II reports descriptions of attendance; characteristics of regular attendees,including gender, race/ethnicity, and participation in special programming; gradeand state assessment levels; and changes in performance. Total attendees aredefined as all studentswho attend a center at least one day during the program.Regular attendees are those students who attended programming for 30 days ormoreduringtheprogramyear.AttendanceStatisticsDataonattendance is crucial forgranteesandstakeholders tobeable toevaluateprogramperformanceandeffectiveness, aswell as assessing theprogram’s reach.GranteesaredefinedinPPICSastheentitiesservingasthefiduciaryagentfor21stCCLCgrants.Thenumberofattendeespercenterisimportantfromaprogrammingperspective; the significance of attendeesper granteehelps to determine the costper student.The21stCCLCprogram isestimated tocost$1,300perstudent.Thusthe number of attendees helps measure success and challenges from a fundingperspective.Table 6 shows APR attendance data for total student attendees and regularattendeesduringschoolyearsendingin2007,2008,and2009.Notethattotalsfromthe2007APRincludeCohorts2through5,whiletotalsfromthe2008APRincludeCohorts3(also3and7E)through6.The2009APRdataincludesCohorts4through7,including4&7Eand3&7E.5

5Granteesthatarelistedinmultiplecohortssuchas3&7Eor4&7Ereceivedanadditionalyearoffundingtotheoriginalcohortyear.Thus3&7Eextendedgranteefundingtoyear2009and4&7Eextendedgranteefundingtoyear2010.

15

Table6.StudentAttendanceDatafromthe2007,2008and2009APR

2007 2008 2009

TotalNumberofStudentAttendees 21,228 15,643 13,879TotalNumberofRegularAttendees* 13,351 10,807 9,544

AverageNumberofRegularAttendeesperCenter 57.06 52.72 50.77

PercentofStudentAttendeesMeetingtheDefinitionofRegularAttendee

62.89% 69.09% 68.77%

*Regularattendeesarestudentswhoattendedthecenterfor30daysormoreduringtheyear.CountsarerevisedperiodicallybyLearningPointtoreflectthelatestdataenteredbygrantees.Source:LearningPoint.2010.APRReports,“NumberofTotalStudentsandRegularAttendeesServedbyCenters(Year).”http://ppics.learningpt.org/

Asthetableindicates,thetotalnumberofstudentattendeesdeclinedsharplyfrom2007to2008.In2009thetotalalsodeclinedbutnotasgreatasthepreviousyear.Inaddition,thenumberofregularattendeesdecreasedsteadilyoverthethree‐yearperiod.Onerationale for thedecrease is thatby2008,Cohort2hadcompleted itsgrantperiod.Theaveragenumberof regularattendeespercenterdecreasedover theperiodof2007‐2009, but not significantly. Centers in North Carolina sustained an averageparticipationof53.52regularattendeesoverthethree‐yearperiod.RegularAttendeesSince attendance informs us of the amount of exposure participants have toafterschoolprogramming,itisessentialforthepurposeofevaluatingthe21stCCLCtoobservethecharacteristicsofregularattendees.Thesearethestudentsthathaveparticipated significantly in the afterschool program towarrant an effect on theirperformance and classroombehavior.Hence they are the focus ofDPI’s efforts tomeasuretheeffectivenessof21stCCLCprograms.Table7displaysdataforregularattendeesongender,race/ethnicity,participationin specialprogramming, and state assessment levels.Thesedata are fromreportsfound on the Learning Point website. Recall that data reported for 2006‐2007includeCohorts2through5,datareportedin2007‐2008includeCohorts3through6,and2008‐2009dataincludeCohorts4through7(includingCohort3&7E).According to Table 7, the program serves roughly equal proportions ofmale andfemale students. Reporting on gender characteristicswas relatively high over the

16

2006‐07,2007‐08,and2008‐09schoolyears.Thisisastarkcontrastfromthe2004‐05schoolyearwheretherewasonlya54%responserate.Sincethe2005‐06schoolyear reporting period, centers have provided better data and have had higherresponserateswithrespecttocollectingthisinformation.Responserates for informationonstudents’race/ethnicitywasalsohigh,withthelowest response rate being 93% in the 2007‐08 reporting period. Regularattendancewithin racial/ethnic groups has fluctuated for both African AmericansandCaucasians.HoweverthehighestoverallattendancehasbeenAfricanAmericanparticipants. The number of regular attendees among Hispanics has steadilyincreased over the three‐year period. This increase in the proportion of Hispanicstudentsmay also explain the increase in number of students designated LimitedEnglishProficiency(LEP)overthe2006‐07,2007‐08,and2008‐09schoolyearsaswell.Table 7 shows that, overall,most students in the program are eligible for free orreducedprice lunch (FRPL).Also theproportionof students eligible forFRPLhasremainedstableovertheyears,whiletheproportionofstudentswithspecialneedsordisabilitiesenrolledinthe21stCCLChasdecreasedeachyearsince2006‐07.There are significant variations from year to year in the proportions of studentsscoringateachreadinglevel,withthesmallestvariationatLevelII.WhiletherewasasignificantincreaseintheproportionofregularattendeesscoringatLevelIfrom2006‐07 to 2007‐08, there was a decrease from 2007‐08 to 2008‐09. Level IIproportions remained relatively stable while students at Level III decreased inproportionfrom2006‐07to2007‐08.LevelIVhasthefewestproportionofregularattendees represented. Proportions also declined over the three years, mostlybetweenthefirsttwoschoolyears.In contrast to reading, the scores for mathematics varied less and indicatenoticeable improvements among students. The proportion of regular attendees inLevelsIandIIdecreasedoverthethree‐yearperiod,whiletheproportionsforLevelIII increased over the same time‐frame. There is fluctuation in the proportion ofregularattendeesscoringatLevelIV,withadecreasefrom2006‐07to2007‐08andaslightincreasefrom2007‐08to2008‐09.WhiletheprogramguidelinesdefineeligiblestudentsasthosethatscoreontheEOGat Levels I or II, some students score at these levels in one subject but score at ahigherlevel intheother.GranteesmayalsopetitionforawaivertoallowLevelIIIstudents toenroll in theprogram if their teachers think that theyare tooclose toscoringatLevelII.

17

Table7.CharacteristicsofRegularAttendeesbyAPRSchoolYear

200607 200708 200809

TotalRegularAttendees 12,984 10,751 9,568

PercentMale 52% 50.7% 50.5%PercentFemale 47.6% 49.1% 49.6%PercentSexUnknown .4% .2% 0%CenterResponseRate 97.1% 94% 96.4%

AmericanIndian/AlaskaNative 2.2% 2.1% 2.8%Asian/PacificIslander 0.2% 0.5% 0.7%Black/AfricanAmerican 61.3% 63.2% 57.3%Hispanic/Latino/‐a 11.3% 12.3% 14.7%White/Caucasian 21% 18.2% 21.6%DataMissing 0.7% 1.5% 0.6%CenterResponseRate 97% 93.1% 95.9%

StudentswithLimitedEnglishProficiency(LEP) 8.2% 9.6% 10%StudentsEligibleforFreeorReducedPriceLunch(FRPL) 74.8% 77.5% 74.9%StudentswithSpecialNeedsorDisabilities 13.6% 9.7% 7.9%CenterResponseRate 92.8% 86.7% 91.3%

Reading/LanguageArts LevelI 7.5% 33.4% 24.1%LevelII 28% 32.9% 36.8%LevelIII 54.6% 30.7% 36.9%LevelIV 9.9% 3% 2.2%MathResults LevelI 16.8% 14.5% 11.6%LevelII 40.9% 39.9% 37%LevelIII 38% 42.6% 48.1%LevelIV 4.3% 3% 3.3%CenterResponseRate 90.7% 83.9% 85.6%

Source:LearningPointAssociates.2010.“APRReports:Gender,Race/Ethnicity,SpecialServicesorProgramsClassificationofStudentAttendees,andPercentofRegularAttendeesatEachProficiencyLevel.”21stCCLCProfileandPerformanceInformationCollectionSystem.http://ppics.learningpt.org/ppics/index.asp

18

Figure6 shows theproportionof regularattendees ineach cohortbygrade level.According to APR results, third graders represented the highest proportion ofstudents in the three school years (2006‐07, 2007‐08, and 2008‐09). Grades fourand five show thenexthighest level of representation.After fifth grade there is anoticeabledeclineintheproprotionofregularattendeesintheafterschoolprogram.Some reasons for this decline include the participation of students in sports andother extracurricular activies following sixth grade. Only a smaller percentage of21st CCLC programs offer programming below third grade and for high schoolstudents.Thatproporitonhasincreasedovertimeforhighschoolstudents.

Figure6.ProportionofRegularAttendeesinEachCohortbyGradeLevel,200607,200708,and200809

Insummary,althoughthetotalnumberofregularattendeeshasdecreasedovertheAPR school years 2006‐07 to 2008‐09, centers are serving students and makingsignificant progress. Theproportions ofAfricanAmerican andCaucasian studentshavebeenconstant,whilethepercentagesofHispanicstudentshaveincreased.Centershaveavested interest incapturing improvementsof regualrattendeesonthe EOG tests. Students are reportedly improving in academic performance,particularly inmathematics.Someof theLevel III students in thedatawere likelythe result of program efforts, as mentioned previously. Since PPICS data do notincludedataonindividualsandtheirperformanceatthetimeofenrollmentin21stCCLC program, the extent to which the Level III counts are indications of the

19

program’simpactonstateasssessmentscannotbedetermined.Inthenextsectionspecificoutcomesonstudentperformanceandbehaviorwillbeexamined.SpecificOutcomesforRegularAttendeesImprovingperformanceandschool‐relatedbehavioramongstudentsisthefocusofthe 21st CCLC program. In this section, APR data are examined with respect tospecificoutcomes forattendees.These includechanges in stateassessment levels,gradesandstudentbehaviorasreportedbyteachers.LookingatthereportedchangesinperformancelevelsamongregularattendeesinFigure7,itisobviousthatalargerproportionofstudentsincreasedinperformancelevelsinReading/LanguageArtsandMathematicsovertime.Whilethepercentageof regular attendees that had no change in the level of performance ranged from32% to 44% in Reading and Math, the percentage of students that reportedlydecreased their performance level was significantly less (8% to 27%). Scores forReading/Language Arts in 2006‐07 and scores forMath in 2007‐08were highestwithregardtoperformancelevelforthethree‐yearperiod.

Source:LearningPointAssociates.2009.“APRReports:DisaggregatedCrossYearStateAssessmentResults.”21stCCLCProfileandPerformanceInformationCollectionSystem.http://ppics.learningpt.org/ppics/index.asp.*Thetotalnumberofcentersreportinggradelevelchangedatainthe2006‐07,2007‐08,and2008‐09APR,was204(88%),60(30%)and153(82%)forreading,and201(86%),172(85%)and158(84%)formath,respectively.Figure7.PercentofRegularAttendeesbyChangesinPerformanceLevelsForReading/LanguageArtsandMathby200607,200708,and200809APR*

20

According to Figure 8, at least 50% of regular attendees increased Reading/LanguageArtsandMathgradesbyhalfagradeormore.Thevastmajorityofcentersreportedchangesingrades.Centersthatrespondedtothe2006‐07APRreportedahigher proportion of regular attendees witnessing an increase in bothReading/Language Arts and Math than in 2007‐08 and 2008‐09. The 2008‐09school year APR showed a higher percentage of regular attendees whose gradesremainedthesame.Thelowestresponseratewasduringthe2007‐08APR,at68%(forReading). The data appears to show that the proportion of regular attendeesthathaveshownimprovement ingradesandtheirperformance levelonEOG/EOCtests is significantly higher than the percentage of regular attendees that havedecreased inperformance levelorgrade.However, it isworthnotingthata largerproportionofcentersreporteddataongradechangesrelativetostateassessments.Forthatreasonthedifferencesorsimilaritiesofresultsshouldbeinterpretedwithcaution.Aspartofthe21stCCLCgrantrequirements,FederalandStateteachersurveyswereadministered to capture information about changes in student performance andclassroom behavior. The 21st Century Teacher Survey Guide notes that regularschool‐dayteacherswereaskedtocompletethesurvey“foreveryregularattendeeinthe21stCCLCprogram.”Guidelinesdifferforelementary,middleandhighschoollevelregularattendees.Forelementaryschoolstudents,regularschool‐dayteacherswereaskedtocompletethesurvey.EitherMathorEnglishteacherswereaskedtofilloutthesurveyformiddleandhighschoolstudents.Resultsfromthesurveywereaggregatedforreportingpurposes.

21

Source:LearningPointAssociates.2009.“APRReports:ChangesinGradesAmongRegularAttendeesServedByCenter.”21stCCLCProfileandPerformanceInformationCollectionSystem.http://ppics.learningpt.org/*Thetotalpercentageofcentersreportingperformancelevelchangedatainthe2006‐07APR,2007‐08and2008‐09was91%,68%and88%forreading,and91%,90%,and87%formath,respectively.

Figure8.PercentofRegularAttendeesbyGradeChangesinReading/LanguageArtsandMath,200607,200708,and2009APR*

Figure9showstheresultsfromtheFederalTeacherSurveyonchangesinstudentbehavior. For school years reporting in 2006‐07, 2007‐08, and 2008‐09, regularclassroom teachers indicated that over 80% of 21st CCCLC attendees showedimprovement in academic performance, completion of homework on time, andcompletionofhomeworktosatisfaction.Theseresultsechothepreviousfigures.Withrespecttoclassroomandschoolbehavior,therehasbeenmorefluctuationinpercentagesofregularattendeesshowingimprovement.Teachersreportedthattheproportionofstudentsshowingimprovedbehaviorinclassrose2%between2006‐07 and 2007‐08, but decreased by 6% in 2008‐09. Similarly the proportion ofstudents thatgotalongwithothers rose5%between2006‐07butalsodecreased9% in 2008‐09. The area of least improvement reported by teachers was thepercentageofstudentsthatvolunteeredinclassandattendedclassregularly.

22

Source:LearningPointAssociates.2009.“APRReports:ChangesinStudentBehaviorAmongRegularAttendees (Federal Teacher Survey Results) 2007, 2008 & 2009.http://ppics.learningpt.org/ppics/index.asp. Percentages based on 93%, 89% and 89% teacherresponseratesofcentersreportingin2007,2008and2009,respectively.Figure9.PercentofRegularAttendeeswithSchoolYearImprovementsinBehaviorFederalTeacherSurveyfor200607,200708,and200809APR*

In summary, the outcomes data suggest that regular attendees are continuing toimprovewithrespecttostateassessmentsandgrades.Althoughbehaviormayhavefluctuated over this reporting period, a vast majority of students has shownimprovement in this area. It would require further disaggregation of data todescribehowvarious groups of regular attendees areprogressing, howoutcomesare distributed across various types of organizations, and how differences inprogrammingrelatetostudentoutcomes.Someaspectsof21stCCLCprogramminghave had indisputable short‐term effects on student performance. Centers haveplacedmoreemphasisonstudents’adequateand timelycompletionofhomeworkand teachers in the survey data have noted the positive outcomes of this basicobjective. Homework completion probably affected other areas of studentachievement and classroom behavior as well. However, without suitablecomparisondataovertimeitcannotbeconcludedwithcertaintywhether21stCCLCprogramminghashadmeasurableimpactsonstudentachievementovertime.In the next section, APR data on the progress for Cohorts 3, 4, and 5 will beexamined.

23

III.APRDataonprogressforCohorts3,4,&5ThissectionofthereportexaminesAPRdatatoevaluatetheprogressforCohorts3through 5 across years. Academic results in the subject areas of Mathematics,Reading,andSciencewillbediscussedaswellasprogramattendanceforgranteesinthesecohorts.Cohorts3,4,and5beganinyears2004,2005,and2006,respectively.Thesecohortsaccountfor70grantees(44%ofthetotalgrantees)andanapproximatetotalof$19million in funds dispersed. According to Grantee Profile data, Cohort 3 was thelargestofallgranteecohortsandwasdispersedthemostfunds(approximately$11million). Incontrast,Cohort5had the fewestgrantees (7)and the leastallocation($1.4million).The centers in Cohorts 3 through 5 were overwhelmingly school districts(approximately 74%). Faith and community‐based organizations followed schooldistrictsascenterswithproportionsof10%and7%respectively.SubjectAreasFigure10showsthatCohorts3through5offeredthenine21stCCLCactivitiesandservicesatthemajorityofitscenters.

Figure10.ProportionofCentersbySubjectAreasProvidedforCohort3

24

Forthe2004‐05reportingyear,thebeginningofCohort3’sgrantperiod,thiscohorthad its highest proportion of centers offering all subject areas. Particularly highwere the proportions of centers offering Reading, Math and Arts instructionalactivities. Only 27% of centers offered other instructional activities in 2004‐05.Recall “Other” activities include, but are not limited to, physical fitness andrecreationaltime.During the reporting periods subsequent to 2004‐05 there was a significantdecrease in the proportion of centers offeringReading,Math,Arts, Tech, Cultural,Health,ScienceandBusinessinstructionalactivities.InmostinstancesthedecreaseamongCohort3’s granteeswas20%ormore.However, in the2006‐07 reportingyear,therewasanincreaseintheproportionofcentersofferingthesamesubjects.This was followed by another dramatic decrease in the proportion of centersoffering those subjects. By 2008‐09most Cohort 3 grantees had completed theirgrant.Onlythosethathadreceivedexpansiongrants, i.e.Cohort3&7Eremained.Theproportionsof centersofferingallninesubjectswereat their lowest level forthisCohort.Figure11showstheproportionofcentersbysubjectareaforCohort4.


There was more equal distribution of centers across the subject areas for thesegrantees. For all subjects, with the exception of two (Business and Other), the

25

proportion of centers offering the subjects increased from the first to the secondyear.This increasebetween the first twoyearswas followedbyadecrease in thepercentage of centers offering the subjects. However the decrease was not asprominentaswhatoccurredwithCohort3.Itisinterestingtonotethatin2006‐07,the highest proportion of centers offered Health instruction and activities. ThissubjectledReadingandMathforthatyearasanofferingbycenters.In Figure 12, Cohort 5 shows a significant increase in the proportion of centersoffering Reading, Math, Science and the Arts after 2006‐07. There was a slightdecline in centers beginning in 2008‐09. Business and “Other” accounted for thelowestproportionof centers in2006‐07and2007‐08(0%).However therewasasignificantincreaseinthepercentageofcentersofferingbusinessin2008‐09.Techalso increased among centers in 2008‐09. Conversely, the proportion of centersofferingCultural andHealth subjects remainedat the same levels in2006‐07and2008‐09(approximately78%).


The following set of figures illustrates the average number of hours per weektypicallyprovidedduringtheschoolbysubjectareaforCohorts3,4,and5.Figure13 shows that Reading andMathwere provided, on average, approximately fourhoursperweekoverthegrantperiod.Thistakesintoconsiderationthatmostofthegrantees completed their grantperiod in2008, leavingCohort3&7Egrantees inthe2008‐09reportingperiod.

26

Figure13.Cohort3Through200809APR

Reading and Math were the subjects that centers provided the most each week.Remainingsubjectsaveragedbetween2to3hoursperweek.Therewasvariationintheamountof timespentonsubjectsacrossyears. In2004‐05,Artswas the thirdhighest subject area by hours provided, while in 2005‐06 “Other” was the thirdhighest.In2006‐07and2007‐08,Sciencewasthethirdhighestsubjectareaandin2008‐09, Science andTechwerenearly tied for third. The lowest amount of timeallottedbycentersforasubjectareawasBusinessoverthecourseofthefive‐yearperiod, on average about one hour perweek. Health, Cultural, and Science hoursincreased slightly over the 2006‐07 and 2007‐08 reporting periods. However theamount of hours spent on these activities per week decreased in 2008‐09. Theoverallnumberofcentersdeclinedovertime,particularlyin2008‐09andofthose20, roughly half continued to offer Reading,Math and Science. The 2008‐09 yearalsosawthesharpestdeclineinthenumberofgranteesremaininginCohort3.Among Cohort 4 grantees, Reading and Math were also the subject area centersspent themost time on perweek as shown in Figure 14. “Other” ranked third in2005‐06and2007‐08,whileTechwasthirdin2006‐07and2008‐09.TimespentonScienceinstructionwasrelativelylowinCohort4.Approximately2.16hourswerespent on it over the duration of this cohort. The lowest offering per week wasBusiness.

HoursPerWeek

27


Cohort 5 hadmore variation than preceding cohorts on time spent per week bysubject. Centers spent approximately 5 hours per week on Reading and Math in2006‐07 and 3.22 hours per week in 2007‐08 and 2008‐09. Though hours diddecline following 2006‐07 APR, the succeeding reporting periods show aconsistency in timeprovided forReading andMath. The diagramalso shows thatTech(2006‐07)andScience(2007‐08and2008‐09)trailedReadingandMathwithregardstotimespentperweek.WiththeexceptionofBusiness,granteesreportedadecrease inhoursspentonmostactivities.Notice the line for “Other.”Of theninegrantees thatwere included inCohort5, initially a veryhigh figurewas given fortimespenton thiscategory followedbyasharpdecline.Reasons for thevariationcouldbeunfamiliaritywithPPICS thatresulted indataentryerrorsor inability toproperlyclassifythegivenactivities.Healthdecreasedfrom2006‐07butremainedconstantandactuallyroseslightlybetween2007‐08and2008‐09.

HoursPerWeek

28


AttendanceStatistics(bycenterandgrantee)Granteesincohorts3through5submittedattendancedataonthetotalnumberofstudentsparticipatingintheprogrammingoverthecourseoftheyearandindicatedthenumberof regularattendeesateachcenter.Whereas the totalattendance isameasure of the breadth of a center’s reach, the regular student data helps tomeasurehowsuccessful the centerwas in retaining studentsacross the reportingperiod. The following table (Table 8) presents attendance data for Cohorts 3through5.The total number of regular attendees rose significantly between2005 and2006,withamodestdeclinein2007.Following2008,themajorityofCohort3completeditsgrantperiod.Cohort3,thelargestofthethreecohorts,maintainedanaverageofapproximately 62 regular attendees per center over the duration of its reportingperiod. Cohort 4 appears to have been successful at recruiting and retainingstudentsacrossgrantees.Totalstudentsandregularattendeesrosebetween2006and2007 and againbetween2008 and2009.Therewere, on average, 52 regularstudentattendeespercenteroverthedurationofthegrantperiod.AlthoughmostCohort4granteesendedtheirgrantperiod in2009,someCohort4&7EgranteeswillreportAPRin2010.

HoursPerWeek

29

Table8.AttendanceDataforcohorts3through5

Cohort 3 + 7E 2005 2006 2007 2008 2009

Total Number of Student Attendees 7869 9012 9183 6567 1443

Total Number of Regular Attendees* 4951 6113 5999 4501 948

Average Number of Regular Attendees per Center 52.11 89.23 62.49 56.26 47.40

Average Number of Regular Attendees per Grantee (33) (6) 150.03 185.24 181.79 136.39 158

Percent of Student Attendees Meeting the Definition of Regular Attendee 63% 67.8% 65.3% 68.5% 65.7%

Cohort 4

Total Number of Student Attendees 5996 7973 6435 7429

Total Number of Regular Attendees* 3734 4742 4506 5425

Average Number of Regular Attendees per Center 41.96 53.89 54.29 57.71

Average Number of Regular Attendees per Grantee (29, 27) 128.76 163.52 166.88 200.92

Percent of Student Attendees Meeting the Definition of Regular Attendee 62.3% 59.5% 70% 73%

Cohort 5

Total Number of Student Attendees 927 1141 1222

Total Number of Regular Attendees* 614 712 672

Average Number of Regular Attendees per Center 68.22 79.11 74.67

Average Number of Regular Attendees per Grantee (7, 5) 87.7 101.7 134.4

Percent of Student Attendees Meeting the Definition of Regular Attendee 66.2% 62.4% 55%

*Regular attendees are students who attended the center for 30 days or more during the year. Counts are revised periodically by Learning Point to reflect the latest data entered by grantees. Source: Learning Point. 2009. APR Downloads, “excel_attendence (Year).” http://ppics.learningpt.org/ For Cohort 5 there are currently three years of APRdata available. Thus far, it isevident that Cohort 5 grantees have shown success at recruiting students for the21stCCLCprogram.Thetotalnumberofstudentsroseby214fromyearonetoyeartwoandby82studentsbetweenyeartwoandthree.However,studentretention,asshownbythetotalnumberofregularattendees,appearstohaveonlygrownfrom

30

2007 to 2008. Overall, the average number of regular attendees per center isrelatively high for this cohort at approximately 74 thus far in its period ofperformance.Thefollowingfiguresfurtherillustratethedatashownintheprevioustable.Figures16 and17 show the attendanceof total students and regular attendeesby cohortacrossyearsforcohorts3through5.Cohort3hadthehighestproportionoftotalstudentsoveritsperiodofperformance.Cohort4hadasignicantincreaseintotalstudentattendeesfrom2006to2007andagainin2009.Bothcohorts3and4werenearlyeveninproportionoftotalstudentsin2008.

Figure16.TotalStudentsbyCohortAcrossYears

Figure 17 shows the success of Cohort 4 in recruitment and retention of regularattendeesacrossgrantees.Between2006and2007therewasasignicantincreaseinthe proportion of regular attendees. Cohort 4 also witnessed an increase in theproportion of regular attendees between 2008 and 2009. Cohort 5, the smallestcohort,showsconsistencyinitsproportionofregularattendees.

31

Figure17.RegularAttendeesbyCohortAcrossYearsofGrant

CostPerStudentThenext table (Table9) shows the costper studentby cohort andAPRyear.Thecostperstudentdecreasedovertime,withrareexception(e.g.year2007‐2008forCohort4).Onereasonfortheincreaseinfundsisthatsomegranteesreportedfewertotalstudentattendeesfromoneyeartothenext.Comparingthefirstthreeyearsofoperationofeachcohort,weseethatCohort3,onaverageforthefirstthreeyearsofits grant, spent $1,150 per student, based on total attendees. Cohort 4 spent, onaverage, $1,040 per student and Cohort 5 spent $1,164 on average per studentbased on total attendees. Asmentioned previously, Cohort 5 will submit its finalAPRin2010.Withregardtocostperregularstudentattendee,Cohort3spentonaverage$1,590per stduent over the period 2005 to 2008. This was the four‐year performanceperiodformostCohort3grantees.Incomparisontothefour‐yearperiodforCohort4,being2006to2009,granteesinthatcohortspent$1,471onaverageperregularattendee. Over its performance period thus far, Cohort 5 has spent $1,884 onaverageperregularstudentattendee.Cohort5isthesmallestofthethreecohortsand has another year of annual performance to report to the state and Federalgovernment.

32

Table9.CostPerStudentbyCohortandAPRYear

Cohort 3 + 7B 2005 2006 2007 2008 2009

Cost Per Student Attendees (based on total attendees) $1,363.52 $1,181.82 $906.94 $729.40 $399.97

Cost Per Student Attendees (based on regular attendee only) $2,167.14 $1,742.27 $1,388.31 $1,064.20 $608.81

Cohort 4

Cost Per Student Attendees (based on total attendees) $1,272.59 $905.67 $955.58 $697.57

Cost Per Student Attendees (based on regular attendee only) $2,043.50 $1,522.76 $1,364.66 $955.25

Cohort 5

Cost Per Student Attendees (based on total attendees) $1,443.31 $1,172.97 $876.18

Cost Per Student Attendees (regular attendee only) $2,179.07 $1,879.72 $1,593.29

RegularAttendeesTable 10 shows the characteristics of regular attendees by cohort/APR year. It isuseful to lookat theparticipationofstudentsofdifferentbackgroundstoexaminethereachofthe21stCCLCprogram.Inadditiontoethnicbackground,participationby students that receive special services or programs indicates the extent whichprogramsareaddressingotherhighneedsconstituentsinthecommunity.Table11alsoshowstheproportionofregularattendeesbytheirlevelofproficiencyincoreacademic areas. This information helps indicate the type of academic assistancecentersshouldprovidetheirstudentpopulationaswellasservesasabaselinefordeterminingtheprogram’simpactintheyearstocome.Cohorts3and4showafairlyevendistributionofmaletofemaleregularattendees,withslightlymoremalethanfemale.TheproportionoffemaleregularattendeesdidincreaseoverthereportingperiodofCohort3withitshighestfigure(53%)in2009.ThegenderdifferencebecomesmoreobviousinCohort5.Toreiterate,thefifthyearofperformanceforCohort3consistsofCohort3&7Egrantees.Anothernoteisthehighpercentageof students thatweremarked “sexunknown” in2005 (16%)and2006 (9%) for Cohort 3. However the percentage of “unknown” was reduced insubsequentyears.

33

Table10.CharacteristicsofRegularAttendeesbyCohortandAPRYear 2005 2006 2007 2008 2009

TotalRegularAttendees

Cohort3 4951 6113 5999 4501 948

Cohort4 X 3734 4742 4506 5425

Cohort5 X X 614 712 672

PercentMale

Cohort3 41.5% 48.3% 51% 51.2% 47.2%Cohort4 X 52.5% 47.4% 49.3% 51%Cohort5 X X 54.1% 53.1% 56.4%

PercentFemale

Cohort3 42.4% 43.1% 47.1% 48.3% 52.8%

Cohort4 X 47.3% 46.9% 49.5% 49.1%

Cohort5 X X 44.5% 46.9% 43.6%

PercentSexUnknown

Cohort3 16.1% 8.6% 1.9% .5% 0%

Cohort4 X .2% 5.7% .2% 0%

Cohort5 X X 1.4% 0% 0%

AmericanIndian/AlaskaNative

Cohort3 1.7% 1.0% 1.9% 2% 0%

Cohort4 X 2.6% 1.9% 2.8% 4.2%

Cohort5 X X 0% 0% 0%

Asian/PacificIslander

Cohort3 0.5% .8% 0% .1% 1.1%

Cohort4 X .2% .2% .5% .7%

Cohort5 X X 1% 3.7% 1.5%

Black/AfricanAmerican

Cohort3 61% 60.9% 64.2% 61.8% 47.6%

Cohort4 X 64.2% 60.6% 65.1% 55.9%

Cohort5 X X 67.6% 50.6% 47.9%

34

Table10.CharacteristicsofRegularAttendeesbyCohortandAPRYear(continued)

2005 2006 2007 2008 2009

Hispanic/Latino/‐a

Cohort3 9.1% 12.7% 12.4% 13.8% 26.4%

Cohort4 X 8.4% 8.4% 10.2% 13%

Cohort5 X X 11.9% 20.5% 26.3%

White/Caucasian

Cohort3 14.8% 19.2% 20.4% 17.9% 23.8%

Cohort4 X 22.3% 20.2% 17.7% 23.7%

Cohort5 X X 18.9% 25.8% 23.4%

DataMissing

Cohort3 12.9% 5.5% 1.2% 4.4% 1.2%

Cohort4 X 2.4% 8.7% 3.6% 2.6%

Cohort5 X X .7% 0% .8%

Students with Limited English Proficiency (LEP)

Cohort3 8.8% 8% 8.1% 9.8% 12.9%

Cohort4 X 6.1% 7.1% 7.3% 9.6%

Cohort5 X X 6.7% 13.8% 19.8%Students Eligible for Free or Reduced Price Lunch (FRPL)

Cohort3 74.8% 64.2% 79.1% 84% 72.6%

Cohort4 X 72.3% 72.9% 76.3% 76.6%

Cohort5 X X 65.8% 58.6% 56.4%StudentswithSpecialNeedsorDisabilities

Cohort3 9.1% 10.4% 11.4% 8.8% 9.5%

Cohort4 X 16.6% 14.3% 10.7% 8.2%

Cohort5 X X 7.3% 12.4% 10.3%

CenterResponseRates

Cohort3 73.1% 73.8% 97.3% 61.5% 100%

Cohort4 X 67.7% 67.7% 63.8% 72.3%

Cohort5 X X 69.2% 69.2% 69.2%

Source:Exportdata‘excelattendance(year)’

35

Table11.TheProportionofStudentsinEachProficiencyLevelforReadingandMathbyCohortandAPRYear

Cohort 3 2005 2006 2007 2008 2009

Reading/ Language Arts

LevelI 13% 9% 11% 3% 1%

LevelII 34% 47% 51% 44% 30%

LevelIII 41% 35% 30% 21% 37%

LevelIV 12% 10% 7% 32% 32%

MathResults

LevelI 9% 1% 4% 2% 6%

LevelII 33% 17% 31% 40% 53%

LevelIII 45% 53% 46% 42% 33%

LevelIV 13% 29% 19% 16% 9%CenterResponseRate 91.7% 87% 73% 63% 84%

Cohort4 2005 2006 2007 2008 2009

Reading/LanguageArts

LevelI 9% 10% 4% 3%

LevelII 51% 55% 24% 44%

LevelIII 33% 27% 36% 35%

LevelIV 7% 8% 37% 18%

MathResults

LevelI 5% 5% 4% 3%

LevelII 39% 42% 47% 50%

LevelIII 40% 38% 37% 37%

LevelIV 17% 15% 12% 10%CenterResponseRate 69% 68% 64% 72%

Cohort5 2005 2006 2007 2008 2009

Reading/LanguageArts

LevelI 5% 4% 3%

LevelII 57% 23% 25%

LevelIII 29% 37% 38%

LevelIV 8% 35% 33%

MathResults

LevelI 1% 1% 3%

LevelII 42% 35% 42%

LevelIII 40% 43% 33%

LevelIV 17% 21% 22%

CenterResponseRate 88% 70% 70%

Source: Learning Point. 2009. APR Downloads, “excel_aprsaii (Year) and “excel_aprlevels(Year).”

36

Table 10 also shows that Cohort 4 had the highest percentage of AmericanIndian/Alaska Native students, compared with Cohort 5. However, Cohort 5 didhave the highest percentage of Asian/Pacific Islander attendees (approximately2%). African American and Caucasian students make up the bulk of regularattendees for the three cohorts and figures across cohorts appear consistent forthesegroups.Hispanic/Latinoattendeesincreasedinpercentageovertime.Cohort3&7EandCohort5hadthehighestproportionatapproximately26%.The proportion of students with Limited English Proficiency (LEP) has alsoincreased across the three cohorts over time.Asmentioned earlier in this report,this may be attributed to the increased proportion of Hispanic/Latino attendees.Thetable indicatesthatthemajorityofparticipants inthe21stCCLCforCohorts3through5areeligibleforFRPL.ThelargestpercentageofstudentseligibleforFRPLwas84%inCohort3(2008)andthelowestthusfarwas56%forCohort5in2009.TheproportionofstudentswithspecialneedsordisabilitieswashighestinCohort4at nearly 17%. In all cohorts there have been fluctuations in the proportion ofspecialneedsstudents,withadecreasingtrendoverall.Table11showsthat,overall,moststudentsintheprogramareperformingatLevelsII and III in Reading andMathematics. This is true across the three cohorts. Theproportion of youth at Levels III and IV has generally increased in Reading anddeclined in Math with the exception of Cohort 5. Fluctuating response rates arelikely to explain some the variation over time and across cohorts. Cohort 4 hasmaintainedthelowestresponseratesofthethreecohortsintheanalysis.Figures18showstheproportionofregularattendeesinCohort3,4and5bygradelevel.InCohort3,atthestartofitsperformanceperiod(2005),grades3through5hadthehighestpercentageofregularattendees.In2009grade6alsohadahigherpercentageofstudents. In fact,2009was theyearwith thehighestpercentagesofstudentsingrades3through6andgrade8.Cohort 4 had an even distribution across years. Barswithin each grade level aremorelevelthaninCohorts3andCohort5.Thehighestproportionofstudentswasthirdgradein2008(over20%).ProgrammingattheK‐2ndgradelevelisapparentaswell as successful efforts at recruiting and retaining high school students. Noticethat2009hadthehighestproportionsofregularattendeesingrades9through12.Cohort5had itshighestpercentageof regularattendees ingradesK‐5 in the firstyearof itsperformance.ThisCohortprovedespeciallysuccessfulatrecruitingandretainingstudents inK‐2ndgrades.Thehighestrepresentationofstudents in2007wasinthe4thgrade(over16%),howeverthatpercentagedroppedsharplyin2008.High school studentswere retained in2008and2009.This is not typical ofmost21stCCLCprogramsandmayindicatethatCohort5granteeswillcontinuetofocusongrades9‐12initsprogramming.

37

Figure18.Cohort3,4and5ProportionofRegularAttendeesbyGradeLevel

Cohort3

Cohort4

Cohort5

38

SpecificoutcomesforregularattendeesAsmentionedpreviously, the progress of regular attendees is a good indicator ofhow well 21st CCLC programs are performing towards meeting their goals andobjectives. In this section specific outcomes for regular grantees in Cohorts 3through5areexamined.Thefollowingfigures19through21showtheproportionof regular student attendees by changes in performance levels and grades inReading/LanguageArtsandMath.Figure19showsthepercentofregularattendeesbychangesinperformancelevelsforreading/languageartsandmathforCohort3.

Source:LearningPointAssociates.2009.“excel_aprgrades(Year)”.http://ppics.learningpt.org/ppics/index.asp.*Responserateforgradelevelchangedatainthe2004‐05,2005‐06,2006‐07,2007‐08,and2008‐09APR,was66%,89%,77%,49%,and85%forreading,and66%,73%,77%,72%,and85%formath,respectively.

Figure19.PercentofRegularAttendeesbyGradeChangesinReading/LanguageArtsandMath,Cohort3

Figure 19 does not show much overall difference between those students thatincreased by half a grade or more and those that remained the same in eithersubject. Students tended to increasebyhalf a gradeormoreupuntil 2009.Years2005 through 2008 had the highest proportion of increase in reading and mathwhile2009hadthelowestproportionofstudentsthatincreasedbyhalfagradeormore.Likewise, in2009therewereahigherpercentageofstudentswhosegrades

39

remained the same in both reading andmath. Increasing proportions of studentsalsodecreasedbyhalfagradeormoreovertimeinbothsubjectareas.

Source:LearningPointAssociates.2009.“excel_aprgrades(Year)”.http://ppics.learningpt.org/ppics/index.asp.*Responserateforgradelevelchangedatainthe2005‐06,2006‐07,2007‐08,and2008‐09APR,was79%,81%,65%,and73%forreading,and70%,80%,80%,73%formath,respectively.


In Cohort 4 (Figure 20) there is a considerable difference in the proportion ofregularattendeesthatincreasedtheirgradesbyhalfagradeormore,proportionsofstudents whose grades remained the same, and those students whose gradesdecreasedbyhalfagradeormore.Infact,in2006and2007,nearlytwiceasmanystudentsincreasedtheirgradesbyhalfagradeormorethanthosestudentswhosegradesremainedthesame.Insomecases,theproportionofstudentsthatincreasedtheir grades is three times as great as those that decreased grades. Also, anincreasingproportionofstudentsremainedatthesamegradelevelinbothsubjectsovertime.Figure21showsthesameinformationforCohort5.Theproportionofstudentsthatincreasedtheirgradesbyhalfagradeormoredeclinedovertimeandthosewhosegrades remained the same increased. Years 2007 and 2008 had the highestpercentagesofregularattendeesthatincreasedtheirgradesinReadingandMath.In2009 a higher proportion of students’ grades remained the same. Although theproportion of students whose grades decreased rose slightly between 2007 and2008,therewasasignificantdecline(10%)inthatnumberthefollowingyear.The

40

decline is probably the result of the high number of students whose gradesremained the same rather than these students increasing their grades by half agradeormore.

Source:LearningPointAssociates.2009.“excel_aprgrades(Year)”.http://ppics.learningpt.org/ppics/index.asp.*Responserateforgradelevelchangedatain2006‐07,2007‐08,and2008‐09APR,was79%,71%,and69%forreading,and75%,75%,and69%formath,respectively.


The final figures (Figures22 through24)of this reportdepict changes in studentbehaviorforCohorts3through5duringthecourseoftheirperformanceperiod.ThedatafromtheFederalteachersurveyshowsthatchangesinstudentbehaviorwereconsistent for Cohort 3 during 2005 and 2006 (Figure 22). Student behaviorincreasedinmostareasovertimeanddeclinedslightlyin2009,particularlyinareasofclassattendanceandbehavinginclass.

41

Figure22.ChangesinStudentBehavior,Cohort3

Figure 23 suggests that a high proportion of students turned in homework andcompletedittoteachers’satisfaction,behavedinclassandweremotivatedtolearn.In2007therewasadecreaseinthepercentageofregularattendeesthatexhibitedpositive behavior towards academic performance, but this went up the next twoyears(nearly100%).Thelowestareaforbehaviorofregularattendeeswasregularclassattendance(lessthan70%),withtheexceptionofyear2007,inwhichteachersrespondedthatover80%ofstudentsattendedtheirclassesregularly.


42


Cohort 5, as shown above, had mixed results across years (Figure 24). Teachersnotedthathighpercentages(80%andabove)completedhomework,participatedinclass and performed well academically. Class attendance, behavior in class andgettingalongwithotherswitnesseddeclines.IV.CONCLUSIONIn this report findingswere discussed fromNorth Carolina’s 21st CCLCprograms.GranteeProfiledataforallactivecohortsfortheschoolyears2006‐07to2008‐09wereusedalongwithAPRdataforthoseschoolyears.TheactivecohortsincludedCohorts5through8. Inadditiontoananalysisof thecurrentgrantees, thisreportalso discussed findings on the progress of Cohorts 3 through 5 over each year ofperformance. Grantee Profile and APR data were compared to get a sense ofachievementwithrespecttoprojectedoutcomes.OverallthefindingspresentedinthisreportsuggestthatNorthCarolina’s21stCCLCprogramsareservingtheirtargetpopulation.Althoughthetotalnumberofstudent

43

attendees fell between the 2006‐07 and 2008‐09 school years, the number ofregularstudentattendeesaveraged11,000over thesameperiod.Theproportionsof students enrolled in special services or programs represented in the 21st CCLChave increased significantly during the three‐year timeframe. With regard toethnicity, the proportion of Caucasian students has remained stable since 2005,while African American and Latino participants have increased steadily over fiveyears.The 21st CCLC program encourages partnerships between grantees and diversecommunitystakeholdersasastrategyforeffectiveandsustainableprograms.Inthestate, there has been a constant increase in the number and percent of granteesdesignatedasfaith‐basedandcommunity‐basedorganizations.Thesegranteeshavesurpassed school districts as the top two leading grantee organization types.Conversely,asan indicationofcommunitypartnerships,schooldistrictsstillmakeupthebulkofactualcentersitesforfaith‐basedandcommunity‐basedprograms.These findings amount to an observed change among grantees and studentattendees. Findings show, that North Carolina 21st CCLC sites are serving theirtargetedpopulation,namelystudentsthathavescoredeitherLevelIorLevelIIontheirend‐of‐gradetests.AlthoughcentershaveanincreasinglyhigherlevelsofLevelII students in Reading/Language Arts, the number of Level I and Level II mathstudentsisdecreasing.ThisdecreaseinLevelIandLevelIImathstudentsbecomesapparent when one sees that regular attendees are improving their academicperformance inMathematicsonend‐of‐gradeandend‐of‐course tests,particularlyatLevelIII.ThisreportalsoshowstheprogressthathasbeenmadefromCohorts3through5.Cohort3includedgranteesthatreceivedexpansiongrantstooperateanadditionalyear. The additional year of funding proved beneficial for Cohort 3 in terms ofregular student attendance at most grade levels, particularly the elementary andmiddleschoollevels.Therewasimprovementinthepercentageofregularattendeesbygrade changes inReading/LanguageArts andMath fromCohort3 toCohort4.Cohort4wasalsotorecruitandretainahighproportionofregularattendeesacrossallgradelevelsoverthecourseofitsperformance.Cohort4&7Eexpandssomeofthiscohort’sgranteesforanotheryearuntil2010.Cohort 5 is the most recent of the three Cohorts and has the fewest grantees.Althoughsignificantprogresshasnotbeenmadeyetwithregardtoregularstudentattendance and improvements in grades, teachers (survey results) note animprovement of the proportion of students completing homework to theirsatisfaction,participationinclass,andoverallacademicperformance.Cohort5hasalsorecruitedandretainedthehighestnumberofhighschoolstudentsofthethreecohorts. This indicates that grantees in this cohort have succeeded wherecounterpartsinothercohortshavenotbyretainingolderstudentpopulations.

44

Although across Cohorts 3 through 7 regular attendees are improving theiracademicperformance in certainareas, thereareunexploredquestions that thesedataareunable to fullyaddress,suchas thedifference inmotivationandparentalsupportbetweentheregularattendeesandthosethatdidnotenrollafterreceivingarecommendationtodosoorenrolledbutdidnotpersist.There are limitations in theway the data are collected and the quality of recordkeeping and reporting done by centers. Due to budget constraints, DPI did notconductregularmonitoringsitevisitstograntees.Withoutregularmonitoringitisdifficult toverifyanddetermine thequalityof reporting.Also, it isnotpossible toaddress some questions because there is no data from the APR for individualstudents,noristheredatadescribingindividualperformancebeforestudentsenrollinthe21stCCLCprogram.Inthepast,turnoveramongprojectdirectorshasnotonlybeenabarriertosustainabilitybutitalsopresentedchallengestodatacollection.North CarolinaDPI allowedUNC‐Pembroke to collect individual‐level data from asample of centers to study outcomes for attendees before, during and after theirparticipation in the program. Data was collected for school year 2005‐06. NorthCarolina might consider, as a recommendation, the continued collection ofindividual‐leveldata.Thesedatacouldbecombinedwithdataonasampleofsimilarnon‐attendees as the basis for a comparison, examining program effectiveness inmore detail (i.e. relationships between student achievement and programming,servicesoffered,typesoforganizations,demographiccharacteristics,fundinglevels,etc).Another step to improving thestrengthof theprogram is forgrantees to take fulladvantage of the possibilities presented by the current database. Learning Pointproduces amultitude of reports that summarize performance data for individualgrantees. In the past, grantees indicated that thesematerialswere useful in theirplanning and implementation. Continuing to enhance technical assistance togranteeswill also improve data collection, the quality of programming, and grantcompliance. Findings from the data show that centers are operating in greatercompliance with the 21st CCLC guidelines, and that is a tribute to the technicalassistance they have received by DPI and Learning Point Associates. Technicalassistancewill continue to play a definitive role in addressing various issues andbringingmorecentersuptostandardsduringthecourseoftheirfour‐yearfunding.

Documents

21st Century Community Learning Centers 2008-09 APR and