What Can My Data Tell Me? Developing Critical Questions To ... · 3 Intended Outcomes Understand...

The CenterforIDEAEarlyChildhoodDataSystems

What Can My Data Tell Me? Developing Critical Questions To Guide Early Childhood Data Analysis and Program Improvement

Gary Harmon, Ph.D.DaSy

PA Data Literacy AcademyApril 4-5, 2017

2

Session Agenda

OverviewofSessionIntendedOutcomesDaSyCriticalQuestionsActivityAnalysisPlansVisualizationsPuttingitAllTogether

3

Intended Outcomes

UnderstandtheDaSyCriticalQuestionsDiscusshowtheDaSyCriticalQuestions canbeusedtohelpPartCandPartB/619programsIntroduceandlearnabouttheDaSyCriticalQuestionsanalysisplansDevelopalistofcriticalquestionsthatareimportanttobringbacktoleadership,co-workers,partnersandstakeholders

RoleDataexperience/comfortabilityEverusedDaSyCQ?

Poll Questions

4

5

Why Do I Need Data?

– Federalandstatereporting

– Highlightgapsandbarriers– Planforandprovideevidenceofprogramimprovement

– Receiveadditionalfunding– Identifyprocesstooutcomerelationships– Improveprogramsandpolicies– Improveoutcomesforchildrenandfamilies

6

DEVELOP CRITICAL QUESTIONS!

7

Questionsthat,whenanswered,provideinformationinareasdeemedmostimportant.Questionsaqualitystatedatasystemshouldprovidethedatatoanswer.Provideinformation,whichwhenanswered,helptoimproveresultsforchildrenandfamilies.

What are Critical Questions?

8

DevelopCritical

Questions

AnalyzeData

UseInformation

Critical Questions Process

9

DevelopCriticalQuestions

AnalyzeData UseInformation

Today’s Focus

10

Createdin2015CollaborationServesastartingpointforfurtherdiscussions

About the DaSy Critical Questions

11

DaSy Critical Questions

DaSy Critical Questions

DaSy Critical Questions

DaSyCriticalQuestions:3sectionsofquestions– Child- andfamily-level– Practitioner-level– LocalEarlyInterventionServices(EIS)program/Localeducationagency(LEA)level

Broad&specificexamplequestionsEssential&aspirationalquestions

DaSy Critical Questions

14

Essential(BreadandButter)Questions

Aspirational(MountainClimber)Questions

Critical Questions You May Have

Amongallyoungchildrenfrombirthtoage5receivingearlychildhoodservices,whatpercentagereceivesmorethanonetypeofearlychildhoodservice?Howmanylow-incomechildrenareenrolledinpublicpreschoolprograms?Whatarethedistrictandschoolenrollmenttrendsforspecificprogramscomparedtotheoverallstatetrends?

16

Critical Questions You May Have…TowhatextentarechildrenwithIndividualFamilyServicePlans/IndividualizedEducationPlans(IFSP/IEPs)attendinggeneralearlycareandeducationprograms?1.B.1– WhatpercentofchildrenenrolledinEI/ECSEareservedingeneral

earlycareandeducationprograms*(e.g.,EarlyHeadStart,HeadStart,homevisiting,childcare,statePre-K)?Howdoesthiscomparetotheoverallpopulationofchildrenbirth-5inthestate?

17

Critical Questions You May Have…

Whatarethecharacteristics ofthepractitioners workingwithchildrenwithIFSP/IEPsingeneralearlycareandeducation?2.A.1– Whatarethelevelsofeducationandyearsofexperienceofthe

generalearlycareandeducationpractitionersworkingwithchildrenwithIFSP/IEPs?

Whatisthecost ofprovidingearlyintervention/earlychildhoodspecialeducationservicestochildrenandfamilies?3.C.1– WhatistheaveragecostperchildforEI/ECSEservicesstatewide?For

eachEISprogram/LEA?

18

Critical Questions You May Have…Whatarethelong-termoutcomesofchildrenwhoparticipateinearlyintervention/earlychildhoodspecialeducation(EI/ECSE)?1.C.3– WhatpercentofchildrenwhoexitEI/ECSEatageexpectationsdonot

haveanIEPinK-12(e.g.,inkindergarten,inthirdgrade,ingrade12,atgraduation,ever)?

– WhatpercentageofchildrenwithIFSP/IEPsarespendingtimeinhigh-qualityearlycareandeducationsettings?

Whoisyouraudience?Whatdotheywanttoknow?Whydotheywanttoknowit?/Howwilltheyuseit?

Where to start?

19

StatelegislatureStateleadership– Departmentlevel– Programlevel

LocalprogramsICC/LICCParentsResearchersOthers(concernedcitizens,advocacygroups,etc.)

Who is your audience?

20

CriticalQuestions– Don’tneedtobeprogramexperts

What do they want to know?

21

Allocationoffunds/resourcesProgramImprovement(gaps,barriers,successes)State/FederalReporting(ex.SSIP)Grantapplications“Alwayswantedtoknow…”

Why do they want to know it?/How will they use it?

22

WhocanparticipateingeneratingCriticalQuestions?

Don’t Go It Alone

23

WhocanparticipateingeneratingCriticalQuestions?

Don’t Go It Alone

24

ACTIVITY TIME

25

Ateachtable,reviewthesubsetofcriticalquestions.– Focusonthebroadquestionsinthegreenboxes(theexamplequestions

helpdefinethescopeofthebroadquestions).Identifythetop2-3broadquestions(thoseinthegreenboxes)thatareapriorityforyourrole/program.Writethequestion“code”(e.g.,1.A.1)ontheprovidedstickynotes.– Identifywhichofthesequestionsyourstate/programcananswerwith

yourdatasystem(s)andwhichquestionscurrentlycannotbeanswered.– Arethereotherquestionsnotlistedthatyouwanttoknowtheanswer

to?Designateonetablerepresentativewhowillsharebackyourtop2-3questions.Asabonus,letusknowifyou’veidentifiedwhetherthedatacurrentlyexists,andifnothowyoucouldgetit.

Critical Questions Activity

26

In2– 3minutes,haveyourdesignatedtablerepresentative:– Sharewiththelargegroupthetop2-3broadquestionsthatareprioritiesforthefolksatyourtable.(providingstickynoteforwallgraph)

– Sharewhethereachquestioncancurrentlybeansweredbyyourstate orlocaldatasystem(s).

Table Share-Back

27

DevelopCriticalQuestions

AnalyzeData UseInformation

Today’s Focus

28

Howmanydatastaffintheroom?

Prepare and Analyze Data

29

HowdoIgofromquestiontoanalysis?– Includeyourdataanalysispartner(s)!!!

Prepare and Analyze Data

30

FindyourdatastaffIncludetheminCQdevelopment– Datayouhave– Datayouneed– Whatanswersyouarelookingfordrivetheanalysis

AnalyzeandReviewRawData– *SNEAKPEEK*DaSyCQAnalysisPlans

Prepare and Analyze Data

31

IndevelopmentforQuestion1.B(expectedreleaseSummer2017)Step-by-stepprocess– Preparingthedataset– Dataelementsneeded– Settingupandconductinganalysis– DataTablesandVisualizations

DaSy CQ Analysis Plans

32

PLACEHOLDERFORREPORTTEMPLATE

DaSy CQ Analysis Plans

33

DevelopedyourquestionsPreparedandanalyzedthedata

NOWWHAT????

Moving Along CQ Process…

34

35

Create Reports to Visually Share Data

Whoisyouraudience?Howwilltheyusethedata?– Inareport?Towhom?

Displaying Information/Data from CQ

36

37

Need Data Display Templates & Ideas?

UsetheDaSy-NCSIDataVisualizationToolkit

INSERT STATE EXAMPLES OF DATA VISUALIZATIONS

38

Beyondjuststandarddatareporting(APR,StateReports)CanbesimpletoverycomplexWhatdatadoIhave?WhatdatadoIneed?– Doesitexist?

• Ifyes,howdoIgetaccesstoit?• Ifno,howdoIgatherit?

CQ Points to Remember

39

40

How Critical Questions Can Guide Your Work

Criticalquestionscan– Beatooltoplanenhancementstoadatasystem– Helporganizeanalysestoinformprogramoperations,programimprovement,andpolicyandaccountability

41

Using the CQs to inform program operations, program improvement, and policy

Identifythequestionsinthedocument(oryourown)thatareapriorityforthestate/locality/stakeholders,etc.Identifywhichofthesequestionsyourstatecananswerwithyourdatasystemandwhichquestionscurrentlycannotbeanswered.Developananalysisplantoanswerthepriorityquestionsthatcanbeansweredwithyourcurrentdatasystem.Analyzethedata.Applytheresultsoftheanalysistoansweryourpriorityquestions– andimproveservicesforchildrenandfamilies.

42

Final presentation slide

VisittheDaSy websiteat:http://dasycenter.org/LikeusonFacebook:https://www.facebook.com/dasycenterFollowusonTwitter:@DaSyCenter

43

ThecontentsofthispresentationweredevelopedunderagrantfromtheU.S.DepartmentofEducation,# H373Z120002.However,thosecontentsdonotnecessarilyrepresentthepolicyoftheU.S.DepartmentofEducation,andyoushouldnotassumeendorsementbytheFederalGovernment.ProjectOfficers,MeredithMiceli andRichelle Davis.