Enhancing evidence-informed policy making

OFFICEOFTHEPRIMEMINISTER’SCHIEFSCIENCEADVISORProfessorSirPeterGluckman,KNZMFRSNZFMedSciFRS

ChiefScienceAdvisor

Enhancingevidence-informedpolicymakingAreportbythePrimeMinister’sChiefScienceAdvisor

July2017OfficeofthePrimeMinister’sChiefScienceAdvisorPOBox108-117,SymondsStreet,Auckland1150,NewZealandTelephone:+6499236318Website:www.pmcsa.org.nzEmail:[email protected]

Enhancingevidence-informedpolicymaking

2

OFFICEOFTHEPRIMEMINISTER’SCHIEFSCIENCEADVISOR

ProfessorSirPeterGluckman,KNZMFRSNZFMedSciFRS

ChiefScienceAdvisor

July5th2017ThePrimeMinisterRtHonBillEnglishWellingtonDearPrimeMinisterRe:EvidenceintheformulationandevaluationofpolicySince my last report1on this topic published in 2013, there has been an encouraging number ofdevelopmentsoccurringattheinterfaceofscience(broadlydefinedasincludingboththenaturalandsocialsciences)andpublicpolicyinNewZealand.Mostnotably,thesedevelopmentsinclude:theappointmentoften departmental science advisors; the establishment of a Committee of Science Advisors (CoSA); theexploration of how to better use data and science to inform public policy development especially in thesocial sector; andagreaterengagementof the scienceadvisory system indisaster risk reductionand riskmanagement. Internationally there is a growing appreciation of the importance of an effective scienceadvisoryecosystemandconsiderableinterestintheemergentNewZealandmodel.Thisisinpartpromptedby the formation of the International Network of Governmental Science Advice (INGSA) following aninternationalmeetingontheissuesupportedbytheNewZealandGovernmentin2014.2Atthesametimegloballytherehasbeenincreasingconcernaboutriskstotheeffectiveinterfacebetweenscienceandpublicpolicy.Forinstance,recentlywehaveseentheriseofpost-trustandpost-expertrhetoricelsewhereinfluencingpublicpolicydecisions,asituationwhichNewZealandhasfortunatelytodatelargelybeenspared,butwecannotbecomplacent.Thereremainpotentialissuesattheinterfacebetweenscienceand policy, and between science and society that largely relate to matters of effective engagement,transparencyandaccessibilityofexpertise.Theseissuesmeritongoingconsideration.Theinterfacebetweenscientificevidenceandpublicpolicyiscomplex.Muchofthiscomplexityisaresultofthe inherently different epistemic underpinnings and processes of these two domains. In general policymakersmustdealwithchoosingbetweenoptionsbasedonconsiderationsthatextendwellbeyondsimplythe scientific evidence.What’smore, the science pertaining to the very issues forwhich decision-makersneed itmost is generally incomplete and sometime ambiguous. This challenge shouldnot undermine theimportant role of scientifically derived evidence in informing policy options – but it does have importantimplicationsforhowtheinterfacebetweenscienceandpublicpolicyshouldbestbestructured.

1Gluckman,P.2013.TheRoleofEvidenceinPolicyFormationandImplementation.OfficeofthePrimeMinister’sChiefScienceAdvisor(http://www.pmcsa.org.nz/wp-content/uploads/The-role-of-evidence-in-policy-formation-and-implementation-report.pdf)2http://www.ingsa.org/


3

Following discussionswith both the former PrimeMinister, the Rt Hon John Key, andmore latterlywithyourself, it was agreed that it would be appropriate for me to again review the state of New Zealand’sscienceadvisoryecosystemandtoidentifywhereprogressiveimprovementsstillcouldbemade.Aspartofthepreparationforthisreport,overthepastyearmyOfficehasconductedextensiveinterviewswithbothprovidersofscientificadviceand,importantly,withMinistersandofficialswhoaretherecipientsofthatadvice.Thatfeedbacktogetherwithagrowingbodyofknowledgefromtheinternationalcommunityofscienceadvicepractitionersandtheorists,includingviaINGSA,informsthisthirdreport.Aswithmypreviousreports,acentralmessageisthatevidencealonerarelyisthebasisofpolicyformation.Itmustalwaysbeacknowledgedthattherearemanyotherconsiderationsandfactorsatplaywithinpolicydecisionmaking.Butitisimportantthatanypublicpolicydecisionisinformedbywhatweknowanddonotknowfromrobustevidentialapproaches.Regardless,theunderlyingpremisemustbethatbetterdecisionsaremorelikelytobemadewhentheyareinformedbyevidence.Theworrisomeriseof‘post-truth’polemicandthegreaterandeasierpromulgationof‘falsenews’thatwehave seen globally in recent times canbe seen as threats to thedemocratic process, social cohesion andgoodgovernance.Ibelievethatacommitmenttoprotectandenhancetheevidentialinputintothepolicyprocessisanincreasinglyimportantdefenceagainstthesetrends.ItisinthisspiritthatIcommendthisreportforyourconsideration.Yourssincerely,

ProfessorSirPeterGluckman,ONZKNZMFRSNZFMedSciFRSChiefScienceAdvisortothePrimeMinisterofNewZealand


4

AcknowledgementsIthankDrLouisePinfoldforherworkresearchingthisreportandpreparingearlydraftsofmaterialfromwhichitdraws.DrPinfoldundertookextensivesemi-structuredinterviewswiththeDepartmentalScienceAdvisorsandseniorofficialsinministries,centralgovernmentagencies(TreasuryandDPMC),andkeygovernmentknowledgeproducerssuchasStatisticsNZ,SuperuandCrownResearchInstitutes.Shealsoconsultedthewidersciencecommunity,includingUniversities,theNZAssociationofScientists,RoyalSocietyofNewZealand,UniversitiesNZ,HealthResearchCouncil,privateresearchinstitutes(MOTU,NZIER)andaportionofthebusinesscommunityviaATEEDandaregionalcouncilHBRC.Allintervieweesweregenerouswiththeirtimeandcontributions,withoutwhichthisreportwouldnotwhatbeenpossible.IalsothanktheDepartmentalScienceAdvisors,DrStephenGoldsonandMsKristiannAlleninmyoffice,forcriticalreviewandinputthroughoutthepreparationofthisreport.


5

Contents

LETTERTOTHEPRIMEMINISTER 2

ACKNOWLEDGEMENTS 4

1.THECONTEXTOFSCIENCEINSOCIETY 6

2.THECONTEXTOFSCIENCEADVICE 6

2.1DEPARTMENTALSCIENCEADVISORS(DSAS) 82.2THEROLESOFDSAS 92.3THECOMMITTEEOFSCIENCEADVISORS(COSA) 10

3.DATAANDPOLICYMAKING 10

3.1DATAANDSCIENCEINFORMEDSOCIALINVESTMENT(‘CITIZEN-BASEDANALYTICS’) 113.2EXTENDINGTHESYSTEMATICUSEOFDATATOINFORMPOLICY 143.3FROMDATATOEVIDENCE:WHATWORKSINDIFFERENTCONTEXTS? 15

4.THEACADEMIC-POLICYINTERFACE 15

4.1THEPOLICYCOMMUNITYPERSPECTIVE 164.2THEACADEMICCOMMUNITYPERSPECTIVE 17

5.RISKMANAGEMENT,CRISESANDEMERGENCIES 19

6.HORIZONSCANNINGANDFUTURESEXERCISES 20

7.OTHERCOMPONENTSOFTHEECOSYSTEM 21

8.THEOFFICEOFTHEPMCSA 21


6

1.ThecontextofscienceinsocietyIn recent years there has been considerable andgrowing reflection on the interface betweenscientific evidence (derived from both from thebroadly defined social and natural sciences) andpublic policy formation. In turn this interfacecannot be seen in isolation from the interactionbetween science and the rest of society itself.This relationship is changing because the natureof science and its contribution to society is itselfchanging, but also because the digital age hasenhanced enormously the accessibility ofknowledge.Greater accessibility of knowledge, especiallythrough the internet, has brought bothopportunities and challenges. The pervasivenessoftoday’scommunicationchannelspromotesthedissemination of reliable, unreliable andsometimes intentionally false and biasedinformation. It has also led to a widespreadassumption that information without expertanalysisandinterpretationissufficienttodrawan‘informed’ conclusion,adding toagrowing senseof a post-trust, post-expert world. The challengeforgoodpolicymakingandforengagedcitizensisthus to reach through this noisy environment todistinguish relatively reliable from non-reliableinformation.In addition, knowledge production itself has alsochanged.Sciencetoday isno longerconsideredalinear search for ‘truth’, but rather increasinglytheanalysisofcomplexsystemsleadstoscientificconclusionsexpressedasprobabilitiesratherthancertainties. Theexplosionof scientific techniqueshasallowedscientiststoexplorenewissuesinthecomputational, life and environmental sciences,forinstance,thatimpingeincreasinglyonsocietalvaluesandthusinevitablyonthepoliticalprocess.Yet, ironically, with the enormous progress inscience, there isa commensurate increase in thecomplexityandincompletenessofourknowledge.The very issues for which policy makers mosturgently need scientific advice are the issues forwhichthescienceisoftenfarfromdefinitive.The

sheer volume of scientific knowledge todaycreatesanewsetofchallenges.Inadditiontothe‘burdenofknowledge’createdbytheexplosionofthe scientific enterprise, there is the addedburdenofassessingthequalityofthatknowledge.Indeed, someof themost important andbiggestvalue judgements that scientists with advisoryfunctions must now make are about the qualityandsufficiencyofavailableevidenceonwhich toreach any conclusion. They must weigh theimplications of any inferential gap between theavailabledataandtheconclusionsreached.Globally there is increasing interest in theuseofrobustly developed knowledge to inform andevaluate policy. Indeed there is no area ofgovernment policy making where scientificallydeveloped evidence cannot contribute to betterdecision-making. But whereas science advisorymechanismsinmostwesterndemocracieslargelygrewoutof themilitaryanddefenceneedsafterthe SecondWorldWar, in the past two decadesthere has been a growing recognition thatscientificevidencehasan importantandbroaderrole to play across all domains of democraticgovernance, from local to national and globallevels. While there is no singular approach to ascience advisory ecosystem, there is a growingglobalconsensusastothequalities itmusthave,which is reflected in how the NZ ecosystem isevolving.To support this report, during the latter half of2016myOfficeconductedastakeholdersurveytoupdate our understanding of the current stateand views of theNZ ecosystem. Findings of thissurvey are reflected in thepresent report.OtherinputscomefromthegrowingglobalconversationledbytheInternationalNetworkforGovernmentScienceAdvice3(INGSA)onthesequestions.

2.ThecontextofscienceadviceScientific evidence alone does not make policy.Policy making involves a broad range ofconsiderations and democratic politicians areproperlyexposedtoawidevarietyofinputsfromboth formal and informal actors. Most decisionsinvolve trade-offs as the political process must3http://www.ingsa.org/


7

consider many societal values and interests thatextendwellbeyondtheevidencebase.Too often scientists having little knowledge ofpolicydynamicsfailtounderstandthefactorsthatinfluencepolicydecisionsbeyondtheevidence,orthey fail to put any scientific conclusion into anappropriate frame of reference (e.g. relativeeffectsize,policyrealities,etc),andthiscan leadto tensions. It has been said that scientists areverygoodatproblemdefinitionandlesspreparedfor finding policy-acceptable solutionswithin thetime frameof the usual policy cycle4. Converselythe policy community needs to reflect on whatsciencecanbringtotheassessmentofoptionsforpolicydevelopmentand implementation. Thoughinherently incomplete, sciencewillalmostalwaysbe able to contribute important and essentialinsights to the policy development process.Properly brokered, scientific knowledge shouldleadtobetterpolicydecisions.Central to this presumption is the concept ofbrokerage. This implies firstly that there is amatchbetweenwhatthepolicymakeristryingtoaddress and the evidence base that is provided.Secondlyit impliesthatthepolicymakerreceivesscientific input in a way that, in as far as ispossible, is not filtered by predetermined biasesor is designed to support predetermineddecisions. Rather, the intent of brokerage is tosummarisewhatisknown,whatisunknown,whatcaveats exist inwhat is knownandwhatoptionsmight thus emerge. Brokerage also assists inconsideration of the implications of each option.Brokerageisnotintendedtoadvocatedirectlyfora specific option or weigh the broaderconsiderationsoffiscalpriority,electoralcontract,public opinion, or diplomatic and politicalimplicationsofanyoptionchosen.Thosemattersareforthepolicyandpoliticalprocessitself.In New Zealand, as elsewhere, the emergentscientific advisory ecosystem is increasingly builtaroundthisnotionofbrokerage,asmanifestedin

4Cairney,Paul.2016ThePoliticsofEvidenceBasedPolicyMaking.PalgraveMacMillanhttp://www.palgrave.com/gb/book/9781137517807

multiple ways. A science advisory ecosystem isnot a unitary system, but comprises multiplecomponents.Theseextendfromsciencetoinformtechnical decisions within regulatory andoperational agencies, to science that informspolicyoptionsandprogrammeevaluation. Ithascomponents that lie inside thepolicy communityandthosethatareexternaltoit.Theseprovideacombination of both deliberative and informalinputs. Science advising operates on multipletime scales, ranging from long term technologyfore-sighting and horizon-scanning to assistingwithacuteemergenciesandcrisissituations.A healthy and politically independent researchcommunity is a critical underpinning of aneffectivescienceadvisorysystem.InNewZealandthiscommunitycompriseslargelyscientistswithinacademia and the Crown Research Institutes(CRIs). Individual scientists and collaborativegroupings within the research institutes anduniversities, professional and academic bodies,non-governmental organization (NGOs) andindustry all produce and can provide policyrelevant knowledge. The high level ofinternational connectedness within our sciencecommunityisalsoanimportantassettothistypeofknowledgeproduction.It is however important to distinguish betweenthe roleof scientists as generatorsof knowledgefrom the need to have systems that collate andtranslate that knowledge for public and policypurposes. We have seen the growing value ofhighly effective science communicators and it isimportantthatsuchdevelopmentscontinuetobeencouraged. But formal public communication isbutasmallpartoftheinterfacebetweenscienceand society. Equally, it is important that thescience capital of the broader community isenhanced in other ways – for example bycontinued development of theNation of CuriousMindsprogramme.5Overthepasteightyears,NewZealandhasbeendeveloping a more complete and formal scienceadvisory system to bridge between the policy

5www.curiousminds.nz


8

community and the science community and toassist the brokerage function. Thesedevelopments have generated considerableinternational interest and are widely regarded.The major elements of this formal scienceadvisory system includes theOffice of the PrimeMinister’s Chief Science Advisor (OPMCSA),departmental science advisors (DSAs),departmental chief scientists and the Committeeof Science Advisors (CoSA). These roles arecomplemented by other ecosystem actors, mostnotably the Royal Society of New Zealand, butalso sectorbodies suchas scientific societiesandprofessional bodies. The Royal Society of NewZealand–ournationalacademy– inparticular isan source of deliberative interpretation ofscientificknowledge.Ithassignificantlyenhancedits capabilities and approach. Academics and CRIscientistsalsoplaycriticalrolesthatarediscussedlaterinthisreport.Thisevolvingecosystemisplayingan increasinglyimportant role in areas ranging from emergencymanagement to casting an evidentiary andscientific lensonannualbudgetbids in thesocialsector.ThisreportfocusesonspecificissuesthatIbelievemeritemphasisandfurtherconsideration.

2.1DepartmentalScienceAdvisors(DSAs)Oneof thecentral recommendationsofmy2011and2013reportswastheestablishmentofanewrolewithinministriesorclustersofministries:theDepartmental Science Advisor (DSA). Since thattime, a number of ministries have appointedscienceadvisors:asof June2017 thesewere theMinistry of Primary Industries (MPI), Ministry ofBusiness,InnovationandEmployment(MBIE–forscience and innovation), Department ofConservation(DOC),MinistryfortheEnvironment(MfE), Ministry of Health (MoH), Ministry ofEducation(MoE),MinistryforSocialDevelopment(MSD) and the Justice sector (encompassingJustice, Police and Corrections). They have allbeen appointed through an open and formalprocess including interview panels advising theChief Executive. The panels have comprisedappropriate senior officials (generally a deputy

ChiefExecutive)andaDSAfromanotherministryorthePMCSA.MostoftheseDSAsarepart-timesecondmentsofpracticing researchers from universities or CRIs,withsomevariationinthetermsofreferenceandconditions of appointment. This variation mayaffect the ability of some of the DSAs to reachtheirfullpotentialandisaddressedinthisreport.Notwithstandingthis, ingeneral theDSApositionhas been seen to be a very valuable addition tothe advisory system and their roles continue togrowbothindividuallyandcollectively.In my view there are some obvious gaps in thecurrentsuiteofappointments.It is noted that Te Puni Kokiri (TPK) has not yetcreated a DSA role, despite my urging, both tobetter bring to bear the considerable formalresearchonMātaurangaMāoriandotherrelevantresearch disciplines as well as providing aMāoriperspective to the collaborative activity of theDSAs. This delay is disappointing. A science (or‘knowledge’)advisorwouldbe ideally situated tohelp unlock the importance of traditionalknowledge especially in areas such as social andenvironmental health to input in an applicableandpolicy-relevantway.Transport, energy and urban science are areasthat are strongly represented in science adviceinternationally and remain poorly supported inNewZealand,althoughInotethattheMinistryofTransporthas committed recently to starting therecruitmentprocessforaDSAinthenearfuture.Perhaps the biggest void in the current portfolioof DSAs is in the area of disruptive digitaltechnologies.Digitalisation is having amajor andtransformational effect on social, economic andpolitical activities. There is an urgent need formore systematic, coordinated and pre-emptiveassessment of the potential of technologies toimpactonNewZealandsocietyandbusinessandto consider and advise on the complex policyissues that are arising. Examples of thetechnologies that fall in this category includemachine learning, artificial intelligence, theinternet of things, blockchain technologies and a


9

variety of related technologies. Issues relatedstrictly to the public use of ‘big data’ alreadyoccupymuchattentionfromtheDSAsystemandareconsideredseparately.The security agencies are generallywell engagedwithscienceadvisorysystems internationallyandindeedintheUKandUSAweretheoriginoftheircurrent science advisory system. However ourstrategic positioning is such that DSAs are likelynot needed in these departments: instead thechief defence scientist, a newly created role, isinvited to be amember of CoSA and the PMCSAhas the appropriate security clearances to assistwhereappropriate.TheplaceofsciencewithintheMinistryofForeignAffairs and Trade (MFAT) is somewhat different.Here, the PMCSA currently fulfils the role ofScienceEnvoy topromote sciencediplomacybuttheincreasingsizeandresponsibilityofbothrolescreates pressure. A number of countries haveidentified science diplomacy as a growing anddistinctrole.WhereMFAThastheneedfor inputon technical matters in the environment andtrade spaces, it either has turned to specificscientistsortotherelevantDSA:thisarrangementappearstohavefunctionedwell.

2.2TherolesofDSAsDSAs operate individually within their ministries,inministry clustersandasa collectivewithin theCoSA, which will be discussed below. Two mainDSA clusters are currently operating – a socialsectorclusterandanenvironmentalcluster.Bothhave been highly effective; for example inprovidinginputstodevelopthesocial investmentmodel and related cross-departmental strategiesand the budget process and to developingenvironmental, conservation and primary sectorresearchroadmapsrespectively.Within their own departments there isconsiderablevariationinthewaysandtheextentto which the DSAs are deployed. Some (e.g. inMPI)arecorepartsoftheseniorleadershipteam.By contrast, others remain more remote fromthat early policy brainstorming phase. This is amissed opportunity to engage where their

informal input into policy ideation may be ofgreatest value. There is also variation across theDSAs with respect to their role and input insigning off on the evidentiary components ofreportstoMinisters.Similarly,acrossthesuiteofDSAs, there are variable linkages to both theinternalandexternalprocurementofknowledge.SomeDSAshavedirect access to theirMinisters,otherssignificantlylessso.Therehavebeenexamplesof situations inwhichfailure to engage DSAs appropriately has led toproblems in policy development orcommunication. There have also been examples(albeit rare)where departments have claimed tohave the support of their DSA (i.e. anendorsement of sufficiency of evidence toproceed with an initiative), whereas this wasnever actually sought, or where the DSA’s viewneeds tobe clarified for thebenefitof theCEorMinister.Ibelievethatitisimportantthattherebeamoreconsistentviewof theDSAroleacrossministries,while acknowledging that there must be someadaptability tomeet specificneedsandcontexts.The Deputy States Service Commissioner iscurrentlyconsultingoncoreandgenerictermsofreference for use byMinistries in establishing orrenewing DSA positions. The DeputyCommissioner shares my beliefs that it isimportantforall theDSAstobebrought intothepolicy development process and that their inputneeds to be directly and explicitly available toMinisters (especially where the evidence iscomplex or where the proposed policy andevidence may be in contradiction). DSAs shouldhavetheopportunitytointeractdirectlywiththerelevant Ministers, especially early in the policyscopingprocess.Whereministriescontractresearchexternallyitisimportant that DSAs take a role in qualityassuranceforboththecontractandtheresultantreport.Equally,theyhaveacriticalroleregardingpolicy-relevant research conducted internally byagencies themselves.Where the research is datadriven it is critical that the expertise of a DSA issought in ensuring appropriate techniques andinterpretation of big data sets. A major role of


10

DSAs must be in assisting analysts to improvetheir understanding and capacity to engagewithcomplexdataifitistobeusedasevidence.AstheroleofeachDSAhasexpanded,thecurrentlevel of adequate support for their functions hasbecomeratelimiting.Whileingeneral,supporttoeachDSAmustbeprovidedwithineachministry,the growing need for DSAs to work in sectoralclustersmeans that either the OPMCSA requiresadditional support to provide clusters withnecessary analytical and operational power, ortheclustersthemselveswillneedtobesupportedas entities. The process to develop sectoralresearch roadmaps in the environmental sectorandtheprimarysector,togetherwiththeongoingcollaborative processwithin the social sector (egregardingmental health) have demonstrated thepotentialofclusteredengagement.

2.3TheCommitteeofScienceAdvisors(CoSA)Once DSAs were appointed to severaldepartments, the CoSA was established afterdiscussion with the State Service Commission inlate 2014. TheCoSAmeets about nine timesperyear and currently comprises the followingmembers:

• ThePMCSA• ThestrategicadvisortothePMCSA6• The PMCSA’s chief of staff as executive

officer• TheDSAs(currentlynine)7• TheGovernmentStatistician• TheChiefEconomistofTreasury• TheChiefScientistofMBIE• The Chief Scientist of the Ministry of

Defence TheCoSAmeetingoriginatedasalargelyinformalopportunity allowing the PMCSA, the DSAs andthe other members to share work programmesandpeerreviewandassisteachother.Whilethis

6PositioncurrentlyheldbyProfessorStephenGoldson7TherearetwodepartmentalscienceadvisorsaswellasachiefscientistwithinMBIE

critical informal role is sustained, increasingly anumber of specific issues are either referred toCoSA by Treasury, the Department of PrimeMinister and Cabinet (DPMC), or from otherministries for comment and assistance. CoSA isone of the only entities other than the sectorboards to have a consistent cross-departmentalview of policy developments and their potentialsynergiesandspill-overimplications,makingitanimportant asset. In addition, MBIE increasinglyuses the group as a sounding board on variousaspectsofpolicyforscience.TheDeputyStateServicesCommissionerisinvitedto attend for relevant items. The President andChief Executive of the Royal Society of NZ areinvitedtoattendCoSAmeetings,wherethereareno government-sensitive agenda items. Thepresence of the RSNZ enables a bettercoordinationofthevariousrolesandcomponentsoftheoverallscienceadvisoryecosystem.

3.DataandpolicymakingOne of the main issues that has occupied CoSAmeetingsinthepastyearhasbeentheuseofbigdata in public policymaking. MyOffice and theCoSA have spent much time considering andproviding inputs into the issues of data hygiene,datamanagement/stewardshipandanalysis.Significantly,from2015thesocialsectorDSAsandthe PMCSA have had increasing input into theannualgovernmentbudgetprocessbyprovidingascientific and evidentiary lens with which toassess the evidence supporting bids in the socialsector. This role is now primarily formalisedthrough the Social Investment Panel whichcomprises the PMCSA, several DSAs, treasuryofficials, and appropriate NGO and social sectorrepresentatives.AkeyroleoftheCoSAhasbeento promote and assist in the development ofconceptual models as a base from which tointerrogate data analytics and budget proposalevaluations.Thisprocesscontinuestoevolve.CoSA has also focused on the issues of socialacceptability of the use of governmentadministrative data for policy development anddecision making. Together, we have provided


11

various inputs into the development and use ofNew Zealand’s pioneering Integrated DataInitiative(IDI)andcitizen-basedanalytics–this isnow termed the ‘social investment’model. Thisdevelopment is at the leading edge globally ofconsideringhowbig-data-derivedevidencemightinform policy both ex ante and ex post. It isconsideredinmoredetail inthenextsectionandina separate reportonevidence-informedpolicymakinginthesocialsector.8

3.1DataandScienceinformedSocialinvestment(‘Citizen-basedAnalytics’)

Arguably thebiggest challenge inpublic policy inthe last few decades has been how to makedecisions about government expenditure in thesocial sector (i.e. in health, education, socialdevelopment, justice etc). In most democraciessuchdecisionsaremadebasedonacombinationof normative argument, political ideology andelectoral considerations. Despite growingexpertise in ‘implementation science’ and socialpolicyresearch(e.g.‘WhatWorksCentresandtheCampbell Collaboration 9 ), the use of a firmevidence base for policy and programmedevelopment and evaluation has beeninconsistent across most liberal democraticgovernments.Often rigorous analysis has been impossiblebecause multiple interventions may have beenintroducedatonce,orthepoliticaltempohasledtoafailuretoobtaingoodbaselinesorundertakepilot work that could be analysed and scaled.Similarly, there has been a general reluctance toenter into formal randomised trials toevaluateapotential intervention. Staggered introduction of

8Gluckman,P.2017Usingevidencetoinformsocialpolicy:Theroleofcitizen-basedanalytics.OfficeofthePrimeMinister’sChiefScienceAdvisor(http://www.pmcsa.org.nz/wp-content/uploads/17-06-19-Citizen-based-analytics.pdf)9TheWhatWorksNetworkisdescribedhere:https://www.gov.uk/guidance/what-works-networkTheCampbellCollaborationisdescribedhere:www.campbell.org

a programme can produce a pseudo-controlledtrial.However in New Zealand such approaches(whether formally- or pseudo-randomised) havetended to be avoided even though they arewellacceptedinotherjurisdictions.Forexamplethereare now randomised trials testing the effects ofminimal basic incomes underway in at least twoEuropean countries and three cities in OntarioCanada.10Evenifhistoricaldataaretobeusedasthe comparator it is critical there is a proactiveassessment to ensure that appropriate baselinedataexist.Failuretodothishascompromisedtheability to assess a number of past interventions.Part of this broad reluctance is probably due toNewZealand’scomparativelyshortpoliticalcycle.Added to this is the problem that normativearguments are easy to make, yet may reflectdiverse biases. The result can be an argument,sometimes valid and sometimesnot, thatoneoranother subpopulation or contextualcircumstance would not be sufficientlyrepresented within any trial. Herein lies theimportanceofexpertdesignandinterpretation.It is also important to distinguish betweeninstances where policy failure is the result ofineffectivepolicydesignandinstanceswhereit isaduetoaprobleminpolicyimplementation.Thelatter may occur at either a central or providerlevel. One of the strengths of ‘behaviouralinsights’and‘whatworks’unitsoverseashasbeento provide trial designs and meta-analyses thatevaluate practical problems in policyimplementation at the provider level.Implementation science is a growing discipline.Given that much of our social services areprovidedbythethirdsectortheremaybevalueinstrengthening our formal capacities in this areas

10TheGovernmentofOntario’sbasicIncomePilotStudyisdescribedhere:https://files.ontario.ca/170508_bi_brochure_eng_pg_by_pg_proof.pdfPilotstudiesintheNetherlandsandFinlandaredescribedhere:https://www.uu.nl/en/news/utrecht-university-and-city-of-utrecht-start-experiment-to-study-alternative-forms-of-socialandhere:http://www.kela.fi/web/en/basic-income-experiment-2017-2018


12

through, for example, the newly-formed SocialInvestmentAgency.11The lack of quality information on policyeffectiveness leads to general frustration at thepolicy level and increasingly at thepolitical level,both in trying to decide between the manypossible interventions for any type of socialchallenge and in stopping programmes that aresuspected to be inefficient or of little impact atscale. The challenge is exacerbated by the factthat a large number of third party providers andthosewhoaccessserviceswillbeaffectedbyanydecision to review a programme. If the situationgains media attention, then it also becomes apolitical challenge which can affect decisionmaking.Thus,thereisgrowingconcernwithinthepolicy community about how to develop andsustain an evidence-informed approach to thesocialsectorpolicyandprogrammedevelopment.The development of big data and citizen-basedanalyticsprovidesaconceptualwaytoassistwiththisdilemma. In theory ifoneknewabout inputsand outcomes for all individuals in a targetpopulation one could use a number of analyticaltechniques to see which inputs might explainsignificant effects on outcomes. Alternatively itmight be possible to identify subpopulationswhichresponddifferentlytodifferentinputs(suchas educational opportunities or access to healthinformationandservices).However this model makes some majorassumptions. In particular, it implies a certainconceptualunderstandingofhowsuchinputsarelikelytodirectly,indirectlyorinteractivelyleadtoa particular outcome. The model also assumesthat the inputswill have a significant effect thatcanbedetectedagainst the inherent ‘noise’ thatexists in such data sets. It is self-evident thatinputs that government would be interested inare those it has some control over (education,health, justice, social welfare services etc.), yetthere are many other factors that affect social

11http://www.legislation.govt.nz/regulation/public/2017/0067/8.0/DLM7219210.html

outcomes – some are easier than others tomeasure(e.g.familystatus,economicstatus)thanothers (e.g. social group interactions). As theprivatesectorhasalreadydemonstratedinitsuseof targeted electronic messaging, even limiteddata on an individual allows better targeting ofservices. So how can a social investment modelbasedongovernmentdatabemadeviable?A number of countries are increasingly using thecapacities in big computing systems to provide avarietyofservicestotheircitizens.NewZealandisdoing this as well, but has also been at theforefront of considering the use of citizen-baseddata to inform social sector decision making.Citizen-based analytics uses administrative,census and service databases, combining thesewith domain expertise from the social sciences(e.g.publichealth,education,orsocial-psychologyfor instance) to develop a conceptual model ofhow to analyse the data based on what theresearch tells us about the issue. Such analyticshavestrongpotentialtoleadtodecisionsthataremore evidence-informed. These in turn shouldleadtobettersocialreturnsintheformofbetterhealth , education and economic outcomes fortheindividuals.The conceptual underpinning is key – there hasbeenenormousprogressinourunderstandingsofhuman development, both biological andbehaviouraland inthesocialsciencesthatallowsmuchoftheargumenttoshiftfromnormativetoscientific.While it has long been self-evident that factorsoperating in one part of a life course can haveimpacts in the latter part of life-course,understandingtheserelationshipsiscomplex.Forinstance, they cross domains (e.g. from earlychildhoodeducationtojusticesysteminteractionslaterinlife)andwhenpathwaysmayoperateoverextremes of the life course, assessing relativeimpact and effect and social benefit and cost isdifficult. There are some real challenges in thedata analysis; for example dealing with complexinteractions, confounders or apparent but notvalid causal associations, particularly if data setsaresmall. Inaddition,thereisagrowingrangeoffactors that interact to influence ensuing


13

outcomes and not all will be necessarily in thedataset.Theseissuescanleadtointensedebateover,forinstance,therelativeroleofpovertyperse versus specific factors that may have aninfluenceonoutcomes.Citizen-basedanalytics,whilenotapanacea,offerthepolicymakerandthepublicgreaterclarityasto the options and their rationale and likelyeffects. While the balance of priorities andacceptabilityofcertainapproacheswillalwaysbethebasisofpoliticaldebate,suchanalyticsshouldincrease the quality of decisions made.Unfortunatelytheterm‘socialinvestment’canbemisunderstood as focused on investment in astrictlyfinancialsense.Instead,itisametaphortomean supporting the most effective socialinterventions for social outcomes identifiedthroughrobustdataanalyticsandsupportedbyascientificunderstandingofthelinkagesinvolved.Attheheartofsocialinvestmentarecitizen-basedanalytics – that is compiling data on thepopulation in a way that allows ex ante and expost assessment of probable deterministicrelationships between inputs, environment (atlevels ranging from family to physical) andoutcomes.Thegovernment’sIDIdatabaseisbeingdevelopedtodoexactly this.With it, researchersand analysts can examine government-collecteddata (andpotentially other data sources) to lookfortrendsandrelationshipsbetweenfactorsthatmay not otherwise have been known. Oncelinked,thedataareanonymizedandplacedunderthe custodianship of Statistics NZ. Thisstewardship arrangement is critical as StatisticsNZ has the highest level of public trust and itsprotocol for access to the data for researchpurposes ensure anonymity and privacy isprotected.Issues relating to data governance and sociallicensearebeyondthispaper.ButInotetheroleoftheDataFuturesPartnershipasacutting-edgeapproach to considering these issues, which arealso explored further in my separate report oncitizen-basedanalytics.As a research tool the IDI is uniquely placed tosupportthegoalsofthesocialinvestment(citizen-

based analytics) approach. It brings data frommultiple sources together. At the moment, thebiggest challenge for the furtherdevelopmentofthe IDI is to includeclient-leveldata fromserviceproviders. These types of data are often seen asthemost sensitive andmust be entered into thedatabase and used in a way that continues toensure protection and anonymity of individuals.ForIDItoproducemeaningfulknowledge,itisnotsufficienttoknowwhetherfundingtoaparticularprogramme has gone to a particular agency.Rather, it is ultimately necessary to understandwhatservicesaparticularindividualaccessedandacrosstheboard,amoreholisticunderstandingofoutcomesforsuchindividuals.Only through access tomore complete data willmeaningful programme evaluation be possible.Furthermore, the big data analytic techniquesnow available allow causal relationships andinteractions to be analysed evenwhen there aremultiple confounding factors. However it needstobeemphasizedthataimlessdatamining isnotsufficient,nor justified. Expert input isneeded inbuilding the models to be tested, testingplausibility of proposed causal relationships,evaluating confounding effects and evaluatingotherassumptionsmade.ThekeyissueforsuchusesoftheIDIistheirsocialacceptability.Itislikelythatsociallicensefordatausecanbeobtainedandmaintainedbecausetheagency responsible for the data is Statistics NZ,which has high public trust. Despite suchassurances,forpublicpeaceofmind,itshouldstillbe emphasized, for social investment modellingpurposes, there is no need whatsoever forindividuals to be identified once databases arelinked.TheusesofdatawithintheIDIcanbeandarebasedonanonymizeddatasets.Theissue,however,getsmorecomplexifdataareheld outside the IDI. Many service deliveryprogrammesaremanagedataministry levelandthey do need access to data to evaluate andmonitorprogrammes.But thesamedepartmentsmayalsoneedaccesstodataforensuringservicedelivery coordination between agencies (e.g. inthe case of family violence) or for compliancereasons (e.g. looking for benefit fraud). These


14

reasons for data collection and use are perfectlylegitimate at the ministry level, but they shouldnotbeconfusedwiththebroaderresearchusesofthedataintheIDI.These multiple uses can create complexity anduncertainty in the minds of the public as to theboundaries of data protection and privacy. As aresult, negotiating social license for data use iscomplicated. To address this, the GovernmentStatistician, the Data Futures Partnership, thePrivacy Commissioner and my Office arecollaboratively working with senior officials torecommendanassuranceandgovernancesystemfordataaccessanduse.Criticalinsuchasystemisto clarify and codify the data use types and therelevantprotectionsandprovisionsforeach.BoththeIDIandthesocial investmentmodelarestill in their infancy. Itwill taketimeforkeydatasetstobeadded,butalreadytheIDIisprovingtobe a valuable tool in changing the nature of thepolicydiscourseandshiftingagenciestobemoreproactive in ensuring that policy development isevidence-informed. Despite this, there aredangers in either understating, or in overstating,its value. There is no doubt that citizen-basedanalyticswill become central to policymaking inthe social sector and New Zealand is at theforefront of this movement. Internationally theattentionthework isacquiring isverysignificant.AlreadyithaschangedtheNZannualgovernmentbudget process dramatically. But there is aconcomitant need to build data analyticcapability, to continue to enhance social licenseandtocontinuetoreducedepartmentalsilos.There is also an ongoing danger of simplisticinterpretation if the appropriate subject matterexperts are not engaged in the analysis processfrom the beginning. For instance, it is easy tooverstate the predictive value of citizen-basedanalytics. Relying on a predictive approach cansometimes wrongly focus policy efforts. Suchlimitations can also lead to politicaloverstatement. Iremainhopefulthat,asdecisionmakers become more familiar with bothpopulation and subpopulation analytics, theseissueswillbecomelessevident.

The social investmentparadigmsupplementsbutdoes not displace other methods andconsiderations for social policy development.There will always need to be values-basedjudgements and some aspects of socialprogramme delivery (e.g. crisis response, servicecapacity management) will always requiretraditionalpolicyapproaches.

3.2ExtendingthesystematicuseofdatatoinformpolicyThe increased focus on environmental reportingwith annual Tier-1 report on the environmentnowproducedbyStatisticsNZjointlywithMfE12isa welcome step forward. It highlights theincreasingimportanceandutilityofdatainpolicymaking and evaluation in the natural resourcedomain. As my own recent report on FreshWater13makes clear, some of the key issues inenvironmental reporting are having uniformstandardsofdataandmeasurementandensuringlongitudinalassessmentofthesamemeasuressotrends become assessable. The environmentalspace is highly contested with differentstakeholdershavingverydifferentviewsregardingpriorities and strategy. Government has thechallenge that while it has regulatory andlegislative levers, much of the operationalmanagement of the natural resource sectoroccursviaregionalandlocalgovernment. Thereare lessons and approaches in the citizen-basedanalytic approach to social policy developmentand assessment that could be applied into thenatural resource sector – particularly regardingthe collection and use of data and its expertanalysis.

12http://www.mfe.govt.nz/more/environmental-reportinghttp://www.stats.govt.nz/browse_for_stats/environment/environmental-reporting-series/indicators-for-freshwater-2017.aspxhttp://www.mfe.govt.nz/publications/environmental-reporting/environment-aotearoa-201513Gluckman,P.2016.NewZealand’sFreshWaters:Values,trends,statesandhumanimpactshttp://www.pmcsa.org.nz/wp-content/uploads/PMCSA-Freshwater-Report.pdf


15

3.3Fromdatatoevidence:whatworksindifferentcontexts?Accessing and appropriately analysing big data isgamechangerforhelpingustocharacterisepublicpolicy problems and target interventions. Butscientifically-informed data driven insights aremost powerfulwhen they canbe combinedwithotherrelevantknowledgeandtestedinreal-worldsituations. Recently, New Zealand has hadopportunities to learn from a number ofpromising initiatives established in the UK andinternationally to do just this. Specifically, theseinclude:

• thenetworkof‘WhatWorks’Centres,whichtakeanimplementationscienceapproachthatconsiderscontextsandotherfactorsinthesuccessorfailureofpolicyobjectives;

• the‘BehaviouralInsightsUnit’,whichappliessocialandpsychologicalinsightstohowpolicyandprogramsaredelivered;and

• effortstobuildcapacitywithinthecivilserviceforbetterpolicydevelopmentanddelivery.

TheWhatWorksandrelatedapproachesarefastbecoming an international public good throughthe development of toolkits and systematicreviewslargelyrelatedtopolicyimplementation.There isalsogrowing interestbygovernments toconsider‘behaviouralinsights’approaches.Inthisapproach, the implementation of policy ideasmakes use of scientifically-derived behaviouralinsights to help ‘nudge’ individual action indesired directions. Dowewantmore people touse public transportation? Let’s learn whatfactorsencourageor impede individuals touse itand apply these to make it easier for people toadopt the practice. Through policy trials, likelyenabling factors can be tested and optimised indifferentcontexts.BoththeUKandAustraliahaveestablished behavioural insight teams withincentral government to support policyimplementationindiversedomains.

Forsuchevidence-informedmethodstobeusefulrequires both enhanced policy skills and

institutions that generate demand as well assupply of policy relevant science. This meansensuringawell-trainedcivilservicemorebroadly.There is also a more structural or institutionaldimension in shaping government anddepartmentstobewillingandabletoexperimentandlearn.Policyinnovationrequiresacommunityof learners and the professional public serviceshouldbenurturedtodothis.

4.Theacademic-policyinterfaceWhereas the above are all fairly recentdevelopments through which government isincreasinglyevidence-ledindecisionmaking,NewZealand’ssciencecommunity,andinparticularitsacademic science community, constitutes alongstanding resource of policy-relevantknowledge. However, there are still someconsiderablechallengesineffectivelytappingintothisresourceforscienceadvicetogovernment.NewZealandpolicymakersappeartoreachouttoacademia less than in many other advanceddemocracies. While there are many establishedbridges between academia and the policycommunity in countries suchas theUKandUSA,such a culture of professional exchange, appearsisnotsoapparenthere. InNewZealand, it isnotcommon to find policy professionals withsignificant post-graduate research experience,althoughthis isstartingtochange. Similarly,fewacademics have much direct experience withinthepolicycommunity.Asabroadgeneralisation,aminorityofscientistsunderstand the complexity of policy making,imaginingittobealogical,linearprogressionorasimple cycle with some iteration. By the sametoken, policy professionals may often assumeWikipedia tobea sufficient sourceof knowledgeon which to build major policy proposals.Whentheydoturntoresearch,itismostoftenthroughthe generic management consulting industry,using conventional cost-benefit approaches andrelated assumptions, rather than to turn tospecialist expertise within academia. Thissituationcouldleadtosomescientificallyunsoundinputs.AmajorargumentforappointingDSAswasthat they could link the NZ policy community


16

better to the NZ academic expert community.Consequently,acentralfocusontheresearchthatunderpins the present report was to understandthe barriers to better interaction between thepolicyandacademiccommunities.

4.1ThepolicycommunityperspectiveThe policy community’s perspective has raised anumber of factors, some of whichmerit specificconsideration if we are to promote moreacademicengagementinpublicpolicymaking:• Timeliness:Policydevelopmentismostoften

on relatively short timelines, particularlygiven New Zealand’s short electoral cycles.Generally, policy makers find the academiccommunity and their institutions slow toconcludeanagreementtoprovideinputs.Bycontrast, the professional consulting sectortends to be able tomove fast contractually.The consulting sector is able to assigndedicated staff immediately to a problem,whereastheacademiccommunityhastofititaroundother professional obligations. Also,policymakersmayfeelmoreabletointeractfreely with the consulting sector becausethey are seen as privileged clients with thissector. The academic community thus maybesuitableforempiricalresearchonalongertime horizon, but is less favoured forliterature reviews and, importantly, forevaluations.

• Access: While the policy community can

reach out directly to the consultantcommunity with which it has a closerelationship (often the consultants are ex-policy staff,which itself creates a narrowingperspective), it is sometimes more difficultreaching out to the academic communityunlesstherehasbeenpriorcontactbetweenthe policy analyst and the academic expert.Not infrequently,policymakers reachout tosomeonetheyknowbutwhomaynotbethebestplacedtoprovidetheneededexpertise.The policy communitymay not knowwhomto contact and there can be layers ofinterface to get through (e.g. the researchoffice, the technology transferoffice)before

apolicymakercanworkwithanacademicofinterest. The DSAs can at least help identifyandreachouttotherelevantexpertsandtothe body of national and internationalacademicliterature.

• Advocacy and brokerage: while the

consultancy community is trained in theknowledgebrokerageroleand‘policy-speak’,theprofessionalprofileofmanyacademicsisbasedontheirrolesas‘criticandconscienceof society,’ which can lead some policymakers to be reluctant about the productthey might receive from academics. Therecan be a preconception within the policycommunity that academicswill approach anissue from a particular angle or there isacademichubris that is intenton instructingthe policy community rather than simplysharingexpertiseandknowledge.

For example, often there can be theimpression among policy makers that theacademic social sciences’ roleasa critic andconsciencecouldinhibititspotentialroleasapartner in policy development. Thisimpression is generally unfair; many in thesocialsciencecommunityarepositionedwellto evaluate and integrate policy relevantevidence. Indeed, it isverymuchwhattheydo in their academic pursuits. The addedchallenge is to collaborate effectively withpolicy makers to ensure that theaccumulated knowledge of the relevantresearchcommunitiescanbebroughttobearon policy decisions, bearing in mind theresponsibility to be brokers rather thanadvocates. Good brokerage is aboutassessing the evidence both as to what isknownandwhatisnotknown.Itmustclarifythe caveats and the implications includingspill-over costs and benefits of the optionsthat arise. But the ultimate policy decisionswill require consideration of many othervalues-based dimensions that are properlythe domain of the policy and politicalcommunities.TheDSAscanhaveamajorroleinaddressingtheseissues.


17

• Understanding policy needs: Notinfrequently, a view within the policycommunity (which is at least partiallyjustified) is that the academic communityoften addresses questions in ways that arenot ideally suited to inform the policyprocess. For instance, either the question isnot directly answered because of theacademic reluctance to draw conclusionswithout themost robust testing, or there isthe frequent call that ‘more research isneeded.’Instead,apreferredapproachistheanalysis of what is known and what is notknown and the implications of each. This issomething that can be addressed by betterbrokeragebetween thepolicy andacademiccommunity. Another issue is that while theanalysis must be robust, to be policy-usefulthe product must be digestible and notpresentedasanacademictreatise.

• Cost:Theissueofcostwasraisedfrequently.

Because universities generally charge a flatoverhead fee that does not distinguishbetween the costs of empirical science andevaluativeand reviewwork, there isoftenaperceived cost barrier regardless of therequest. Compared to organisations such asconsulting companies that do not have tosupport expensive laboratory researchfacilities, this is a considerable barrier toengaging the academic community. It is abarrier that, in the current model ofuniversity funding, is likely to remain. It isnoteworthy that in countries whereuniversity infrastructure and indirect costsareaddressedindifferentwaysthantheyareinNewZealand, thereappearstobegreaterengagement with university-basedresearchers.

• Confidentiality and control: There can be atension between academia, for whom earlypublication is generally important, and thetraditional desire for confidentiality in thepolicy process. However this can be anoverstated issue in that many parts ofacademia are accustomed to these delayswhendealingwith theprivate sector. It couldbe that those disciplines most likely to be

dealing with the policy sector may not havesimilar collaborative experience with industryfrom which to draw. However, of muchgreater concern is the desire, that issometimes expressed by the policycommunity,tomaintaincontroloverpotentialacademic publication even after the policyprocess is complete. Provided that anyacademic publication is free of overt politicaladvocacy, there is absolutely noneed to limitpublicationexceptincasesofnationalsecurityrisk or where the research involves sensitivedataorotherelementswithethicalorprivacyimplications. Again the DSAs can assist inaddressingthistensionandtheremaybevaluealso in establishing some generic protocols toaddress this matter. The government’scommitment toaccountabilityand theOfficialInformation Act mean that policy makingappliestoevidentialinputs.

In general, it seems thatmany in the policy andacademic communities are equally sceptical ofeachother.Theirdifferentperspectivesrepresenta general lack of familiarity between thesedistinctive cultures.Ahelpful trendhasbeen thegrowingrecognitionbetweentheNationalScienceChallenges and the Centres of ResearchExcellence of the desirability of closerrelationshipswiththepolicycommunity.

4.2TheacademiccommunityperspectiveFromtheacademiccommunity’sperspective, themainissuesinclude:• Control/independence: The discussion above

highlightstheissuesaroundconfidentialityandcontrol of research information. These issuesare not particularly complex to deal with butrequire maturity and mutual understandingfrom both the policy community and theacademic community. Unfortunately I amaware of examples where governmentagencieshavewantedtomaintaincontroloverreports produced by academics even whenthereisnoimmediatepolicyissueinquestion.For this reason, protocols for research accessand dissemination should be developed,


18

perhaps with the assistance of the StatesServicesCommissionandCoSA.

• Performance-BasedResearchFund (PBRF)and

relatedincentives:Theacademiccommunityislargely incentivisedby how they are regardedby their peers. The ability to receive researchgrants, invitationstospeak,prizesandawardsand academic promotions are all part of theculture of academic merit. The PBRF hasfurther institutionalised this culture byrewarding institutions according to theperformance of staff in these traditionalactivities of research (and in particular in thevolume and supposed quality of academicpublications).MorerecentlythePBRFhasalsoacknowledged the role of commercialinteractions, but policy-related impacts arepoorly acknowledged. Yet policy relevantresearch is critically needed and indeed is anincreasinglyimportantjustificationfortheNewZealand taxpayer supporting our researchcommunity. There are real career-limitingbarriers for an academic to become tooengaged in policy research however. This isparticularlysoformorejuniorresearcherswhomust meet a number of tacit careermilestones. If engagement and impact on thepolicyprocessweremoreexplicitlyrecognisedwithin the academic merit system, it wouldencourage more interaction between policyandacademiccommunities.

Notallresearchersarehousedwithinuniversities.A large number of government-fundedresearchers are based in CRIs. Their enablinglegislationandtheirmodeofoperationmeanthatingeneralanumberoftheissuesraisedaboveareless prominent within relationship between CRIsandpolicymakers.However,theCRImodelraisesitsownsetofissueswithregardtoscienceadviceto government. For one thing, CRIs do notrepresent the full range of disciplines within theuniversitycommunity,inparticularpolicy-relevanthealth and social research must come fromacademia or private consultants. Also, with amandatetoself-fundthroughbothprivatesectorandpublicinterestresearch,theCRImodelcanbeat risk of a conflict of interest if ever privatelycontractedresearchrevealsanyriskthat is inthe

public interest. For this reason contractingagreements must be – and generally are –managed very carefully,with an alertness to thepublicinterest.The Royal Society of New Zealand includes ournationalsciencesandhumanitiesacademy. It isasource of independent self-determined reportsand public education events andmaterial, but itcanalsobea sourceof reportsat the requestofthe policy community. A model that has beensuccessfully piloted in recent years is the jointapproachbythePMCSAandthePresidentoftheRoyal Society of New Zealand to convenecommittees providing technical reports onpotentiallycontentioustopics. Inthis,thePMCSAplays the role of broker with the policycommunity, while the Royal Society enables theauthoritativevoiceofthesciencecommunity.This brokering process is an important step inaddressingwhathasbeena frequent criticismofacademy reports worldwide in that they do notanswerthequestionposed,butthisissometimesasmuch a function of the question being poorlyframed and explained. The OPMCSA and therelevant ministry negotiate the framing of thequestion.ThePresidentoftheRoyalSocietythenestablishes an expert panel, which is thensupported by a research analyst/writer from theOPMCSA.Followingcompletionofthedraftreportthe PMCSA arranges peer review (includinginternational reviewers). Once finalised, thereportisjointlysignedoffbyboththePMCSAandPresidentoftheRoyalSociety.Theresultismuchclearer process for framing and answeringtechnical questions that satisfies both policy-specific needs while remaining scientificallyindependent. The brokerage role played by thePMCSAisextrapolatedtoaroleplayedbyofDSAsin brokering between the policy community andacademicspreparingreportsattheministrylevel.The above discussion highlights the value anddesirabilityofenhancingthelinkagesbetweenthepolicycommunityandacademia.ThePMCSAandtheDSAshaveacriticalroletoplayasindividualsat the interface, but structured institutionalapproaches seem desirable also. The value andutilityofprotocols that canensureconfidenceof


19

the policy community while respecting academicindependenceandintegrityseemself-evident.In many other comparable countries there iswidespread use of academic secondments,internshipsand fellowships toprovidea cadreofoutside expertise within policy departments.There are also opportunities for policyprofessionals to spend sabbaticals within anacademic context. Many other countries’ keyministries now provide for limited–termfellowships for academics to spend time withinagenciesandthemutualvalue indoingsoseemsobviousandshouldbeconsideredhere.In addition to encouraging greater exchangebetweenthetwocommunities,thereisabroadercultural change that can be fostered at thetrainingstageofbothfuturescientistsandfuturepolicy professionals. While universities arebeginning to encourage science communicationcourses, there is a need for a more holisticapproach that considers the broader ‘civics ofscience’ including science communication,science-in-societyapproachesandthephilosophyand ethics of science and science-policyinteractions. It would be logical to imagine thatsuch courses should be encouraged at seniorundergraduate and graduate levels and open tostudentsinfacultiesofartsandhumanitiesaswellasfacultiesofscience.My own Office has started to explore some oftheseissuesinapilotvoluntaryexerciseknownastheSciencePolicyExchange.Thispilotendeavourbrings together competitively selected early-career scientists and policy professionals forcritical engagement on practical complexproblems. The activities of INGSA offer furtheropportunitiesinthisregard.

5.Riskmanagement,crisesandemergenciesIt has been said that themost critical role for ascience advisor is in crises/emergency

management14. Depending on the situation thescience advisor must be able to reach out tospecific and sometimes unanticipated sources ofexpertise, help identify the range of issues thatneeds tobeaddressed, translate technical inputsfor decision makers, and be part of the publiccommunication and reassurance process. All ofthis must be done within a highly constrainedtimeframe and often tense operationalconditions. Several natural disasters experiencedby New Zealand in recent times havedemonstratedtheneedforeachoftheseroles.Quite apart from the acute phase of crises,science also plays a critical role in the actuarialsideofriskidentificationandinhelpingtodecideon risk-reduction strategies. This has been aparticularly important role of the OPMCSA andDSAsover thepastyearas risk identificationandprevention has attracted more governmentattention.ThePMCSAservesontheStrategicRiskand Resilience Panel (SRRP). More recently thePMCSA has been made a member of OfficialsCommittee for Domestic and External SecurityCoordination (ODESC) for relevant crises. AnOPMCSA staffmemberwith appropriate securityclearancehasalsobeenaddedtorelevantWatchGroups.While for some ‘typical’ emergencies (e.g. acutenatural disasters), the core scientific expertise isgenerallypre-identifiedandavailablewithinNewZealand, for other types of emergencies theexpertiseismoredistributed.Expertiseisneededtoensurescientificadvicetothedecisionmakersunder urgency, to ensure or interpret particularanalyses to inform the situation, and to provideinformedcommunicationsinatrustedmannertothe public according the their changinginformationneedsatthevariousstagesofagivencrisis.In the case of the latter role, it may not bepossible for the individual involved in the actual14SirMarkWalportinhisplenaryaddresstothefirstINGSAconference.Availablehere:https://www.youtube.com/watch?v=aak-WYvQV_E&list=PLqk2_xL-kgou1tpglGytbDXUM7idJOm_o&index=5


20

studies or analyses also to be the one tocommunicate these. This issue is particularlyproblematicwhen the CRIs are involved.On onehand,ourCRIs are themajor sourceof expertisein many policy-relevant areas, but on the otherhand,CRIChiefExecutivesmayfeelcompelledtolimit the public role of their scientists in acutesituationsfearofpotentialliabilityissues.Indeed,the Chief Executives themselves are keen thatthereisgreaterclarityonhowtheirscientificstaffmight be deployed for public communication inemergency situations. This is a matter that myOffice is currently addressing with DPMC, theMinister of Science and Innovation, TheMinisterofCivilDefenseandScienceNewZealand,whichrepresentstheCRIs.Given the potential range of expertise required,and the importance of scientists interactingdirectly with the policy community in mostemergency situations that we might envisage,somecountrieshaveestablishedformalprocessesto engage the science community for thispurpose. A particularly salient model for NewZealand is that of the UK. Central to the UK’semergency preparedness model is SAGE (theScientific Advisory Group for Emergencies). Inmajor emergencies, SAGE is called into actionbyorder of the Cabinet Office. It is chaired by theUK’s Chief Science Advisor, and involves otherScience Advisors relevant to the situation andother scientists (both from public and privatesector). TheroleofSAGE ispracticedalongwithother parts of the emergency managementsystemoftheUKonaregularbasis.InNewZealand,wehaveundertakenagooddealofriskidentificationandplanningworkalreadyforthe development of a possible national riskregister,asexistsinmanyEuropeancountries.

6.HorizonscanningandfuturesexercisesTechnologies and their social contexts changerapidly. There will inevitably be opportunities,challenges and sometimes controversies arisingfromnewtechnologiesormodesofoperationas

science and innovation increasingly interact withalmostallaspectsofhumanlife.Technologies are emerging at a very fast rateparticularly in the digital, transportation and lifescience fields of application. Many of thesetechnologies could have profound effects on thepolicy-making and decision-makingmanoeuvrability of States, as well as on theireconomy,ontheirsocialorganisation.Technologyincreasingly will have broader policy, ethical,diplomatic and trade implications. In general,New Zealand and other democratic states havefoundthemselvesinareactivemodeinmanagingor otherwise mediating new technologies. Theexception has been those related to humanreproductionorgeneticintegrity.Itisnoteworthythat, in both these cases, the challenges createdby non-adaptive regulation are increasinglydiscussed.In reality, the implications of many new andemerging technologies are such that moreproactivepolicypositions seemdesirable,even ifthese will always need to be adaptive andpreliminary. Some of the technologies that maymerit consideration include artificial intelligenceanddeepmachinelearning;theinternetofthings,advanced genetic technologies, autonomousvehicles and mind–enhancing pharmacologicalagents, to name but a few. Every one of thesetechnologies andmanyotherswill be introducedin some way in some societies, with potentiallyprofound effects internationally. Adaptiveregulation or policy approaches will clearly beneeded.The digital transformation highlights howprofoundly and rapidly technologies can impacton society, and this exposes the vulnerability ofsovereign state to control use in mostcircumstances. Already, we are seeing howautomation and machine learning could andindeed already is impacting onmany current jobsectors,and it remainsmostuncertainas tohoweducation and employment training can andshouldadapt.For these reasons, many countries now includehorizon-scanning and technology futures scoping


21

and assessment as an important part of theirpolicy toolkit. There are a number of formal andwell-detailedprocessesthatcanbeemployedforsuch purposes. Yet, such exercises have beenlargelyabsentinNewZealandpolicydevelopmentcontext (except in Defence and Transport). Thisdeficiencyinlong-termplanningforpolicymakingisinpartaspill-overfromtheshortlengthofourelectoralcycles.Thesegapsandtheskillsneededto fill them have been recognised and are nowbeing considered more generically by the PolicyProject15beingundertakenbycentralagencies.Morerecently,NewZealandhasbeenofferedtheopportunitytoundertakesomespecificaspectsoftechnology assessment and fore-sighting inpartnership with Australia through the newScience,Technologyand Innovationagreement.16IthasbeenproposedthattheRoyalSocietyofNZjoin with the Australian College of LearnedAcademies (ACOLA) in horizon scanning activitiesin areas identified by the Prime Ministers andreportedthroughtotheirScienceAdvisors.

7.OthercomponentsoftheecosystemThis report was not intended to becomprehensive or to repeat matters covered inmy previous reports. For example, it has notdiscussed at length the critical role of manyscientistsemployedwithindepartments likeDOCorthosehavinganalyticaloradvisoryroleswithinregulatoryordeliveryagenciessuchasPHARMACand the Environmental Protection Authority(EPA).Some countries have specific science supportservices for their parliamentarians. To a largeextent, these are sources of brief deliberativereviews and summaries of the literature ratherthan targeted tools for policy development (e.g.the UK’s Parliamentary Office for Science and

15https://www.dpmc.govt.nz/policyproject16http://www.mbie.govt.nz/info-services/science-innovation/international-science-partnerships

Technology - POST). In New Zealand, anequivalent service is provided by theParliamentary Library, which employs severalscientists and which, on occasion, has contactedmy own office for support and information.Additionally,thebriefreportsoftheRoyalSocietyofNZareverysimilarto‘POSTNotes’seriesintheUK. Where parliamentary select committees inother jurisdictions take on a more investigativerole than is traditional in NZ, they are oftensupportedbyspecificscientificanalysts.

8.TheofficeofthePMCSAFinally, it is timely to consider the future of theOffice of the PMCSA itself. This office wasestablishedin2009toplayaninstrumentalroleindeveloping the science advisory ecosystem.However,becausethefulltermsoftherolewereinitially uncertain, it was established as a part-time secondment, to be based within theappointee’s home institution. This guaranteedindependence but created a set of practicalproblemsastherolehasgrowntobeaeffectivelya full-time commitment and the terms ofreference have evolved17.While this institutionalstructure has been made workable for theinaugural position, in the longer term, it willbecome difficult to maintain the required levelinteractionfromanywhereotherthanWellington.Thus, it is suggested that any future PMCSAappointment will need to be Wellington-basedand effectively a full-time position. There are anumberofwaysinwhichtheindependenceoftherole can be protected. The States ServicesCommission andDPMC are currentlyworking onrecommendationsonthefuturearrangementsfortheOffice.

17Gluckman,Peter.2016AnnualReportofthePrimeMinister’sChiefScienceAdvisorhttp://www.pmcsa.org.nz/wp-content/uploads/Annual-report_10-09-2016.pdf

Documents

Enhancing evidence-informed policy making