Click to edit Master title style
Click to edit Master subtitle style
1
ResearchapplicationswithNOAA’sClimateDataRecord
Program
JessicaMatthewsNovember29,2016
2
NOAA’sCDRProgram
Mission:Todevelopandimplementarobust,sustainable,andscientificallydefensibleapproachtoproducingandpreservingclimaterecordsfromsatellitedata.
https://www.ncdc.noaa.gov/cdr
3
WhatareCDRs?
• TheNationalResearchCouncil(NRC)definesa CDR asatimeseriesofmeasurementsofsufficientlength,consistency,andcontinuitytodetermineclimatevariabilityandchange.
• NOAAisapplyingmoderndataanalysismethodstohistoricalglobalsatellitedatatoreturnneweconomicandscientificvaluefromtherecords.
4
WhyareCDRsimportant?
• Providetrustworthyinformationonhow,where,andtowhatextenttheland,oceans,atmosphereandicesheetsarechanging.
• Thisinformationwillbeusedbyenergy,waterresources,agriculture,humanhealth,nationalsecurity,coastalcommunityandotherinterestgroups.
5
FundamentalandthematicCDRs
• CDRs aregroupedbyFundamental CDRs andThematic(Atmospheric,Oceanic,andTerrestrial) CDRs.
• Fundamental CDRs aresensordata(e.g.calibrated radiances,brightnesstemperatures)thathavebeenimprovedandqualitycontrolledovertime,togetherwiththeancillarydatausedtocalibratethem.
• Thematic CDRs aregeophysicalvariablesderivedfromthe FCDRs.
6
CDRexamples
• Fundamental:– Brightnesstemperatures,Reflectances,Meanlayertemperatures
• Atmospheric:– Aerosolopticalthickness,Cloudproperties,Ozone,Precipitation
• Oceanic:– Seaiceconcentration,Seasurfacetemperature
• Terrestrial:– Snowcoverextent,LAI,NDVI,FAPAR
7
NOAACDROperationalSpiralDevelopment
6 – BenchmarkCDR
5 – FullOperationalCDR
4 – SustainedCDR
3 – InitialOperationalCDR
2 – ResearchCDR
1 - ConceptualCDR
Ope
ratio
ns
Research
8
ResearchtoOperations•Goal- TomovefromtheResearchandDevelopmentofaCDRtoInitialOperationalCapability(IOC).
• IOC- ACDRdevelopmentstateachievedwhenaCDR(includingthedataset,sourcecode anddocumentation)isscientificallydefensibleandisarchived,maintained,andmadepubliclyavailable byNOAA.Itsdatamayormaynothavebeenindependentlyreproduced.Itisnotregularlymaintainednorisitnecessarilyextendedasnewdatabecomeavailable.
• FOC- ACDRdevelopmentstatewhenaCDRwillbesystematicallyandoperationallymaintainedinacycleofcontinuousimprovement.Sourcecodeanddocumentationmeethighstandardsandresultsarereproducible.Therecordwillberoutinelyextendedasnewdatabecomeavailableortoreprocesstherecordifrecalibrateddatabecomeavailable.
R2O IOC FOC
9
CDRR20ProcessDiagram1)Assess
– AssembleIPT: PI,PIAssistant,SME,PM,Ops,Archive,Access– Assesssamplesofdata,documentation,sourcecode– Developtransitionschedule
2)Prepare– PIUpdatesSourceCode(CDRPcodingstandards,README)– PICreatesDocumentation(CATBD,FlowChart,MaturityMatrix)– PIPrepsDatafordelivery(netCDFFormat,Metadata,SubmissionAgreement)
3)Transfer– PItransfersacopyoftheSourceCode,Documentation,andDatatoNCDC
4)Verify– IPTverifiesSourceCode,Documentation,andData– Iterativeprocessuntilcode,documentation,anddatameetstandards
5)Archive– NCDCpreservesacopyofSourceCode,DocumentationandDatainthearchive
6)Access– ORR,thenprovidepublicaccesstoeverythingviatheCDRaccessportal
5/30/2012 OperationalReadinessReview
• ProductDescription• ORRChecklist• Websitedemo
Codemadeavailable
Documentationmadeavailable
Productmadeavailable
6)Access
ArchiveAncillaryData
ArchiveCodePackage
ArchiveDocumentation
ArchiveProductData
5)Archive
Verify‘README’
4)Verify
VerifyDataandMetadata
KDP
OperationsBranch ArchiveBranch AccessBranchPI
CDRAssessment
1)Assess
Entercodeintoversioncontrol
2)Prepare
Data
Documents
Code
VerifyFlowDiagram,
CATBD,andMaturityMatrix
3)Transfer
TransferDocuments
Verifysourcecodepackage
ConductSecurityReview
ITBranch
CreateDocuments
CDRP
TransferData
Iterativeuntilallpartshavebeenverified
CreateData
Updatesourcecode
Submissionagreement
TransferSourcecode
IOC
ORR
CDRIPT
Science Branch
10
IntegratedProductTeam
• R2OTransition– NotataskthePIdoesalone,partnershipwiththeIPT
• IPTconsistsof:– PIandanyAssistants– R2OProjectManager– InternalNCEIScientist– ArchiveBranchrepresentative– OperationsBranchrepresentative– AccessBranchrepresentative– AdditionalsupportfromIT,UserEngagement,andotherCDRPmembers
• NCEIpersonnelspend400-600hourstransitioningeachCDRtoIOC
• TypicalR2Otransitiontakestheteam8-12monthstocomplete
11
CICS-NCsupportofCDRs
Science
Scientificprogramming
Stewardship
12
Tropicalmonitoringandprediction
• TheMadden–JulianOscillation(MJO),equatorialRossby waves,andKelvinwavesaredominantsourcesofsynoptic-to-subseasonalvariabilityinthetropics
• Divergentcirculationsfromtheirconvectioncaninfluencetropicalcyclonesandotherweatherpatternsaroundtheglobe
• ForecastersintheenergyindustryharnesstheselongtimescalesandglobalimpactstoanticipateenergydemandintheUS
13
Tropicalmonitoringandprediction
• TropicalmonitoringderivedinpartfromOutgoingLongwaveRadiationCDRdata
• 18hr lag• ForecastsfromNCEPClimate
ForecastSystem• Tropicalcyclogenesistracking• Diagnosticsusedoperationally
byNOAACPCandAirForce14th WeatherSquadron
https://ncics.org/portfolio/monitor/mjo/
14
High-ResolutionNEXRADReanalysis
• Providesrainfallamountsat1-kmresolutioneveryfivemin
• Available from2000topresent• Processingstats:– 130millionradarfiles– 500kjobs– 3millionhrs CPU-time– ~1yr real-time
15
High-ResolutionNEXRADReanalysis
● Israinfallassociatedwithanincrease/decreaseinfatalaccidents?
● Howdoesthisvary...- byregion?- byseason?- bytimeofday?
● Comparefrequencyofcrasheswhichtakeplaceduringtheraintothefrequencyofrainitself
● Intheabsenceofacorrelation,thesefrequenciesshouldberoughlyequal
16
High-ResolutionNEXRADReanalysis
• Blue=climatologicalfrequencyofrainfall
• Red=frequencyofcrashesduringrain
• Allregionsshowahigherpercentageofrain-associatedcrashesthanwouldbeexpectedbasedonclimatology
17
SeaIceConcentrationCDR
• ThePassiveMicrowaveseaiceconcentrationsareanestimateofthefractionofoceanareacoveredbyseaicemean STD
18
SeaIceConcentrationCDR
• Verificationandvalidation(a)CDR-GSFC (b)CDR-BT
(c)CDR-NT (d)BT-NT
bias
biasbias
biasRMSD
RMSDRMSD
RMSD
19
SeaIceConcentrationCDR
WholeArctic
CentralArctic(lightgreen) Greenland(darkgreen) HudsonBay(darkbrown)
SpatialandTemporalVariabilityofSeaIceCoverage
20
Obs4MIPS
• TheObservationsforModelInter-ComparisonProject(obs4MIPs)isanefforttomakeobservationaldataavailabletotheclimatemodelingcommunityinthesameformatthattheyusetoshareclimatemodeloutputs.
• TosupportCDRdataaccess,CICS-NCcreatedatooltoconvertCDRdatatoobs4MIPscompliantformatandspecifications.
21
Obs4MIPS
https://ncics.org/data/obs4mips/
OptimalInterpolationSeaSurfaceTemperature(OISST)
HIRSOutgoingLongwaveRadiation(OLR)
PassiveMicrowaveSeaIceAreaFraction
22
Otherresearchhighlightedhere
• “ScientificStewardship- WhatIsItandWhatDoesItMeantoUs?”– GePeng
• “DiurnalCycleofLandSurfaceTemperaturesunderAll-skyConditions”– Anand Inamdar
• “Long-termHIRS-BasedTemperatureandHumidityProfiles”– JessicaMatthews
• “AnEvaluationofSatelliteQuantitativePrecipitationEstimates(QPEs)Products”– OlivierPrat
• “LinkingnetCDF DatawiththeSemanticWeb-EnhancingDataDiscoveryAcrossDomains”– JimBiard
23
Summary
• NOAA’sCDRsarerobust,sustainable,andscientificallydefensible.
• CICS-NCsupportstheeffortwithscience,scientificprogramming,andstewardshipexpertise.
• Tolearnmore:https://www.ncdc.noaa.gov/cdr