1
Introducing NOAA's Climate Data Record Best Practices into NCEI's Center for Weather and Climate (CWC) Products Daniel Wunder, Gary Ellingson, Candace Hutchins, and Brian Newport, Global Science & Technology, Inc. • Jeff Privette, NOAA/NCEI • Terry McPherson, LMI Poster: 877 January, 2017 Introduction NOAA’s Climate Data Record (CDR) Program was established at the National Centers for Environmental Information (NCEI) (formerly the National Climatic Data Center) in order to develop and implement a robust, sustainable, transparent, and scientifically defensible approach to producing and preserving authoritative climate records from satellite data Since its inception in 2009, the CDR Program has transitioned 37 CDRs developed by various research groups to an initial operating state at NCEI For transparency and scientific defensibility, the CDR dataset, documentation, and source code are preserved in NCEI’s archive and made accessible to the public Configuration Control is Necessary for Scientific Integrity Tailored Process to Baseline an Operational Product Tailor existing processes as necessary to encompass a much broader suite of products Rank products based on user needs, economic impact, and compliance with directives Initiate portfolio management across similar products and services Maintain the core values of transparency and scientific integrity Implementing across CWC Routine extensions of the operational product are made while allowing for systematic development and deployment of algorithm improvements A change request (CR) is submitted against the baseline version when significant improvements are achieved The configuration control board (CCB) approves/disapproves the CR after evaluating the level of improvement vs. cost of implementing and sustaining a new version CDR Website: https://www.ncdc.noaa.gov/cdr 50 CDRs Tailored processes 150 CWC Products Hundreds of NCEI Products Apply Lessons Learned National Oceanic and Atmospheric Administration | National Centers for Environmental Information Research to Operations Steps 1) Assessment Submission Agreement Data Documents Source Code KDP* Algorithm Developers Document Header Information 2) Submission Sample Data Transfer 3) Transfer Source Code Transfer Enter Code into Version Control Software Documents Transfer Main Data Transfer 4) Validation Validate Code Package Validate ‘Read MeValidate Data Flow Diagram Validate Maturity Matrix Validate Metadata Validate Data Code Package Archived Documen- tation Archived Product Data Archived Ancillary Data Archived 5) Archival 6) Access Code Made Available to Public Data Made Available to Public CDR Assessment *ORR = Operational Readiness Review *KDP = Key Decision Point Documen- tation Made Available to Public Step 1: Assessment Determine if a candidate research product is sufficiently mature for transition to operations. Step 2: Submission Identify the complete package (code, documents, and data) in a formal Submission Agreement, such that NCEI can plan for archive and access of the materials. Step 3: Transfer Transfer the complete dataset, source code, and documentation materials to NCEI for Validation. Step 4: Validation Integrated Product Team (IPT) ensures the source code, documentation, and data are consistent with the current coding and metadata standards. Step 6: Access Conduct an ORR*, then provide a fully transparent and scientifically defensible product to the public. Available at: www.ncdc.noaa.gov/cdr/operationalcdrs.html Step 5: Archival Product is put under version control and submitted to NCEI archive. This initiates stewardship for a minimum of 20 yrs. Climate Data Record: A time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change (National Research Council, 2004) ORR*

AMS supported meetings - DanielWunder, Gary Ellingson ......DanielWunder, Gary Ellingson, Candace Hutchins, and Brian Newport, Global Science & Technology, Inc. • Jeff Privette,

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: AMS supported meetings - DanielWunder, Gary Ellingson ......DanielWunder, Gary Ellingson, Candace Hutchins, and Brian Newport, Global Science & Technology, Inc. • Jeff Privette,

Introducing NOAA's Climate Data Record Best Practices into NCEI's Center for Weather and Climate (CWC) ProductsDaniel Wunder,GaryEllingson,CandaceHutchins,andBrianNewport,GlobalScience&Technology,Inc.•JeffPrivette,NOAA/NCEI•TerryMcPherson,LMI

Poster:877

January,2017

Introduction• NOAA’sClimateDataRecord(CDR)ProgramwasestablishedattheNationalCentersfor

EnvironmentalInformation(NCEI)(formerlytheNationalClimaticDataCenter)inordertodevelopandimplementarobust,sustainable,transparent,andscientificallydefensibleapproachtoproducingandpreservingauthoritativeclimaterecordsfromsatellitedata

• Sinceitsinceptionin2009,theCDRProgramhastransitioned37CDRsdevelopedbyvariousresearchgroupstoaninitialoperatingstateatNCEI

• Fortransparencyandscientificdefensibility,theCDRdataset,documentation,andsourcecodearepreservedinNCEI’sarchiveandmadeaccessibletothepublic

ConfigurationControlisNecessaryforScientificIntegrity

TailoredProcesstoBaselineanOperationalProduct

• Tailorexistingprocessesasnecessarytoencompassamuchbroadersuiteofproducts• Rankproductsbasedonuserneeds,economicimpact,andcompliancewithdirectives• Initiateportfoliomanagementacrosssimilarproductsandservices• Maintainthecorevaluesoftransparencyandscientificintegrity

ImplementingacrossCWC

• Routineextensionsoftheoperationalproductaremadewhileallowingforsystematicdevelopmentanddeploymentofalgorithmimprovements

• Achangerequest(CR)issubmittedagainstthebaselineversionwhensignificantimprovementsareachieved• Theconfigurationcontrolboard(CCB)approves/disapprovestheCRafterevaluatingthelevelof

improvementvs.costofimplementingandsustaininganewversion

CDRWebsite:https://www.ncdc.noaa.gov/cdr

50CDRs

Tailoredprocesses150CWCProducts

HundredsofNCEIProducts

ApplyLessonsLearned

NationalOceanicandAtmosphericAdministration| NationalCentersforEnvironmentalInformation

ResearchtoOperationsSteps

1)Assessment

SubmissionAgreement

Data

Documents

SourceCode

KDP*

AlgorithmDevelopers

DocumentHeader

Information

2)Submission

SampleDataTransfer

3)Transfer

SourceCodeTransfer

EnterCodeintoVersionControlSoftware

DocumentsTransfer

MainDataTransfer

4)Validation

ValidateCodePackage

Validate‘ReadMe’

ValidateDataFlowDiagram

ValidateMaturityMatrix

ValidateMetadata

ValidateData

CodePackageArchived

Documen-tation

Archived

ProductDataArchived

AncillaryDataArchived

5)Archival 6)Access

CodeMadeAvailableto

Public

DataMadeAvailableto

Public

CDRAssessment

*ORR=OperationalReadinessReview

*KDP=KeyDecisionPoint

Documen-tation MadeAvailableto

Public

Step1:AssessmentDetermineifacandidateresearchproductissufficientlymaturefortransitiontooperations.

Step2:SubmissionIdentifythecompletepackage(code,documents,anddata)inaformalSubmissionAgreement,suchthatNCEIcanplanforarchiveandaccessofthematerials.

Step3:TransferTransferthecompletedataset,sourcecode,anddocumentationmaterialstoNCEIforValidation.

Step4:ValidationIntegratedProductTeam(IPT)ensuresthesourcecode,documentation,anddataareconsistentwiththecurrentcodingandmetadatastandards.

Step6:AccessConductanORR*,thenprovideafullytransparentandscientificallydefensibleproducttothepublic.Availableat:www.ncdc.noaa.gov/cdr/operationalcdrs.html

Step5:ArchivalProductisputunderversioncontrolandsubmittedtoNCEIarchive.Thisinitiatesstewardshipforaminimumof20yrs.

ClimateDataRecord:A timeseriesofmeasurementsofsufficientlength,consistency,andcontinuitytodetermineclimatevariabilityandchange (NationalResearchCouncil,2004)

ORR*