NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, 2005 1 March 29, 2005 Presentation for

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages

March 29, 20051

March 29, 2005Presentation for

NOAA’s Climate Working Group

Tom Karl, Observations and Analysis Climate Program ManagerJohn Bates, SDS Co-Lead

Mitch Goldberg, SDS Co-Lead

1

NOAA’s Scientific Data Stewardship Program:Implementation, Governance, Performance

Measures and Linkages


March 29, 20052

Outline

BackgroundDefining Scientific Data Stewardship and

CDRsNOAA Scientific Data Stewardship Program

Governance and ManagementProgram Drivers and Performance MeasuresLinkages to Other ProgramsQuestions?


March 29, 20053

Climate Goal/Climate Program Element 1 – Observations and Analysis

Scientific Data Stewardship (SDS) Funds to support SDS

• Reprogrammed from EDSM (ESDIM)

President’s FY06 Total EDSM Budget: $9.384M• Funding shared among several programs including SDS

• SDS from EDSM ~$2.5M

• OGP’s Applied Research Center for Data Set Development

~ $1.4M (Maybe more or less, depending on FY06 distribution)

• FY06 PBA 100% Requirement for SDS is $3.5M

• FY07 - FY12 100% requirement substantially more

Background


March 29, 20054

NOAA’s Scientific Data Stewardship rooted in NRC dialogue and reports

NOAA/NRC SDS leads

– Bates

– Goldberg

Background


March 29, 20055

Data Stewardship is a subset of Data Management and consists of the application of rigorous analyses and oversight to ensure that data sets meet the needs of users (NOSC definition). For environmental satellite observations, SDS priorities include:

A. Observing System Performance Monitoring

i. Documenting measurement practices and processing practices (metadata)

ii. Providing feedback on observing system performance, including recommending corrective action for errant or non-optimal operations.

B. Climate Data Recordsi. Reprocessing (incorporate new data, apply new algorithms, perform

bias corrections, integrate/blend data sets from different sources or observing systems)

ii. Inter-comparison of data sets for validation

Defining Scientific Data Stewardship


March 29, 20056

Climate Data Records

Notional Functions of Scientific Data Stewardship for Climate

Network Performance Monitoring

Observations & Metadata

Analyses and Quality Control

Feedbacks

Stewardship Teams

Archives

Climate Analyses

Reference Data Sets

(Reprocessing)

Climate Quality

Products

Scientific Data StewardshipReal time and retrospective management of climate data

Model Re-analyses

Defining Scientific Data Stewardship


March 29, 20057

Sensor DataRecords (SDRs)

Data (Direct & Remotely Sensed)

Fundamental Climate Data

Records (FCDRs)

Thematic Climate Data Records

(TCDRs)

Climate Data Records or Homogenized Time Series

Homogenization and Calibration

Time-tagged Geo-Referenced

Converted to Bio-Geophysical

Variables

EnvironmentalData Records

(EDRs)

Converted to Bio-Geophysical

Variables

Defining CDRs

Climate Data Records


March 29, 20058

Climate Working

Group

NOAA Science Advisory

Board

Scientific Data Stewardship

CLASS Working Group

Scientific Data Stewardship

Program Management

Research Climate Data Science Teams

FCDR Teams

Observations Scoping

Requirement Systems

C2D2 NCDC ORA

FY06EDSM $2.5M

C2D2 $1 to $1.5MGovernance and Management Structure

TCDR Teams

R&D Products and

Services Theme Areas

NOAA

(5-10%)

External

(1-2%)

NOAA, Other

Agencies External Scientific

(~90%)

NOAA Climate Program & Climate Board

NOAA Observing System Council

Operational CDR Generation & Data Mgmt.

CDR Generation

NOAA

Other Agencies

Universities

Private Sector

CLASS

NOAA IT Infrastructure

operators

Research to

Operations

Currently exists FY05 FY06

NOAA’s Scientific Data Stewardship Program

NOAA’s Scientific Data Stewardship Program:

Implementation, Governance, Performance Measures & Linkages

8


March 29, 20059

Program Drivers

What is …..

– Changing– Causing the Change– The Impact

$


March 29, 200510

Program DriversClimate Program Essential Global Climate Variables (from CCSP & GCOS)

The following essential atmospheric variables are required over land, sea and ice:

1.1 Surface•a. Air temperature•b. Precipitation•c. Air pressure

1.2 Upper-air•a. Earth radiation budget (including solar irradiance)•b. Upper-air temperature (including MSU radiances)•c. Wind speed and direction•d. Water vapor•e. Cloud properties

1.3 Composition•a. Carbon dioxide•b. Methane•c. Ozone

*Including nitrous oxide (N2O), chloroflurorocarbons (CFCs), hydrofluorocarbons (HFCs), sulpher hexaflouride (SF6), and perfluorocarbons (PFCs).

•d. Surface radiation budget•e. Wind speed and direction• f. Water vapor

•d. Other long-lived greenhouse gases *•e. Aerosol properties

1. Atmospheric Variables

•g. Evaporation & evapotranspiration


March 29, 200511

Program DriversEssential Global Climate Variables (from CCSP & GCOS)

2.1 Surface•a. Sea-surface temperature•b. Sea-surface salinity•c. Sea level•d. Sea state•e. Sea ice• f. Current•g. Ocean color (for biological activity)•h. Carbon dioxide partial pressure• i. ocean surface wind & wind stress• j. Surface air temp/humidity•k. Precip (fresh water/salinity flux)• l. Evaporation•m. Fresh water flux from rivers & ice melt•n. CO2 flux across the air sea interface•o. Geothermal heat flux – ocean bottom

2. Ocean Variables

2.2 Sub-surface•a. Temperature •b. Salinity•c. Current•d. Nutrients•e. Carbon• f. Ocean tracers•g. Phytoplankton


March 29, 200512

Program DriversEssential Global Climate Variables (from CCSP & GCOS)

•a. Snow cover•b. Glaciers and ice caps•c. Permafrost and

seasonally-frozen ground•d. Albedo•e. Land cover (including

vegetation type)

3. Terrestrial Variables

•f. Fraction of absorbed photosynthetically active radiation (FAPAR)

•g. Leaf area index (LAI)•h. Biomass•i. Land surface temp


March 29, 200513

Performance Measures for CDRs

(PERFORMANCE)

SOCIETAL IMPACTS

Time-dependent biases

SOFTWARE ENGINEERING

CRITERIASCIENTIFIC

CRITERIA

Reproducibility

Multiple Observing Systems

Multiple analysis teams

Error structure

MetadataData formats and data

modelsData archive & access

Standards, NARA, EGDC,

EFA, etc.Version control

New knowledge

Outreach & network building

New tools & techniques

New products & services

Assessments & decision support


March 29, 200514

Century

Multi-Century to Multi-Millennia

Decadal

Linking Ice Ages to Space Age

Multi-year

Minor Observing Changes

Major Observing Changes

Direct versus Indirect

Observables


March 29, 200515

Performance Measures for Climate Data Records

All CDRs address time-dependent biases and random errors in the data set and are reproducible by independent analysis teams.

BronzeBronze --- --- A single time series produced by a single analysis team from a single observing system (e.g. MSU) or Silver or Gold level failures to meet

SilverSilver --- --- Multiple time series:

– produced by multiple analysis teams based a common observing systemOR

– produced from multiple independent observing systems by a single analysis team (e.g. MSU vs. Radiosondes)

Gold Gold ------ Multiple time series produced by multiple analysis teams from multiple

independent observing systems.

Note: Trends within data sets must be larger than the differences among data sets to reach Gold or Silver status.

Scientific Criteria

Essential Climate VariablesMulti-Century to Multi

MillenniaCentury Decadal Multi-Year

1.1 Surface

a. Air temperature

b. Precipitation

c. Air pressure

d. Suface radiation budget

e. Wind speed and direction

f. Water vapor

g. Evaporation & evapotranspiration

1.2 Upper Air

a. Earth radiation

b. Upper-air temp

c. Wind speed & dir.

d. Water vapor

e. Cloud properties

1.3 Composition

a. Carbon dioxide

b. Methane

c. Ozone

d. Other greenhs. Gas

e. Aerosol properties

Gold Silver Bronze No CDR

Notional Status of Global Scale CDRs - Atmosphere

16

Scientific Criteria

Scientific Criteria

Essential Climate VariablesMulti-Century to Multi

MillenniaCentury Decadal Multi-Year

2.1 Surface

a. SS temp

b. SS salinity

c. Sea level

d. Sea state

e. Sea ice

f. Current

g. Ocean color

h. Carbon dioxide

i. Ocean surface wind & wind stress

j. Surface air temp/humidity

k. Precip (fresh water/salinity flux)

l. Evaporation

m. Fresh water flux from rivers & ice melt

n. CO2 flux across the air/sea interface

o. Geothermal heat flux-ocean bottom

2.2 Sub-Surface

a. Temperature

b. Salinity

c. Current

d. Nutrients

e. Carbon

f. Ocean tracers

g. Phytoplankton Gold Silver Bronze No CDR

Notional Status of Global Scale CDRs - Ocean

17

Essential Climate VariablesMulti-Century

to Multi Millennia

Century Decadal Multi-Year

a. Soil moisture

b. Glaciers & ice caps

c. Permafrost & seasonally frozen ground

d. Albedo

e. Land cover (including vegetation type)

f. Fraction of absorbed photosynthetically active radiation (FAPAR)

g. Leaf area index (LAI)

h. Biomass

i. Land surface temp

Gold Silver Bronze No CDR

Notional Status of Global Scale CDRs – Terrestrial Variables

18

Scientific Criteria


March 29, 200519

• a. fire occurrence • b. volcanic effects (on surface) • c. biodiversity • d. chemical (fertilizer/pesticide & gas exchange)• e. waste disposal & other contaminants • f. earthquakes, tectonic motions • g. nutrients and soil microbial activity • h. coastal zones/margins• i. erosion, sediment transport• j. land surface structure & topography

3. Terrestrial Variables

• a. Organic & inorganic effluents (into ocean)

2. Ocean Variables

Secondary VariablesExternal Forcing or Feedback Variables

Scientific Criteria


March 29, 200520

Working Group Report on Implementation Plans for the CCSP Deliverable:

CDR production enablesCDR production enables:: Identification, reduction and removal of discontinuities in Re-analyses

Reducing impact of changes in the observing system

Linkage to Other Programs

Re-analysis

“Re-analyses of historical climate data for key atmospheric features. Implications for attribution of causes of observed

change”


March 29, 200521

General CDR Development Quality control and bias correct

satellite radiance data

Develop, acquire, and quality control needed forcing data sets, e.g. SST, land use, solar variability, green house gases, aerosols

Homogenize key observations, especially upper air data that have suffered from a rapidly changing observing system



March 29, 200522

Modern Satellite Era (1979-present)

Upper-air Era (approx. 1950-present)

Surface Era (approx. 1900-present)


CDR Relevant Strategies & data streams aimed at developing improved Re-analyses for:

Linkage to Other GEOSS Programs

23

International Project Management, (GEO)

Regular Monitoring Reports for NOAA’s

Earth Observation Partnership of the America’s (EOPA)

Activities/Products

Improved Data Analysis

CDRs

Model Reanalysis

State of “…” Monitoring Reports

User Requirement

Document

Programmatic Guidance

(e.g., GEO, GCOS, GTOS,

WHYCOS, IGBP, IGOS, etc.Data Management

Climate Liaison Team (Stakeholder Consultation)

Application of Data ProductsImproved Earth System Understanding and Forecasting

Applications & Assessments

Validation of Earth System Modeling

Projections & Prediction (including Extreme Events)

Atmosphere

Terrestrial

Oceans

Cryosphere

24

IEOS Near-Term Deliverables and Potential Regional Linkages

Drought (NIDIS)

Air Quality

Sea Level

Land Use/Land Cover

Natural Hazards

IWGEO Near-TermDeliverables

Integrated Data sets

24


March 29, 200525

Questions

Collection- Internet, private networks, satellite, media

Ingest/Management- Collect/create/maintain metadata- Catalog and inventory

Quality Control- Basic-missing data, identifying and correcting suspected errors including - Model/Data intercomparison- Observing system performance monitoring- Developing Climate Data Records

Preservation/Archive- Sort/Reformat- Storage- Backup

Data Management +(Analysis)

{CLASS

{ScientificData

Steward-ship

Analysis- Monitor climate indicators- Data products (including integration)- Reprocessing in time and space (means and extremes)- Data filtering (filter high frequency noise)- Trend analysis

Discovery/Access- Interactive browse on-line, near-line- Via customer services

Data Delivery- On-line, Internet- Media

{CLASS

Overarching the above data management components are interoperable systems, effective user feedback and standards/protocols

{CLASS

Documents

NOAA’s Scientific Data Stewardship Program: Implementation, Governance, Performance Measures & Linkages March 29, 2005 1 March 29, 2005 Presentation for