NOAA NDE COPC Meeting Offutt AFB November 13, 2008 1

NDETitle Page

NDE

NPOESS Data Exploitation – NESDIS

James G. (Jim) Yoe, NESDIS/OSDNDE Integration Coordinator

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 2

NDEOverview

• NESDIS NDE Review/Overview• COPC Action Item 2008-1.10

– Response and Recommendation• NDE Project Status• Summary – Issues

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 3

NDENDE Mission and Objectives

Deliver NPOESS data products and assist NOAA and other civilian operational users to realize the potential of NPOESS observations

• Disseminate NPOESS Data Records to real-time civilian users• Generate and disseminate tailored NPOESS Data Records• Generate and disseminate NOAA-unique products (enhanced

data products constructed from NPOESS Data Records) • Deliver NOAA-unique products, product processing elements,

and associated metadata to CLASS for long-term archiving • Provide services to customers, including NDE product training,

product enhancement, and implementation support across NOAA• Provide software for NPOESS Data Record format translation

and other data manipulations

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 4

NDENPOESS & NDE

NPOESS DataExploitation

SafetyNet

NPOESS Ground Systems

Command and Control

Operational User

CommunityArchive and Access

NPP

NPOESS C1

NDE EnhancedProducts

Quality Control

Near Real-time Delivery

GenericNPOESS

Data

NOAAScience Center

(operated in ESPC)

Process

Ingest

NDE will be a critical link to achieve return on the NPOESS investment – beginning with NPP

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 5

NDE NDE Management Team

AWIPS

Design Partners

NDEProduct

Development Teams

(STAR & Others)

ESPC

InfrastructureAcquisition,Operations(OSDPD)

NDEProject

ManagerJim Silva

Data CentersCLASS

STAR

NDESystem Architect

Geof Goodrum

NOAASystem Integration

Teams(NCEP, CoastWatch,

AWIPS & Others)

SystemDevelopment

Team(OSDPD)

NDESystems

IntegrationCoordinator

Jim Yoe

NDEProduct

Development Manager

Tom Schott

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 6

NDECOPC Action Item 2008-1.10

• The prioritization of NESDIS NDE Initiatives as it relates to the time phasing of the NPP/NDE, requires further coordination. The NESDIS should work more closely with DoD partners and the JCSDA to ensure coordination of priorities. NESDIS should provide a review of these initiatives at the next COPC meeting.– AI arose from COPC reaction to the NPP Product

Development phasing efforts (see next slides).

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 7

NDENDE Product Development

• NDE supporting 3 major NOAA-Unique Product (NUPs) development lines – CrIS/ATMS Thinned Radiance– Microwave Integrated Retrieval System (MIRS)– Sea Surface Temperature (SST)

• Phased Approach for NPP– Phase I to provide legacy products for NOAA

mission continuity– Phase II to provide other legacy products

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 8

NDENPP Phase 1

CrIS Thinned Radiances SST Analysis Sea Surface Temperature (SST)

CrIS Cloud Cleared Radiances Blended SST Nadir Profile Ozone

Total Precipitable Water (ATMS) SST Anomalies Ozone Total Column

Snow Cover (ATMS) Coral Reef Degree Heating Snow Cover and Depth

Precipitation Type/Rate (ATMS) Coral Reef Bleaching Imagery

Surface Emissivity (ATMS) Total Ozone (CrIS) Ocean Color/Chlorophyll

Cloud Liquid Water (ATMS) Trace Gases (Carbon) Vegetation Index

Sea Ice Cover/Concentration (ATMS) SST (AVHRR-like) Active Fires

Snow Water Equivalent (ATMS) Aerosol (AVHRR-like) Atmospheric Temperature Profile

Ice Water Path (ATMS) ATMS Radiances Atmospheric Moisture Profile

Land Surface Temperature (ATMS) CrIS Radiances Aerosol Optical Thickness

Temperature Profiles (ATMS) VIIRS Radiances Land Surface Type

Moisture Profiles (ATMS) OMPS Radiances Surface Albedo

Rain Water Path/Profile (ATMS) Cloud Mask Cloud Cover/Layers

Y NUPs

B xDRs

Phase 1 (3-20 months after NPP Launch): Legacy mission continuity replacement products comprised of currently funded NOAA Unique Products (NUPs) and xDRs

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 9

NDENPP Phase 2

Polar Winds (VIIRS) Fire Potential Cloud Top Height (VIIRS)

Limb Profile Ozone Near Coast Ocean Color Ice Surface Temperature

Blended Snow Cover Integrated xDRs at CrIS Resolution Net Heat Flux

Harmful Algal Blooms (VIIRS) Cloud Liquid Water Path (VIIRS) Sea Ice Characterization (VIIRS)

Blended Ozone Cloud Ice Water Path (VIIRS) Suspended Matter

Blended Total Precipitable Water Cloud Top Temperature (VIIRS) Atmospheric Pressure Profile

Tropical Rainfall Potential Aerosol Particle Size Quarterly Surface Type Gridded

Clear Sky Radiances (VIIRS) Cloud Top Temperature Soil Moisture

Vegetation Health Cloud Top Pressure

Vegetation Moisture Land Surface Temperature (VIIRS)

Drought Indices Cloud Base Height

Vegetation Thermal Conditions Cloud Effective Particle Size

Leaf Area Index Cloud Optical Thickness

Phase 2 (20 months after NPP Launch to 1 month prior to C1 launch): Additional legacy products and enhanced products comprised of NUPs and xDRs not linked to mission continuity

Y NUPs

B xDRs

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 10

NDENDE Response to AI 2008-1.10

• JCSDA Director presented NDE’s NPP product phasing to Executive Team (DoD, NASA, & NOAA)– With understanding that ability to re-prioritize NUP development

sharply limited by need to support NWP, funding availability, and time

• JCSDA partners most interested in obtaining– CrIS/ATMS radiances– VIIRS Atmospheric Motion Vectors (AMV winds)

• Note that NOAA is supporting development of AVHRR AMVs w/o water vapor channel; if successful will apply to VIIRS

• NOAA NDE also has identified additional “internal NUPs” which may be made available to DoD – CrIS/ATMS Principal Components from radiances - USN

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 11

NDENDE Response to AI 2008-1.10

• JCSDA ranks NPP preparation as highest priority– Prior to NPP launch, JCSDA will

• Ingest simulated datasets to ensure that file formats and processing chain including CRTM (Community-based radiative transfer Model) are ready

– Immediately after launch, JCSDA will• Start monitoring data quality of CrIS and ATMS against

background and analysis fields; diagnostics will be made available to NPP team but not to the general public

– Excerpted from L.P. Riishojgaard’s presentation to Sounding OAT

• Recommendation: Close the Action Item

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 12

NDENDE Project Status

SYSTEM DEVELOPMENT : Subsystem Builds

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 13

NDE

• Testing w/NPOESS Ground System– System Interface Tests ahead

• ICT 14 (C3S): ~20 Jan 2009• ICT 10 (IDPS): ~9 Mar 2009• ICT 13 (Ancillary Data): ~May 2009• NCT 3 (EtE): ~15 June 2009

– COTS Software Requirement for Java API• Borland Visibroker licensing

• IT Security– Identify NIST SP800-53 Security Controls

• Common and NDE System Specific• Due 31 Oct 2008

NDE Architecture Status

13

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 14

NDENDE Architecture Status

• IDPS Face to Face (8-9 Oct)– Performance improvements (4 copy at NESDIS)– Prototyping software for NPOESS– Evaluating hardware for NPOESS

• Shared Architecture– Presented NDE to GAS IPT as notional architecture for ESPC (16

Oct)– Update and present at ESPC Evolution meeting (19 Dec)

14

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 15

NDENDE Implementation

• Progress– Data Handling System Development;

• Successful Ingest Demonstration at the September 26th NDE Quality Review• Completed NDE Tailoring Tool Evaluation and Analysis

– IDPS to NDE testing support;• Continued support of IDPS interface testing• Integrated, configured, and tested SAN disks with previously received 9222i

Cisco switches• Received two Dell servers to support StorNext Meta Data Control (MDC)• Estimate shipment of MDC RAID from LSI on October 29th

– Created the NDE Subsystem Boundary Diagram for security documentation

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 16

NDENDE Implementation

• Plans– Continue work with IDPS on interface testing;

• Continue to receive and configure network, disk storage, and metadata servers hardware, and StorNext software

– Test Readiness Review for the Ingest Subsystem scheduled for November

– Begin development of the HDF to Net CDF Data Reader Prototype in November

– Define the NDE Security Controls by the end of November

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 17

NDE

NDE Systems IntegrationInterface Development

• NDE-ESPC ICD– Signature version delivered 11/7/2008– Goal: signed ICD by 2/3/2009

• NDE-CLASS ICD– Signature version delivered for review 10/24/2008– Goal: signed ICD by 2/3/2009

• NDE-STAR/CE ICD– First Draft being prepared based on input from 7/7/2008

• NDE-NWS/TOC ICD– NDE-SIC met with new TOC Director 10/6/2008

• Joint NDE-TOC team to outline way forward• Need to determine what document(s) needed between TOC-AWIPS

Technology Infusion, and NDE-AWIPS-TI

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 18

NDESystem Integration - Users

• AWIPS Technology Infusion Project– Coordinate schedules, architectures, smart push-pull for data– Work with NWS HQ to identify product requirements

• Collaboration with Hydrometeor Testbed– HMT gets enhanced data to support forecasts/warning– NDE gets data/expertise to validate precipitation products

• Outreach and Training– Identify new applications and new users– Develop/support training modules for products– Exploit existing resources

• COMET, NWS Training Division, VISIT, SHYMET

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 19

NDENDE -COPC Issues

• Distribution of NPP NUPS to COPC Partners– Determine detailed requirements – formats, volumes, etc.– Establish interim methodology & protocol

• NDE Distribution Build will not be complete until 2011– Adequacy of Network

• Encourage co-development by NDE and the DoD Centrals– For data product development

• NDE and NOAA CoastWatch users would like VIIRS proxy data– For tools and system elements

• NDE data tailoring tools may be of interest to AFWA– For (user) training

• Back-up options for ingest data, processing, and distribution in the NPOESS era

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 20

NDENPP Status

• VIIRS status– Senior Government Manager on site at Raytheon, appointed

as COTR– EMI testing regression ongoing– April 2009 delivery to NPP doubtful

• NASA maintains LRD 14 months after VIIRS delivery• Tri-agency Team established to evaluate several

NPP options– NDE prepared to evaluate impacts on current baseline and

identify its resource requirements for each option

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 21

NDEBack-Up Slides

NOAA NDE COPC Meeting Offutt AFB November 13, 2008 22

NDENPP Reformatting to Support JCSDA

Prioritized Product BUFR GRIB2 Comments

ATMS Radiances X   BUFR Table expected end Nov 2008 

CrIS Radiances X   BUFR Table expected end Oct 2008

Nadir Profile Ozone (OMPS) andOMPS (NP) Radiances

X  

VIIRS Radiances X    

Snow Cover   X

Vegetation Index   X  

Aerosol Optical Thickness X    

Sea Surface Temperature X