AMSR-E SIPS Processing Status

AMSR-E SIPS Processing Status

Presented by Helen ConoverInformation Technology and Systems Center

at the University of Alabama in Huntsville

AMSR-E Joint Science Team MeetingSept 4-5, 2013

Oxnard, CA

[email protected] www.itsc.uah.edu ghrc.nsstc.nasa.gov 256.961.7807

mailto:[email protected]


http://www.itsc.uah.edu/

http://ghrc.msfc.nasa.gov/

2

Outline

• Introduction• AMSR-E Reprocessing Status

– L2A with Wentz V7 complete– Standard Browse Imagery Updated– Collecting Data Provenance– Inserting Data Object Identifiers

• Global Imagery Browse Services (GIBS) Overview

• AMSR2 Processing Plans

AMSR-E Joint Science Team Meeting Oxnard, CA Sept. 4-5, 2013

3

AMSR-E SIPS Personnel

• NASA MSFC: H. Michael Goodman, PI• UAHuntsville ITSC (Staff shared with GHRC DAAC):

Management: Dr. Sara Graves, Director, ITSCDevelopment: Helen Conover, Lead

Bruce BeaumontEvans CriswellMichael McEniry

Mary NairEngineering: Kathryn Regner

Sandra Harper Data Management: Lamar Hawkins Sherry HarrisonSystems Administration: Will Ellett, Lead Michele Garrett


AMSR-E SIPS Processing and Dependencies

4

Level-2A Brightness Temperatures(with Metadata and QA files)

L2B Ocean(1/2-orbit)

L2B Land(1/2-orbit)

L2B Rain(1/2-orbit)

L3 Ocean(Daily)

L3 Land(Daily)

L3 Snow(Daily)

L3 Sea Ice(Daily, 3 Res)

L3 Snow(Pentad)

L3 Ocean(Weekly)

L3 Ocean(Monthly)

L3 Rain(Monthly)

L3 Snow(Monthly)

Once per day

Once every

five daysOnce per

week

Once permonth

ProductGeneration

at SIPS-GHRC

-------------~ 29 times

per day

Level-2A Brightness Temperatures

L3 Sea IceDrift

~ 29 times per day at SIPS-RSS


5

Algorithm DAP Version Summary

Algorithm Short Name Current Version

L2A Tb V12

L2B Land V10

L2B Ocean V07

L2B Rain V11

L3 Land V07

L3 Ocean V05

L3 Rain V07

L3 Sea Ice V14

L3 Sea Ice Drift B03

L3 Snow V10


Reprocessing

• Earlier this year, the SIPS began the most comprehensive reprocessing effort to date.

• The ESDT Version 3 data set will feature the following enhancements:– L2A products generated with Wentz

Version 7 algorithm– Inclusion of ISO-compliant lineage

information and digital object identifiers (DOIs) in data files

– Updated browse imagery


6

Wentz Version 7 algorithm

• The new brightness temperature products generated at SIPS-RSS – use AMSR-E L1A V03 dataset as source input – use the new RSS Version 7 radiative

transfer model algorithm, intercalibrated with other microwave radiometers, such as SSM/I F13 and WindSat, thus bringing all satellite microwave radiometers to a common calibration standard

– show improved calibration over land, based on heavily vegetated tropical rainforest scenes

– improve Radio Frequency Interference (RFI) Flagging in response to new geostationary sources.


7

Provenance Capture

• All AMSR-E ESDT Version 3 products will include new metadata routines that contain or capture dataset DOIs and ISO Lineage.– The digital object identifier (DOI) uniquely

identifies a dataset. Metadata about the data is correlated with the DOI. The DOI for an object is permanent, whereas its location and other metadata may change.

– Storing ISO-compliant lineage information in data files means that processing history stays with the files.


8

Enhanced Browse Imagery

• All AMSR-E ESDT Version 3 products will benefit from a new browse image process which includes– new 24-bit PNG images with embedded metadata, such

as source science data file name, parameters, date/time stamp, projection, etc.

– KMZ files suitable for display on map or globe– the traditional “framed” image with title, color bar and

annotations • Full resolution browse imagery, data only, will be

provided as map layers to NASA’s Global Image Browse Service (GIBS)– Selected channels or variables from each product,

suitable for overlay on MODIS background images


9

Reprocessing Plan

• L2A reprocessing (addition of metadata and browse image generation for RSS-produced L2A data) completed at SIPS-GHRC in July.

• L2B and L3 datasets will be reprocessed individually throughout 2014-15 as algorithm updates are received from TL-SCF.

• Assuming a final L2A dataset update by SIPS-RSS in late 2015, SIPS-GHRC will perform a full mission reprocessing of all products in 2016 using final L2A as input with latest available science algorithm.


10

Global Imagery Browse ServicesOverview

AMSR Science Team MeetingMatt Cechini, GIBS Lead

Helen Conover, AMSR-E SIPS

AMSR2 Processing Planning at the SIPSPresented by Helen Conover

Information Technology and Systems Center at the University of Alabama in Huntsville

AMSR-E Joint Science Team MeetingSept 4-5, 2013

Oxnard, CA

[email protected] www.itsc.uah.edu ghrc.nsstc.nasa.gov 256.961.7807



http://www.itsc.uah.edu/

http://ghrc.msfc.nasa.gov/

Introduction

• The AMSR-E SIPS is currently negotiating with NASA HQ to do standard and near real-time product generation– SIPS in Huntsville, AL will generate climate

research quality products from JAXA Level-1R data, using algorithms adapted from AMSR-E processing by the US Science Team. These products will be archived at the NSIDC DAAC.

– SIPS will also generate LANCE (Land Atmosphere Near real-time Capability for EOS) products for users who are interested in monitoring natural and man-made hazards in near real time (no archive).


13

Proposed AMSR2 Data Flow


14

TLSCFScience SoftwareIntegration and Test,Metadata & QA software

SIPS-Huntsville

JAXA-EOCL1R Product Generation

L2 and L3 Product Generation(science, metadata, qa, ph, subsets, browse)

NSIDC DAAC L2 and L3

Data Archive and

Distribution

L2 & L3

L1R

L0

Product Delivery Record Server (PDRS)

Science AlgorithmsAlgorithm Teams

Ocean Sea Ice

Rain

Land

Snow

Users

Products and Dependencies


15

L2B Ocean(1/2-orbit)

L2B Land(1/2-orbit)

L2B Rain(1/2-orbit)

L3 Ocean(Daily)

L3 Land(Daily)

L3 Snow(Daily)

L3 Sea Ice(Daily,3 Res)

L3 Snow(Pentad)

L3 Ocean(Weekly)

L3 Ocean(Monthly)

L3 Rain(Monthly)

L3 Snow(Monthly)

Once per day

Once every

five daysOnce per

week

Once permonth

ProductGeneration

-------------~ 29 times

per day

Level-1R Brightness Temperatures

Ingest from JAXA~ 29 times per day

L3 Sea Ice Drift

AMSR2 Near-Real-Time Processing

• NASA’s LANCE system is eager for AMSR2 products. AMSR-E was second only to MODIS with LANCE users:– “Our flood detection system depends on AMSR-E data,

and is used by governments in many countries (e.g., http://www.gdacs.org/flooddetection/ ).”

Tom De Groeve, Joint Research Centre of the European Commission– “At the Cooperative Institute for Meteorological Satellite

Studies at the University of Wisconsin, we make continuous use of the AMSR-E retrievals of total precipitable water shared in real-time on the LANCE servers... Microwave-frequency retrievals of low-level water vapor are usually the earliest indicator available for tropical cyclone development in remote ocean environments.”

Chris Velden, Cooperative Institute for Meteorological Satellite Studies at the University of Wisconsin


16

http://www.gdacs.org/flooddetection/




Discussion

• SIPS plans to use JAXA L1R as input to the US Level-2B and Level-3 algorithms – Already signed GCOM-W1 AMSR2 User Agreement and

registered with JAXA GCOM-W1 data system.– What other operations agreements do we need to

prepare and with whom do we coordinate?– What arrangements do we need to make for routine,

automated data delivery and with whom do we coordinate?

– Can we receive L1R per ½ orbit in near real (similar to L1B delivery to GSFC PPS)?

• LANCE data latency requirement is within 3 hours of observation, but sooner is better…

– What else do we need to consider as we plan to receive Level-1R data directly from JAXA?


17

Thank you!


18

AMSR2 Primary POCs

• Helen Conover, Technical [email protected]

• Kathryn Regner, Systems [email protected]

• Sherry Harrison, Data Management and User [email protected]


19



Backup

TLSCF ScienceSoftware I&T SIPS-GHCC

PO.DAAC

JAXA-EOCL1A Product Generation

SIPS-RSSL2A Product Generation

L2B and L3 Product Generation(science, metadata, qa, ph, subsets, browse)

L2 and L3algorithms

L2A algorithm

NSIDC DAAC Data Archive and Distribution

Level-1

Level-2 and Level-3

Level-1

Level-1

NASAEDOS

GBAD (Pre Ephem) & PDS

GroundStations

L2B and L3 Algorithms

Level-0

Product Delivery Record Server (PDRS)

Algorithm Teams

Ocean Sea Ice

Rain

Land

Snow Data

NASA Flt Dynamics

GBAD(Def Ephem)

AMSR-E Data Flow


22

SIPS-GHCC HardwareOperational Configuration (Mar

‘13)

Multiple processing environments across three machines results in much greater flexibility for processing and

reprocessing.

Priv

ate

Netw

ork

(1 G

B/se

c)

Forward Processing

Reprocessing

Dev, Integration & Test

Sun Fire x4270 x642 x 2.93 GHz Sun Intel Xeon Processors

4 x 2GB DIMM , RHEL 5AMSR 1-3


Near line storage for all Level-2A, Level-2B and Level-3 AMSR-E products

Sun Storage 7310With J4400 Disk

Array10TB NAS

0123

4567

89

10USB

13

1112

1415

REARPS

TOPFAN

Storage 7110

micros ystems

SysSN: 0829Q

ASO

OC

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

OK

Sun Storage 712022TB NAS

Documents

AMSR-E SIPS Processing Status