Tools Developed For AVHRR Data Production and Analysis The AVHRR modular framework can produce three...

Tools Developed For AVHRR Data Production and Analysis

The AVHRR modular framework can produce three output datasets in NetCDF format when AVHRR NetCDF inputs are used

1)AVHRR observations, cloud property retrieval, and cloud mask with necessary ancillary fields required for clear sky OH albedo updating2)AVHRR observations, cloud mask, and debug parameters3)Ancillary fields (i.e. snow mask, surface elevation, IGBP, etc…) required for NOAA-9/10 ERBE radiation CDR

Tools suggested by team members and developed for product production and analysis

•Scripting infrastructure on AMI to 1) pull AVHRR LV1B files from cloudsgate1 NAS, 2) convert to NetCDF, 3) farm out to a user-defined maximum number of jobs, 4) each job process orbit in 1000 scanline chunks, 5) write NetCDF file types 1 and/or 2 above, and 6) push output back to cloudsgate1 NAS

- Files internally compressed and most parameters written as short integers

1)IDL quicklook image creator for file types #1 and #2 above

2)Web-based quicklook image viewer with animation, fade, and toggle tools

3)Fortran monthly average gridding software for 8 parameters (cloud fraction, cloud effective/top height/pressure/temperature, optical depth, particle size), separated by day/night, water/ice, land/water/snow or ice land surface

4)IDL monthly average image viewer for modular, CERES, CALIPSO, and PATMOS-X outputs

5)Fortran cloud fraction analysis software where cloud fraction is computed for each IGBP type, sunglint yes/no, and day/night

ORBIT 1

ORBIT 2

ORBIT 3

….

ORBIT 15

DAY 1

ORBIT 1

ORBIT 2

ORBIT 3

….

ORBIT 15

DAY 2

30 Jobs Processing Orbit 15 Times In

1000 Scanline Chunks

AMI Processing With Standard NetCDF-4

Update Clear Sky Reflectance Map

Standard NetCDF does not allow 2 jobs to simultaneously write to same NetCDF file

We cannot continue to process Day 3 until previous 2 days are completed and clear sky OH albedo maps are updated

Process Days 3 and 4

Chunk 2Chunk 3…..Chunk 15

ORBIT 1

ORBIT 2

ORBIT 3

….

ORBIT 15

DAY 1

ORBIT 1

ORBIT 2

ORBIT 3

….

ORBIT 15

DAY 2

Up to 450 Simultaneous

Jobs Processing 2 Days Worth of

Orbits

AMI Processing With Parallel NetCDF-4

Update Clear Sky Reflectance Map

Chunk 1 Chunk 1Chunk 2Chunk 3…..Chunk 15

Parallel NetCDF allows simultaneous writes

We distribute jobs in 1000 scan line chunks across processors, greatly accelerating our processing

Parallel NetCDF has been installed on AMI, but I have not yet figured out how to use it

Process Days 3 and 4

Monthly Mean Datasets To Be Shown In The Following Slides

1) Modular AVHRR framework using late 2011 mask and pre-Ed4 cloud retrievals from October 2008 and October 1986

2) CERES MODIS Edition 4 spring 2012 delivery gridded output from October 2008

3) PATMOS-X AVHRR produced by UW-CIMSS for 2010 GEWEX cloud assessment from October 2008 and October 1986

4) CALIPSO 5-degree cloud fraction from October 2008 via Qing

October 2008 Results

Modular Using CERES October 2008Clear Sky OH Albedo Map

Modular Using AVHRR October 2008 Snow and Non-Snow Clear Sky OH Albedo Map

Modular Using CERES October 2008Clear Sky OH Albedo Map

NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to

significantly impact optical depth retrieval

Modular Using AVHRR October 2008 Snow and Non-Snow

Clear Sky OH Albedo Map

NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to

significantly impact optical depth retrieval

NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to significantly impact optical depth retrieval

NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to significantly impact optical depth retrieval

October 1986 Results