Update on COPC Action Item 2008-1.5

Presented byDr. Bradley Ballish

Co-Chair JAG/ODAA andMember of Dropout Team

17 May 2011COPC Meeting

AFWA

May 17, 2011 COPC Dropout Update 2

Outline

• COPC Action Item (AI) 2008-1.5 (real-time monitoring)

• Quad chart and status of work

• Forecast-Forecast correlations

• Extreme analysis differences

May 17, 2011 COPC Dropout Update 3

AI 2008-1.5COPC Action Item 2008-1.5: Develop a monitoring

system to analyze differences between the NCEP and FNMOC global models and the ECMWF global model in real-time and make this real-time system available to OPCs as a daily tool.

Initial Planned Deliverables• NCEP will generate warnings on a restricted website

when the GFS global analysis has extreme localized differences from ECMWF analyses on a 1x1 degree grid

• NCEP will develop a real-time warning system to alert us when the GFS global model 5-day forecast height correlations with the ECMWF exceed normal limits, which will give an early warning of a possible dropout

• NCEP website will have graphics to allow staff to study the divergence in forecasts in real time and show likely analysis problem areas

May 17, 2011 COPC Dropout Update 4

AI 2008-1.5Status

• Project is getting close to initial operating capability, see quad chart on next page

• A number of slides will follow showing more on the status and usefulness of this work

• Recommend AI remain open

May 17, 2011 COPC Dropout Update 5

NCEPGFS Forecast Divergence vs. ECMWF - PMO124

Project Status as of 4/6/2011

Issues/Risks

Issues:Other tasks and projects have higher priority causing the schedule

to slipRisks: The correlations cannot be computed in real-time resulting in no

viable information for the forecasters.Mitigation:Conduct testing at key points in the development to determine

viability of the project and the results.

G Finances

SchedulingProject Information (Scope/Benefits)

Lead: Scott JacobsScope: Provide a R/Y/G indication, on a web site, to the users when GFS

global 5-day forecast height correlations, compared to the ECMWF forecasts, are beyond the given criteria

Provide graphics on a web site to show the GFS global forecast correlations

Create the criteria and the software to support the web siteCreate work flow process for when a significant difference in the

forecasts occursExpected Benefits:Better forecastsBetter quality controlAbility to predict low forecast correlationsTraining and action plan for the users

Associated Costs:No hardware or software costsLabor: 4 person-months (625 person-hours) from personnel in NCO,

for a duration of 6 calendar monthsBrad Ballish – 20%Krishna Kumar – 20%Shucai Guan – 10%Joe Carr / Kelly Kempisty – 10%Scott Jacobs – 5%

Funding Sources: N/A

Management Attention Required Potential Management Attention Needed

On TargetGYRv1.0 10/02//06

Milestone Date Status

Project Start 9/20/2010 Complete

Requirements/Design for support software

10/10/2010 Complete

Requirements/Design for displaying results

10/20/2010 Complete

Support software 12/15/10 Complete

Testing of support software 1/14/2011 Complete

Software to present the information 1/21/2011 Complete

Sponsor briefing 2/16/2011 Complete

Website available 4/11/2011 In Progress

Testing of website 4/15/2011

Training documentation and class outline 4/15/2011

Project Close / Lessons Learned 4/29/2011

YG

G

May 17, 2011 COPC Dropout Update 6

Status and Results from Work on COPC AI-2008-1.5

• ECMWF-GFS forecast-forecast correlations are working very well

• GFS forecast-forecast correlation with previous GFS not quite ready

• Extreme analysis difference code and related plotting codes are proving useful

• Code to plot forecast divergence versus time is not ready

• Website to display results not quite ready

May 17, 2011 COPC Dropout Update 7

F-F Correlation Dec. 2010

Predicting low GFS forecast skill valid dates 16 Dec. 2010 12Z and 3 Jan. 2010 12Z

ECMWF-GFS

May 17, 2011 COPC Dropout Update 8

Anomaly Correlation Verification for 11 Dec. 2010 12Z

GFS low-skill is consistent with low F-F correlation

Extremes code shows important finding for this case in later slides

May 17, 2011 COPC Dropout Update 9

Anomaly Correlation Verification for New Year 2011

GFS dropout (< 0.7) predicted by F-F correlation

May 17, 2011 COPC Dropout Update 10

T700

Website will have a graphic like this to show where and atwhat levels and variables there were extreme differences

There will be more website options below to view graphics

May 17, 2011 COPC Dropout Update 11

The extremes code diagnosed this large analysis minus guess change, that appearsto lead to a forecast dropout

This example is important as we never expected satellite mass-field data to havesuch large impact

May 17, 2011 COPC Dropout Update 12

These large analysis wind changes were diagnosed by the extremes code

Radiosonde 61415 had deep very suspect winds that would have been rejectedby profile QC as done at the ECMWF

May 17, 2011 COPC Dropout Update 13

This is a case with large amplitude wind differences over a large areafor a period of time – case is still being analyzed

We are analyzing Mars, with ECMWF analyzing Venus

May 17, 2011 COPC Dropout Update 14

Background Slides

May 17, 2011 COPC Dropout Update 15

00Z 5-day AC Scores in March/April 2011

Example of low score in NH,which has not happened oftenwith the latest GFS model

Dropouts in SH are still too frequent

May 17, 2011 COPC Dropout Update 16

New adjoint code for impactof observations on the analysisshowed the GPSRO data washaving large impact

May 17, 2011 COPC Dropout Update 17

Dropout Related project areas

• Data QC and data usage

• Dropout analysis

• Analysis and model changes

May 17, 2011 COPC Dropout Update 18

Dropout Related ProjectsProject Status

Using Quickscat QC Flags Implemented 2009

Aircraft Track-Checking Implementation of NRL code Q2 2011

Major Dictionary Corrections Upper air changes in operations 2009 North American surface changes implemented in June 2010

NWSTG waypoint updates in Feb 2011

Observational Data Impact Tests Continuous testing at NCEP plus Langland has started tests

Adjoint Estimates of Observation

Impact

Langland has been running tests and NCEP has adjoint of GSI analysis running with help from NASA/Goddard

Surface Pressure Bias Corrections Preliminary testing

Data QC and Data Usage

May 17, 2011 COPC Dropout Update 19

Dropout Related Projects

Project Status

Aircraft Temperature Bias Corrections Waiting for NRL aircraft QC upgrade -

Satellite Wind QC Upgrades Both speed dependent QC and ECMWF filtering rules showed positive impact - NCEP implementation TBD

Profile QC Preliminary investigation

Improved Reject-list Preliminary investigation

Use-list for Conventional Obs Preliminary testing

Data QC and Data Usage

May 17, 2011 COPC Dropout Update 20

Dropout Related Projects (cont)

Establish Dropout Investigation Procedures

Is done and performed for dropout cases

Dropout Relation to Data Counts RTDMS extended to 30 days with January 2011 implementation

Dropout Climatology Studies Common origin areas found – Langland has shown preferred synoptic patterns

ECM Runs Journal publication in old hold– High resolution ECMWF input being tested

Improved Diagnostics for Analysis Differences for Dropouts

Preliminary investigation at both NCEP and FNMOC

Dropout Analysis

May 17, 2011 COPC Dropout Update 21

Dropout Related Projects (cont)

GSI Analysis Upgrades Operational GSI updates Q2 2011 implementation

GFS T574 Model Bundle Implemented on 27 July 2010

Analysis and Model Changes

May 17, 2011 COPC Dropout Update 22

Interagency Participants• NCEP/EMC

– Jordan Alpert– Yangrong Ling – DaNa Carlis

• NCEP/NCO– Bradley Ballish– Krishna Kumar– Joe Carr

• NCEP/HPC– James Cisco

• NRL– Rolf Langland

• FNMOC– Chuck Skupniewicz

• NESDIS – Kathy Kelly– Tom Renkevens

• The NCEP dropout team meets weekly with the EMC director, John Derber and sometimes with the JCSDA director and reports quarterly to the NCEP director

May 17, 2011 COPC Dropout Update 23

Latest Dropout Team Findings (continued)

• For more information on dropout research, see the reports presented at the AMS meetings:

http://ams.confex.com/ams/89annual/techprogram/paper_142644.htm http://ams.confex.com/ams/89annual/techprogram/paper_142649.htmhttp://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154268.htmhttp://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154282.htm

May 17, 2011 COPC Dropout Update 24

• Based on the code from the NASA/Goddard/GMAO

GSI, the dropout team has the GSI analysis adjoint code running

• Further work is needed to develop an adjoint of the NCEP global model that with the analysis adjoint will produce estimates of how every observation helps or hurts the forecast skill

• We will determine if smart statistics from these promising new tools will give us more precision on how to better use observations, such as:– Satellite radiance data impact estimates by type, channel,

surface conditions (water, ice, etc.) and more– Possibly better rules on filtering of satellite winds– Improved reject-list criterion– Possibly better analysis of dropout cases

Adjoint Codes