WDSS-II Training Module IV

Algorithms and Tools

General Notes Output from WDSS-II applications may be

shared across multiple machines Any application can use the output of

another application as input The wg display is an example of this It provides input/launch to the “Filter”

algorithms It uses products from other algorithms

Real-time and “data playback” modes are essentially the same modes of operation

WDSS-II application types Data ingest applications (“ingestors”) Single-source algorithms

Usually single-radar applications Multi-source algorithms

Combine input data from multiple sources of one or more instrument types

General use tools Data filters, objective analysis tools, data

remapping, data converters, verification tools, etc.

WDSS-II primary data types LatLonGrid: geographic projection

Equal spacing in degrees latitude and longitude RadialSet: cylindrical projection

Accommodates any number of radials with variable radial widths

PolarGrid: an indexed RadialSet DataTable: for point data

Trends tracks

CartesianGrid: equidistant projection equal spacing in N/S/E/W directions

Other types to be described in a later presentation

Data ingest

Data-ingesting programs read “raw” data files and convert them to one of the internal WDSS-II formats New input types are easy to add Maintains a consistent internal

structure for data sharing among applications

WDSS-II Real-time data flow

ldm2netcdf

Reflectivity Velocity Sp. W.

w2qcnn

ReflectivityQC

swatScit2D

Scit2D (table)

w2circ

AzShear

Divergence

AzShear layers

Single-radar products

WSR-88D data (level 2)

netssap

CellTable

MesoTable

TvsTable

RUC analysis data (grib)satellite data*

gribToNetcdf

nse1w2cloudcover

*Satellite data are required to be in netcdf format.

w2vil

Reflectivity OR ReflectivityQC

w2hail

MESH

POSH

MESHTracking

Echo Tops (H_*)

VIL

Comp. Ref.

Other optional algorithms

Dashed lines represent optional inputs, data sources, or applications

Applications are in boxes

Data sources are in ovals

Legend

1If nse is not used as an input, then PolarHail.xml and ssaparm.dat should be updated twice daily. It is highly recommended to use nse data if accurate hail guidance are desired.

The most-used single-source algorithms

w2qcnn: quality control neural network May use radar-only data, or radar plus cloud

cover information Output: ReflectivityQC &

ReflectivityQComposite http://

cimms.ou.edu/~lakshman/Papers/qcnnjam.pdf w2circ: radial velocity derivatives

Produces rotational (AzShear) and divergent (Divergence) shear fields for every tilt

Also produces layer maxima (e.g. 0-3 km MSL)

The most-used single-source algorithms

nse: near-storm environment Parameters are derived from the RUC

model analysis Provides input to other algorithms Output similar to SPC mesoanalysis web

page

Other single-source algorithms

w2hail: hail grids and echo tops w2vil: VIL and composite reflectivity netssap: the original SSAP

MDA, TDA, SCIT, HDA, DDPDA Requires copy of *.dat configuration files in

working directory dealias: independent executable of WSR-

88D build 10 dealiasing Note that dealiasing is usually done

automatically in data ingest process for WSR-88D data (ldm2netcdf)

WDSS-II Real-time data flow

scit3D

Reflectivity[QC]

(x N radars)

Scit2D

(x N radars; or from

w2merger)

w2merger

ClusterTable

MergedReflectivity[QC]CompositeForecast (15,30,45,60 min)

Windfield

Multi-radar products

w2segmotion

MergedReflectivity[QC]

MergedReflectivity[QC]Composite

VIL products

Reflectivity_X1C

EchoTop_Y2

HY2_Above_HX1 (“Height Above Isosurface”)

MESH /POSH / SHI (Hail)

Scit2D (from 3D grids)1isosurface(C); 2reflectivity value (dBZ)

nse*

qcinfo

Dashed lines represent optional inputs, data sources, or applications

Applications are in boxes

Data sources are in ovals

Legend

MR_Celltable

QCTimeInfo

w2accumulator

MESH Tracks (2 hr, 6 hr, etc)

*If nse is not used as an input, then MRScitHail.xml should be updated twice daily. It is highly recommended to use nse data if accurate hail guidance are desired.

AzShear[layer]

w2merger

MergedAzShear[layer]

(RotationTracks)

w2merger Multi-radar data merging

2D or 3D Continuously updating

The grid is updated each time data are received from any source

Writes output at user-specified time intervals Any resolution (Vertical/horizontal) Also runs algorithms on the 3D data field http://cimms.ou.edu/~lakshman/Papers/w

2merger.pdf

w2merger preparations: cache

Pre-compute the radars that will sample the grid point (the “cache”) Makes all computations faster Beam blockage is considered Use program “createCache” (once for each

radar) w2merger will create a cache on-the-fly if one

is not available, but: It will not include terrain data Data will not be processed until the cache creation is

complete (which might take a while)

w2merger preparations: cache

By default, the cache is stored in ~/.w2mergercache It might be big! If you are finished processing

a domain, you should delete it A cache may be extracted from a cache

with larger spatial extents (“createCache –e”) Within NSSL: extract from /mnt/radararchive

Another option: createSubdomains – create caches for all radars in the domain

w2merger preparations: cache

You may reduce the number of radars that affect a point by running “postprocessCache” e.g. if you only want the 3 “best”

radars to impact the calculation at a point

Merging strategies Different products may require different

ways of combination Set through the ‘-C’ option Some examples:

Reflectivity: ExponentialTimeAndDistance or Distance

AzShear: MagnitudeMaximum Velocity: InverseVAD or MultiDoppler

Choose the most appropriate method for the product you are merging.

There are others: see w2merger usage for list If you need a different merging option, add it!

Running merging and algorithms separately

Algorithms may be run each time w2merger writes out 3D grids of reflectivity data

If the merger is CPU-intensive or I/O-intensive, then run the algorithms separately, perhaps on another machine w2merger option “-C 10”

w2merger algorithms(-a option) Composite or VerticalMaximum

vertical maximum at each lat/lon VerticalMinimum

vertical minimum product at each lat/lon AbsMax or AbsoluteMaximum

abs-max product at each lat/lon. The result retains the sign of the maximum.

VIL vertical integrated liquid product at each

lat/lon (assumes that the 3D grid is a grid of Reflectivity)

Includes different integration strategies (e.g. along storm tilt, VIL Density, etc)

w2merger algorithms(-a option) HDA

produces SHI, POSH, and MESH at each lat/lon (assumes that the 3D grid is a grid of Reflectivity).

SCIT creates 2D storm cell features from the multi-

radar grid (assumes a grid of Reflectivity). LayerAverage or Isotherms

produces Reflectivity at various isotherms (0,-10 and -20C), ReflectivityBelowZero, LowestReflectivity, etc.

w2merger supplemental output

MergerInputRadarsTable Provides information about the

current data streams Age Tile VCP

Useful for determining which radars went into the output

w2segmotion: storm segmentation and motion estimation

Multiple scales Can generate statistics based on

storm areas Motion estimates feed back into

w2merger for time/space correction

http://cimms.ou.edu/~lakshman/Papers/kmeans_motion.pdf

Mr. SCIT (Multi-radar storm cell identification and tracking “scit3D” executable

Use “-g” option for Scit2D features generated by w2merger

Use “-t” option to ingest grid fields of various parameters that should be added to the output table

Environmental data from RUC analysis Precipitation rate field Etc.

Produces “MR_CellTable” output

w2accumulator Take the:

Maximum Minimum, or Sum

of all tables or grids produced over a specified time interval. E.g.: 2-hour max MESH = a hail swath 6-hour precipitation rate integration 4-hour max of 0-3 km Azimuthal Shear

(“Rotation Tracks”) DataTable, RadialSet, or LatLonGrid

Other useful algorithms

w2cloudcover: estimate cloud cover over a region using IR satellite and surface temperature

w2vortdiv: compute vorticity and divergence from a 2D wind field

w2alarm: collect statistics within an earth-relative polygon for any grid

Data Converters w2awipsnc: convert WDSSII netcdf

grid files to AWIPS format w2cropconv: convert and remap

any WDSSII RadialSet or LatLonGrid to a LatLonGrid

w2csv2table: convert a CSV file (spreadsheet) to a WDSS-II DataTable

w2table2csv: vice versa

Data Converters w2geotiff: convert a WDSSII netcdf file

to a geoTIFF file A TIFF image file with geographic

information tags (for GIS) w2grib2conv: convert a WDSS-II file to

GRIB2 netcdf2ldm: convert a set of WDSSII

netcdf files to WSR-88D level II format Can replace AliasedVelocity with Velocity,

Reflectivity with ReflectivityQC for example

Objective analysis / filters w2smooth: smooth the data using

one of many strategies: Gauss Cressman Percent (e.g. median) Oriented Ellipse Various wavelets

Objective analysis / filters

w2threshold: Thresholds one field based on another Example, remove VIL in areas where

the IR temperature is > 250K Various options to smooth (using

w2smooth internally) and/or segment field

Objective analysis / filters

w2oban: convert point data to a LatLonGrid

w2morph: morphological filters Dilate Erode

w2contour: create contours of a data field

File manipulation w2get: copy a file via rssd w2mirror: mirror all the files listed in an lb

to a different machine Limits the number of users “hitting” a real-

time machine w2simulator: simulate real-time data

playback w2stitcher: stitch together two different

domains into one larger one

Suggested exercise on archive data Download KTLX and KINX data from May 20,

2001 from 21:00 to 22:00 UTC from NCDC Convert it into WDSS-II netcdf format Run w2vil to produce VIL estimates in rapid-

update mode Merge the VIL estimates using w2merger

What is a valid combination strategy here? createCache before merging!

Compute VIL from merging reflectivity data Compare the two VIL estimates

Find their difference field using w2scoregrid

Suggested exercise on real-time data Connect to two adjacent radars that are

currently experiencing weather Look at the 2DConUS index Overlay the radarsites shapefile Find LB names from the tensor list

Create cache for domain using createSubdomains.

Extract from /mnt/radararchive Run w2vil, w2merger and w2scoregrid as

described before. Set up a w2alg.conf to do this.

End of WDSS-II Training Module IV What to do next :

Practice running some algorithms and tools. You will not be able to follow module 6 (writing

a WDSS-II algorithm) unless you are familiar with how WDSS-II algorithms in general work.

Run both a single-radar algorithm and a multi-sensor algorithm.

Run both on archived cases and real-time cases.

Next module: Configuring WDSS-II