Advanced Weather Interactive Processing System (AWIPS) Overview

for System-of-Systems

Workshop

October 7, 2009

Ronla Henry & Steve SchotzNWS – Office of Science and Technology

AGENDA

• AWIPS Overview– Mission and Functions– Brief History

• AWIPS II Technology Infusion Scope and Milestones

• AWIPS II Architecture Overview– Major Software Components– Communications Interfaces– Standards and Formats

• AWIPS II Extended Data Delivery Project Overview

AWIPS Overview - Functions/MissionAWIPS is the Critical Link to the NWS Forecaster

NEXRAD

AWIPS Workstations andServers

GOES/POES

ASOS

NCEPModels

Buoys,River Gauges

Forecasts

Warnings

Advisories

Watches

AWIPSCommunications

Service provided to3066 US Counties

24 hrs/day, 365 days/yr.

169 separate AWIPS systems at137 geographical locations

~900 Workstations (total)~1200 Servers (total)

AWIPS Overview - Service Delivery Facilities in Six NWS Regions

AWIPS A Brief History

• The original prime contract for system development was awarded to PRC, Inc on December 29, 1992– To replace Automation of Field Operations and Services (AFOS)– PRC, Inc later acquired by Northrop Grumman IT (NGIT)– Commissioned - 2000

• Operations phase of original contract expired on September 30, 2005

• Re-compete contract awarded to Raytheon Technical Services (RTS) on August 17, 2005– Proposal included a high level plan to re-engineer AWIPS

software into a Service Oriented Architecture (SOA), AWIPS II

AWIPS II Technology Infusion Scope

AWIPS II Technology Infusion (FY2005 – FY2015) – A long-term project which delivers a modern, robust software infrastructure that

provides the foundation for future system level enhancements for the entire NWS enterprise

• Phase 1: (FY2006-FY2011)– Migration of WFO/RFC AWIPS (AWIPS I) to a modern Service Oriented Architecture

(SOA) infrastructure executed incrementally through a series of task orders

• Phase II: (FY2009-FY2012) – AWIPS SOA Extension– Creation of a seamless weather enterprise spanning NWS operations

• Migration of NAWIPS into the AWIPS ISOA• Delivery of thin client to support for the Weather Service Offices, Center Weather Support

Units, Incident Meteorologists, (e.g., Fire Weather, backup support for RFCs and National Centers)

• Integration of Weather Event Simulator)• CHPS Integration into AWIPS SOA

• Phase III: (FY2009 – FY2015) – Enterprise Level Enhancements• Data delivery enhancements: “Smart push-smart pull” data access• Integrated visual collaboration• Information generation enhancements • Visualization enhancements

Migration Schedule

Task Order Delivery DateDevelop of AWIPS I SW Product Improvement Plan (TO1) June 2006

Conduct Initial system analysis (TO2) October 2006

Develop ADE/SDK (TO3-6) July 2007

Plan baseline application migration (TO7) October 2007

Migrate primarily D2D/Warngen capabilities migrate (TO8) February 2008

Migrate primarily GFE capabilities (TO9) September 2008

Migrate primarily hydrologic capabilities and infrastructure improvements (TO10)

February 2009

Complete AWIPS SOA Release 1.0 (TO11) March 2010

System Operational Test and Evaluation (OTE) August 2010

Field (OTE)

Target deployment

December 2010

2011

AWIPS I to AWIPS IIRe-Architecture Approach

• Perform “black-box” conversion– Preserve existing functionality, look and feel on top of new

infrastructure

• Thorough field validation and acceptance before deployment

• No loss of functionality– Deployed system current with deployed AWIPS capability (i.e.,

OB9)

• Use open source projects - No proprietary code– JAVA and open source projects enable AWIPS II to be platform

and OS independent

• Objective is to make AWIPS II available for collaborative development

AWIPS-II: Reference ArchitectureBased on Plug In Extensible services

AWIPS-II Reference Architecture

<<Service Framework >>

EDEX

<<Visualization Framework >>

CAVE

Meta

DataData

+Ingest Service +Persist Service

+Metadata Service +Subscription Service

+Request/Transform Service +Adapter Service

+Collaboration Service +Disseminate

Core Generic Services

Core Libraries

+Eclipse RCP PlugIns +Data Animation

+GIS Raster Rendering +Data Editing

+GIS Vector Rendering +Data Analysis

+GIS Point Rendering +Drawing

Enterprise Service Bus (ESB) - Camel

AWIPS-II Dependencies: Open Source BasedUpdated Oct. 05, 2009

CAMEL + Spring Enterprise service bus and dependency injection container for SOA services (decoupled services)

activeMQ Java messaging provider with clustering and JMS tunneling over HTTP

PostgreSQL Relational database for storing Metadata from Data plug ins and spatially enables ingested data

PostGIS Spatially enables PostgreSQL

Hibernate3 Relational RDBMS to Java Object mapping

GeoTools Enables GIS capabilities and map projection framework

JOGL Java API to OpenGL enables Gaming level visualization performance

HDF5 High performance file persistence of large data sets such as satellite, radar, and parsed point data

JAVA + ANT Primary programming language and software build framework

Python + numPY Data transform scripting languages with high performance math library

Apache Velocity Provides a mechanism for automatic text product generation

Eclipse RCP Plug In driven visualization framework

OpenFire Real time collaboration server based on XMPP

Batik Scalable Vector Graphics (SVG) used for plots

Apache Thrift Binary service message transfer serialization

Jetty Web application container

SB1A

AMC4 Americom CommercialSatellite

AWIPSWIDE AREANETWORK(NOAANET)

AWIPS Communications Interfaces

GINI

CISCO2514

AWIPS NetworkControl Facility

CISCO7000

CPID CPIC

SBIG

DSIA

CISCO2514

CSU/DSU

NWSTGAS1A WK1BDS1A

ApplicationServer

DataServer

WFO or RFC

DataServer

ApplicationServer

WorkStation

SBP

DEMOD

CISCO4500

AWIPS Master Ground Station

CombCSU/DSU

ModHPA

NCEP

NESDIS

Source of AWIPS Model Data

Source of AWIPS Satellite Imagery

Other Gov, WMO, Academic, andPrivate Industry Interfaces

NEXRAD ASOS CRS RRS

Site AWIPS System

NOAAPortReceive System

Non-Gov Data Users

DEMODSBP

SP1D

Server

LDAD

AWIPS II Primary Standards & Formats

• Ingested Data Formats– GRIB1/GRIB2 – Gridded data– NetCDF3 – Support AWIPSI/II interoperability– BUFR – Observational Data, e.g., soundings– METAR, SHEF – Surface and hydrological data– GINI – Satellite Imagery– OPRG L3 – Radar Imagery– Text Messages – Text products

• Data Store Formats– PostGres – Metadata and select data type store, e.g., text– HDF5 – Binary store for grids, imagery and select observations

• Product Distribution– NetCDF3 – NDFD Grids– ASCII Text – Text products

AWIPS II Extended Data Delivery Overview

• Objective – Develop robust data delivery system within AWIPS II infrastructure that enables efficient access to high volume datasets – Address significant growth in data volumes, e.g., ensembles, GOES-R,

NPOESS and mitigate impacts on SBN

• High-Level requirements in common with NextGen– Data registry and discovery services– “Smart” push/pull technology

• Sub-setting by user selectable space, time, and parameter• Complex retrievals, e.g., derived parameters, coordinate transformations, etc

– Ad hoc and subscription services– Operationally robust – supports availability, latency and security

requirements for operational users

• Multi-Phase Implementation– IOC Focus – NWS data providers, e.g., NOMADS, MADIS, possibly with

basic services only, discovery, sub-setting – Target FY12

AWIPS II Data Delivery – Significant Challenges

Challenge Potential Mitigation Strategies• Dependence on Data Provider readiness

– Net enabled services, e.g., adapters– HW infrastructure to support operational requirements, e.g., availability and latency– Required resources/funding– Schedule constraints

• Coordinate/pool resources with NextGen

• Focus on NWS data providers such as Nomads and MADIS for IOC

• Develop AWIPS II Distribution Servers

• Dependence on network capacity (NOAANET)

• Identify current/planned capacity

• AWIPS PPBES FY12-16 submission

• Develop governance requirements

• Coordinate with NextGen requirements

• Security between AWIPS, NOAANET and Data Providers

• Engage organization security representatives early

• Establish SLAs and ICDs

• Coordinate with NextGen requirements

QUESTIONS ????