1

OB8.3 DCS3452 OSIP 07-020GFE SvcBu – ISC Routing

August 2007

2

Agenda

Requirements Review Design Review

– ISC Routing Table– Request/Reply of ISC

User Interface Review– ISC Routing Table– Request/Reply of ISC

3

Requirements Review

4

Requirement and Initial Plans

Derived from OSIP 07-020– Service Backup Improvements – Phase 1– To address the delay/reliability of service backup

Original Plans to address deficiency:– Real-Time configuration synchronization– Hot Backup – grids from ISC– Expanded Domain– Automated ISC Routing

5

Requirement Migration for OB8.3

SMTP / iscMosaic/ ifpnetCDF / scope– Resulted in sufficient performance to eliminate the

OB8.3 need for hot backup.– Overall changes were too large for single build

and too much impact on the field.

Scope reduced for OB8.3:– ISC Routing Table to correctly route ISC traffic as

ifpServers are added to the network

6

Requirement to Implement

ISC Traffic shall be automatically routed to the sites (ifpServers) requiring ISC data.

– As the service backup is invoked, the routing will be automatically updated so that the service backup server will get the correct ISC grids from its surrounding sites

– The need for manual calls / configuration changes to neighboring sites for ISC data should be eliminated.

A technique to manually request data will exist, i.e., to initially populate the ISC database when service backup is started.

7

Assumption

The overall service backup paradigm remains as it is today:– Download configuration and grid data from the

IFPS Central Server– Bring up a second ifpServer for the failed site’s

domain.

8

Design Review

ISC Traffic Routing

9

How it works today…

The sending site must modify a gfe configuration file to list the sites that will get the ISC data.– No clean way to adopt for changing

configurations.– Error-prone. Using the wrong config file will cause

issues.– Requires phone calls to sites to change

configuration.

10

ISC Traffic Routing addresses:

The deficiency (and manual setup) required to reroute ISC traffic when service backup configuration changes.

– s/w currently doesn’t know who needs the data. The need to know about every machine requiring

ISC data. The complicated, and perhaps error-prone ISC

configuration options in GFESuite that the focal point needs to consider.

– The possibility of sending HUGE amounts of ISC data by accident.

11

ISC Traffic Routing solution:

Knowledge is known in real-time who needs the ISC data and how it needs to be routed.

Provides additional information about non-AWIPS GFEs that are getting ISC data.

Greatly simplifies the ISC configuration options in GFESuite that the focal point needs to consider.– Eliminates, or greatly reduces, the ability of a

forecaster to “flood” the network with ISC data.

12

Current Incoming ISC Data

AB

C

D

ESite B sends ISC data to A. Site B sends a lot more data than necessary to A and most of that data is dropped when merged.

13

Addressing the “data volume”

AB

C

D

ESite B sends clipped data to A – only what is necessary.

14

Current ISC Routing – determined by sender

A

B

C

D

ifpServer at “A”ifpServer at “C”

C

AISC routedto site “A”

Normal Operations

15

Current ISC Routing – determined by sender

A

B

C

D

ifpServer at “A”

A

B

C

D

ifpServer at “B”, configured for “A”(performing backup)

ifpServer at “C”

C

AISC routedto site “A”

Service Backup Mode

Site “C” fails to route ISC for “A” to site “B”Since “C” has no knowledge of “B” doing “A”.

16

Planned Incoming ISC Data – determined by receiver

A

B

C

D

ifpServer at “A”

A

B

C

D

ifpServer at “B”, configured for “A”(performing backup)

ifpServer at “C”

C

AISC routedto site “A”

Service Backup Mode

Site “C” knows to route ISC for “A” to site “B”,and no longer routes ISC to site “A”.

17

ISC Routing

Correct routing depends upon knowing:– Each ifpServer’s configured domain and site

Internal routing within a WFO also requires correct routing:– If two ifpServers are running, they should not

necessarily be getting the same ISC data

Correct routing will dynamically change.

18

Who to send to? (1)

Use IFPS Central Server (or similar) and run a web service that:– Knows the pairing (configuration) of all of the sites– Knows who is doing service backup– Knows who needs a particular piece of ISC data

Web service updated in real-time as configuration changes.

19

Who to send to? (2)

Each ifpServer upon startup/shutdown, and svcbu start/stop would send to the web service:

– Its requirements for ISC data– Its configuration (domain and site)– Its physical location (mhs id, server host, server port)

Example:– Dx4/98000000 at BOU is configured for BOU domain, and

requires ISC from ABQ, GJT, PUB, GLD, CYS for a particular rectangular box. It wants T, Td, Wind, Wx, and Sky for weather elements.

20

Who to send to? (3)

Each ifpServer upon sending ISC would contact the web service to determine where to send its data.– Allows for correct routing for ISC, even in service

backup scenarios No change in the format of ISC data is

implied with this architectural change.– There is additional information in the ISC

message which requires a patch to OB8.2.

21

IRT (ISC Routing Table) Service

Use web service on IFPS Central Server ifpServers would communicate with the server to

register, and to determine who to send data Footprint:

– Virtually no disk space– Small amount of http data transfers.– Potential of frequent requests from sites

Almost everytime ISC is sent, web service is contacted.– Single point of failure for ISC, so s/w fallback to default (or

last known configuration) if web service not accessible

22

2. On transmit, discover who needs “C” data

IRT Flow (1)

A

B

C

D

ifpServer at “A”

ifpServer at “C”

C

A4. ISC routedto site “A”

Normal Mode

IRT

“A” running “A” needs ISC from “B”, “C”, “D”

1. Register on startup

3. Site “A” needs “C” data

23

IRT Flow (2)

A

B

C

D

ifpServer at “A”

ifpServer at “C”

C

A4a. ISC routedto site “A”

Service Backup Mode, site “B” preparing

IRT

“A” running “A” needs ISC from “B”, “C”, “D”“B” running “A” needs ISC from “B”, “C”, “D”

1. Register on startup.

2. On transmit, discover who needs “C” data

3. Site “A” and “B” need “C” data

A

B

C

D

ifpServer at “B”, configured for “A”(getting ready to perform backup)

4b. ISC routedto site “B”

24

IRT Flow (3)

A

B

C

D

ifpServer at “A”

ifpServer at “C”

C

A

Service Backup Mode, site “B” active,Site “A” offline.

IRT

“B” running “A” needs ISC from “B”, “C”, “D”

2. On transmit, discover who needs “C” data

3. Site “B” and “B” need “C” data

A

B

C

D

ifpServer at “B”, configured for “A”(performing backup)

4. ISC routedto site “B”

1. Unregister

25

ISC Software Changes (in GFE/ifpServer)

Initial ifpServer startup (and when svcbu configuration changes), contacting IRT server to define needs for ISC data

Sending ISC, to consult IRT server to know what/who to transmit. [iscExtract script]

Generation of ISC data for transmission, produce what is required based on IRT entries. [iscExtract script]

26

ISC Software Changes (in GFE/ifpServer)

Iscd/iscDataRec concept, along with routing files currently used, changes– Iscd no longer needed (dx4 can call msg_send)– Routing files replaced with IRT

Occasional “pings” to IRT to let IRT know that an ifpServer is still there.– Handles the disappearing ifpServer case.– Maybe every 8 hours (?). Defined by IRT.

27

ISC Software Changes (at central server)

Development and Installation of simple web service– Dynamic Table containing domains, configuration,

which ISC data needed, for every ifpServer in the NWS.

– Responds to GET/POST requests for: Register, unregister, query

– Responds with XML information: Contains destination information for ISC sending domain

28

IRT – additional advantages

EVERY ifpServer on the network would need to register if ISC data is desired.

– This includes “RPP” machines Allows for detailed real-time analysis of:

– Who is doing service backup– Who is getting ISC data– Who has RPP (non-baseline) machines on network

A monitor could be written to view the status, and perhaps make IRT table changes.

– Not in scope for OB8.3

29

OB8.2 Compatibility Issues

Backward compatibility for OB8.2 ISC.– OB8.2 does not have an IRT. – We will have a web interface to allow entries to be manually

placed into the IRT. – We will pre-configure the IRT for ISC for all sites in order to

be compatible with OB8.2. May be a script to be run at each OB8.2 site.

– IRT doesn’t have to exist, however, until the very first OB8.3 site is activated.

Failure to do this will stop OB8.3 ISC data from going to OB8.2 sites.

30

OB8.2 Compatibility Issues

Will likely need a patch provided to OB8.2 sites.– We use two attachments in the WAN message for

OB8.3.– We use one attachment in the WAN message for

OB8.2.– An OB8.2 site won’t understand this second

attachment, nor will it get purged.

31

Other AWIPS Compatibility Issues

Some (very little) of the service backup code existing in OB8.2 will need to be changed/removed for OB8.3.– The code currently creates isc routing files.

These routing files will no longer be needed.– Failure to remove the code won’t cause harm

since the routing files will be ignored.

32

Design Review

Request/Reply ISC Data

33

Request/Reply Design

Only needed to initially populate the ISC database.– Only needed if you *must* have ISC data right

now.– Eventually (12 hours?) ISC will populate itself.

A request can be made from a GFE for ISC data for a particular domain.– Example: “I would like to get Wx data from the

SLC domain” into my ISC database.

34

Request/Reply Design

Request consists of the requestor’s and requestee’s ifpServer info (mhs host, server host, server port), along with weather elements desired.

Request goes from ifpServer through iscRequestReply script to package up info for msg_send. [new script]

35

Request/Reply Design

Upon receipt of data request, remote ifpServer packages up data just like normal ISC send operations, and sends it (without consulting the routing table) via msg_send to the requestor.

Data is ingested into the requestor’s ifpServer and the data appears in the ISC database.

36

User Interface Review

ISC Traffic Routing

37

UI Changes to GFE

Due to automatic ISC routing, the manual dialogs that dealt with routing need to be removed or changed.– Normal configuration….all data saved to the Fcst

database will be sent via ISC.

38

Deleted Dialogs

Removal of the Send Intersite Grids Dialog

39

Changed Dialogs

Elimination of the “Send ISC Grids” button on the Save Weather Element Dialog.

OLDNEW

40

Changed Dialogs

Elimination of the “Send ISC Grids” button on the Publish Weather Element Dialog.

OLDNEW

41

User Interface Review

Request/Reply ISC

42

Two Designs

Two dialogs or One dialog?– Affects the internal data flow on implementing request/reply

for ISC data.– Degree of software complexity varies depending upon the

GUI choice Two Dialog Option

– First – select the data you want– Second – select where you want the data from

One Dialog Option– Select the data you want and where you want the data

from.

43

Two Dialog Choice: Request ISC Data Dialog

From Consistency -> Request ISC Data menu

OK Cancel

Requested Weather Elements

TTdWind(multi checkbox,Preselected)

Domain

CLEPBZBUF(multi checkbox,Nothing preselected)

44

Two Dialog Choice: Requestee ISC Data Dialog

Pops up automatically after previous dialog for each domain.

OK Cancel

Data Source for CLE

Domain AWIPSSite Serverhost ServerPortCLE cle dx4f 98000000CLE pbz dx4f 98000001CLE cle rpp-1 98000000

The most probable entry is pre-selected. Can only select one.

45

Single Dialog Choice: Request/Reply Dialog

Cancel

Requested Weather Elements

TTdWind(multi checkbox,Preselected)

OK

Domain

CLEPBZCTPBUFIWXDTX

Multicheck,Defaults allOff.

Data Source (ifpServer)

CLE cle dx4f 98000000CLE pbz dx4f 98000001CLE cle rpp-1 98000000

IWX iwx dx4f 98000000

Domain MHS Host Port

46

Request/Reply Dialog

Only data sources for the selected domains (1st checkbox) appear in the 2nd checkbox.

Ensures that only one ifpServer for each domain is on. Defaults to the “most likely” ifpServer.

47

Summary

48

Summary

ISC Routing Table Changes with central web server.– Some configuration/training needed for focal points to

activate the new ISC.

ISC Request/Reply to handle initial population of ISC database.

Minor GUI changes. Some work on OB8.2 will be needed for

compatibility.