AbstractAbstractComplex real-time seismic networks often develop an extensive set of dataflow connections, especially when one includes the virtual networks established by data-sharing. As these systems grow, the task of managing existing connections and adding new ones has the potential to rapidly overwhelm network operators and system architects. This is amplified by data-source deletions and additions and temporary network problems, to say nothing at all of the the changing presence of individual data streams.

We have created dynamic status-monitoring tools for Antelope-based seismic networks, expanding on capabilities of the Antelope system, which allow us to monitor the topology of orbserver connectivity and the dynamic dataflow for individual streams. These tools are important for monitoring network health, avoiding common traps such as data loops, and engineering sensible data-flow architectures.

Status monitors run on each orbserver, delivering metadata about server connectivity, clients, and data streams to a central site. These metadata are saved in a near-real-time relational database. We generate directed-graphs from this database. From these, using the GraphViz graphical processor, we create interactive overview images of connectivity and dataflow.

We present the current status of this technology; challenges we have encountered in its development; and application to multi-signal-domain sensor grids. We discuss the potential for further growth, including more sophisticated analysis algorithms and the possibility of automated, dynamic sensor-network reconfiguration.

Dynamic Dataflow Topology Monitoring for Real-time Seismic NetworksK.G. Lindquist *, T.S. Hansen‡, B. Ludäscher‡, F.L. Vernon†

* Lindquist Consulting† Institute of Geophysics and Planetary Physics, UCSD (IGPP)‡ San Diego Supercomputer Center (SDSC)

For more information:http://roadnet.ucsd.edu

The above diagram shows a dynamically generated snapshot of Anza network topology. The main components shown are Antelope orbservers (Antelope is a product of Boulder Real-Time Technologies, Inc.). The orbserver is a near-real-time object ring buffer for packetized data, in this case seismic data. Each one of these connections is created by an instance of the Antelope orb2orb command. Figures like the above are created in near-real-time based on status snapshots of the running ORBs. The pforbstat command creates ORB-packets with parameter-files containing summaries of clients and packet types. This information is distilled and used to create a description file for a directed graph, which is plotted via the open-source AT&T GraphViz application.

Application StructureApplication Structure

Future DirectionsFuture Directions

The diagram above shows the basic components of the monitoring system. A pforbstat utility runs on each orb of interest. The results are encapsulated as parameter-file orb packets and placed back on the orb, where the new orbpf2db utility puts them into a database in the registry1.0 schema.

Starting from the prototypes displayed here, we are currently working on a web-based implementation driven by the new Antelope interface to PHP, the web scripting language. This will allow web-clickable interfaces which can provide much more information about each orb, including available data sources and clients.

Registry1.0 database

Antelope PPInterface:

Clickable, interactive Connectivity map

Specific Orbserver

Client Client

srcname

srcname

Connectivity Database…

…InteractiveExploration

Active Orbservers

Real-time Databases

Dataflow Latencies

Connectivity StatusConnectivity Status

The symbols with single-line outlines are Object Ring Buffers (ORBs), managed by instances of the Antelope orbserver command.

Symbols with double-line outlines are near-real-time relational databases. These actually represent the orb2db client programs running on a given orb. As with the orb2orb instances, the orb2db clients are recognized in the pforbstat parameter file packets.

The blue lines linking each box represent network connectivity with the dataflow latency (in seconds) above each respective line. Each of these connections is an instance of the Antelope orb2orb command.

Change DetectionChange DetectionIn addition to presenting the current connectivity status, the orb topology-monitoring scripts are able to detect changes in the layout, most notably additions of new orbs or disappearances of existing ones. The diagrams at right are graphical summaries of detected changes that are emailed to interested parties. Green objects are new appearances; red objects have disappeared from the pforbstat structures.

The figure below shows a hypothetical future application, in which one could select a specific orb source-name from a list, and get a map of the orbs containing that source, with their latencies.