Monitoring data db

Monitoring data database

Paola GrossoSCS Networking

The idea (from Warren) is to store the data from various monitoring programs into an Oracle database for:

– Faster retrieval– Easier data selection – Trend studies– Archiving purposes

Monitoring data db

Oracle Database

We are using the new Oracle 9 db, called SLACPROD, instead of the older one, SLAC_TCP.

This will allow us to grow the data size more easily and possibly take advantage of new Oracle feature as partitioning

(quoting Ian MacGregor)

Monitoring data db

Measurements

The data we are storing comes from 3 different sources:

• RIPE_TT measurement• ABwE measurements• IEPM-BW measurements

There are separate tables for each one of the data sources.

Monitoring data db

RIPE_TT table

The data is collected in real time from the data that has been received from the various RIPE TT boxes:

SQL> descr RIPE_TT: Name Null? Type ----------------------------------------- -------- ---------------------------- ALIASNAME VARCHAR2(20) NODENAME VARCHAR2(30) VERSION NUMBER(38) IPDEST VARCHAR2(30) PORTDEST NUMBER(38) IPSDR VARCHAR2(30) PORTSDR NUMBER(38) ARRIVALTIME VARCHAR2(20) DELAY NUMBER(63) NTPSRC VARCHAR2(20) NTPDEST VARCHAR2(20) SEQUENCE NUMBER(38) NTPERRSDR NUMBER(63) NTPERRDEST NUMBER(63)

Monitoring data db

ABWE table

The data stored in this table is extracted in real time from the ABwE measurements performed every minute:

SQL> descr abwe; Name Null? Type ----------------------------------------- -------- ---------------------------- ALIASNAME VARCHAR2(30) NODENAME VARCHAR2(30) IPADDRESS VARCHAR2(30) TIMESTAMP NUMBER(38) ABW NUMBER(63) XTR NUMBER(63) DBCAP NUMBER(63) AVABW NUMBER(63) AVXTR NUMBER(63) AVDBCAP NUMBER(63)

Monitoring data db

IEPM-BW_ROUTE table

IEPM_BW_ROUTE contains the data for the ctrace,rtrace and trace measurements

• SQL> descr IEPM_BW_ROUTE;• Name Null? Type• ----------------------------------------- -------- ----------------------------• ALIASNAME VARCHAR2(30)• NODENAME VARCHAR2(30)• IPADDRESS VARCHAR2(30)• TIMESTAMP NUMBER(38)• TOOLNAME VARCHAR2(30)• ROUTENUMBER NUMBER(38)• ROUTE VARCHAR2(1024)

Monitoring data db

IEPM_BW table

This table contains the data from the iperf,qiperf,gftp and bbftp measurements.

SQL> descr IEPM_BW; Name Null? Type ----------------------------------------- -------- ---------------------------- ALIASNAME VARCHAR2(30) NODENAME VARCHAR2(30) IPADDRESS VARCHAR2(30) TIMESTAMP NUMBER(38) TOOLNAME VARCHAR2(30) THROUGHPUT NUMBER(63) STREAM NUMBER(38) WINDOW_SIZE NUMBER(38)

Monitoring data db

Current status

We are debugging the tables layout and the scripts that store the data.

We have already stored:RIPE_TT ~350k entriesABWE ~50k entriesIEPM_BW ~7k entriesIEPM_BW_ROUTE ~20k entries

Monitoring data db

Code structure

The scripts that perform the data archiving are located in the AFS area:

/afs/slac/g/www/www-iepm/tools/netrat/tools

There 3 Perl scripts and one Perl module for the actual archiving, and 3 Shell scripts for the management of the programs.