Channel Archiver Introduction

Channel Archiver Introduction

[email protected]

mailto:[email protected]

OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY

2

Basic Pieces of the Channel Archiver

ArchiveEngine Collect

samples as a CA client

Network Data Server Serve the

archived samples

Java Viewer Plot, …


3

ArchiveEngine

Receives data via CA, writes to disk

'Monitored' Mode Every received sample is written to disk.

Limited by memory used to buffer between disk writes. Configurable, based on estimated period.

'Scanned' Periodically writes the most recent value to disk.

Internally uses 'get' or also 'monitor'. Marks repeated values as such to conserve space.


4

Engine's XML Configuration File

<?xml version="1.0"?>

<engineconfig>

<group>

<name>Stuff</name>

<channel> <name>tx:aiExample</name> <period>10</period> <scan/> </channel>

<channel> <name>tx:setpoint</name> <period>10</period> <monitor/> </channel>

<channel> <name>tx:room</name> <period>60</period> <monitor/> </channel>

<channel> <name>tx:PID</name> <period>1</period> <monitor/> </channel>

<channel> <name>tx:tank</name> <period>1</period> <monitor/> </channel>

</group>

</engineconfig>


5

Sampling stores original time stamps!

Data @ 9.96, 10.98, 11.76 s ?


6

Engine's Web Server for Status URL of engine’s HTTPD: http://<machine>:<port>

Local machine, default port: "localhost:4812" Port set via command-line option.

Status & Config., not data!

“Client Pull”: Updates on reload

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.


7

Network Data Server

Usually a 'CGI' plug-in to a web server Hosted by web server, but the result is 'XML-RPC'

protocol for special archive client, not HTML.

"Real" web server, not the one inside the engine. Good, because we can use security mechanisms as

common to web servers, get through firewalls, … Bad, because you have to understand httpd config to

get going.


8

Idea


9

Supported Export Options

Original Time Stamps:ill-suited for Spreadsheets

“Staircase” interpolation, repeat values

Linear Interpolationfor given period

Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 #N/A -0.0860063/22/00 17:02:37.401 0.054358 #N/A3/22/00 17:02:37.511 #N/A -0.111776

...

Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 0.071824 -0.086013/22/00 17:02:37.401 0.054358 -0.086013/22/00 17:02:37.511 0.054358 -0.111783/22/00 17:02:39.411 0.139948 -0.11178...

Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 #N/A -0.086013/22/00 17:02:30.000 0.069216 -0.089813/22/00 17:02:32.000 0.065201 -0.095663/22/00 17:02:34.000 0.061186 -0.101513/22/00 17:02:36.000 0.057171 -0.10736…


10

Export for plot

Creates“Plot-Bins”of requestedwidth.

Bins with>2 valuesreplaced by:Initial,min, max,final.

Result:Plots withhardly anyvisual difference, but a lot less data.


11

ArchiveDataServerStandalone Combines the data server with a simple web

server ('abyss') into server for single archive:

cd /ade/training/ubuntu_stuff/ext/src/ChannelArchiver/DemoData

ArchiveDataServerStandalone abys.conf /ade/training/groups/tx/whereever/index

… as long as you only run one per machine. Otherwise need different abys.conf

Then run viewer: archiveviewer -u http://localhost:8080/RPC2 &


12

Java Data Viewer

Search PV names

Plot, zoom, pan

Multiple axes

Formulas

Export spreadsheet-type text files.


13

One Archive = Directory with …

Data files '20060120', …

Index File 'index'

Do not separate these! Data will be lost!

One typically gets more then one Stop & restart in new directory every week

limits possible data loss. Split config into one engine per sub-system


14

How do I look at more than one Archive?

Create 'list' index file Basically XML file that lists individual index files. Point data server to that one. Degrades since each query looks everywhere until

something's found. Problems when same channel in more then one sub-

archive.

Run ArchiveIndexTool on 'list' index file Creates binary index with info from sub-indices. Optimal for retrieval, but creation takes some time. Need to re-run index tool when sub-archives change.


15

Features support long-term history, not data acquisition for experiments

'Monitored' (with buffer limit) and 'scanned'

'Disabling' feature for groups allows suspension of sampling Example: Skip power supply data while PS is off.

Not implemented: 'On demand' snapshots to save/restore settings 'triggered': Save A, B, C, … whenever X=2

Also not implemented: pre/post trigger to get detailed samples on some event, while ignoring the rest.

More detailed correlation: Wait for matching time stamps on channels A, B, C, …, and X=2, …


16

Missing Data Management Features

Index type in between 'list' and 'binary' Not as complete as the 'binary' index, but with

acceptable creation/update time to allow looking at "everything"

Any type of data compression (Convenient) ways of deleting selected channel

or time range Replace samples with slower-sampled average

or other statistical data.


17

More Stuff

ArchiveExport Command-line data extraction

ArchiveDaemon Starts/stops/restarts engines automatically

Scripts for copying data,headaches with managing terabytes of data, …

Documentation, Snapshots:http://www.aps.anl.gov/epics,Extensions, ChannelArchiver.


18

Acknowledgements

Bob Dalesio implemented the first version

Craig McChesney: data server protocol

Sergei Chevtsov: index file algorithm,Java Archive Viewer (initially w/ Craig)

Thomas Birke, Chris Larrieu, Greg Lawson, Peregrine McGehee, Nick Pattengale, Ernest Williams, Noboru Yamamoto: Ideas and/or code

Documents

Channel Archiver Introduction