Upload
chul
View
36
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Report from A+M Data Centre, RRC “Kurchatov Institute”. Yu.V.Martynenko 19th Meeting of the Atomic and Molecular Data Centers and ALADDIN Network Vienna, 03-05 October 2007. The main activities on A+M/PSI DATA: 1. The new A+M Data generation. (Experiment, theory, codes). - PowerPoint PPT Presentation
Citation preview
Report from A+M Data Centre,
RRC “Kurchatov Institute”
Yu.V.Martynenko
19th Meeting of the Atomic and Molecular
Data Centers and ALADDIN Network
Vienna, 03-05 October 2007
The main activities on A+M/PSI DATA:
1. The new A+M Data generation. (Experiment, theory, codes).
2. Computer code for tokamak plasma processes
3. Data Acquisition System + (DAS+). This software is developed for operation with experimental
data of various devices of controlled nuclear fusion (storage, transmission, processing and results representation). The software allows to create Data warehouse for data and gives possibility for multi-devices fusion researches.
.
3. Data base.
3
DATA GENERATIONA+M
1. Atomic particles cross section measurement (split beam methods).
2. Theory of atomic particles interaction.3. Analytic models and codes for atomic
kinetic and radiations.
PSI1. Surface erosion and modification• ion beams, plasma exposition• theory2. H retention and permeability
4
COMPUTER CODES FOR TOKAMAK
PLASMA PROCESSES
The integration of already existing codes
for comprehensive SOL description:
- Plasma dynamic, convection transport (Shurygin / Pastukhov)
- Universal Numerical Code for
atomic kinetics and radiation (V. Lisitsa)
- Material response on plasma affect (Martynenko, Moskovkin)
(DB sputtering yields and reflection coefficients)
5
THE UNIFICATION OF ACCESS TO EXPERIMENTAL DATA OF FUSION DEVICES: Т-10, GTB, PN-3, S300, L-2M, Tuman, Globus, PS-3, Castor NFI RRC “Kurchatov Institute”. http://t10.fusion.ru, [email protected], M. Sokolov.
This software is developed for operation with experimental data of various devices (storage, transport, processing and results presentation). It allows to create the Data warehouse and to carry multi devices researches.
The software includes:• DB DASSQL for storage of the diagnostic and descriptive information of various devices.
• Local archives DASFile for transport of data and off-line work with fragments of DB DASSQL.
• Network service DASWeb for remote access to DB DASSQL from the Internet browser.
• Network service DASTCP for remote access to DB DASSQL from applications.
• Cross-platform library DAS Access for creation of user's applications in the environment of MS Windows and Linux, having remote access through the Internet to DB DASSQL, and also to local archives DASFile.
•The set of Mex-functions of data reading from DB DASSQL and DASFile in MATLAB.
• Utilities DASText, DASLoad and DASDec for data exchange as the text and binary files between DB DASSQL, DASFile and user's applications.
• Multifunctional graphics shell DASTools with a package of mathematical processing DAS Processing for operation with experimental data of plasma devices.
Data Acquisition System +
6
Data files
Application server
High perf. PC
Runs servicesand data handlingroutines.Acts as a server for remote terminals.
Data math.processing
DB loading/extracting
Data files reading
Service1 for client programs (TCP)
Service2 forremote
terminals
.
System modules and data flows.
T-10
_||_… _||_
Experimental shots
Data Readers
Industrial PCs with ADCs installed.Read multi-channelraw data.Compress data.Create data files.
Data Center
High perf. PC or blade-servers unit.Integrates and saves many data files into one, loads data into database.
Database Server
High perf.PC.SQL-databasestores raw and processed data of experiments, incl.data from other experim. devices.Currently 200 Gb +
Web server
High perf. PC
Runs Website with static descriptive information and facility of interactive access to the experimental data and to do more calculations.
Service3 for Web clients to access
data via browsers
Service4 for moremathematic calculations
to be executed
(http)
Data files
Data convertersImport data of other experimental devices. Export data into other formats (Text, XML, XLS, GIF,…)
Compressed data
Researchers
LAN / Internet
Researchers are provided with full-featured program utilityfor selecting, viewing and analysing the data.
IntegratedData file of a shot300 Mb +
Researchers can integrate provided library modulesinto their specific applications, and thus gain access to the data.
Efficientbinary format
…
Other experimental devices
7
Program facilities for data access and analysis.Researchers can access the experimental data in several ways:
Internet browser with HTML-input forms as interfaces to access the experimental data.The data are selected and rendered on the server and the results are displayed in the user’s browser,in binary, text, graphic, image and media presentation.
Internet browser (SigmaWeb) with input forms allowing users to run certain mathematic routines on the server with user-defined initial data.The calculations are carried out on the server and the resultsare displayed in the browser.
Other user applications for experimental data processing and presentationcan integrate library modules which implement connection to Application Server and thus the applications gain access to the experimental data.The user applications list includes Matlab, MDS, any Pascal,C++,Python, …- programs.
Full-featured program utility for data selecting, viewing and analysing,with a graphic user interface and rich variety of data processing and presentation ways (tables, 2D-, 3D-graphics, scaling, profiles).Can run on users’ PCs (usually in LAN of the experimental installation)or on Application Server while in remote terminal mode*. Can work with local copies of the experimental data (data files).Provides a unified user interface for researchers (remote included).
to Service1
to Service2
to Service3
to Service4
to Service1
*remote terminal mode is an advantageous way of using full-featured program utility to access the data via the Internet, as it results in minimal data traffic over the network.In this mode a user is provided with the program interface, while fetching the data, calculations and rendering are carried out on the server.
8
Flow-chart of the main components
Database
DASSQL
Internet service
SigmaWeb
Internet service
DASTCP
Remote access Data sources
Graphics shell DAS Tools
Internet browser
Program library DAS Access
Utilities: DASТехт, DASLoad, DASDec
Applications and codes. Mathematical packages like MATLAB MDS Plus, IDL
HTTP
TCP
DASFile
Data acquisition systems: T-10 (DASHost) CASTOR TUMAN …
Codes
Decoders
Local archives DASFile
Files: ASCII, media, image, binary
Internet service
DASWeb
9
Unification of data access for various devices
Common DB DASSQL data of device-1 … data of device-N
DASTCP DASWeb
Shared data access
Data warehouse REMOTE USERS The common tools for the analysis of experimental data:
Internet browser DASTools DASAccess-Applications DASMex-MATLAB …
MDSPlus
DASFile
DB of device-1 DASTCP / DASWeb Decoder
DB of device-N DASTCP / DASWeb Decoder
…
Unified databases
DASSQL of device-1 DASTCP / DASWeb
DASSQL of device-N DASTCP / DASWeb
…
TCP/HTTP
This software forms the common information system, which is built over the existing systems of data acquisition and over the existing data bases of experimental devices.
10
NETWORK RESOURCES OF T-10
Network services T-10:DASTCP (for DASTools, DASAccess) 195.9.0.8:8888DASWeb http://cpunfi.fusion.ru/dassql/dasweb2.dll/showgl SigmaWeb http://cpunfi.fusion.ru:8088/SIGMAFRAME.htmlWebsite T-10:http://T10.fusion.ru (also, useful links at the page “Database”)http://T10.fusion.ru/JE page of Joint experimenthttp://T10.fusion.ru/Files page of shared documents (from directory JE T10.WWW)http://T10.fusion.ru/DAST10 page of software DAS T-10Network T-10 disks:Netware network, server Pls:\\pls\nw1 DAS T-10 software and documentation\\pls\nw2 T-10 users archives\\pls\jet10 common directory of JE T10 participantsT-10 disks also can be accessed as:MS Windows network, workgroup Soflab, server NS:\\Softlab\\NS\DASPLS DAS T-10 software and documentation\\Softlab\\NS\T10Common T-10 users archives\\Softlab\\NS\JE T-10 Common directory of JE T-10 participantsT-10 disks also can be accessed via FTP.FUSION.RU (secure SSL is preferred, for example WS_FTP client): /!JE T-10 Common directory of JE T-10 participantsThe Common directory of JE T-10 participants contains two folders:JECommon folder for archives and files exchangeJEFile.WWW files from this folder are published on website http://T10.fusion.ru/Files
11
Structure of data records
Data record
Header Passport of a variable length (up to 2Gb)
Data of a variable length (up to 2Gb)
Name of device
Shot number Name of record Hyperlink Date-time Record type Comment The auxiliary fields
Metadata (XML or the Train of descriptive parameters):
Date-time Text (string) Number Structures (binary
string) Data records
DB DASSQL
The quick reference on shots
(Indexed parameters)
The quick reference on devices,
diagnostics, channels and indexed parameters The main types:
Variable arrays Tables of variable
dimension Media (video) Images ASCII Documents
The fields Passport and Data are not mandatory; they have the variable size and may be compressed.At storage and data transfer via a network, the data compression by a combination of delta-method and open algorithm Zlib is carried out.
12
Tree of records in DASSQL Device 1 (Globus)
… Device N (T-10) Shot 1
… Shot M Indexed parameters for the quick reference on shots
Additional shot’s description (record “0.inf”) Diagnostics Diagnostic 1
… Diagnostic K (@Ece) Record 1
… Record J (Ece.xxx)
SQL search:•Device, Shot, •Date Time •Diagnostics, Name•Commentary•Indexing parameters (B, Ipl,U, …)
Data ownership (Signals and parameters):•Group of names - Ch001..Ch999•Domain – Ch.*
Key: Device-Shot-ParameterName. Values: Double / Datetime / Text
The dynamic table of Indexing parameters:
13
Various ways of data access and viewing.
DAS Tools
MATLAB
DASWeb - Интернет браузер
MDS Plus
14
DASSQL data acquisition system, storage, access and device search
• Choice of data by criteria using the resources of SQL• For pulse experiments• Used in several Russian facilities• Rich set of interfaces• Unified format and the common methods of operation with
various data. • Data compression in 3-6 times at storage and transmission via
Internet.• Remote multi-user operation with data of many devices.• Specialized processing and the unified representation of
results.• Possibility of the subsequent integration with the Grid
infrastructure.
15
DASSQL/DASFile -- MATLAB
Applications/server MATLAB
Mex-functions DASMex
Library DAS Access
Internet service DASTCP
Database DASSQL
Local archive DASFile
Reading from the DASSQL/DASFile » DASMexDevices - get list of devices » DASMexShots - get list of shots » DASMexShotInf - get description of shot » DASMexNames - get list of names » DASMexData - get data » DASMexParamByName - get parameter
ADO/TCP
TCP/IP
Examples:[Names, ErrorMessage] =DASMexNames (‘dassql.ini’, ‘T10’, 37856);[Data, Passport, ErrorMessage] =DASMexData (‘dassql.ini’, ‘T10’, 37856, ‘Bol.d-01’, 1, 500, 1e5, 7e5);
16
DASWEB
• DB DASSQL are automatically published in the Internet.
• The Web-interface provides formation of inquiries through the Internet-browser.
• Dynamic creation HTML and images, compression transmitted by Deflate.
• Data processing are made on the side of a server, results in the form of graphic (GIF), tables (ASCII) or binary (DAS File) and transmitted to the user.
17
SigmaWeb: Run codes Start of a code is carried out on a following chain:{The client, the Internet a browser} - Internet HTTP/Post- {Application server, SigmaWeb} - Files: Input, OutPut, Errors, Status - {the Code}At start on platform SigmaWeb the code representing the simple console application working with files, becomes multi-user and receives the graphic interface in the form of the Internet of a browser. The code can represent group of the executed modules incorporated in a command file. Performance of a code in the multi-user mode is provided with creation of a temporary context for each started copy of a code. All network connections of clients with SigmaWeb are started in separate streams.
SigmaWeb console on Applications server.
18
SigmaWeb: Run math code AT-Light on the Server (Service4)
Задание параметров для кода
Результаты выполнения кода
Setting the initial data -> Viewing in browser the results of the
calculations carried out on the server.Sending request ->
19
DAS TOOLS• Access to a DB via Internet.
• Work with local archives, text, binary and media files.
• Effective navigation and work with data sets.
• Fast mathematical processing - DAS Processing.
• Integrated with the special codes.
• Wide set of means of data presentation.
• Work with big multichannel data and media formats.
20
DAS PROCESSING
1. Methods of the moving average (MA, WMAH, WMAT, Average).2. Median filter.3. Subtraction of shelf (Zero line).4. Transformation of the AXIS «Y» (dimension).5. Transformation of the AXIS «X» (dimension).6. Suppression of casual emissions.7. Inverting of the signal.8. Allocation of positive values of the signal.9. Detailed elaboration.10. Extracting.11. Normalizing the signal on the maximum.12. Calculation of the chord.13. Interpolation given by multinominal of Lagrange.14. Derivatives and integrals.15. Data presentation of multichannel diagnostics in the form of structures and
surfaces.16. Addition, multiplication and division for groups of signals.17. Fourier analysis.18. Use of graphics of harmonious fluctuations.19. Fitting of function approximation of many variables to experimental data.
Methods DAS Processing are accessible through network services DASWeb and DASTCP, library DAS Access and task interface DASText.
21
Nuclear Fusioninter-facilities collaboration
A lot of standards applied
Collaboration:No common interoperation
standardAsdex
Mast
JET
TCVTore Supra
W-7X
L-2M
GOL-3
T-11M
T-10
Globus
FTU
Textor
22
EGEE
infrastructuremiddleware &
Asdex
Mast
JET
TCVTore Supra
W-7X
L-2M
GOL-3
T-11M
T-10
Globus
FTU
Textor
Unified interoperation needed
Nuclear Fusioninter-facilities collaboration
23
PROSPECTIVE
The DAS is going to be modernized to accommodate increasing amounts of experimental data.Modern electronic equipment and computational concepts are going to be used,including blade-servers and parallel multi-channel data processing.
The System future development will keep up a cooperation with other researchers for working out common standards for better interoperability of different data-systems in nuclear fusion experimental field.This cooperation includes issues of integration into scientific structures based on Grid-technologies and other prospective formations.
The System as the whole or its parts can be used in different applications: - development of new data-systems for any devices;- integration its data viewing/processing/analysis utilities into existing data-systems;- storing large amounts of data from other data sources in the System database with ability to manipulate this data with the DAS utilities.
24
CONCLUSION
This software forms the information system, which is built over the existing systems of data acquisition and over the existing data bases of experimental devices.
At present, the records from devices: T-10, GTB, PN-3, PS-1, S-300 (RRC KI), CASTOR (host lab experiment, IPP, Praguesep.2005) and also separate shots from devices: TUMAN-3M, GLOBUS, L-2M are presented in the DB DASSQL.
The following possibilities of operations with experimental data have been realized:
• Choice of data by criteria using the resources of SQL• Rich set of interfaces• Unified format and the common methods of operation with various data. • Data compression in 3-6 times at storage and transmission via Internet.• Remote multi-user operation with data of many devices.• Specialized processing and the unified representation of results.• Possibility of the subsequent integration with the Grid infrastructure.