Embed Size (px)
DESCRIPTION
ICAT Integration at ISIS. Tom Griffin, ISIS Facility ICAT Developer Workshop The Cosener’s House, Abingdon August 2009 [email protected]. ISIS. Neutrons and Muons Operating since 1984 1600 user base; ~200 staff Volume of data ~6TB ~5M datafiles (27 instruments, 500/hour) - PowerPoint PPT Presentation
Citation preview
ICAT Integration at ISIS
Tom Griffin, ISIS FacilityICAT Developer WorkshopThe Cosener’s House, Abingdon
August 2009
ISIS
• Neutrons and Muons• Operating since 1984• 1600 user base; ~200 staff• Volume of data ~6TB• ~5M datafiles (27 instruments,
500/hour)• >6.7GB metadata, >33M rows
Current Pre-Experiment
Data.ISIS
Data.ISIS
• Main point of access for ISIS data• Maintains index of Run number ->
Folder– GEM12345– \\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw– .txt .log .sav0
• Ranges• Zip files
Data.ISIS Authentication
Data Portal
ICAT API
Data.ISIS
User performs search via application e.g. Data Portal
Search is executed in ICAT
Permitted results are returned to application
11
11
22
33
33
22
Results are displayed to the user44
44
55
User performs request to download datafile, multiple datafiles or dataset
55
66
ICAT creates http GET link and passes to back to user (routed through application)
sessionIdemail (optional)fileId(s) or datasetIdaction (i.e. download, zip,
compressed)
66
User clicks http link77
77
Data.ISIS call ICAT API to check permissionssessionId & datafileId(s) or
datasetId
88
88
Return Exception on failure or DownloadObject on success- userId- array [filename, cycle, run
number]
99
99
User gets their data!1010
1010
Current Pre-Experiment
‘To Be’ Pre-Experiment
Problems
• RB Numbers• Business System
• Not released to users• Replace DataPortal with TopCat
Related Projects
• Grid MP – distributed PC network• Analysis and simulation• New portal required anyway ->
1) Single point of access for Grid MP2) Use ISIS data in Grid jobs3) Store results and process in ICAT
Where to store data?
Freddie Akeroyd (Data.ISIS)Chris Moreton-Smith (ISIS DFS/FRS)Hannah Guest (Facilities Business System & ERAS)Matt Clarke (SECI modifications)
Damian FlanneryStuart Ansell (writeRaw)
Laurent Lerusse (NXIngest)
Database Services, Roger Downing, FBU IT Service Desk (servers and hosting)
Core ICAT – Glen, Shoaib, Michael.....
Acknowledgments(who did what at ISIS)
ISIS Computing Group
ISIS Data Analysis Group
Questions...
