1Lee Lueking

The Sequential Access Model for Run II Data Management and Delivery

The Sequential Access Model for Run II Data Management and Delivery

Lee Lueking, Frank Nagy, Heidi Schellman, Igor Terekhov, Julie Trumbo, Matt Vranicar, Rich Wellner,

Vicky White.

URL: www-d0.fnal.gov/~lueking/sam/sequential.html.

CHEP98

Sept. 3, 1998

2Lee Lueking

What is The Sequential Access Model: SAM?

What is The Sequential Access Model: SAM?

Sequential events: Data is stored in files as sequential events. Data Tiers: Each event is stored in each of several data tiers.

» The Event Data Unit (EDU) is the unit of data stored in each tier.

» Physical event size: EDU5=5kB/event, EDU50=50kB/event, et cetera.

Physical streaming (clustering): Data categories based on Trigger or reconstruction information

Database catalog: File, Event and Processing Database; Information about the data - event-level, file-level, run-level. Also processing information; static and dynamic.

3Lee Lueking

Data OrganizationData Organization

Physical Clustering

File & Event Database

EventInformationTiersWarm

Cache

User and physics group(derived) data

4Lee Lueking

How Do I Access Data?How Do I Access Data?

Pipelines: Data access channels tailored for particular processing and analysis patterns.

Pipeline segments: Tapes, drives + Automated Tape Library + Storage Management System, network, group-shared and/or user-private analysis disk.

Example access modes:» Database: Access to event, trigger & other FEDB info. » Thumbnail: Disk resident sketch of each event. » Freight Train: Large data stream file server. » Event Picking: Random event selection from any data tier.» Small Data-set: One or a few files from any data tier.

5Lee Lueking

Data AccessData AccessMass Storage Pipeline Consumers

=Disk Storage

=Tape Storage

=File

=Event

=Data flow =Group of Users

=Single User=Pipeline Name

6Lee Lueking

D0 SpecificationsD0 Specifications Data sizes

Per event Total Storage Comment

Events 1.2e9 1.2e9Raw size 250kB 300 TBLarge EDU 300 kB * 75-100 TB *< 20% of events

Medium EDU 50-100 kB 60-120 TB

Small EDU 5-10 kB 6-12 TB Stored on diskEvent DB ~150 B ~200 GB Stored on diskFile DB ~100 GB Stored on disk

Further details» 10-15 exclusive streams preferred. Based on L3 and/or Reconstruction

information.» 10% warm (tape or disk) caches of Raw and Medium EDU data.» Possible on-demand reconstruction.

7Lee Lueking

Will SAM Scale to Run II?Will SAM Scale to Run II?Access Time for 5,50, and 250 kB/event

D0 data sets - Drive Read speed 5MB/sec

0

2

4

6

8

10

Stream Size Relative to Total Data

Mo

nth

s t

o A

cc

es

s

5 kB/ev

50 kB/ev

250 kB/ev

8Lee Lueking

Exclusive StreamingExclusive StreamingExclusive Streaming

Jet47%

e11%

e+Jet8%

Low Pt m + nojet13%

Jet

e

e+Jet

M Et

M Et+jet

M Et +e

M Et+jet+e

High Pt m

High Pt m +jet

High Pt m + nojet +others

High Pt m + jet +others

Low Pt m + nojet

Low PT m + jet +others

Low Pt m + nojet +others

Low PT m + jet +others

Luminosity

Zero BiasSee Talk #182: Heidi Schellman, “Assurance of Data Integrity in a Petabyte Data Sample”

9Lee Lueking

Data Handling SystemFarm User ProcessesAnalysis and Creation of DataSets -- User Processes ----

Buffer CacheDisk

Data Files Databases

Tapes in Robot

Tapes on a shelf outside a Robot

SupportingDatabases

Data Access

FarmProduction

Storage ManagementSystem (Enstore)

(SAM)

RIP(RAW datato tape)

File/VolCatalog

Event/FileCatalog + Bookkeeping

Run Conditions

Joint Projects related to Data Handling System

Data Files

?

Buffer and Cache

10Lee Lueking

SAM Design DetailsSAM Design Details Network distributed. Easily scalable. Works for all access modes. Uses CORBA interfaces between modules. Modules being written in JAVA, Python and C++. File, Event and Processing Database uses ORACLE 8. Not tightly coupled to:

» Tape Mass Storage System.» CPU availability or Batch processing facilities on Farm or Analysis

machines. » The D0 event data model.

11Lee Lueking

Main ComponentsMain Components File and Event Database: Info about data location and

processing details. (see poster session #127: Vicky White, “Use of ORACLE in Run II for D0” )

Global Optimizer: Optimizes tape access and regulates bandwidth to various stations and activities.

Station: Management for a set of processing resources, including buffer and Data I/O.

Project Master: Responsible for managing projects which are lists of files to process.

Consumer/producer: Actual data processing GUI and API user interfaces: Allow users to access data and

administrators to control the system.

12Lee Lueking

Components of SAMComponents of SAM

Station A

Mass StorageSystem

Global Optimizer

Project Master

Station B

Station C

Station D

Station E

Station F

Consumer/Producer

ProjectUser & Admin.Interface

(API and GUI)

DB and Information

Servers

Project

Project

Project

Consumer/Producer

Consumer/Producer

Consumer/Producer

Consumer/Producer

13Lee Lueking

FilesID

NameFormat

Size# Events

File and Event DatabaseFile and Event Database

EventsID

Event NumberTrigger L1Trigger L2Trigger L3

Off-line FilterThumbnail

Volume

Project

Data Tier

PhysicalData Stream

TriggerConfiguration

Processing Info

Run

Event-FileCatalog

14Lee Lueking

(Mass Storage System Needs)(Mass Storage System Needs) Provide access to data through file-level semantics. Manage all tape activity within the ATL(S) and to/from shelf. Allow data to be physically clustered in tape groupings or “file

families”. A mechanism for sending priorities with file requests to allow

control over allocation of resources for various activities. System must optimize the use of resources such as arm time

and tape mounts. Retry and fail-over features for failed tape read/write activities. Open tape format to allow removal of tapes and exchange of

data with other sites. Reliable and unattended operation.

See ENSTORE presentation #126: Don Petravick, “ENSTORE - An Alternative Data Storage System”

15Lee Lueking

Access to Data through SAM

Access to Data through SAM

•

User or group defines a “project” by sending a list of constraints or file list to the Database Server.

DB Server returns a summary of the project (number of files, size and availability).

User is provided a list of possible “stations” where the project might run. He chooses one.

User registers with the station for a given (new or existing) project. He is given a unique “key” to use.

User’s client “consumer/ producer” sends the “project master” on the chosen station the “key”, and is given the next available file in the “project”.

16Lee Lueking

Consumer- Read from Storage

Consumer- Read from Storage

17Lee Lueking

Producer - Write to Storage

Producer - Write to Storage

18Lee Lueking

SAM PrototypeSAM Prototype

Status: Being built, ready early October. Goals:

» Populate and exercise the SAM database.» Specify projects - data to be accessed for processing or analysis. » Attach to a ‘Station’ which makes files for that Project accessible.» Interface to ENSTORE - get/put files - using SAM “Global Optimizer”.» Build Analysis programs using D0 framework.» Demonstrate multiple Stations, Projects, Analysis consumers .

Testing: Further testing in fall with SAM PC test-bed. Beta version: Plan to make MC data available

through SAM late ‘98.

19Lee Lueking

SAM Prototype PC test-bed

Example configuration

Enstore Warehouse

Consumers/Producers

SAM Station ServersNetworkHUB

Main BackboneTo Database Server

20Lee Lueking

SummarySummary Dzero plans to use a file based Sequential Access Model for

run II data access. The design is network distributed with CORBA communication

between modules written in JAVA, PYTHON and C++. ORACLE 8 is used for the DB.

A SAM prototype is being built now and will be ready in Early October.

Hardware to construct a SAM test-bed will be assembled this

fall to more fully test and understand the system. We plan to employ the system for MC data by the end of `98,

and perform large-scale testing with Run II hardware the first part of next year.