Upload
red
View
35
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Moving Pre-Archive Pipeline Processing March 14, 2003 Forrest Hamilton/OPUS Ops. Goals. Switch over will not disrupt the routine flow of science data processing or archiving Free up processing power for OTFR on ACDSDOPS Will help when ACS data become public - PowerPoint PPT Presentation
Citation preview
Implementation Review 1
Moving Pre-Archive Pipeline Processing
March 14, 2003Forrest Hamilton/OPUS Ops
Implementation Review 2
Goals Switch over will not disrupt the routine flow of
science data processing or archiving Free up processing power for OTFR on
ACDSDOPS Will help when ACS data become public
Gain experience to use when we switch over the more complex OTFR pipelines
Implementation Review 3
Plan Switch pre-archive science processing space over
from ACDSDOPS storage array to EMC Migrate pre-archive science processing on
ACDSDOPS to the Sun Fire 15K Both of these configurations will have already
been successfully tested in the new test environment
Implementation Review 4
Implementation Review 5
EMC StorageArray
2GB Switch
SUN FIRE15K
EMC BackupManager
SUN BackupManager
HST
FROM: ScienceCommunity(thru StarView)
FUSEEngineering Team
TO: ScienceCommunity
Fatkat
Imagemaker
Fiber Channel
Ethernet 10/100Connections to
Institute Networkand Internet
ARCHC SCARAB
Ops-Jukeboxes
Corsair
Dev-Jukeboxes
DVD DrivesW DVDW DVD
W DVD
DVD DrivesW DVDW DVD
W DVD
CD Burners
CD Burners
Implementation Review 6
OPUS Installation Primer
opus_login.csh
# setenv SOGS_DISK setenv OAPI_LIB setenv OPUS_DEFINITION setenv OPUS_HOME_DIR setenv DSQUERY setenv OPUS_DB setenv jref setenv jtab
opus_definitions_dir =
*.path
STAGE_FILE =OPUS_HOME_DIR =OPUS_TEMP_DIR =POD_IN_DIR = ACS_CAL_DIR = STI_CAL_DIR = NIC_CAL_DIR =WF2_CAL_DIR
*.pipeline
dp_acs professor acdsdopsss_acs professor acdsdopsdv_acs professor acdsdopswcsacs professor acdsdopsgc_acs professor acdsdopscalacs professor acdsdops
/home/opus/defs/ /store/opus14_2/defs/unix/
source opus_login.csh
.alias
Pre-archive pipeline process
Implementation Review 7
Post-installation Configuration
opus_definitions_dir
/home/opus/defs/
Ops
files
OPUS Installation Primer
opus_definitions_dir
Nominal Configuration
Ops
files
DSB
files
/home/opus/defs/
/store/opus14_2/defs/unix/
Pre-installation Configuration
Ops
files
DSB
files
Modified Ops files
New DSB files
/home/opus/defs/
/home/opus/opus14_3/
/store/opus14_2/defs/unix/
/store/opus14_3/defs/unix/
opus_definitions_dir
Modified Ops files
/store/opus14_3/defs/unix/
New DSB
files
Implementation Review 8
Migration Process: ACDSDOPS EMC Perform pipeline account access checks Deployment Review
Needed to determine readiness and appropriate time for switch to EMC
Copy OPUS software onto EMC No new OPUS software installation required
Create a copy of pipeline configuration files and edit in the changes needed to point to the EMC
Standard procedure used for OPUS software upgrades Terminate the CDBS (reference files) pipeline Copy reference data onto EMC
Implementation Review 9
Migration Process: ACDSDOPS EMC
Terminate pre-archive pipeline processing to allow data to clear off of the ACDSDOPS disks Done to prevent major cleanups if problems arise Migrate any trouble data over
Activate modified configuration files Bring up pre-archive processing using EMC
Implementation Review 10
Migration Process: ACDSDOPS Sun Fire 15K Verify PACOR-A interface Verify DADS interface Migrate OPUS pipeline configuration directory hierarchy
from ACDSDOPS to Sun Fire 15K Similar process has already been done to activate the EDPS
Verify pipeline account access Deployment Review Terminate pre-archive processing on ACDSDOPS Install the OPUS software version that supports Sun Fire
15K processing
Implementation Review 11
Migration Process: ACDSDOPS Sun Fire 15k Activate configuration files pointing to the Sun
Fire 15K Make use of new reference file area (GEIS format is
OS dependent)
Bring up pre-archive processing using the Sun Fire 15K
Implementation Review 12
Risk Management Transitioning the easiest and most stable pipeline over
onto new hardware first Migration of OTFR, which is a more complex pipeline, will occur
later If problems arise, we can revert back to ACDSDOPS until
problem resolution This is the case for either transition
ACDSDOPS EMC ACDSDOPS Sun Fire 15K
The pre-archive pipeline will contain as little data as possible (a.k.a draining the pipes)
Again this will be done for both transitions
Implementation Review 13
Schedule: EMC
March 26, 2003: Connect 0.5 TB of EMC space to ACDSDOPS
March 28, 2003: Aliveness Test for ACDSDOPS/EMC
April 7, 2003: Acceptance Test for ACDSDOPS/EMC
April 14, 2003: Deployment Review April 21, 2003: ACDSDOPS/EMC operational
Implementation Review 14
Schedule: Sun Fire 15K