Enterprise Removable Media Manager

IBM TotalStorage

© 2005 IBM Corporation

Enterprise Removable Media Manager

A Service Offering forOpen System Tape Management

Ulf Troppens, eRMM Development, IBM [email protected]

2 Enterprise Removable Media Manager (eRMM) © 2005 IBM Corporation

IBM TotalStorage

Disclaimer


IBM TotalStorage

Agenda

eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSMeRMM RoadmapConclusion


IBM TotalStorage

Agenda



IBM TotalStorage

Customer Problems and Requirements today

Libraries with SCSI Media Changer(e.g. IBM 3584)

Removable Media Hardware

Media Changer Devices

Streaming Devices (Drives)

SCSI/FC SCSI/FC

SAN

Removable Media Hardware

Media Changer Devices

Streaming Devices (Drives)

SCSI/FC prop. InterfaceLibrary Mgmt.

Libraries with integratedManagement (e.g. IBM 3494)

Problems:• No centralized access control, administration and reporting

• Media management has to be done by every application

• Difficult (or nearly impossible) to share libraries, drives and scratch pools, especially in heterogeneous environments

• Multitude of Media Changer andLibrary Management Interfaces(SCSI, IBM 3494, STK ACSLS/LS, …)

• Adding storage or upgrading storage technology requires changes to every application

Requirements:• Improved manageability to cope with rapidly growing storage• Reduced TCO• Efficient usage of tape storage capacity• Increased utilization of tape drives• Integrated support for vaulting


IBM TotalStorage

Customer Scenarios

…

Statically partitioned IBM 3584

SAN

Legato Networker and HiBack Server

TSM Server

…EDT

EDT

EDT

STK SN 6000

STK Silos

STK ACSLSServer

…

IBM Tape Hardware

SAN

TSM on z/OSTSM LAN-free clients for open systems

Customer wants:• consolidated media

management for Open Systems first, later with z/OS too

• centralized administration and control

• to get rid of Gresham EDT

Customer wants:• enhanced utilization of drives• to create drive groups and

cartridge pools that span multiple libraries

• centralized media management administration and control for Networker and HiBackapplications

• enhanced monitoring and reporting

Customer wants:• consolidated media

management for heterogenous environment

• Integrated support for vaulting• policy-based cartridge

lifecycle management

Scenario IScenario II

Scenario III


IBM TotalStorage

Case Study: Backup Consolidation on zLinux (1/2)

Business needThe customer wants to consolidate its backup infrastructure with TSM on zLinux.

Business valueThe customer builds a backup infrastructure which is very flexible and which scales very well. The advanced management functions of zLinux/VM allow to provision new TSM Servers on demand.

Use case (planned new service)At the customer a business unit can request to establish a new security zone including a dedicated VPN to protect certain applications and its confidential data. The infrastructure team automatically creates a new zLinux Instance with a new TSM server which is only visible in the new VPN. The concept of having a dedicated TSM server for each security zone simplifies the mechanisms to secure confidential backup data.


IBM TotalStorage

Case Study: Backup Consolidation on zLinux (2/2)

Similar setup available in collocated data centerBoth data centers connected via DWDM (Gigabit Ethernet + Fibre Channel)Fail-over of zLinux instances between both data centers planned for laterNew TSM servers added on demand

eRMM TSM1

TSM2

… TSMn

TSMn+1

TSMn+2

TSMn+3

… TSMn+m

z/VM 1 z/VM 2

z900

40 zFCP Ports for Tape, 8 zFCP Ports for disk

Fibre Channel SAN

IBM 3584 Tape Library65 x IBM 3592 Tape Drive2320 x 3592 JA Cartridge

IBM DS 830014,7 TB RAID 5 netDisk cache for TSM

eRMM’s advanced tape management capabilitieshelp to master the complexity of this frequently changing setup.


IBM TotalStorage

Value of the Enterprise Removable Media Manager

Improve the flexibility of the removable media infrastructureResults

Improved Application Availability– Create cross library drive and scratch pools

Optimized Storage Resource Utilization– Share drives between heterogeneous applications– Provide a cross application scratch pool– Define policies that priories concurrent mount requests– Define policies to move cartridges to vaults

Enhanced Storage Personnel Productivity– Create a single point of control, administration, reporting, security and auditing

for removable media resources – Move, add or change removable media resources (drives, cartridges, libraries)

without requiring modifications to applications like TSM


IBM TotalStorage

Tape Configuration without eRMM

SAN

IBM 3584 IBM 3494 StorageTek


IBM TotalStorage




SAN


IBM TotalStorage


SAN

IBM 3584


IBM 3494 StorageTek

TSMCartridges

Networker Cartridges

STK DrivesLTO Drives

Common Scratch Pool

Drive Pool• Pools of capacity• Segmented basedon business need

Scratch Pool• For all applications• Segmented basedon business needs

Cartridge Pool• Access for owningapplication only


IBM TotalStorage


SAN

IBM 3584


IBM 3494 StorageTek

TSMCartridges

Networker Cartridges

STK DrivesLTO Drives

Common Scratch Pool

Policies• Drive and cartridgeallocation based onpriorities

• Cartridge movement• Cartridge life-cyclemanagement

New Efficiencies• Cross applicationdrive sharing

• Cross applicationscratch pool

• Mount request queuing• Centralized reporting• Audit trails• Integrated vaulting


IBM TotalStorage

Dynamic grouping and pooling of drives and cartridges

DriveGroup 1 DriveGroup 2

CartridgeGroup 1

Cartridge Group 2

Library 1 Library 2

DriveGroups:• Used to aggregate drives• Drives can be added and removed dynamically

• Used to implement access permission model and preferential usage policy

• May span multiple libraries

CartridgesGroups:• Used to aggregate cartridges• Cartridges can be added and removed dynamically

• Used to implement access permission model and preferential usage policy

• May span multiple libraries


IBM TotalStorage

Access Roles and Preferential Usage Policy for DriveGroups and CartridgeGroups

DriveGroup 1 DriveGroup 2

CartridgeGroup 1

CartridgeGroup 2

Library

App 1 App 2

100

10

10

100

100

100

100

100

Preferential Usage Policy:• DriveGroup-Application relations have priority attribute

• Drives have priority attribute too• Priority is used by eRMM to find drive for a mount operation

Access Roles:• Applications are subject to access control• Two – level access permission model• Administrative applications may access every Drive• Standard applications may only access Drives within DriveGroups which have been assigned to them by an administrative application


IBM TotalStorage

eRMM Features

Dynamic library and drive sharing across heterogeneous application boundaries and heterogeneous operating systemsDynamic drive and cartridge poolingMount request queuingPolicy-based drive and cartridge allocationPolicy-based media life cycle management (*)Integrated off-site media management and tracking (Vaulting) (*)Centralized access control, administration and reportingAdvanced reporting and auditing (*)

(*) Planned for future releases


IBM TotalStorage

SAN

App 3App 2App 1

SAN

App 3App 2App 1

Media Management and Sharing

Eliminate changes when storage is added and minimize impact of technology upgrades.

Adding storage or upgrading storage technology requires changes to every application

Provide integrated support for manually operated libraries and vaults together with policy-based cartridge life cycle management (planned for future release).

No integrated support for Vaulting

Provide centralized access control, administration and reporting accessible through open standards based API (IEEE1244) and SSG Console WUI

No centralized access control, administration and reporting

Provide centralized mainframe-class media management accessible through open standards based API (IEEE1244)

Media management has to be done by every application

Provide dynamic sharing of tape drives including mount request queuing + additional utilization enhancements through dynamic grouping and pooling of drives

Inefficient drive utilization because its difficult to share drives (physical mapping App->Lib. Partition)

Provide access to and sharing of a common scratch pool and dynamically resizable cartridge groups with access control.

Inefficient capacity usage because tape storage is statically partitioned by application

ERMM SolutionTop Customer Problems/Requirements


IBM TotalStorage

Agenda



IBM TotalStorage

eRMM Architecture

IBM 3584 IBM 3494 STK ACSLS

Media Manager

LibraryManager

SAN

eRMM

Admin Console

DM DM DM DM


IBM TotalStorage

eRMM Architecture


Media Manager

LibraryManager

SAN

eRMM

Admin Console

•Coordinates accessto drives and cartridges

• Central repository• Log of all activities

• Controls libraries onbehalf of the mediamanager

• Encapsulateslibrary hardware

• Command lineinterface and GUI

DM DM DM DM

• Controls driveson behalf of themedia manager


IBM TotalStorage

eRMM Control Flow


Media Manager

LibraryManager

SAN

eRMM

Admin Console

(1.) TSM Server wants to mount a scratch tape

(2.) TSM ELM sends request to Media Manager via TCP

DM DM DM DM


IBM TotalStorage

eRMM Control Flow


Media Manager

LibraryManager

SAN

eRMM

Admin Console

(3.) Media Manager selects scratch cartridge and idle drive

(4.) Media Manager forwards request via TCP

(6.) Drive Manager loads cartridge into drive

(5.) Library Manager moves cartridge to drive

DM DM DM DM


IBM TotalStorage

eRMM Control Flow


Media Manager

LibraryManager

SAN

eRMM

Admin Console

(8.) Response to TSM Server via TCP and ELM

(7.) Response to Media Manager via TCP

DM DM DM DM


IBM TotalStorage

eRMM Control Flow


Media Manager

LibraryManager

SAN

eRMM

Admin Console

(9.) Direct access to drive

DM DM DM DM


IBM TotalStorage

Agenda

eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSM eRMM RoadmapConclusion


IBM TotalStorage

eRMM Offline Demo (Screenshots)

I. SetupII. Drive and cartridge sharing

(between EMC/Legato Networker and TSM)III. ReportsIV. Advanced Reporting


IBM TotalStorage

Setup (1) – managed libraries


IBM TotalStorage

Setup (2) – managed drives


IBM TotalStorage

Setup (3) – managed cartridges


IBM TotalStorage

Setup (4) – authorized applications


IBM TotalStorage

Setup (5) – drive groups


IBM TotalStorage

Setup (6) – drive groups (authorized applications)


IBM TotalStorage

Setup (7) – cartridge group


IBM TotalStorage

Setup (8) – cartridge group (authorized application)


IBM TotalStorage

Sharing (1) – common scratch pool


IBM TotalStorage

Sharing (2) – Networker begins using one drive …

1. Allocate scratch volume

3. Start backup

2. Label volume


IBM TotalStorage

Sharing (2) – TSM wants two drives …

4. TSM is waiting for second drive

3. MOVE DATA needs two drives


IBM TotalStorage

Sharing (3) – eRMM synchronizes drive access …

5. TSM is blocked


IBM TotalStorage

Sharing (4) – Networker is done and TSM continues …

6. Networker is ready and unmounts cartridge

7. TSM gets second drive


IBM TotalStorage

Reports (1) – drive “LTO263” was shared

Drive Sharingbetween TSMand Networker


IBM TotalStorage

Reports (2) – cartridge groups have changed


IBM TotalStorage

Reports (3) – drive usage statistics


IBM TotalStorage

Advanced Reporting

The following screen shotsshow prototyped features,

which may become part of eRMMin a future release.


IBM TotalStorage

Advanced Reporting (1a) – Audit trailsStep 1: Select filter in GUI

Which Cartridge was mounted in

“DRIVE1” in January 2004?


IBM TotalStorage

Advanced Reporting (1b) – Audit trailsStep 2: Download report as CSV file


IBM TotalStorage

Advanced Reporting (1c) – Audit trailsStep 3: Import CSV file into Excel


IBM TotalStorage

Advanced Reporting (2a) – Error logsCentral error log file for all drive errors


IBM TotalStorage

Advanced Reporting (2b) – Error logsDetails for a specific error


IBM TotalStorage

Advanced Reporting (2c) – Error logsError statistics by day


IBM TotalStorage

Advanced Reporting (3a) – Drive UtilizationTotal number of drives in use


IBM TotalStorage

Advanced Reporting (3b) – Drive UtilizationUtilization of a single drive


IBM TotalStorage

Advanced Reporting (4) – PerformanceTotal throughput by day


IBM TotalStorage

Agenda



IBM TotalStorage

Value add in a homogeneous TSM environment (1/6)

TSM provides its own tape management capabilities(e.g. TSM Library Sharing)eRMM is designed to support heterogeneous applications

Why do I need eRMM in a homogeneous TSM environment?


IBM TotalStorage


Flexible provisioning of drives and cartridges– sharing of TSM drives with other applications like EMC Legato NetWorker

and OS tools like mksysb and tar– simplified setup of TSM LAN-free environments

no need to define tape paths inside TSMall tape paths are automatically generated inside eRMMcentral check whether drive is used by TSM Server or Storage Agent

– flexible adding and removing of drives to TSM storage poolsall changes are handled inside eRMMupdates of the TSM servers or other applications are not required

– enabling of cross library scratch pools– enabling of cross library drive pools– automatic labeling of scratch cartridges (can be disabled in config file)


IBM TotalStorage


Advanced reporting and auditing– eRMM provides statistics on how many drives were used when by

which application (future).– eRMM provides statistics on how many data was written during a

mount. This data can be aggregated to provide throughput reportsfor drives, drives groups, applications and a whole library.

– eRMM provides statistics on how often a cartridge was mounted and how many data was written/read for the whole cartridge life cycle (future). TSM looses its data, when a cartridge is returned to the TSM scratch pool.

– The TSM server stores tape alerts in the TSM activity log. In addition to that, eRMM stores all log sense pages in the eRMM database. This additional data can be very useful for error analysis.


IBM TotalStorage


Enhanced access control to library resources– In an outsourcing environment eRMM can share the drives across several

TSM servers whilst it strictly separates the cartridges of each TSM server.– eRMM can also provide a separate scratch pool for each TSM server thus

cartridges are not exchanged between the TSM servers.

Additional operational benefits– dismount of cartridge when TSM server goes down– physical erase of cartridges before return to scratch pool (future)– improved cartridge life-cycle management

(e.g. threshold for wornout tapes) (future)


IBM TotalStorage

Value add in a homogeneous TSM environment (5/6)Customer EnvironmentThe customer backs-up a large application server using TDP for Oracle. This backup job starts four sessions and thus uses four drives.

Customer IssueWhen TSM starts a backup of the TSM database and all drives are in use, then TSM dismounts one volume and uses it for the TSM database backup. As a result the backup of the large application server fails, even when it was running for several hours and even when this backup job was nearly complete.

eRMM SolutionConfigure TSM with two external libraries which use eRMM managedresources. Use one library for the backup of the application server and the other for the backup of the TSM database. Inside eRMM map these two libraries to the same resources (DriveGroup, CartridgeGroup).

As a result eRMM will queue the backup of the TSM database.Now the TSM administrator can decide which job to cancel.


IBM TotalStorage


TSM provides its own tape management capabilities(e.g. TSM Library Sharing)eRMM is designed to support heterogeneous applications

eRMM’s advanced tape management capabilitiesprovides value add even in a homogeneous TSM environment!


IBM TotalStorage

Agenda



IBM TotalStorage

Supported platforms (eRMM 1.2)eRMM Media Manager

– SLES 8 for zSeries GA– AIX 5.2/5.3 GA– SLES 8 for Intel RPQ

Applications– Tivoli Storage Manager GA– Native OS commands (e.g. tar, mksysb) GA– EMC Legato Networker RPQ– Veritas NetBackup future– iSeries BRMS future– zSeries DFSMSrmm future– Other backup applications future

Tape Libraries– IBM 3584 Open Systems GA– IBM 3494 Open Systems GA– StorageTek ACSLS GA– StorageTek HSC-LS RPQ– Other tape libraries future


IBM TotalStorage

Implemented features (eRMM 1.2)

Dynamic/heterogeneous drive sharing GADynamic/heterogeneous drive and cartridge pooling GAMount request queuing GATape library virtualization GAPolicy-based drive and cartridge allocation GACentralized access control, administration and reporting GAPolicy-based cartridge lifecycle management futureOffsite-media management and tracking (vaulting) futureAdvanced reporting and auditing future


IBM TotalStorage

Agenda



IBM TotalStorage

Summary

eRMM is an enterprise tape library virtualization solution whichscales very well and provides very high availability and reliability.eRMM provides mainframe class, centralized management functions like tape cartridge pooling, tape drive pooling, tape drive sharing, mount request queuing, policy based cartridge lifecycle management, offsite media management, centralized access controland auditing, and advanced reporting functions.eRMM’s intelligent sharing capabilities increase tape drive, tape cartridge and therefore tape library utilization whilst it reduces the time spent for tape library administration.

The goal of eRMM is to dramatically reduce the Total Cost of Ownership for data stored on tape by improving its access and its management across heterogeneous application, server, and tape library boundaries.


IBM TotalStorage

eRMM Resources

Internethttp://www.ibm.com/Search/?q=eRMM&v=14&lang=en&cc=us

eRMM Business DevelopmentDetlef KrauseMail: [email protected]: +49 171 3 33 87 41

Documents

Enterprise Removable Media Manager