Oxford University TSM Symposia - Preview of Future ...tsm-symposium.oucs.ox.ac.uk/2007/papers/Dave Cannon...Preserve externalized TSM SQL schema so customer-developed queries and autti

Tivoli Storage, IBM Software Group

Preview of Future Enhancements in Tivoli Storage Manager

D CDave CannonTivoli Storage Manager ArchitectOxford University TSM SymposiumSeptember 2007

© 2007 IBM Corporation


DisclaimerDisclaimer

This presentation describes potential future enhancements to the IBM Tivoli Storage Manager family of productsStorage Manager family of products

All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice and represent goals and objectiveschange or withdrawal without notice, and represent goals and objectives only

Information in this presentation does not constitute a commitment to deliver pthe described enhancements or to do so in a particular timeframe

IBM reserves the right to change product plans, features, and delivery h d l di t b i d d i tschedules according to business needs and requirements

This presentation uses the following designations regarding availability of potential product enhancementspotential product enhancements– Planned 5.5: Planned for delivery in TSM v5.5 (2007) – Next Release Candidate: Candidate for delivery in the next release after v5.5

Future Candidate: Candidate for delivery in future release

© 2007 IBM Corporation2 Preview of Future Enhancements in Tivoli Storage Manager

– Future Candidate: Candidate for delivery in future release


Symposium Presentations on Future DirectionSymposium Presentations on Future Direction

Session Title Presenter Future Topics Coveredp

Trends and Directions Freddy Saldana TSM roadmap 5.5 and next release items (high level)

Future Enhancements Dave Cannon

DatabaseManagement of stored dataAdministrationS itSecurity

HSM Stefan Bender TSM for Space ManagementTSM HSM for Windows

Copy Services Del Hoobler TSM for Copy ServicesTSM for Advanced Copy Services

Understanding Disk Storage Dave Cannon Sequential-access diskg g q

Data Deduplication Dave Cannon Deduplication



TopicsTopics

Database enhancement

Management of stored data

Administration

Security



Database Evaluation and Decision CriteriaDatabase Evaluation and Decision Criteria

Assessment of customer valuePerformance evaluation and optimization using prototype TSM servers with DB2Interviews and other customer input

Enhance proprietary database

Upgrade database to DB2

Interviews and other customer inputExperience from integration of other IBM products with DB2Extensive collaboration with DB2 teams and IBM Research

Primary Evaluation Criteria Based on Customer Value

Ease of AdministrationPerformanceScalability

Server Availability

Other Considerations

Evaluation Result: Decision to

upgrade proprietary

Development Cost

Other Considerations

Impact of Upgrade

Reporting & Monitoring

Total Cost of Ownership

upgrade proprietary database to DB2



Upgrade of TSM Database to DB2Upgrade of TSM Database to DB2

TSM server will require use of DB2 as database

Continue ongoing maintenance of prior releases with proprietary database

Consider extending support time for previous release to allow time for customers to upgrade to release with DB2

Ongoing, active consultation with TSM customers


Next Release CandidateIncreased availability and scalability of TSM server


DB2 TSM Server AvailabilityDB2 TSM Server AvailabilityGoals

Online, automated reorganization of database while server operations continue Eliminate or greatly reduce need for offline database repair through exploitation of online integrity checks and repair

PlansPlansExploit existing online reorganization capability in DB2 (will consume available CPU cycles)Exploit DB2 capabilities for real-time integrity validation

–Foreign-key relationships for referential integrity–Constraints and triggers–Type checking

Implement alternate indices to avoid out-of-synch conditions between base tablesIf integrity errors are detected automatically take corrective action without offline database auditIf integrity errors are detected, automatically take corrective action without offline database audit Customers can optionally use DB2 HADR (High Availability and Disaster Recovery) to reduce impact of planned and unplanned outages

DB2Online database repair

or reorganization (compaction)


TSM Server(compaction)


Backup of TSM DB2 DatabaseBackup of TSM DB2 Database

Database and recovery log will be backed up directly to sequential media (equivalent to TSM database backup today)

Backup types equivalent to those with proprietary database plus additionalBackup types equivalent to those with proprietary database, plus additional differential type– Full

I t l ( l ti h i l t f ll b k )– Incremental (non-cumulative changes since last full backup)

– Differential (cumulative changes since last full backup)

– Snapshotp



DB2 ScalabilityDB2 ScalabilityGoals

Improve ability to support concurrent operations Increase maximum number of objects that can be managed by a TSM serverSatisfy current TSM requirements for concurrency and management of large amounts of data, while positioning TSM to meet future requirementsReduce required number of TSM servers (total cost of ownership)

PlansExploit DB2 archive recovery logging

–Supports roll-forward recovery–256 GB maximum log size allows greater concurrency (13 GB maximum for proprietary database)

Maximum size for DB2 much higher than for proprietary database (practical size limitations to be determined)

TSM Server

Proprietary Database DB2


TSM ServerTSM Server


DB2 Performance and Resource UtilizationDB2 Performance and Resource UtilizationGoals

In first release with DB2, performance should be at least comparable to thatIn first release with DB2, performance should be at least comparable to that with the proprietary database for mainline TSM operations (backup, recovery, expiration)Utilization of resources (memory, CPU) should not increase significantly

PlansIteratively profile, analyze and optimize TSM performance using DB2Extensively restructure TSM table schema and application code to optimize for DB2

Incremental backup to TSM servers on open-system platforms now has better performance with pDB2 than proprietary database



DB2 Administration (UNIX Linux Windows)DB2 Administration (UNIX, Linux, Windows)Goals

Customers should not need to hire a DB2 DBA to administer TSM (total cost ofCustomers should not need to hire a DB2 DBA to administer TSM (total cost of ownership)DB2 should be hidden so TSM administrator does not need to manage DB2 directlyDatabase backup/recovery and other maintenance operations should be as easy as p y p ywith proprietary database

PlansPackage, install, and service DB2 embedded with TSMDrive DB2 management operations, including backup/recovery, from TSM administrative interfaceCommands for database operations will change, but should not be more complexExploit evolution of DB2 toward autonomic self-administration

DB2

TSM

DB2 embedded and managed via TSM

administrative interface


TSM ServerTSM

Administratoradministrative interface


DB2 Administration (z/OS)DB2 Administration (z/OS)Goals

Customers who do not already have a DB2 DBA should not need to hire one to d i i t TSM (t t l t f hi )administer TSM (total cost of ownership)

DB2 administration should fit a model that is familiar to most z/OS customers

PlansDesign TSM z/OS server to run as a DB2 application, rather than encapsulating with the TSM z/OS server

– Familiar model to most z/OS customers– Many z/OS customers already have a DB2 DBA who would manage DB2 separately from TSM

Drive DB2 management operations, including backup/recovery, from TSM administrative interfaceinterfaceCommands for database operations will change, but should not be more complexExploit evolution of DB2 toward autonomic self-administration

TSM runs as DB2 applicationTSM

DB2


TSM ServerTSM/DB2

Administrator

ppTSMAdministrator


Server Externals Changes (UNIX Linux Windows)Server Externals Changes (UNIX, Linux, Windows)Eliminated commandsDBBACKUPTRIGGER commandsDBVOL commands

New commandsDSMSERV DISPLAY DBSPACEEXTEND DBSPACE

Eliminated optionsBUFPOOLSIZEDBPAGE SHADOWDBVOL commands

DSMSERV DUMPDB/UNLOADDBESTIMATE DBREORGSTATSEXTEND/REDUCE DBQUERY DBVOL

EXTEND DBSPACEQUERY DBSPACEDSMSERV DISPLAY LOG

Changed commands

DBPAGE SHADOWDB PAGESHADOWFILELOGPOOLSIZELOGFULLWARNPERCENTMIRRORREADQUERY DBVOL

DEFINE/DELETE LOGVOLDSMSERV DISPLAY LOGVOLUMESDSMSERV EXTEND LOGEXTEND LOG

Changed commandsBACKUP DBDSMSERV RESTORE DBDEFINE/DELETE DBCOPYDSMSERV AUDITDB

MIRRORREADMIRRORWRITESELFTUNEBUFPOOLSIZE

EXTEND LOGQUERY LOGVOLREDUCE LOGSET LOGMODERESET BUFPOOL

DSMSERV AUDITDBQUERY DBDEFINE/DELETE LOGCOPYQUERY LOGVOLHISTORY commands

New optionsACTIVELOGSIZEACTIVELOGDIRECTORYMIRRORLOGDIRECTORYRESET BUFPOOL

RESET DBMAXUTILIZATIONRESET LOGCONSUMPTIONRESET LOGMAXUTILIZATIONCONVERT ARCHIVE

VOLHISTORY commandsSPACETRIGGER commandsDSMFMTDSMSERV FORMATQUERY OPTION

FAILARCHLOGDIRECTORYOVERFLOWLOGDIRECTORYDBMEMPERCENT

CONVERT ARCHIVEUNDO ARCHCONVERSIONUPDATE ARCHIVE

QUERY OPTIONSETOPTQUERY STATUSGRANT/REVOKE AUTHORITYQUERY ADMIN

Changes due toSelf administration by DB2


QUERY ADMIN Self-administration by DB2Different functionality of DB2Analyst privilege class eliminated


DB2 Reporting and MonitoringDB2 Reporting and MonitoringGoals

Preserve externalized TSM SQL schema so customer-developed queries and t ti ti t kautomation continue to work

Complete SQL functionImproved performance for standard, built-in TSM query functionsCompatibility with reporting tools based on ODBC or JDBC

PlansProvide DB2 views for virtual tables currently available in TSMExploit existing DB2 capabilities for SQL, ODBC, JDBCEncourage ISVs who offer TSM reporting/monitoring tools to integrate with TSMEncourage ISVs who offer TSM reporting/monitoring tools to integrate with TSM using DB2

TSMDB2


TSM ServerTSM

Administrator


Upgrade of Proprietary Database to DB2Upgrade of Proprietary Database to DB2Goals

Accurate upgrade Minimal administrator interventionMinimal TSM downtime

PlansDevelop automated, offline upgrade utility to extract records from TSM proprietary database and load into DB2 Perform extensive testing using error injection and as many customer databases as possibleOffer programs for early customer evaluation (technology preview, alpha, beta)

Proprietary Database DB2

Upgrade


TSM Server TSM Server


Database Upgrade UtilityDatabase Upgrade Utility

TSM server will be offline during the upgrade

Utility will extract records from v5.5 proprietary database and load into DB2 database

Source and target databases must be on the same platform, but not necessarily the same system

Data transfer can be either via network or using intermediate media

Where TSM table schema have changed for DB2, data will be loaded using the newWhere TSM table schema have changed for DB2, data will be loaded using the new schema

Utility will exploit pipelining and/or parallel processing to reduce upgrade timeUtility will exploit pipelining and/or parallel processing to reduce upgrade time



TopicsTopics



Administration

Security



Restartable Export/ImportRestartable Export/Import

Source Server Target ServerImported FilesRemaining Export Files

Source Server Target ServerI A B C D

E

IJ

H

J K

M N O F G

L

P

Export/import operation is interrupted (last checkpoint after file F exported)

Source Server Target Server Imported Files

G

Remaining Export FilesA B C DGHH I J

E

GH

HK L M F GN

O P

Export/import restarts from checkpoint (some files may be resent)


Reduced time to complete interrupted export/import Planned 5.5


Export Todate/Totime OptionsExport Todate/Totime Options

F d t /F ti T d t /T tiFromdate/Fromtime Todate/TotimeObjects stored prior to this

time are not exportedObjects stored after this

time are not exportedObjects stored in this time

window are exported

Date/time objects are stored on the TSM export server

Todate/Totime based on insertion date/time (when the object was stored on the TSM )server)

Todate/Totime, together with existing Fromdate/Fromtime will allow an administrator to specify a time window for object data that will be exported

Facilitates synching of servers– Facilitates synching of servers– Satisfy business requirements to capture and retain data stored within a specified time window


Improved control over what is exported Planned 5.5


Server-Server Storage Pool Volume TransferServer Server Storage Pool Volume TransferSource Server Target Server

DBDB DBServer-server export of metadata

S St Hi hTransfer storage pool volumes

Metadata transferred between servers using export/importR bl t l l h i ll d t th t t

Storage Hierarchy Storage Hierarchyg p

Removable storage pool volumes physically moved to the target serverEspecially attractive when used with shared librariesCould be used for– Splitting/balancing servers– Consolidating servers, especially after upgrade to DB2


Future CandidateReduced time and bandwidth consumption for export/import of object data


Expiration EnhancementsExpiration Enhancements

Expiration can be initiated for specified– Nodes

– Node group

Policy domain– Policy domain

– Data types (backup, archive)

Increased parallelism in expiration processing

Ad i i t t t l d f i ti iAdministrator can control resources used for expiration processing

Statistics reported as expiration processing completes for each nodep p p g p


Next Release Candidate


Simultaneous Migration and Storage Pool BackupSimultaneous Migration and Storage Pool BackupCopy pool 1

2. Storage

Cli t

pool backup

2 Storage

2. Migration1. Backup

Client 2. Storage pool backup

Server

Data flow

Copy pool 2

Can reduce client backup window as compared to simultaneous write to primary and copy storage pools during backup operation

Combines windows for migration and storage pool backupCombines windows for migration and storage pool backup

Periodic storage pool backup required to ensure all data copied


Future CandidateReduced total time for migration plus storage pool backup


Simultaneous Migration and Storage Pool BackupSimultaneous Migration and Storage Pool Backup

Use COPYSTGPOOL parameter to specify one or more copy pools for next t l i hi hstorage pool in hierarchy

When data is migrated to next pool, it is also be written to designated copy pools, if any

Simultaneous write to copy pools is incremental (data written only if not already in the copy pool)

If data cannot be stored in next pool it may be written to subordinate poolIf data cannot be stored in next pool, it may be written to subordinate pool which would inherit the copy pools specified for the next pool

May be limited to migration from sequential-access source pools

Not supported for other data movement operations such as reclamation or Move Data

Use COPYCONTINUE attribute to specify what to do if error is encounteredUse COPYCONTINUE attribute to specify what to do if error is encountered writing to copy pool during migration (continue or abort)



TopicsTopics



Administration

Security



Administration Center: Infrastructure ImprovementsAdministration Center: Infrastructure Improvements

Upgrade to new, lighter-weight version of Integrated Solutions Console (ISC) i f t t(ISC) infrastructure– Significant reduction in footprint– Faster installation

Improved response time to administrator actions– Reduced time for initial page load and page switchesp g p g– Enhanced processing of large queries– Progress indicators

U f AJAX t h lTSM administrator

– Use of AJAX technology

TSM servers


Improved administrative experience Next Release Candidate


Administration Center: Functional EnhancementsAdministration Center: Functional EnhancementsEnhanced navigation, especially for tasks involving client nodes

Use of saved, customized values for repetitive creation of multiple nodes

Basic and advanced dialogs for creation of maintenance plang p

For enterprise configuration, show relationships between managing and managed servers

Enhancements to health monitor and policy domain

Hyperlink to message help from messages displayed in Administration Center

Hover help to show default action for hyperlinks

Add support for backup set enhancements


Improved administrative experience Next Release Candidate


Forms for Node RegistrationForms for Node Registration

Allows values to be customized and

saved for repetitivesaved for repetitive registration of nodes



Reporting/Monitoring OverviewReporting/Monitoring OverviewImproved capability for out-of-the-box historical reporting and operational monitoring

G hi l t ti f t d d t t tGraphical representation of trends and current status

Ability to run on all open platforms supported by TSM server

Complement and integrate with the Administration Center (staged)

Sufficiently lightweight for managing one TSM serverSufficiently lightweight for managing one TSM server

Scalable to allow management of many TSM servers (potentially hundreds) within an enterprise

Allow administrators to define their own reports

Integration with reporting tools from other IBM productsIntegration with reporting tools from other IBM productsTSM

Administrator


Improved reporting and analysis Next Release Candidate


Reporting/Monitoring Architecture ISC (Web Browser)TEP

TEP Workspaces

TSM Admin Center/Health Monitor

TSM Monitoring Data

p

SAP Client ITM Agent(mySAP)

Optional SAP Client

TSM Server ITM Agent(TSM) ITMBIRT based reports

TSM TSM

DB/2

TDWReporting

Infrastructure

DB/2

HTMLTSM Activity Log

TSM Operational

Database TSM AttributeGroups

BIRTReporting

Engine HTMLPDFCSV

E-mailWeb URL

HTML

PDF

Distribution

BIRT Report Design Tool(Eclipse)

Primary TSM Server ITM & Reporting Server


Report Definitions


Predefined ReportsPredefined ReportsClient reports

Client job statusServer reports

Server job statusClient backup currencyStorage capacity protectedBackup details

Server throughputServer resource usage Database details

Top 10 backupsBackup missed filesBackup historyR t d t il

Disk usageTape usageOther storage usageT l l iRestore details

Top 10 restoresRestore historyArchive details

Tape volume usage analysisTape capacity analysisTape device errorsDevice usage historyArchive details

Top 10 archivesArchive historyRetrieve details

Device usage historyServer machine utilization

Retrieve detailsTop 10 retrievesRetrieve history



Operational MonitoringOperational Monitoring

Scheduled client events

Scheduled server events

Current client activity

Current server activityCurrent server activity

Current failures

TSM database status

Tape device status

Storage pool status

Client backup statusClient backup status



Client Backup CurrencyClient Backup Currency

Time since last successful backup, by

client node



Server ThroughputServer Throughput

Throughput of server operations by time



Server Resource UsageServer Resource Usage

Utilization of disk, tape and database

resources by time



Historical Audit Trail: Data ObjectsHistorical Audit Trail: Data Objects

Information regarding initial store of objectWhen object was storedWhen object was storedWho initiated store operationHow store was initiated (schedule, GUI)Initial storage pool / volumesTransport mechanism for store (LAN-free)Type of encryptionClient compressionInitial management class

TSM server

DBInitial management classObject size

Details of later operations on objectA Client restore/retrieve/recall attempts

Outcome of client access operationsDeletion (who/what initiated)Move/copy operationsMove/copy operations Management class rebindingStorage hierarchy


Future CandidateImproved tracking of historical information on data objects


Historical Audit Trail: Server ConfigurationHistorical Audit Trail: Server Configuration

Server configuration historyN / h d t t

DB

New/changed constructs- Policy definitions- Schedules- Storage pools/device classes

TSM server

DB Sto age poo s/de ce c asses- Nodes

Set commandsChanges to server optionsCh t l lChanges to server level


Future CandidateImproved tracking of historical information on server configuration


TopicsTopics



Administration

Security



Authentication via External User DirectoryAuthentication via External User Directory

TSM ServerTSM ClientSSL session

TSM Server

a. Legacy authentication using password in TSM

b. Authentication against password in

external directory

TSM Client

TSM Server/ Storage Agent

TSM Database(client passwords)

database

External Directory(client passwords)

g g

Admin Center

Option to store selected TSM user information, including passwords, in external directory (e g LDAP Tivoli Access Manager)directory (e.g., LDAP, Tivoli Access Manager)TSM would provide following authentication options

a. Legacy authentication using password stored in TSM databaseb Authentication against password stored in external directoryb. Authentication against password stored in external directory

Allows enforcement of strong password rules


Future CandidateConfiguration and enforcement of strong password rules


TSM Privilege Classes After DB2 ReleaseTSM Privilege Classes After DB2 Release

•System settingsEnterprise administration

Command-based administrative authorities

System •Enterprise administration•Administrators•All admin tasks

administrative authorities

Unrestricted Policy

Unrestricted Storage OperatorPolicy Storage

•Nodes•Policies•Schedules

•Storage devices•Storage pools

•Server operation•Removable media•Client sessions

Restricted Policy

•Selected domains

Restricted Storage

•Selected storage pools

NodeAll TSM administrators can perform server

Validated at sign-on


•Backup/archive via webAll TSM administrators can perform server

queries, even if not assigned a privilege class


Authorization via External EngineAuthorization via External Engine

Administrative Client TSM Server Admin command

E t lAuthorization can be performed using

L h i ( t d i il

TSM Database(privilege classes)

External Authorization

EngineResources

Legacy mechanism (granted privilege classes stored in TSM database)External authorization engine with preconfigured resources/permissions corresponding to privilege classes

If external authorization (e.g., Tivoli Access Manager) is used as TSM authorization engine

Permissions

p g p g

Engine would be preconfigured with– Resource definitions corresponding to current privilege classes (system, policy, storage, etc)– Each resource mapped to assigned permissions (operations allowed for that resource)

Administrator assigns authority to TSM administrators using authorization engine interfaceAdministrator assigns authority to TSM administrators using authorization engine interfaceTSM server checks external authorization engine for authorizationRestricted authority would require resource entries corresponding to domains/pools


Future CandidateAuthorization based on enterprise-wide policies and infrastructure


Authorization with Customer-Defined AuthoritiesAuthorization with Customer Defined Authorities

Administrative Client Admin command

TSM Server

Authorization can be performed usingLegacy mechanism (granted privilege

E t lclasses stored in TSM database)External authorization engine with preconfigured resources/permissions corresponding to privilege classesExternal authorization engine with

TSM Database(privilege classes)

External Authorization

EngineResources

External authorization engine with customer-defined authorities for each administrator Permissions

If external authorization engine is used with customer-defined authorities Engine would be preconfigured with resource and permission definitionsCustomer could alter, add, or remove resource/permission definitions to achieve desired authorizations for each administrator and command


Future CandidateFlexible, fine-grained assignment of authorities


SummarySummary

This presentation has described a sampling of potential future enhancements t Ti li St Mto Tivoli Storage Manager

Topics have includedTopics have included– Upgrade to DB2 for improved availability and scalability– Enhanced management of data in server storage– Improved ease of administration– Enhanced security options for authentication and authorization


Documents

Oxford University TSM Symposia - Preview of Future ...tsm-symposium.oucs.ox.ac.uk/2007/papers/Dave Cannon...Preserve externalized TSM SQL schema so customer-developed queries and autti