System z features-custom

Mainframe Fundamentals

© 2009 IBM Corporation1

System z Features

Run fastRun a lotGrow without bounds

Performance & Scalability

Nancy SteinIBM Advanced Technical Support - [email protected]

2


© 2009 IBM Corporation

Performance and Scalability

• Performance• Do some one thing really fast• i.e. Rapid Response Times

• Throughput• Not just one thing…• but lots and lots of things…• all at the same time…• and all of ‘em fast!• i.e. High Transaction Rates

• Scalability• Grow and grow

• With acceptable performance and through-put

Seconds Per Transaction

Transactions Per Second

Meet Goals (Seconds Per Transaction)

At Peak Volumes(Transactions Per Second)

3



Hardware Platform

Operating System

Shared Application Model … Remember This?

DiskLAN

I/OAdapter

NetworkAdapter

RealMemory

Processor

API

Container

Application

FileSystemTCP/IP

VirtualMemory

ThreadAPI

Container

Application

FileSystemTCP/IP

VirtualMemory

Thread

API

Container

Application

FileSystemTCP/IP

VirtualMemory

Thread API

Container

Application

FileSystem

TCP/IP

VirtualMemoryThread API

Container

Application

FileSystem

TCP/IP


Container

Application

FileSystem

TCP/IP


Container

Application

FileSystem

TCP/IP

VirtualMemoryThread

API

Container

Application

FileSystemTCP/IP

VirtualMemory

ThreadAPI

Container

Application

FileSystemTCP/IP

VirtualMemory

ThreadAPI

Container

Application

FileSystemTCP/IP

VirtualMemory

Thread

1000s of running applicationsMany different containers

Various priorities and goalsVarious inter-dependencies

Lots and Lots of Files and Disks and DataMany different networks (LAN segments)

4



HardwarePlatform

OperatingSystem

Mainframe Platform Performance – Scaling Resources

LAN

I/OAdapter

NetworkAdapter

RealMemory

Processor

API

Container

Application

FileSystemTCP/IP

VirtualMemory

ThreadAPI

Container

Application

FileSystemTCP/IP

VirtualMemory

Thread

API

Container

Application

FileSystemTCP/IP

VirtualMemory

Thread API

Container

Application

FileSystem

TCP/IP


Container

Application

FileSystem

TCP/IP


Container

Application

FileSystem

TCP/IP


Container

Application

FileSystem

TCP/IP

VirtualMemoryThread

API

Container

Application

FileSystemTCP/IP

VirtualMemory

ThreadAPI

Container

Application

FileSystemTCP/IP

VirtualMemory

ThreadAPI

Container

Application

FileSystemTCP/IP

VirtualMemory

Thread

LAN

NetworkAdapter

Disk

I/OAdapter

Processor

Disk

I/OAdapter

LAN

NetworkAdapter

Processor

1000’s of Terabytes of Storage!!!Dozens of GbE LAN connections!!!…

Dozens of processors!!!... 100’s of GBs of Memory!!!...

100’s of Fiber Connections…

5



What does a mainframe look like?

HybridCooling

Processor Books(CPs and Memory)

CEC* Cage

STI cables

SupportElements

3 x I/Ocages

PowerSupplies

InternalBatteries

Front View

Distances are critical when

clock rates and signaling rates approach multi-

GHz ranges

Technology “packaging” has never been more

important!For performanceAnd much more..

Speedof

Light!

Memory-Time

I/O-Time CPU-Time

z9 Technical Guide - Redbook

z10 Technical Guide - Redbook

6



zSeries BookAdvanced Packaging Technology

��

��

��

� � � � � � � � � � � � � � � � � � � � � � � � � � � �

Up to 4 Books in a Server

MCMMulti-Chip

Module

STISelf-Timed Interfaces

(I/O Buses)

Memory-Time

I/O-Time CPU-Time

7



172.8 GB/sec

IBM System z – Balanced System Design

Memory

System I/O Bandwidth

Processors

ITR for 1-way

288 GB/sec*

1.5 TB**

64-way

~920

172.8 GB/sec*

~600512 GB

54-way

96 GB/sec

450256 GB

32-way

24 GB/sec

30064 GB

16-way

z10 EC (2008)

z9 EC (2005)

zSeries 990 (2003)

zSeries 900 (2000)

Balanced SystemCPU, nWay, Memory,

I/O Bandwidth*

*Servers exploit a subset of its designed I/O capability** Up to 1 TB per LPAR

8



z10 Multi-Chip Module� 96mm x 96mm MCM

103 Glass Ceramic layers7 chip sites7356 LGA connections17 and 20 way MCMs

�CMOS 11s chip Technology PU, SC, S chips, 65 nm5 PU chips/MCM – Each up to 4 cores

One memory control (MC) per PU chip21.97 mm x 21.17 mm994 million transistors/PU chipL1 cache/PU core

64 KB I-cache128 KB D-cache

L1.5 cache/PU core3 MB

4.4 GHz0.23 ns Cycle Time6 km of wire

2 Storage Control (SC) chip21.11 mm x 21.71 mm1.6 billion transistors/chipL2 Cache 24 MB per SC chip (48 MB/Book)L2 access to/from other MCMs3 km of wire

4 SEEPROM (S) chips2 x active and 2 x redundantProduct data for MCM, chips and other

engineering informationClock Functions – distributed across PU and SC

chipsMaster Time-of-Day (TOD) and 9037 (ETR)

functions are on the SC

PU 0PU 2

PU 4 PU 3

SC 0SC 1

PU 1

S 0

S 1

S 2

S 3

Memory-Time

I/O-Time CPU-Time

9



z10 Quad Core� Up to Four cores per PU

4..4 GHz L1 cache/PU core

64 KB I-cache128 KB D-cache

3MB L1.5 cache/PU coreEach core with its own Hardware Decimal Floating

Point Unit (HDFU)� Two Co-processors (COP)

Accelerator engines Data compression Cryptographic functions

Includes 16KB cacheShared by two cores

� L2 Cache interfaceShared by all four coresEven/odd line (256B) split

� I/O Bus Controller (GX)Interface to Host Channel Adapter (HCA)Compatible with System z9 MBA

� Memory Controller (MC)Interface to controller on memory DIMMs

MC

CoreL1 + L1.5

&HDFU

COP

COP

L2 Intf GXL2 Intf

CoreL1 + L1.5

&HDFU

CoreL1 + L1.5

&HDFU

CoreL1 + L1.5

&HDFU

CPU-Time

10



Processing Unit Characterizations

System Assist Processor (SAP)

Integrated Coupling Facility (ICF)

zSeries Application Assist Processor (zAAP)

SPARE

zSeries Information Integration Processor (zIIP)

Integrated Facility for Linux (IFL)

SPARE

SPARE

SPARE

SPARE

Central Processor (CP)

SPARE

Every microprocessor

on an MCM can take on

one of several“personalities”

SPARE is an acronym

for …

1994

2004

2006z990+

z9+

z900+

~1996

2001

I/O-Time CPU-Time

z10

11


© 2009 IBM CorporationMore on Specialty Engines

Offload Processors and Accelerators

System Assist Processor (SAP)

Integrated Coupling Facility (ICF)

zSeries Application Assist Processor (zAAP)

zSeries Information Integration Processor (zIIP)


Central Processor (CP)

HardwareAssisted

Data Compression

CP Assist ForCryptographic

SupportInstructions

Crypto Accelerators

CryptoExpressPCI-Card



Memory-Time

I/O-Time CPU-Time

The 1st mainframe was built as a multi-processor (it had an I/O offload engine) and every machine

generation since has improved on those multi-processing

capabilities!

12



Throughput - Overlapping and Context Switching

CPU-Time Memory-Time I/O-Time Ready & Waiting Time

CPU-Time Memory-Time I/O-Time

High Priority Workload

Low Priority Workload

Application 1

Context Switching on Mainframe5 to 7 X more efficient then distributed servers

Sw

itchApplication 2 Application 1

Sw

itch

Sw

itch

In a system that context switches very well…Where a lot of I/O is going on (asynchronously)…

A great deal of overlapped work can be done…Driving system efficiency (i.e. utilization) very high!!!!!

Memory-Time

13



z10 EC HiperDispatch for z/OS

� HiperDispatch – z10 unique functionDispatcher Affinity (DA) - New z/OS DispatcherVertical CPU Management (VCM) - New PR/SM Support

� Mitigate impact of scaling differences between processor and memoryAccess to memory and remote caches not scaling with processor speedIncreased performance sensitivity to cache misses in multi-processor system

� Optimize performance by redispatching units of work to same processor groupKeep processes running near their cached instructions and dataMinimize transfers of data ownership among processors / books

� Tight collaboration across entire z10 hardware/firmware/OS stackConcentrate logical processors around shared L2 cachesCommunicate effective cache topology for partition to OSDynamically optimize allocation of logical processors and units of work

Memory-Time

I/O-Time CPU-Time

14



STIz990/z890

2003

STIz9

2005

InfiniBand I/O Busz10 2008

STIz900/z800

200x

6 GBps

2.7 GBps

2 GBps

1 GBps

z10 Upgraded I/O Subsystem – Infiniband

z10 Infiniband6GBpsec

16 per book

6 GBps X16 IBs X

4 Books +384GBps

“Backside”Bandwidth

Host bus interconnect speeds in GBps

I/O-Time

15



When a Single System is NOT Big Enough…Scaleability with The Parallel Sysplex Cluster

z/OS Operating System

DB2

TCP/IP

DiskLAN

I/OAdapter

NetworkAdapter

VirtualMemoryThread

RealMemory

Processor

WAS

(JVM)

FileSystem

VirtualMemoryThread IMS

FileSystemTCP/IP

VirtualMemoryThread

COBOLTransactionJSP/EJB

TCP/IP

SQLFile

System

Processor Processor

Disk

I/OAdapter

Disk

I/OAdapter

LAN

NetworkAdapter


DB2

TCP/IP

DiskLAN

I/OAdapter

NetworkAdapter

VirtualMemoryThread

RealMemory

Processor

WAS

(JVM)

FileSystem


FileSystemTCP/IP

VirtualMemoryThread


TCP/IP

SQLFile

System

Processor Processor

Disk

I/OAdapter

Disk

I/OAdapter

LAN

NetworkAdapter

� Scale capacity “horizontally”By adding servers (1 to 32 total servers in a sysplex cluster)

� Workload is balanced across the systems in the clusterWorkload “affinities” hinder clustering flexibility (limit routing)

Affinity to “data” is the most common and most limiting affinity

� Sysplex Cluster – UniqueThe Only “Shared Everything” cluster in the enterprise server market

Minimize affinities – maximize flexibility!

16



Scaleability with The Parallel Sysplex Cluster


DB2

TCP/IP

Disk

I/OAdapter

NetworkAdapter

VirtualMemoryThread

RealMemory

Processor

WAS

(JVM)

FileSystem


FileSystemTCP/IP

VirtualMemoryThread


TCP/IP

SQLFile

System

Processor Processor

Disk

I/OAdapter

Disk

I/OAdapter

LAN

NetworkAdapter


DB2

TCP/IP

Disk

LAN

I/OAdapter

NetworkAdapter

VirtualMemoryThread

RealMemory

Processor

WAS

(JVM)

FileSystem


FileSystemTCP/IP

VirtualMemoryThread


TCP/IP

SQLFile

System

Processor Processor

Disk

I/OAdapter

Disk

I/OAdapter

LAN

NetworkAdapter

DataSwitch

TheNetwork

TheNetwork

SharedNetwork

Sysplex Cluster Coupling Facility

SharedLocks

SharedLists

SharedBuffers

CouplingLinks

Cluster Programming

API

SharedApps

Single-System“Image”

Management Scope

Up to 32 System

Other Clustering Topologies

SharedData

17



0

500

1000

1500

2000

2500

3000

3500

4000

1997G4

1998G5

1999G6

2000z900

2003z990

2005z9 EC

2008z10 BC and EC

MH

z

300MHz

420 MHz

550 MHz

770 MHz

1.2 GHz

1.7 GHz

� G4 – 1st full-custom CMOS S/390®

� G5 – IEEE-standard BFP; branch target prediction� G6 – Copper Technology (Cu BEOL)

� z900 – Full 64-bit z/Architecture®

� z990 – Superscalar CISC pipeline� z9 EC – System level scaling

4.4 GHz

� z10 EC – Architectural extensions

IBM z10 EC Continues the CMOS Mainframe Evolution

3.5 GHz

18



QUESTIONS ???

19



Internal Coupling Facility (ICF)

1997


2001

IBM System z9 Integrated Information Processor (IBM zIIP)

2006IBM System z Application Assist Processor (zAAP)

2004

Building on a strong track record of technology innovation with specialty engines, IBM introduces the System z9 Integrated Information Processor

�Support for Linux workloads and open standards

�Designed to help improve resource optimization for eligible data (DB2) workloads within the enterprise�Centralized data

sharing across mainframes

�Designed to help improve resource optimization for z/OS Java technology-based workloads

Specialty Engines – An Evolution

Return

20



Specialty Engines - Defined

� Processing Unit Characterization– i.e. not “special” hardware

– But slightly customized microcode load

� Are Specialty Engines New?– Not really

– SAPS were 1st gen CMOS - aka 1994

– ICFs followed, then IFLs, zAAPs, and zIIPs

� What is the Role of Specialty Engines?– Value – aka Reduce Total Cost of Ownership (TCO)

– Promote “strategic” mainframe growth areas

Return

21



Specialty Engines – Value Proposition

� Capacity at a Fraction of the Normal Hardware Cost…– And to the extent that constraints may exist specialty engines can

improve performance as well!

– zAAPs – capacity for Java-based processing

– zIIPs – for a variety of information-based processing ops.

– IFLs – capacity for Linux-based processing� Without “Software” Loading…

– zAAPs and zIIPs – (to date) have no effect on software billing from IBM – nor from any ISV (to date)

– i..e zAAPs and zIIPs have no associated MSUs or VUs– IFLs – the bulk of the Linux software provider community (aka

IBM, Oracle, etc.) treats IFLs just like any other “core” (or engine)� With Investment Protection…

– “Upgraded” specialty engines have no cost!!!

Return

22



40% utilization

Consider a WebSphere Application that is transactional in nature and requires 1000 MIPS today on zSeries.

In this example, with zAAP, we can reduce the standard CP capacity requirement for the Application to 500 MIPS or a 50% reduction.* * For illustrative purposes only

1000 MIPS for WebSphere App

500 MIPS for WebSphere App +500 MIPS now available for additional workloads

JAVA execution powered by a zAAP

JAVA

JAVA

JAVA

JAVA

JAVA

JAVA

80% utilization

zAAPs Explained…

Return

z/OS JVMDispatcherInterface

23



zIIPs Explained…� z/OS manages and directs work between the general

purpose processor and the zIIPNo changes anticipated to DB2 UDB for z/OS V8 apps

� DB2 UDB for z/OS V8 will be first IBM exploiter of the zIIP:System z9 and z/OS 1.6 or later DB2 UDB for z/OS V8

� Portions of the following DB2 UDB for z/OS V8 workloads may benefit from zIIPs:

ERP, CRM, Business Intelligence and other enterprise applications – via DRDA over a TCP/IP connection

Data warehousing applications** – requests that utilize star schema and/or parallel queries

DB2 UDB for z/OS V8 utilities used to maintain index maintenance structures

And of late…� DB2 Remote Native SQL Stored Procedures (in DB2 9)� IPSEC offload processing (in z/OS 1.9)� z/OS System XML Parsing Services (in z/OS 1.9)

Announcing Availability

Return

Enclave SRBz/OS Dispatcher

Proprietary Interface

24



Differences between zAAP and zIIP

Exploiters include:� ANYTHING that uses Java via the IBM SDK (IBM

Java Virtual Machine (JVM)) such as:– WebSphere Application Server– IMS™

– DB2– CICS®

– Java batch� z/OS XML System Services

– DB2 9 (New Function Mode)– Enterprise COBOL V4.1– IBM XML Toolkit for z/OS, V1.9

Underlying technology is z/OS ability to re-direct portions of TCB mode work to the zAAP

Intended to help implement new application technologies on System z, such as Java and XML

System z Application Assist Processor (originally the zSeries Application Assist Processor).Available on System z10 EC and z10 BC™, z9 EC and z9 BC and IBM eServer zSeries 990 and 890 (z990, z890)

Introduced in 2004

zAAP

Exploiters include:� DB2 V8 for z/OS, DB2 9 for z/OS

– Data serving– Data Warehousing

� z/OS Communications Server– Network encryption

� z/OS XML System Services – DB2 9 New Function Mode

� z/OS Global Mirror (XRC), System Data Mover (SDM)� IBM GBS Scalable Architecture for Financial Reporting

Underlying technology is z/OS ability to re-direct portions of enclave SRB work to the zIIP

Intended to help integrate data and transaction processing across the enterprise and on to System z9 and System z10

System z9 Integrated Information Processor and System z10 Integrated Information ProcessorAvailable on IBM System z9 EC and z9 BC, and IBM System z10 EC and z10 BC

Introduced in 2006

zIIP

Return

25



� Java eligible for zAAP – lowering the cost of computing for WebSphere® Application Server and other Java technology-based applications

� Centralized data serving eligible for zIIP - workloads such as BI, ERP, and CRM applications running on distributed servers with remote connectivity to DB2 V8 (with z/OS 1.6)

� Network encryption on zIIP – zIIP becomes an IPSec encryption engine helpful in creating highly secure connections in an enterprise (with z/OS V1.8)

� z/OS XML System Services eligible for zAAP and zIIP – helps make hosting XML data and transactions on System z more attractive. DB2 9, Enterprise Cobol V4.1, and XML Toolkit for z/OS V1.9 are first IBM exploiters (introduced with z/OS V1.9 and rolled back to V1.8 and V1.7)

� Remote mirror on zIIP – zIIP assisted z/OS Global Mirror function (zGM, formerly XRC) Most of the System Data Mover (SDM) processing eligible for zIIP. Helps reduce server utilization at recovery site (z/OS V1.10, z/OS V1.9 with PTF UA39510, z/OS V1.8 with PTF UA3950)

� HiperSockets – z/OS Communications Server allows the HiperSockets Multiple Write operation for outbound large messages (originating from z/OS) to be performed by a zIIP. Application workloads based on XML, HTTP, SOAP, Java, etc as well as traditional file transfer, can benefit.

� Business Intelligence – IBM Scalable Architecture for Financial Reporting provides a high-volume, high performance reporting solution by running many diverse queries in z/OS batch– can be eligible for zIIP.

Specialty Engines, zAAPs and zIIPs –Designed to help implement, integrate, optimize new technologies

Return

26



Clustering Technology Topologies

Active

Data

Passive

Failover

� Active / Passive� Microsoft Cluster � HACMP � Veritas

High Availability& Capacity

� Active / Active� Shared Data� Oracle RAC

Active Active

Data

Replication

� Active / Active� Replicated data

and/or code� WebSphere� xkoto

Active Active

Data Data

Sprayer

Comprehensive

� Active / Active� Shared Data� Shared

Applications� Shared Mgt� Extensive

exploitation� Parallel Sysplex

Active Active

Data

Workload Mgt

Return

27



IMSPlex – Data Sharing / Shared Queues

z/OSPR/SM

IMS-ABufferPoolsLocks

IMS-ALog

CouplingFacility

IMSRECONS

IMS-BLog

IMSDBs

IMSPROCLIB

z/OSPR/SM

IMS-BBufferPoolsLocks

Tape

Rollingz/OSMigrations

ConcurrentMicrocodeUpdates

ConcurrentSysplex NodeAdd/Remove

ConcurrentCoupling FacilityUpdates

RollingIMS ReleaseMigration andMaintenance

ConcurrentCF Rebuilds

ImmediateFailover & Restart

RecoveryControl :LOGs/DBs

Online/OfflineUtilities Configuration

Parameters

Duplexed LogsAnd Archives

DuplexedCF &

Timers

Return

Duplexed LogsAnd Archives

28



People Costs to Manage Databases

� Distributed Database ModelMultiple O/S images

Multiple instances of DBMS

Replicated IP and SAN connections

Multiple H/A clusters

Multiple DR functions

Duplicate copies of data

� Mainframe Database ModelShared O/S mage

Single instance of DBMS

Shared IP and SAN connections

Shared H/A cluster

Shared DR capability

Shared data

Lock Management

Data

z/OS

DB Mgr

DB DB DB DB DB

Data

O/S

DB Mgr

DB

Data

O/S

DB Mgr

DB

Data

O/S

DB Mgr

DB

Return

Documents

System z features-custom