Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
IBM z Analytics
IBM z Analytics
ANY Data for ANY ApplicationExploring IBM Data Virtualization Manager for z/OS
in the era of API Economy
Francesco Borrello | [email protected]
IBM z Analytics Data Virtualization Manager
Traditional Data Integration Inadequate
No longer Viable to Move Data(using extract, transform, load ETL)
• Risk to data security
• Data inconsistency
• Rigid, limits business agility
• High cost and latency
IBM z Analytics Data Virtualization Manager
A New Approach is Needed – Data Virtualization
a data services layer that integrates data and content on demand from disparate sources in real time
provides a layer of abstraction that shields developers from unique data implementation
provides a metadata catalog to keeps track of data, location, availability, and state
supports transactions that write back to the original data sources regardless of location
doesn’t mandate a API or schema - supports “many to many”
Unlike traditional data integration�
data virtualization creates virtualized and integrated views of disparate
data in-memory for immediate read/write access rather than waiting
on moving the data and holding the integrated views in an EDW
IBM z Analytics Data Virtualization Manager
What is expected of Data Virtualization?
IBM z Analytics Data Virtualization Manager
What is expected of Data Virtualization?
Provide ANY data for ANY application
Cloud Mobile
Analytics/Search ESB, ETL
Transactional DataData Virtualization
IBM z Analytics
Data Virtualization on IBM Z
IBM z Analytics Data Virtualization Manager
� accelerate mainframe modernization initiatives involving Big Data
� gain real-time business insights across z/OS and enterprise data
� eliminate coding of complex z/OS apps via built-in APIs and
interfaces
• ensures data is secure and in-place with real-time data virtualization
• supports Hybrid architectures (on-premises, Cloud, Hadoop and MF)
• broad API support via SQL, NoSQL, SOAP, and REST via z/OS Connect EE
• z/OS resident optimization for improved performance and TCO
• abstraction layer for improved productivity and business agility
IBM Data Virtualization Manager for z/OS
The industry’s only Z-resident data
virtualization solution!
Virtualize z/OS data with other enterprise data sources in
real-time without data movement to provide
comprehensive information that is readily consumable by
analytics, cloud and cognitive applications
Take action now!
IBM z Analytics Data Virtualization Manager
IBM Data Virtualization Manager for z/OS
IBM z Analytics Data Virtualization Manager
Runs Without A Processing Penalty
WLM
Enclave
WLM
Enclave
Network I/O
Hybrid SRB/TCB
SRB/TCB
Mode Switch
API
WLM
Enclave
Network I/O
Hybrid SRB/TCB
SRB/TCB
Mode Switch
API
WLM
Enclave
WLM
Enclave
Data I/O
Hybrid SRB/TCB
SRB/TCB
Mode Switch
API
WLM
Enclave
Data I/O
Hybrid SRB/TCB
SRB/TCB
Mode Switch
API
zIIP
Ability to route up to
99% of data
processing time to
zIIP
Ability to route up to
99% of data
processing time to
zIIP
Ability to integrate
mainframe data in
place with
transformation
running on zIIP
Ability to integrate
mainframe data in
place with
transformation
running on zIIP
IBM z Analytics Data Virtualization Manager
IBM DVM Address Space
Pro
vid
er I/O
Hyb
rid
SR
B/T
CB
Pro
vid
er I/O
Hyb
rid
SR
B/T
CB
Pro
vid
er I/O
Hyb
rid
SR
B/T
CB
Co
ns
um
er I/O
Hyb
rid
SR
B/T
CB
Co
ns
um
er I/O
Hyb
rid
SR
B/T
CB
Co
ns
um
er I/O
Hyb
rid
SR
B/T
CB
zIIP
Map/Reduce Implemented on z/OS
Data Consumer
SQL
ODBC/JDBC
NoSQL
(JSON)
Web
Services
(SOAP/REST)
Data Provider
Db2, ADABAS, IMS
IDMS, VSAM
Physical
Sequential, SMF,
Sys Logs , MQ
Db2 LUW, SQL
Server, Oracle
Write to ClientWrite to Client
Request for DataRequest for Data
MapReduce LDU
MapReduce LDU
MapReduce LDU
MapReduce LDU
MapReduce LDU
MapReduce LDUBuffer
IBM z Analytics
Adabas Db2 for z/OS
IMS DB IDMS
SEQ files VSAM
SMF records Log Streams
Virtual Tape zFS
Db2 WarehouseDb2 Warehouse
on Cloud
Db2 Informix
MAINFRAME
z/OS Systems
Distributed
More than just data access )
JDBC/ODBC
REST AP/SOAPI
NoSQL
Access to all data from all applications
without
need to know which format the data it is in
using the API of your choice
IBM Data Virtualization
Manager for z/OS
IBM z/OS Connect EEIBM z/OS Connect EE
IBM z Analytics
Selective direct Distributed Relational Database Architecture support
IBM Data Virtualization
Manager for z/OS
Oracle
Data
Provider for DRDA
(Linux)
Oracle
SQL
ServerMicrosoft
Service for DRDA
Apache
Derby
DRDA
IBM z Analytics Data Virtualization Manager
Enabling APIs for Data on IBM Z
CICS
IMS
WAS
MQ
DB2
� Agile web, mobile, cloud access via REST to
mainframe applications such as CICS, IMS, WAS*
CICS
IMS
WAS
MQ
DB2
z/OS Connect
Enterprise Edition
Trusted zone
Load b
ala
ncer
Inte
lligent
Conte
nt
routing
Gate
way /
Data
Pow
er
Security
layer
DMZ Inner APIsOuter APIs
Discovery Services
REST Services
REST API
Consumers
IBM z Analytics Data Virtualization Manager
Enabling APIs for Data on IBM Z
CICS
IMS
WAS
MQ
DB2
Data
Virtualization
Manager for z/OS
Adabas
IMS DB
IDMS
SMF
VSAM
� Agile web, mobile, cloud access via REST to
mainframe data such as Adabas, IMS, VSAM*
CICS
IMS
WAS
MQ
DB2
z/OS Connect
Enterprise Edition
Trusted zone
Load b
ala
ncer
Inte
lligent
Conte
nt
routing
Gate
way /
Data
Pow
er
Security
layer
DMZ Inner APIsOuter APIs
Discovery Services
REST Services
REST API
Consumers
DVM Non z/OS
IBM z Analytics
Automated generation of code to access data from Spark &
Python
� Reduce coding of data access via automated generation of Scala code
(Python in progress)
� Automatically optimized based on available parallelism to generate
best performance results without manual coding
IBM z Analytics
16