Better IT Operations and Security through Enhanced z/OS Analytics: New Features for Syncsort Ironstream

Syncsort Mainframe Customer Education Webinar

Syncsort Ironstream® Version 1.4 New Features For Enhanced z/OS Analytics – Part 2

4Q 2017

Today’s Presenters

Ed Wrazen Director, Mainframe Product Management is responsible for the product strategy & roadmap for Syncsort’s Mainframe products and solutions. With a career in Enterprise IT spanning 35 years, Ed has held roles in software development, database administration, product management, consulting and marketing in global businesses and enterprise technology companies. Ed has experience in Enterprise systems architectures, performance management, database and data management technologies and is a regular speaker at industry events worldwide.

2Syncsort Confidential and Proprietary - do not copy or distribute

Ed Hallock is a highly experienced Information Technology Professional with a broad experience base in software product development, support, product management, marketing, and business development. In his diverse career Ed has benefited from working for some of the largest independent software vendors, in a variety of roles, providing enterprise solutions to Global 1000 corporations. Ed has extensive experience in performance and availability management for systems and applications. He holds a bachelor’s degree in Computer Science from Montclair State University in Upper Montclair, New Jersey and has presented at numerous industry events as well as corporate related conferences and seminars.

Agenda

Introduction to Ironstream®

Recent New Features:– SMF Versioning

– Enclave support for RMF III data

– Multi-Send API

– Transaction Tracing

– Advanced Filtering for SMF data

– Data Loss Protection

Splunk: The Industry-Leading Platform For Machine Data

Syncsort Confidential and Proprietary - do not copy or distribute

Machine Data: Any Location, Type, Volume

Online

ServicesWeb

Services

ServersSecurity

GPS

Location

StorageDesktops

Networks

Packaged

Applications

Custom

AppsMessaging

TelecomsOnline

Shopping

Cart

Web

Clickstreams

Databases

Energy

Meters

Call Detail

Records

Smartphones

and Devices

RFID

On-

Premises

Private

Cloud

Public

Cloud

Platform Support (Apps / API / SDKs)

Enterprise Scalability

Universal Indexing

Answer Any Question

DeveloperPlatform

Report &analyze

Custom dashboards

Monitor & alert

Ad hoc search

Mainframe

4

Critical Mainframe Data Normalized and Streamed to Splunk with Ironstream®

Log4jFile

Load

SYSLOGSYSLOGD

logs

security

SMF

50+

types

RMF

Up to 50,000

values

DB2SYSOUT

Live/Stored

SPOOL Data

Alerts

Network

Components

Ironstream

API

Application Data

AssemblerC

COBOL

REXX

USS

Value of an End-to-End View, Inclusive of Mainframe

Extend What Splunk Does Already, to include critical z/OS systems:– 360ᵒ Degree View: Make the Splunk View of the Enterprise Complete via

Including Mainframe Data

– Same Splunk Dashboards, Bigger, More Complete Data Sets; Free Ironstream Splunk Apps and Modules

Security and Compliance/SIEM- Ensure Audits Passed

IT Operational Analytics/ITOA-Ensure Ops SLAs Met

IT Service Intelligence/ITSI-Ensure Services Health

Polling Question #1

What analytics platforms are you using today for z/OS IT operational intelligence:

Splunk

Hadoop

ELK (Elastic Stack)

Spark

Custom/Home Grown solution

None

7


SMF Versioning

New in 4Q 2017

z/OS SMF Changes

SMF record structures can and do change from one release of z/OS to another

Significant No. of SMF changes with z/OS 2.3 (avail Sep 29, 2017)– Up to 2048 record types supported

– Up to 65k subtypes per record type

– DSECT changes - Fields Added or Moved

– Extended Headers now available

Changes with subsystems– Different versions of CICS or Db2

– Different versions running on same LPAR


SMF Processing requirement for different z/OS levels

Technologies that process SMF data from z/OS need to be mindful of changes between releases

An SMF Processing Solution compiled on one z/OS may need to process SMF data from another z/OS

Ability to process SMF data on a z/OS level that did not generate the original data

Ability to detect subsystem version that generated SMF

– CICS 4.2, CICS 5.1, CICS 5.2 CICS 5.3, CICS 5.4

– Db2 v10, Db2 v11, Db2 v12


New Ironstream Support for SMF Versioning

Dynamically detects the z/OS level

Determines the structure of z/OS SMF records

Determines the subsystem SMF record structure

Supports multiple versions of z/OS SMF data in a single instance

Ironstream now includes a framework for supporting multiple versions of z/OS SMF data

• Positioned for future z/OS releases

• Correctly formats the SMF record for Splunk

• Multiple versions/formats supported



Enclave support for RMF III data

New in 4Q 2017

What are Enclave Attributes?

An enclave is a transaction that can span multiple dispatchable units (SRBs and tasks) in one or more address spaces and is reported on and managed as a unit.

The enclave is managed separately from the address spaces it runs in.

A program can create an enclave, schedule SRBs into it, or join tasks to it.

A multisystem work manager can process a transaction on multiple systems by using a multisystem enclave.

An enclave represents a “business unit of work” – Used by a variety of workloads – Db2, Db2 DDF, Websphere, MQ, LDAP, TCP/IP

– Enclave created and used for work across multiple address spaces and systems


Enclave support for RMF III data

Enclave is used to keep track of address space independent transactions

– Can consist of multiple TCBs or SRBs executing across multiple address spaces

Enclave used by Db2

– Distributed Data Facility (DDF)

– Db2 Stored Procedures

– Db2 Sysplex query parallelism

– Db2 sequential prefetch

Other users

– MQSeries

– Websphere

– TCP/IP

– LDAP


Displaying Enclave activity using RMF

Configuring Ironstream to collect Enclave Attributes

1. Activate IronstreamDesktop Browser

2. On Admin menu, select Ironstream instance

3. Select RMF Filters– Automatically built using RMF

DDS

4. Select Enclave attributes from MVS Image Attributes


Ironstream API EnhancementMulti-Send API

New in 4Q 2017!

Ironstream API Enhancements

The Ironstream API enables Ironstream instances to programmatically collect user-defined EBCDIC or ASCII data and forward it on to Splunk.

The API includes a configurable IRONSTREAM_API data source and a SSDFAPI routine, which is a standalone load module that can be called or link edited statically into the calling program

New capability allows an API instance to be INITiated and TERMinated

– Enables multiple records to be sent while the connection is active


Advantages of the Multi-Send API

Good for multiple, sequential send requests

– E.g. Processing file input and forwarding to Splunk

RACF check performed only on the initiation of the connection

Connection persists over multiple send requests

Eliminates the need for the API to allocate and release storage for each operation

Performance Improvement over Single-Send API


Using the Multi-Send API


CALL SSDFPAPI,(NUMPARM,REQUEST,CLASS,TYPE,SUBTYPE,TOKEN,RETCODE, RSNCODE)

|||

PROCESS DATA 1-NCALL SSDFPAPI,(NUMPARM, REQUEST, TOKEN, DATA, LENGTH, RETCODE, RSNCODE)

|END DATA |

||

CALL SSDFPAPI,(NUMPARM,REQUEST,TOKEN,RETCODE, RSNCODE)

Requesting Program Ironstream

INIT Request

SEND Request…1

SEND Request…N

TERM Request

▪ Validate Request▪ Perform RACF Check▪ Build Control Block Chain▪ Send a valid Token to Requestor

▪ Validate request▪ Process record for each SEND

request

▪ Validate request▪ Release Storage

1. Request: Request type can be “INIT”, “SEND” or “TERM”

2. Class: Identifies the Ironstream instance running with the same class

3. Type: Identifies the Ironstreaminstance running with same class and type combination.

4. Subtype: Identifies the Ironstream instance running with Same class, type and subtype combination

5. Token: Used by API 6. Data: Data Address7. Length: Data Length

API Parameter Options


Transaction Tracing

Announced in 3Q 2017

What is Transaction Tracing

• Enables organizations to get deep insight into web-based and mobile transactions’ impact on the mainframe.

• Unprecedented granularity that enables you to monitor and improve application performance.

• Lightweight solution with a minimal footprint


• Leverages Syncsort Ironstream® to

deliver IBM z/OS machine data in

real-time to leading platforms like

Splunk® for operational analytics.

How Does It Work?

• Uses a transaction identifier to correlate transaction workloads through CICS and Db2 on z/OS.

• Ties the transaction identifier to SMF 110 and 101 records generated by CICS and Db2.

• Provides the correlated SMF data to Splunk for a visualization of various performance attributes of the units of work.

• Time spent in the CICS and Db2 sub-systems along with resources consumed to support the transactions is clearly reported.



Advanced Filtering for SMF Data


Why Filter SMF Data?

SMF volumes can be enormous – large CICS and Db2 installations can generate TBs of data daily

Transferring data that is not useful puts a strain on network and other system resources

Need to provide control over volume of SMF data processed and forwarded by Ironstream to Splunk

Need to eliminate data clutter by forwarding only those fields that are truly needed


SMF Filtering and WHERE Processing

Ability to select only desired fields within individual SMF records– INCLUDE statement in configuration file or via field selection in the Ironstream

Desktop GUI

New extension enables selection of fields based upon the value of field

– WHERE clause in configuration file


Basic WHERE Syntax

"SELECT":"SMFnnn"

"INCLUDE":"field_1,field_2,...,field_n" Optional Statement– If omitted, INCLUDE defaults to ALL

"WHERE":"search_condition AND/OR search_condition“– Any number of search conditions can be specified

– If multiple search conditions are given, each must be separated by a logical AND or OR operator

– Search_condition: Field_1 operator operand• Field_1 must be the name of a field from the SMF record

• The operator can be: EQ, NE, LT, LE, GE, GT

• Operands can be another field name, character strings, decimal values, hex values, date, time

• Wildcards supported for character strings



Data Loss Protection (DLP)


Why is DLP Needed?

To prevent loss of data forwarded by Ironstream to Splunk

– Early implementations of Splunk did not include any mechanism for ensuring that data forwarded by Ironstream was both received and successfully indexed by the Splunk platform

• If Splunk encountered an error prior to indexing the data it received from Ironstream, that data was lost even though Ironstream had successfully forwarded it

– Network failures preventing Ironstream from forwarding data for a long enough period would cause the in-storage data buffers to overflow resulting in data loss


New Feature: Data Loss Protection (DLP)

Minimizes data loss during times of network or other external failures.

Uses IBM’s Coupling Facility’s System Logger functions, and Splunk’sIndexer Acknowledgement feature.

– Splunk indexer acknowledgement feature allows Ironstream to detect when data it has forwarded has been successfully received and indexed by the Splunk platform.


New Feature: Data Loss Protection (DLP)

Minimizes data loss during times of network or other external failures.

Uses IBM’s Coupling Facility’s System Logger functions, and Splunk’sIndexer Acknowledgement feature.

– Splunk indexer acknowledgement feature allows Ironstream to detect when data it has forwarded has been successfully received and indexed by the Splunk platform.

Optional feature that must be enabled….– Must define and configure a log stream within a coupling facility and make

Ironstream configuration parameter changes.

– No modifications required to existing Ironstream configuration files for those customers not requiring DLP.

– More information is available in the Ironstream Configuration and Users Guide.


Coming Soon! – IMS Log Data


Mainframe

TCP/IPSSL or non-SSL

Data Forwarder DCE IDT

Ironstream DesktopData Collection Extension

Data ForwarderData Forwarder

z/OS

Enterprise

Security

IT Service

Intelligence SPLUNK

DB2SYSOUT

Live/Stored

SPOOL Data

Alerts

Network

Components

Ironstream API

Application Data

Assembler

C

COBOL

REXX

USSLog4jFile

Load

SYSLOGSYSLOGD

logs

security

SMF

65+

types

RMF

Up to 50,000

values

IMS

Polling Question #2

What analytics platforms are you considering or evaluating to use for z/OS IT operational intelligence within the next 12 months:

Splunk

Hadoop

ELK (Elastic Stack)

Spark

Custom/Home Grown solution

Other

32

Summary: Value Today for Enterprises with a z/OS Mainframe


Less ComplexityCollect mainframe data; correlate with data from other platforms; no mainframe expertise required

Clearer Security InformationIdentify unauthorized mainframe access, other security risks; prepares and visualizes key data for compliance audits

Healthier IT OperationsReal-time alerts identify problems in all key environments View latency, transactions per second, exceptions, etc.

Effective Problem-Resolution ManagementReal-time views to identify real or potential failures earlier; view related 'surrounding' information to support triage repair or prevention

Higher Operational EfficiencyEnhanced event correlation across systems; Staff resolves problems faster; “do more with less”

Eliminate Your Mainframe “Blind-Spot”Splunk + Ironstream = Your 360ᵒ Enterprise View

Industry Leader in Mainframe Software Products

What Now?

35

Get Ironstream® for SYSLOG for free

VISIT: HTTP://WWW.SYNCSORT.COM/EN/PRODUCTS/MAINFRAME/IRONSTREAM

CONTACT: [email protected]

http://www.syncsort.com/en/TestDrive/Ironstream-Starter-Edition

http://www.syncsort.com/en/Products/Mainframe/Ironstream

mailto:[email protected]

http://www.syncsort.com/en/TestDrive/Ironstream-Starter-Edition

Thank You.Questions?