1
Shadowbase Data Replication and Data Integration Solutions
Paul J. Holenstein
Executive Vice President
Shadowbase Products Group
GTUG - 2014
www.gravic.com/shadowbase 2
Agenda
• Introduction to Gravic • Shadowbase Product Overview
o Business Continuity o Data Integration & Synchronization o Application Integration
• Shadowbase Success Stories • What’s New for Shadowbase • What’s Next for Shadowbase
Questions? Please ask as we go along…
1979 – Founded, Development Organization & Service Bureau 1984 – Introduced NonStop Replication Product TMF Auditor 1995 – Shadowbase for HP NonStop Released 1996 – Shadowbase Heterogeneous Platform Support Released 1997 – Shadowbase for Oracle, SQL Server, and Sybase Released 1998 – Gravic Labs Formed 2000 – Patented & Released Shadowbase NonStop Bi-Dir Support 2002 – Released Bi-Dir Heterogeneous Platform Support 2005 – Surpassed 500 Licensed Shadowbase Systems 2008/9 – Released SQL/MX, ACI BASE24®, GE Centricity™ Enterprise Support 2011 – Gravic Won Two Industry Awards from HP & Intel for Technical Excellence 2012/13 – New Synchronous Replication Features Delivered/Beta; Gravic won STEVIE Award 2014 – Gravic working with HP NED on a HP Reseller Deal for Shadowbase Products
Introduction to Gravic
www.gravic.com/shadowbase 3
A History of Excellence: 35+ Years and Going Strong!
• Leverage product, services, and partners to offer complete business problem solution
• Industries include financial services, trading systems, banks, telecommunications, and healthcare
Total Replication Solutions®
Banking & Financial Services:
Telecommunications:
Other:
U.S. Cellular
Selected Shadowbase Customers
• Active/Passive Disaster Recovery • Sizzling-Hot-Takeover • Active/Active Continuous Availability • Eliminate Planned Downtime for Migrations & Upgrades (ZDM)
• Homogenous & Heterogeneous Environments • Data Transformation, Scrubbing, Filtering & Cleansing • Extend Replication Capabilities with Embedded Application Logic
• Build Event-driven Architectures o No more polling for needed data
• Build Real-Time Architectures o Process events as the occur; no more working with “stale” data
• Integrate Disparate Applications with no Application Code Changes o Integrate at data-layer, avoiding costly adapters & middleware
Shadowbase Product Suite Overview
www.gravic.com/shadowbase 5
Business Continuity & Application Availability Environments
Data Integration & Data Synchronization
Application Integration
Shadowbase Suite: Enabling, Extensible Technology!
HP NonStop
SQL/MP SQL/MX
Enscribe
AS400
DB2
HP NonStop
SQL/MP SQL/MX
Enscribe
VMS
Oracle Sybase
Windows
Oracle SQL
Server DB2
Sybase
Unix/Linux
Oracle MySQL
DB2
Linux, Unix, Windows
SQL Server
Oracle Sybase
Any ODBC Target Platform/Database (e.g., Teradata)
Shadowbase Product Suite Overview
HP NonStop
SQL/MP SQL/MX
Enscribe
Linux, Unix, Windows
SQL Server
Oracle Sybase
HP NonStop
SQL/MP SQL/MX
Enscribe
Linux, Unix, Windows
SQL Server
Oracle Sybase
Homogeneous & Heterogeneous Uni/Bi-Directional Data Replication
6 www.gravic.com/shadowbase
• Audit Compliance Reporting & Analysis
• Validate Target Matches Source
• Load/Refresh Data Warehouses & Data Marts Using ETL
• Build Real-Time Business Intelligence &
Active Data Warehouse Systems
• Integrate Operational Processing with Ancillary Systems
• Eliminate Planned Application Downtime (ZDM)
• Restore Corrupted Databases On-line Eliminating Application Outages
• “Chases” Non-audited Log Files to Inject Events into Replication
7
Shadowbase Product Suite Overview Shadowbase Product Extensions/Example Uses
www.gravic.com/shadowbase
www.gravic.com/shadowbase 8
Shadowbase Product Suite Overview Shadowbase Product Extensions/Example Uses
• Performs Optional Disk-Based Source or Target-Side Queuing of Data
• Online Loading Facility Performs High-Volume Initial & Refresh Loading of Target from Source
• Modernize an Application By Allowing Both Legacy & New Versions to Coexist with No Application Downtime
• Manage, Command & Control GUI from Your Workstation
• Test Database Creation, QA Database Refresh, Data Obfuscation
• No-Obligation Trial, Test, Train, Tune
– We’ll Build Your Environment in Our Lab or On-site & Demo
Business Continuity Overview
Last Saved Data Point
RPO/RTO Relationship
Potential for Data Loss
Failure Point Time to Recover
RPO RTO
Recovery Complete
System Processing Time
9
• Recovery Point and Recovery Time Objectives – Recovery Point Objective (RPO) and Recovery Time Objective (RTO) are two commonly
used terms to describe business continuity requirements. – They are business decisions based on your applications’ needs
RPO - describes the point in time to which the data must be recovered RTO - describes the time from when a failure occurs until the business process must
become available/active again.
www.gravic.com/shadowbase 9
Primarily Influenced by Technology (Async vs Sync)
Primarily Influenced by Architecture (Active/Passive to Active/Active)
Business Continuity Overview
• Recovery Point and Recovery Time Objectives – Recovery Point Objective (RPO) and Recovery Time Objective (RTO) are two commonly
used terms to describe business continuity requirements. – They are business decisions based on your applications’ needs
RPO - describes the point in time to which the data must be recovered RTO - describes the time from when a failure occurs until the business process must
become available/active again.
RPO/RTO Relationship Last Saved Data Point
Potential for Data Loss
Time to Recover
RTO
Recovery Complete
RPO
10 www.gravic.com/shadowbase 10
Failure Point
www.gravic.com/shadowbase 11
Shadowbase for Business Continuity (SYNC) Shadowbase Product Suite
1. Current-Shadowbase Disaster Recovery (Async, Active/Passive) 2. Current-Shadowbase Sizzling-Hot-Takeover (Async, Active/‘Almost’ Active) 3. Current-Shadowbase Continuous Availability (Async, Active/Active) 4. Future-Shadowbase Disaster Recovery (Sync, Active/Passive) 5. Future-Shadowbase Sizzling-Hot-Takeover (Sync, Active/‘Almost’ Active) 6. Future-Shadowbase Continuous Availability (Sync, Active/Active)
BC “Sweet Spot”
RTO (faster recovery)
RPO
(less, or no, data loss)
Async Disaster
Recovery
Async Active/ Active
Sync Active/ Active
Async Sizzling-Hot-
Takeover
No Data Loss, Fastest “Recovery”
Little Data Loss, Fastest “Recovery”
Sync Disaster
Recovery
Sync Sizzling-Hot-
Takeover
www.gravic.com/shadowbase 12
Shadowbase for Business Continuity Zero Downtime Upgrades and Migrations (ZDM)
• Minimize or eliminate “planned” application downtime • Build & test new environments without impacting existing
production users • Avoid the risk of the outage window “big-bang” cut-over –
Avoiding “failover faults” o Cut-over to a known-working application/database environment o Very similar to the sizzling-hot-takeover benefits!
• Use for: o O/S or Platform Upgrades o Developing new application releases o Database upgrades or conversions o Moving “other” application or database environments onto NonStop
• Many datatypes mapped automatically o For example: CHAR, numerics, many dates/times, etc.
• Complex datatypes mapped via either Data Mapping scripts or the Shadowbase embedded User Exit capability
• Blank and null-fill processing, NULL mapping, date/time cleansing, etc.
• Remove events being replicated based on data content • Consolidate multiple events into one
• Data normalization support provided (e.g., for redefinition support, array fan-out, etc.)
• For example NonStop Enscribe to Oracle
Shadowbase for Data Integration
www.gravic.com/shadowbase 13
Non-Relational to/from Relational Data Formats
Data Filtering and/or Aggregation
Data Scrubbing and Cleansing
Shadowbase Manages Data Mapping for Source to Target
14
Integrating HP NonStop trading applications with online trading applications
Internet client
Windows Linux Unix IBM
Firewall Firewall
Traditional Brokers
NonStop Cluster
Oracle
Sybase
SQL Server
DB2
MySQL
Flat Files
Internet client
Shadowbase
www.gravic.com
Internet
Large Scale SaaS
Shadowbase for Data Integration
200+
www.gravic.com/shadowbase 15
Bi-Directional Active/Active NonStop with Oracle/RAC
sales
refunds
exchanges
Oracle
RAC
cluster interchanges
AJB RTS
AJB FIPay
Combined Stores
Linux servers
Integrated Refunds and Exchanges
Brand 1's
NonStop
BASE24
sales
refunds
exchanges
interchanges
Brand 1 Stores
SQL/MP
Enscribe
Brand 2's
NonStop
BASE24
sales
refunds
exchanges
interchanges
Store POS terminals
Brand 2 Stores
SQL/MP
Enscribe
Store POS terminals
ACI BASE24™ AJB
Shadowbase for Data Integration
• Shadowbase monitors the TMF transaction log and can “trigger” on all DML or DDL database activity (e.g., inserts, updates, or deletes)
• As soon as the event occurs in the database, Shadowbase processes it
• No need to modify application code (assuming you *have* the code) • Avoid inefficient polling of the database for changes • Shadowbase acts as the hub, feeding pertinent database change events
to all other system(s)… • Build efficient data-driven Publish/Subscribe event-notification
architectures using replication as the middleware
Shadowbase for Application Integration
www.gravic.com/shadowbase 16
Shadowbase Enables Event-Driven Architectures
Shadowbase Provides Real-Time Data Delivery
Shadowbase Integrates at the Data Layer
www.gravic.com/shadowbase 17
Selected DB Activity
Application IPC Feed
Business Logic
Function:
Shadowbase “sees” all changes to the application’s data…and can act on them.
Uses:
• Shadowbase acts as a capture process for change events from the database/audit trail and notifies or delivers them to downstream files, applications, or middleware.
Middleware
Feed
OR OR
Consumer Collector
Customer Application
Customer Database
Audit Trail
Shadowbase for Application Integration Database Event Capture and Delivery
Consumer
www.gravic.com/shadowbase 18
Fraud Detection Application
Shadowbase
Shadowbase
UNIX NonStop
NonStop-based ATM/POS application feeding transactions into RiskShield (a Unix-based real-time fraud detection system)
Business Logic
Consumer
Business Logic
Shadowbase for Application Integration Real-Time Fraud Detection
Collector
ATM/POS Application
ATM/POS Database
Audit Trail
19
2) Business Continuity – Active/Active Switch
Shadowbase
\NonStop \NonStop
ATMs
3) Data Integration – Active Data Warehouse/Real-time Fraud Detection
\NonStop Oracle
Shadowbase Client Services
1) Business Continuity – Disaster Recovery
\NonStop
Shadowbase
Clients
\NonStop
Shadowbase Multi-Use Case Study Implementations at First Data Corporation
www.gravic.com
www.gravic.com 20
BASE24
Regions 1,2,3,4,5
Non-Shadowbase
Uni-Directional Replication
Availability Modernization – The Old Way
Key Issues:
• Paying for idled system capacity/licenses
• All users affected at failure
• More data loss at failure
• Worse recovery time – application not running
• Will target application come up???
• Target DB read-only (inconsistent for reporting)
Shadowbase Success Stories Active/Passive: Uni-Directional Disaster Recovery
ACI BASE24™
www.gravic.com 21
Partitioned
Regions
Availability Modernization – The New Way
BASE24
Regions 3, 4, 5
Key Benefits:
• No idled system capacity
• Fewer users affected at failure
• Less data loss at failure
• Faster recovery time - application already running
• Failover to known-working system
• Both DB’s active/available for application work
BASE24
Regions 1 & 2
Shadowbase Bi-Directional Replication
Shadowbase Success Stories Active/Active: Partitioned Application Users
ACI BASE24™
• Shadowbase Synchronous Data Replication Engine with Guaranteed Data Delivery (Zero Data Loss at Failure)
• Incorporates HP TMF Synchronous Gateway API o Working with HP TMF Development on Various Enhancements
• Active/Passive, SZT, and Active/Active Architectures Supported o Data Collisions Possible
• File/Table and TX Level Granularity • POC Completed, Beta Programs Underway
• Leverages Shadowbase ZDL Architecture to Provide: o Zero Data Loss if a Failure o Collision Avoidance for Active/Active Architectures
• Beta Program to Follow Shadowbase ZDL
What’s Next for Shadowbase
www.gravic.com/shadowbase 22
Sync Replication – Shadowbase ZDL Plus SR™
Sync Replication – Shadowbase ZDL™ (Zero Data Loss)
Interested in Learning More? See Separate Presentation on Shadowbase Synchronous Replication
NOTE: Shadowbase ZDL™ and Shadowbase ZDL Plus SR ™ are future technologies and require HP’s NonStop Synchronous Replication Gateway; specifications are subject to change without notice and delivery dates are not guaranteed…
• Shadowbase is deployed at hundreds of sites, including many of the most-demanding NonStop sites
• Business Continuity, Data Integration & Synchronization, Data Warehouse Feeds, Application Integration, Real-time Business Intelligence
• Gravic’s expert global 24x7 support organization partners with our local reseller support
• New Strategic Alliances with HP in Process
• Improves TCO via Overall Cost Advantage and Features o “One Product, Many Solutions”
• We are partnering and investing with HP in many innovative enhancements o “Only on NonStop”
Why Choose Shadowbase?
www.gravic.com/shadowbase 23
Affordable, and Committed to the NonStop Platform
Global 24x7 Support Organization
Flexible Solutions for Your Business Challenges
Proven Technology
• Survivable Systems for Enterprise Computing
• Achieving Century Uptimes with Active/Active
• Active/Active Systems in Practice
For More Information
www.gravic.com/shadowbase 24
Volume 1
Volume 2
Volume 3
Breaking the Availability Barrier Book Series
Also, Visit our Web Site for Case Studies, Executive Briefs, and White Papers: http://www.gravic.com/shadowbase/news.html
www.gravic.com/shadowbase 25
Questions?
Gravic, Inc. 17 General Warren Blvd. Malvern, PA 19355 USA
[email protected] [email protected]
www.gravic.com
Phone: +1.610.647.6250 Fax: +1.610.647.7958
Find us on…
www.gravic.com/shadowbase 26
Supplemental Slides Follow…
www.gravic.com/shadowbase 27
Disclaimer
This presentation contains forward-looking statements and describes potential technologies that are under consideration, and either do not exist or are only starting to become available…there is no guarantee that the technologies described in this presentation will become available on the HP NonStop platform. Specifications are subject to change without notice and delivery dates/timeframes are not guaranteed…
www.gravic.com/shadowbase 28
Making Continuous Availability the Goal…
HP NonStop Technology makes localized fault-tolerance a reality Shadowbase extends this technology to geographic fault-tolerance, while minimizing risk by selecting the right business continuity architecture for your business’ needs
Source: HP Solution Brief # 4AA4-1791ENW
The question is: When is it okay for your business to be unavailable to your customers? Never? Then let’s talk…
www.gravic.com/shadowbase 29
Async: Disaster Recovery Protection
Classic DR: • Active → Passive • Uni-Directional Disaster Recovery • Application active on primary node
only • Passive node may be used for read-
only OLQP
Key Issues: • Difficult to test backup and failover (requires outage) • High testing costs • Failover faults occur • Backup capacity mostly unused • All users affected at failover • More data loss at failover • Backup database may be
inconsistent
GE Centricity® Enterprise
Uni-Directional Replication
Clients
Backup
All Transactions
Primary For these reasons of failover uncertainty,
decision to execute failover requires senior
management decision which takes more time,
extending the outage.
Shadowbase for Business Continuity
30
Shadowbase for Business Continuity Async: Sizzling-Hot-Takeover (SZT)
Billing/Customer Service
Bi-Directional Replication
Step-up from Classic DR: • Active → Almost Active • Backup application hot, database
open R/W (improves RTO) • Bi-Dir replication configured
(facilitates recovery) • All transactions routed to one node • No data collisions • Easy to validate backup - submit
verification Tx • Failovers easy to test (no outage) • Excellent solution when application
cannot run fully A/A
Key Issues: • All user affected at failover • More data loss at failover • Backup capacity mostly unused
www.gravic.com
Active/Active Partitioned: • Active ↔ Active • Applications active on all nodes • Bi-Dir replication • Applications and/or data
partitioned (tx’s routed to specific node based on data content)
• Users load balanced across nodes (pre-assigned a primary node)
• Avoids data collisions • Fewer users affected at failover • Less data loss at failover
Key Issues: • Not all applications/data can be
“partitioned” • Imbalanced load distribution • More complex to implement
31
Shadowbase for Business Continuity Async: Active-Active – No Collisions
Customers
A-M
Bi-Directional Replication
Customers
N-Z
www.gravic.com
32
Shadowbase for Business Continuity
www.gravic.com
Async: Active-Active – Collisions can Occur
ATM Switch
Bi-Directional Replication
Active/Active Route Anywhere: • Active ↔ Active • Applications active on all nodes • Bi-Dir replication • Tx’s routed anywhere • Requests evenly load balanced
across nodes • Fewer users affected at failover • Less data loss at failover
Key Issues: • Data collisions can occur which must
be automatically identified and resolved via application logic embedded into replication engine.
(E.g., most recent update wins) • More complex to implement
33 www.gravic.com
Shadowbase Success Stories Multi-Shadowbase-Use Case Studies
Application and Data Synchronization – First Data Corporation
Availability Modernization – Royal Bank of Canada