Upload
ca-technologies
View
69
Download
2
Embed Size (px)
Citation preview
World®’16
HighPerformanceMaskingandTestDataGenerationforIMSandDB2PrashantPandeyCATechnologies
MFX59E
MAINFRAMEANDWORKLOADAUTOMATION
2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.
Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.
ForInformationalPurposesOnlyTermsofthisPresentation
3 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Abstract
Withfinesaveragingmillionsofdollars,andprohibitivelyhighinfrastructurecosts,maskingandsubsetting themainframeisamust.ThissessionwillcoverhowtheCATestDataManagerarchitectureavoidsthediresecurityandperformanceimplicationsofmovingdatatoandfromz/OS.
PrashantPandeyCATechnologiesSr PrincipalConsultant,Presales
4 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataonz/OS(theBadNews)
§ z/OSsystemofrecord(70%datastillresideshere)§ ManyorganisationsrelyoncopiesofProductionfortesting§ Secureandreliableinproduction– whatabouttest?§ WhatandWhereisPIIdata§ Legislation– GDPR,demonstratecompliance§ BreachNotification– reputation&finesrisk§ Costofmovingtestdatato/fromplatform
(theGoodNews)BestPracticeTestDataManagementalreadyaddressesthisimpact
5 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
WhatisTestDataManagement(TDM)?
‘…isthepracticeofapplyingastructuredandcentralizedapproachtothemanagementoftestdataatanenterpriselevel,inordertoreducecostwhilstincreasingefficiencyandqualityindevelopmentandtesting’
6 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CA-TestDataManagerDeployment&Process
1. Centralize2. Discover3. Subset4. Mask5. Synthesize6. Find&Reserve7. Automate
Synthetic/CloneData
Masked
Subset
Created
ProductionDevelopm
ent
Web- SOATestHarness
Virtualization
DataMasking
Subsetting/DataSlicing
DataProfilingCoverageMetrics
Agile-Designer
TestDesign
DataDesign
TestDataonDemand
UI- LoadTestHarness
TestMart
FindandReserve
REQUEST/RESPONSEPOOLS
VERSIONCONTROLEXPECTEDRESULTS
REGRESSIONPACKS
DATADRIVENTESTPACKS
TEMPLATES
Agile-DataHPALM/QC
Automation
DataVisualization
Javelin
–Da
taOrche
stratio
n
7 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CA-TestDataManagerDeployment&Process
Adabas
z/Series
IMSDB2
VSAM
Unisys
DMSII
LUW
Informix
MSSQLOracle
Teradata
DB2/400 FlatFiles(CSV,Excel,XMLetc.)
i/Series
WindowsUI
ODBC/JDBCCOBOL/JCL
JAVASQL
LUW
CA– TestDataManager
Register&Profile
DefineTestDataEngineeringOperation
ProcessRun-time
EnableasOn-DemandRequest
TestDataOn-Demand
IMGR&ServiceLayerOracleorSQL
ServerRepository
8 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
SecureMaskedSubsetsof
Data
MainframeEnvironment
DB2z/OSVSAMIMSnon-indexedflatfilesOtherDBs
ValidationProfilingPIIDiscoveryDataSubsetDefinitionBuilt-InMaskingFunctionsSyntheticDataRules
RulesandDefinitions
MetaData
SecureandeffectivetestdataengineeringonMainframe
CATDMWindowsWorkbench
DataneverhastoleavetheMainframe
MainframeBatchOperation
Rules
Repository(RulesandMetaStore)
PDSJCL&CTL)
FTP
ODBC/Copybooks
9 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
End-EndIntegrity
§ Acrossmultiplesource(andplatforms!)– z/Series:DB2,IMS,VSAM(KSDS,ESDS,etc.)orflatfiles(PS)etc.– i/Series– AS400:DB2/400– Distributed:Oracle,SQLServer,DB2UDBetc.– Filestructures:CSV,Excel,XML,SOAP,REST,JSONetc.
Maintainingdatabase,businessandlogicalintegrity
CustID Item# OrderDate
07221 80-25442 2015-01-25
07221 80-25442 2015-03-14
CustID Item# OrderDate
10002 80-25442 2015-02-15
10002 80-25442 2015-05-19
Dataretainsstructure,contextandreferentialintegritybutdoesnotcontain“real”information.
CustID Name SSN Street
08052 AndrewChapman 181-23-7171 87Featherbed Lane
09112 Brian James 112-32-4435 12ParkLane
08052 AndrewChapman 181-23-7171 87Featherbed Lane
07221 TomAngle 771-11-8972 8Boardwalk
CustID Name SSN Street
10000 Hikaru Sulu 545-95-1245 87Featherbed Lane
10001 Nyota Uhura 802-41-3411 12ParkLane
10000 Hikaru Sulu 545-95-1245 87Featherbed Lane
10002 PavelChekov 325-74-6517 8Boardwalk
10 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
ProjectandVersionCentralizedControl
§ Re-usableassetsinaTestDataWarehouse
§ Providesacentrallibraryoftestdataengineeringoperations
§ Assetscanbeupgradedacrossversions(releases)
§ Regressionpackscanbestored
CATestDataManagerTestDataWarehouse(Repository)
11 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Automaticallydiscoversensitivedatastoredenterprise-wide
§ Powerful,mathematically-basedalgorithmsdiscoverandtagallsensitivedata,enterprise-wide
§ Simple, easytousefilteringbytagwhenselectingcolumnsortablestomask
§ Supportforeverymajordatabasetype,mainframeplatforms andflatfiles
CATestDataManagerDataProfiler
12 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Subsetting:datasetsareextractedbasedonexactcriteriaCATestDataManagerDataSubset
Referentiallyintactsubsetsofdata
ProductionAttributes
ScenarioSubset
CoveredSubsetProduction&Enhanced
Attributes
13 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
De-sensitizemillionsofrowsofreferentiallyintactdatainminutesCATestDataManagerFastDataMasker
Theseincludesubstitution,randomization,hashingandseeddata.Oncecreated,maskingrulescanbestoredandre-usedfromacentralrepository.
GTFastDataMaskerwillonlydisplaysuitablemaskingroutines,basedontheselectedcolumnandtables
Deterministicmaskingfunctionsandbuilt-incross-referencingensureconsistency.Thereferentiallyintact,realisticdata,canbeinjectedintomultiplesystemsatonce
NumberofRows Timeto Mask
201,722,392 9minutes, 42seconds
453,877,152 11minutes,22seconds
768,088,071 7minutes,57seconds
17,422,541 1minute
46,579,485 1minute, 25seconds
1,759,612 13seconds
47,895 5seconds
GTFastDataMaskerusesnativedatabasescriptstoproducethehighestpossibleperformancewhenmaskingOracle,MSServer,Teradataandmainframeplatforms
§ Maskmillionsofrowsofdata inminutesusingnativemaskingengines
§ Over 80built-indatamaskingfunctions
§ Replacementswithrealistic,referentiallyintact, appliedtomultiplesystems
§ DemonstrateCompliancewiththeEUGDPR,GLBA,HIPAA,PCIDSS,PIPEDA,andmore
14 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GenerateRequiredDataScenariosCATestDataManagerSyntheticDataGeneration
Thismeansthatnewdatacanefficientlyandquicklyinjectedintoexistingdatabasesforexactlytherequirementsyouneed
§ ExtensivealgorithmsforSyntheticdataappliedinaDataPainterstyle
§ UseSymbolic values,combine functions,include conditional valuesandmine datafromothersourcesintothenewdata
§ Publish(inject/load)singledataobjectasmultiple/variednewrowsinexistingdatabase
§ Storedataobjectscentrally,asre-usableassetsinaTestDataWarehouse
§ Datacanbere-usedacrossversionsandreleases,withversionedregressionpacks
15 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Find&ReserveExistingDataCATestDataManagerTestMatching
SAP-1
DataAttributeMining
DataTestConditions
DatamakerTestMatching
DB-2
XML
MatchedTest
QTPScripts
16 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CombineTDMOperationstoProvideComprehensiveSetofData
Referential Integrity
Profiling
PII Discovery
Native DB Engines
Seed Replacement
Audit
CATestDataManager
1 2 3SecureDataSubsets
Test/DevEnvironments
Gold Copy
Subset
Seed Tables
Built-in Masking Functions
Cross Referencing
Masked
Generated Scenarios
Find & Reserve Data
SyntheticContent
SyntheticContent
Productiondatasourcesandfiles
XMLFilesExcelFilesSQLFiles
CSVFiles FixedDefinitionFiles
HTMLFiles
VSAM/ISAMSwift
TXTFiles
TestData
Warehouse
17 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Avoidcostlymistakeswhenmanagingz/OSTestData
Avoiduseofrawproductiondataisamandatoryoperationformodernorganizations.Theaveragecostperincidentofdatabreachworldwideis$3.5m.IntheUS,theriskwasashighas$5.85m1.
Inasmanyas59%oftheseincidents,theriskcouldhavebeenmitigatedbyimplementingTestDataManagementbestpractices
1 Ponemon Institute,2014
CATDMallowsuserstoapplyTestDataManagementpracticesdirectlyonthemainframe reducingtherelianceonslow,expensiveExtract,Transformandreloadtechnologiestoensure complianceandtakeadvantageoffit-for-purposetestdata.
Processingdatadirectlyonthemainframe canprovideteamswithmillionsofrowsofhighquality,meaningful,referentiallyintactdata,inashortperiodoftimeandatlessrisktoquality!
18 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
RecommendedSessions
SESSION# TITLE DATE/TIME
MFX58ECCATestingToolsEducationalSession(Part3of3):CAMainframeApplicationTunerDemoandWhat'sNew 11/15/2016at2:00pm
MFX112S MainframeApplicationDevelopmentPortfolioReview 11/16/2016at12:45pm
MFT148SAchieveDevOpsontheMainframeforFasterTimetoMarket 11/16/2016at4:30pm
19 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
MustSeeDemos
ContinuousDevelopmentandTestingMainframeTheater
ContinuousDelivery
MainframeTheater
20 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Questions?
21 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Thankyou.
Stayconnectedatcommunities.ca.com
@CAWORLD#CAWORLD ©2016CA.AllRIGHTSRESERVED.22 @CAWORLD#CAWORLD
MainframeandWorkloadAutomation
Formoreinformationpleasevisitca.com/appdev
PrashantPandeySr PrincipalConsultant,[email protected]