Upload
vohanh
View
240
Download
4
Embed Size (px)
Citation preview
Internal and Confidential
Methodology forSAP Data Migration using
Informatica
NON LIVE FEED SCENARIO
For live feed scenario, where site/country data keeps changing on ongoing basis even after go live and needs to be incorporated in target SAP/other
system, please refer to our other presentation.
Agenda
Agenda
What's New Now ?
New Conversion platform Design (AIP)
Screen Shoots of Future AIP Platform
Nomenclature
What’s New
• Benefit Highlightsq We present you a architecture model, that will meet your immediate and long term needsq A architecture that has a proven track record of being one of the best. It has been built over few
years of continuous inputs and acts as great referenceq Eliminate try and try approach by inexperienced resources.
Ø This approach can be costly in terms of time and moneyØ Often this is a continuous ongoing patchy work to existing solutionØ Often becomes more and more unmanageable due to patch works
q Delivery of proven platform will deliver great values and provide cost saving on business front tooq This platform gives the country data owners high level of transparency in term of seeing there data
that gets selected, the data that would get loaded along with errors at each check level.q Save multi millions euros over the next 1-2 years in terms of learning curve, avoiding fixes to
existing work, eliminate waste of time by dependent users to get right reports etcØ Shorten delivery of right platformØ Delivery capability to manage / roll-out more sites at the same time within fixed deadlinesØ Shorten each phase time due to better platform and quick turnaround time for checks and
appropriate reports
NEW METHODOLGYA proven methodology and mature technological architecture to
be used for Data Migration projects by big clients with multiplecountries/sites to be migrated in a phased plan over a period
of time.
What’s New
• New Experienced resourcesq We bring on board new resources with proven experience of working on this large scale SAP
Migration projects. These resources has the needed exposure to plan, design and run data conversion in a controlled fashion
q These resources understand the data dependencies and complications across objectsq They understand the pre-checks and SAP data inclusion to be done prior to object loadsq They are highly methodical and capable of crisis management
• Better Controlq Introduction of new template to manage tasks, progress and issues at SAP object levelq Introduction of new template for enhanced SAP configuration values managementq These templates will deliver the appropriate graphs and figures for management reports
• Financial ROIq This approach is a candidate for projects with more then xx million € budget and more then 1 siteq It can assure 10 upto 30% financial savingsq It can guarantee higher quality data migration and thus ensuring further saving of 10%, which
normally is spent to fix poor data quality migration. This is ensured by this platform approach and further supported by our suggested strategy to manage real time data quality validations.
General Client issues at hand
• Tight deadlines ahead• Enhance reconciliation and error logging capabilities to offer desired level of
data records report to client for each country• Certain enhancements needed for most objects implemented up to S2 level• Final touches needed for objects implemented in S3 level (IDOC writing phase)• S3 Mappings need enhancement for
q Inclusion of SAP existing data wherever needed eg. Contact persons, partner functionsq Inclusion of further checks against SAP data
• Standardize all connection parameters to ensure smooth run across DEV, TEST and PROD system load runs
Certain clients already has a ongoing project; in those scenarios too we come across below mentioned issues to resolve on urgent basis apart from building the future
to be ideal solution
Agenda
Agenda
What's New Now ?
New Conversion platform Design (AIP)
Screen Shoots of Future AIP Platform
Nomenclature
Team Setup Approach
• Based on add-on of new experience in the team and to meet project goals in time with high satisfaction, below is new setup proposal
SAPSystem Load
& Reconciliation
S1) Basic Mappings for extracting Site data and
map it to object templates
S3) SAP Check & Load Mappings
Highly experienced resources with knowledge of different
SAP data objects and there correlations.They will also design the needed templates
and reconcilliation model.
General Informatica Resources
S2) Mappings for Basic checks &
Data accumulation
Informatica Site Process
LegacySystem Mapping
DataCleansing
FreeFormat
FlatFile to Central Team, via FTP
SiteResponsibility
Off-LineQualityChecksLLLL
JJJJ
•Check KDS values•Check Currency formats, values•Check Date formats•Analyse missing Req field values
•Check 0, Site Mapping•Check 1 Mapping
Informatica Cluster/Central Process
SAPProduction
System
ClusterResponsibility
JJJJ
ChecksChk1Chk2Chk3
Online Quality Checks
LegacySystem Mapping
DataCleansing
FreeFormat
FlatFile
SiteResponsibility
Off-LineQualityChecksLLLL JJJJ
LLLL
OverAll Process Flow
LEGACYSYSTEM SAP
R/3
Target/Source
fileSource
fileLoadData
CLUSTERSITE
Check 1
Check 2
Check 4
Load
Check 6&7
Check 1 errors
Check 2 errors
Review comments on checks
Check 3 Duplicates/Load errors
Sourcefile
Check 3
Site MappingCheck 0
Check 1
Check 5
SQL
Excel Template/
KDS
CONFIGREPO
Sourcefile
ExtractData
ExtractData
LKP
Web Portal- One Point Control Center
Informatica Mappings Central (C)- Team
• Check 1 Mappingq Perform Relevance check (quality checks level 1)q Running this mapping to ensure that all the necessary data fields are available from the data
ownersq Generated separate files for good and error records
• Check 2 Mapping : FD business rules checkq Perform Completeness check (quality checks level 2)q It checks if the data conforms to the business rules as specified in the FD. The business rules
cover, among others, data ranges, data formats and value dependencies. Check completeness of local data with cluster scope, identify incomplete data (e.g. no post code on an address) and missing data (e.g. no date of birth or address for permanent employee).
q Examples:Ø Site X can only have Plants Y and Z.Ø Why do we try to load a PIR for a Material which was not created or extended for you plant.Ø Why do we want to load a PIR when the specified Vendor was not in the Vendor file.
•q Perform duplicates check and generates output file with duplicate recordsq Mapping 3 loads all records which passed Chk-2 into a SQL table.q Assess inner duplicate ratio (percentage of duplicated records within site legacy systems) and outer
duplicate ratio (local data already existing in SAP)q A version of this mapping also deletes all the previously loaded data for this delivery for this specific
site
Informatica Mappings C- Team
• Check 4 Mapping (SAP Load)q These mappings writes the data to IDOCsq Some of these mappings also generate text load files in the required format for loading
into SAP using other tools like LSMW, WinRunner, WinShuttel, ABAP etc
• Check 5 Mapping (SAP_SENT_FLAG)q It takes a file/table with the SQL keys back against the SQL table and set the
SAP_SENT_FLAG to 'Y' to indicate the rows which were sent to SAP for loading.Ø This step also helps capture SAP-NO for common objects like HR load, vendor
create, customer create
•q It is used to reconcile the data loaded into SAP. There is no standard layout for this
mapping. The checks are specified in the BII (business check documents).
•q checks the integrity between data objects in SAP. There is no standard layout for this
mapping. The checks are specified in the BII.
•q These mappings downloads the data from SAP tables on daily basis or regular
intervals.
OverAll Process Flow – Error Reporting
SAPR/3
LoadData
CLUSTERSITE
LoadC
heck 1 errors
Check 2 errors
Review comments on checks (Email)
Check 3 D
uplicates/Load errors
CHECKS SQL CHECKS
Informatica Check Reports (Email)
CTQuality
Checks & Feedbacks
Web Portal-One Point Control Center
Business User / Data owners
Checks & Reviews
& Corrections
Online Status and Error Log access
File Server
Informatica –File Server Overview - Central
• It covers the understanding of the location of in & out files used for the mapping and tracing of them on server; these are primarily the I/O files related to mappings.
• Also we cover the following things in context of specifying the sessions for mappings:
q Speciying of Source files locationq Speciying of Target files locationq Locating error files generatedq Locating duplicates/bad files generated
• In brief, we highlight the tracing of following In & out files types of files:q Source, Target Error, Duplicates Files –Chk1q Source, Target Error Files –Chk1 (C)q Source, Target Error Files –Chk2q Duplicates Files –Chk3
• The initial decision for the source, target locations are specified by the Informatica expert OR the person responsible for administration of your informatica Server.
• The location of the files generated is somewhere on a central server, which has restricted access. It is highly organized by SiteCodes and ObjectNames.
The Informatica I/O are usually structured as:INF_ROOT/
SITE_ID/<<Folders>>
Informatica –File Server Overview - Central
• Source Fileq Your source file is with the same name that you used in FTP name.q In above case it is with <object_name>.out (eg. c002_vendor_partner_function.out)
• Target Files –Chk1q Files generated after the automatic running of the CHK1 central mappings are
<object_name>_chk1.out & <object_name>_chk1_error.outq <object_name>_chk1.out represents the file with correct recordsq <object_name>_chk1_error.out represents the file with error recordsq In above eg. the concerned files are:
Ø c002_vendor_partner_function_chk1.out Ø c002_vendor_partner_function_chk1_error.out
• Target Files –Chk2q Files generated after the automatic running of the CHK2 central mappings are
<object_name>_chk2.out & <object_name>_chk2_error.outq <object_name>_chk2.out represents the file with correct recordsq <object_name>_chk2_error.out represents the file with error recordsq In above eg. the concerned files are:
Ø c002_vendor_partner_function_chk2.out Ø c002_vendor_partner_function_chk2_error.out
• Target Files –Chk3q Files generated after the automatic running of the CHK3 central mappings are
<object_name>_sql.badq <object_name>_sql.bad represents the file with duplicate recordsq In above eg. the concerned files are:
Ø c002_vendor_partner_function_sql.bad
• Each time you FTP your file to central server, a special code and a directory with that code name is created to track all actions on it. Eg. 20090815_1609 on right side. It consist of date and a random number.
Informatica – Disorganised Site Data
• Project Scenario - where we are not organised for Site data?q Not easy to remember last used txt file version of a FDq Not easy to takeover the work of the colleague during his holidays, illness etc scenario‘sq Mappings might pick an old file OR overwrite a similar filename of other object.q Not easy for data super users/Data Owners to update new changes/ correctionsq More stress to locate or navigate through different objects
Informatica Mappings – Organised Site Data
Objective is to map Legacy data to Target Mapping in structured approach.
• Performed by Site/(designated central team resources) Informatica resource, this process involves feeding of the raw data in the mappings and producing a file format as desired for Site Check1 mapping.
q This is a tedious process and involves careful analysis of input data. The Informatica resource has to make the mappings to perform this transformation and it might involve lot of data manipulations etc.
q Please do not perfom complex business checks, which are already infact taken care in the C-Team mappings, to avoid duplication of activities
Chk0 Mapping-• Basic activies in the basic site mappings are :
q Legacy Unit Codes into SAP Unit Codesq Format Codes – eg. Sales Material begin with ‘A’ and have no ‘-’q Set the KDS Valuesq Set Default Valuesq Lookup tables to compare values, return new data values.q Test for small relationships eg. bankcode of vendors not in Banks fileq Filter out any unwanted records Or take only desired ones eg. Same source file contains all
customers and we need to filter out External, Internal etc.
Chk1 Mapping- Use the shortcut to C-Team Chk1 mapping to verify your data
Informatica –File Server Overview - Site
• It covers the understanding of the location of in & out files used for the mapping and tracing of them on server; these are primarily the I/O files related to mappings.
• Also we cover the following things in context of specifying the sessions for mappings:
q Speciying of Source files locationq Speciying of Target files locationq Locating error files generatedq Locating duplicates/bad files generated
• In brief, we highlight the tracing of following In & out files types of files:q Source, Target, Error, Duplicates Files –Chk0q Source, Target Error, Duplicates Files –Chk1
• The initial decision for the source, target locations are specified by the Informatica expert OR the person responsible for administration of your informatica Server.
• It is OK to establish subdirectories under the folders SrcFiles, TgtFiles and BadFiles for better management of your files.
Informatica –File Server Overview - Site
• Site Basic mapping (Chk0 ) Contextq Source files
Ø SrcFiles used as the input to mappings; should be a location on the Informatica serverØ Files with <object name>.txt extension are stored here.
q Target files (under)Ø TgtFiles used to generate the output files, lying at your Informatica serverØ Files with <object name>.txt extension are generated here.
q Error filesØ Basically not generated as such, all targets are output files; in case you generate one, it should
be kept under TgtFiles location. The session run errors are however located under SessLogsq Duplicates/BAD records files (under)
Ø BadFiles directory is used to generate these files by default.Ø A file could be generated in case your target is a DB table and you have implemented some
primary keys that resulted in not loading of some records.
• Site mapping (Chk1) Context The CHK1 mapping you use also make use of the above directories but some times in a different way.
q Source FilesØ This directory SrcFiles is not used for picking the source file for these mappings rather we use
the files from TgtFiles directoryØ Files with <object name>.txt extension lying under TgtFiles are used as source files
q Target FilesØ Files with <object name>.out extension are generated under TgtFiles by these mappingsØ Once these .out files are considered good enough, please activate the FTP option and send
this file to appropriate folder on the central conversion site server.q Error Files
Ø Same as above
Agenda
Agenda
What's New Now ?
New Conversion platform Design (AIP)
Screen Shoots of Future AIP Platform
Nomenclature
AIP (Automated Informatica Platform)- Main Functions
• Scope Controlq Global Repository for all data objects (documentation)q Site specific scope control
• Manage File Deliveriesq Display error files directly in excelq Rerun – Submit- Follow Up on errors
• Planningq Set Delivery / Load dates
• Statisticsq Follow Up on delayed deliveries / loads
• SAP Downloadsq Manage all table downloads from SAPq Instantly refresh tables – Selective access to end users
• Easy To Maintainq Maintain multiple sites at the same time in the AIP q Adding additional sites will have no lead-time < 1 minute to setup.q Whole solution can be maintained by 2-3 Informatica Developers at the central data
conversion location
Agenda
Agenda
What's New Now ?
New Conversion platform Design (AIP)
Screen Shoots of Future AIP Platform
Nomenclature
Nomeclature
• P3 Targets, Sourcesq Central Team generates these Targets, Sources.q The structure specified in the FD needs to be built in Informatica as a target. The sites will map their
source to this target in the mapping process.• SAP Extracts
q Central Team Extracts current production SAP Data for Sites to use as reference tables. eg. Vendors,Customers, Materials, Banks.
• Site Mappingsq Used to refer Mappings used for the purpose of extraction/formating of data from legacy systems
into the P2 Target format.q It could be 1 or more mappings for each FD object based on complexity
• Site Chk1 Mapping q A shortcut to the Central teams Chk1 mappingq It checks data for Mandatory Fields and generates a error fileq If no errors during running of this mapping, we can FTP the file to the central team, else analyse the
errors and fix them.q The source and the Target structure is same in this mapping
• Central Chk Mappings q These represent the reusable set of object mappings. They require minimal set of changes to
include new countries• KDS
q It refers to Key Data structure values which are configured inside SAP for each country/site• Localization Matrix
q It refers to country specific exceptions• Site Team
q This is the team of technical and business data owners who focuses on extracting site data, site level checks and reconciliations and delivery it further
• Central Teamq This is the technical and Functional designers who ensures implementation of all site level, project
phase level and SAP level checks and reconciliations