Upload
gabriele-bartolini
View
2.353
Download
5
Embed Size (px)
DESCRIPTION
Migrating an Oracle database to Postgres is never an automated operation. And it rarely (never?) involve just the database. Experience brought us to develop an agile methodology for the migration process, involving schema migration, data import, migration of procedures and queries up to the generation of unit tests for QA. Pitfalls, technologies and main migration opportunities will be outlined, focusing on the reduction of total costs of ownership and management of a database solution in the middle-long term (without reducing quality and business continuity requirements).
Citation preview
Higher impact of license costs
Agile Oracle to PostgreSQL migrations
PGConf.EU 2013, Dublin, Nov 1st
Gabriele Bartolini
2ndQuadrant Italia / PostgreSQL [email protected]
@_GBartolini_
Gabriele Bartolini•Co-Founder and Manager of 2ndQuadrant Italia
• Data Architect, Business critical environments
• Data warehousing
•Co-Founder Italian PostgreSQL Users Group
•Co-Founder PostgreSQL Europe
•PostgreSQL Contributor and Advocate
Outline• Introduction
•The migration project
•Open source tools
•Conclusions
Target•Chief Technical Officers
•Chief Information Officers
•Oracle users
• Evaluate moving to PostgreSQL 9.3
Goals•Describe the migration process
•Spot the major sources of costs
•Help you plan a migration in the mid/long term
Part IIntroduction
Oracle vs Postgres
(*) not sure it is in the TODO List, yet
Upgrade automaticallyWould you ever do an upgrade to a higher version of
Oracle in a completely automated way?
PostgreSQL•Hot Standby (~ Data Guard)
•Barman (~ RMAN)
•Londiste/PgQ (GoldenGate)
•PostGIS (Spatial)
•SE Linux integration (~ Database Vault)
•Memcache integration (~ In-Memory Database Cache)
•Partitioning (*)
•Future: Multi-master replication (~ RAC)
License•Very simple and clear
•Applies to every part (module, product, option) of Postgres
• server, client, libraries, additional modules
• 100% open source
•No need to hire a lawyer to understand it
•Highly Permissive
• Not subject to monopoly nor acquisitions
•Fosters market competition in professional services offer
Oracle DBA, do not be afraid!•Transferrable knowledge
•PostgreSQL focused professional training
•100% open source
Part IIThe Migration project
Migration = projectbudget, scope, quality, time
Budget•Crucial variable
• Usually triggers the migration process
•Cost analysis drives the migration project
• short term
• mid term
Quality•PostgreSQL reliability
•Tests
Components•Applications
•Database
•Processes
•Human resources
Migration cost components
25,00% 50,00%
Dev Testing Processes Training Licensing
fictit
ious
dat
a
Comparison with Oracle upgrade
25,00% 50,00%
Dev Testing Processes Training Licensing
fictit
ious
dat
a
External applications•Force your supplier to provide Postgres support for their
application
• You set the rules
•Typical case of the public sector
• e.g.: Land registry
• Financial crisis and competition are on your side
Agile approach• Requirements change
• Every context is different from the other
• No one-size-fits-all scenario
• Be dynamic:
• SCRUM (with the customer)
• Kanban (internally)
• DevOps
Focus•Applications
•Database
•Processes
•Human resources
Database migration•Schema
•Queries
•Data
•Stored procedures
•Tests
Schema conversion•Tables
• Data type mapping
• Indexes
•Views
•Constraints
Schema major differences•USER = schema/database
•UPPERCASE = lowercase
•Partitioning (*)
•Tablespaces
•SYNONYMs
• search_path / auto updatable views / foreign tables
Queries•ANSI/ISO SQL
•PostgreSQL has no hints (optimiser)
•Major differences:
• NULL = NULL
• Postgres: NULL
• Oracle: TRUE
• OUTER JOINs
• (+)
• use LEFT/RIGHT JOIN
• DUAL
• ROWNUM and ROWID
• ...
Data•Might require specialised
consulting activities
•Major solutions:
• ETL tools (e.g. Kettle)
• External scripts (e.g. ora2pg or custom)
• COPY t FROM stdin;
• Foreign tables with Oracle FDW
• INSERT INTO t SELECT * FROM ft;
• Cut-over time
• > 0
• ~ 0
Stored Procedures•Most critical and complex component of the whole migration
•Package = extension (schema)
•Two major ways:
• 1:1 conversion (not always possible)
• complete rewriting (optimised for PostgreSQL)
• It might required specialised consulting activities
PL/SQL vs PL/pgSQL•Syntactical differences
•PROCEDURE = FUNCTION that returns VOID
•TRIGGER = TRIGGER FUNCTION + TRIGGER
•Lack of autonomous transactions
•Lack of global variables
• ...
Test•Vital component for a successful migration process
• It measures the QUALITY of the project
• It can be reused in the future for PostgreSQL upgrades
• strategic investment in the long period
Part IIIOpen source tools
ora2pg•URL: http://ora2pg.darold.net/
•Written in Perl - GNU GPL 3
•Reads the information schema from Oracle
•Generates DDL instructions for PostgreSQL
•Highly configurable (mapping data type conversions)
•Basic support for PL/SQL (regexp)
orafce•URL: http://orafce.projects.pgfoundry.org/
•C and SQL - BSD licensed
•Set of functions, objects and modules that are part of Oracle:
• NVL function (COALESCE)
• DBMS_ALERT, DBMS_OUTPUT, UTL_FILE, ...
• DUAL
Oracle FDW•URL: http://oracle-fdw.projects.pgfoundry.org/
•SQL/MED
• uses the API for Foreign Data Wrapper
• CREATE FOREIGN TABLE
• Currently READ ONLY
•Useful for ETL
Example of Oracle FDW...CREATE FOREIGN TABLE oracle_table ( id integer NOT NULL, ...) SERVER oradb OPTIONS (schema 'ORAUSER', table 'ORATAB');...-- pushes down the WHERE clauseSELECT * FROM oracle_table WHERE id=10;
PgTap•Unit tests suite for PostgreSQL
•Extensions (functions)
•Produces TAP output
•Test Driven Development in databases
Vagrant•Open Source software
•Creation and configuration of virtual environments
•Provisioning: Puppet, Chef
•Useful for local testing
PostgreSQL•Every major release of Postgres reduces the technology gap
with Oracle
•5 years ago, Postgres did not have hot standby, streaming/synchronous/cascading replication, window functions, CTEs, column permissions, foreign tables, serialisable isolation level, XML, JSON, updatable and materialised views ...
• Is it called “OraGIS” or “PostGIS”?
•Postgres does have transactional DDL
Master Standby
App
Business Continuity stack
DR
server C
repmgr
Barmanserver A server B
If you really miss it ...CREATE USER scott WITH PASSWORD ‘tiger’;CREATE DATABASE scott WITH OWNER scott;\c scott scottCREATE SCHEMA scott;
Part IVConclusions
0
30.000
60.000
90.000
120.000
150.000
180.000
210.000
240.000
270.000
300.000
Costs in Euro
35.000
70.000
140.000
16.000
32.000
64.000
7.5001 CPU Socket 2 CPU Socket 4 CPU Socket PostgreSQL*
License Support
Cost analysis of a database solution on a period of 3 years for 1 server
* PostgreSQL with 24/7 support from us
015.00030.00045.00060.00075.00090.000
105.000120.000135.000150.000
Costs in euro70.000
16.000 16.000
50.000
2.500 2.500
Anno 1 Anno 2 Anno 3
Oracle PostgreSQL
Cost analysis of a migration to PostgreSQL in a 3 year period
* PostgreSQL with 24/7 support from us
Conclusions•Final solution: 100% open
source
• reduced TCO
• No vendor lock-in
•Migration must be focused on costs evaluation
•Supervised process (not automated)
• If a feature is missing, you can simply sponsor it
• If needed, professional companies can help you before, during and after the migration process (you won’t be alone)
QuestionsE-mail: [email protected]
Twitter: @_GBartolini_
Thank you!Licenza Creative Commons BY-NC-SA 3.0
http://creativecommons.org/licenses/by-nc-sa/3.0/it/deed.it