Agile Oracle to PostgreSQL migrations (PGConf.EU 2013)

Higher impact of license costs

Agile Oracle to PostgreSQL migrations

PGConf.EU 2013, Dublin, Nov 1st

Gabriele Bartolini

2ndQuadrant Italia / PostgreSQL Europegabriele.bartolini@2ndQuadrant.it

@_GBartolini_

Gabriele Bartolini•Co-Founder and Manager of 2ndQuadrant Italia

• Data Architect, Business critical environments

• Data warehousing

•Co-Founder Italian PostgreSQL Users Group

•Co-Founder PostgreSQL Europe

•PostgreSQL Contributor and Advocate

Outline• Introduction

•The migration project

•Open source tools

•Conclusions

Target•Chief Technical Officers

•Chief Information Officers

•Oracle users

• Evaluate moving to PostgreSQL 9.3

Goals•Describe the migration process

•Spot the major sources of costs

•Help you plan a migration in the mid/long term

Part IIntroduction

Oracle vs Postgres

(*) not sure it is in the TODO List, yet

Upgrade automaticallyWould you ever do an upgrade to a higher version of

Oracle in a completely automated way?

PostgreSQL•Hot Standby (~ Data Guard)

•Barman (~ RMAN)

•Londiste/PgQ (GoldenGate)

•PostGIS (Spatial)

•SE Linux integration (~ Database Vault)

•Memcache integration (~ In-Memory Database Cache)

•Partitioning (*)

•Future: Multi-master replication (~ RAC)

License•Very simple and clear

•Applies to every part (module, product, option) of Postgres

• server, client, libraries, additional modules

• 100% open source

•No need to hire a lawyer to understand it

•Highly Permissive

• Not subject to monopoly nor acquisitions

•Fosters market competition in professional services offer

Oracle DBA, do not be afraid!•Transferrable knowledge

•PostgreSQL focused professional training

•100% open source

Part IIThe Migration project

Migration = projectbudget, scope, quality, time

Budget•Crucial variable

• Usually triggers the migration process

•Cost analysis drives the migration project

• short term

• mid term

Quality•PostgreSQL reliability

•Tests

Components•Applications

•Database

•Processes

•Human resources

Migration cost components

25,00% 50,00%

Dev Testing Processes Training Licensing

fictit

ious

dat

a

Comparison with Oracle upgrade

25,00% 50,00%

Dev Testing Processes Training Licensing

fictit

ious

dat

a

External applications•Force your supplier to provide Postgres support for their

application

• You set the rules

•Typical case of the public sector

• e.g.: Land registry

• Financial crisis and competition are on your side

Agile approach• Requirements change

• Every context is different from the other

• No one-size-fits-all scenario

• Be dynamic:

• SCRUM (with the customer)

• Kanban (internally)

• DevOps

Focus•Applications

•Database

•Processes

•Human resources

Database migration•Schema

•Queries

•Data

•Stored procedures

•Tests

Schema conversion•Tables

• Data type mapping

• Indexes

•Views

•Constraints

Schema major differences•USER = schema/database

•UPPERCASE = lowercase

•Partitioning (*)

•Tablespaces

•SYNONYMs

• search_path / auto updatable views / foreign tables

Queries•ANSI/ISO SQL

•PostgreSQL has no hints (optimiser)

•Major differences:

• NULL = NULL

• Postgres: NULL

• Oracle: TRUE

• OUTER JOINs

• (+)

• use LEFT/RIGHT JOIN

• DUAL

• ROWNUM and ROWID

• ...

Data•Might require specialised

consulting activities

•Major solutions:

• ETL tools (e.g. Kettle)

• External scripts (e.g. ora2pg or custom)

• COPY t FROM stdin;

• Foreign tables with Oracle FDW

• INSERT INTO t SELECT * FROM ft;

• Cut-over time

• > 0

• ~ 0

Stored Procedures•Most critical and complex component of the whole migration

•Package = extension (schema)

•Two major ways:

• 1:1 conversion (not always possible)

• complete rewriting (optimised for PostgreSQL)

• It might required specialised consulting activities

PL/SQL vs PL/pgSQL•Syntactical differences

•PROCEDURE = FUNCTION that returns VOID

•TRIGGER = TRIGGER FUNCTION + TRIGGER

•Lack of autonomous transactions

•Lack of global variables

• ...

Test•Vital component for a successful migration process

• It measures the QUALITY of the project

• It can be reused in the future for PostgreSQL upgrades

• strategic investment in the long period

Part IIIOpen source tools

ora2pg•URL: http://ora2pg.darold.net/

•Written in Perl - GNU GPL 3

•Reads the information schema from Oracle

•Generates DDL instructions for PostgreSQL

•Highly configurable (mapping data type conversions)

•Basic support for PL/SQL (regexp)

orafce•URL: http://orafce.projects.pgfoundry.org/

•C and SQL - BSD licensed

•Set of functions, objects and modules that are part of Oracle:

• NVL function (COALESCE)

• DBMS_ALERT, DBMS_OUTPUT, UTL_FILE, ...

• DUAL

Oracle FDW•URL: http://oracle-fdw.projects.pgfoundry.org/

•SQL/MED

• uses the API for Foreign Data Wrapper

• CREATE FOREIGN TABLE

• Currently READ ONLY

•Useful for ETL

Example of Oracle FDW...CREATE FOREIGN TABLE oracle_table ( id integer NOT NULL, ...) SERVER oradb OPTIONS (schema 'ORAUSER', table 'ORATAB');...-- pushes down the WHERE clauseSELECT * FROM oracle_table WHERE id=10;

PgTap•Unit tests suite for PostgreSQL

•Extensions (functions)

•Produces TAP output

•Test Driven Development in databases

Vagrant•Open Source software

•Creation and configuration of virtual environments

•Provisioning: Puppet, Chef

•Useful for local testing

PostgreSQL•Every major release of Postgres reduces the technology gap

with Oracle

•5 years ago, Postgres did not have hot standby, streaming/synchronous/cascading replication, window functions, CTEs, column permissions, foreign tables, serialisable isolation level, XML, JSON, updatable and materialised views ...

• Is it called “OraGIS” or “PostGIS”?

•Postgres does have transactional DDL

Master Standby

App

Business Continuity stack

DR

server C

repmgr

Barmanserver A server B

If you really miss it ...CREATE USER scott WITH PASSWORD ‘tiger’;CREATE DATABASE scott WITH OWNER scott;\c scott scottCREATE SCHEMA scott;

Part IVConclusions

0

30.000

60.000

90.000

120.000

150.000

180.000

210.000

240.000

270.000

300.000

Costs in Euro

35.000

70.000

140.000

16.000

32.000

64.000

7.5001 CPU Socket 2 CPU Socket 4 CPU Socket PostgreSQL*

License Support

Cost analysis of a database solution on a period of 3 years for 1 server

* PostgreSQL with 24/7 support from us

015.00030.00045.00060.00075.00090.000

105.000120.000135.000150.000

Costs in euro70.000

16.000 16.000

50.000

2.500 2.500

Anno 1 Anno 2 Anno 3

Oracle PostgreSQL

Cost analysis of a migration to PostgreSQL in a 3 year period

* PostgreSQL with 24/7 support from us

Conclusions•Final solution: 100% open

source

• reduced TCO

• No vendor lock-in

•Migration must be focused on costs evaluation

•Supervised process (not automated)

• If a feature is missing, you can simply sponsor it

• If needed, professional companies can help you before, during and after the migration process (you won’t be alone)

QuestionsE-mail: gabriele.bartolini@2ndQuadrant.it

Twitter: @_GBartolini_

Thank you!Licenza Creative Commons BY-NC-SA 3.0

http://creativecommons.org/licenses/by-nc-sa/3.0/it/deed.it