Upload
enterprisedb
View
1.449
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Postgres has been proven to process document database workloads faster than MongoDB in benchmark testing. But there are multiple benefits to using Postgres over a specialized solution for such applications. Application developers are finding new ways to combine schema-less data with traditional relational tables and deliver innovative applications faster while meeting evolving DevOps strategies and goals. Providing a single, ACID-compliant enterprise-ready database that can manage both structured and unstructured data supports the development process and reduces overall complexity. Learn what Postgres can help you achieve. This covers: -- Using JSON/JSONB and HSTORE to combine schema-less data with enterprise information -- Build on existing skillsets while using web 2.0 development technologies -- Reduce complexity that comes with using multiple heterogeneous platform and disparate application demands -- New performance benchmark results showing Postgres outperforms MongdoDB
Citation preview
© 2014 EnterpriseDB Corporation. All rights reserved. 1
Postgres NoSQL - Delivering Applications Faster Marc Linster – SVP, Products and Services
© 2014 EnterpriseDB Corporation. All rights reserved. 2
• EDB Introduction
• BigData and NoSQL: Volume, Variety and Velocity
• Postgres – NoSQL for the Enterprise • Performance Evaluation – Phase 2
• FDW – Coexistence of NoSQL only and SQL
• Leveraging a proven skill set to create Web 2.0 applications
• Postgres as a NoSQL/SQL Integration Platform
Agenda
© 2014 EnterpriseDB Corporation. All rights reserved. 3
POSTGRES innovation
ENTERPRISE reliability
24/7 support
Services & training
Enterprise-class features & tools
Indemnification
Product road-map
Control
Thousands of developers
Fast development
cycles
Low cost
No vendor lock-in
Advanced features
Enabling commercial adoption of Postgres
© 2014 EnterpriseDB Corporation. All rights reserved. 4
Postgres Plus Advanced Server Postgres Plus
Cloud Database
High Availability Performance Management
REMOTE DBA 24x7
SUPPORT PROFESSIONAL
SERVICES
TRAINING
EDB Serves All Your Postgres Needs
PostgreSQL
Security
© 2014 EnterpriseDB Corporation. All rights reserved. 5
• Common Myth (or misconception) − Volume, Velocity, Variety and ACID are not compatible! − You cannot have it all!
• Their (incorrect) reasoning: − Very High Data Volumes (log analysis, append only applications,
social media) applications cannot be handled by relational systems − The data is sent to the application at high speeds and has to be
ingested quickly – not in batch mode − Web 2.0 applications require more than SQL – they also need
documents, IMs, emails, graphs, links − ACID (Atomicity, Consistency, Isolation and Durability) compliant
systems are too inflexible, too slow and cannot handle the new data types
BigData, NoSQL and SQL: Myths about Volume, Variety, Velocity and ACID
© 2014 EnterpriseDB Corporation. All rights reserved. 6
$$$$$$
ACID, Volume, Velocity and Variety in Context ACID Transactional
Ingestion Velocity
Variety
Data Volume
Postgres
Data Mining/BigData
Document/NoSQL-Only
© 2014 EnterpriseDB Corporation. All rights reserved. 7
• Performance Benchmarks − Postgres is very fast and can handle huge
amounts of data − Postgres can selectively relax key ACID features
to increase performance
• Data Types − Postgres has JSON, JSONB, Key-Value Pair,
plus arrays, ranges, timezones, dates, integer, floating point, etc.
• Proven track record − ACID compliant − Open source − ANSI SQL − Huge developer and vendor community
Postgres – NoSQL for the Enterprise
© 2014 EnterpriseDB Corporation. All rights reserved. 8
• HSTORE − Key-value pair − Simple, fast and easy − Postgres v 8.2 – pre-dates many
NoSQL-only solutions
• JSON − Hierarchical document model − Introduced in Postgres 9.2, perfected in 9.3
• JSONB − Binary version of JSON − Faster, more operators and even more robust − Postgres 9.4
NoSQL Data in Postgres
© 2014 EnterpriseDB Corporation. All rights reserved. 9
• Creating a table with a JSONB field CREATE TABLE json_data (data JSONB);!
• Simple JSON data element: {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1}!
• Inserting this data element into the table json_data INSERT INTO json_data (data) VALUES !
!(’ { !"name": "Apple Phone", !! !"type": "phone", !! !"brand": "ACME", !! !"price": 200, !! !"available": true, !! !"warranty_years": 1 ! !!!} ')!
JSON Examples
© 2014 EnterpriseDB Corporation. All rights reserved. 10
SELECT DISTINCT !!data->>'name' as products !
FROM json_data; ! products !------------------------------! Cable TV Basic Service Package! AC3 Case Black! Phone Service Basic Plan! AC3 Phone! AC3 Case Green! Phone Service Family Plan! AC3 Case Red! AC7 Phone!
A simple query for JSON data
This query does not return JSON data – it returns text values associated with the key ‘name’
© 2014 EnterpriseDB Corporation. All rights reserved. 11
SELECT data FROM json_data;!
data !
------------------------------------------!
{"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1}!
A query that returns JSON data
This query returns the JSON data in its original format
© 2014 EnterpriseDB Corporation. All rights reserved. 12
• JSON is naturally integrated with ANSI SQL in Postgres
• JSON and HSTORE are elegant and easy to use extensions of the underlying object-relational model
• JSON and SQL queries use the same language, the same planner, and the same ACID compliant transaction framework
JSON and ANSI SQL – A Great Fit
© 2014 EnterpriseDB Corporation. All rights reserved. 13
SELECT DISTINCT product_type, data->>'brand' as Brand,
data->>'available' as Availability FROM json_data JOIN products ON (products.product_type=json_data.data->>'name') WHERE json_data.data->>'available'=true; product_type | brand | availability ---------------------------+-----------+-------------- AC3 Phone | ACME | true
JSON and ANSI SQL Example
ANSI SQL
JSON
No need for programmatic logic to combine SQL and NoSQL in the application – Postgres does it all
© 2014 EnterpriseDB Corporation. All rights reserved. 14
Bridging between SQL and NoSQL Simple ANSI SQL Table Definition
CREATE TABLE products (id integer, product_name text );! Select query returning standard data set
SELECT * FROM products; ! id | product_name !----+--------------! 1 | iPhone! 2 | Samsung! 3 | Nokia!
Select query returning the same result as a JSON data set
SELECT ROW_TO_JSON(products) FROM products; {"id":1,"product_name":"iPhone"} {"id":2,"product_name":"Samsung"} {"id":3,"product_name":"Nokia”}
© 2014 EnterpriseDB Corporation. All rights reserved. 15
• Start unstructured, and become structured as you learn more − Use the quick-to-get-started capabilities of NoSQL − Complete the initial sprints without a DBA − Move data between unstructured and structured − Embrace corporate data standards as you move
from the stand-alone application towards integrated applications with a bigger value proposition
Postgres Provides Great Flexibility
By 2017, 50% of data stored in NoSQL DBMSs will be damaging to the business due to lack of applied information governance policies and programs. Gartner, December 2013
© 2014 EnterpriseDB Corporation. All rights reserved. 16
• Goal − Help our customers understand when to chose Postgres and
when to chose a specialty solution − Help us understand where the NoSQL limits of Postgres are
• Setup − Compare Postgres 9.4 to Mongo 2.6 − Single instance setup on AWS M3.2XLARGE (32GB)
• Test Focus − Data ingestion (bulk and individual) − Data retrieval
Postgres NoSQL Performance Evaluation
Load Generator
Postgres 9.4
MongoDB 2.6
© 2014 EnterpriseDB Corporation. All rights reserved. 17
Postgres NoSQL Performance Evaluation Generate JSON Documents
10 M documents & 50 M documents
Load into MongoDB 2.6 (IMPORT)
Load into Postgres 9.4
(COPY)
10 Million individual INSERT commands
10 Million individual INSERT commands
Multiple SELECT statements
Multiple SELECT statements
T1
T2
T3
© 2014 EnterpriseDB Corporation. All rights reserved. 18
Test 1: 10 Million Documents
Test Criterion MongoDB 2.6 Postgres 9.4
Data load (s) 2,624 1,193
Inserts (s) 16,491 5,637
Selects (s) 187 67
DB Size (GB) 20.05 14.89
0%
50%
100%
150%
200%
250%
300%
350%
Data load (s) Inserts (s) Selects (s) DB Size (GB)
Mongo DB 2.4/Postgres 9.4 Rela9ve Performance Comparison (10M Documents)
MongoDB 2.6
Postgres 9.4
© 2014 EnterpriseDB Corporation. All rights reserved. 19
Test 2: 50 Million Documents
Test Criterion MongoDB 2.6 Postgres 9.4
Data load (s) 15,391 7,319
Inserts (s) 85,639 29,125
Selects (s) 1,929 753
DB Size (GB) 92.63 69.36
0%
50%
100%
150%
200%
250%
300%
350%
Data load (s) Inserts (s) Selects (s) DB Size (GB)
Mongo DB 2.4/Postgres 9.4 Rela9ve Performance Comparison (50M Documents)
MongoDB 2.6
Postgres 9.4
© 2014 EnterpriseDB Corporation. All rights reserved. 20
• Tests confirm that Postgres can handle many NoSQL workloads
• EDB is making the test scripts publically available
• EDB encourages community participation to better define where Postgres should be used and where specialty solutions are appropriate
• Download the source at https://github.com/EnterpriseDB/pg_nosql_benchmark
• Join us to discuss the findings at http://bit.ly/EDB-NoSQL-Postgres-Benchmark
Postgres NoSQL Performance Evaluation
© 2014 EnterpriseDB Corporation. All rights reserved. 21
• FDW implements SQL/MED ("SQL Management of External Data")
• PostgreSQL 9.1 - read-only support
• PostgreSQL 9.3 – read/write support • FDW
− Makes data on other servers (or services) look like tables in Postgres
− available for databases (MongoDB, MySQL, Oracle, …), files, services (Twitter, …)
• MongoDB FDW: https://github.com/EnterpriseDB
Foreign Data Wrappers – Co-Existence Platform
© 2014 EnterpriseDB Corporation. All rights reserved. 22
CREATE EXTENSION mongo_fdw;!
CREATE SERVER mongo_server !FOREIGN DATA WRAPPER mongo_fdw !OPTIONS (address '172.24.39.129', port '27017');!
CREATE USER MAPPING FOR enterprisedb !SERVER mongo_server !OPTIONS (username 'mongo', password 'mongo');!
CREATE FOREIGN TABLE mongo_data( !name text, !brand text, !type text) !SERVER mongo_server !OPTIONS ( ! !database 'benchmark', ! !collection 'json_tables');!
MongoDB FDW Example
© 2014 EnterpriseDB Corporation. All rights reserved. 23
SELECT * FROM mongo_data WHERE brand='ACME' limit 10;!
!
name | brand | type -------------+-------+------- AC7553 Phone | ACME | phone AC7551 Phone | ACME | phone AC7519 Phone | ACME | phone AC7565 Phone | ACME | phone AC7555 Phone | ACME | phone AC7529 Phone | ACME | phone AC7528 Phone | ACME | phone AC7547 Phone | ACME | phone AC7587 Phone | ACME | phone AC7541 Phone | ACME | phone!
(10 rows)!
!
MongoDB FDW Example
© 2014 EnterpriseDB Corporation. All rights reserved. 24
INSERT INTO mongo_data(name, brand, type) !VALUES('iphone6 phone','Apple Inc','phone');!
SELECT* FROM mongo_data WHERE brand='Apple Inc';!
_id | name | brand | type --------------------------+----------------+-----------+------- 53ea4f59fe5586a15714881d | iphone6 phone | Apple Inc | phone!
!
UPDATE mongo_data SET brand='Apple Product' !WHERE brand='Apple Inc’;!
SELECT * FROM mongo_data WHERE brand='Apple Product’;!
_id | name | brand | type --------------------------+----------------+---------------+------- 53ea4f59fe5586a15714881d | iphone6 phone | Apple Product | phone!
!
MongoDB FDW Example
© 2014 EnterpriseDB Corporation. All rights reserved. 25
• Postgres allows you to − Use the same Infrastructure DBAs (no change!!!)
− Use the same management, maintenance, backup, restore, high availability, disaster recovery and compliance processes for SQL and NoSQL
− Start unstructured and align with corporate governance standards as your project matures
− Start unstructured (data logic in the application) and migrate shared data logic into the database è enhances reuse
− Build on the existing PL/pgSQL programming skillset
− EDB Postgres Plus adds Oracle PL/SQL compatibility (fully integrated with JSON and HSTORE)
Leveraging a Proven Skill Set
© 2014 EnterpriseDB Corporation. All rights reserved. 26
• Postgres has many NoSQL features without the drawbacks: − Schema-less data types, with sophisticated indexing
support − Schema changes with rapid addition and removal of
keys and tags − Durable by default, but configurable per-table or
per-transaction − Foreign Data Wrappers (FDW) support co-existence
and tight integration − Standards based with very low technology risk − Highly available skill set
Postgres: The Best of Both Worlds
© 2014 EnterpriseDB Corporation. All rights reserved. 27
• Structures and standards emerge! • Data has references (products link to catalogues;
products have bills of material; components appear in multiple products; storage locations link to ISO country tables)
• When the database has duplicate data entries, then the application has to manage updates in multiple places – what happens when there is no ACID transactional model?
“No SQL Only” or “Not Only SQL”?
© 2014 EnterpriseDB Corporation. All rights reserved. 28
• Postgres is Not Only SQL (NoSQL is No SQL only) • Fully ACID compliant • Proven track record
• Fully capable of handling the variety, velocity and volume requirements of most applications
• Tackle NoSQL projects without leaving the capabilities of the relational model behind you
Postgres is the best of both worlds, and it’s open source!
Say ‘Yes’ to ‘Not Only SQL’
© 2014 EnterpriseDB Corporation. All rights reserved. 29
• Contact us for an Enterprise NoSQL Assessment ([email protected])
• Review EDB’s NoSQL Resources − Whitepaper -
http://www.enterprisedb.com/wp-using-nosql-features-postgres − NoSQL Solution Information -
http://www.enterprisedb.com/nosql-for-enterprise − Benchmarks- https://github.com/EnterpriseDB
© 2014 EnterpriseDB Corporation. All rights reserved. 30
© 2014 EnterpriseDB Corporation. All rights reserved. 31