Upload
selena-deckelmann
View
11.745
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
Some Conference
OSCO
N 2011
PostgreSQL 9.1Selena Deckelmann
PostgreSQL Global Development Group
Stronger
Harder Better
Faster
OSCO
N 2011
Some Conference
OSCO
N 2011
Say “Postgres.”
Some Conference
OSCO
N 2011
Some Conference
OSCO
N 2011
“World’s most advanced open source database.”
Some Conference
OSCO
N 2011
Some Conference
OSCO
N 2011
Some Conference
OSCO
N 2011
Some Conference
OSCO
N 2011
Postgres disrupts the commercial relational
database industry.
Some Conference
OSCO
N 2011
Licensing
“Enterprise” Web development
Proprietary DBA Career Path
Disruptive forces
Some Conference
OSCO
N 2011
Licensing
Drop-in replacement for custom application development.
Cheap Data Warehousing
Ecosystem!
Some Conference
OSCO
N 2011
“Enterprise”Web development
Postgres is a “Developers database” - Extension is encouraged!
Database of choice for Django
Oracle wants PHP developers
Some Conference
OSCO
N 2011
“Visit the Ruby and Python Topicfor more information pertainingto Ruby/RoR, Python and other
non-PHP scripting languages.”
Some Conference
OSCO
N 2011
DBA Career path
Companies can’t hire Postgres DBAs fast enough.
Postgres is close enough to Oracle, that Oracle DBAs can jump in.
What we do better: Data loading, automation, TUI and GUI admin tools
Some Conference
OSCO
N 2011
And... Stability
Support major releases for at least 5 years.
Commits to HEAD are stable.
Bugs reported are fixed in <48 hrs.
Some Conference
OSCO
N 2011
Stronger
Harder Better
Faster
More secure & robust:SE PostgresReplication Tools
We made us better: Writeable CTEPer-column collationsExtensions
Performance:KNN IndexesUnlogged Tables
First in class:Serializable Snapshot IsolationForeign Data WrappersSynchronous Replication
10
features
Some Conference
OSCO
N 2011
Harder
Some Conference
OSCO
N 2011
Replication Tools
Harder
WAL-based replication:File-based (default 16-MB files)Streaming (per-WAL record) (9.0)AsynchronousWarm standbyHot standby (9.0)Synchronous (9.1)Cascading replication (just committed... 9.2)
http://www.pgcon.org/2010/schedule/attachments/149_PGCon2010-Built-in-Replication.pdf
new!
new!
Some Conference
OSCO
N 2011
Harder Replication Tools
pg_stat_replication viewshows all replicas and their statuses
pg_basebackupsingle-command cloning over port 5432 http://www.postgresql.org/docs/9.1/static/app-pgbasebackup.html
promote to masterpg_ctl promoteformerly only could do this by creating a trigger file
new functions pg_last_xact_replay_timestamp() pg_xlog_replay_pause()
Some Conference
OSCO
N 2011
HarderSE-Postgres
SE-Linux security label support!Originally a NSA-funded Linux kernel extension
For tablespace, schema, table, column, sequence, view and procedure object classes
Will have integrated support in pgAdmin!http://rhaas.blogspot.com/2010/09/se-linux-for-postgresql-part-1.html
http://rhaas.blogspot.com/2010/09/se-linux-for-postgresql-part-2.html
http://wiki.postgresql.org/wiki/SEPostgreSQL_Documentation
Some Conference
OSCO
N 2011
BetterWriteable CTE
But first - What’s a CTE? Common Table Expressions A temporary table or VIEW that exists for just a single query.
http://developer.postgresql.org/pgdocs/postgres/queries-with.html#QUERIES-WITH-MODIFYING
Some Conference
OSCO
N 2011
Better
Some Conference
OSCO
N 2011
Better Common TableExpression Example
WITH RECURSIVE t1(a, b) AS ( VALUES(0,0) UNION ALL SELECT CASE CAST(b AS BOOLEAN) WHEN b % 3 = 0 THEN b WHEN b % 5 = 0 THEN b END, b + 1 FROM t1 WHERE b < 1000)SELECT sum(a) FROM t1;
Euler Project, question 1: If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23. Find the sum of all the multiples of 3 or 5 below 1000.
http://wiki.postgresql.org/wiki/Euler_Project,_Question_1
Some Conference
OSCO
N 2011
BetterWriteable CTE
WITH deleted_posts AS (
DELETE FROM posts WHERE created < now()
- '6 months'::INTERVAL RETURNING * )SELECT user_id, count(*) FROM deleted_posts group BY 1;
Makes maintaining partitions easier:
http://www.depesz.com/index.php/2011/03/16/waiting-for-9-1-writable-cte/
http://xzilla.net/blog/2011/Mar/Upserting-via-Writeable-CTE.html
Some Conference
OSCO
N 2011
Better Per-columncollation
Supports multiple languages in the same table, and have ORDER BY sort by language order (assuming you’ve loaded the correct collation library)BEFORE:
$ select * from normal_polish order by some_text; some_text----------- alfa car ćma coś lama
Some Conference
OSCO
N 2011
Better Per-columncollation
AFTER:
$ select * from normal_polish order by some_text; some_text----------- alfa car coś ćma lama
Some Conference
OSCO
N 2011
BetterExtensions
Postgres-specific package management for contrib/, PgFoundry projects, tools.
Like Oracle "packages"
End user view: http://facility9.com/2011/03/postgresql-extensions/
Docs: http://developer.postgresql.org/pgdocs/postgres/sql-createextension.html
Some Conference
OSCO
N 2011
Faster
Some Conference
OSCO
N 2011
Faster K-Nearest Neighbor Indexes
Incredibly fast search for things that are "near" a particular object
Operator is: <->Datatypes supported: pg_trgm contrib and btree_gist datatypes (point)
Wiki: http://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9.1#K-Nearest-Neighbor_IndexingDeveloper: http://www.sai.msu.su/~megera/postgres/talks/pgday-2010.pdf
Some Conference
OSCO
N 2011
Faster K-Nearest Neighbor Indexes
$ create table test ( position point );$ insert into test (position) select point( random() * 1000, random() * 1000) from generate_series(1,1000000);
$ create index q on test using gist ( position );$ select *, position <-> point(500,500) from test order by position <-> point(500,500) limit 10;
$ explain analyze select *, position <-> point(500,500) FROM test ORDER BY position <-> point(500,500) limit 10;
QUERY PLAN Limit (cost=0.00..0.77 rows=10 width=16) (actual time=0.164..0.475 rows=10 loops=1) -> Index Scan using q on test (cost=0.00..76512.60 rows=1000000 width=16) (actual time=0.163..0.473 rows=10 loops=1) Order By: ("position" <-> '(500,500)'::point)
Some Conference
OSCO
N 2011
FasterUnlogged Tables
Background: every write to a table in Postgres is essentially *two* writes because of the Write Ahead Log (basis for replication, and durability guarantees)
Unlogged tables are FREED of this durability constraint!
Could think of them as Global Temporary tables (see Haas' post)
http://www.depesz.com/index.php/2011/01/03/waiting-for-9-1-unlogged-tables/http://rhaas.blogspot.com/2010/05/global-temporary-and-unlogged-tables.html
Example:=$ perl -le 'my $x = "aaaaaaaa"; for (1..450000) {print $x++}' | \ perl -pe 's/.*/(\047$&\047)/;s/\s*\z/,/ if $.%3' | \ sed 's/.*/insert into test (some_text) values &;/'
Some Conference
OSCO
N 2011
Stronger Serializable Snapshot Isolation
PostgreSQL's is the first production implementationIncludes: per transaction support
Why use it?* No more blocking on table locks!* No more "select for update"* Simplifies programming
Wiki: http://wiki.postgresql.org/wiki/SSIDeveloper:My notes from Kevin's version of the talk: http://www.chesnok.com/daily/2011/03/24/raw-notes-from-kevin-grittners-talk-on-ssi/
Some Conference
OSCO
N 2011
Stronger
Some Conference
OSCO
N 2011
Stronger Foreign Data Wrappers
* Remote datasource access* Initially implemented TEXT, CSV data sources* Twitter: http://pgxn.org/dist/twitter_fdw/* Google Summer of Code student is working on other data sources* Underway currently: Oracle and MySQL sources
Awesome because:* 6-function API for implementing* Good for implementing imports, things which would otherwise fail if you just used COPY (imporoving COPY is a whole other rant...)
Stronger
Some Conference
OSCO
N 2011
Stronger Foreign Data Wrappers
Not awesome because:* Nothing other than sequential scans are possible for query plans* Not that many data sources implemented yet
Expect tons of FDWs to be implemented once we get 9.1 to production release
Stronger
Some Conference
OSCO
N 2011
Stronger Syncronous Replication
This is *the* most important use case for WAL-level replication.
User:http://developer.postgresql.org/pgdocs/postgres/warm-standby.html#SYNCHRONOUS-REPLICATION(kind of terrible for a person trying to understand how it works)
Somewhat better:http://wiki.postgresql.org/wiki/What%27s_new_in_PostgreSQL_9.1#Synchronous_replication_and_other_replication_features
Wiki: http://wiki.postgresql.org/wiki/Synchronous_replication
Stronger
Some Conference
OSCO
N 2011
Stronger Syncronous Replication
WAL stands for "write ahead log" and it is what ensures durability for all write operations
* WAL-shipping 16-MB chunks of these records was the pre-9.0 method for built-in replication* 9.0 and later have Streaming rep - per WAL record (per-commit) shipping of changes* Logical changes to DB, NOT statement-based* All-or-nothing wrt databases and tables
Some Conference
OSCO
N 2011
Stronger Syncronous Replication
* Create a replication role (no longer a forced role creation like in 9.0)* pg_hba.conf (host based authorization) update to allow replication user to access system* Five key configuration options: wal_level, max_wal_senders, archive_mode, archive_command, synchronous_standby_names* Create a base backup that will be the slave* recovery.conf config: restore_command, standby_mode, primary_conninfo, trigger_file
Some Conference
OSCO
N 2011
Terrifyingly live demo.
Some Conference
OSCO
N 2011
Thanks toJosh Berkushttp://www.pgexperts.com/document.html?id=50
Depeszhttp://www.depesz.com
Stronger
Harder Better
Faster
Some Conference
OSCO
N 2011
Other Postgres talks: Pro PostgreSQLRobert Treat, 1:40pm Wednesday
Unbreaking your Django ApplicationChristophe Pettus, 9am (tutorial happening NOW)
Stronger
Harder Better
Faster
Some Conference
OSCO
N 2011
Selena Deckelmann @selenamariehttp://chesnok.com
For hire! :)
Stronger
Harder Better
Faster