Upload
ontico
View
106
Download
5
Embed Size (px)
Citation preview
PostgreSQL @ Alibaba Cloud
Xianming Dou 2016.11.05
How to create a PostgreSQL instance from ZERO?
Create PostgreSQL Instance From ZERO
Create PostgreSQL Instance From ZERO
Create PostgreSQL Instance From ZERO in 10 minutes
HA
• Transparent• Replication
Lag limit
After Created
Monitoring
• Performance• Exception
and Alarm• Auto-Repair
Backup And Restore• Backup
Scheduling • Time
Specified Restore
PostgreSQL
• OOM Handling• IO Hung• Privilege
Management• Transparent
Switch-Over• Transparent
Connection Pool
High Availability
Meta Data of instances
HA Worker PG
Master
PG SlaveHA Worker
…
HA Worker
HA Worker
…
PG Master
PG Slave
PG Master
PG Slave
replication
HA Master
HA Master
Detect
Promote
Monitoring
Host Collector
Host Collector
Host Collector
Host Collector
Database StormingAPI
Rules And Check Databas
e
Performance
Alarm Auto-Repairing
Instant Detect
Backup and Restore
Backup
Restore
Instance 1
Instance 2
Instance …
Instance N
Backup Scheduler
Persistent Storage
Meta DB
Control Flow
Data Streaming
Data Streaming
OOM Handling• Standalone Handler• Signal Process
IO Hung• ext4• IO sync
Privilege Management• No super-user• Normal user is not enough
PostgreSQL Kernel Enhancement
Transparent Connection• Connection Pool• Connection Switch
PG
OOM Handling
OOM Handler
PG Process 1
Instance Resource Container
Public Resource Container
OOM Signal
PG Process N…Most
Memory Usage
PG Process 2PG Process 2
OOM Handling
OOM Handler
PG Process 1
Instance Resource Container
PG Process 2
Public Resource Container
Specified signal for statement canceling
PG Process N…
PG Process 2Received cancel and report OOM to frontend.
PG Process 2
MovePG Process 2
OOM Handling
OOM Handling• Standalone Handler• Signal Process
PG
Privilege Management
• Super-user is dangerous• User need lots of privileges• Compatibility
rolname sanityrolsuper frolinherit trolcreaterole trolcreatedb trolcatupdate trolcanlogin trolreplication trolconnlimit -1rolpassword ****rolvaliduntil
OOM Handling• Resource Limit• Process Container Control
Privilege Management• No super-user• Normal user is not enough
PostgreSQL Kernel
PG
IO Hung
ext4 journal
lock write
Data Pages
Could be a lot, and IO consuming
The problem of fsync during checkpoint
BLOCK File System
write
unlock
More Page WriteIO limit
GET WORSE
IO Hung
ext4 journal
lock write
Data Pages
Sync data before fsync during checkpoint
write
unlock
OOM Handling• Resource Limit• Process Container Control
IO Hung• ext4• IO sync
Privilege Management• No super-user• Normal user is not enough
IO Hung
PG
Transparent Connection Control
Master
Slave
Net Proxy HA
User Switch
Keep Connected
Shutdown
Switch
Promote
Transparent Connection Control
Master
Slave
Net Proxy
Keep Connected
Statement Clean
HA
• Transparent• Replication
Lag limit
After Created
Monitoring
• Performance• Exception
and Alarm• Auto-Repair
Backup And Restore• Backup
Scheduling • Time
Specified Restore
PostgreSQL
• OOM Handling• IO Hung• Privilege
Management• Transparent
Connection Control