Open PostgreSQLMonitoring

A brand new & free monitoring suite

Jean-Paul Argudo / Julien Rouhaud

Dalibo - www.dalibo.org

PostgreSQL Conference Europe 2014 - October, 22

1 / 26

OriginsComparison with Oracle

I Multimaster : Bi-Directional Replication (BDR)(2014-07)

I Materialized views : version 9.3 (2013-09)I Easy partitionning : TODOI Parallel queries : TODOI AWR : PoWAI Oracle Enterprise Manager, Oracle Grid : OPM!

2 / 26

State of the ArtA tool is missing

I proprietary softwareI existing Open Source projects to assembleI no real Open Source tool dedicated to PostgreSQL

3 / 26

Main ideas

To be accepted by the community the tool must beI completely free and Open SourceI crafted for PostgreSQLI extensible with a ”pluggable” architecture

4 / 26

Objectives

I a powerful monitoring platformI the standard for PostgreSQL monitoringI must compete with proprietary software

5 / 26

Functionalities

I an extensible architectureI Suitable probesI graphsI alerts

6 / 26

An active project

I OPM has 5 releases alreadyI 13 releases for check_pgactivityI fast pace dev : release often, release early

7 / 26

HistoryVersion 1.X

I creation of the architecture (opm_core and wh_nagios)I first metrics in check_pgactivityI graphs management

I aspect (Stacked, filled, etc)I human-readable unitsI export graphs into images

8 / 26

HistoryVersion 2.0 to 2.3

I new designI new authentication and grant/revoke methodsI pluggable architecture to accept pluginsI tags managementI bug fixes

9 / 26

OPM in details1/2

I A long projectI Composed of some new projectsI Still using some well-known ones, like Nagios

10 / 26

OPM in details2/2

I A graphing engine opm_coreI Modules to store data, at the moment, only

wh_nagiosI Probes dedicated to PostgreSQL : check_pgactivity

11 / 26

Graphing engine1/3

I PostgreSQL extension (opm_core) written in pl/pgsqlI web GUI in Perl (Mojolicious) and JavascriptI access control

12 / 26

Graphing engine2/3

I discovers automatically the data found (servers,services, etc)

I generates automatically the associated graphs, andassociates them to the correspond server

I tags to aggregate various graphs

13 / 26

Graphing engine3/3

I an API to facilitate writing to the warehouseI sampling functionI storing tables to inheritI . . .

14 / 26

Authorizations

I groups and UsersI 1 server = 1 groupI each user is part of one or many groupsI two levels of users : Administrator and User

15 / 26

wh_nagiosStorage module

I standard warehouse dedicated to Nagios data(perfdata)

I storing and archiving of all the valuesI contains a dispatcher to handle Nagios data

16 / 26

check_pgactivityNagios probe (1/3)

I standard Nagios probeI very simple code, and an APII performance data oriented to graphing

I managing of units in the perfdatasI the right metric is sent, no transformation needed

17 / 26

check_pgactivityNagios probe (2/3)

I better or new probes, based on our experienceI local file storage to compute deltasI probe on bgwriterI writing speed into WALsI . . .

18 / 26

check_pgactivityNagios probe (3/3)

I still many probes to invent or addI roadmap

I make it more modularI create a dedicated probe for the system

19 / 26

Overview

20 / 26

Create a warehouse1/3

I specify a schemaI table inheritance

I servicesI metricsI series

I And all the warehousing necessary

21 / 26

Create a warehouse2/3

I implement functionsI get_metric_data()I (grant/revoke)_dispatcher()I purge_service()

I and all the warehousing necessary

22 / 26

Create a warehouse3/3

I optional, add elements in the interfaceI decorateors, e.g. :

I details_server()I details_service()

I or dedicated screensI e.g. : Administration screens wh_nagios

23 / 26

Roadmap

I remove the dependency on Nagios or another toolI push and pull modes supportI have the dashboard customizableI metrics correlationI and much more

24 / 26

Some numbers

I more than 100 servers generatesI more than 2000 servicesI more than 17000 metrics

I over 9 monthsI represents 8.6 Gb of data

25 / 26

Demo

Because a good demo is better than boring slides !I demo.opm.ioI login : opmI pass : demo

More informations on opm.io

26 / 26