Embed Size (px)
Citation preview
Usually big production database runs at• Firebird 2.5• Classic or SuperClassic architecture
• They can scale accross multiple CPU/cores!
• In Firebird 3 SuperServer is the better choice… • but now let’s talk about 2.5 Classic and SuperClassic
What are performance indicators to monitor in Firebird 2.5?1. Lock table parameters2. Monitoring record versions and garbage collection3. Transactions
1. Long running transactions2. Rolled back transactions
4. Temporary files – quantity and size5. Connections, SQL queries and transactions
LOCK TABLE MONITORING
fb_inet_serverfb_inet_server
How lock table works
fb_inet_server
Firebird
Page cache Page cache Page cache
Page X
How to resolve a write conflict when changing the same page?
How lock table works
fb_inet_server fb_inet_server fb_inet_server
mapped
Lock table
mapped mapped
shared memoryAll requests to access internal database objects go through lock table
Why should we monitor lock table parameters?• Lock table is a critical part of Classic and SuperClassic
architectures• Access to shared objects is implemented through locks in
Lock table…
Analysis of lock table parameters is the easiest way to reveal problems with concurrent access
Lock table analysis - raw• Fb_lock_print –d <database_name>• LOCK_HEADER BLOCK• Version: 17, Active owner: 0, Length: 6291456, Used: 5517236• Flags: 0x0001• Enqs: 10906251, Converts: 58907, Rejects: 22373, Blocks: 210859• Deadlock scans: 5841, Deadlocks: 0, Scan interval: 10• Acquires: 13636997, Acquire blocks: 558879, Spin count: 0• Mutex wait: 4.1%• Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26• Remove node: 0, Insert queue: 0, Insert prior: 0• Owners (107): forward: 26696, backward: 5517140• Free owners: *empty*• Free locks (1630): forward: 3878196, backward: 2264580• Free requests (793): forward: 5412916, backward: 1906516• Lock Ordering: Enabled
Lock table analysis – where to look• Fb_lock_print –d <database_name>• LOCK_HEADER BLOCK• Version: 17, Active owner: 0, Length: 6291456, Used: 5517236• Flags: 0x0001• Enqs: 10906251, Converts: 58907, Rejects: 22373, Blocks: 210859• Deadlock scans: 5841, Deadlocks: 0, Scan interval: 10• Acquires: 13636997, Acquire blocks: 558879, Spin count: 0• Mutex wait: 4.1%• Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26• Remove node: 0, Insert queue: 0, Insert prior: 0• Owners (107): forward: 26696, backward: 5517140• Free owners: *empty*• Free locks (1630): forward: 3878196, backward: 2264580• Free requests (793): forward: 5412916, backward: 1906516• Lock Ordering: Enabled
Thresholds and recommendationsEssential values: Firebird.conf params to tune:
Length: 6291456 LockMemSize - default is 1MbSet at least:Cache_pages * max_connections_count * 100
Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26
LockHashSlots = 1009 Prime number. Set x20, like 20011
Mutex wait: 4.1% No parameter to adjust. More than 10% could be a problem, check hardware and concurrency settings
Owners (107) Number of connections for server
Use optimized Firebird configuration files from IBSurgeon:http://ib-aid.com/en/optimized-firebird-configuration/
How to monitor lock table• 1) Command prompt (on the server only), run every N min
fb_lock_print -d E:\OLTP-EMUL\oltp30.fdb
• 2) Alerts and automatic recommendations
Monitoring record versions and garbage collection in Firebird
What is a record version?
TR50 TR60
UPDATE set data =200read
N Tx Data1 10 100
60 200
...
readwrite
Different transactions can see different versions of the record.
Why many record versions are bad
The more versions record has, the more scan operations server does to find necessary version
After data load or restore – no versions
• When UPDATE changes indexed fields, indices also must be updated, and - UPDATE does not update keys in the index, it adds new keys for new record version!
• DELETE does not delete index keys!
N Tx Data1 10 100
60 200
...
Index key = 100
Index key = 200
Why many record versions are bad
What to monitor?
Database statistics• Database statistics shows information about Record
Versions and Max Versions• Taking database statistics can be long!
How to monitor1. With gstat command line tool2. With HQbird Firebird Database Analyst
More details: https://www.youtube.com/watch?v=_KZqXcgwN98
Goal of every Firebird developer is
to avoid creating versions for records!
Not only because server slowly reads multiple record versions, but also because of
GARBAGE COLLECTION
What is garbage and how it is collected
• When record versions become obsolete and non-interested by any transaction, it is considered as garbage and need to be cleaned.
• Garbage collection is a process of removing unnecessary records versions• It can be cooperative (Classic or SuperClassic) or
background (SuperServer)
Why should we monitor garbage collection?
• It consumes resources.• We should locate and fix those parts of the application(s) which produce many record versions which increase an intensity of GC
How to monitor GC?• 1) Manual SQL queries to MON$ tables• 2) HQbird Firebird MonLogger
How to decrease number of garbage versions?
1. Use as short write transactions as possible2. For read-only operations use (if possible)
transactions READ COMMITTED READ-ONLY
If you see a lot of record versions, and you are not sure how to fix it, consider to order our Database Optimization service:http://ib-aid.com/en/firebird-interbase-performance-optimization-service/
Monitoring transactions in Firebird
Why to monitor transactions?
Transactions are key indicators of versioning processes in Firebird database.
2 main problems1. Long-running [writeable] transactions2. Rolled back transactions
Gstat –h gives us snaphot of transactionsDatabase header page information: Flags 0 Checksum 12345 Generation 1564 Page size 4096 ODS version 10.1
Oldest transaction 10009 Oldest active 20001 Oldest snapshot 20001 Next transaction 25007 Bumped transaction 1 Sequence number 0 Next attachment ID 1 Implementation ID 16 Shadow count 0 Page buffers 3000 Next header page 0 Database dialect 1 Creation date Nov 1, 2015 15:55:55
Long-running transactions• All transactions have sequential numbers from 1 to…• Oldest Active Transaction – currently active transaction
with the minimal numberOldest transaction 10009Oldest active 20001Oldest snapshot 20001Next transaction 25007
Interval NEXT – OAT shows number of potentially active transactions – server cannot clean versions created by transactions with numbers > OAT
How to track long running active transactions in Firebird?
• 1. Manual tracking of gstat –h output (check every N minutes)
• 2. Alerts from HQbird FBDataGuard
How to fix long running active transactions in Firebird?
It is possible to find who is running long-running transaction and CANCEL it
1. Manual query to MON$ tables2. HQbird Firebird MON$Logger (GUI)
Rolled back transactions• When some transaction is marked in transaction inventory
as rolled back, it prevents record versions beeing cleaned by collective or background garbage collection
Oldest transaction 10009Oldest active 20001Oldest snapshot 20001Next transaction 25007
Interval Oldest Snapshot – Oldest Interesting determines the need for sweep
Sweep• Sweep reads whole database from the beginning to the end and cleans garbage record versions
• Sweep is necessary when OST-OIT interval becomes big
How to make sweep• Autosweep
• by default 20000• Starts immediately when interval > threshold• It causes slowness at unpredicted moments
• Manual sweep• Scheduled sweep during the inactivity period of time
• Can be run manually with gfix –sweep or scheduled in HQbird FBDataGuard
Sweep must be controlled!• If sweep did not succeed to align transaction markers, it can happen due to the serious problem/corruption!
• HQbird FBDataGuard checks sweep status
TEMP FILES
Temp files• Temp files are created in the default temp folder or in folders specified by TempDirectories in firebird.conf
• Actually they are written to the disk only if size of all sort files is more than TempCacheLimit parameter
• It is better to have sorting in memory
How to track Temp files1. Manually check size and quantity of temp files
(fb_sortxxx) in all temp folders2. HQbird FBDataGuard monitors temp files size and
quantity, and sends warnings if necessary
How to move temp files to memory• Increase TempCacheLimit parameter• Warnings:
• At Classic TempCacheLimit is allocated for each process• 32bit processes have 2Gb limitation of memory to
address
Use optimized configuration files from IBSurgeon and HQbird!http://ib-aid.com/en/optimized-firebird-configuration/
Monitor connections, transactions, and SQL queries
Why to monitor Firebird connections?• Number of connections should be constantly monitored:
• Peaks of connections• Firebird Classic/SuperClassic consume RAM as
Number_Of_Connections x Page Buffers• In case of peaks can consume all RAM and go to swap – slowness!
• Application can do more than 1 connection to the database• By mistake• By design
• In case of connection pool – adjust the size of the pool and its increment
• Suspicious connections• Long running connections and connections from developer tools• Connections from non-standard applications
How to monitor connections1. Manually run SELECT count(*) FROM
MON$ATTACHMENTS2. Use HQbird
1. FBDataGuard constantly monitors number of connections and send alerts if their number grows too much
2. MonLogger can be used to go deep and get detailed information about connections: memory, IO, garbage collection
Transactions• Yes, again!• Besides monitoring of intervals between transactions
markers (Next-OAT, OIT-OST), also we should check these important things, which can affect Firebird performance:
1. Transaction’s parameters (isolation level, wait/nowait) 2. Transactions flows and their relation with business
logic3. Transaction-level statistics (IO, time to live)
Transaction’s parameters• Transactions parameters must correspond to business
logic.• Several often mistakes
• Non-modyfing queries (SELECT) are started in read-write mode• Should be in READ ONLY
• Snapshot are used for operational queries, instead of read-commited• It should be used only for queries where consistency is really required
(snapshots)
Transactions flows mistakes• Transaction (other then read-committed read only) never
ends, expires due to connection closing, and leads to ROLLBACK
• Only 1 sentence in the transaction• For example, refreshing query with timer:
• Quickly exhausts 2 billion limit of transactions• it should be run in permanent read committed read only transaction
• Using the only read-write transaction for all operations• Better split read-only and write operations
How to track transactions flows and transactions parameters? - FBScanner
HQbird FBScanner allows to track transactions flows: parameters, containing queries, plans, etc.
How to track transactions flows and transactions parameters? - PerfMon• HQbird Firebird Performance Monitor (PerfMon) uses
TraceAPI to capture all engine events – connections, queries, transactions, stored procedures, triggers, execution plans, execution statistics
• Works correctly only at Firebird 2.5.3+ versions
Quick check of problems with transactions
• HQbird MonLogger gives a fast and detailed overview of active transactions in the system
HQbird tools for SQL and transactions monitoring and optimizationHQbird FBScanner
+ works as a proxy, non-invasive tool, can be installed on separate server and track only selected workstations— cannot track stored procedures and triggersHQbird PerfMon
+ works with TraceAPI, can monitor everything— TraceAPI significantly supresses performanceHQbird MonLogger
+ Great ad-hoc tool, fast— Captures only snapshots, lack of important information
IBSurgeon Firebird Optimization service
We can increase the performance of your databasehttp://ib-aid.com/en/firebird-interbase-performance-optimization-service/
Thank you and don’t forget these links:
IBSurgeon optimized configuration fileshttp://ib-aid.com/en/optimized-firebird-configuration/
Firebird Hardware Guidehttp://ib-aid.com/download/docs/Firebird_Hardware_Guide_IBSurgeon.pdf
Contact us: [email protected]
