Ugif 10 2012 ppt0000001

User Group Informix France

Partition Defragmenter

Yoram [email protected]

Mercredi 3 Octobre 2012Mercredi 3 Octobre 2012


AgendaAgenda

�A bit of partition background�The problem with fragmented data�How does the feature work�Functionality & considerations�Defragmenter in OAT�Example


A bit of partition background�What is an extent ?

– Collection of physically contiguous pages on disk

– Extent can hold data page, index page, bitmap page etc…

�What is a partition ?– Logical collection of all the extents allocated for a

given table or index.

– Space for partition is allocated in units of extents and is not necessarily contiguous.

�Each partition are unique within the system– oncheck –pt <partnum>


A bit of partition background

Partition logical to physical extent relationship



Extents for table A

Extents for

other tables

Chunk 14

Free pages

�Before Defragmenting


�After Defragmenting

Chunk 14

Extents for table A




7

�Previous Solutions– Table unload and reload

�Oncheck –me– Off-line table reorganization– Non-recoverable

– Unsupported



�How many extents exist ?– Run oncheck –pe.– Run oncheck –pt.

– SQLdatabase sysmaster; select unique te_partnum, count(*) from systabextents group by 1 having count(*) > NN order by 1 desc;

• Where NN is a value.


The problem with fragmented data

�What’s the problem with many extents? – Full partition page.

• Prohibits new inserts.– Might not be at maximum number of rows.

• Need to either delete older entries, if possible.– or

• Requires a manual table recreation.

– (FYI: Actually, this can no longer exist in 11.70)


�Many extents degrades performance.– Need to skip over non-contiguous disk areas.– Slower retrieval of data.

– Mapping logical page numbers to physical addresses is slower.

– Chunk allocations are slower with long free list pages

– Archive extent-sorting phase is slower



�What’s the problem with many extents? – Many tables with many extents feels

overbearing.• Outputs from oncheck –pe & -pt.



How does the feature work

�Run new SQL API functions.– sysadmin database.

– execute function [admin|task] ("<defragment","<database>:[<owner>.]<table>[#index]")

• or

– execute function [admin|task] ("<defragment partnum", "<partnum> [, <partnum>, ... ]”);



– What goes on behind the scenes.• Check validity.

• Create new extent.• Might need just one adding

• Start a loop and process all old extents.– Read old extent.– Move all old extent pages to the new extent.– Delete old extent.

• Loop end.



�Log the work using five new log types.– Start defragmentation.

– End defragmentation.– Add new defragmentation extent.

– Remove old defragmentation extent.– Move page.

�Will result in a surge in logical log usage.



�Monitoring progress.– onstat -g defragment.

• Searching for extents.• Merging extents.• Defrag_completed.• Defrag_failed.



�Recoverable if server shuts down or crash

– Defragmenter cleaner thread will remove extra extents after fast recovery

• Rolls back current unit of work (UOW) if needed.– 08:52:06 Defragmenter cleaner thread now running– 08:52:06 Defragmenter cleaner thread cleaned:1 partitions

– Will the object be consistent ?• Yes.

– Choose to run again to complete the job.


�Can defragment table online�No locks placed on object�Only allowed one defragmenter running per

dbspace�Replication is supported (HDR, MACH11, ER)�Run via the SQL API interface.

Functionality


�Functionality not allowed – Functional indexes.

– VTI & VII objects.– Temporary tables.

– Sort tables.– Optical blobs.

– System catalog tables

Considerations


�Cannot stop once job submitted�Cannot run simultaneously when table being

– Altered– Truncated

– Drop– Compress, Repack, Shrink

�Not the same as compress/repack/shrink

Considerations

�Enterprise Replication.– Table needs to be in CDR alter mode.

• Mode unset automatically at end of job


�Operation is logging-intensive– Each page move is logically logged (DFMVPG)

• 70-80 bytes per page moved

�Small units of work to avoid long transaction.– It may send other transactions over the threshold

�New extents created at end of dbspace.�Schedule during low activity.�Perform after a backup.

Considerations


Defragmenter in OAT

21

On the OAT menu, expand

Space Administration > Space Optimization


Defragmenter in OAT

22

Select the tables.

On the Actions menu, click Optimize Space.


Defragmenter in OAT

23

Select Defragment extents and click OK.


�Automatic Storage Optimization– Can automate defragmenting using OAT.

– Enabled via scheduler task “auto_crsd”• This can also do compressing, repacking, shrinking.• ‘crsd’ stand for compress, repack, shrink, defragment

– This can also be enabled via SQL

Defragmenter in OAT


�‘auto_crsd’ task details– In OAT -> Task Scheduler -> Task Details

• UPDATE ph_task SET tk_enable = 'T‘ WHERE tk_name = 'auto_crsd';

Defragmenter in OAT


�‘auto_crsd’ task parameters.

– UPDATE ph_threshold SET value = 'F WHERE name = 'AUTODEFRAG_ENABLED';

– UPDATE ph_threshold SET value = ‘100‘ WHERE name = 'AUTODEFRAG_EXTENTS';

Defragmenter in OAT


Example

� SQL to create three fragmented tables– create table tab1 (a serial, b int, c int);– create table tab2 (a serial, b int, c int);– create table tab3 (a serial, b int, c int);– create procedure ins (a int, b int) define c int;

• for c = a to b• insert into tab1 values (0, 88, 99);• insert into tab2 values (0, 88, 99);• insert into tab3 values (0, 88, 99);• end for;

– end procedure;– execute procedure ins (1, 1000000);


Example

�oncheck –pe before defragment.


Example

� oncheck –pt before defragment.– FYI : in 11.70 No longer limited to one partn page.


Example

�SQL API function.– execute function task ("defragment", "oltr:tab1");

(expression) OK


Example

�oncheck –pe after defragment.


Example

�oncheck –pt after defragment.


Example

�onlog output


SummarySummary

�Informix Defragmenter Capabilities

– Partition Extents Defragmentation– New System Admin SQL API

– OAT GUI Interface– Automatic Defragmentation

– Can be completed with compress/repack/shrink for optimal storage optimization


Questions ?


MerciMerci

Yoram [email protected]

Mercredi 3 Octobre 2012Mercredi 3 Octobre 2012

Documents

Ugif 10 2012 ppt0000001