Informix Architecture

© 2010 IBM CorporationSeptember 13, 2010

Informix Architecture(Informix on Campus Lecture Series)

Information Management – Informix

© 2010 IBM Corporation2

Agenda

Architecture overview

Process model

Memory model

Storage model



Architecture Overview



IDS Server components

Process component– One or more processes that

do the tasks requested by the database server instance

Shared memory component– Shared memory segments

used to cache table data from disk and maintain and control resources needed by processes

Disk component– Disk is a storage facility to

hold table data and system information necessary to maintain the server



PROCESS MODEL



Process based database server

Each process receives a time-slice of CPU time, represented by “CPU Usage Over Time”

Notice that the oninitprocesses fall “in line”between other processes (p1, p2, p4, p5…).

As more users connect, the number of processes increase and the demand for resources does as well.

* Informix database server processes are called oninit



Process based database server cont…

Process-based database servers are highly inefficient!

0

100

Can waste 50% or more of CPU resources!!

CPUUsage

CPU being used

CPU idle, not working on process



Dynamic Scalable Architecture (DSA)

1993 – Informix creates Dynamic Scalable Architecture and IDS is born!

Multi-threaded from the core out Custom libraries, not Posix

– Resource efficient– Not O/S dependent

• Easily ported

Integrated parallelism– Dynamic– Scalable



Dynamic Scalable Architecture (DSA) cont…

Processes– Each database server process (oninit) is known as a “virtual

processor” because it schedules and runs its own threads– Virtual Processors (VPs) are divided into classes based on

functionality• E.g. Writing to logical or physical log, reading data from disk,

performing administrative tasks

Threads– Services requests from client application (user threads)

• E.g. Select query issued by a client is serviced by an user thread– Accomplish internal tasks like database I/O, logging I/O, page

cleaning, administrative functions (internal threads)– A thread can run on any VP in its class– To run a thread, VP retrieves thread’s data and environment

stack from ready queue and executes request



Dynamic Scalable Architecture (DSA) cont…

Provides fan-in and fan-out parallelism

PhysicalProcessor

VirtualProcessor

Fan-in Fan-out

PhysicalProcessor

1

PhysicalProcessor

2

PhysicalProcessor

3

VirtualProcessor

1

VirtualProcessor

2

VirtualProcessor

3



DSA - Thread based database server

Each thread receives a “chance” to run as a part of the oninit process on

a physical processor.

Notice the bottom axis shows “Virtual Processor Usage Over Time”. This represents time the thread is running on a pre-existing oninit process

As more users connect, the number of processes remains static as each

connection represents a request to run a

“thread”, not a “process”



DSA - Thread based database server

Result: almost complete usage of system resources and significantly better scalability!

0

100

CPUUsage

CPU being used

CPU idle, not working on process



DSA – Putting it all together

Serial akaSingle threaded

Scan

Join

SortTime toProcess

Parallel Parallelized

DSAprocesses tasks

concurrentlyDSA

breaks tasksinto subtasks

Write

(Two table join)



DSA - Putting it all together

A single query example:

select * from flightsorder by flightno;

Introduction to Informix Dynamic Server 14

Disk Scan

Scan Thread

Sort Thread



DSA – Putting it all togetherA more complexexample using aparallel tree

Exchange Threads

Exchange Threads

Hash Join Threads

Scan Threads (flightleg)

Scan Threads (ticket)

Exchange Threads

Sort Threads

select count(*) from tickets a,flightleg b where a.flno =b.flno andorig=“DFW”order by a.class



MEMORY MODEL



Shared Memory

O/S feature that allows database server processes to share data by sharing access to pools of memory

Reduces disk I/O by caching data from disk

Provides the fastest method of inter-process communication

Provides communication channels for local client applications that use IPC communication

Virtual processor Bmemory space

Shared MemorySegments

Unallocatedspace

Privatedata

Program text

Unallocatedspace

Privatedata

Program text

Virtual processor Amemory space

Data Client

Client

Client



Unallocatedspace

Privatedata

Program text

Unallocatedspace

Privatedata

Program text




Unallocatedspace

Privatedata

Program text

Unallocatedspace

Privatedata

Program text


DataData Client

Client

Client

Client

Client

Client

Client

Client

Client



Shared Memory Segments

Resident segment– Contains the buffer pool and other system information

• Buffer pool caches data from table– Can be configured to remain resident in main memory– Fixed size

Virtual segment– Contains information about the threads and sessions, and the

data used by them– Could be paged out to disk by the operating system– Expandable

Message segment– Holds the message buffers used in client-server communication

if shared memory communication is configured



Shared Memory Structure



STORAGE MODEL



Disk Page

The basic unit of storage in a server

All database and system information is stored on pages

The minimum unit of I/O in a server is a page

The size of a page is by default 2KB on most UNIX systems and 4KB on AIX and Windows

Page size can also be configured (max of 16KB)



Extents and Tablespaces

Extent

– Collection of contiguous pages

– Space for table is allocated inunits of extents

Tablespace

– Logical collection of extents

– Storage for table

– A table can have multiple tablespaces

ExtentTablespace



Chunks

Largest unit of contiguous disk dedicated to database server data storage

Chunk can be a raw device (character-special device), a piece of a raw device, or a UNIX file (cooked file)

Maximum size of chunk is 4TB

Maximum allowable chunks is 32767



Dbspaces

Logical collection of one or more chunks

Can have between 1 and 32767 chunks

Maximum allowable dbspaces is 2047



Storage Model

Chunk

Chunk

Chunk

Dbspace

Tablespace

Page

Extents



Blobspaces

Special dbspace that stores simple large objects (TEXT and BYTE data)

Basic unit of storage in blobspace is a blobpage

Blobpage size can be configured as a multiple of the database server page size

Database server writes data stored in a blobspace directly to disk (not buffered)



Sbspaces

Special dbspace that stores smart large objects (BLOB and CLOB data) and user defined data

Pages in sbspace are called sbpages

Sbpage size is same as the database server page size and is not configurable

Basic unit of allocation in sbspace is an extent

Writes to sbspace are buffered by default– Buffering can be turned off at sbspace creation time



Mirroring

Process of automatically writing same data to two disks

Eliminates data loss due to disk failure– If one disk fails, the data is still

available on the other disk

Costs– Additional disk space– Performance cost of having to

write to two locations

Primary Chunk Mirrored Chunk

Writes

Primary Chunk Mirrored Chunk

Writes



Reading and Caching data

When a client issues a query, the table data is read from disk into buffers in shared memory

I/O is performed in page units

Subsequent access to the same data is from buffer pool

Changes made to the data are visible to all database server process



Physical and Logical Logging

Physical logging– Process of storing before-images of pages that are being modified– Before-images are stored in physical log– Physical log is a collection of contiguous pages on disk– Required for recovery purpose in event of a system failure

Logical logging– Process of recording transaction details– Transaction records are stored in logical log– Logical log is comprised of logical log files, which are collections of

contiguous pages on disk– Required for transaction rollback and recovery purpose in event of a

system failure



Checkpoints and Recovery

Checkpoints– Periodic system event during which all modified buffers are written to

disk– Establishes a consistent state for database server– Information on checkpoint event is recorded in the system pages and

logical log (needed for recovery purposes)

Recovery– If a system failure occurs, the database server restarts at that

established point– Before-images of pages modified since checkpoint are restored from

physical log– Transactions since checkpoint are then replayed from the logical log to

get the server to the consistent state just before the time of failure



Resources

The Online IDS Information Center– http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp

IBM Informix DeveloperWorks Technical Articles– http://www.ibm.com/developerworks/db2/products/informix/index.html

IBM DeveloperWorks IDS Blogs– http://www-128.ibm.com/developerworks/blogs/page/roundrep (IDS

Replication)

– http://www-128.ibm.com/developerworks/blogs/page/gbowerman (IDS Application Development)

– http://www-128.ibm.com/developerworks/blogs/page/idsteam (IDS Experts Blog)

Documents

Informix Architecture