© 2010 IBM CorporationSeptember 13, 2010
Informix Architecture(Informix on Campus Lecture Series)
Information Management – Informix
© 2010 IBM Corporation2
Agenda
Architecture overview
Process model
Memory model
Storage model
Information Management – Informix
© 2010 IBM Corporation3
Architecture Overview
Information Management – Informix
© 2010 IBM Corporation4
IDS Server components
Process component– One or more processes that
do the tasks requested by the database server instance
Shared memory component– Shared memory segments
used to cache table data from disk and maintain and control resources needed by processes
Disk component– Disk is a storage facility to
hold table data and system information necessary to maintain the server
Information Management – Informix
© 2010 IBM Corporation5
PROCESS MODEL
Information Management – Informix
© 2010 IBM Corporation6
Process based database server
Each process receives a time-slice of CPU time, represented by “CPU Usage Over Time”
Notice that the oninitprocesses fall “in line”between other processes (p1, p2, p4, p5…).
As more users connect, the number of processes increase and the demand for resources does as well.
* Informix database server processes are called oninit
Information Management – Informix
© 2010 IBM Corporation7
Process based database server cont…
Process-based database servers are highly inefficient!
0
100
Can waste 50% or more of CPU resources!!
CPUUsage
CPU being used
CPU idle, not working on process
Information Management – Informix
© 2010 IBM Corporation8
Dynamic Scalable Architecture (DSA)
1993 – Informix creates Dynamic Scalable Architecture and IDS is born!
Multi-threaded from the core out Custom libraries, not Posix
– Resource efficient– Not O/S dependent
• Easily ported
Integrated parallelism– Dynamic– Scalable
Information Management – Informix
© 2010 IBM Corporation9
Dynamic Scalable Architecture (DSA) cont…
Processes– Each database server process (oninit) is known as a “virtual
processor” because it schedules and runs its own threads– Virtual Processors (VPs) are divided into classes based on
functionality• E.g. Writing to logical or physical log, reading data from disk,
performing administrative tasks
Threads– Services requests from client application (user threads)
• E.g. Select query issued by a client is serviced by an user thread– Accomplish internal tasks like database I/O, logging I/O, page
cleaning, administrative functions (internal threads)– A thread can run on any VP in its class– To run a thread, VP retrieves thread’s data and environment
stack from ready queue and executes request
Information Management – Informix
© 2010 IBM Corporation10
Dynamic Scalable Architecture (DSA) cont…
Provides fan-in and fan-out parallelism
PhysicalProcessor
VirtualProcessor
Fan-in Fan-out
PhysicalProcessor
1
PhysicalProcessor
2
PhysicalProcessor
3
VirtualProcessor
1
VirtualProcessor
2
VirtualProcessor
3
Information Management – Informix
© 2010 IBM Corporation11
DSA - Thread based database server
Each thread receives a “chance” to run as a part of the oninit process on
a physical processor.
Notice the bottom axis shows “Virtual Processor Usage Over Time”. This represents time the thread is running on a pre-existing oninit process
As more users connect, the number of processes remains static as each
connection represents a request to run a
“thread”, not a “process”
Information Management – Informix
© 2010 IBM Corporation12
DSA - Thread based database server
Result: almost complete usage of system resources and significantly better scalability!
0
100
CPUUsage
CPU being used
CPU idle, not working on process
Information Management – Informix
© 2010 IBM Corporation13
DSA – Putting it all together
Serial akaSingle threaded
Scan
Join
SortTime toProcess
Parallel Parallelized
DSAprocesses tasks
concurrentlyDSA
breaks tasksinto subtasks
Write
(Two table join)
Information Management – Informix
© 2010 IBM Corporation14
DSA - Putting it all together
A single query example:
select * from flightsorder by flightno;
Introduction to Informix Dynamic Server 14
Disk Scan
Scan Thread
Sort Thread
Information Management – Informix
© 2010 IBM Corporation15
DSA – Putting it all togetherA more complexexample using aparallel tree
Exchange Threads
Exchange Threads
Hash Join Threads
Scan Threads (flightleg)
Scan Threads (ticket)
Exchange Threads
Sort Threads
select count(*) from tickets a,flightleg b where a.flno =b.flno andorig=“DFW”order by a.class
Information Management – Informix
© 2010 IBM Corporation16
MEMORY MODEL
Information Management – Informix
© 2010 IBM Corporation17
Shared Memory
O/S feature that allows database server processes to share data by sharing access to pools of memory
Reduces disk I/O by caching data from disk
Provides the fastest method of inter-process communication
Provides communication channels for local client applications that use IPC communication
Virtual processor Bmemory space
Shared MemorySegments
Unallocatedspace
Privatedata
Program text
Unallocatedspace
Privatedata
Program text
Virtual processor Amemory space
Data Client
Client
Client
Virtual processor Bmemory space
Shared MemorySegments
Unallocatedspace
Privatedata
Program text
Unallocatedspace
Privatedata
Program text
Virtual processor Amemory space
Virtual processor Bmemory space
Shared MemorySegments
Unallocatedspace
Privatedata
Program text
Unallocatedspace
Privatedata
Program text
Virtual processor Amemory space
DataData Client
Client
Client
Client
Client
Client
Client
Client
Client
Information Management – Informix
© 2010 IBM Corporation18
Shared Memory Segments
Resident segment– Contains the buffer pool and other system information
• Buffer pool caches data from table– Can be configured to remain resident in main memory– Fixed size
Virtual segment– Contains information about the threads and sessions, and the
data used by them– Could be paged out to disk by the operating system– Expandable
Message segment– Holds the message buffers used in client-server communication
if shared memory communication is configured
Information Management – Informix
© 2010 IBM Corporation19
Shared Memory Structure
Information Management – Informix
© 2010 IBM Corporation20
STORAGE MODEL
Information Management – Informix
© 2010 IBM Corporation21
Disk Page
The basic unit of storage in a server
All database and system information is stored on pages
The minimum unit of I/O in a server is a page
The size of a page is by default 2KB on most UNIX systems and 4KB on AIX and Windows
Page size can also be configured (max of 16KB)
Information Management – Informix
© 2010 IBM Corporation22
Extents and Tablespaces
Extent
– Collection of contiguous pages
– Space for table is allocated inunits of extents
Tablespace
– Logical collection of extents
– Storage for table
– A table can have multiple tablespaces
ExtentTablespace
Information Management – Informix
© 2010 IBM Corporation23
Chunks
Largest unit of contiguous disk dedicated to database server data storage
Chunk can be a raw device (character-special device), a piece of a raw device, or a UNIX file (cooked file)
Maximum size of chunk is 4TB
Maximum allowable chunks is 32767
Information Management – Informix
© 2010 IBM Corporation24
Dbspaces
Logical collection of one or more chunks
Can have between 1 and 32767 chunks
Maximum allowable dbspaces is 2047
Information Management – Informix
© 2010 IBM Corporation25
Storage Model
Chunk
Chunk
Chunk
Dbspace
Tablespace
Page
Extents
Information Management – Informix
© 2010 IBM Corporation26
Blobspaces
Special dbspace that stores simple large objects (TEXT and BYTE data)
Basic unit of storage in blobspace is a blobpage
Blobpage size can be configured as a multiple of the database server page size
Database server writes data stored in a blobspace directly to disk (not buffered)
Information Management – Informix
© 2010 IBM Corporation27
Sbspaces
Special dbspace that stores smart large objects (BLOB and CLOB data) and user defined data
Pages in sbspace are called sbpages
Sbpage size is same as the database server page size and is not configurable
Basic unit of allocation in sbspace is an extent
Writes to sbspace are buffered by default– Buffering can be turned off at sbspace creation time
Information Management – Informix
© 2010 IBM Corporation28
Mirroring
Process of automatically writing same data to two disks
Eliminates data loss due to disk failure– If one disk fails, the data is still
available on the other disk
Costs– Additional disk space– Performance cost of having to
write to two locations
Primary Chunk Mirrored Chunk
Writes
Primary Chunk Mirrored Chunk
Writes
Information Management – Informix
© 2010 IBM Corporation29
Reading and Caching data
When a client issues a query, the table data is read from disk into buffers in shared memory
I/O is performed in page units
Subsequent access to the same data is from buffer pool
Changes made to the data are visible to all database server process
Information Management – Informix
© 2010 IBM Corporation30
Physical and Logical Logging
Physical logging– Process of storing before-images of pages that are being modified– Before-images are stored in physical log– Physical log is a collection of contiguous pages on disk– Required for recovery purpose in event of a system failure
Logical logging– Process of recording transaction details– Transaction records are stored in logical log– Logical log is comprised of logical log files, which are collections of
contiguous pages on disk– Required for transaction rollback and recovery purpose in event of a
system failure
Information Management – Informix
© 2010 IBM Corporation31
Checkpoints and Recovery
Checkpoints– Periodic system event during which all modified buffers are written to
disk– Establishes a consistent state for database server– Information on checkpoint event is recorded in the system pages and
logical log (needed for recovery purposes)
Recovery– If a system failure occurs, the database server restarts at that
established point– Before-images of pages modified since checkpoint are restored from
physical log– Transactions since checkpoint are then replayed from the logical log to
get the server to the consistent state just before the time of failure
Information Management – Informix
© 2010 IBM Corporation32
Resources
The Online IDS Information Center– http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp
IBM Informix DeveloperWorks Technical Articles– http://www.ibm.com/developerworks/db2/products/informix/index.html
IBM DeveloperWorks IDS Blogs– http://www-128.ibm.com/developerworks/blogs/page/roundrep (IDS
Replication)
– http://www-128.ibm.com/developerworks/blogs/page/gbowerman (IDS Application Development)
– http://www-128.ibm.com/developerworks/blogs/page/idsteam (IDS Experts Blog)