Scalable Trigger Processing

Eric N. Hanson et alICDE 1999

Presented byShyamshankar D

MotivationUse of Triggers

Integrity constraint checkingAlerting etc.

Commercial database systems Limited triggering capabilities1-100 triggers/table.

Doesn’t scale to internet and World wide web requirements

An Example A Continuous query

Stock holding: 100*IBMQuery: Inform me whenever the price of the stocks crosses 10,000$

Create Trigger stock-watch from quotes q

on update(q.price)when q.name=‘IBM’ and 100*q.price > 10,000do raise event ThresholdCrossed(100*q.price).

What Next?The problemThe key conceptTriggerMan system architecturePredicate IndexTrigger processingConcurrency

The ProblemRequires millions of triggers.Steps for trigger processing

Event monitoringCondition evaluationExecuting the trigger action

Response time for the database operations.

The Key ConceptIf large number of triggers are created, then most of them have the same format.Triggers share the same expression signature except that one constant has been substituted by another.Group predicates from trigger conditions based on expression signatures and store these in an efficient main memory data structure.

The TriggerMan SystemTrigger system built on Informix databaseCould also be implemented in a trigger system in a DBMS server.Trigger syntaxcreate trigger <triggerName> [in setName][optionalFlags]from fromList[on eventSpec][when condition][group by attributeList][having groupCondition]do action

Example 1

Update Fred’s salary when Bob’s salary is updated

create trigger updateFred

from emp

on update(emp.salary)

when emp.name = ’Bob’

do execSQL ’update emp set salary=:NEW.emp.salary where emp.name=’’Fred’’’

Example 2house(hno,address,price,nno,spno)salesperson(spno,name,phone)represents(spno,nno)neighborhood(nno,name,location)

create trigger IrisHouseAlerton insert to housefrom salesperson s, house h, represents rwhen s.name = ‘Iris’ and s.spno=r.spno andr.nno=h.nnodo raise eventNewHouseInIrisNeighborhood(h.hno, h.address)

System Architecture

ComponentsTriggerMan DatabladeData Sources

Local/remote tables/streams

TriggerMan Client applicationsCreate drop triggers etc.

TriggerMan DriverCondition testing and action TmanTest() fn.

TriggerMan consoleDirect user interaction interface

Trigger Condition structureExpression signature

Expression signature consists ofData source IDOperation codeGeneralized Expression

=

Emp.name CONSTANT

Data src: empEvent : update

Condition structure (contd)

Steps to obtain canonical representation of WHEN clause

Translate expression to CNFGroup each conjunct by data source they refer.

Predicate will be of form (C11 OR C12 OR ..) AND ... (Ck1 OR …), where each Cij refers to a tuple variable.

Very large number of predicates created only if different triggers contain distinct CONSTANT values.If expression has m constants replace xth constant by CONSTANTX

The TablesPrimary tables

trigger_set (tsID, name, comments, creation_date, isEnabled)

Trigger (triggerID, tsID, name, comments, trigger_text, creation_date, isEnabled, …)

Trigger cache in main memory for recently accessed triggers.

Predicate Index

Predicate IndexTables

expression_signature(sigID, dataSrcID, signatureDesc, constTableName, constantSetSize, constantSetOrganization)const_tableN(exprID, triggerID, nextNetworkNode, const1, … constK, restOfPredicate)

Root of predicate index linked to set of data source predicate indexEach data source contains an expression signature listFor each expression signature containing constants, a constant table.Index expressions on most selective conjunct.

Condition TestingFor Predicate to be satisfied all conjuncts should be true. This is checked using an A-Treat network.A-Treat network is a discrimination network for trigger condition testing.

A-Treat network(Hans 92)Define rule SalesClerkIf emp.sal>30,000And

emp.dno=dept.dnoAnd

dept.name=“sales”And emp.jno=job.jnoAnd job.title=“clerk”Then Action

Processing trigger definition

Parse the trigger and validate it Convert the when clause to conjunctive normal form and group the conjuncts by the distinct sets of tuple variables they refer toForm a trigger condition graph. Undirected graph with node for each tuple variable and edge for join predicates.Build the A-Treat network

Processing trigger definition(contd)

For each selection predicate over a alpha node

If predicate with same signature, not seen before

Add signature of predicate to list and to expression_signature tableIf signature has a constant placeholder in it, create a constant table for the signature.

Else if predicate has constants, add a row to the constant table for the expression

Alternate Organization strategies

Storage for the expression signature’s equivalence class:

Main memory listsMain memory indexNon-indexed database tableIndexed database table

For each expression signature use a structure depending on number of triggers.

efficiency

Scalability

Processing update descriptors

On getting an update descriptor/token (data src ID, oper code, old/new tuple)

Locate data source predicate index from root of predicate index.For each expression signature, find constant matching the token.Check additional predicate clauses against the token.When all predicate clauses of a trigger have matched, pin the trigger in main memoryIf trigger condition is satisfied, execute action.

Some optimizations

For I = 1 to N Create trigger Ti from R when R.a=100 do …

List of trigger ids for R.a=100Constant sets and trigger Ids for equality conditions stored as

ListsClustered constant index

All entries of [const1,..constk] stored togetherEnables fast retrieval of triggers ids together

Concurrent processingDifferent types of concurrency

Token level concurrencyMultiple tokens in parallel

Condition level concurrencyTest multiple selection condns against token concurrently

Rule action concurrencyProcess multiple actions fired at same time.

Data level concurrency

Current implementation supports token level concurrency

ConcurrencyN=[NUM_CPUS*TMAN_CONCURRENCY_LEVEL]

N driver processesEach driver process calls TmanTest() after T timeT=future workBalance switching overhead and avoid long executions

TmanTest()while(total execution time of this invocation of TmanTest <

THRESHOLD and work is left in the task queue)

{

Get a task from the task queue and execute it.

Yield the processor so other Informix tasks can use it

}

if task queue is empty

return TASK_QUEUE_EMPTY

return TASKS_REMAINING

Concurrency(contd)Task can be

Process a token to check matching ruleRun a rule actionProcess a token against a set of conditionsProcess a token to run a set of rule actions triggered by token.

ConcurrencyFor k=1 to M

Create trigger T_K

from R

when R.company = "IBM"

do raise event

notify_user("userK", …. )

Partition trigger sets into N sets.

ConclusionAn architecture to build a truly scalable trigger systems.

Future workScalability for temporal conditionsAggregates