Artificial Immune Systems (AIS)

Artificial Immune Systems (AIS)

AJ Graaff

AIS - Lectures

• The Natural Immune System– 6 Sept, Biological overview of immunity

• The Artificial Immune System– 13 Sept, Models based on different theories of

the natural immune system• The Artificial Immune System (cont.)

– 20 Sept, more on network based AIS models

The Natural Immune SystemOverview

• Classical view of immunity– Basic components of the immune system– Antibodies/Antigens as Self/Non-self– White Cells and Immunity Types

• Clonal Selection theory– Process of Affinity Maturation

• Idiotypic Network theory– Consequences of Affinity Maturation

• Danger Theory– From Self/Non-self to Dangerous/Non-dangerous

The Natural Immune SystemClassical view

• Distinguishes between normal (Self) and foreign (Non-self)– Self: all normal cells in the body– Foreign: pathogenic material (bacteria,

viruses, parasites)• Lymphocytes detect non-self (antigens)

– Secrete antibodies which bind to antigen– Complementary match in shape – Binding strength, affinity






The Natural Immune SystemClonal Selection

• Learning in the immune system– Increase lymphocyte count with frequent antigen recognition

• Process of Affinity Maturation– Selection for Cloning– Somatic hyper-mutation of clones

• Different to Classical View– Not only antibodies are secreted, but mutated clones are

generated• Lymphocyte count regulated

– Increase in size of some clones, decreases other clones– Previously learned antigen structures are forgotten– Survival of the fittest lymphocytes

The Natural Immune SystemIdiotypic Network

• Somatic hyper-mutation in clonal selection– Occurs in variable region– Mutated variable region can be antigenic– Invoke immune response from neighbouring

lymphocytes• Antigenic region known as idiotype

– Binding to idiotypes form idiotypic networks– Antibody-Antibody OR Antibody-Lymphocyte

The Natural Immune SystemIdiotypic Network

The Natural Immune SystemDanger Theory

• Different to Classical View– Distinguish between dangerous and non-

dangerous– Non-self could be non-dangerous (ex.

Yoghurt)• Respond only to danger

– Occurs when necrotic cell death occurs– Co-stimulation signal from antigen presenting

cell

The Natural Immune SystemDanger Theory

The Artificial Immune SystemOverview

• Summarise NIS capabilities• Generic AIS framework

– Basic AIS algorithm• Classical View models

– Negative Selection– Evolutionary Approaches

• Clonal Selection models– CLONALG– MARIA

The Artificial Immune SystemOverview(cont.)

• Idiotypic Network based models– Artificial Immune Network (AINE)– Self Stabilising AIS (SSAIS and SMAIN)– aiNet

• Danger Theory Models– Adaptive Mailbox– Intrusion Detection

The Artificial Immune SystemNIS capabilities

• Needs to know structure of self/non-self cells• Can distinguish between self and non-self• Non-self can be sensed as dangerous/non-

dangerous• Learn and adapt through cloning and mutation• Build-up of memory

– Faster secondary response• Cooperation and co-stimulation among

lymphocytes– Network formation

The Artificial Immune SystemGeneric AIS

• Trained detectors (artificial lymphocytes) to detect non-self patterns

• Needs repository of self and/or non-self patterns to train artificial lymphocytes (ALCs)

• Measure affinity – between ALC and pattern– between two ALCs– requires that ALCs and patterns have similar structure

• Memory• ALC can be cloned and mutated

– More diversity in search space

The Artificial Immune SystemBasic AIS algorithm

• Initialize a set of ALCs as population B;• Determine the antigen patterns as training set A;• while some stopping condition(s) not true do

– for each antigen pattern aj A ∈ do• Select a subset of ALCs for exposure to aj, as population S B;⊆• for each ALC, bi S ∈ do

– Calculate the antigen affinity between aj and bi;• end• Select a subset of ALCs with the highest calculated antigen affinity as

population H S;⊆• Adapt the ALCs in H with some selection method, based on the calculated

antigen affinity and/or the network affinity among ALCs in H ;• Update the stimulation level of each ALC in H ;

– end• end

The Artificial Immune SystemNegative Selection

• Censoring process– Determine size of ALC set– While ALC set size not met

• Generate random ALC• Measure affinity between the ALC and each

pattern in self set• If affinity higher (lower) than the affinity threshold

– Add to ALC set

• Affinity measured using r-continuous matching rule

The Artificial Immune SystemNegative Selection

• r-continuous matching rule

The Artificial Immune SystemEvolutionary Approaches

• ALC not randomly generated for screening• ALCs are evolved towards non-self patterns

– Maintain diversity and generality among ALCs

• Multiple class problem– Evolve ALC towards patterns of a class, further away

from patterns of different classes– Until most of non-self are detected– ALC set represents specific class– Repeat for other classes in the set

The Artificial Immune SystemEvolutionary Approaches(cont.)

• Adaptive negative selection (GAIS)• Each ALC has affinity threshold• Hamming distance as affinity measure

The Artificial Immune SystemCLONALG

• Initialise a set of random ALCs (set C)• Subset of random ALC set as memory set M (size equal to number

of patterns in training set), remainder for affinity maturation (set R)• Each pattern in training set

– Affinity with all ALCs in C– Select n of the ALCs with highest affinity as H– Number of clones for each ALC in H proportional to calculated affinity– Each clone mutated, mutation rate inversely proportional to affinity– Affinity between mutated clone and training pattern is calculated– Mutated ALC clone with highest affinity replace corresponding ALC in M

if affinity with training is higher– Number of ALCs with lowest affinity replaced with randomly generated

ALCs• Repeat until maximum number of generations reached

The Artificial Immune SystemMARIA

• Multi-layered– Free-antibody layer, B-Cell layer and Memory

layer• Layers interact to adapt and learn

structure of antigen patterns• Each layer has affinity and death

thresholds• Euclidean distance as affinity measure

The Artificial Immune SystemMARIA (cont.)

• Antigen first enters free-antibody layer– Randomly presented to n free-antibodies– Number of free-antibody bindings stored as nb– If free-antibody binds, then remove from layer

• Antigen then enters B-Cell layer– Randomly presented to each B-Cell until it binds to one of B-

Cells– Mutated clone of B-Cell is generated if nb above stimulation

threshold (mutated clone added to B-Cell layer)– Activated B-Cell generates mutated clones as free-antibodies

which are added to free-antibody layer– If antigen does not bind to any B-Cell then add antigen as B-Cell

to B-Cell layer and generate mutated clones as free-antibodies

The Artificial Immune SystemMARIA (cont.)

• Antigen with clone of activated B-Cell enter Memory layer– Memory cell with lowest affinity to activated B-

Cell clone is selected• If affinity higher than memory affinity threshold

then add B-Cell clone as new memory cell• If affinity lower than memory affinity threshold and

affinity of B-Cell clone less than affinity of selected memory cell with antigen pattern, then replace memory cell with B-Cell clone

The Artificial Immune SystemAINE

• Concept of Artificial Recognition Balls (ARB)• Resource limited environment• ARB allocates resources• ARB represents region of B-Cells• AINE consists of

– Population of ARBs– Links between ARBs– Set of antigen patterns– Clonal operations

The Artificial Immune SystemAINE (cont.)

• Euclidean distance as affinity measure• ARBs connected (linked) if affinity below

Network Affinity Threshold (NAT)• NAT influences number of ALC networks• After each iteration of antigen set

– Stimulation level of each ARB calculated– Number of resources allocated based on stimulation

level– Weakest ARBs (zero resources) removed from ARB

population– Mutated clones of remaining ARBs integrated into

ARB population (re-linking with remaining ARBs)

The Artificial Immune SystemAINE (cont.)

• Stimulation of an ARB– Antigen stimulation: sum of all antigen

affinities (below NAT)– Network stimulation: affinities with linked

ARBs– Network suppression: dissimilarity with linked

ARBs• Need to set upper limit on resource pool• SSAIS/SMAIN improves on AINE

The Artificial Immune SystemSSAIS and SMAIN

• No shared pool of resources• No limit on number of resources• Resource of ARB with highest stimulation is

increased• Resource level local to each ARB• ARBs with resource level lower than mortality

threshold are culled from the population• Stimulation of an ARB

– Network suppression discarded– Network stimulation: average of the summation of

affinities with linked ARBs

The Artificial Immune SystemSSAIS and SMAIN (cont.)

• Resource levels of ARBs also geometrically decayed with certain rate

• ARB with highest stimulation level– Generates mutated clones– Mutated clones linked to ARBs in population

• Poor data compression• Sometimes overfitting the data

The Artificial Immune SystemSSAIS and SMAIN (cont.)

• SMAIN– No mutation operator on clones of highest

stimulated ARB– Highest stimulated ARB is cloned and half of

the parent’s resources are assigned to the ARB clone

– Stimulation level• Network stimulation: summation of affinities with

linked ARBs

• SMAIN also tends to overfit the data

The Artificial Immune SystemaiNet

• Evolves population of linked memory ALCs (clonal selection) to cluster data

• ALC networks– ALCs connected by edges (ALC pairs)– Weight value assigned to each edge (indication of

similarity)– Results into edge-weighted graph

• Edges are pruned (similarity threshold)• Pruning results into sub-networks• Sub-network potential cluster in data

The Artificial Immune SystemaiNet (cont.)

• Two phases– Clonal selection (based on CLONALG)– Network formation/suppression

• Clonal selection– Subset of highest affinity ALCs are cloned and mutated– Clonal memory set selected from mutated clones (mutated

clones with highest affinity)– Memory clones with affinity lower than threshold removed

• Network formation/suppression– Remaining memory clones linked.– Weighted edges pruned (below threshold)– Remaining memory clones added to existing memory ALCs

• After each iteration of training antigens, percentage of lowest affinity memory ALCs replaced by random ALCs

The Artificial Immune SystemaiNet (cont.)

• Final network of memory ALCs– Minimal spanning tree– Hierarchical Agglomerative Clustering

• Other models– MOM-aiNet (multi-objective, multipopulation)– Opt-aiNet (multi-modal function optimisation)– Dopt-aiNet (improve opt-aiNet, non-stationary

environments)

The Artificial Immune SystemAdaptive Mailbox

• Distinguish between interesting and uninteresting emails

• Two phases– Initialisation phase

• Monitor user action for each new email• User deletes email, generate ALC which can

detect deleted email• Add ALC to ALC set

– Clone and mutate existing ALCs in set to improve generalisation

• ALC set represents uninteresting emails

The Artificial Immune SystemAdaptive Mailbox (cont.)

• Two phases– Running phase

• Label all deleted emails as uninteresting• Buffers uninteresting emails as antigen patterns• When buffer reaches specific size, present antigen

patterns to ALC set of uniteresting emails (init phase)

• ALC set adapts to buffer (clonal selection)• Thus, ALC set adapts to user behaviour

The Artificial Immune SystemAdaptive Mailbox (cont.)

• Danger in mailbox– Determined by number of unread emails– Number of unread emails reaches limit

• Unread emails presented as antigen to ALC set• Unread emails classified as uninteresting if highest

affinity with an ALC exceeds affinity threshold• Unread emails which are classified as

uninteresting then automatically moved to temp folder or deleted

The Artificial Immune SystemIntrusion Detection

• Simple IDS– Monitor incoming traffic of specific host– Creates profile of normal user traffic– Signals alarm for abnormal traffic (i.e. traffic not part of profile)– Drawback: normal traffic changes through time, profile gets

outdated• Danger theory approach

– Danger signal of abnormal CPU usage, memory usage or certain security attacks

– IDS only signal alarm if danger signal is also received– No danger signal means that profile needs to be updated with

abnormal traffic as normal traffic

The Artificial Immune System Network based AIS models

(overview)

• Network AIS in context of data clustering• Drawback of existing Network AIS models• Alternative network topologies• Local Network Neighbourhood AIS

The Artificial Immune System Data clustering

• Partitioning of data set such that similar patterns are grouped together and are more similar compared to patterns across different groups

• Group (cluster) identified by centroid• ALC networks represent potential clusters in

data• Antigen affinity measures similarity

• Drawback of existing Network AIS models– Proximity matrix for network linking– Number of parameters– Determining number of clusters

• Existing models do not need user specified value for number of clusters, but

• Techniques used to determine final clustering do however

The Artificial Immune SystemNetwork AIS Disadvantages

• Proximity matrix of affinity

The Artificial Immune SystemNetwork AIS Disadvantages

The Artificial Immune System Alternative network topologies

• Linear Topology


• Simple Cyclic Topology


• Cayley Tree Topology

The Artificial Immune System Local Network Neighbourhood AIS• Index based ALC neighbouring technique• No network affinity threshold• Model also not dependant on number of user

specified clusters– Technique to determine ALC network (cluster)

boundaries much simpler• Has only three user parameters

– Maximum ALC population size– ALC activation threshold– ALC neighbourhood size

The Artificial Immune System Local Network Neighbourhood

AIS• Index based ALC neighbouring technique

The Artificial Immune System Local Network Neighbourhood

AIS• Technique to determine ALC network

(cluster) boundaries much simpler

Documents

Artificial Immune Systems (AIS)