Introduction to Artificial Immune Systems (AIS)

Introduction to Artificial Introduction to Artificial Immune Systems (AIS)Immune Systems (AIS)

BIC 2005: BIC 2005: International Symposium on Bio-Inspired ComputingInternational Symposium on Bio-Inspired Computing

Johor, MY, 5-7 September 2005Johor, MY, 5-7 September 2005

Dr. Leandro Nunes de [email protected]

Catholic University of Santos - UniSantos/Brazil

BIC 2005 - Introduction to Artificial Immune Systems - Dr. Leandro Nunes de Castro 2

Introduction to the Immune System

Artificial Immune Systems A Framework to Design Artificial

Immune Systems (AIS) Representation Schemes Affinity Measures Immune Algorithms

Discussion and Main Trends

Outline

Part I

Brief Introduction to the Immune System


Brief Introduction to the Immune System: Outline

Fundamentals and Main Components Anatomy Innate Immune System Adaptive Immune System Pattern Recognition in the Immune System Basic Immune Recognition and Activation Clonal Selection and Affinity Maturation Self/Nonself Discrimination Immune Network Theory Danger Theory


The Immune System (I) Fundamentals:

Immunology is the study of the defense mechanisms that confer resistance against diseases (Klein, 1990)

The immune system (IS) is the one responsible to protect us against the attack from external microorganisms (Tizard, 1995)

Several defense mechanisms in different levels; some are redundant

The IS is adaptable (presents learning and memory) Microorganisms that might cause diseases

(pathogen): viruses, fungi, bacteria and parasites Antigen: any molecule that can stimulate the IS


Innate immune system: immediately available for combat

Adaptive immune system: antibody (Ab) production specific to a

determined infectious agent

The Immune System (II)

G ran u locytes M ac rop h ag es

In n a te

B -ce lls T-ce lls

L ym p h ocytes

A d ap ta tive

Im m u n ity


Anatomy

The Immune System (III)

Lymphatic vessels

Lymph nodes

Thymus

Spleen

Tonsils andadenoids

Bone marrow

Appendix

Peyer’s patches

Primary lymphoidorgans

Secondary lymphoidorgans


All living beings present a type of defense mechanism

Innate Immune System first line of defense controls bacterial infections regulates adaptive immunity composed mainly of phagocytes and the

complement system PAMPs and PRRs

The Immune System (IV)


Adaptive Immune System vertebrates have an adaptive immune

system that confers resistance against future infections by the same or similar antigens

lymphocytes carry antigen receptors on their surfaces.

These receptors are specific to a given antigen is capable of fine-tuning the cell receptors

of the selected cells to the selective antigens

is regulated and down regulated by the innate immunity

The Immune System (V)


Pattern Recognition: B-cell

The Immune System (VI)

Epitopes

B-cell Receptors (Ab)

Antigen


Pattern Recognition: T-cell

The Immune System (VII)


The Immune System (VIII)

after Nosssal, 1993

Basic Immune Recognition and Activation Mechanisms


Antibody Synthesis:

The Immune System (IX)

... ... ... V V

V library

D D

D library

J J

J library

Gene rearrangement

V D J Rearranged DNA

after Oprea & Forrest, 1998


Clonal Selection and Affinity Maturation

The Immune System (X)


Maturation and Cross-Reactivity of Immune Responses

The Immune System (XI)

Antigen Ag1AntigensAg1, Ag2

Primary Response Secondary Response

Lag

Responseto Ag1

Ant

ibo

dy

Con

cen

tra

tion

Time

Lag

Responseto Ag2

Responseto Ag1

...

...

Cross-ReactiveResponse

...

...

AntigenAg1’

Response toAg1’

Lag


Affinity Maturation somatic hypermutation receptor editing

The Immune System (XII)


Self/Nonself Discrimination repertoire completeness co-stimulation tolerance

Positive selection B- and T-cells are selected as

immunocompetent cells Recognition of self-MHC molecules

Negative selection Tolerance of self: those cells that recognize the

self are eliminated from the repertoire

The Immune System (XIII)


Self/Nonself Discrimination

The Immune System (XIV)

Clonaldeletion

AnergyUnaffected cell

Clonal Expansion Negative SelectionClonal Ignorance

Effector clone

Self-antigen

OR receptor editing

Nonselfantigens


Immune Network Theory The immune system is composed of an

enormous and complex network of paratopes that recognize sets of idiotopes, and of idiotopes that are recognized by sets of paratopes, thus each element can recognize as well as be recognized (Jerne, 1974)

Features (Varela et al., 1988) Structure Dynamics Metadynamics

The Immune System (XV)


Immune Network Dynamics

The Immune System (XVI)

after Jerne, 1974


The Immune System (XVII)

Danger Theory

after Matzinger, 1994


Pathogen, Antigen, Antibody Lymphocytes: B- and T-cells Affinity 1ary, 2ary and cross-reactive response Learning and memory

increase in clone size and affinity maturation Self/Nonself Discrimination Immune Network Theory Danger Signals

The Immune System— Summary —

Part II

Artificial Immune Systems


Artificial Immune Systems (AIS) Remarkable Immune Properties Concepts, Scope and Applications Brief History of AIS

An Engineering Framework for AIS The Shape-Space Formalism Measuring Affinities Algorithms and Processes

Artificial Immune Systems: Outline


Remarkable Immune Properties uniqueness diversity robustness autonomy multilayered self/nonself discrimination* distributivity reinforcement learning and memory predator-prey behavior noise tolerance (imperfect

recognition)

Artificial Immune Systems (I)


Concepts Artificial immune systems are data manipulation,

classification, reasoning and representation methodologies, that follow a plausible biological paradigm: the human immune system (Starlab)

An artificial immune system is a computational system based upon metaphors of the natural immune system (Timmis, 2000)

The artificial immune systems are composed of intelligent methodologies, inspired by the natural immune system, for the solution of real-world problems (Dasgupta, 1998)

Artificial immune systems (AIS) are adaptive systems, inspired by theoretical immunology and observed immune functions, principles and models, which are applied to problem solving (de Castro & Timmis, 2002)

Artificial Immune Systems (II)


Artificial Immune Systems (III)

Scope (de Castro & Timmis, 2002): Pattern recognition Fault and anomaly detection Data analysis (classification, clustering, etc.) Agent-based systems Search and optimization Machine-learning Autonomous navigation and control Artificial life Security of information systems


Examples of Applications Pattern recognition; Function approximation; Optimization; Data analysis and clustering; Machine learning; Associative memories; Diversity generation and maintenance; Evolutionary computation and programming; Fault and anomaly detection; Control and scheduling; Computer and network security; Generation of emergent behaviors.

Artificial Immune Systems (IV)


Artificial Immune Systems (V)

The Early Days: Developed from the field of theoretical

immunology in the mid 1980’s. Suggested we “might look” at the IS

1990 – Bersini first use of immune algorithms to solve problems

Forrest et al – Computer Security mid 1990’s

Work by IBM on virus detection Hunt et al, mid 1990’s – Machine learning


The Early Events

Part III

A Framework to Engineer AIS


Representation How do we mathematically represent immune

cells and molecules? How do we quantify their interactions or

recognition? Shape-Space Formalism (Perelson & Oster,

1979) Quantitative description of the interactions between

cells and molecules Shape-Space (S) Concepts

generalized shape recognition through regions of complementarity recognition region (cross-reactivity) affinity threshold

A Framework for AIS (I)


Recognition Via Regions of Complementarity and Shape Space (S)

Cross-Reactivity

A Framework for AIS (II)

V

V

V

V

after Perelson, 1989


Representation Set of coordinates: m = m1, m2, ..., mL, m

SL L

Ab = Ab1, Ab2, ..., AbL, Ag = Ag1, Ag2, ..., AgL

Some Types of Shape Space Hamming Euclidean Manhattan Symbolic

A Framework for AIS (III)


A Framework for AIS (IV)

Affinities: related to distance/similarity Examples of affinity measures

Euclidean

Manhattan

Hamming

L

iii AgAbD

1

2)(

L

iii AgAbD

1

L

i

ii AgAbD

1 otherwise0

if1δwhereδ,


Affinities in Hamming Shape-Space

A Framework for AIS (V)

Hamming r-contiguous bit Affinity measure distance rule of Hunt

i

lH

iDD 2

Flipping one string


Algorithms and Processes Generic algorithms based on specific

immune principles, processes or theoretical models

Main Types Bone marrow algorithms Thymus algorithms Clonal selection algorithms Immune network models

A Framework for AIS (VI)


A Bone Marrow Algorithm

A Framework for AIS (VII)

An individual genome corresponds to four libraries:

Library 1 Library 2 Library 3 Library 4

A1 A2 A3 A4 A5 A6 A7 A8

A3 D5C8B2

A3 D5C8B2

A3 B2 C8 D5

= four 16 bit segments

= a 64 bit chain

Expressed Ab molecule

B1 B2 B3 B4 B5 B6 B7 B8 C1 C2 C3 C4 C5 C6 C7 C8 D1 D2 D3 D4 D5 D6 D7 D8

after Perelson et al., 1996


Thymus Algorithms: Negative Selection Store information about the patterns to be

recognized based on a set of known patterns

A Framework for AIS (VIII)

Selfstrings (S)

Generaterandom strings

(R0)Match Detector

Set (R)

Reject

No

Yes

No

Yes

Detector Set(R)

SelfStrings (S)

Match

Non-selfDetected

Censoring Monitoring phase phase

after Forrest et al., 1994


A Clonal Selection Algorithm

A Framework for AIS (IX)

after de Castro & Von Zuben, 2001a


Somatic Hypermutation Hamming shape-space with an alphabet of

length 8

Real-valued vectors: inductive mutation

A Framework for AIS (X)


Affinity Proportionate Hypermutation

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

D*

= 5

= 10

= 20

A Framework for AIS (XI)

0 20 40 60 80 100 120 140 160 180 200

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Iterations

after de Castro & Von Zuben, 2001a after Kepler & Perelson, 1993


A Discrete Immune Network Model: aiNet

A Framework for AIS (XII)

1. For each antigenic pattern Agi,1.1 Clonal selection: Apply the pattern recognition version

of CLONALG that will return a matrix of memoryclones for Agi;

1.2 Apoptosis: Eliminate all memory clones whose affinitywith antigen are below a threshold;

1.3 Inter-cell affinity: Determine the affinity between allclones generated for Agi;

1.4 Clonal Suppression: Eliminate those clones whoseaffinities are inferior to a pre-specified threshold;

1.5 Total repertoire: Concatenate the clone generated forantigen Agi with all network cells

2. Inter-cell affinity: Determine the affinity between all networkcells;

3. Network suppression: Eliminate all aiNet cells whose affinitiesare inferior to a pre-specified threshold.


Guidelines to Design an AIS

A Framework for AIS (XIII)

1. Problem definition2. Mapping the real problem into the AIS domain

2.1 Defining the types of immune cells and moleculesto be used

2.2 Deciding the immune principle(s) to be used in thesolution

2.3 Defining the mathematical representation for theelements of the AIS

2.4 Evaluating the interactions among the elements ofthe AIS (dynamics)

2.5 Controlling the metadynamics of the AIS3. Reverse mapping from AIS to the real problem

Part IV

Discussion and Main Trends


Discussion

Growing interest for the AIS Biologically Inspired Computing

utility and extension of biology improved comprehension of natural

phenomena Example-based learning, where different

pattern categories are represented by adaptive memories of the system

A new computational intelligence approach


The use of a general framework to design AIS

Main application domains Optimization, Data Analysis, Machine-

Learning, Pattern Recognition Main trends

Innate immunity, hybrid algorithms, use of danger theory, formal aspects of AIS, mathematical analysis, development of more theoretical models

Main Trends


Dasgupta, D. (Ed.) (1998), Artificial Immune Systems and Their Applications, Springer-Verlag.

de Castro, L. N., & Von Zuben, F. J., (2001a), “Learning and Optimization Using the Clonal Selection Principle”, submitted to the IEEE Transaction on Evolutionary Computation (Special Issue on AIS).

de Castro, L. N. & Von Zuben, F. J. (2001), "aiNet: An Artificial Immune Network for Data Analysis", Book Chapter in Data Mining: A Heuristic Approach, Hussein A. Abbass, Ruhul A. Sarker, and Charles S. Newton (Eds.), Idea Group Publishing, USA.

Forrest, S., A. Perelson, Allen, L. & Cherukuri, R. (1994), “Self-Nonself Discrimination in a Computer”, Proc. of the IEEE Symposium on Research in Security and Privacy, pp. 202-212.

Hofmeyr S. A. & Forrest, S. (2000), “Architecture for an Artificial Immune System”, Evolutionary Computation, 7(1), pp. 45-68.

Jerne, N. K. (1974a), “Towards a Network Theory of the Immune System”, Ann. Immunol. (Inst. Pasteur) 125C, pp. 373-389.

Kepler, T. B. & Perelson, A. S. (1993a), “Somatic Hypermutation in B Cells: An Optimal Control Treatment”, J. theor. Biol., 164, pp. 37-64.

Klein, J. (1990), Immunology, Blackwell Scientific Publications. Matzinger, P. (1994), “Tolerance, Danger and the Extended Family”,

Annual Reviews of Immunology, 12, pp. 991-1045.

References (I)


References (II)

Nossal, G. J. V. (1993a), “Life, Death and the Immune System”, Scientific American, 269(3), pp. 21-30.

Oprea, M. & Forrest, S. (1998), “Simulated Evolution of Antibody Gene Libraries Under Pathogen Selection”, Proc. of the IEEE SMC’98.

Perelson, A. S. (1989), “Immune Network Theory”, Imm. Rev., 110, pp. 5-36. Perelson, A. S. & Oster, G. F. (1979), “Theoretical Studies of Clonal Selection:

Minimal Antibody Repertoire Size and Reliability of Self-Nonself Discrimination”, J. theor.Biol., 81, pp. 645-670.

Perelson, A. S., Hightower, R. & Forrest, S. (1996), “Evolution and Somatic Learning in V-Region Genes”, Research in Immunology, 147, pp. 202-208.

Starlab, URL: http://www.starlab.org/genes/ais/ Timmis, J. (2000), Artificial Immune Systems: A Novel Data Analysis Technique

Inspired by the Immune Network Theory, Ph.D. Dissertation, Department of Computer Science, University of Whales, September.

Tizard, I. R. (1995), Immunology An Introduction, Saunders College Pub., 4th Ed.

Varela, F. J., Coutinho, A. Dupire, E. & Vaz, N. N. (1988), “Cognitive Networks: Immune, Neural and Otherwise”, Theoretical Immunology, Part II, A. S. Perelson (Ed.), pp. 359-375.

de Castro, L. N., & Timmis, J. (2002), Artificial Immune Systems: A New Computational Intelligence Approach, Springer-Verlag.

[email protected]

Thank You!

Questions? Comments?

Documents

Introduction to Artificial Immune Systems (AIS)