An Introduction to InductionPrinciples

Patricia HILL

School of ComputingUniversity of Leeds, United Kingdom

Roberto BAGNARA

Department of MathematicsUniversity of Parma, Italy

Copyright c� 2004 Patricia Hill. Distributed under the terms of the GNU Free Documentation License 1.2 1

MATHEMATICAL INDUCTION

Let � ��� � be a property of natural numbers� � � ,� , , . . . Theprinciple of mathematical induction says that to show � �� � holds forall natural numbers, it is sufficient to show:

The base case: � � � � holds.

The step case: If � ��� � holds then so does � �� � � � , for anynatural number� .

� ��� � is called inductive hypothesis.

Formally,

� � � ��� ��� � �� � � �� �� �� �

ind. hyp.

� � � ��� � � � � � � � � � � � � � �� ���

MATHEMATICAL INDUCTION 2

MATHEMATICAL INDUCTION: EXAMPLE 1

Let us show that � �� � � � � � �! �#" " " � � � � �� � � � � :

Inductive hypothesis: � ��� � � � $%& ' ( � � �� � � � � .

Base case,� � � : � � � � � � � � � � � � � � � .

Step case,� ) � : Assume � ��� � holds.

$ *+% & '

( � � � � " " " � � � �� � � �

� � � � � � � � � � � � ��� � � � [by rearranging]

� � ��� � � � � �� � � � [by ind. hyp. � ��� � ]

� �� � � � ��� � � [by rearranging]

so that � ��� � � � holds.

MATHEMATICAL INDUCTION: EXAMPLE 1 3

EXERCISES

Prove the following identities by mathematical induction:

� �, �- �#" " " � � � � � � � ��� � � �/. 0

�� " � � " , � �, " ! �#" " " � ��� � � � �� � �

� � � �� � 0

� �1 �1 . �#" " " �1 2 � � 31 2 * +� 31 �

EXERCISES 4

MATHEMATICAL INDUCTION: EXAMPLE 2

Let 4 ��� � be the property that, for all symbols 5 687 689 + 6 � � � 69 2 , we have

� 5 9 + " " " 9 2 � 9 + " " " 9 2 7 � � � 5 � 7 . We show that� � � � � 4 ��� � .

Inductive hypothesis:

4 ��� � � � � 5 9 + " " " 9 $ � 9 + " " " 9 $ 7 � � � 5 � 7 � .

Base case,� � � : 4 � � � � � 5 � 7 � � 5 � 7 � .

Step case,� ) � : Suppose

5 9 + " " " 9 $ 9 $ *+ � 9 + " " " 9 $ 9 $ *+ 7 0

then7 � 9 $ *+ and therefore

5 9 + " " " 9 $ � 9 + " " " 9 $ 9 $ *+ � 9 + " " " 9 $ 7 �By the inductive hypothesis, 5 � 7 , and thus 4 �� � � � holds.

MATHEMATICAL INDUCTION: EXAMPLE 2 5

COMPLETE MATHEMATICAL INDUCTION

Let � ��� � be a property of natural numbers� � � ,� , , . . . Theprinciple of complete mathematical induction says that to show � ��� �

holds for all natural numbers, it is sufficient to show:

The base case: � � � � holds.

The step case: If � ��: � holds for each: � � ,� , . . . � then sodoes � �� � � � , for any natural number� .

The conjunction ; $ < & ' � �: � is called inductive hypothesis.

Formally,

� � � �� � � �� �$

< & '� �: �

� �� �

ind. hyp.

� � � �� � � � � � � � � � � � ��� � �

COMPLETE MATHEMATICAL INDUCTION 6

COMPLETE MATHEMATICAL INDUCTION: EXAMPLE

We prove that, in IMP, arithmetic evaluation is deterministic:�= 5 68> ? 3 @ � � = 5 6> ? 3 @ � A � � � � � � A 6

for all arithmetic expressions 5 , stores> and numbers� and� A .

Inductive hypothesis: ; $ < & ' � �: � , where � �: � is this property fora derivation tree of depth: .

Base case (a): � �� � is the case when 5 is a number� , so that, if

= 5 6> ? 3 @ � A then� � � A ;

Base case (b): � �� � is also the case when 5 is a variable B .

If= 5 6> ? 3 @ � then the only rule for a variable requires> � B � � �

as a premise. Similarly, if= 5 68> ? 3 @ � A then the only rule for avariable requires> � B � � � A as a premise. Therefore� � � A .

COMPLETE MATHEMATICAL INDUCTION: EXAMPLE 7

COMPLETE MATHEMATICAL INDUCTION: EXAMPLE, CONT.

Step case (b): 5 is a sum 5 ' � 5+ with der. tree of depth: � � .

...

= 5 ' 6> ? @ � '

...

= 5+ 68> ? @ � +

if� � � ' � � += 5 ' � 5+ 6> ? @ �...

= 5 ' 6> ? @ � A'...

= 5+ 6> ? @ � A+ if� A � � A' � � A+= 5 ' � 5+ 6> ? @ � A

By induction hypothesis for derivation trees of depth: for

= 5 ' 68> ? @ � ' and= 5 ' 6> ? @ � A' we have� ' � � A' . Similarly,

� + � � A+ . Therefore� � � A .This same reasoning applies when 5 is 5 ' 3 5+ or 5 'DC 5+ .

COMPLETE MATHEMATICAL INDUCTION: EXAMPLE, CONT. 8

STRUCTURAL INDUCTION

The principle of structural induction is:

In order to show that a property is true of all expressions, itsuffices to show it is true of all atomic expressions and ispreserved by all methods of forming the expressions.

STRUCTURAL INDUCTION 9

STRUCTURAL INDUCTION: EXAMPLE 1

Structural induction can also be used to prove that the arithmeticevaluation in IMP is deterministic.

To show that � � 5 � holds for all 5 �E FG H we need to show:

� ��� � holds;

� � B � holds;

if � � 5 ' � and � � 5+ � hold, then � � 5 ' � 5+ � holds;

if � � 5 ' � and � � 5+ � hold, then � � 5 ' 3 5+ � holds;

if � � 5 ' � and � � 5+ � hold� � � �

ind. hyp.

, then � � 5 'C 5+ � holds.

STRUCTURAL INDUCTION: EXAMPLE 1 10

STRUCTURAL INDUCTION: EXAMPLE 1, CONT. I

We prove that�= 5 68> ? 3 @ � � = 5 6> ? 3 @ � A � � � � � � A 6

for all arithmetic expressions 5 , stores> and numbers� 6 � A .Let � � 5 �� � 5 68> 6 � 6 � A � �= 5 6> ? 3 @ � � = 5 68> ? 3 @ � A � � � � � � A .First the cases where 5 is an atomic expression:

For all> �I JKL FM , � �� � holds as only one rule applies in thiscase;

for all> �I J KL F M , � � B � holds as only one rule applies in this case.

Then the cases where 5 is formed by composing subexpressions. . .

STRUCTURAL INDUCTION: EXAMPLE 1, CONT. I 11

STRUCTURAL INDUCTION: EXAMPLE 1, CONT. II

Suppose that 5 � 5 ' � 5+ . Then only one rule applies:

– there exist� ' and� + such that

= 5 ' 68> ? 3 @ � ' � = 5+ 6> ? 3 @ � + and� � � ' � � + ;

– there exist� A' and� A+ such that

= 5 ' 68> ? 3 @ � A' � = 5+ 6> ? 3 @ � A+ and� A � � A' � � A+ .

As 5 ' and 5+ are proper subexpressions (sub-structures) of 5 , wecan assume that the inductive hypothesis applies to � � 5 ' � and

� � 5+ � , so that:

– if �= 5 ' 6> ? 3 @ � ' � = 5 ' 68> ? 3 @ � A' � then� ' � � A' ;

– if �= 5+ 6> ? 3 @ � + � = 5+ 68> ? 3 @ � A+ � then� + � � A+ .

So� � � ' � � + � � A' � � A+ � � A .

Similarly for 5 � 5 ' 3 5+ and 5 � 5 'DC 5+ .

STRUCTURAL INDUCTION: EXAMPLE 1, CONT. II 12

STRUCTURAL INDUCTION: EXAMPLE 2

Prove that the evaluation of arithmetic expressions terminates. If

5 �E F G H , let � � 5 � denote the property:

for all> �I J KL F M , there exist� �N O J such that there is a(finite) derivation tree for= 5 6> ? 3 @ � .

� ��� � hold;

For all> �I JKL FM , � � B � holds;

Suppose that 5 � 5 ' � 5+ .

Then � � 5 ' � and � � 5+ � hold so that, for all> �I J KL F M , there exist

� ' ,� + and derivation trees for= 5 ' 6> ? 3 @ � ' and

= 5+ 68> ? 3 @ � + . Using the rule for ‘ � ’ we have a derivation treefor= 5 68> ? 3 @ � , with� � � ' � � + . Therefore, for all> �I JKL FM ,

� � 5 � holds. Similarly for 5 � 5 ' 3 5+ and 5 � 5 'C 5+ .

STRUCTURAL INDUCTION: EXAMPLE 2 13

STRUCTURAL INDUCTION: EXERCISE

Definition:

1. “ J J ” and “ P ” are boolean expressions;

2. If7 is a boolean expression, “ � O K J 7 � ” is a boolean expression;

3. If7 ' and7 + are boolean expressions, “ �7 'RQ O S 7 + � ” is a booleanexpression;

4. If7 ' and7 + are boolean expressions, “ �7 ' KL 7 + � ” is a booleanexpression.

5. Nothing is a boolean expression unless it is constructed by rules1–4.

Prove using structural induction that in every boolean expression thenumber of ‘(’ equals the number of ‘)’.

STRUCTURAL INDUCTION: EXERCISE 14

WELL-FOUNDED INDUCTION: INTRODUCTION

Mathematical and structural induction are instances of well-foundedinduction.

Mathematical induction relies on the fact that for any number�

every descending sequence� T � 3 � T � 3 T " " " is finite.

Structural induction relies on the fact that for any expression 5 '

every subexpression sequence of the form

5 'VU 5+ U 5 . U " " "is finite. ( 5 A W 5 denotes 5 A X� 5 is a subexpression of 5 .)

These rely on an ordering relation that is well-founded.

WELL-FOUNDED INDUCTION: INTRODUCTION 15

WELL-FOUNDED RELATIONS

A well-founded relation is a binary relation ‘ Y ’ on a set Z such thatthere are no infinite descending chains

5 '\[ 5+ [ " " " [ 5 %[ " " "

This is equivalent to saying:

Every non-empty subset of A has a least element.

If ‘ Y ’ is well-founded, then the transitive closure ‘ Y * ’ iswell-founded.

We write ‘ ] ’ for the reflexive closure of ‘ Y ’.

5 ] 7 ^ � 5 � 7 _ 5 Y 7Exercise: Show that ‘ ] ’ is not a well-founded relation.

WELL-FOUNDED RELATIONS 16

PARTIAL ORDERING

Suppose ` is a binary relation on a set Z . ` is:

1. Reflexive: if, for all 5 � Z , 5 ` 5 0

2. Transitive: if, for all 5+ 6 5 . 6 5 a � Z �

� 5+ ` 5 . � 5 . ` 5 a � � � 5+ ` 5 a 0

3. Antisymmetric: if, for all 5+ 6 5 . � Z �

� 5+ ` 5 . � 5 . ` 5+ � � � 5+ � 5 . �

A partial ordering is a binary relation that is reflexive, transitive andantisymmetric.

A complete ordering on a set Z is a partial ordering ] on Z suchthat, for all 5+ 6 5 . � Z , either 5+ ] 5 . or 5 . ] 5+ .

PARTIAL ORDERING 17

WELL-FOUNDED RELATIONS AND PARTIAL ORDERINGS

Exercise: Let ‘ Y ’ a well-founded relation over a set Z .

Show that ‘ Y * ’, the transitive closure of ‘ Y ’, is well-founded.

Show that ‘ Ycb ’, the reflexive and transitive closure of ‘ Y ’, is apartial ordering.

WELL-FOUNDED RELATIONS AND PARTIAL ORDERINGS 18

WELL-FOUNDED RELATIONS: EXAMPLES

Let Z � � be the set of natural numbers and ‘ Y ’ be the relation

� Y � if and only if� � � � � .

Let Z � � be the set of natural numbers and ‘ Y ’ be the relation‘ d ’.

Let Z � E FG H be the set of all arithmetic expressions in IMP. Let

7 W 5 be defined to hold if and only if7 is an immediatesubexpression of 5 .

Then ‘ W ’ is a well-founded relation.

Let Z be the set of all finite character strings.

Let e f 9 be defined to hold if and only if e is a proper substring of

9 .

Then ‘ f ’ is a well-founded relation.

WELL-FOUNDED RELATIONS: EXAMPLES 19

WELL-FOUNDED INDUCTION

The principle of well-founded induction is:

Let Y be a well-founded relation on a set Z . Let � be aproperty. Then � � 5 � holds for all 5 in Z if and only if

� 5 � Z � �� 7 Y 5 � � �7 � � � � � � 5 � � �

So, to prove that a property holds of all elements of a well-foundedset (i.e., a set with a well-founded ordering) we just have to provethat if it holds for all predecessors of any 5 � Z in this ordering, thenit holds for 5 .

WELL-FOUNDED INDUCTION 20

EXAMPLE: GCD I

Consider the IMP program gh i that computes the greatest commondenominator of two numbers:

gh i � while O K J ��j � k � do

ifj l k

then k � � k 3 j

elsej � � j 3 k

Then we show that, for all> �I J KL F M , the command gh i terminates:

� > �I JKL FM � > ��j � ) � � > � k � ) � � � m > A � = gh i 6> ? @ > A�

EXAMPLE: GCD I 21

EXAMPLE: GCD II

Letnpo qr� s > �I J KL F Mutt > ��j � ) � 6> � k � ) � v �

We have to show that

m > A � = gh i 6> ? @ > A

holds for all stores> � n . We call this property � �> � .The relation ‘ Y ’ on n is defined as follows, for each> A 6> � n :

> A Y > ^ � �> A � j � l > ��j � � > A � k � l > � k � �

� �> A ��j � X� > � j � _ > A � k � X� > � k � � � �

Exercise: Show that ‘ Y ’ is well-founded.

EXAMPLE: GCD II 22

EXAMPLE: GCD III

Case> � j � � > � k � .Then

= not ��j � k � 68> ? @ P �

From the rules for while statements, we have

= not ��j � k � 68> ? @ P

= gh i 6> ? @ > �Hence, in this case, � �> � holds.

EXAMPLE: GCD III 23

EXAMPLE: GCD IV

Case> � j � X� > � k � .Then= not ��j � k � 68> ? @ J J .Let w be the if statement

ifj l k then k � � k 3 j elsej � � j 3 k �

From the rules, we can derive= w 6> ? @ > A A 6 where

> A A � xy z

> {> � k � 3 > ��j � | k } 6 if> � j � d > � k � 0

> {> ��j � 3 > � k � | j } 6 if> � k � d > ��j � �

In both cases,> A A Y > so we can apply the inductive hypothesis to

> A A : for some> A � n ,

= gh i 6> A A ? @ > A�

EXAMPLE: GCD IV 24

EXAMPLE: GCD V

Case> � j � X� > � k � cont.

From the rules for while statements, we have

...

= not � j � k � 6> ? @ J J...

= w 68> ? @ > A A

...

= gh i 68> A A ? @ > A

= gh i 68> ? @ > ATherefore we have a derivation of= gh i 6> ? @ > A and hence, in thiscase � �> � holds.

For all> � n where> � j � X� > � k � , by well-founded induction, � �> �

holds.

EXAMPLE: GCD V 25

DEFINITIONS BY INDUCTION

Consider the following definition: for all 5 �E FG H we define:

~ F O � J� � 5 �o qr�x��������

y ��������z� 6 if 5 � � 0

� 6 if 5 � B 0

~ F O � J � � 5 ' � � ~ F O � J� � 5+ � 6 if 5 � � 5 ' � 5+ � 0

~ F O � J � � 5 ' � � ~ F O � J� � 5+ � 6 if 5 � � 5 ' 3 5+ � 0

~ F O � J � � 5 ' � � ~ F O � J� � 5+ � 6 if 5 � � 5 'C 5+ ���

Definitions of this form are often called inductive or recursive.

DEFINITIONS BY INDUCTION 26