COEN 352 Data Structures and Algorithms: Mathematical...

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

COEN 352 Data Structures and Algorithms:Mathematical Preliminaries

Stuart Thiel

Concordia UniversityDepartment of Electrical Computer Engineering

Summer, 2015

1/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Outline

Notation

Sets

Sets and Relations

Identities

Recursion

Proof Techniques

2/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Size vs. Time

I Size is often a power of 2

I Time is usually decimal

I Shaffer talks about his preferred notation in 2.2, I willattempt to avoid ambiguity

3/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Stirling’s Approximation of n!

I n! ≈√

2πn(ne )n

I We often write n! but are using the approximation forresulting analysis

I proving this not for the faint of heart, but fascinating!

I This and other stuff Stirling did is critical to theunderstanding of advanced algorithm analysis.

4/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Combinatorics

I Permutation S is the members of S arranged in someorder

I n members leads to n! possible permutations

I Often need to generate random permutations foranalysis

I How many do you need to test?

I What does “random” mean?

I How real is “random”?

5/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Notations

I A boolean value is true or false

I A =⇒ B means if A then B

I A ∧ B means A and B

I A ∨ B means A or B

I A means ∼ A means not A, or the negation of AI a%b is the modulus operator, gives remainder of integer

divisionI different languages can do modulus differently, be

careful

6/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

More Notations

I dxe ceiling rounds up to the nearest larger integer

I bxc floor rounds down to the nearest smaller integer

7/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Sets

I useful abstraction for algorithm design

I distinguishable elements of a base type.

I elements are primitive elements of base type

I elements may also be sets

I no duplication

I no order

8/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Example

I P = {7, 11, 42}I elements are 7, 11 and 42

I P is the set

I Integers are the base type

I Sets can be empty: ∅

9/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Definitions

I enumerating each element can be long

I formal definition can communicate well

I “All positive integers”

I “All even integers”

I “All prime numbers”

10/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Definition Example

I P = {x |x is a positive integer}I P = {x |x > 0}I more specific: P = {x |x > 0 ∧ x ∈ Z}

11/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Operations with Elements

I in a set

I not in a set

I cardinality of a set

12/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Operations with Elements Example

I x ∈ P

I x /∈ P

I Q = {7, 11, 42}I |Q| = 3

13/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Operations with Other Sets

I subset

I supersets

I union

I intersection

I difference

14/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Operations with Other Sets Example 1

I P = {x |x > 0 ∧ x ∈ Z}I Q = {x |x ∈ Z}I P ⊆ Q

I Q ⊇ P

15/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Operations with Other Sets Example 2

I P = {4, 12, 18}I Q = {10, 12, 22}I P ∩ Q = {12}I P ∪ Q = {4, 10, 12, 18, 22}I P \ Q = {4, 18}

16/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Powerset

I All possible subsets of a set

I Includes empty set!

17/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Bags

I Bags are Sets that allow duplication

I using square brackets is a common distinction

I P = [4, 12, 12, 18]

I also called multisets

I useful in advanced analysis!

18/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Sequence

I A bag with an order

I using angle brackets is a common distinction

I P = 〈4, 12, 12, 18〉I They need not be ordered a particular way

I also called tuple or vector

I like our implemented data structures

I 〈4, 12, 12, 18〉 6= 〈12, 4, 12, 18〉

19/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Set Context

I Set? Bag? Sequence?

I The context is often the guide

I If lucky, brackets distinguish

I Sorting problem? Looks like sets, probably sequences.

20/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Relations

I A relation over a set is a set of order pairs

I A set of sequences of cardinality 2

I elements in the ordered pairs are elements from the set

21/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Relations example

I Set P = {a, b, c}I Relation R1 = {〈a, c〉, 〈b, c〉, 〈c , b〉}I Relation R2 = {〈a, a〉, 〈a, c〉, 〈b, b〉, 〈b, c〉, 〈c , c〉}I How the left item relates to the right item

I lRr

I How many different relations can there be on a set?

22/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Properties of Relations

I Useful to classify relations

I Good for describing, identifying types of problem

I reflexive, ∀a ∈ S : aRa

I symmetric, ∀a, b ∈ S : aRb =⇒ bRa

I antisymmetric, ∀a, b ∈ S : aRb ∧ bRa =⇒ a = b

I transitive, ∀a, b, c ∈ S : aRb ∧ bRc =⇒ aRc

I useful to group of these properties

23/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Properties of Relations Example

I ordinal relation of less than or equal is reflexive

I being a partner is symmetric

I natural numbers are antisymmetric

I being a relative is transitive

24/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Equivalence Relation

I reflexive, symmetric and transitiveI breaks a set into equivalence classesI These partitions are disjoint from all othersI Elements in these partitions are “equivalent”

I “less than or equal” is an ok example

I modulus is a better example

25/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Partial Order Relation

I Partial Ordering Definitions Differ. We use Shaffer.

I antisymmetric and transitive

I “older than” is a good example

26/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Advantage of Identities

I You can replace expressions with other equivalentexpressions

I You can use identities to change the tools you can useI summation may be tricky in math, but it has an identity

that is regular arithmeticI Identities that lead to arithmetic expressions are called

closed-form solutions

I You can make patterns more visible

27/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Outline I

Notation

Sets

Sets and Relations

IdentitiesLogarithmsSummation

Recursion

Proof Techniques

28/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Logarithms

I logarithm operation finds the exponent that a givenbase is raised to in order to equal the parameter

I log usually implies base 2

I ln usually implies base e

I log 2b = b

I ln eb = b

I log3 3b = b

29/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Logarithmic Identities

I log ab = log a + log b

I log a/b = log a− log b

I log xb = b log x

I log b√x = 1

b log x

I eg: log3 81 = log3 92 = 2 log3 9 = 2 log3 32 = 4

30/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Outline I

Notation

Sets

Sets and Relations

IdentitiesLogarithmsSummation

Recursion

Proof Techniques

31/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Analysing Loops and Recurrences

I We want to count costs

I We are looking at algorithms

I algorithms have repeating structures

I count these structures

I evaluate the cost of each

I summation gives us a tool to help

I a powerful tool for analysis

In∑

i=1f (i)

32/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Summation identities

In∑

i=1i = n(n+1)

2

In∑

i=1i2 = n(n+1)(2n+1)

6

33/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Identifying Recurrence Relations

I Looking for closed-form solutions can be done

I Simply, one can try to unfold a summation thatdescribes an algorithm

I You look for a pattern

I Using identities can swap out unfamiliar expressions andhelp you see patterns

I An advanced topic, but we will look at one importantexample analysing a sorting algorithm later this semester

34/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Loops or Recursion

I an algorithm may call itself on a smaller sub-problem

I this makes it a recursive algorithm

I such algorithms have one or more base cases (orterminating cases) that can give a direct solutionwithout recursion

I such algorithms otherwise break the problem down,including the recursive call that will eventually lead to abase case

35/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Recursion Example

I Factorials

I Towers of Hanoi

36/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Proofs

I Do not be sloppy

I Understanding proofs is key

I As Engineers, proofs are the basis for justifyingeverything you do

I You should always be comfortable justifying everythingyou do

I People question you, but you should always be able toprove what you did is correct

37/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Direct Proof

I The easy one. Logical Explanation.

I Logic or arithmetic leads directly there.

I I also include the use of accepted identities here.

I If the exam is tomorrow, it is not today. Today is nottomorrow.

38/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Proof by Contradiction

I Can disprove with one counter-example.

I No number of examples can prove something true.

I Not true, ALL examples are a proof, but that is usuallyintractable.

I We can do something similar:

I Assume what you want to prove is false.

I Find a counter-example for this.

I eg: There is no largest integer!

39/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Proof by Induction

I Looks like Recursion, really

I Not a coincidence!

I Base case(s)

I Induction Step

40/41

COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries

Stuart Thiel

Notation

Sets

Sets and Relations

Identities

Logarithms

Summation

Recursion

Proof Techniques

Proof by Induction Examples

I Summation

I Coin Example

41/41