DATA STRUCTURES AND ALGORITHMS - cs.ubbcluj.romarianzsu/DSA_MI/Lectures/Lecture03.pdf · Amortized analysis In asymptotic time complexity analysis we consider a single run of an algorithm

DATA STRUCTURES AND ALGORITHMSLECTURE 3

Lect. PhD. Onet-Marian Zsuzsanna

Babes - Bolyai UniversityComputer Science and Mathematics Faculty

2019 - 2020

Lect. PhD. Onet-Marian Zsuzsanna DATA STRUCTURES AND ALGORITHMS

In Lecture 2...

Algorithm Analysis

Dynamic Array


Today

Dynamic Array

Iterator

ADT Set


Quiz - 1

What is the correct recurrence relation for computing thecomplexity of the code below?

function recursiveProblem(n) is://n is a positive number

if n ≤ 1 thenrecursiveProblem ← 1

elserecursiveProblem ← 1 + recursiveProblem(n - 5)

end-ifend-function

a. T (n) =

{1 if n ≤ 1

T ( n5) + n otherwise

b. T (n) =

{1 if n ≤ 1

T ( n5) + 1 otherwise

c. T(n) =

{1 if n ≤ 1

T (n − 5) + 1 otherwised. T(n) =

{1 if n ≤ 1

T (n − 5) otherwise


Quiz - 2

If an array of integer numbers starts at the memory address1038 and an integer number uses 4 bytes, what is the addressof the element from position 100 (assume that the firstelement is at position 0)?

a. 1038b. 1138c. 1436d. 1438e. We cannot determine


Quiz - 3

What makes a dynamic array dynamic?

a. It is dynamically allocated

b. We can add elements to it and we can remove elements from it

c. The elements are dynamically spread over the memory insteadof occupying consecutive memory locations

d. The number of positions allocated to the array canchange (increase or decrease)


Quiz - 4

What is the condition for a dynamic array to be full?

a. The capacity is a power of 2b. The length is 0c. The capacity is equal to the lengthd. The capacity is half of the lengthe. The length is greater than the capacity


Recap

Remember the representation of a dynamic array

DynamicArraylen: Integercap: Integerelems: TElem[]


Recap 2

Remember the function addToEnd

subalgorithm addToEnd (da, e) is:if da.len = da.cap then//the dynamic array is full. We need to resize it

da.cap ← da.cap * 2newElems ← @ an array with da.cap empty slots//we need to copy existing elements into newElemsfor index ← 1, da.len execute

newElems[index] ← da.elems[index]end-for//we need to replace the old element array with the new one//depending on the prog. lang., we may need to free the old elems arrayda.elems ← newElems

end-if//now we certainly have space for the element eda.len ← da.len + 1da.elems[da.len] ← e

end-subalgorithm


Amortized analysis

In asymptotic time complexity analysis we consider a singlerun of an algorithm.

addToEnd should have complexity O(n) - when we have toresize the array, we need to move every existing element, so thenumber of instructions is proportional to the length of the arrayConsequently, a sequence of n calls to the addToEnd operationwould have complexity O(n2)

In amortized time complexity analysis we consider a sequenceof operations and compute the average time for theseoperations.

In amortized time complexity analysis we will consider the totalcomplexity of n calls to the addToEnd operation and dividethis by n, to get the amortized complexity of the algorithm.


Amortized analysis

In asymptotic time complexity analysis we consider a singlerun of an algorithm.

addToEnd should have complexity O(n) - when we have toresize the array, we need to move every existing element, so thenumber of instructions is proportional to the length of the arrayConsequently, a sequence of n calls to the addToEnd operationwould have complexity O(n2)

In amortized time complexity analysis we consider a sequenceof operations and compute the average time for theseoperations.

In amortized time complexity analysis we will consider the totalcomplexity of n calls to the addToEnd operation and dividethis by n, to get the amortized complexity of the algorithm.


Amortized analysis

We can observe that we rarely have to resize the array if weconsider a sequence of n operations.

Consider ci the cost (≈ number of instructions) for the i th callto addToEnd

Considering that we double the capacity at each resizeoperation, at the ith operation we perform a resize if i-1 is apover of 2. So, the cost of operation i, ci , is:

ci =

{i , if i-1 is an exact power of 2

1 otherwise


Amortized analysis

Cost of n operations is:

n∑i=1

ci ≤ n +

[log2n]∑j=0

2j < n + 2n = 3n

The sum contains at most n values of 1 (this is where the nterm comes from) and at most (integer part of) log2n termsof the form 2j .

Since the total cost of n operations is 3n, we can say that thecost of one operation is 3, which is constant.


Amortized analysis

While the worst case time complexity of addToEnd is stillO(n), the amortized complexity is Θ(1).

The amortized complexity is no longer valid, if the resizeoperation just adds a constant number of new slots.

In case of the addToPosition operation, both the worst caseand the amortized complexity of the operation is O(n) - evenif resize is performed rarely, we need to move elements toempty the position where we put the new element.


Amortized analysis

In order to avoid having a Dynamic Array with too manyempty slots, we can resize the array after deletion as well, ifthe array becomes ”too empty”.

How empty should the array become before resize? Which ofthe following two strategies do you think is better? Why?

Wait until the table is only half full (da.len ≈ da.cap/2) andresize it to the half of its capacityWait until the table is only a quarter full (da.len ≈ da.cap/4)and resize it to the half of its capacity


Iterator

An iterator is a structure that is used to iterate through theelements of a container.

Containers can be represented in different ways, using differentdata structures. Iterators are used to offer a common andgeneric way of moving through all the elements of a container,independently of the representation of the container.

Every container that can be iterated, has to contain in theinterface an operation called iterator that will create andreturn an iterator over the container.


Iterator

An iterator usually contains:

a reference to the container it iterates over

a reference to a current element from the container

Iterating through the elements of the container means actuallymoving this current element from one element to anotheruntil the iterator becomes invalid

The exact way of representing the current element from theiterator depends on the data structure used for theimplementation of the container. If the representation/implementation of the container changes, we need to changethe representation/ implementation of the iterator as well.


Iterator - Interface I

Domain of an Iterator

I = {it|it is an iterator over a container with elements of type TElem }


Iterator - Interface II

Interface of an Iterator:


Iterator - Interface III

init(it, c)

description: creates a new iterator for a container

pre: c is a container

post: it ∈ I and it points to the first element in c if c is notempty or it is not valid


Iterator - Interface IV

getCurrent(it)

description: returns the current element from the iterator

pre: it ∈ I, it is valid

post: getCurrent ← e, e ∈ TElem, e is the current elementfrom it

throws: an exception if it is not valid


Iterator - Interface V

next(it)

description: moves the current element from the containerto the next element or makes the iterator invalid if no elementsare left

pre: it ∈ I, it is valid

post: it ′ ∈ I, the current element from it’ points to the nextelement from the container (compared to where it pointed to),or, if no more elements are left, it ′ is invalid

throws: an exception if it is not valid


Iterator - Interface VI

valid(it)

description: verifies if the iterator is valid

pre: it ∈ Ipost:

valid ←

{True, if it points to a valid element from the container

False otherwise


Types of iterators I

The interface presented above describes the simplest iterator:unidirectional and read-only

A unidirectional iterator can be used to iterate through acontainer in one direction only (usually forward, but we candefine a reverse iterator as well).

A bidirectional iterator can be used to iterate in bothdirections. Besides the next operation it has an operationcalled previous.


Types of iterators II

A random access iterator can be used to move multiple steps(not just one step forward or one step backward).

A read-only iterator can be used to iterate through thecontainer, but cannot be used to change it.

A read-write iterator can be used to add/delete elementsto/from the container.


Using the iterator

Since the interface of an iterator is the same, independently ofthe exact container or its representation, the followingsubalgorithm can be used to print the elements of anycontainer.

subalgorithm printContainer(c) is://pre: c is a container//post: the elements of c were printed//we create an iterator using the iterator method of the container

iterator(c, it)while valid(it) execute

//get the current element from the iteratorelem ← getCurrent(it)print elem//go to the next elementnext(it)

end-whileend-subalgorithm


Iterator for a Dynamic Array

How can we define an iterator for a Dynamic Array?

How can we represent that current element from the iterator?

In case of a Dynamic Array, the simplest way to represent acurrent element is to retain the position of the currentelement.

IteratorDA:da: DynamicArraycurrent: Integer

Let’s see how the operations of the iterator can beimplemented.



How can we define an iterator for a Dynamic Array?

How can we represent that current element from the iterator?

In case of a Dynamic Array, the simplest way to represent acurrent element is to retain the position of the currentelement.

IteratorDA:da: DynamicArraycurrent: Integer

Let’s see how the operations of the iterator can beimplemented.


Iterator for a Dynamic Array - init

What do we need to do in the init operation?

subalgorithm init(it, da) is://it is an IteratorDA, da is a Dynamic Array

it.da ← dait.current ← 1

end-subalgorithm

Complexity: Θ(1)






end-subalgorithm

Complexity:

Θ(1)






end-subalgorithm

Complexity: Θ(1)


Iterator for a Dynamic Array - getCurrent

What do we need to do in the getCurrent operation?

function getCurrent(it) is:if not valid(it) then

@throw an exceptionend-ifgetCurrent ← it.da.elems[it.current]

end-function

Complexity: Θ(1)






end-function

Complexity:

Θ(1)






end-function

Complexity: Θ(1)


Iterator for a Dynamic Array - next

What do we need to do in the next operation?

subalgorithm next(it) is:if not valid(it) then

@throw exceptionend-ifit.current ← it.current + 1

end-subalgorithm

Complexity: Θ(1)






end-subalgorithm

Complexity:

Θ(1)






end-subalgorithm

Complexity: Θ(1)


Iterator for a Dynamic Array - valid

What do we need to do in the valid operation?

function valid(it) is:if it.current <= it.da.len then

valid ← Trueelse

valid ← Falseend-if

end-function

Complexity: Θ(1)





valid ← Trueelse


end-function

Complexity:

Θ(1)





valid ← Trueelse


end-function

Complexity: Θ(1)



We can print the content of a Dynamic Array in two ways:

Using an iterator (as present above for a container)

Using the positions (indexes) of elements


Print with Iterator

subalgorithm printDAWithIterator(da) is://pre: da is a DynamicArray//we create an iterator using the iterator method of DA

iterator(da, it)while valid(it) execute

//get the current element from the iteratorelem ← getCurrent(it)print elem//go to the next elementnext(it)

end-whileend-subalgorithm

What is the complexity of printDAWithIterator?


Print with indexes

subalgorithm printDAWithIndexes(da) is://pre: da is a Dynamic Array

for i ← 1, size(da) executeelem ← getElement(da, i)print elem

end-forend-subalgorithm

What is the complexity of printDAWithIndexes?



In case of a Dynamic Array both printing algorithms haveΘ(n) complexity

For other data structures/containers we need iterator because

there are no positions in the data structure/container

the time complexity of iterating through all the elements issmaller


An example

You have discussed ADT Bag at Seminar 1

it has no positions

it can contain duplicates

You have also discussed the interface of the Bag, and itcontained the following operations (for complete specificationscheck Seminar 1)

init (b)add(b, e)remove(b, e)search(b, e)nrOccurrences(b, e)size(b)iterator(b)destroy(b)


ADT Bag on Dynamic Array

When we implement an ADT we have to implement theoperations from the interface of the ADT respecting the preand postconditions of the operations ⇒ for a Bag we have toimplement the operations from above

Before the actual implementation, we need to decide how arewe going to store the elements of the container ⇒ we need adata structure

Implementing a container on a given data structure meansstoring the elements in the given data structure andimplementing the operations from the interface of thecontainer (as required by the data structure).


ADT Bag on Dynamic Array - Representation

If we want to implement a Bag on a dynamic array, how canwe store the elements?

Version 1 - keep the elements simply in a dynamic array

Version 2 - keep a dynamic array of unique elements and keepthe frequency of the elements in a different dynamic array.













Let’s see how to implement version 2. What would be theexact representation (what fields do we need)?

When deciding the representation you have to answer thefollowing two questions:

what are the specific fields needed for a Dynamic Array?is there anything special/extra that we need for the Bag?

Bagelements: TElem[] //array of unique elementsfrequencies: int[] //array of frequencieslen: int //number of occupied positions from the arrayscapacity: int //capacity of the arraysnrElems: int //total number of elements from the Bag



Let’s see how to implement version 2. What would be theexact representation (what fields do we need)?

When deciding the representation you have to answer thefollowing two questions:

what are the specific fields needed for a Dynamic Array?is there anything special/extra that we need for the Bag?

Bagelements: TElem[] //array of unique elementsfrequencies: int[] //array of frequencieslen: int //number of occupied positions from the arrayscapacity: int //capacity of the arraysnrElems: int //total number of elements from the Bag


ADT Bag on Dynamic Array - operation init

How do we implement operation init?

In general operation init has to initialize the elements (fields)from the representation.

subalgorithm init(b) is://b is a Bag

b.len ← 0b.nrElems ← 0b.capacity ← 10 //we can chose any initial capacityb.elements ← @an array of b.capacity elements of type TElemb.frequencies ← @an array of b.capacity elements of type int

end-subalgorithm

Complexity: Θ(1)







end-subalgorithm

Complexity:

Θ(1)







end-subalgorithm

Complexity: Θ(1)


ADT Bag on Dynamic Array - operation add

How can we implement the add operation? What casesshould we consider?

We use a representation with frequencies, so we first need tosee if the element is already in the bag.

If the element is already in the bag, we increase its frequencyby 1.

If the element is not in the bag, we should add it and add afrequency of 1 for it. Representation is a dynamic array.Where in the array should we put the element?

In a dynamic array the easiest is to add an element to the end.

Special case is when the array is full, because we have toresize. Since we have two arrays, we resize both of them.

Since we have a nrElems field, we need to increment it.





















subalgorithm add(b, e) is://b is a Bag, e is a TElem

index ← 1found ← false//first we check if e is already in the bagwhile index ≤ bag.len and not found execute

if b.elements[index] = e then//if we find e, we increase the frequency

found ← trueb.frequencies[index] ← b.frequencies[index] + 1

end-ifindex ← index + 1

end-whileif not found thenif e is not in the bag yet, we have to add it. The array might be full

if b.len = b.capacity thenb.capacity ← b.capacity * 2

//continued on the next slide...


newElems ← a new array of b.capacity elements of type TElemnewFrequencies ← a new array of b.capacity elements of type intfor i ← 1, b.len, 1 execute

newElems[i] ← b.elements[i]newFrequencies[i] ← b.frequencies[i]

end-forb.elements ← newElemsb.frequencies ← newFrequencies

end-ifb.len ← b.len + 1b.elements[b.len] ← eb.frequencies[b.len] ← 1

end-if//we increment the total number of elementsb.nrElems ← b.nrElems + 1

end-subalgorithm

Complexity:

O(n) - where n is the number of elements in thearrays (len) - which is the number of unique elements in theBag


newElems ← a new array of b.capacity elements of type TElemnewFrequencies ← a new array of b.capacity elements of type intfor i ← 1, b.len, 1 execute

newElems[i] ← b.elements[i]newFrequencies[i] ← b.frequencies[i]

end-forb.elements ← newElemsb.frequencies ← newFrequencies

end-ifb.len ← b.len + 1b.elements[b.len] ← eb.frequencies[b.len] ← 1

end-if//we increment the total number of elementsb.nrElems ← b.nrElems + 1

end-subalgorithm

Complexity: O(n) - where n is the number of elements in thearrays (len) - which is the number of unique elements in theBag


ADT Bag on Dynamic Array - operation remove

How do we implement operation remove? What cases shouldwe consider?

We first have to search for the element that we want toremove.

If we do not find it, we return false.

If we find the element, we look at its frequency:

If frequency is greater than 1, we simply subtract 1 from it.

If the frequency is 1, we have to remove the element from thearray. What is the simplest way of removing an element froma dynamic array? Think about the order of the elements andwhether it is important for a bag.

Since the order of the elements is not important, the simplestway of removing an element is to move the last element of thearray to the position where we remove from.




















function remove (b, e) is://b is a Bag, e is a TElem

index ← 1found ← falsewhile index ≤ b.len and not found execute

if b.elements[index] = e thenfound ← trueb.nrElems ← b.nrElems - 1if b.frequencies[index] > 1 then

b.frequencies[index] ← b.frequencies[index] - 1else//we do not need to move all elements, we can just move the//last one to position index

b.elements[index] ← b.elements[b.len]b.frequencies[index] ← b.frequencies[b.len]b.len ← b.len-1

end-ifend-if

//continued on the next slide...


index ← index + 1end-whileremove ← found

end-function

Complexity:

O(n) - where n is the number of elements in thearray


index ← index + 1end-whileremove ← found

end-function

Complexity: O(n) - where n is the number of elements in thearray


Another container

Consider the following problem: in order to avoid electoralfraud (especially the situation when someone votes multipletimes in different locations) we want to build a softwaresystem which stores the personal numerical code (CNP) ofeveryone who votes.

What would be the characteristics of the container used tostore these personal numerical codes?

The elements have to be unique

The order of the elements is not important

The container in which the elements have to be unique andthe order of the elements is not important (there are nopositions) is the ADT Set.


Another container

Consider the following problem: in order to avoid electoralfraud (especially the situation when someone votes multipletimes in different locations) we want to build a softwaresystem which stores the personal numerical code (CNP) ofeveryone who votes.

What would be the characteristics of the container used tostore these personal numerical codes?

The elements have to be unique

The order of the elements is not important

The container in which the elements have to be unique andthe order of the elements is not important (there are nopositions) is the ADT Set.


ADT Set - Domain

Domain of the ADT Set:S = {s|s is a set with elements of the type TElem}


ADT Set - Interface I

init (s)

descr: creates a new empty set

pre: true

post: s ∈ S, s is an empty set.


ADT Set - Interface II

add(s, e)

descr: adds a new element into the set if it is not already inthe set

pre: s ∈ S, e ∈ TElem

post:s ′ ∈ S, s ′ = s ∪ {e} (e is added only if it is not in s yet.If s contains the element e already, no change is made).add ← true if e was added to the set, false otherwise.


ADT Set - Interface III

remove(s, e)

descr: removes an element from the set.


post: s ∈ S, s ′ = s \ {e} (if e is not in s, s is not changed).remove ← true, if e was removed, false otherwise


ADT Set - Interface IV

search(s, e)

descr: verifies if an element is in the set.


post:

search←

{True, if e ∈ s

False, otherwise


ADT Set - Interface V

size(s)

descr: returns the number of elements from a set

pre: s ∈ Spost: size ← the number of elements from s


ADT Set - Interface VI

isEmpty(s)

descr: verifies if the set is empty

pre: s ∈ Spost:

isEmpty ←

{True, if s has no elements

False, otherwise


ADT Set - Interface VII

iterator(s, it)

descr: returns an iterator for a set

pre: s ∈ Spost: it ∈ I, it is an iterator over the set s


ADT Set - Interface VIII

destroy (s)

descr: destroys a set

pre: s ∈ S

post:the set s was destroyed.


ADT Set - Interface IX

Other possible operations (characteristic for sets frommathematics):

reunion of two sets

intersection of two sets

difference of two sets (elements that are present in the firstset, but not in the second one)


ADT Set on a Dynamic Array

If we want to implement a Set we can use a dynamic array asrepresentation.

Do we have two ways of representing the Set, like in case ofthe Bag?

How can we represent a Set using a dynamic array?

what are the specific fields needed for a Dynamic Array?is there anything special/extra that we need for the Set?

Set:elems: TElem[]len: intcap: int
















ADT Set - on a Dynamic Array


Preconditions tell us, that an element can be added only if itis not in the set already ⇒ we first need to see if the elementis in the set.

If the element is in the set, we return false and add nothing

If the element is not in the set, we should add it.Representation is a dynamic array. Where in the array shouldwe put the element?

In a dynamic array the easiest is to put an element at the end.

Since it is a dynamic array, we have to consider the case whenthe array is full and we have to resize it.


















ADT Set - in a Dynamic Array

function add (s, e) is://s is a Set, e is a TElem

index ← 1found ← false//the search partwhile index ≤ s.len and found = false execute:

if s.elems[index] = e thenfound ← true

end-ifindex ← index + 1

end-whileif found = true then

add ← false //it is already in the set, return falseelse//continued on the next slide ...


if s.len = s.cap then //resizes.cap ← s.cap * 2newElems = @a new array with s.cap elementsfor i ← 1, s.len, 1 execute

newElems[i] ← s.elems[i]end-ifs.elems ← newElems

end-if//actually add the elements.len ← s.len + 1s.elems[s.len] ← eadd ← true

end-ifend-function

Complexity:

O(n)


if s.len = s.cap then //resizes.cap ← s.cap * 2newElems = @a new array with s.cap elementsfor i ← 1, s.len, 1 execute

newElems[i] ← s.elems[i]end-ifs.elems ← newElems

end-if//actually add the elements.len ← s.len + 1s.elems[s.len] ← eadd ← true

end-ifend-function

Complexity: O(n)


ADT Set - Iterator

How would you define an iterator for a Set on a dynamicarray?

How would you represent the current element from theiterator?

SetIterator:s: SetcurrentPosition: int


ADT Set - Iterator

How would you define an iterator for a Set on a dynamicarray?

How would you represent the current element from theiterator?

SetIterator:s: SetcurrentPosition: int


Iterator - init


subalgorithm init(sit, s) is://sit is a SetIterator, s is a Set

sit.s ← ssit.currentPosition ← 1

end-subalgorithm

Complexity: Θ(1)


Iterator - init




end-subalgorithm

Complexity:

Θ(1)


Iterator - init




end-subalgorithm

Complexity: Θ(1)


Iterator - getCurrent


function getCurrent(sit) is:if not valid(sit) then

@throw an exceptionend-ifgetCurrent ← sit.s.elems[sit.currentPosition]

end-function

Complexity: Θ(1)






end-function

Complexity:

Θ(1)






end-function

Complexity: Θ(1)


Iterator - next


subalgorithm next(sit) is:if not valid(sit) then

@throw exceptionend-ifsit.currentPosition ← sit.currentPosition + 1

end-subalgorithm

Complexity: Θ(1)


Iterator - next




end-subalgorithm

Complexity:

Θ(1)


Iterator - next




end-subalgorithm

Complexity: Θ(1)


Iterator - valid


function valid(sit) is:if sit.currentPosition <= sit.s.len then

valid ← Trueelse


end-function

Complexity: Θ(1)


Iterator - valid



valid ← Trueelse


end-function

Complexity:

Θ(1)


Iterator - valid



valid ← Trueelse


end-function

Complexity: Θ(1)


Documents

DATA STRUCTURES AND ALGORITHMS - cs.ubbcluj.romarianzsu/DSA_MI/Lectures/Lecture03.pdf · Amortized analysis In asymptotic time complexity analysis we consider a single run of an algorithm