Data Structures and Algorithms - Pradžiaalgis/dsax/Data-structures-2.pdfData Structures and Algorithms ... functions that operate on these objects ... Implementation 10/3/2013 3 10

Data Structures and

Algorithms Course’s slides: Abstract data types,

Stacks, Queues, Lists, Heaps

www.mif.vu.lt/~algis

Abstract data type

ò  A theoretical description of an algorithm, if realized in application is affected very much by:

ò  computer resources,

ò  implementation,

ò  data.

ò  Such a theory include fundamental concepts:

ò  Abstract Data Type (ADT) or data type, or data structures

ò  tools to express operations of algorithms;

ò  computational resources to implement the algorithm and test its functionality;

ò  evaluation of the complexity of algorithms.

What is a Data Type?

ò  A name for the INTEGER data type

ò  E.g., “int”

ò  Collection of (possible) data items

ò  E.g., integers can have values in the range of -231 to 231 – 1

ò  Associated set of operations on those data items

ò  E.g., arithmetic operations like +, -, *, /, etc.

Abstract data type

ò  An abstract data type (ADT) is defined as a mathematical model of the data objects that make up a data type, as well as the functions that operate on these objects (and logical or other relations between objects).

ò  ADT consist of two parts: data objects and operations with data objects.

ò  The term data type refers to the implementation of the mathematical model specified by an ADT

ò  The term data structure refers to a collection of computer variables that are connected in some specific manner

ò  The notion of data type include basic data types. Basic data types are related to a programming language.

Example: Integer Set Data Type

ò  ABSTRACT (Theoretical) DATA TYPE

ò  Mathematical set of integers, I ò  Possible data items: -inf, …, -1, 0, 1, …, +inf ò  Operations: +, -, *, mod, div, etc.

ò  Actual, Implemented Data Type (available in C++):

ò  It’s called “int” ò  Only has range of -231 to 231 – 1 for the possible data

items (instead of –inf to +inf) ò  Has same arithmetic operations available

ò  What’s the relationship/difference between the ADT and the Actual, Implemented Data Type in C++?

ò  The range of possible data items is different.

Summary of ADT

ò  (Abstract or Actual) Data Types have three properties:

ò  Name

ò  Possible Data Items

ò  Operations on those data items

ò  The Data Type declaration goes in the .h (header) file – e.g., the class declaration

ò  The Data Type definitions go in the .cpp (implementation) file – e.g., the class definition

Stacks

ò  Stacks are a special form of collection with LIFO semantics

ò  Two methods ò  int push( Stack s, void *item );

- add item to the top of the stack

ò  void *pop( Stack s ); - remove an item from the top of the stack

ò  Like a plate stacker

ò  other methods   int IsEmpty( Stack s );

/* Return TRUE if empty */ void *Top( Stack s ); /* Return the item at the top, without deleting it */

Stacks

This ADT covers a set of objects as well as operations performed on these objects: ò  Initialize (S) – creates a necessary structured space in computer

memory to locate objects in S; ò  Push(x) – inserts x into S; ò  Pop – deletes object from the stack that was most recently inserted

into; ò  Top – returns an object from the stack that was most recently inserted

into; ò  Kill (S) - releases an amount of memory occupied by S. The operations with stack objects obey LIFO property: Last-In-First-Out. This is a logical constrain or logical condition. The operations Initialize and Kill are more oriented to an implementation of this ADT, but they are important in some algorithms and applications too. The stack is a dynamic data set with a limited access to objects.

Stacks - Implementation

10/3/2013

3

10 Elementary Data Structures10.1 Stacks and queues

• In stacks and queues the element removedfrom the set by DELETE is prespeci¿ed

• In a stack, the element deleted is the one mostrecently inserted: the stack implements a last-in, ¿rst-out (LIFO) policy

• In a queue the element deleted is the one thathas been in the set for the longest time: thequeue policy is ¿rst-in, ¿rst-out (FIFO)

• We consider array to implementation of stacksand queues

3-Oct-13MAT-72006 AA+DS, Fall 2013 270

Stacks

• INSERT operation on a stack is called PUSH

• DELETE operation, which does not take anelement argument, is called POP

• These names are allusions to physical stacks

3-Oct-13MAT-72006 AA+DS, Fall 2013 271


10/3/2013

4

• We can implement a stack of at most elements with an array [1. .ሿ�

• The array has an attribute Ǥ ݐ that indexesthe most recently inserted element

• The stack consists of elements [1. . Ǥ ,[ݐwhere [1] is the element at the bottom of thestack and ሾǤ [ݐ is the element at the top

• When Ǥ ݐ = 0, the stack is empty• We can test to see whether the stack is

empty by query operation STACK-EMPTY

• Popping an empty stack leads to underÀow3-Oct-13MAT-72006 AA+DS, Fall 2013 272

STACK-EMPTYሺ)1. if Ǥ �ݐ == �02. return TRUE

3. else return FALSE

PUSHሺǡ (ݔ1. Ǥ ݐ ൌ Ǥ ݐ + 12. Ǥ ݐ ൌ ݔ

POPሺ)1. if STACK-EMPTYሺ)2. error “underÀow”3. else . top ൌ . top െ 14. return Ǥ ݐ + 1

• Each stack operationtakes (1) time

3-Oct-13MAT-72006 AA+DS, Fall 2013 273


10/3/2013

4

• We can implement a stack of at most elements with an array [1. .ሿ�

• The array has an attribute Ǥ ݐ that indexesthe most recently inserted element

• The stack consists of elements [1. . Ǥ ,[ݐwhere [1] is the element at the bottom of thestack and ሾǤ [ݐ is the element at the top

• When Ǥ ݐ = 0, the stack is empty• We can test to see whether the stack is

empty by query operation STACK-EMPTY

• Popping an empty stack leads to underÀow3-Oct-13MAT-72006 AA+DS, Fall 2013 272

STACK-EMPTYሺ)1. if Ǥ �ݐ == �02. return TRUE

3. else return FALSE

PUSHሺǡ (ݔ1. Ǥ ݐ ൌ Ǥ ݐ + 12. Ǥ ݐ ൌ ݔ

POPሺ)1. if STACK-EMPTYሺ)2. error “underÀow”3. else . top ൌ . top െ 14. return Ǥ ݐ + 1

• Each stack operationtakes (1) time

3-Oct-13MAT-72006 AA+DS, Fall 2013 273

12

Array Stack Implementation

ò  We can use an array of elements as a stack

ò  The top is the index of the next available element in the array

top integer

Object of type T Object of type T null T [ ] stack

13

Linked Stack Implementation

ò  We can use the same LinearNode class that we used for LinkedSet implementation

ò  We change the attribute name to “top” to have a meaning consistent with a stack

Object of type T

top LinearNode next; T element;

Object of type T

LinearNode next; T element;

null

count integer

The N-Queens Problem

ò  Suppose you have 8 chess queens...

ò  ...and a chess board


Can the queens be placed on the board so that no two queens are attacking each other

?


Two queens are not allowed in the same row...


Two queens are not allowed in the same row, or in the same column...


Two queens are not allowed in the same row, or in the same column, or along the same diagonal.


The number of queens, and the size of the board can vary.

N Queens

N columns


We will write a program which tries to find a way to place N queens on an N x N chess board.

How the program works

The program uses a stack to keep track of where each queen is placed.


Each time the program decides to place a queen on the board, the position of the new queen is stored in a record which is placed in the stack.

ROW 1, COL 1


We also have an integer variable to keep track of how many rows have been filled so far.

ROW 1, COL 1

1 filled


Each time we try to place a new queen in the next row, we start by placing the queen in the first column...

ROW 1, COL 1

1 filled

ROW 2, COL 1


...if there is a conflict with another queen, then we shift the new queen to the next column.

ROW 1, COL 1

1 filled

ROW 2, COL 2


If another conflict occurs, the queen is shifted rightward again.

ROW 1, COL 1

1 filled

ROW 2, COL 3


When there are no conflicts, we stop and add one to the value of filled.

ROW 1, COL 1

2 filled

ROW 2, COL 3


Let's look at the third row. The first position we try has a conflict...

ROW 1, COL 1

2 filled

ROW 2, COL 3

ROW 3, COL 1


...so we shift to column 2. But another conflict arises...

ROW 1, COL 1

2 filled

ROW 2, COL 3

ROW 3, COL 2


...and we shift to the third column.

Yet another conflict arises...

ROW 1, COL 1

2 filled

ROW 2, COL 3

ROW 3, COL 3


...and we shift to column 4. There's still a conflict in column 4, so we try to shift rightward again...

ROW 1, COL 1

2 filled

ROW 2, COL 3

ROW 3, COL 4


...but there's nowhere else to go.

ROW 1, COL 1

2 filled

ROW 2, COL 3

ROW 3, COL 4


When we run out of

room in a row:

ò pop the stack,

ò reduce filled by 1

ò and continue working on the previous row.

ROW 1, COL 1

1 filled

ROW 2, COL 3


Now we continue working on row 2, shifting the queen to the right.

ROW 1, COL 1

1 filled

ROW 2, COL 4


This position has no conflicts, so we can increase filled by 1, and move to row 3.

ROW 1, COL 1

2 filled

ROW 2, COL 4


In row 3, we start again at the first column.

ROW 1, COL 1

2 filled

ROW 2, COL 4

ROW 3, COL 1

Pseudocode for N-Queens

❶  Initialize a stack where we can keep track of our decisions.

❷  Place the first queen, pushing its position onto the stack and setting filled to 0.

❸  repeat these steps

ò if there are no conflicts with the queens...

ò else if there is a conflict and there is room to shift the current queen rightward...

ò else if there is a conflict and there is no room to shift the current queen rightward...




Increase filled by 1. If filled is now N, then the algorithm is done. Otherwise, move to

the next row and place a queen in the first column.





Move the current queen rightward, adjusting the record on top of the stack

to indicate the new position.






Backtrack! Keep popping the stack, and reducing filled by 1, until you reach a row where the queen

can be shifted rightward. Shift this queen right.






Backtrack! Keep popping the stack, and reducing filled

by 1, until you reach a row where the queen can be shifted rightward. Shift this queen right.

Stacks - Relevance

ò  Stacks appear in computer programs

ò  Key to call / return in functions & procedures

ò  Stack frame allows recursive calls

ò  Call: push stack frame

ò  Return: pop stack frame

ò  Stack frame

ò  Function arguments

ò  Return address

ò  Local variables

43

Stacks and Queues

ò  Array Stack Implementation

ò  Linked Stack Implementation

ò  Queue Abstract Data Type (ADT)

ò  Queue ADT Interface

ò  Queue Design Considerations

ò  Stacks and FIFO queues are identifying items according to the time that they were inserted into the queue.

44

Queue Abstract Data Type

ò  A queue is a linear collection where the elements are added to one end and removed from the other end

ò  The processing is first in, first out (FIFO)

ò  The first element put on the queue is the first element removed from the queue

ò  Think of a line of people waiting for a bus (The British call that “queuing up”)

Queues

This ADT covers a set of objects as well as operations performed on objects:

queueinit (Q) – creates a necessary structured space in computer memory to locate objects in Q; ò  put (x) or enqueue (x) – inserts x into Q;

ò  get or dequeue – deletes object from the queue that has been residing in Q the longest;

ò  head – returns an object from the queue that has been residing in Q the longest;

ò  kill (Q) – releases an amount of memory occupied by Q.

The operations with queue obey FIFO property: First-In-First-Out. This is a logical constrain or logical condition. The queue is a dynamic data set with a limited access to objects. The application to illustrate usage of a queue is:

ò  queueing system simulation (system with waiting lines) ò  (implemented by using the built-in type of pointer)

46


10/3/2013

5

Queues• The INSERT operation on a queue is ENQUEUE

• We call the DELETE operation DEQUEUE

– like the stack operation POP, DEQUEUE takesno element argument

• The queue has a head and a tail• When an element is enqueued, it takes its

place at the tail of the queue• The element dequeued is always the one at

the head of the queue3-Oct-13MAT-72006 AA+DS, Fall 2013 274

3-Oct-13MAT-72006 AA+DS, Fall 2013 275

47


10/3/2013

6

• One way to implement a queue of at most െ 1 elements using an array [1. .ሿ�

• The queue has an attribute Ǥ thatindexes (points) to its head

• The attribute Ǥ ݐ indexes the next locationat which a newly arriving element will beinserted into the queue

• We “wrap around”: location 1 immediatelyfollows location in a circular order

• When Ǥ ൌ Ǥ the queue is empty ,ݐ• Initially, we have . = . ݐ = 1�

3-Oct-13MAT-72006 AA+DS, Fall 2013 276

ENQUEUEሺǡ (ݔ1. Ǥ ݐ ൌ ݔ2. if Ǥ ݐ ==

Ǥ ݐ3. Ǥ ݐ = 14. elseǤ ݐ ൌ Ǥ ݐ + 1

• Both operations take(1) time

DEQUEUEሺ)1. ݔ ൌ ሾǤ ]2. if Ǥ ==

Ǥ ݐ3. ��Ǥ = 14. else

Ǥ ൌ Ǥ + 15. return ݔ

3-Oct-13MAT-72006 AA+DS, Fall 2013 277

48


10/3/2013

6

• One way to implement a queue of at most െ 1 elements using an array [1. .ሿ�

• The queue has an attribute Ǥ thatindexes (points) to its head

• The attribute Ǥ ݐ indexes the next locationat which a newly arriving element will beinserted into the queue

• We “wrap around”: location 1 immediatelyfollows location in a circular order

• When Ǥ ൌ Ǥ the queue is empty ,ݐ• Initially, we have . = . ݐ = 1�

3-Oct-13MAT-72006 AA+DS, Fall 2013 276

ENQUEUEሺǡ (ݔ1. Ǥ ݐ ൌ ݔ2. if Ǥ ݐ ==

Ǥ ݐ3. Ǥ ݐ = 14. elseǤ ݐ ൌ Ǥ ݐ + 1

• Both operations take(1) time

DEQUEUEሺ)1. ݔ ൌ ሾǤ ]2. if Ǥ ==

Ǥ ݐ3. ��Ǥ = 14. else

Ǥ ൌ Ǥ + 15. return ݔ

3-Oct-13MAT-72006 AA+DS, Fall 2013 277

49

A Conceptual View of a Queue

Rear of Queue (or Tail)

Adding an Element Removing an Element

Front of Queue (or Head)

50

Queue Design Considerations

ò  Although a queue can be empty, there is no concept for it being full. An implementation must be designed to manage storage space

ò  For first and dequeue operation on an empty queue, this implementation will throw an exception

ò  Other implementations could return a value null that is equivalent to “nothing to return”

51

Queue Design Considerations

ò  No iterator method is provided

ò  That would be inconsistent with restricting access to the first element of the queue

ò  If we need an iterator or other mechanism to access the elements in the middle or at the end of the collection, then a queue is not the appropriate data structure to use

Queue implementation

ò  This procedure works well until the first time when the tail marker reaches the end of the array. If some removals have occurred during this time, there will be empty space at the beginning of the array. However, because the tail marker points to the end of the array, the queue is thought to be 'full' and no more data can be added.

ò  We could shift the data so that the head of the queue returns to the beginning of the array each time this happens, but shifting data is costly in terms of computer time, especially if the data being stored in the array consist of large data objects.

of the queue. Because queues process data in the same order in which they are received, a queue is said to be a first-in-first out or FIFO data structure. Just as with stacks, queues can be implemented using arrays or lists. For the first of all, let’s consider the implementation using arrays. Define an array for storing the queue elements, and two markers: x one pointing to the location of the head of the queue, x the other to the first empty space following the tail. When an item is to be added to the queue, a test to see if the tail marker points to a valid location is made, then the item is added to the queue and the tail marker is incremented by 1. When an item is to be removed from the queue, a test is made to see if the queue is empty and, if not, the item at the location pointed to by the head marker is retrieved and the head marker is incremented by 1. This procedure works well until the first time when the tail marker reaches the end of the array. If some removals have occurred during this time, there will be empty space at the beginning of the array. However, because the tail marker points to the end of the array, the queue is thought to be 'full' and no more data can be added. We could shift the data so that the head of the queue returns to the beginning of the array each time this happens, but shifting data is costly in terms of computer time, especially if the data being

stored in the array consist of large data objects. A more efficient way of storing a queue in an array is to “wrap around” the end of the array so that it joins the front of the array. Such a cfrcular array allows the entire array (well, almost, as we'll see in a bit) to be used for storing queue elements without ever requiring any data to be shifted. A circular array with QSIZE elements (numbered from 0 to QSIZE-1) may be visualized: The array is, of course, stored in the normal way in memory, as a linear block of QSIZE elements. The circular diagram is just a convenient way of representing the data structure. We will need Head and Tail markers to indicate the location of the head and the location just after the tail where the next item should be added to the queue, respectively. An empty queue is denoted by the condition Head = Tail:

At this point, the first item of data would be added at the location indicated by the Tail marker, that is, at array index 0. Adding this element gives us the situation:

of the queue. Because queues process data in the same order in which they are received, a queue is said to be a first-in-first out or FIFO data structure. Just as with stacks, queues can be implemented using arrays or lists. For the first of all, let’s consider the implementation using arrays. Define an array for storing the queue elements, and two markers: x one pointing to the location of the head of the queue, x the other to the first empty space following the tail. When an item is to be added to the queue, a test to see if the tail marker points to a valid location is made, then the item is added to the queue and the tail marker is incremented by 1. When an item is to be removed from the queue, a test is made to see if the queue is empty and, if not, the item at the location pointed to by the head marker is retrieved and the head marker is incremented by 1. This procedure works well until the first time when the tail marker reaches the end of the array. If some removals have occurred during this time, there will be empty space at the beginning of the array. However, because the tail marker points to the end of the array, the queue is thought to be 'full' and no more data can be added. We could shift the data so that the head of the queue returns to the beginning of the array each time this happens, but shifting data is costly in terms of computer time, especially if the data being

stored in the array consist of large data objects. A more efficient way of storing a queue in an array is to “wrap around” the end of the array so that it joins the front of the array. Such a cfrcular array allows the entire array (well, almost, as we'll see in a bit) to be used for storing queue elements without ever requiring any data to be shifted. A circular array with QSIZE elements (numbered from 0 to QSIZE-1) may be visualized: The array is, of course, stored in the normal way in memory, as a linear block of QSIZE elements. The circular diagram is just a convenient way of representing the data structure. We will need Head and Tail markers to indicate the location of the head and the location just after the tail where the next item should be added to the queue, respectively. An empty queue is denoted by the condition Head = Tail:

At this point, the first item of data would be added at the location indicated by the Tail marker, that is, at array index 0. Adding this element gives us the situation:

Queue implementation

Let us use the queue until the Tail marker reaches QSIZE-1. We will assume that some items have been removed from the queue, so that Head has moved along as well:

Now we add another element to the queue at the location marked by Tail, that is, at array index QSIZE-1. The Tail marker now advances one step, which positions it at array index 0. The Tail marker has wrapped around the array and come back to its starting point. Since the Head marker has moved along, those elements at the beginning of the array from index 0 up to index Head-1 are available for storage. Using a circular array means that we can make use of these elements without having to shift any data. In a similar way, if we keep removing items from the queue, eventually Head will point to array index QSIZE-1. If we remove another element, Head will advance another step and wrap around the array, returning to index 0. We have seen that the condition for an empty queue is that Head == Tail. What is the condition for a full queue? If we try to make use of all the array elements, then in a full queue, the tail of the queue must be the element immediately prior to the head. Since we are using the Tail marker to point to the array element immediately following the tail element in the queue, Tail would have to point to the same location as Head for a full queue. But we have just seen that the condition Head == Tail is the condition for an empty queue. Therefore, if we try to make use of all the array elements, the conditions for full and empty queues become identical. We therefore impose the rule that we must always keep at least one free space in the array, and that a queue becomes full when the Tail marker points to the location immediately prior to Head. We may now formalize the algorithms for dealing with queues in a circular array. x Creating an empty queue: Set Head = Tail = 0. x Testing if a queue is empty: is Head == Tail? x Testing if a queue is full: is (Tail + 1) mod QSIZE == Head? x Adding an item to a queue: if queue is not full, add item at location Tail and set Tail = (Tail +

1) mod QSIZE. x Removing an item from a queue: if queue is not empty, remove item from location Head and

set Head = (Head + 1) mod QSIZE. The mod operator ensures that Head and Tail wrap around the end of the array properly.

Let us use the queue until the Tail marker reaches QSIZE-1. We will assume that some items have been removed from the queue, so that Head has moved along as well:

Now we add another element to the queue at the location marked by Tail, that is, at array index QSIZE-1. The Tail marker now advances one step, which positions it at array index 0. The Tail marker has wrapped around the array and come back to its starting point. Since the Head marker has moved along, those elements at the beginning of the array from index 0 up to index Head-1 are available for storage. Using a circular array means that we can make use of these elements without having to shift any data. In a similar way, if we keep removing items from the queue, eventually Head will point to array index QSIZE-1. If we remove another element, Head will advance another step and wrap around the array, returning to index 0. We have seen that the condition for an empty queue is that Head == Tail. What is the condition for a full queue? If we try to make use of all the array elements, then in a full queue, the tail of the queue must be the element immediately prior to the head. Since we are using the Tail marker to point to the array element immediately following the tail element in the queue, Tail would have to point to the same location as Head for a full queue. But we have just seen that the condition Head == Tail is the condition for an empty queue. Therefore, if we try to make use of all the array elements, the conditions for full and empty queues become identical. We therefore impose the rule that we must always keep at least one free space in the array, and that a queue becomes full when the Tail marker points to the location immediately prior to Head. We may now formalize the algorithms for dealing with queues in a circular array. x Creating an empty queue: Set Head = Tail = 0. x Testing if a queue is empty: is Head == Tail? x Testing if a queue is full: is (Tail + 1) mod QSIZE == Head? x Adding an item to a queue: if queue is not full, add item at location Tail and set Tail = (Tail +

1) mod QSIZE. x Removing an item from a queue: if queue is not empty, remove item from location Head and

set Head = (Head + 1) mod QSIZE. The mod operator ensures that Head and Tail wrap around the end of the array properly.

We may now formalize the algorithms for dealing with queues in a circular array.

54


10/3/2013

5

Queues• The INSERT operation on a queue is ENQUEUE

• We call the DELETE operation DEQUEUE

– like the stack operation POP, DEQUEUE takesno element argument

• The queue has a head and a tail• When an element is enqueued, it takes its

place at the tail of the queue• The element dequeued is always the one at

the head of the queue3-Oct-13MAT-72006 AA+DS, Fall 2013 274

3-Oct-13MAT-72006 AA+DS, Fall 2013 275

Linked list

ò  The List ADT

ò  A list is one of the most fundamental data structures used to store a collection of data items.

ò  The importance of the List ADT is that it can be used to implement a wide variety of other ADTs. That is, the LIST ADT often serves as a basic building block in the construction of more complicated ADTs.

ò  A list may be defined as a dynamic ordered n-tuple:

ò  L == ( l1, 12, ....., ln )

Linked list

ò  The use of the term dynamic in this definition is meant to emphasize that the elements in this

ò  n-tuple may change over time. Notice that these elements have a linear order that is based upon their position in the list. The first element in the list, 11, is called the head of the list. The last element, ln, is referred to as the tail of the list. The number of elements in a list L is refered to as the length of the list.

ò  Thus the empty list, represented by (), has length 0. A list can homogeneous or heterogeneous.

Linked list

ò  0. Initialize ( L ). This operation is needed to allocate the amount of memory and to give a structure to this amount.

ò  1. Insert (L, x, i). If this operation is successful, the boolean value true is returned; otherwise, the boolean value false is returned.

ò  2. Append (L, x). Adds element x to the tail of L, causing the length of the list to become n+1. If this operation is successful, the boolean value true is returned; otherwise, the boolean value false is returned.

ò  3. Retrieve (L, i). Returns the element stored at position i of L, or the null value if position i does not exist.

ò  4. Delete (L, i). Deletes the element stored at position i of L, causing elements to move in their positions.

ò  5. Length (L). Returns the length of L.

Linked list

ò  6. Reset (L). Resets the current position in L to the head (i.e., to position 1) and returns the value 1. If the list is empty, the value 0 is returned.

ò  7. Current (L). Returns the current position in L.

ò  8. Next (L). Increments and returns the current position in L.

ò  Note that only the Insert, Delete, Reset, and Next operations modify the lists to which they are applied. The remaining operations simply query lists in order to obtain information about them.

Linked lists

ò  Flexible space use

ò  Dynamically allocate space for each element as needed ò  Include a pointer to the next item

➧  Linked list

ò  Each node of the list contains ò  the data item (an object pointer in our ADT)

ò  a pointer to the next node

Data Next

object

Linked lists

ò  Collection structure has a pointer to the list head

ò  Initially NULL

ò  Add first item

ò  Allocate space for node

ò  Set its data pointer to object

ò  Set Next to NULL

ò  Set Head to point to new node

Data Next

object

Head

Collection node

Linked lists

ò  Add second item

ò  Allocate space for node

ò  Set its data pointer to object

ò  Set Next to current Head

ò  Set Head to point to new node

Data Next

object

Head

Collection

node Data Next

object2

node

Linked lists

ò  Add time

ò  Constant - independent of n

ò  Search time

ò  Worst case - n

Data Next

object

Head

Collection

node Data Next

object2

node

Linked lists - LIFO and FIFO ò  Simplest implementation

ò  Add to head ➧  Last-In-First-Out (LIFO) semantics

ò  Modifications

ò  First-In-First-Out (FIFO) ò  Keep a tail pointer

struct t_node { void *item; struct t_node *next; } node; typedef struct t_node *Node; struct collection { Node head, tail; };

tail is set in the AddToCollection

method if head == NULL

head

tail

Linked list

3. Retrieve (L, i). Returns the element stored at position i of L, or the null value if position i does

not exist. 4. Delete (L, i). Deletes the element stored at position i of L, causing elements to move in their positions. 5. Length (L). Returns the length of L. 6. Reset (L). Resets the current position in L to the head (i.e., to position 1) and returns the value 1. If the list is,empty, the value 0 is returned. 7. Current (L). Returns the current position in L. 8. Next (L). Increments and returns the current position in L. Note that only the Insert, Delete, Reset, and Next operations modify the lists to which they are applied. The remaining operations simply query lists in order to obtain information about them.

Sequential Mapping If all of the elements that comprise a given data structure are stored one after the other in consecutive memory locations, we say that the data structure is sequentially mapped into computer memory. Sequential mapping makes it possible to access any element in the data structure in constant time. Given the starting address of the data structure in memory, we can find the address of any element in the data structure by simply calculating its offset from the starting address. An array is an example of a sequentially mapped data structure:

Because it takes the same amount of time to access any element, a sequentially-mapped data structure is also called a random access data structure. That is, the accessing time is independent of the size of the data structure, and therefore requires O(l) time.

Schematic representations of (a) a singly-linked list and (b) a doubly-linked list:

List Operations Insert and Delete with Single-linked List and Double-linked List:

The Skip Lists:

Linked list

The Skip Lists:

The Skip Lists:

Linked list

10/3/2013

7

10.2 Linked lists

• The linear order in a linked list is determinedby a pointer in each object

• Each element of a doubly linked list ܮ is anobject with an attribute ݕ and two otherpointer attributes: ݐݔ and ݒݎ

• Ǥݔ ݐݔ points to successor in the linked list,and Ǥݔ ݒݎ points to predecessor

• If Ǥݔ ݒݎ ൌ�NIL, the element ݔ is the ¿rstelement, or head, of the list

3-Oct-13MAT-72006 AA+DS, Fall 2013 278

• If Ǥݔ ݐݔ = NIL, the element ݔ is the lastelement, or tail, of the list

• Attribute Ǥܮ points to the ¿rst element ofthe list

• If .ܮ = NIL, the list is empty3-Oct-13MAT-72006 AA+DS, Fall 2013 279

Linked list

10/3/2013

7

10.2 Linked lists

• The linear order in a linked list is determinedby a pointer in each object

• Each element of a doubly linked list ܮ is anobject with an attribute ݕ and two otherpointer attributes: ݐݔ and ݒݎ

• Ǥݔ ݐݔ points to successor in the linked list,and Ǥݔ ݒݎ points to predecessor

• If Ǥݔ ݒݎ ൌ�NIL, the element ݔ is the ¿rstelement, or head, of the list

3-Oct-13MAT-72006 AA+DS, Fall 2013 278

• If Ǥݔ ݐݔ = NIL, the element ݔ is the lastelement, or tail, of the list

• Attribute Ǥܮ points to the ¿rst element ofthe list

• If .ܮ = NIL, the list is empty3-Oct-13MAT-72006 AA+DS, Fall 2013 279

Linked list - searching

10/3/2013

8

Searching a linked list

LIST-SEARCHሺܮǡ )1. ݔ ൌ Ǥܮ 2. while ݔ NIL and Ǥݔ ݕ 3. ݔ ൌ Ǥݔ ݐݔ4. return ݔ• To search a list of objects, LIST-SEARCH

takes ȣሺ) time in the worst case, since itmay have to search the entire list

3-Oct-13MAT-72006 AA+DS, Fall 2013 280

Inserting into a linked list

LIST-INSERTሺܮǡ (ݔ1. Ǥݔ ݐݔ ൌ Ǥܮ 2. if Ǥܮ NIL3. Ǥܮ Ǥ ݒݎ ൌ ݔ4. Ǥܮ ൌ ݔ5. Ǥݔ ݒݎ = NIL

• The running time for LIST-INSERT is (1)3-Oct-13MAT-72006 AA+DS, Fall 2013 281

Linked list - insert

10/3/2013

8

Searching a linked list

LIST-SEARCHሺܮǡ )1. ݔ ൌ Ǥܮ 2. while ݔ NIL and Ǥݔ ݕ 3. ݔ ൌ Ǥݔ ݐݔ4. return ݔ• To search a list of objects, LIST-SEARCH

takes ȣሺ) time in the worst case, since itmay have to search the entire list

3-Oct-13MAT-72006 AA+DS, Fall 2013 280

Inserting into a linked list

LIST-INSERTሺܮǡ (ݔ1. Ǥݔ ݐݔ ൌ Ǥܮ 2. if Ǥܮ NIL3. Ǥܮ Ǥ ݒݎ ൌ ݔ4. Ǥܮ ൌ ݔ5. Ǥݔ ݒݎ = NIL

• The running time for LIST-INSERT is (1)3-Oct-13MAT-72006 AA+DS, Fall 2013 281

Linked list - delete

10/3/2013

9

Deleting from a linked list

LIST-DELETEሺܮǡ (ݔ1. if Ǥݔ ݒݎ NIL2. Ǥݔ Ǥݒݎ ݐݔ ൌ Ǥݔ ݐݔ3. else Ǥܮ ൌ Ǥݔ ݐݔ4. if Ǥݔ ݐݔ NIL5. Ǥݔ Ǥݐݔ ݒݎ ൌ Ǥݔ ݒݎ• This runs in (1) time, if we wish to delete an

element with a given key, ȣሺ) time is required

3-Oct-13MAT-72006 AA+DS, Fall 2013 282

Sentinels

• LIST-DELETE would be simpler if we couldignore the boundary conditions at the headand tail of the list:

LIST-DELETE’ሺܮǡ (ݔ1. Ǥݔ Ǥݒݎ ݐݔ ൌ Ǥݔ ݐݔ2. Ǥݔ Ǥݐݔ ݒݎ ൌ Ǥݔ ݒݎ

3-Oct-13MAT-72006 AA+DS, Fall 2013 283

Dynamic set ADT

The concept of a dynamic set to investigate data structures and algorithms that support efficient implementation of operations on sets. Another important difference between the mathematical concept of a set and the sets considered in computer science: ò  • a set in mathematics is unchanging, while the sets in CS are

considered to change over time as data elements are added or deleted.

Thus, sets are referred here as dynamic sets. Each element in a dynamic set contains an identifying field called a key, and that a total ordering relationship exists on these keys. It will be assumed that no two elements of a dynamic set contain the same key.

Dynamic set ADT

ò  The concept of a dynamic set as an DYNAMIC SET ADT is to be specified, as a collection of data elements, along with the legal operations defined on these data elements.

ò  If the DYNAMIC SET ADT is implemented properly, application programmers will be able to use dynamic sets without having to understand their implementation details. The use of ADTs simplifies design and development, promotes reusability of software components.

ò  A list of general operations for the DYNAMIC SET ADT. In each of these operations, S represents a specific dynamic set:

Dynamic set ADT

ò  Search(S, k). Returns the element with key k in S, or the null value if an element with key k is not in S.

ò  Insert(S, x). Adds element x to S. If this operation is successful, the boolean value true is returned; otherwise, the boolean value false is returned.

ò  Delete(S, k). Removes the element with key k in S. If this operation is successful, the boolean value true is returned; otherwise, the boolean value false is returned.

ò  Minimum(S). Returns the element in dynamic set S that has the smallest key value, or the null value if S is empty.

ò  Maximum(S). Returns the element in S that has the largest key value, or the null value if S is empty.

ò  Predecessor(S, k). Returns the element in S that has the largest key value less than k, or the null value if no such element exists.

ò  Successor(S, k). Returns the element in S that has the smallest key value greater than k, or the null value if no such element exists.

Dynamic set ADT

In many instances an application will only require the use of a few DYNAMIC SET operations, they are given special names:

ò  the ADT that supports Search, Insert, and Delete operations is called the DICTIONARY ADT;

ò  the STACK, QUEUE, and PRIORITY QUEUE are all special types of dynamic sets.

Each of the data structures described will be analyzed in order to determine how efficiently they support the implementation of these operations. In each case, the analysis will be performed in terms of n, the number of data elements stored in the dynamic set.

Dictionary/Dynamic set

What is the asymptotic worst-case running times for each of the seven fundamental dictionary operations when the data structure is implemented as

• A singly-linked unsorted list,

• A doubly-linked unsorted list,

• A singly-linked sorted list,

• A doubly-linked sorted list.

Dictionary/Dynamic set Solution

singly double singly doublyDictionary operation unsorted unsorted sorted sortedSearch(L, k) O(n) O(n) O(n) O(n)Insert(L, x) O(1) O(1) O(n) O(n)Delete(L, x) O(n)∗ O(1) O(n)∗ O(1)Successor(L, x) O(n) O(n) O(1) O(1)Predecessor(L, x) O(n) O(n) O(n)∗ O(1)Minimum(L) O(n) O(n) O(1) O(1)Maximum(L) O(n) O(n) O(1)∗ O(1)


10/3/2013

2

Operations on dynamic sets

• Operations on a dynamic set can be groupedinto queries and modifying operations

3-Oct-13MAT-72006 AA+DS, Fall 2013 268

SEARCHሺǡ )• Given a set and a key value ,

return a pointer ݔ to an elementin such that Ǥݔ ݕ ൌ , or NIL ifno such element belongs to

INSERTሺǡ (ݔ• Augment the set with the

element pointed to by We .ݔassume that any attributes inelement ݔ have already beeninitialized

DELETEሺǡ (ݔ• Given a pointer ݔ to an element

in the set , remove ݔ from .Note that this operation takes apointer to an element not a ,ݔkey value

MINIMUMሺ)• A query on a totally ordered set

that returns a pointer to theelement of with the smallestkey

MAXIMUMሺ)• Return a pointer to the element

of with the largest keySUCCESSORሺǡ (ݔ• Given an element ݔ whose key is

from a totally ordered set ,return a pointer to the next largerelement in , or NIL if ݔ is themaximum element

PREDECESSORሺǡ (ݔ• Given an element ݔ whose key is

from a totally ordered set ,return a pointer to the nextsmaller element in , or NIL if ݔ isthe minimum element

• We usually measurethe time taken toexecute a setoperation in terms ofthe size of the set

• E.g., we later describea data structure thatcan support any of theoperations on a set ofsize in time (lg )

3-Oct-13MAT-72006 AA+DS, Fall 2013 269


10/3/2013

2

Operations on dynamic sets

• Operations on a dynamic set can be groupedinto queries and modifying operations

3-Oct-13MAT-72006 AA+DS, Fall 2013 268

SEARCHሺǡ )• Given a set and a key value ,

return a pointer ݔ to an elementin such that Ǥݔ ݕ ൌ , or NIL ifno such element belongs to

INSERTሺǡ (ݔ• Augment the set with the

element pointed to by We .ݔassume that any attributes inelement ݔ have already beeninitialized

DELETEሺǡ (ݔ• Given a pointer ݔ to an element

in the set , remove ݔ from .Note that this operation takes apointer to an element not a ,ݔkey value

MINIMUMሺ)• A query on a totally ordered set

that returns a pointer to theelement of with the smallestkey

MAXIMUMሺ)• Return a pointer to the element

of with the largest keySUCCESSORሺǡ (ݔ• Given an element ݔ whose key is

from a totally ordered set ,return a pointer to the next largerelement in , or NIL if ݔ is themaximum element

PREDECESSORሺǡ (ݔ• Given an element ݔ whose key is

from a totally ordered set ,return a pointer to the nextsmaller element in , or NIL if ݔ isthe minimum element

• We usually measurethe time taken toexecute a setoperation in terms ofthe size of the set

• E.g., we later describea data structure thatcan support any of theoperations on a set ofsize in time (lg )

3-Oct-13MAT-72006 AA+DS, Fall 2013 269

Generalized queue

Alternatively, the abstract concepts may be identified in terms of a sequential listing of the items in order, and refer to the basic operations of inserting and deleting items from the beginning and the end of the list:

ò  if we insert at the end and delete at the end, we get a stack (precisely as in array implementation);

ò  if we insert at the beginning and delete at the beginning, we also get a stack (precisely as in linked-list implementation);

ò  if we insert at the end and delete at the beginning, we get a FIFO queue (precisely as in linked-list implementation);

ò  if we insert at the beginning and delete at the end, we also get a FIFO queue (this option does not correspond to any of implementations given).

Generalized queue

ò  The algorithm can expect to get any of the items on the queue with equal probability. The operations of a random queue can be implemented:

ò  in constant time using an array representation (it requires to reserve space ahead of time)

ò  using linked-list alternative (which is less attractive however, because implementing both, insertion and deletion efficiently is a challenging task)

ò  Random queues can be used as the basis for randomized algorithms, to avoid, with high probability, worst-case performance scenarios.

Generalized queue

Building on this point of view, the dequeue ADT may be defined, where either insertion or deletion at either end are allowed. The

implementation of dequeue is a good exercise to program.

The priority queue ADT is another example of generalized queue. The items in a priority queue have keys and the rule for deletion is: "remove the item with the smallest key"

The priority queue ADT is useful in a variety of applications, and the problem of finding efficient implementations for this ADT

has been a research goal in computer science for many years.

Heaps and the Heapsort

ò  Heaps and priority queues

ò  Heap structure and position numbering

ò  Heap structure property

ò  Heap ordering property

ò  Removal of top priority node

ò  Inserting a new node into the heap

ò  The heap sort

ò  Source code for heap sort program

Heaps and priority queues

Heaps are an abstract data structure, where each object has a key value (the priority), and the operations are:

ò  insert an object,

ò  find the object of minimum key (find min)

ò  delete the object of minimum key (delete min).

A heap is a data structure used to implement an efficient priority queue. The idea is to make it efficient to extract the element with the lowest priority - the next item in the queue to be processed.

Heap structure

A heap can be visualised as a binary tree in which every layer is filled from the left. For every layer to be full, the tree would have to have a size exactly equal to 2n-1, e.g. a value for size in the series 1, 3, 7, 15, 31, 63, 127, 255 etc.

So to be practical enough to allow for any particular size, a heap has every layer filled except for the bottom layer which is filled from the left.

Heap structure and position numbering

Heap structure and position numbering

In the above diagram nodes are labelled based on position: •  the left child of each node is numbered node*2 and •  the right child is numbered node*2+1. The parent of every node is obtained using integer division (throwing away the remainder) so that for a node i's parent has position i/2 . Because this numbering system makes it very easy to move between nodes and their children or parents, a heap is commonly implemented as an array with element 0 unused.

Basic Heap Properties

ò  A heap T storing n keys has height h = ⎣log(n + 1)⎦, which is O (log n)

4

6

207

811

5

9

1214

15

2516

Heap ordering

Heap Insertion

ò  Insert 6

Heap Insertion

ò  Add key in next available position

Heap Insertion Begin Unheap

Heap Insertion

Heap Insertion

Terminate unheap when ò  reach root

ò  key child is greater than key parent

Removal of top priority node

ò  The rest of these notes assume a MinHeap will be used.

ò  Removal of the top node creates a hole at the top which is "bubbled" downwards by moving values below it upwards, until the hole is in a position where it can be replaced with the rightmost node from the bottom layer. This process restores the heap ordering property.

Heap Removal

ò  Remove element

from priority queues?

removeMin( )

Heap Removal ò  Begin downheap

Heap Removal

Heap Removal

Heap Removal

Terminate downheap when ò  reach leaf level

ò  key parent is greater than key child

The Heapsort

ò  Using a heap to sort data involves performing N insertions followed by N delete min operations as described above.

ò  Memory usage will depend upon whether the data already exists in memory or whether the data is on disk. Allocating the array to be used to store the heap will be more efficient if N, the number of records, can be known in advance.

ò  Dynamic allocation of the array will then be possible, and this is likely to be preferable to pre-allocating the array.

Comparing Quick-Heap-Insert

ò  Quicksort is generally faster

ò  Fewer comparisons and exchanges ò  Some empirical data

n Quick Heap InsertComp Exch Comp Exch Comp Exch

100 712 148 2842 581 2595 899200 1682 328 9736 9736 10307 3503500 5102 919 53113 4042 62746 21083

What are Heaps Useful for?

ò  To implement priority queues

ò  Priority queue = a queue where all elements have a “priority” associated with them

ò  Remove in a priority queue removes the element with the smallest priority

ò  insert

ò  removeMin

Heap or Not a Heap?

Building a Heap

build (n + 1)/2 trivial

one-element heaps

build three-element

heaps on top of them

Building a Heap

downheap to preserve the order property

now form seven-element heaps

Building a Heap

Building a Heap

Heap-ordered pointers

222 5 Heaps

has to occur together with its object: there are no two possible models like theyexisted for search trees, but each node contains a key with its object. Thus, thestructure of a node of a (binary) heap-ordered tree is as follows:

typedef struct hp_n_t {key_t key;object_t *object;struct hp_n_t *left;struct hp_n_t *right;

/* possibly additional information */} heap_node_t;

We named the two pointers again left and right, but different from thesearch tree, there is no order relation between them. Again we define a heap-ordered tree recursively: the heap-ordered tree is either empty or contains inthe root node a key, an object, and two pointers, each of which might be eitherNULL or point to another heap-ordered tree in which all keys are larger thanthe key in the root node.Any structure with these properties is a heap-ordered tree for its objects and

key values.

5obj1

6obj2

42obj3

99obj4

10obj5

50obj6

11obj7

15obj8

78obj9

12obj10

72obj11

16obj12

89obj13

80obj14

13obj15

91obj16

81obj17

14obj18

22obj19

92obj20

97obj21

Heap-Ordered Tree

We have to establish some convention to mark the empty heap; this isdifferent from the situation in the search trees, where we could use NULL fieldsin left and right pointers; but in a heap-ordered tree, both pointers mightlegitimately be NULL pointers. We could use the object field, but there might

Generic Insert for Heap-ordered pointers 224 5 Heaps

3

5 6

8 11 28 10

13 9 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

7

3

5 6

7

8

11 28 10

13 9

30 32 29 23

21 15 27 22

35 40

31 16 42 33

37 41

Heap-Ordered Tree: Simple Generic Insertion Method.The Entire Subtree at the Inserted Node Is Moved Down

This way, we do not even need to go down until we reach a leaf, butwe increase the depth of everything below the newly inserted node by 1.Alternatively, we could insert the new key in the existing node and then pushevery following key one step downward to the leaf on this path, creating a newleaf in the end. Here the depth of the nodes stays the same; only the final leaf isa new node with possibly high depth. We do not violate the heap-order propertyby this pushing down along any path, because in each node we exchange thecurrent key for a smaller key. The complexity of this operation is the length ofthe path taken, so we just need to be able to find one short path. Any tree withn nodes must contain some path of length ⌊log(n + 1)⌋; we just have to find it.

3

5 6

8 11 28 10

13 9 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

7

3

5 6

7 11 28 10

8 9 30 32 29 23

13 15 27 22 35 40

21 16 42 33 37 41

31

Heap-Ordered Tree: Alternative Generic Insertion Method.The Elements along the Arbitrary Chosen Path Are Pushed Down

For the delete min operation, the situation is more difficult; the obviousmethod would be to remove the key and object from the root, compare the keysof its left and right lower neighbors, and move the smaller one down, deletingit recursively from its subtree. Thus we have no choice; we have to take thepath from the root to a leaf that we get when we always take the smaller key,and along this path we move everything one step up to the root, deleting thelast, now empty, node. Because we have no control over the path we take, thisworks only in O(log n) time if all paths from the root to any leaf have lengthO(log n).

Alternative Insert for Heap-ordered pointers

224 5 Heaps

3

5 6

8 11 28 10

13 9 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

7

3

5 6

7

8

11 28 10

13 9

30 32 29 23

21 15 27 22

35 40

31 16 42 33

37 41

Heap-Ordered Tree: Simple Generic Insertion Method.The Entire Subtree at the Inserted Node Is Moved Down

This way, we do not even need to go down until we reach a leaf, butwe increase the depth of everything below the newly inserted node by 1.Alternatively, we could insert the new key in the existing node and then pushevery following key one step downward to the leaf on this path, creating a newleaf in the end. Here the depth of the nodes stays the same; only the final leaf isa new node with possibly high depth. We do not violate the heap-order propertyby this pushing down along any path, because in each node we exchange thecurrent key for a smaller key. The complexity of this operation is the length ofthe path taken, so we just need to be able to find one short path. Any tree withn nodes must contain some path of length ⌊log(n + 1)⌋; we just have to find it.

3

5 6

8 11 28 10

13 9 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

7

3

5 6

7 11 28 10

8 9 30 32 29 23

13 15 27 22 35 40

21 16 42 33 37 41

31

Heap-Ordered Tree: Alternative Generic Insertion Method.The Elements along the Arbitrary Chosen Path Are Pushed Down

For the delete min operation, the situation is more difficult; the obviousmethod would be to remove the key and object from the root, compare the keysof its left and right lower neighbors, and move the smaller one down, deletingit recursively from its subtree. Thus we have no choice; we have to take thepath from the root to a leaf that we get when we always take the smaller key,and along this path we move everything one step up to the root, deleting thelast, now empty, node. Because we have no control over the path we take, thisworks only in O(log n) time if all paths from the root to any leaf have lengthO(log n).

Root delete for Heap-ordered pointers 5.3 Heap-Ordered Trees and Half-Ordered Trees 225

5 6

8 11 28 10

13 9 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

5

6

8 11 28 10

13 9 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

5

8 6

11 28 10

13 9 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

5

8 6

9 11 28 10

13 30 32 29 23

21 15 27 22 35 40

31 16 42 33 37 41

5

8 6

9 11 28 10

13 22 30 32 29 23

21 15 27 35 40

31 16 42 33 37 41

5

8 6

9 11 28 10

13 22 30 32 29 23

21 15 27 35 40

31 16 42 33 37 41

Heap-Ordered Tree: Generic Deletion Method.The Root Is Deleted, and the Hole Moves Down to a Leaf

We thus need some sort of balancing information. We could attempt toreuse any of the balancing methods of the search trees, for example, creatinga height-balanced heap-ordered tree. Because the height of the tree would bebounded by O(log n), we would support all the update operations in O(log n)time. The problem here is that the rotations cannot be applied to heap order.If we want to rotate a subtree, the key in the root of the subtree must stay thesame by the heap-order condition, so the key in the other node of the rotationalso stays the same. But this other node receives a new lower neighbor and thatlower neighbor might violate the heap-order condition. Thus, we cannot justreuse the balancing methods we developed for search trees.

An alternative with a weaker order condition is the half-ordered trees. Theseare the same trees as before, but we demand only that for each node, ev-ery key in its right subtree should be larger. For the left subtree, there is nocondition.

x x

>x >x >xno

restriction

Order Conditions below a Node: Heap Order and Half Order

Documents

Data Structures and Algorithms - Pradžiaalgis/dsax/Data-structures-2.pdfData Structures and Algorithms ... functions that operate on these objects ... Implementation 10/3/2013 3 10