HKOI 2005 TrainingIntroduction to Algorithms

Alan, Tam Siu Lung

What is Algorithm?• Algorithm

– A set of data manipulations instructions to solve a problem (informatics problem)

• Problem– A mapping from each input (in a specific format) to an

output (in a specific format)• Pseudo-Code

– Realization of Algorithm in the current domain• Mapping from problem input to algorithm input• Mapping from algorithm output to problem output

• Program– Realization of Pseudo-Code given

• A compiler and the language supported• Data in plain text format and the IO routines supported

Visualization of Program

A BAlgorithm1

Coding Pseudo-Code

Source-Codeint main() {

A a = Read();

B b = Algorithm1(a);

C c = Algorithm2(b);

D d = Algorithm3(c);

Write(d);

}

B CAlgorithm2

C DAlgorithm3

Algorithm Example• Sorting Problem

– Given an array of N comparable elements with comparable keys

– Output a permutation of the elements– such that no element is has a key greater

than that of the one there before• Note

– Array = a sequence of elements– Element = a key-value pair– Permutation = ...– Greater than = …

Realization Example• To print in sorted order a set of points in a

cetacean plane by the distance from origin:• Realization contains

– Key = distance from origin– Value = coordinates = (x, y)– Comparison of 2 elements = comparison of the

numerical values of distances• Pseudo-Code contains

– Store the set in an array– Sort them– Output the array in order

Why Algorithms?

• Computational Biology• Database• Cryptography• Quantitative Analysis• Artificial Intelligence• Scheduling

Example – Multiple Sequence Alignment

• Input– GCTAGCTAAC– AACTAGC– AGCGCTAGCTA– TAACGACTAT

• Find a string with minimum which contains all these sub-strings

Example – Multiple Sequence Alignment

• Find a string with minimum which contains all these sub-strings

• OutputAACTAGCGCTAGCTAACGACTAT GCTAGCTAACAACTAGC AGCGCTAGCTA TAACGACTAT

Example – Sorting By Reversal

• Given a sequence of integers 1..N• Every step you can reverse a

continuous sub-subsequence• Find the minimum number of steps

to sort it

Example – Sorting By Reversal

• Input– 4 3 1 5 6 2

• Output– 4 3 1 5 6 2 – 1 3 4 5 6 2– 1 6 5 4 3 2– 1 2 3 4 5 6

To know what algorithms are

• Mathematics– Symbols and notations– Methods

• Data Structures– Specification of input and output– Internal storage during processing

• You can recite them.• It is science. You can also derive

them.

To know how to use algorithms

• Techniques– Common ways that algorithms are built

upon• Formulations

– How to map the problem domain into entities in algorithms

• It is art. We have no way to teach you!

• That’s why you need exercises.

Techniques• Recursion

– Reduction into sub-problems• Divide and Conquer

– Reduction into >1 sub-problems– Collect the results

• Exhaustion– Expand all reductions into sub-problems

• Greedy– Use only 1 reduction into sub-problem(s)

• Dynamic Programming– Try all and remember the one

To know why algorithms work

• Complexity Analysis• Proof of correctness• Mathematics-oriented• Out of our scope, but …• To “invent” new algorithms …

HKOI concerns• All basics and techniques specified

above– Mathematics, Data Structures– Recursion, Exhaustion, Greedy,

• Algorithms which implement the techniques– Sorting, Searching, DFS, BFS, Prim, Kruskal– Dijkstra, Bellman-Ford, Warshall-Floyd

• Famous IO models used by algorithms– Tree, Graph– Search Space (Graph Searching)– State Space (Constraint Satisfaction)

Introduction to Algorithms

Questions?

HKOI also concerns

• Modifications to algorithms– To accommodate the situation

• Multi-state BFS

– For speed• Branch & Bound

– For memory• Bit-wise States

• Other paradigms– Open test data (known input)– Interactive (hidden input)

Complexity

• An approximation to the runtime and memory requirement of a program.

• Note that there are– Best-case complexity– Average-case complexity– Worst-case complexity

• In most cases, we concern runtime only.

Measuring Runtime• Definition

– An operation– Problem size

• Consider bubble sort:for i := N - 1 downto 1 dofor j := 1 to i doif A[j] > A[j+1] thenswap(A[j], A[j+1]);

• In worst case how many “if” operations?“swap” operations? “for j” operations?“for j” operations? Integer comparisons?

Complexity

• Give a name to a group of functions with similar growth property, e.g. use O(n2) to mean n2 + n, 2n2 – 10n, 10n2, …

• Since the constant won’t be too big practically, it is a good indicator of expected runtime.

• We uses “worst case runtime complexity” (a.k.a. order) extensively to describe runtime of a program

Graph

1

2

3

4

5

76

8

Graph

WA

NT

SA

Q

NSW

V

T

Graph

• A directed graph G=(V,E), where– V = the set of Node/Vertex– E = the set of Edge/Arc, where

• E=(v1,v2) where v1 and v2 are in V

• Extreme: |E| ≤ |V|2

• So we have real algorithms with order:– O(|V|3)– O(|V| + |E|)– O(|V| log |V| + |E|)– O(|E| log(log* |E| – log* |E|/|V|))

Difficulty of Problem• Definitions (INCORRECT)

– A problem with order being a polynomial is called polynomial-time solvable (P)

– A problem whose solution is verified in polynomial time is said to be polynomial-time verifiable (NP)

– A problem with no known polynomial-time solution to date is called NP-hard

• Difficulty of problems are roughly classified as:– Easy: in P (of course all P problems are also in NP)– Hard: in NP but not in P (NP-complete)– Very Hard: not even in NP

Complexity

• When you have a problem, given input (output) constraints and runtime constraints, you can guess whether an algorithm works or not.

• But sometimes the stated order of an algorithm depends on some tricky data structures, which we won’t have skills or time to implement!

IOI/NOI Problems Requires

• Understanding of basic algorithms• Innovation on ways to realize the

algorithms• Technical capability of the

realization• Remember: Different people care

different things in different situations

Finally

Information, What

Knowledge, How

Understanding, Why InductionDed

uction