HKOI 2005 Training

Introduction to AlgorithmsAlan, Tam Siu Lung

What is Algorithm?

• Algorithm– A set of data manipulations instructions to solve a

problem (informatics problem)• Problem

– A mapping from each input (in a specific format) to an output (in a specific format)

• Pseudo-Code– Realization of Algorithm in the current domain

• Mapping from problem input to algorithm input• Mapping from algorithm output to problem output

• Program– Realization of Pseudo-Code given

• A compiler and the language supported• Data in plain text format and the IO routines supported

Visualization of Program

A BAlgorithm1

CodingPseudo-Code

Source-Codeint main() {

A a = Read();

B b = Algorithm1(a);

C c = Algorithm2(b);

D d = Algorithm3(c);

Write(d);

}

B CAlgorithm2

C DAlgorithm3

Algorithm Example

• Sorting Problem– Given an array of N comparable elements

with comparable keys– Output a permutation of the elements– such that no element is has a key greater

than that of the one there before

• Note– Array = a sequence of elements– Element = a key-value pair– Permutation = ...– Greater than = …

Realization Example

• To print in sorted order a set of points in a cetacean plane by the distance from origin:

• Realization contains– Key = distance from origin– Value = coordinates = (x, y)– Comparison of 2 elements = comparison of the

numerical values of distances

• Pseudo-Code contains– Store the set in an array– Sort them– Output the array in order

Why Algorithms?

• Computational Biology

• Database

• Cryptography

• Quantitative Analysis

• Artificial Intelligence

• Scheduling

Example – Multiple Sequence Alignment

• Input– GCTAGCTAAC– AACTAGC– AGCGCTAGCTA– TAACGACTAT

• Find a string with minimum which contains all these sub-strings

Example – Multiple Sequence Alignment

• Find a string with minimum which contains all these sub-strings

• OutputAACTAGCGCTAGCTAACGACTAT GCTAGCTAACAACTAGC AGCGCTAGCTA TAACGACTAT

Example – Sorting By Reversal

• Given a sequence of integers 1..N

• Every step you can reverse a continuous sub-subsequence

• Find the minimum number of steps to sort it

Example – Sorting By Reversal

• Input– 4 3 1 5 6 2

• Output– 4 3 1 5 6 2 – 1 3 4 5 6 2– 1 6 5 4 3 2– 1 2 3 4 5 6

To know what algorithms are

• Mathematics– Symbols and notations– Methods

• Data Structures– Specification of input and output– Internal storage during processing

• You can recite them.• It is science. You can also derive

them.

To know how to use algorithms

• Techniques– Common ways that algorithms are built

upon

• Formulations– How to map the problem domain into

entities in algorithms

• It is art. We have no way to teach you!

• That’s why you need exercises.

Techniques

• Recursion– Reduction into sub-problems

• Divide and Conquer– Reduction into >1 sub-problems– Collect the results

• Exhaustion– Expand all reductions into sub-problems

• Greedy– Use only 1 reduction into sub-problem(s)

• Dynamic Programming– Try all and remember the one

To know why algorithms work

• Complexity Analysis

• Proof of correctness

• Mathematics-oriented

• Out of our scope, but …

• To “invent” new algorithms …

HKOI concerns

• All basics and techniques specified above– Mathematics, Data Structures– Recursion, Exhaustion, Greedy,

• Algorithms which implement the techniques– Sorting, Searching, DFS, BFS, Prim, Kruskal– Dijkstra, Bellman-Ford, Warshall-Floyd

• Famous IO models used by algorithms– Tree, Graph– Search Space (Graph Searching)– State Space (Constraint Satisfaction)

Introduction to Algorithms

Questions?

HKOI also concerns

• Modifications to algorithms– To accommodate the situation

• Multi-state BFS

– For speed• Branch & Bound

– For memory• Bit-wise States

• Other paradigms– Open test data (known input)– Interactive (hidden input)

Complexity

• An approximation to the runtime and memory requirement of a program.

• Note that there are– Best-case complexity– Average-case complexity– Worst-case complexity

• In most cases, we concern runtime only.

Measuring Runtime

• Definition– An operation– Problem size

• Consider bubble sort:for i := N - 1 downto 1 do

for j := 1 to i doif A[j] > A[j+1] then

swap(A[j], A[j+1]);

• In worst case how many “if” operations?“swap” operations? “for j” operations?“for j” operations? Integer comparisons?

Complexity

• Give a name to a group of functions with similar growth property, e.g. use O(n2) to mean n2 + n, 2n2 – 10n, 10n2, …

• Since the constant won’t be too big practically, it is a good indicator of expected runtime.

• We uses “worst case runtime complexity” (a.k.a. order) extensively to describe runtime of a program

Graph

1

2

3

4

5

76

8

Graph

WA

NT

SA

Q

NSW

V

T

Graph

• A directed graph G=(V,E), where– V = the set of Node/Vertex– E = the set of Edge/Arc, where

• E=(v1,v2) where v1 and v2 are in V

• Extreme: |E| ≤ |V|2

• So we have real algorithms with order:– O(|V|3)– O(|V| + |E|)– O(|V| log |V| + |E|)– O(|E| log(log* |E| – log* |E|/|V|))

Difficulty of Problem

• Definitions (INCORRECT)– A problem with order being a polynomial is called

polynomial-time solvable (P)– A problem whose solution is verified in polynomial

time is said to be polynomial-time verifiable (NP)– A problem with no known polynomial-time solution to

date is called NP-hard

• Difficulty of problems are roughly classified as:– Easy: in P (of course all P problems are also in NP)– Hard: in NP but not in P (NP-complete)– Very Hard: not even in NP

Complexity

• When you have a problem, given input (output) constraints and runtime constraints, you can guess whether an algorithm works or not.

• But sometimes the stated order of an algorithm depends on some tricky data structures, which we won’t have skills or time to implement!

IOI/NOI Problems Requires

• Understanding of basic algorithms

• Innovation on ways to realize the algorithms

• Technical capability of the realization

• Remember: Different people care different things in different situations

Finally

Information, What

Knowledge, How

Understanding, Why InductionDed

uctio

n