Scientific Programming [24pt]Backtrackingdisi.unitn.it/~montreso/sp/slides/B07-backtrack.pdfSi consideri ancora una scacchiera nán; lo scopo trovare un Ògiro di cavalloÓ, ovvero

Scientific Programming

Backtracking

Alberto Montresor

Università di Trento

2018/12/15

This work is licensed under a Creative CommonsAttribution-ShareAlike 4.0 International License.

references

http://creativecommons.org/licenses/by-sa/4.0/

http://creativecommons.org/licenses/by-sa/4.0/

Table of contents

1 Introduction2 Enumeration

SubsetsPermutationsGames

Introduction

Introduction

Problem classes (decisional, search, optimization)

Definition bases on the concept of admissible solution: a solutionthat satisfies a given set of criteria

Typical problems

Build one or all admissible solutionCounting the admissible solutionsFind the admissible solution "largest", "smallest", in general"optimal"

Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 1 / 37

Introduction

Typical problems

Enumeration

List algorithmically all possible solutions (search space)

Example: list all the permutations of a set

Build at least a solution

We use the algorithm for enumeration, stopping at the firstsolution found

Example: identify a sequence of steps in the Fifteen game


Introduction

Typical problems

Count the solutions

In some cases, it is possibleto count in analytical way

Example: counting thenumber of subsets of kelements taken by a set of nelements

n!

k! (n− k)!

In other cases, we build thesolutions and we count them

Example: number of subsetsof a integer set S whose sumis equal to a prime number


Introduction

Typical problems

Find optimal solutions

We enumerate all possible solutions and evaluate them through acost function

Only if other techniques are not possible:Dynamic programmingGreedy

Example: Hamiltonian circuit (Traveling salesman)


Introduction

Build all solutions

To build all the solutions, we use a "brute-force" approach

Sometimes, it is the only possible way

The power of modern computer makes possible to deal withproblems medium-small problems

10! = 3.63 · 106 (permutation of 10 elements)220 = 1.05 · 106 (subsets of 20 elements)

Sometimes, the space of all possible solutions does not need to beanalyzed entirely


Introduction

Backtracking

Approach

"Prova a fare qualcosa, e se non va bene, disfalo e provaqualcos’altro"

"Ritenta, e sarai più fortunato"

How it works?

A systematic methods to iterate all possible instances of a researchspaceIt is an algorithm technique that, as the others, need to be"customized" for the specific problem to be solved.


Introduction

General organization

General organization

A solution is represented by a list SThe content of element S[i] is taken from a set of choices C thatdepends on the problem

Examples

C generic set, possible solutions permutations of CC generic set, possible solutions subsets of CC game moves, possible solutions a sequence of movesC edges of a graph, possible solutions paths


Introduction

Partial solutions

At each step, we start from a partial solution S where k ≥ 0choices have been already taken

If S[0 : k] is an admissible solution, we "process" itE.g., we can print itWe can then decide to stop here or keep going by listing/printing allsolutions

If S[0 : k] is not a complete solution:If possible, we extended solution S[0 : k] with one of the possiblechoices to get a solution S[0 : k + 1]Otherwise, we "cancel" the element S[k] (backtrack) and we goback to to solution S[0 : k − 1]


Introduction

Decision trees

Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions

S[0]

S[3]


Introduction

Decision trees


S[0]

S[1]

S[3]


Introduction

Decision trees


S[0]

S[1]

S[2]

S[3]


Introduction

Decision trees


S[0]

S[1]

S[2]

S[3]


Introduction

Decision trees


S[0]

S[1]

S[2]

S[3]


Introduction

Decision trees


S[0]

S[1]

S[2]

S[3] S[3]


Introduction

Decision trees


S[0]

S[1]

S[2]

S[3]


Introduction

Decision trees


S[0]

S[1]

S[3]


Introduction

Decision trees


S[0]

S[1]

S[3]

S[2]


Introduction

Decision trees


S[0]

S[1]

S[3]

S[2]

S[3]


Introduction

Pruning

"Branches" of the trees that do not bring to admissible solutionscan be "pruned"The evaluation is done in the partial solutions corresponding tointernal nodes


Introduction

Pruning

"Branches" of the trees that do not bring to admissible solutionscan be "pruned"The evaluation is done in the partial solutions corresponding tointernal nodes


Table of contents

1 Introduction2 Enumeration

SubsetsPermutationsGames

Enumeration

Enumeration

boolean enumeration(object[ ] S, int n, int i, . . . )Set C = choices(S, n, i, . . .) % Compute C based on S[0 : i− 1]foreach c ∈ C do

S[i] = cif isAdmissible(S, n, i) then

if processSolution(S, n, i, . . .) thenreturn True

if enumeration(S, n, i+ 1, . . .) thenreturn True

return False


Enumeration

Enumeration

S: list containing the partial solutions

i: current index

. . .: additional information

C: the set of possible candidates to extend the current solution

isAdmissible(): returns True if S[0 : i] is an admissible solution

processSolution(): returnsTrue to stop the execution at the first admissible solutionFalse to explore the entire tree


Enumeration Subsets

Example 1

List all subsets of {0, . . . , n− 1}

def subsets(S,n,i):C = [1, 0] if i<n else []for c in C:

S[i] = cif i==n-1:

if (processSolution(S)):return True

else:subsets(S,n,i+1)

return False

def processSolution(S):for i in range(len(S)):

if (S[i]==1):print(i, "", end="")

print("")return False

n = 4S = [0]*nsubsets(S,4,0)


Enumeration Subsets

Example 1 (Clean-up version)

List all possible subsets of {0, . . . , n− 1}

def subsets(S,n,i):for c in [1, 0]:

S[i] = cif i == n-1:

processSolution(S)else:

subsets(S,n,i+1)


Enumeration Subsets

Example 1

There is no pruning. All the possible space is explored.But this is required by the definition of the problem

Computational complexity O(n2n)

In which order sets are printed?

Is it possible to think to an iterative version, ad-hoc for thisproblem?(non-backtracking)


Enumeration Subsets

Example 1

subsets(int n)for j = 0 to 2n − 1 do

print ’{’for i = 0 to n− 1 do

if (j and 2i) 6= 0 thenprint i

print ’}’


Enumeration Subsets

Example 2

List all possible subsets of size k of a set {0, . . . , n− 1}

Iterative versionWhat is the cost?

Solution based on backtrackingCan we prune?


Enumeration Subsets

Example 2 - Attempt 1

def subsets(S, k, n,i):C = [1,0]for c in C:

S[i] = cif i == n-1:

if admissible(S, k):processSolution(S)

else:subsets(S, k, n, i+1):

def admissible(S, k):return sum(S) == k


if (S[i]==1):print(i, "", end="")


n = 4k = 2S = [0]*nsubsets(S, 2, 4, 0)


Enumeration Subsets


def subsets(S, k, n, i, count):C = [1, 0]for c in C:

S[i] = ccount = count + cif i == n-1:

if count == k:processSolution(S)

else:subsets(S,k, n, i+1, count)

count = count - c

def admissible(S, k):tot = sum(S)return tot == k


if (S[i]==1):print(i, "", end="")


n = 4k = 2S = [0]*nsubsets(S, 2, 4, 0, 0)


Enumeration Subsets


def subsets(S, k, n, i, count):if count < k and count + (n-i) >= k:

C = [1,0]else

C = []for c in C:

S[i] = ccount = count + cif count == k:

processSolution(S)else:

subsets(S, k, n, i+1, count)count = count - cS[i] = 0


if (S[i]==1):print(i, "", end="")


n = 4k = 2S = [0]*nsubsets(S, 2, 4, 0, 0)


Enumeration Subsets

Example 2: advantages

© Alberto Montresor 20

Esempio 2: vantaggi


Enumeration Subsets

Example 2 - Summary

Cosa abbiamo imparato?

By "customizing" the generic algorithm, we can get a moreefficient solution

It is efficient forsmall values of k (close to 1)large values of k (close to n)

No great saving for n/2

It is difficult to obtain the same efficiency with an iterativealgorithm


Enumeration Permutations

Example 3

Print all permutations of a set A

def permutations(S, i, original):for c in original:

S[i] = coriginal.remove(c)if (len(original)==0):

print(S)else:

permutations(S, i+1, original)original.add(c)

original = { 1,2,3,4 }S = [0]*len(original)permutations(S, 0, original)Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 23 / 37

Enumeration Games

Eight queens puzzle

ProblemThe eight queens puzzle is the problem of placing eight chess queens onan 8× 8 chessboard so that no two queens threaten each other

History:Introduced by Max Bezzel(1848)Gauss found 72 of the 92solutions

Let’s start from the stupidestapproach, and let’s refine thesolution step by step


Enumeration Games

Eight queens puzzle

Idea: There are n2 possible positions where a queen may be located

S[0 : n2] binary array S[i] = True⇒ "queen in i"

isAdmissible() i == n2

choices(S, n, i) {True,False}

pruning if the new queen threaten one of theexisting queens, returns ∅

# Solutions for n = 8 264 ≈ 1.84 · 1019

Comments

Maybe we have a representation problem?Binary matrix, really sparse


Enumeration Games

Eight queens puzzle

Idea: For each queen, there are n2 possible positions

S[0 : n] coordinates in{0 . . . n2 − 1}

S[i] coordinate of queen i

isAdmissible() i == n

choices(S, n, i) {0 . . . n2 − 1}

pruning returns the subset of legal moves

# Solutions for n = 8 (n2)n = 648 = 248 ≈ 2.81 · 1014

Comments

Better, but the space is still huge . . .Problem: how to distinguish between a solution "1-7-. . . " and asolution "7-1-. . . " ?


Enumeration Games

Eight queens puzzle

Idea: do not place queens in positions that occur "before" the onesalready taken

S[0 : n] coordinates in{0 . . . n2 − 1}

S[i] coordinate of queen i


choices(S, n, i) {0 . . . n2 − 1}

pruning returns legal moves, S[i] > S[i− 1]

# Solutions for n = 8 (n2)n/n! = 248/40320 ≈ 6.98 · 109

Comments

Very good, but there is still space for improvementAlberto Montresor (UniTN) SP - Backtracking 2018/12/15 27 / 37

Enumeration Games

Eight queens puzzle

Idea: each row of the board must contain exactly one queen

S[0 : n] coordinates in{0 . . . n− 1}

S[i] column of queen i, where row = i


choices(S, n, i) {0 . . . n− 1}

pruning returns only legal columns

# Solutions for n = 8 nn = 88 ≈ 1.67 · 107

Comments

Almost there


Enumeration Games

Eight queens puzzle

Idea: every column must contain exactly one queen

S[0 : n] coordinates in{0 . . . n− 1}

permutations of {1 . . . n}


choices(S, n, i) {0 . . . n− 1}

pruning removes diagonals

# Solutions for n = 8 n! = 8! = 40320

Comments

Solutions actually visited = 15720


Enumeration Games

Eight queens puzzle

def queens(S, i, columns):for c in columns:

S[i] = ccolumns.remove(c)if (diagonalsOK(S,i)):

if len(columns)==0:printBoard(S)

else:queens(S,i+1,columns)

columns.add(c)


Enumeration Games

Knight’s tour

ProblemA Knight’s tour is a sequence of moves of a knight on a chessboard suchthat the knight visits every square only once.

© Alberto Montresor 30

Giro di cavallo

✦ Problema: ✦ Si consideri ancora una scacchiera n·n; lo scopo è trovare un “giro di cavallo”,

ovvero un percorso di mosse valide del cavallo in modo che ogni casella venga visitata al più una volta

✦ Soluzione: ✦ Tramite backtrack


Enumeration Games

Knight’s tour

Solution

Board n× n whose cells contains:0 if the cell has not been visitedi if the cell has been visited at step i

# Solutions: 64! ≈ 1089

But: at each step, there are at most 8 possible cells, so themaximum number of visits is 864 ≈ 1057

In reality, much less thank to pruning


Enumeration Games

Knight’s tour

def knight(S, i, x, y):n = len(S)C = moves(S, x, y)if (i == n*n):

for k in range(n):print(S[k])

return Trueelse:

S[x][y] = ifor c in C:

if knight(S, i+1, x+dx[c], y+dy[c]):return True

S[x][y] = 0return False


Enumeration Games

Knight’s tour

dx = [-1, +1, +2, +2, +1, -1, -2, -2]dy = [-2, -2, -1, +1, +2, +2, +1, -1]

def moves(S, x, y):res = set()for i in range(8):

nx = x + dx[i]ny = y + dy[i]if (0 <= nx < 8) and (0 <= ny < 8) and S[nx][ny] ==0 :

res.add(i)return res


Enumeration Games

Sudoku - "Suuji wa dokushin ni kagiru"

2 5 9 7 6

2 4

1 5 3 9

8 9 4 5 2 6

1 2 4

2 5 6 7 3

8 3 2 1

9 7

3 7 8 9 2

2 5 3 8 9 1 4 7 6

8 9 7 2 6 4 3 1 5

6 4 1 5 7 3 9 2 8

7 8 9 4 3 5 2 6 1

1 3 6 7 2 9 8 5 4

4 2 5 6 1 8 7 3 9

9 6 8 3 5 2 1 4 7

5 1 2 9 4 7 6 8 3

3 7 4 1 8 6 5 9 2


Enumeration Games

Sudoku

boolean sudoku(int[ ][ ] S, int i)

int x = i mod 9int y = bi/9cSet C = Set()if i ≤ 80 then

if S[x, y] 6= 0 thenC.insert(S[x, y])

elsefor c = 1 to 9 do

if check(S, x, y, c) thenC.insert(c)

int old = S[x, y]foreach c ∈ C do

S[x, y] = cif i = 80 then

processSolution(S, n)return True

if sudoku(S, i+ 1) thenreturn True

S[x, y] = oldreturn False


Enumeration Games

Sudoku

boolean check(int[ ][ ] S, int x, int y, int c)for j = 0 to 8 do

if S[x, j] = c thenreturn False % Column check

if S[j, y] = c thenreturn False % Row check

int bx = bx/3cint by = by/3cfor ix = 0 to 2 do

for int iy = 0 to 2 do % Subtable checkif S[bx · 3 + ix, by · 3 + iy] = c then

return False

return True


Documents

Scientific Programming [24pt]Backtrackingdisi.unitn.it/~montreso/sp/slides/B07-backtrack.pdfSi consideri ancora una scacchiera nán; lo scopo trovare un Ògiro di cavalloÓ, ovvero