J. Glenn Brookshear

1

Slide 12-1 Copyright © 2009 Pearson Education, Inc.

J. Glenn Brookshear

C H A P T E R 3

Chapter 12

J. Glenn Brookshear蔡文能

Theory ofComputation

2


Chapter 12: Theory of Computation

• 12.1 Functions and Their Computation

• 12.2 Turing Machines

• 12.3 Universal Programming Languages

• 12.4 A Noncomputable Function

• 12.5 Complexity of Problems

• 12.6 Public-Key Cryptography

3


Functions

• Function: A correspondence between a collection of possible input values and a collection of possible output values so that each possible input is assigned a single output

• Computing a function: Determining the output value associated with a given set of input values

• Noncomputable function: A function that cannot be computed by any algorithm

4


Complexity of Problems

• Time Complexity: The number of instruction executions required– Unless otherwise noted, “complexity” means “time

complexity.”

• A problem is in class O(f(n)) if it can be solved by an algorithm in (f(n)).

• A problem is in class (f(n)) if the best algorithm to solve it is in class (f(n)).

5


Software Efficiency

• Measured as number of instructions executed

– Let f (n) be a function. f (n) is the maximum number of basic operations performed by the algorithm on any input size n.

• notation for efficiency classes– O( ? )

– ( ? ) ( ? )

• Best case, worst case, and average case

Time Complexity vs. Space Complexity

How shall we measure the amount of work done by an algorithm?

6


Asymptotic Upper Bound (Big O)

f(n)

c g(n)• f(n) c g(n) for all n n0 • g(n) is called an asymptotic upper bound of f(n).• We write f(n)=O(g(n))• It reads f(n) equals big oh of g(n).

n0

7


Asymptotic Lower Bound (Big Omega)

f(n)

c g(n)

• f(n) c g(n) for all n n0 • g(n) is called an asymptotic lower bound of f(n).• We write f(n)=(g(n))• It reads f(n) equals big omega of g(n).

n0

8


Asymptotically Tight Bound (Big Theta)

f(n)

c1 g(n)

• f(n) = O(g(n)) and f(n) = (g(n))• g(n) is called an asymptotically tight bound of f(n).• We write f(n)=(g(n))• It reads f(n) equals theta of g(n).

n0

c2 g(n)

9


P versus NP

• Class P: All problems in any class (f(n)), where f(n) is a Polynomial

• Class NP: All problems that can be solved by a nondeterministic algorithm in polynomial time

Nondeterministic algorithm = an “algorithm” whose steps may not be uniquely and completely determined by the process state

• Whether the class NP is bigger than class P is currently unknown.

10


Figure 12.12: A graphic summation of problem classification

I s ? P NPUS$1,000,000 dollars for a solution:

http://www.claymath.org/millennium/

11


Figure 12.1 An attempt to display the function that converts measurements in yards into meters

12


Figure 12.2 The components of a Turing machine

13


Turing Machine Operation

• Inputs at each step– State– Value at current tape position

• Actions at each step– Write a value at current tape position– Move read/write head– Change state

14


Figure 12.3 A Turing machine for incrementing a value

15


- - ADD, *, L

RETURN,1,R CARRY,0,L CARRY,*,R

RETURN,1,R CARRY,0,L Overflow,1,L

RETURN,*,R RETURN,*,R RETURN,*,R

RETURN,0,R RETURN,1,R HALT,*, stop

START

ADD

CARRY

Overflow

RETURN

0 1 *

16


CHURCH-TURING THESIS

• The functions that are computable by a Turing machine are exactly the functions that can be computed by any algorithmic means.

• Any well-defined procedure that can be grasped and performed by the human mind and pencil/paper, can be performed on a conventional digital computer with no bound on memory.

The Church-Turing Thesis is NOT a theorem. It is a statement of belief concerning the universe we live in.

17


Universal Programming Language

A language with which a solution to any computable function can be expressed– Examples: “Bare Bones” and most popular

programming languages

18


The Bare Bones Language

• Bare Bones is a simple, yet universal language.

• Statements– clear name; // while(x){--x;}– incr name; // ++x;– decr name; // --x;– while name not 0 do; … end;

如何做出 x = y; // copy y to x

如何做出 if(x) state_1; else state_2 ; 的效果 ?

如何做出 invert(x); ?

19


Figure 12.4 A Bare Bones program for computing X x Y

Z = X * Y;

問題 :

原來的 X 還在嗎 ?

如何保存 ?

Hint: 原來的 Y 還在 !

20


Figure 12.5: A Bare Bones implementation of the instruction “copy Today to Tomorrow”

tmp = 0;Tomorrow = 0;while(Today) { tmp++; Today--;}while (tmp) { Today++; Tomorrow+; tmp--;}

Tomorrow = Today;

21


Alan Turing (1912-1954)

• Published On Computable Numbers … (1936)

– Introduced the Halting Problem – Formal model of computation

(now known as “Turing Machine”)

• Codebreaker at Bletchley Park– Broke Enigma Cipher ( 二次大戰德軍用 )– Perhaps more important than Lorenz

• After the war: convicted of homosexuality (then a crime in Britain), committed suicide eating cyanide apple ( 含氰化物的蘋果 )

Lorenz: 另一種加密法

22


Turing 的重要貢獻 Turing 1936 年 11 月號的 Journal of Symnbolic Logic 發表 On

Computable Numbers, with an Application to the Entscheidungsproblem ( 論可計算數及其在判定性問題上之應用 )

Turing 於 1950 年發表了一篇思慮清晰的哲學論文：計算機械與智慧

(Computing Machinery and Intelligence) ，是人工智慧研究上，一篇歷史性的文獻。 Turing test (Imitation Game; 模仿遊戲 ) Turing 1954 年 6 月 8 日 Turing 死後，其在邏輯與計算機科學領域

的貢獻慢慢獲得肯定，美國計算機協會 (Association for Computing Machinery;ACM) 從 1966 年起設立 Turing 獎，做為對計算科學有貢獻人士的最高獎項。 (Turing 獎被譽為計算機科學界的諾貝爾獎 )

1998 年 6 月 22 日，英國下議院通過修改法條，使得十六歲以上同性或異性間的自願性行為均屬合法。 2009 年 9 月 10 日，英國首相布朗發出公函，正式向 1952 年被政府因其同性戀性傾向受迫害的Alan Mathison Turing 正式道歉。 (2010 年五月英國大選後首相已換David Cameron)

23


Bletchley Park ( 布雷奇萊園區 )

• During World War II the German armed forces top secret codes were broken at Bletchley Park, providing the allies with vital information towards their war effort. Situated 50 miles North-West of London, the site played host to a diverse group of code breakers, including Alan Turing and Dilly Knox. Among the ciphers that were broken were Enigma( 奇謎密碼 ) and Lorenz.

• 在第二次世界大戰期間破解 Enigma 因此大西洋之役中使英國船隊免於遭到德國潛艇 (U-boot) 狼群戰術的攻擊

英國破解密碼的情報單位

24


The Caesar Cipher

• One of the simplest examples of a substitution cipher is the Caesar cipher, which is said to have been used by Julius Caesar to communicate with his army. Caesar is considered to be one of the first persons to have ever employed encryption for the sake of securing messages.

• Caesar decided that shifting each letter in the message would be his standard algorithm, and so he informed all of his generals of his decision, and was then able to send them secured messages. Using the Caesar Shift (3 to the right), the message,

"RETURN TO ROME"

would be encrypted as:

"UHWXUQ WR URPH"

凱撒密碼

25


The Enigma Cipher ( 奇謎密碼 )

• The Enigma cipher is most well known for it's contributions to World War II on the Germans' side. They developed what came to be known as The Enigma Machine.

• The machine was based on a system of three rotors that substituted cipher text letters for plain text letters. The rotors would spin in conjunction with each other, thus performing varying substitutions much like the Caeser Shift.

26


The CODE Book (by Simon Singh ; 賽門‧辛 )

• 人類自開始書寫以來即伴隨著密碼術的產生，我們可以從若干歷史事件當中看出密碼術的演化過程，以及密碼本身又是如何扮演著改變歷史的關鍵性角色。

一場持續了三千多年的攻防戰至今仍方興未艾，編碼者不斷努力建造更強的密碼系統來防衛通訊，解碼者則不斷發明更有威力的方法來破解密碼。

兩者之間的爾虞我詐，動員了各種學科的知識與技術，將人類科技文明推向不可思議的極限。編碼者和解碼者之間這場演化競爭的最終贏家是誰？我們且拭目以待！

http://www.kmt.org.tw/Forum/ViewOpinion.asp?Art_Id=1865

http://www.books.com.tw/exep/prod/booksfile.php?item=0010131762

27


The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography

第一章　蘇格蘭瑪麗女王的密碼第二章　無法破解的密碼第三章　秘密書寫的機械化第四章　破解「奇謎」 (Enigma Cipher)第五章　語言障礙第六章　愛麗絲和巴伯的公開鑰匙第七章　極佳隱私 (PGP)第八章　躍進量子世界密碼挑戰‧十階通達一萬英鎊

附錄Ａ　小說《虛空》開頭第一段附錄Ｂ　頻率分析的基本要訣附錄Ｃ　所謂的「聖經密碼」附錄Ｄ　豬圈密碼附錄Ｅ　波雷費密碼附錄Ｆ　 ADFGVX 密碼附錄Ｇ　單次鑰匙簿的回收缺點附錄Ｈ　《每日電訊報》縱橫字謎的秘密附錄Ｉ　尚待解譯的古文字附錄Ｊ　 RSA所使用的數學

碼書

28


Windtalkers獵風行動 -- 主演 : Nicolas Cage• 密碼法發展史上，第一個重要的方法－維吉內爾密

碼法 (Vigenere cypher) ，這個密碼法在十六世紀被發展出來，也一舉擊潰當時破解密碼者擅長的頻率分析法。

• 但是，這個當時堪稱無敵的密碼法到了十九世紀，仍舊被破解了，破解之人則是提出現代計算機模型的先驅—現代計算機之父 : 巴貝奇 (Charles Babbage) 。

• 另一種加密法－使用少數民族語言，例如在二次世界大戰中當時美軍為了不讓日軍破解密碼所使用的納瓦荷語 (Navajo) 。

• 2001/07/26 美國總統布希 26 日頒獎給一批納瓦荷族美國印第安人，表揚他們錯綜複雜的語言提供美軍一種無法破解的密碼，在第二次世界大戰期間協助美國擊敗日本的貢獻。將此一事蹟拍成電影的華人導演吳宇森也應邀觀禮。

29


Computability Theory

The theory of what can and can’t be computed by an ideal computer is called Computability Theory or Recursion Theory.

30


Fix any finite set of symbols, . Fix any precise programming language, e.x., Java. A program is a finite string of characters that is syntactically valid.A function f: * -> * is computable if there is a program P that when executed on an ideal computer, computes f. That is, for all strings x belongs to *,

P(x) = f(x).

Computable Function

31


A problem P is decidable if some Turing machinedecides (solves) the problem.

Decidable problems:

• Does Machine M have three states ?

• Is string w a binary number?

• Does DFA M accept any input?

Decidability

Consider problems with answer YES or NO

32


Decision problems

Example: Is the number n prime?

Why focus on decision problems?

A decision problem is a computational problem witha yes or no answer.

Decision problems are simple: This makes it easyto develop a rigorous mathematical theory.

Decision problems are surprisingly general:Many other problems can be recast in terms ofdecision problems that are essentially equivalent.

33


Recasting other problems as decision problems

Multiplication decision problem: Is the kth bit ofthe product of m and n a one?

Factoring problem: What is the smallest non-trivialfactor of n?

Multiplication problem: What is the product of m and n?

Factoring decision problem: Does n have a non-trivialfactor smaller than k?

Time required to solve one of these problems is the same (to within a small overhead) as the time required to solve the other.

Time required to solve one of these problems is the same (to within a small overhead) as the time required to solve the other.

34


Turing MachineInputprobleminstance

YES

NO

The Turing machine that decides (solves) a problem answers YES or NO for each instance of the problem

35


The machine that decides (solves) a problem:

• If the answer is YES then halts in a yes state

• If the answer is NO then halts in a no state

These states may not be final states

36


YES states

NO states

Turing Machine that decides a problem

YES and NO states are halting states

37


Some problems are undecidable:which means: there is no Turing Machine that solves all instances of the problem

A simple undecidable problem:

The Halting problem

A problem is undecidable if it cannot be solved by any Turing machine that halts on all inputs.

38


The Halting problem

P(x) means the output that arises from running program P on input x

P(P) means the output obtained when we run P on the text of its own source code.

• Write a program HALT such that:

HALT(P) = yes, if P(P) haltsHALT(P) = no, if P(P) does not halt

• Given the encoded version of any program, return 1 if the program is self-terminating, or return 0 if the program is not.

39


THEOREM: There is no program to solve the halting problem

(Alan Turing 1937)•Suppose a program HALT, solving the halting problem, existed:

•HALT(P)= yes, if P(P) halts•HALT(P)= no, if P(P) does not halt

• We will call HALT as a subroutine in a new program called CONFUSE.

40


CONFUSE(P):If HALT(P) then loop_for_ever;Else return (i.e., halt);<text of subroutine HALT goes here>

--------------------------------------------Does CONFUSE(CONFUSE) halt?

YES implies HALT(CONFUSE) = yes thus, CONFUSE(CONSFUSE) will not halt

NO implies HALT(CONFUSE) = no thus, CONFUSE(CONFUSE) halts

CONTRADICTION

41


Figure 12.6 Testing a program for self-termination

42


Figure 12.7 Proving the unsolvability of the halting program

43


Does program x halt on input of x?

0 if program halts on input ( )

1 otherwise

x xh x

Is there an algorithm to solve the halting problem, thatis, to compute h(x)?

Suppose such an algorithm exists.PROGRAM: TURING(x)

IF h(x) = 1 THEN HALTELSE loop forever

Let T be the program number forTURING.

TURING(T) halts h(T) = 1 h(T) = 0

Contradiction!

The halting problem again

44


Figure 12.8 A procedure MergeLists for merging two lists

45


Figure 12.9 The merge sort algorithm implemented as a procedure MergeSort

46


Figure 12.10 The hierarchy of problems generated by the merge sort algorithm

47


Efficiencyinsolvable

solublesoluble

insolvable

solublehard

easy

Nomenclature: easy = “tractable” = “efficiently computable”

hard = “intractable” = “not efficiently computable”Definition: A problem is easy if there is a Turing machineto solve the problem that runs in time polynomial in the size of the problem input. Otherwise the problem is hard.

This definition is usually applied to both decision problems and more general problems.

48


Can it be decided efficiently?

• Class P problems: problems that can be solved in polynomial time with a deterministic machine.

• Class NP problems (Cook 1971): problems that can be solved in Polynomial time with a Non-deterministic machine.

Theory: unknown if P = NP.

• Class NP-Complete problems: a problem X is NP-Complete if it can be converted in polynomial time to an NP-Complete problem Y.

49


NonDeterministic Turing Machine

• States–0 Initial–1 Record–2 Look for 0–3 Look for 1–4 Scan Left–5+ Rest of program

1,B,R — —

Read Symbol

B 0 1

Cu

rren

t S

tate

accept,B,R 2,B,R 3,B,R

— 2,0,R 4,2,L 2,1,R

— 3,0,R 3,1,R 4,2,L

5,B,R 4,0,L 4,1,L

0

1

2

3

4

Nondeterministic TM: 2 possible actions from single point

‘ —’ means no possible action from this point

Deterministic TM: At most one possible action at any point

50


Figure 12.11 Graphs of the mathematical expressions n, lg n, n lg n, and n2

51


Many important problems aren’t known to be in P

Example: Factoring.

Example: The Traveling Salesman Problem (TSP).

22 km

16 km

12 km15 km

23 km

14 km19 km

Goal: Find the shortest tour through all the cities.

Traveling salesman decision problem: Given a networkand a number, k, is there a tour through all the cities of length less than k?

It is widely believed that neither of these problems is in P.

52


P vs. NP

Polynomial Time PTIME = [k nk

L is in P if there exists a Turing Machine M which for every x, decides if x is in L in polynomially many steps.

Non-Deterministic Polynomial Time

L is in NP if there exists a Turing Machine M s.t. for every x

• If x is in L then there exists w s.t. M(x,w)→“Yes” in PTIME.• If x is not in L then there is no such w.

53


Input for Problem B

Output for Problem B

Algorithm for Problem B

Reductionfrom B to A

Algorithmfor A

x R(x) Yes/No

Reducibility

NP-Hard: A problem Π is NP-hard if every problem in NP

has a polynomial-time reduction to Π.If Π is in ΝP and NP-hard then Π is NP-

complete.Moral: Π is At least as hard as any other problem in

NP

B p A

PNP p Π

54


NP-Complete

• What, intuitively, does it mean if we can reduce problem P to problem Q?– P is “no harder than” Q

• How do we reduce P to Q?– Transform instances of P to instances of Q in polynomial time

s.t. Q: “yes” iff P: “yes”

• What does it mean if Q is NP-Hard?– Every problem PNP p Q

• What does it mean if Q is NP-Complete?– Q is NP-Hard and Q NP

Q is At least as hard as any other problem in NP

P p Q

Q至少與 P 一樣難

55


Boolean SAT problem

• Given a proposition logic formula, decide if it is satisfiable. ( 有沒有一組輸入使其輸出為 1 ?)

• A formula is satisfiable if there exists a variable assignment such that the formula becomes true.

• Boolean SAT is NP complete (Cook 1971)

56


Boolean SAT problem (Circuit Satisfiability )

• Satisfying Assignment: truth assignment inducing output = 1

• Circuit Satisfiability is NP-complete

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

Boolean Boolean Combinational Combinational Gate Types: Gate Types: AND, NOT, ORAND, NOT, OR

single circuit output

circuit inputs

Language DefLanguage Def: CIRCUIT-SAT = : CIRCUIT-SAT =

{<C>:C is a satisfiable boolean {<C>:C is a satisfiable boolean combinational circuit.}combinational circuit.}

34.8

57


SAT is NP-complete

• Cook’s Theorem– Can generate Boolean formula that checks whether NDTM

accepts string in polynomial time

• Translation Procedure– Given

• NDTM M• Polynomial function p• Input string x

– Generate formula F• F is satisfiable iff M accepts x in time p(|x|)

– Size of F is polynomial in |x|– Procedure generates F in (deterministic) time polynomial in

|x|

Translation

M

x

p

F

58


NP-Completeness Proofs

• Proving a Language NP-Complete

source: 91.503 textbook Cormen et al.source: 91.503 textbook Cormen et al.

•Proving a Language NP-Hard– Do all the above steps except (1)

59


Expanding List of Hard Problems via Reduction

source: textbook Cormen et al.source: textbook Cormen et al.

Circuit-SATCircuit-SAT

TheoremTheorem: :

- If any NP-complete problem is polynomial-time solvable, then P=NP. - If any NP-complete problem is polynomial-time solvable, then P=NP.

- Equivalently, if any problem in NP is not polynomial-time solvable, - Equivalently, if any problem in NP is not polynomial-time solvable,

then no NP-complete problem is polynomial-time solvable.then no NP-complete problem is polynomial-time solvable.

• Relationships among some NP-complete problems

SATSATCIRCUIT P

CNFSAT P 3 CNF = Conjunctive Normal FormCNF = Conjunctive Normal Formconjunctionconjunction: AND of clauses: AND of clausesclauseclause: OR of literal(s): OR of literal(s)

60


Public-Key Cryptographic Algorithms

• RSA and Diffie-Hellman • Diffie-Hellman in 1976

– Echange a secret key securely– Compute discrete logarithms

• RSA - Ron Rives, Adi Shamir and Len Adleman at MIT, in 1977.– RSA is a block cipher– The most widely implemented– Relies on the (presumed) intractability of the problem of factoring

large numbers• Key: A value used to encrypt or decrypt a message

– Public key: Used to encrypt messages– Private key: Used to decrypt messages

一次切取一區塊 ( 例64Bytes) 來加解密

Block cipher vs. Stream cipher

Encryption vs. digital signature

61


Figure 12.13 Public key cryptography

62


The RSA Algorithm – Key Generation

1. Select p,q p and q both prime2. Calculate n = p x q3. Calculate 4. Select integer e5. Calculate d6. Public Key KU = {e,n}7. Private key KR = {d,n}

)1)(1()( qpn)(1;1)),(gcd( neen

)(mod1 ned

63


Example of RSA Algorithm

1. Select p,q p =7, q =172. Calculate n = p x q =7 x 17 = 1193. Calculate = 964. Select integer e=5 5. Calculate d =776. Public Key KU = {e,n} = {5, 119}7. Private key KR = {d,n} = {77, 119}

)1)(1()( qpn)(1;1)),(gcd( neen

)(mod1 ned

因為 77 x 5 = 385 = 4 x 96 + 1

64


Example of RSA Algorithm (cont.)

65


Diffie-Hellman Key Echange 和 q 是雙方先約好或由一方送給另一方 (A 送給B)

雙方算出的 K 會相等

66


Checksumming: Cyclic Redundancy Check(CRC)

• view data bits, D, as a binary number

• choose r+1 bit pattern (generator), G

• goal: choose r CRC bits, R, such that– <D,R> exactly divisible by G (modulo 2)

– receiver knows G, divides <D,R> by G. If non-zero remainder: error detected!

– can detect all burst errors less than r+1 bits

• widely used in practice (ATM, HDCL)

67


CRC Example

Want:D.2r XOR R = nG

equivalently:D.2r = nG XOR R

equivalently: if we divide D.2r by

G, want remainder R

R = remainder[ ]D.2r

G

68


Chapter 12Theory of Computation

謝謝捧場[email protected]

蔡文能

Thank You!Thank You!

Documents

J. Glenn Brookshear