ITP 20005 Programming Languages Chapter 1 1.6 Higher-Order …contents.kocw.net/KOCW/document/2014/handong/kimyoungsup/... · 2016-09-09 · ITP 20005 Programming Languages Chapter

ITP 20005 Programming LanguagesChapter 1 1.6 Higher-Order Functions

Major references: 1. Structure and Interpretation of Computer Programs(SICP) by Abelson and Sussman, MIT

Composing Programs by John DeNero, Google2. MITx OPENCOURSEWARE(OCW) 6.00 SC Lecture 3. UC Berkley CS61A Lecture 4. Python, Wikipedia and many on-line resources.Prof. Youngsup Kim, [email protected], 2014 Programming Languages, CSEE Dept., Handong Global University

mailto:[email protected]

2

1.6 HOF: Environment for Function Composition

output?

3


Step 12 of 21

4


Then, what is the

output ?

compose1(f, g)(3)f = squareg = make_adder(2) make_adder(2) returns adder which is adder(k) and has n = 2. compose1(f, g) returns h which is h(x) = f(g(x))

h(3) = square(adder(3))= square(5)

5


6

1.6.2 Example: Functions as General Methods

Challenge: Implement a general method for iterative improvement to compute the golden ratio "phi" which is near 1.618.

http://www.youtube.com/watch?v=AzfHbETDbN8

Fibonacci and Golden ratio(mean)

7



Mat 7:12 So in everything, do to others

what you would have them do to you,

for this sums up the Law and the Prophets.

그러므로 무엇이든지 남에게 대접을 받고자 하는 대로 너희도 남을 대접하라 이것이율법이요 선지자니라.

8



How do you get this? HW?

9



𝑎 + 𝑏

𝑎= 1 +

𝑏

𝑎= 1 +

1

𝜑≡ 𝜑

s𝑖𝑛𝑐𝑒 1 +1

𝜑= 𝜑, 𝑡ℎ𝑒𝑛 1 + 𝜑 = 𝜑2

𝜑2 − 𝜑 − 1 = 0

∴ 𝜑 =1 + 5

2

10



11



1. Begin with a guess of a solution to an equation.

2. It repeatedly applies an update function to improve that guess.

3. Apply a close comparison to check whether current guess is close enough to end.

def improve(update, close, guess = 1):while not close(guess):

guess = update(guess)return guess

function

def update(guess): # get the next guessreturn 1 + 1/guess # 1 + 1/r = r by definition

def close_enough(guess):return approx_eq(guess * guess, guess + 1)

def approx_eq(x, y, tolerance=1e-10):return abs(x-y) < tolerance

12

1.6.5 Example: Newton’s Method

Finds approximations to zeroes of differentiable functions.

• Application: a method for (approximately) computing square roots, using only basic arithmetic.

• The positive zero of 𝒇 𝒙 = 𝒙𝟐 − 𝒂 is 𝑎

13


𝑥0

𝑥1

𝑓(𝑥)

𝑥𝑛

Get the next guess 𝑥1 that closes to the solution.

Begin with a function 𝑓(𝑥) and an initial guess 𝑥0.

Find 𝑥𝑛 such that 𝑓(𝑥𝑛) = 0.

By the formula, the next guess 𝑥1 = 𝒙𝟎 −𝒇(𝒙𝟎)

𝒇′(𝒙𝟎)

(𝑥0, 𝒇 𝑥0 )

− 𝒇 𝑥0

14


𝑥0

𝑥1

𝑓(𝑥)

𝑥𝑛




Compute the value of f at the initial guess: 𝒇 𝑥0

Compute the derivative of f at the initial guess: 𝒇′ 𝑥0

Update next guess 𝑥1 to be:𝑥1 = 𝒙𝟎 −𝒇(𝒙𝟎)

𝒇′(𝒙𝟎)

Algorithm:

Visualization: http://en.wikipedia.org/wiki/File:NewtonIteration_Ani.gif

15


𝑥0

𝑥1

𝑓(𝑥)

𝑥𝑛




By the formula, the next guess 𝑥1 = 𝒙𝟎 −𝒇(𝒙𝟎)

𝒇′(𝒙𝟎)

(𝑥0, 𝒇 𝑥0 )

− 𝒇 𝑥0

How do you get this formula to

begin with ?

16


𝑥0

𝑥1

𝑓(𝑥)

𝑥𝑛




(𝑥0, 𝒇 𝑥0 )

− 𝒇 𝑥0

𝑥2

17


𝑥0

𝑥1

𝑓(𝑥)

𝑥𝑛

(𝑥0, 𝒇 𝑥0 )

− 𝒇 𝑥0

𝑥2


𝒙𝟏 = 𝒙𝟎 −𝒇(𝒙𝟎)

𝒇′(𝒙𝟎)

𝒙𝒏 = 𝒙𝒏−𝟏 −𝒇(𝒙𝒏−𝟏)

𝒇′(𝒙𝒏−𝟏)

𝒙𝟐 = 𝒙𝟏 −𝒇(𝒙𝟏)

𝒇′(𝒙𝟏)

18


𝑥0

𝑥𝑛

𝑥1

(𝑥0, 𝒇 𝑥0 )

− 𝒇 𝑥0

𝒙𝟏 = 𝒙𝟎 −𝒇(𝒙𝟎)

𝒇′(𝒙𝟎)

𝒙𝒏 = 𝒙𝒏−𝟏 −𝒇(𝒙𝒏−𝟏)

𝒇′(𝒙𝒏−𝟏)

𝑥2

𝒙𝟐 = 𝒙𝟏 −𝒇(𝒙𝟏)

𝒇′(𝒙𝟏)

𝑓(𝑥)


Use the tangent of 𝒇(𝒙) at 𝒙𝟎 to formulate the relation between 𝒙𝟎 and 𝒙𝟏.

𝒇′ 𝒙𝟎 =𝟎 − 𝒇(𝒙𝟎)

𝒙𝟏 − 𝒙𝟎

(𝒙𝟏 −𝒙𝟎) 𝒇′ 𝒙𝟎 =− 𝒇(𝒙𝟎)

𝒙𝟏 𝒇′ 𝒙𝟎 = 𝒙𝟎 𝒇′ 𝒙𝟎 − 𝒇(𝒙𝟎)

Therefore, 𝒙𝟏 = 𝒙𝟎 −𝒇(𝒙𝟎)

𝒇′(𝒙𝟎)

the slope or tangent of 𝒇(𝒙) at 𝒙𝟎 is 𝒇′ 𝒙𝟎 .

19


Visualization: http://en.wikipedia.org/wiki/File:NewtonIteration_Ani.gif

Compute the value of f at the guess: 𝒇 𝑥0

Compute the derivative of f at the guess: 𝒇′ 𝑥0

Update next guess 𝑥1 to be:𝑥1 = 𝒙𝟎 −𝒇(𝒙𝟎)

𝒇′(𝒙𝟎)

𝑥0

𝑥1

(𝑥0, 𝒇 𝑥0 )

− 𝒇 𝑥0

𝑓(𝑥)


𝑥𝑛

20

1.6.5 Example: Using Newton’s Method

How to find the square root of 2?

>>> f = lambda x: x*x - 2

>>> find_zero(f)

1.4142135623730951 𝑓 𝑥 = 𝑥2 − 2

How to find the log base of 1024?

>>> g = lambda x: pow(2, x) – 1024

>>> find_zero(f)

10.0 𝑔 𝑥 = 2𝑥 − 1024

21


How to compute square_root(a)

Idea: Iteratively refine a guess x about the square root of a.

Update: 𝑥 = 𝑥 −𝒇 𝒙

𝒇′ 𝒙= 𝑥 −

𝒙𝟐−𝑎

2𝑥=

𝑥+𝑎

𝑥

2

initial guessnext guess

𝒇 𝒙 = 𝒙𝟐 − 𝒂

𝒇′ 𝒙 = 𝟐𝒙

22




Update: 𝑥 =𝑥+

𝑎

𝑥

2

𝒙 – 𝒇(𝒙)/𝒇’(𝒙)

𝑩𝒂𝒃𝒚𝒍𝒐𝒏𝒊𝒂𝒏𝑴𝒆𝒕𝒉𝒐𝒅

Implementation questions:

next guess

initial guess

23




Update: 𝑥 =𝑥+

𝑎

𝑥

2

𝒙 – 𝒇(𝒙)/𝒇’(𝒙)

𝑩𝒂𝒃𝒚𝒍𝒐𝒏𝒊𝒂𝒏𝑴𝒆𝒕𝒉𝒐𝒅


• What guess should start the computation?

• How do we know when we are finished?

24

1.6.5 Example: Square Roots – Special Case



Demo:>>> def square_root(a):... x = 1... while x*x != a:... x = (x + a/x) / 2... return x>>> square_root(4)2.0>>> square_root(2)Traceback (most recent call last):File "<stdin>", line 1, in <module>File "<stdin>", line 4, in square_root

KeyboardInterrupt

Why not working?

It never reached the right number since the limitation of the floating point number representation of the computer.

25

1.6.5 Example: Square Roots – Special Case



def sqrt(n):approx = n/2.0better = (approx + n/approx)/2.0while better != approx:

approx = betterbetter = (approx + n/approx)/2.0

return approx

By repeatedly applying the formula until the better approximation is equal to the previous one.

Demo:

Difference?:

26

1.6.5 Example: Cube Roots – Special Case

How to compute cube_root(a)


Update:

𝒇 𝒙 = 𝒙𝟑 − 𝒂

𝒇′ 𝒙 = 𝟑𝑥2

27





𝒇′ 𝒙

next guess initial guess

28





𝒇′ 𝒙= 𝑥 −

𝒙𝟑−𝑎

3𝑥2=

2𝑥+𝑎

𝑥2

3

next guess initial guess

29




Update: 𝑥 =2𝑥+

𝑎

𝑥2

3

𝒙 − 𝒇(𝒙)/𝒇’(𝒙)next guess

30




Update: 𝑥 =2𝑥+

𝑎

𝑥2

3

𝒙 − 𝒇(𝒙)/𝒇’(𝒙)


• What guess should start the computation?

• How do we know when we are finished?

31

1.6.5 Iterative Improvement

First, identify common structure.

Then define a function that generalized the procedure.

def iter_improve(update, done, guess=1, max_updates=1000):

"""Iteratively improve guess with update until done

returns a true value.

>>> iter_improve(golden_update, golden_test)

1.618033988749895

"""

k = 0

while not done(guess) and k < max_updates:

guess = update(guess)

k = k + 1

return guess

32

1.6.5 Newton’s Method for nth Roots

• 𝒇 𝒙 = 𝒙𝒏 − 𝒂 𝒇 𝒙 = 𝟎, 𝒙= 𝑛 𝑎= 𝒑𝒐𝒘 𝒙,𝒏 − 𝒂

• 𝒇′ 𝒙 = 𝒏 𝒙𝒏−𝟏

= 𝒏 ∗ 𝒑𝒐𝒘(𝒙, 𝒏 − 𝟏)• Update: 𝒙 = 𝒙 − 𝒇(𝒙)/𝒇’(𝒙)

• Reminder: for square root 𝒇 𝒙 = 𝒙𝟐 − 𝒂 = 0 𝒙 = 𝑎

Documents

ITP 20005 Programming Languages Chapter 1 1.6 Higher-Order …contents.kocw.net/KOCW/document/2014/handong/kimyoungsup/... · 2016-09-09 · ITP 20005 Programming Languages Chapter