Functional languages and how to implement them

Functional languages and how to implement them

Karl-Filip FaxénKTH/IMIT/LECS

Functional programming

A functional program computes a result from a set of argumentscompute a target program from a source programcompute a bitmap image from a scene description

A function (subprogram) is a function (mapping)computes a result from arguments withno side effects!

Well known functional languages includeStandard ML (not purely functional)Haskell (purely functional)

Functional languages

Garbage collection Polymorphic typing

Catches all type errors at compile timeTypes are inferred and need not be givenPolymorphism allows generic functions to be

written

First class functionsAllows for higher order functions

Eager (strict) or lazy (nonstrict) evaluationStandard ML is strictHaskell is nonstrict

Strict or nonstrict?

let z = x/y in if y == 0 then 1 else z

What happens if y=0?

Strict or nonstrict?

let z = x/y in if y == 0 then 1 else z

What happens if y=0? - In a strict language (SML): a run-time error - In a lazy language (Haskell): return the value 1

Strict or nonstrict?

if y == 0 then 1 else x/y

What happens if y=0?

Strict or nonstrict?

if y == 0 then 1 else x/y

What happens if y=0? - In a strict language (SML): return the value 1 - In a lazy language (Haskell): return the value 1

Data structures

data List a = Nil | Cons a (List a)

Data structures

data List a = Nil | Cons a (List a)

Defines a new type name List ...

Data structures

data List a = Nil | Cons a (List a)

Defines a new type name List ...and two constructors, Nil and Cons.

Data structures

data List a = Nil | Cons a (List a)

The type List is parameterized by the type parameter a which represents the type of the list elements (think of templates in C++)

Data structures

data List a = Nil | Cons a (List a)

The constructor Nil has no arguments ...

Data structures

data List a = Nil | Cons a (List a)

The constructor Nil has no arguments ...but the constructor Cons has two ...

Data structures

data List a = Nil | Cons a (List a)

The constructor Nil has no arguments ...but the constructor Cons has two ...the first is a list element ...

Data structures

data List a = Nil | Cons a (List a)

The constructor Nil has no arguments ...but the constructor Cons has two ...the first is a list element ...and the second is the rest of the list.

Data structures

data List a = Nil | Cons a (List a)

Some examples of lists:

Cons 1 (Cons 2 (Cons 3 Nil)) :: List IntCons True (Cons False (Cons False Nil)) :: List Bool

Data structures

data List a = Nil | Cons a (List a)

Some examples of lists:

Cons 1 (Cons 2 (Cons 3 Nil)) :: List IntCons True (Cons False (Cons False Nil)) :: List Bool

The type of the list elements instantiate thetype parameter of List

Data structures

data List a = Nil | Cons a (List a)

Some examples of lists:

Cons 1 (Cons 2 (Cons 3 Nil)) :: List IntCons True (Cons False (Cons False Nil)) :: List Bool

The type of the list elements instantiate thetype parameter of List

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

Defines the function take; take n xs returns the first n elements of xs or xs if xs is shorter than n

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

Definition by pattern matching; a kind of implicit case statement.The first equation that matches is selected.

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

A literal pattern matches if the argument is that literal(in this case 0) ...

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

A literal pattern matches if the argument is that literal(in this case 0) ...a variable pattern matches any value (and binds the variable to it) ...

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

A literal pattern matches if the argument is that literal(in this case 0) ...a variable pattern matches any value (and binds the variable to it) ...so the first equation is chosen if the first argument is 0

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

A constructor pattern matches if ...

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

A constructor pattern matches if ...the argument is built with the same constructor ...

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

A constructor pattern matches if ...the argument is built with the same constructor ...and the component patterns match the values of the componentsof the argument ...

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

A constructor pattern matches if ...the argument is built with the same constructor ...and the component patterns match the values of the componentsof the argument ...so the second equation is chosen if

the first argument is not zero, andthe second argument is not the empty list

Computing with data structures

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

The function take is polymorphic; its type istake :: Int -> List a -> List a

since the list elements are only extracted from and stored indata structures

Higher order functions

filter p (Cons x xs) = if p(x) then Cons x (filter p xs) else filter p xsfilter p Nil = Nil

The parameter p is a function ...

Higher order functions

filter p (Cons x xs) = if p(x) then Cons x (filter p xs) else filter p xsfilter p Nil = Nil

The parameter p is a function ...and filter p xs returns the elements of xs for which p returns True

Lazy evaluation

sieve (Cons x xs) = Cons x (sieve (filter g xs)) where g y = remainder y x /= 0

from n = Cons n (from (n+1))

primes n = take n (sieve (from 2))

Compute the first n primes as the first n elements of the(infinite) list of all primes

Lazy evaluation

sieve (Cons x xs) = Cons x (sieve (filter g xs)) where g y = remainder y x /= 0

from n = Cons n (from (n+1))

primes n = take n (sieve (from 2))

Compute the first n primes as the first n elements of the(infinite) list of all primes

Lazy evaluation

sieve (Cons x xs) = Cons x (sieve (filter g xs)) where g y = remainder y x /= 0

from n = Cons n (from (n+1))

primes n = take n (sieve (from 2))

Compute the first n primes as the first n elements of the(infinite) list of all primes

Representing data structures

Header

list element

next element

GC info

Constructor #2

Constructor #Nil 1Cons 2

In the stack or In the heap or In the text segment in registers in the data segment

Header

Value (e.g. 1)

Cons node Cons descriptor

Representing functions

sieve (Cons x xs) = Cons x (sieve (filter g xs)) where g y = remainder y x /= 0

The function g is passed to filter; how is it represented?

Representing functions

sieve (Cons x xs) = Cons x (sieve (filter g xs)) where g y = remainder y x /= 0

The variable x is free in the body of g, so just a code pointeris not enough!

Representing functions

Header

value of x

GC info

Code for thebody of g

Header

Value (e.g. 1)

Function node Function descriptor

This pointer ispassed tofilter

Implementing lazy evaluation

from n = Cons n (from (n+1))

The function from produces its result one element at a time(Just In Time evaluation!)

Implementing lazy evaluation

from n = Cons n (from (n+1))

The suspended computation contains the free variable n

Implementing lazy evaluation

Header

value of n

GC info

Code forevaluating the

expressionfrom (n+1)

Header

Value (e.g. 1)

Thunk node Thunk descriptor

This pointer isstored in the Cons node

Implementing lazy evaluation

from n = Cons n (thunk from (thunk (eval n)+1))

Thunks and evals can be made explicit in an intermediate languagefor lazy evaluation (Faxén -95, ...)

Why lazy functional programs are (sometimes) slow

There are overheads from some featuresExtra cost for building thunks which are

eventually evaluated (most thunks are)Evaluating a thunk is expensive since it involves

an indirect call (bad for optimization and pipelines)

Polymorphism and GC needs uniform representations; thunks force this representation to be boxed

Difficult to make array operations efficient

Why lazy functional programs are (sometimes) slow

An inefficient programming style is encouragedMany small functionsFrequent use of higher order functionsLinked data structures rather than monolithic

onesMany intermediate data structures

But this style is good for programmers!

Optimizing functional programs

Standard optimizationsInlining, constant propagation and folding, etc

Optimizations of lazy evaluationStrictness analysisCheap eagerness (Faxén -95, -00, -02)Sharing analysis (to avoid updates) (Faxén -97)

Representation analysis (Faxén -99) Deforestation (fusion) Cloning (Faxén -01)

Strictness analysis

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

The function take is strict in its first argument since it is necessaryto know its value in order to produce any output ...

Strictness analysis

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

The function take is strict in its first argument since it is necessaryto know its value in order to produce any output ...because it is needed for pattern matching

Strictness analysis

take 0 xs = Niltake n (Cons y ys) = Cons y (take (n-1) xs)take n Nil = Nil

The function take is not strict in its second argument since it is not necessary to know its value if the first argument is 0

Cheap eagerness

from n = Cons n (thunk from (thunk (eval n)+1))

primes n = take n (thunk sieve (from 2))

The function from is not strict since it can produce a Cons nodewithout using its argument ...

Cheap eagerness

from n = Cons n (thunk from (thunk (eval n)+1))

primes n = take n (thunk sieve (from 2))

The function from is not strict since it can produce a Cons nodewithout using its argument ...so the thunk can not be eliminated by strictness analysis

Cheap eagerness

from n = Cons n (thunk from (thunk (eval n)+1))

primes n = take n (thunk sieve (from 2))

Only the eval is expensive in the thunk body ...

Cheap eagerness

from n = Cons n (thunk from (thunk (eval n)+1))

primes n = take n (thunk sieve (from 2))

Only the eval is expensive in the thunk body ...and only if n can bound to a thunk ...

Cheap eagerness

from n = Cons n (thunk from (thunk (eval n)+1))

primes n = take n (thunk sieve (from 2))

Only the eval is expensive in the thunk body ...and only if n can bound to a thunk ...which it unfortunately can since the thunk is passed to from ...

Cheap eagerness

from n = Cons n (thunk from ( (eval n)+1))

primes n = take n (thunk sieve (from 2))

Only the eval is expensive in the thunk body ...and only if n can bound to a thunk ...which it unfortunately can since the thunk is passed to from ...but if the thunk is eliminated, the eval becomes cheap ...

Cheap eagerness

from n = Cons n (thunk from ( ( n)+1))

primes n = take n (thunk sieve (from 2))

Only the eval is expensive in the thunk body ...and only if n can bound to a thunk ...which it unfortunately can since the thunk is passed to from ...but if the thunk is eliminated, the eval becomes cheap ...and can even be eliminated!

Cheap eagerness

Cheap eagerness is about speculative evaluationOnly legal if the speculated computation

terminates with no run-time errorOnly good if the speculated expression takes

little time to evaluate or is very likely to be evaluated anyway

Reduces execution time by 12-58% (Faxén -00)

Cloning

Several versions of each function is generatedDifferent versions optimized for different call sitesParticularly important for representation analysis

Code growth might be a problemIf identical clones are shared, code growth is

typically at most 20%Some programs actually shrink!

Reduces execution time by 11-29%

Conclusion

Functional languages are good for programmersProgramming on a high level of abstractionSubprograms can be combined in interesting

ways

Lazy languages in particular are not efficientHigh overheadsEncourage inefficient programming style

But the compiler can save the day!