node.js? done.scala!days2011.scala-lang.org/sites/days2011/files/56. node...implicit def...

node.js?done.scala!Implementing Scalable Async IO using

Delimited Continuations

Tiark Rompf, EPFL

Saturday, June 4, 2011

who has heard about node.js?

6/3/11 7:18 AMnode.js

of 4http://nodejs.org/

Evented I/O for V8 JavaScript.

An example of a web server written in Node which responds with"Hello World" for every request.

var http = require('http');http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World\n');}).listen(1337, "127.0.0.1");console.log('Server running at http://127.0.0.1:1337/');

To run the server, put the code into a file example.js and executeit with the node program:

% node example.jsServer running at http://127.0.0.1:1337/

Here is an example of a simple TCP server which listens on port1337 and echoes whatever you send it:

var net = require('net');

var server = net.createServer(function (socket) { socket.write("Echo server\r\n"); socket.pipe(socket);});

DownloadChangeLogAboutv0.4.8 docs

WikiBlogCommunityDemoJobs

Hash InsightsChickRx

purely asynchronous

callback driven

no blocking operations

good scalability

6/3/11 7:22 AMGoogle Trends: erlang, node.js, clojure,

of 1http://www.google.com/trends?q=erlang%2C+node.js%2C+clojure%2C&ctab=0&geo=all&date=all&sort=0

tiark.rompf@gmail.com | Signout

erlang, node.js, clojure,

Search Trends

Tip: Use commas to compare multiple search terms.

Searches Websites All regions All years

Scale is based on the average worldwide traffic of erlang in all years. Learn more

erlang 1.00 node.js 0.08 clojure 0.12

Rank by erlang

Scaling Large Projects With ErlangSlashdot - Jul 6 2008

The AZ of Programming Languages: ErlangComputerworld - Jun 16 2009

The AZ of Programming Languages: ClojureComputerworld - Aug 10 2009

Erlang Solutions, Ltd. and Basho Technologies, Inc. EnterBroad Partnership to Deliver Scalable, Fault TolerantApplications to a Global MarketCNNMoney.com - Mar 30 2010

Programming ClojureSlashdot - May 17 2010

Erlang and OTP in ActionSlashdot - Dec 8 2010

More news results »

Regions

1. Sweden

2. RussianFederation

3. SouthKorea

4. SouthAfrica

5. India

6. Singapore

7. Ukraine

8. Indonesia

9. China

10. Philippines

Cities

1. Goteborg,Sweden

2. Stockholm,Sweden

3.Moscow,RussianFederation

4. Beijing,China

5.SanFrancisco,CA, USA

6. Shanghai,China

7. Guangzhou,China

8. Seattle,WA, USA

9. Delhi, India

10. Bogota,Colombia

Languages

1. Swedish

2. Korean

3. Russian

4. Chinese

5. English

6. Indonesian

7. Danish

8. Hungarian

9. Japanese

10. German

Export this page as a CSV file

Google Trends provides insights into broad search patterns. Please keep in mind that several approximations areused when computing these results.

why would Scala programmers care?

#1: it’s about scalability

#2: no equivalent Scala libs

a little experiment: port it over to Scala!

done.scala

• a minimal scala port of node.js

• only net, http and supporting modules

• very incomplete (and not at all done!)

an interesting exercise:

JavaScript Scaladynamic language statically typed

• added type annotations in a few places

• helper classes for anonymous objects{ host: ‘google.com’, port: 80 }

• overloading, default arguments and varargs instead of dynamic typeof

quite smooth overall:

use Java NIO underneath

http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}) res.end('Hello World\n')}).listen(8000, "127.0.0.1")console.log('server running')

http.createServer { (req, res) => res.writeHead(200, "Content-Type" -> "text/plain") res.end("Hello World\n")}.listen(8000, "127.0.0.1")println("server running")

JavaScript

now what?

node’s conscious choice:only callback-driven apis

but is it the right choice?

small examples are fine but does it scale?

println(“hello”)setTimeout(1000) {

println(“world”)}

hello<pause>world

setTimeout(1000) {println(“world”)

}println(“hello”)

setTimeout(1000) {println(“world”)

}println(“hello”)

hello<pause>world

for (x <- List(1,2,3) {setTimeout(1000) {

println(“found ” + x)}

for (x <- List(1,2,3) {setTimeout(1000) {

println(“found ” + x)}

<pause>found 1found 2found 3

try {setTimeout(1000) {

throw new Exception}

} catch {case e =>

println(“caught: ” + e)}

try {setTimeout(1000) {

throw new Exception}

} catch {case e =>

println(“caught: ” + e)}

?Saturday, June 4, 2011

many things no longer work as expected

callback-driven programmingis really hard

val host = “http://search.twitter.com/”val query = “#scaladays”

http.get(host,80,“search.atom?q=”+query) { res => println(“STATUS:” + res.statusCode) res.onData { chunk => println(“BODY:” + chunk) } res.onEnd { println(“DONE”) }}

val host = “http://search.twitter.com/”val query = “#scaladays”

http.get(host,80,“search.atom?q=”+query) { res => println(“STATUS:” + res.statusCode) res.onData { chunk => println(“BODY:” + chunk) } res.onEnd { println(“DONE”) }}

STATUS: 400BODY: <data>BODY: <more data>DONE

val host = “http://search.twitter.com/”val queries = List(“#scala”,“#scaladays”, “@odersky”)

val results = for (q <- queries) yield { http.get(host,80,“search.atom?q=”+query) { res => ... }}

println(results)

val results = for (q <- queries) yield { http.get(host,80,“search.atom?q=”+query) { res => ... }}

println(results)

List(ClientRequest, ClientRequest, ClientRequest)

can we do something about it?

execute asynchronously

program synchronously

scalac -P:continuations:enable

def foobar: Int @suspendable = shift { retrn: (Int=>Unit) => retrn(7)}

def abort: Unit @suspendable = shift { retrn: (Unit=>Unit) => // just don’t return...

reset {println(“A”)abortprintln(“B”)

}println(“C”)

def foobar: Int @suspendable = shift { retrn: (Int=>Unit) => retrn(7)}

def abort: Unit @suspendable = shift { retrn: (Unit=>Unit) => // just don’t return...

reset {println(“A”)abortprintln(“B”)

}println(“C”)

def setTimeout(delay: Int)(callback: => Unit): Unit = ...

hello<pause>world

def sleep(delay: Int) = shift { retrn: (Unit=>Unit) => setTimeout {

retrn()}

println(“hello”)sleep(1000)println(“world”)

hello<pause>world

def sleep(delay: Int) = shift { retrn: (Unit=>Unit) => setTimeout {

retrn()}

println(“hello”)sleep(1000)println(“world”)

hello<pause>world

http.get(...) { res => res.onData { chunk => println(“BODY:” + chunk) } res.onEnd { println(“DONE”) }}

val res = http.get(...)

for (chunk <- res.data) { println(“BODY:” + chunk)}

println(“DONE”)

val res = http.get(...)

for (chunk <- res.data) { println(“BODY:” + chunk)}

println(“DONE”)

def data = new { def foreach(yld: Buffer=>Unit) = shift { retrn: (Unit=>Unit) =>

onData(yld)onEnd(retrn)

}def mkString = {

val s = new StringBuilder; foreach(s+=_); s.result}

}Saturday, June 4, 2011

val results = for (q <- queries) yield {val res = http.get(host,80,“search.atom?q=”+query)val xml = XML.loadString(res.data.mkString)xml \ "entry" \ "title" text

println(results)

val results = for (q <- queries) yield {val res = http.get(host,80,“search.atom?q=”+query)val xml = XML.loadString(res.data.mkString)xml \ "entry" \ "title" text

println(results)

<console>:11: error: no type parameters for method map: (f: (java.lang.String) => B)(implicit bf: scala.collection.generic.CanBuildFrom[List[java.lang.String],B,That])That exist so that it can be applied to arguments ((java.lang.String) => String @scala.util.continuations.cpsParam[Unit,Unit])

val results = for (q <- queries.suspendable) yield {val res = http.get(host,80,“search.atom?q=”+query)val xml = XML.loadString(res.data.mkString)xml \ "entry" \ "title" text

println(results)

val results = for (q <- queries.suspendable) yield {val res = http.get(host,80,“search.atom?q=”+query)val xml = XML.loadString(res.data.mkString)xml \ "entry" \ "title" text

println(results)

List(List(“#Scala highlighted in article from @TheEconomist on parallel programing and multicore: http://econ.st/jkGchu @typesafe”, ...), List(“There are so many great talks today at #scaladays; it's hard to decide which to attend!”, ...), List(“@odersky is there any info on Cascade up somewhere?”, ...))

implicit def richIterable[A,Repr](xs: IterableLike[A,Repr]) = new { def suspendable = new {

def foreach(yld: A => Unit @suspendable): Unit @suspendable = {val it = xs.iteratorwhile (it.hasNext) yld(it.next)

}def map[B, That](f: A => B @suspendable)

(implicit bf: CanBuildFrom[Repr, B, That]): That @suspendable = {val b = bf(xs.repr)foreach(x => b += f(x))b.result

val results = for (q <- List(1,2,3).suspendable) yield {sleep(1000); println(“tick”)

implicit def richIterable[A,Repr](xs: IterableLike[A,Repr]) = new { def suspendable = new {

def foreach(yld: A => Unit @suspendable): Unit @suspendable = {val it = xs.iteratorwhile (it.hasNext) yld(it.next)

}def map[B, That](f: A => B @suspendable)

(implicit bf: CanBuildFrom[Repr, B, That]): That @suspendable = {val b = bf(xs.repr)foreach(x => b += f(x))b.result

}<pause>tick<pause>tick<pause>tick

val results = for (q <- List(1,2,3).suspendable) yield {sleep(1000); println(“tick”)

val results = for (q <- queries.suspendable) yield {println(“start ”+q)val res = http.get(host,80,“search.atom?q=”+query)val xml = XML.loadString(res.data.mkString)println(“done ”+q)xml \ "entry" \ "title" text

start #scaladone #scalastart #scaladaysdone #scaladaysstart @oderskydone @odersky

def spawn(body: => Unit @suspendable) = {eventLoop.submitTask(() => reset(body))

class DataFlowCell[A] {private[this] var value: Option[A] = Noneprivate[this] var queue: List[A=>Unit] = Nil

def apply() = shift { retrn: (A=>Unit) =>value match {

case Option(v) => retrn(v)case None => queue ::= retrn

def set(v: A) = value match {case Option(_) => assert(false, “can’t set value twice”)case None => value = Some(v); queue.foreach(f => spawn(f(v)))

def future[A](body: => A @suspendable) = {val cell = new DataFlowCell[A]spawn { cell.set(body) }cell

def par[A,B](a: => A @suspendable)(b: => B @suspendable) = {val (u,v) = (future(a), future(b))(u(),v())

par { sleep(1000); println(“a”)

} { sleep(1000); println(“b”)

}println(“done”)

def future[A](body: => A @suspendable) = {val cell = new DataFlowCell[A]spawn { cell.set(body) }cell

def par[A,B](a: => A @suspendable)(b: => B @suspendable) = {val (u,v) = (future(a), future(b))(u(),v())

par { sleep(1000); println(“a”)

} { sleep(1000); println(“b”)

<pause>badone

implicit def richParIterable[A,Seq,Repr](xs: ParIterableLike[A,Seq,Repr]) = new {def suspendable = new {

def foreach(yld: A => Unit @suspendable): Unit @suspendable = {val futures = xs.seq.map(x => future(yld(x)) // sequential list of futuresfutures.suspendable.foreach(_.apply())

}def map[B, That](f: A => B @suspendable)(... bf ...): That @suspendable = {

val futures = xs.seq.map(x => future(yld(x))futures.suspendable.map(_.apply())

val results = for (q <- List(1,2,3).par.suspendable) yield {sleep(1000); println(“tick”)

implicit def richParIterable[A,Seq,Repr](xs: ParIterableLike[A,Seq,Repr]) = new {def suspendable = new {

def foreach(yld: A => Unit @suspendable): Unit @suspendable = {val futures = xs.seq.map(x => future(yld(x)) // sequential list of futuresfutures.suspendable.foreach(_.apply())

}def map[B, That](f: A => B @suspendable)(... bf ...): That @suspendable = {

val futures = xs.seq.map(x => future(yld(x))futures.suspendable.map(_.apply())

<pause>tickticktickdone

val results = for (q <- List(1,2,3).par.suspendable) yield {sleep(1000); println(“tick”)

val results = for (q <- queries.par.suspendable) yield {println(“start ”+q)val res = http.get(host,80,“search.atom?q=”+query)val xml = XML.loadString(res.data.mkString)println(“done ”+q)xml \ "entry" \ "title" text

start #scalastart #scaladaysstart @oderskydone #scaladone #scaladaysdone @odersky

Conclusions

• Scala needs good networking libs

• asynchronous execution does not meancallback-driven programming

• continuations can re-introduce synchronictiy

node.js? done.scala!days2011.scala-lang.org/sites/days2011/files/56. node...implicit def...

Documents

ÅRSRAPPORT SYKKYLVEN KOMMUNE 2017 · Venstre: 2 repr. Arbeidarpartiet: 6 repr. Senterpartiet: 4 repr. Kristeleg folkeparti: 2 repr. Sosialistisk venstreparti: 1 repr. Talet på politiske

ÅRSRAPPOR T 2018 - Sykkylven · Venstre: 2 repr. Arbeidarpartiet: 6 repr. Senterpartiet: 4 repr. Kristeleg folkeparti: 2 repr. Sosialistisk venstreparti: 1 repr. Talet på politiske

ANNAL REPR 2017 1 - uploads5.craft.co

Špin Mlyn-HS-fon-def-def-def

General Repr

Simulado 1 - static.educacaoadventista.org.brstatic.educacaoadventista.org.br/portal/canais/pdf/SimuladoF1... · 30328 - Simulado 1 Ano 1 Bim F1 Repr 184 9610 19 evereir 1998 Repr

days2011.scala-lang.orgdays2011.scala-lang.org/sites/days2011/files/31. Kafka.pdf · pub-sub messaging system Easy Kafka integration hint: Logger logger = Logger.getLogger(KafkaAppender.class)

Telecommunications UTILITY REPR - South Carolina

Repr jnted fr

s repr. masculino

SC65 Def trac - cwmagscwmags.com/brochures/Soilmec/SC-65/files/sc-65.pdf · SOILMEC NORTH AMERICA erasil UK SOILMEC France Deutsch I a n d SOILMEC Italia SOILMEC Moscow Repr. Office

A trip to the moon. Repr

STAKEHOLDER INV OLVEMENT IN KCE WORKING ESSES...Pres CEO Insur Pres Food presi Pres Secu Gene Medi Repr Repr Repr Interm Profe phys Profe nurse Hosp Socia Hous entre elgian Health

0HJDSWHUD QRYDHDQJOLDH na área de repr

M.r.u. repr

Female repr sy

To: Permanent Repr esentatives Committee From: Presidency

5-Dsr repr btn2

The Scala Programming Language - Delite: A Framework for …days2011.scala-lang.org/sites/days2011/files/06. Delite.pdf · 2011-09-09 · Heterogeneous Parallel DSLs Hassan Chafi,

Def Def Def 2