Upload
err
View
20.454
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
Chris Wanstrath
http://defunkt.github.com
hi everyone, i’m chris wanstrath
i love randy rhoads
and kurt vonnegut
i live in san francisco
and work at github
(which is written in rails)
but! i’m not gonna talk about any of that stuff
today i want to talk about python
and comet
(among other things)
the real-time web (And other buzzwords)
By Chris Wanstraththis talk is titled “the real-time web (and other buzzwords)”
?so what is the “real-time” web?
techcrunch and readwriteweb would have you believe it’s a way to get your RSS stories
faster
but is that it?
no.
it’s all about pushing
instead of polling
!it’s getting told what’s new
?instead of asking for what’s new
instant chat in the browser
and information the moment it’s available
Client Server
one persistent connection
ClientServer
instead of many, short lived connections
right now when we say “real-time web” we usually mean one of three things
comet
flash’s XMLSocket
or HTML5’s WebSocket
let’s start with comet
?how many people know what comet is?
how many people know how it differs from XML socket or WebSocket?
good! for a long time i had no idea what comet was
!i think it has a big marketing problem.
see, comet is a cleaning product.
just like ajax
so it’s kind of like,
Me, too!
“me too”
I, too, am a revolutionary web
technique you didn’t know existed and can start using
today that will forever change the way you imagine the
web experience.
“I, too, am a revolutionary web technique you didn’t know existed and can start using today that will forever change the way you think about the web experience.”
...
and you’re like
Uhh...
uh...
AJAX!!!
if i had my way...
i’d call it something else
like maybe asteroid
i dunno, there’s some cool imagery there
i dunno, there’s some cool imagery there
i dunno, there’s some cool imagery there
i dunno, there’s some cool imagery there
i dunno, there’s some cool imagery there
i dunno, there’s some cool imagery there
i dunno, there’s some cool imagery there
anything but comet
anyway, comet is any standards compliant technique which pushes or streams data to the browser over HTTP
with comet you can essentially fake a socket connection between a browser and a backend server
how about an example
well, for this year’s django dash
me
alex gaynor... whoops...
alex gaynor
and leah culver
( World’s smallest park )
(seen here next to the world’s smallest park)
built leafychat
irc in your browser using comet
you can connect to freenode channels
see who’s online
and do the irc thing
right in your browser (demo?)
all without ajax
?how?
well, there are a few components at play here
Browser
first you’ve got the browser (naturally)
Browser Apache80
Apache80
then we have apache
Browser Apache80
Django8000
Apache80
Django8000
sitting behind apache is django (via mod_wsgi or whatever)
Browser Apache80
Django8000
Apache80
Django8000
the browser hits leafychat.com, port 80, which hits apache. apache sees it’s a django request
Browser Apache80
Django8000
Apache80
Django8000
and hands off the request.
Browser Apache80
Django8000
Apache80
Django8000
once django generates a response, apache takes it and delivers it
Browser Apache80
Django8000
Apache80
Django8000
back to the browser. that’s, more or less, our HTTP request cycle
Browser
comet connections work in a similar fashion
Browser Orbited8100
Orbited8100
instead of apache, we have Orbited sitting on port 8100
orbited is an open source, python comet server powered...
by twisted.
it does all of the comet heavy-lifting for us
Browser Orbited8100
Orbited8100
so.
Browser Orbited8100
Orbited8100
Zeddicus8200
sitting behind orbited we have our app-specific comet code. for leafy chat it was a twisted-based daemon named zeddicus. it handled all the IRC stuff - connecting to channels, sending messages, receiving private messages, logging, all that
Orbited8100
Orbited8100
orbited handles generic browser stuff...
Zeddicus8200
while our app-specific daemon (zeddicus) deals with the business logic.
in this case, irc stuff
Browser Orbited8100
Orbited8100
Zeddicus8200
does this look familiar?
Browser Apache80
Django8000
Apache80
Django8000
no? ok.
Browser Orbited8100
Orbited8100
Zeddius8200
Zeddicus8200
anyway, the browser (using orbited’s bundled js library) makes a request...
Browser Orbited8100
Zeddius8200
Orbited8100
Zeddius8200
Zeddicus8200
to port 8100. orbited sees it’s a request for zeddicus and hands it off to our backend
Browser Orbited8100
Zeddius8200
Orbited8100
Zeddius8200
Zeddicus8200
zeddicus sees a new socket connection open, does some stuff, then writes to the socket
Browser Orbited8100
Zeddius8200
Orbited8100
Zeddius8200
Zeddicus8200
orbited reads what zeddicus wrote to the socket connection they share
Browser Orbited8100
Zeddius8200
Orbited8100
Zeddius8200
Zeddicus8200
then sends it back the browser via comet
Browser Apache80
Django8000
Apache80
Django8000
just like before
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><head> <script> // session data var session_nick = "" var session_channels = [] </script> <script type="text/javascript" src="/static/js/soundmanager2/soundmanager2-nodebug-jsmin.js"></script> <script type="text/javascript" src="/static/js/audio-player.js"></script>
<title>Leafy Chat</title>
<link rel="icon" href="/static/img/favicon.ico"/>
<link rel="stylesheet" href="/static/css/reset.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/static/css/facebox.css" type="text/css" media="screen/> <link rel="stylesheet" href="/static/css/base.css" type="text/css" media="screen" />
<script> document.domain = document.domain; </script> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script> <script type="text/javascript" src="http://leafychat.com:8100/static/Orbited.js"></script> <script type="text/javascript" src="/static/js/facebox.js" charset="utf-8"></script> <script type="text/javascript" src="/static/js/cookie.js" charset="utf-8"></script> <script type="text/javascript" src="/static/js/leafy.js" charset="utf-8"></script> <script type="text/javascript" src="/static/js/kahlan.js" charset="utf-8"></script>
<div id="main"> <div id="header"> <div id="navigation" class="rounded"> <ul class="nav-list"> <!-- this should be a list --> <li>Hi <span id="welcome-user">there</span>!</li>
but while apache and django deal with html
Browser Orbited8100
Zeddius8200
Orbited8100
Zeddius8200
Zeddicus8200
zeddicus, our back-end daemon, concerns itself only...
with json.
orbited supports STOMP, XMPP, and raw IRC, too, but JSON is the way to go.
why do i say that?
well, unfortunately this is where orbited does not rock
orbited only cares about giving you a socket, you have to do everything else yourself.
that’s the deal
so you need to design your own protocol...
how it interacts with your back-end daemon
how events are fired and responded to
all that boring stuff
we ended up using a combination of django signals and json
Browser Orbited8100
Zeddius8200
Orbited8100
Zeddius8200
Zeddicus8200
so this is our overview, except with one small change
Browser Orbited8100
Zeddius8200
Zeddicus8200
these are actually persistent connections
Browser Orbited8100
Zeddius8200
Zeddicus8200
between orbited and zeddicus it’s a tcpsocket
Browser Orbited8100
Zeddius8200
Zeddicus8200
between the browser and orbited it’s a comet technique
Browser Zeddicus8200
we jump through all these hoops because it allows us to write our app as if the browser is connecting DIRECTLY to zeddicus via a tcpsocket
Browser Zeddicus8200
which is what the “real time” web is all about and what comet gives us
the browser writes to and reads from a socket, our back-end daemon does the same.
fast communication
Browser Orbited8100
Zeddius8200
Zeddicus8200
alright, so let’s talk about this part a bit more
the comet part
Browser Orbited8100
Zeddius8200
Zeddicus8200
there are a few different ways to fake a persistent connection to a server with modern browsers
xhr long polling
- hang an xhr request until we get a response or 30s, then re-open and wait again
<script>script tag long polling
- dynamically create a script tag pointing to a url that hangs or times out, rinse and repeat
<iframe>forever frame
- open a Content-Encoded: chunked url in an iframe, each chunk is a <script> tag that runs
xhr streaming
- set content-encoding: chunked and trigger onreadystate callback with each chunk
so now you know our white lie: we don’t have real persistence, or a real socket.
we fake it at both ends.
or, really, orbited fakes it at both ends
all that to get irc in your browser
well, kinda
Browser Orbited8100
Zeddius8200
Zeddicus8200
freenode6667
see, with orbited, you can connect directly to an irc server
and why not? it’s just a socket connection
in fact you can demo a (somewhat functional) irc connection on the orbited website
Browser Orbited8100
Zeddius8200
Zeddicus8200
Zeddius8200
Zeddicus8200
freenode6667
but with leafychat we wrote our own backend daemon that connected to irc.
why?
logging.
Browser Orbited8100
Orbited8100
Zeddius8200
Zeddicus8200
if we loaded our django code into zeddicus, we could easily log irc chats you’re interested in based on your session id.
it works very well.
Browser
Zeddius8200
Browser
Django8000
Apache80
Orbited
at this point i should mention the older tutorials online explaining how to load orbited into django.
Browser
Apache80
Django8000
Orbited8100
Zeddicus8200
but i the best (and simplest) way is to let each component be its own daemon
it works great for production as well as dev mode - the django dash judges were able to start and run our app locally, despite the number of daemons that needed to run
if you want to get a comet app up and running locally, check out orbited
it supports a ton of comet transports and is actively maintained
using java? jetty has comet support
the ruby world has juggernaut
while perl has meteor
there’s the interesting in-progress Ajax Push Engine
and in erlang there’s erlycomet (built on mochiweb)
so, flash’s XMLSocket
Browser Orbited8100
Zeddius8200
Zeddicus8200
it turns this
Browser Zeddicus8200
into this
how? flash allows you to make tcp connections in actionscript.
by providing a javascript api to those tcp connections, we can use flash to create persistent, socket connections from the browser
no lying needed
of course, it may be non-standard and might not work great across firewalls. but if you can use it, it’s pretty great
a popular technique is to attempt to open a flash socket, then fall back to standards based comet methods if it fails
it’s good backup
there are a few nice libraries for xml socket on github
tmm1 / jssocket
my favorite is tmm1’s jssocket
defunkt / jssocket
(i have a fork which removes the jquery dependencies)
finally: HTML5’s WebSocket
it’s still a proposed draft
but it’ll let you open a socket to any serve that speaks the special WebSocket protocol
ws://servers need to speak the WebSocket protocol - you can’t open arbitrary connections to irc or xmpp gateways
this plugs up the obvious security holes but also makes it a bit harder to implement than something like XMLSocket
your server needs to speak ws
?so, in review
comet
should be called asteroid
flash’s XMLSocket
is non-standard but nice
HTML5’s WebSocket
not yet here
but futuristic
you should give orbited a shot
with json
\0probably null terminated
Django8000
Orbited
don’t load orbited in django
Orbited8100
use them alongside each other
?is that it?
!that’s it
Thank You
http://www.flickr.com/photos/mojombo/3785549701/sizes/l/http://www.flickr.com/photos/voteprime/2361330726/sizes/o/http://www.flickr.com/photos/johnkerr/2371310025/sizes/l/http://www.flickr.com/photos/h19/182898904/http://www.flickr.com/photos/ukinindia/3595042998/sizes/l/http://www.flickr.com/photos/scott1027/3189137578/sizes/l/http://www.flickr.com/photos/foxypar4/2124673642/sizes/l/http://www.flickr.com/photos/nickwheeleroz/2166114756/sizes/l/http://www.flickr.com/photos/nickwheeleroz/2178146080/sizes/l/http://www.flickr.com/photos/diyromarcade/3006368260/sizes/o/http://www.flickr.com/photos/boopsiedaisy/3611187885/sizes/o/http://www.flickr.com/photos/bastispicks/2834869959/sizes/l/http://www.flickr.com/photos/courtenay/2536259393/sizes/l/http://www.flickr.com/photos/jardinle/3335907363/sizes/o/http://www.flickr.com/photos/tim-miley/3569809538/http://www.flickr.com/photos/dexterousartisan/3209508363/sizes/l/http://www.flickr.com/photos/equanimity/3763158824/sizes/l/http://www.flickr.com/photos/24617281@N04/2329660856/sizes/o/http://farm4.static.flickr.com/3555/3767120120_8f43f885e1.jpghttp://www.flickr.com/photos/raffaella/64701476/http://www.flickr.com/photos/monicareyes/405402858/sizes/o/http://www.flickr.com/photos/11016633@N07/2232831953/
flickr