Tuesday, September 16th, 2008

Comet and Highly Interactive Websites – Joe Walker, @Media Ajax

Category: Comet, DWR

Joe Walker of DWR fame on Comet at @media Ajax.

Motivation for Comet

Interested in what our friends are up to, even if no-one else cares (as twitter illustrates). And also interested in what systems/things are doing, not just people. Chat (meebo), collaborative editing (google docs), streaming financial data (lightstreamer), async updates (yes.com), online gaming, async server processing (polar rose, i.e. shift processing complexity to the server – note this could be seen as an alternative to Gears-style local processing in some situations).

The web was designed to be connectionless – Comet blatently aims to make it connected.


Actually does scale. Joe shows the following graph:


Seven ways (!) to implement Comet.

Long polling – “slow” XHR request. Server doesn’t answer immediately. Special part of HTTP to do this – chunked mode. However, IE “lies to you” – keeps saying it’s got something, but you can’t actually inspect it.

Forever frame – Send text/plain and 4k whitespace to flush IE’s buffer. Flush with script tag for each data block. Must keep restarting to avoid memory leak.

Script tags – Dynamic script blocks, can point to any domain.

WebSockets – HTML 5 standard. Cleaner solution.

ActiveXObject(“htmlfile”) – htmlfile is an activeX control similar to XHR. Normally causes “clicking” noise in browser, but there’s a hack to turn it off in most cases. Obviously, this only works in IE. Most bizzare thing is you get 49 Javascript before garbage collection, leading to hair-pulling moments when you try to work out why your 50th command isn’t executing!

Mime messaging – “What push was built on 11 years ago” – x-multipart-replace. Works really well, though with memory leakage issues, but not in IE and historically not in the other browsers either.

Flash remoting

Forever GIF (bonus technique) – keep sending out new slices of an animated GIF!

Technical Issues

Co-ordination when browser has multiple tabs open to the same server – using window.name or cookies, but better solution is multi-home DNS – each call points to the server using a different name (1.example.com, 2.example.com etc all pointing to the same IP address).

Issues detecting when browser or connection has broken.

Proxies which don’t let the stream go through in real-time – hold on to the chunks for a while before releasing them all at once. Can detect if this is happening from the browser using a timestamp technique.

… so just like with Ajax, we have to come up with hacks, likewise with Comet. Facebook and GMail show it’s possible to work around these problems and get Comet working at scale.

API Styles

e.g. with WebSocket you’re simply posting and receiving data. Event handlers for onOpen, onRead etc. Joe says this will be too low-level in many cases, hence the following styles.

PubSub – e.g. cometd. Low coupling (server-browser separation – Joe describes demo where servers hot-swapped without affecting client), inter-language interop. Good analogy to SOAP, which has gradually shifted from RPC to document exchange.

API – this looks to me like Ruby’s remote JS – server controls what’s happening on the browser. e.g. Effect.shake(“price”) on the server will make the price div shake on the client.

Data Syc API – Keep changing/updating data store. Simplest to understand.

Posted by Michael Mahemoff at 4:27 am

4.6 rating from 14 votes


Comments feed TrackBack URI

Another problem is the size of the http header. I really wouldn’t suggest using any of these hacks if you are building an online realtime game. For each request send, take into account the number of bytes in the header. As small as it seems, it builds up rather quickly.

Comment by TNO — September 16, 2008

@TNO… no, clearly many of them would be bad. But a persistent Flash Socket works great for just such a task. And there’s no reason why an invisible flash “proxy” swf couldn’t be used for that purpose, while the rest of the game/presentation layer would be controlled via javascript.
flXHR (http://flxhr.flensed.com/) works very similarly, in that it’s an invisible swf for XHR-style transport. A future suggestion for flXHR is that another version of it be put out which instead exposes some socket-like API to javascript in very much the same way that it exposes a drop-in replacement of native XHR right now.

Comment by shadedecho — September 16, 2008

Leave a comment

You must be logged in to post a comment.