Saturday, March 4th, 2006

Comet: A New Approach to Ajax Applications

Category: Comet, Dojo, Programming, Remoting

<p>Alex Russell has coined a term for a flavour of Ajax that’s been getting more attention of late. Comet describes applications where the server keeps pushing – or streaming – data to the client, instead of having the browser keep polling the server for fresh content. Alex identifies several buzzworthy examples:

Comet uses a modified form of the canonical Ajax architecture:

As is illustrated above, Comet applications can deliver data to the client at any time, not only in response to user input. The data is delivered over a single, previously-opened connection. This approach reduces the latency for data delivery significantly.

The architecture relies on a view of data which is event driven on both sides of the HTTP connection. Engineers familiar with SOA or message oriented middleware will find this diagram to be amazingly familiar. The only substantive change is that the endpoint is the browser.

While Comet is similar to Ajax in that it’s asynchronous, applications that implement the Comet style can communicate state changes with almost negligible latency. This makes it suitable for many types of monitoring and multi-user collaboration applications which would otherwise be difficult or impossible to handle in a browser without plugins.

Like Ajax, Comet’s a pattern that’s been around for a while, but Alex felt there was no suitable name for it, to help describe the problems and solutions growing around it: “(I)n the spirit of improved communication (and not technology invention), I’m proposing a new name for this stuff”

The article explains why Comet is better for users (responsive) and how it can scale. He also points out that Dojo has supported Comet-style architecture for some time.

As a sidenote, it’s interesting that 37Signals’ Campfire chat app went with polling as opposed to "Comet" architecture – it polls the server every three seconds for new messages. Since 37Signals presumably has similar constraints to Meebo or JotSpot, why would they prefer polling? More generally, when to use polling and when to use Comet?

(UPDATE: We recently interviewed Jamis Buck of 37signals and addressed the push versus poll question. The answer is less about technology and more about following the agile development process — more on that when we post the cast in the coming weeks. – Ben)

Remember the Push vision?

(A) new medium is arising, surging across the Web in the preferred, many-to-many way: anything flows from anyone to anyone – from anywhere to anywhere – anytime. In other words, a true network like the telephone system, rather than a radiating system like radio or TV. This new medium doesn’t wait for clicks. It doesn’t need computers. It means personalized experiences not bound by a page – think of a how-to origami video channel or a 3-D furry-muckers VR space. It means information that cascades, not just through a PC, but across all forms of communication devices – headlines sent to a pager, or a traffic map popping up on a cellular phone. And it means content that will not hesitate to find you – whether you’ve clicked on something recently or not.

– Wired, March 1997.

Related Content:

Posted by Michael Mahemoff at 2:06 pm
46 Comments

++++-
4 rating from 124 votes

46 Comments »

Comments feed TrackBack URI

Why does Campfire use polling?

They might not have the infrastructure for push. Or they found the ressource situation better for their usage.

Push requires one permanently open connection per client. There goes only so far. If their server process is fast in accepting a connection, and the application is fast in initializing the environment for the requesting user etc., polling might well be the better option.

Comment by Martin Bialasinski — March 4, 2006

Wouldn’t keeping that connection open with the server be a problem for bandwidth? Are we going to see the bandwidth hosting companies offer more and more bandwith as this “Comet” technique becomes more widespread?

Comment by Phil Ridlen — March 4, 2006

I can only presume that Campfire didn’t use this method because of the amount of RAM overhead it would take. RoR/FCGI uses a few megs for each connection. This usually opens/closes quickly but holding it open would up the server requirements by a lot. Given the nature of their apps and popularity, well lets just say they would have to know how to scale today, verse doing it tomorrow. :)

Comment by Adam Lindsay — March 4, 2006

�Ajax】Comet

Ajaxian.comã?Œã€? Comet: A New Approach to Ajax Applications http://ajaxian.com/archives/comet-a-new-approach-to-ajax-applications ã?¨ç´¹ä»‹ã?™ã‚‹ã€?Cometã?ªã‚¢ãƒ—ローãƒ?。 Comet: Low Latency Data for the Browser http://alex.dojotoolkit.org/?p=545 æ…

Trackback by JavaScript++ã?‹ã‚‚日記 — March 4, 2006

Wouldn’t keeping that connection open with the server be a problem for bandwidth?

No. This is a lighter bandwidth solution, possibly by a wide margin too. This would negate most, if not all of the client http headers (can add up to a lot of savings). On a local network, it’s about the equivelent of a 0.2 second poll rate, but without any of the bandwidth used to poll. On a reasonable tcp/ip stack & web server platform it’s not even resource intensive.

The only real problem is that it eats 1 of the 2 connections you can have open to a web server (given http 1.1 recommendations). This means you have a single channel for the browser to retrieve images, html, or other content (not part of the trickle). This *can* be a real problem, but you can negate it by having this content come from a “different server”.

This isn’t new. I’ve been using it for a few years (with iframes). It’s a secret I’ve kept for a time, but I don’t mind that it’s coming out. Maybe the standard web platforms (*/apache or windows/iis) will tune themselves to not react funny to the “load it creates”.

Comment by cooperpx — March 4, 2006

This technology already has a name in wide use – pushlet technology, which I personally prefer to ‘comet’. The javascript world is starting to rival Microsoft for rebranding old technologies with new names.

Comment by steve H — March 4, 2006

Steve: I considered a post that just revives one of the old names (“pushlets” being just one). Unfortunantly, they are all tied to particular implementations with pushlets being one of the least scalable of the bunch.

In the end I decided that we need a new name to describe the architecture. Individual project names will take care of themselves (and have for some time).

Regards

Comment by Alex Russell — March 4, 2006

Too big a hack for me at this point, even given Apache’s mpm_event and such. Adobe’s Flex Enterprise Services (though currently in beta) looks like a *much* cleaner approach for push apps.

Comment by Hans — March 5, 2006

[...] Here’s a podcast about Comet – exploring the two-way web with Ajax. From my Ajaxian post earlier today: [...]

Pingback by Software As She’s Developed - Comet: It’s Ajax for “Push” — March 5, 2006

=?utf-8?B?ZGVsLmljaW8udXMvcG9wdWxhcg==?=

From the Googleplex
http://video.google.com/googleplex.html  &nbsp;

Unleashing 10-User Conferencing in Skype 2.0 / Windows for all CPUs
http://maxxuss.com/home/skype.html  &nbsp;

Zend Framewor…

Trackback by The.RSS.Reporter — March 5, 2006

[...] Here’s a podcast about Comet – exploring the two-way web with Ajax. From my Ajaxian post earlier today: [...]

Pingback by Software As She’s Developed - Comet: It’s Ajax for “Push” (Podcast) — March 5, 2006

This article failed to mention what specific push technology is being used in this COMET thing. Is it similar to JSAN’s push or Netscape’s never-ending multipart mime transfer or other? Does it work on both IE and Mozilla-derived browsers? Some actual send/receive socket dumps would be better than the diagram provided with the tiny unreadable font.

Comment by Bill — March 5, 2006

Check out my paper on the subject:
http://rubyforge.org/docman/view.php/1251/314/CooperativeMediaLab_RP_Amsterdam_TechnicalDocumentation.pdf

We handled the rail fcgi memory problem by adding a simple ruby daemon that holds the push connection and acts as a mediator between rails and the clients.
A real problem is detecting closed connections. In FTP, the protocol has a BYE command, the client executes it before closing the conneciton. In HTTP streams, it is not possible to use the HTTP push connection to send this, so we have to periodically write to the socket to detect a closed connection. Ugly…

Why COMET? is that an abbreviation like AJAX? Why not call it HTTP streaming?
Hans: I have to aggree it is hackish. It is probably easier and more stable at this point to use a flash object to open a real socket connection and use it to access a javascript api. But I guess sometime in the future it will become usable.

Comment by wooyay — March 5, 2006

Too funny…AJAX for the kitchen, COMET for the bathroom. Looks like an early April Fools post. :)

Seriously though – client hacks and server scalability issues have kept http streaming out of the mainstream, but the real challenges to broad deployment come from proxies and firewalls. Outside of the lab, buffer padding eats your bandwidth, and keepalives eat your scalability.

If you need the lowest latency and can justify these costs, there are plenty of mature implementations to choose from. I can’t speak to Pushlets, LivePage, etc, but companies like KnowNow and Kenamea commercialized this stuff a loooong time ago. And of course SmartClient does it. :)

Comment by Jeff Dill — March 5, 2006

I did something similar using Flash, coined AJAF. I will write a post on it soon. The nice thing about this is it can directly connect to a socket server :)

Comment by Ethan Kaplan — March 6, 2006

用于Server Push的

Trackback by yingfeng — March 6, 2006

The argument of Push v/s poll is missing an important link: This.
According to the “>RFC specs, a standard browser can keep only two connections alive at a time with a given server. More on this in the MSDN blogs link above.

Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy

Most of the servers today are compliant with the HTTP RFC specs. so any attemt to initiate more than two connections might impact in compatibility with Servers and proxies. What we really need is a unique Browser/Session id. One that identifies a Browser right down to its Tab uniquely.

Don’t see a chance of that happening in the near future…

Or will it?

Regards,
Shri.

Comment by Shrikant Joshi — March 7, 2006

[...] Alex Russell has posted slides for his ETech presentation on Comet. Comet (which we mentioned the other day) is Alex’s new term for push-style server-to-browser communication. [...]

Pingback by Ajaxian » Comet ETech Slides Available — March 8, 2006

[...] Via ajaxian llegeixo en què consisteix això del Comet, en què s’assembla i en què difereix de l’AJAX. Una mica més de discussió al web de DOJO [...]

Pingback by ignots abismes » Arxiu » Comet — March 10, 2006

[...] Comet: A new approach to Ajax applications(Slides) [...]

Pingback by Ajaxian » Audible Ajax 16: March State of Ajax — March 14, 2006

The “comet” approach is formalized in the XMPP specification (Jabber protocol) under JEP-124. The “http polling” model under JEP-25. As far as I’m aware all client-side XMPP libraries support JEP-25 and most have, or are adding JEP-124 support.

When using XMPP for chat it seems that a reasonable approach is to poll and adjust the next polling interval based on heuristics like the amount of time since an incoming packet was received. In other words, if you’re actively chatting the polling is every second or two. If both parties have stopped actively chatting the polling interval can be adjusted downward since neither party is actively monitoring the chat window and a 10 or 15 second latency in polling will hardly be noticed. While this can’t be accomplished using a simple setInterval it can be accomplished by leveraging closures and setTimeout computations. As an aside, this is the default polling strategy used in TIBET’s XMPP support, which has be available for the past 3 years.

When using XMPP for real-time event distribution in a worfklow-style environment the sustained connection model is required by the use case so issues of whether it’s more efficient for the server are less relevant. An interesting aspect of this approach is that the ejabberd, written in Erlang, is currently being enhanced with JEP-124 support. That server promises to be highly scalable even when using JEP-124 connections given the nature of the Erlang language itself.

Comment by Scott Shattuck — March 14, 2006

[...] Alex Russell coined the term Comet as a way to discuss a different interaction model to standard Ajax. It all sounds good, but how can we implement the server side piece? Should we use Twisted? Apache mods? [...]

Pingback by Ajaxian » Comet is here with LightStreamer? — March 24, 2006

[...] I’ve read Ajaxian comments and mention of old 1997 Push technology, but COMET looks very different: it doesn’t start with general ideas but with real implementations, based on today’s products (web servers & browsers). [...]

Pingback by WebAPI.org - Web API scripts, directory, samples and reviews. » Welcome to COMET — March 27, 2006

Excellent site you have! Awesome content. Thank you.

Comment by car insurance — April 5, 2006

[...] Read more about Comet. Also read about Comet from Phil Windley. [...]

Pingback by Richard K Miller dot coooooooooom » Comet — April 29, 2006

Comet: A New Approach to Ajax Applications

Comet: A New Approach to Ajax Applications

Trackback by Moo がおすすめする Microsoft Visual Web Developer — May 15, 2006

[...] Ajaxian Comet: A New Approach to Ajax ApplicationsAlex Russell has coined a term for a flavour of Ajax that s been getting more attention of late. Comet describes applications where the server keeps pushing – or streaming – data to the client, instead [...]

Pingback by Ajaxian Comet: A New Approach to Ajax Applications - Comet — May 26, 2006

Hi, I’m looking for a tutorial or some example to develop in COMET. I’m creazing!!! Could you help me to find something? tnx in advance.

Comment by John — June 5, 2006

Hi there,
I’ve suggested, and we, the Tomcat team are playing around with extending this model a little bit to make it true push for *both* client and server, with zero latency.

http://www.halosg.com/Comet.png

As you can see, the client can push data over and over again, once the client has made a HTTP request, it can append to it through push, and the server will read just the appended data, or all the data

Let me know what you think of it,
Filip

Comment by Fiip Hanik — June 16, 2006

[...] Comet is mostly considered a server-to-browser thing, but how about a permanent connection in the opposite direction, from browser to server? I’ve been talking about this on my blog and received some interesting thoughts from Alex Russell. [...]

Pingback by Testing The Web Dot Com » Blog Archive » Two-Way Web: Can You Stream In Both Directions? — July 17, 2006

What ever happened to Steve Martin? When I was growing up, he was one of my favorite actors. Now he seems to be churning out zillions of movie which don’t do him justice.

Comment by mr skin — October 20, 2006

boy, i didn’t understand a word you just said.

Comment by napoleon dynamte — October 29, 2006

Hi, I’m Sara. In my opinion is yours blog is fine. Thank you dear writer. Has you made a lot of this great site?

Comment by Auto — November 8, 2006

We are breeders of all sort of animals, and we sale at the most affordable price
We deal on animals expecialy on dogs, cats, fish, birds, monkeys e.t.c.
Please do contact us through our email: infoonveterinary@yahoo.com
We can offer spectacular dogs and puppies that will grow up to be wonderful, confident family companions, excellant therapy or seeing eye dogs, as well as exceptionally great show dogs. from a month to three year health guarantee.

Comment by stacy — November 13, 2006

Leggo ed imparo sul vostro luogo. grazie!

Comment by Trenitalia — December 1, 2006

Cheap Satellite Internet

One of the best blog…

Trackback by Cheap Satellite Internet — December 5, 2006

this is terible look what they did above me

yes thats right they didnt use a anchor link

happy christmas

Comment by free xbox 360 — December 11, 2006

above one connection to your host?
test it…

Comment by cheap hosting — December 31, 2006

anyone got a comet code sample i can test on my own server? any help would be appreciated.

blessings…

Comment by zack — January 12, 2007

claim a free PS3 http://www.free-games-consoles.co.uk

Comment by Free-PS3 — November 19, 2007

Check out this site for details of how to get a free PS3 (Playstation 3)

Comment by Free PS3 — November 26, 2007

Get a Free PS3 from http://something4free.net/

Comment by AFreePS3 — January 20, 2008

Wicked post :) Thanks a lot!

Comment by Wii — December 10, 2008

Comet has come a long way since this article first appeared, its now in use almost everywhere: GMail Chat, Facebook etc.. We’ve started to put some Comet features in our web 2.0 framework, we started out trying to do the logic ourselves using just PHP but found there was so much work to do to fix all the various niggles in different browsers. In the end we opted for StreamHub comet server – they have a free community edition and have solved all the browser intricacies for us.

Comment by CometDude — July 22, 2009

Is this not moderated now?

Anyway, wouldn’t this create a bandwidth issue?

Does SteamHub community edition cover all the bases well?

Comment by FreeXbox360 — July 29, 2009

Thanks for the great info!

Comment by freesatnav — December 14, 2009

Leave a comment

You must be logged in to post a comment.