Opera Software showcased an almost unknown great feature that their flagship desktop product has: the support for persistent connections from the client to the server.
I think that’s great to have (it’ll be better to have in on every browser too) because the kind of applications you can build with that are in our everyday life (or should be 
. See meebo, Gmail chat, JotLive and so on.
These applications are not AJAX, they take advantage of the so-called AJAX to go one step further and keep a persistent connection between client and server to stream data back and forth. Alex Russell, the author of Dojo rebranded this “pattern” as Comet and started a project to write an HTTP-based event routing bus, Cometd. Now there are Perl and Twisted implementations.
Guess what which web framework was supporting Comet before all this hype? Nevow, it’s obvious. See Athena to taste it.
Going back to the title of the post: Opera 9 does Comet on the client side, so they chose to implement the test server in… Twisted (here we go again!) and one of the Opera’s developers says:
Easy, simple and clean. Which is the same on the server. We have built a neat little demo IRC-like chat client that is open for testing. The server is built on Twisted, an event-driven networking framework written in Python. During initial testing we tried to put the server through pure, unadulterated load abuse, but were unable to bring it down, which says something about the advantages for this approach on the server as well.
We should learn to hype our stuff as the Rails community does…
7 Comments
I don’t think Twisted needs hype. It needs the 20 minute wiki. It needs better examples in general.
There’s plenty of 20 line Twisted demos flying around showing how to setup various services, what’s lacking is demos showing how you go beyond that, integrating other features of Twisted such as Cred, Perspective Broker, etc. There seems to be the assumption that the async part of Twisted is what boggles people (maybe it does, a little), but I know for me async wasn’t terribly difficult, knowing where to go next is what I found daunting. I can write simple, single-purpose servers in Twisted without much of a problem. I wouldn’t attempt a fully-integrated application like I would with TurboGears.
The information on how to accomplish this might be there, but I doubt I’d be able to sort it out in any timely fashion (or properly choose between the usual state of “don’t use this because it’s deprecated and its replacement will be done real soon now”). I think herein lies Twisted’s reason for not dominating the Python webspace: any path one might choose outside the core (i.e. anything non-trivial) is either labeled “deprecated” or “unstable”. That’s not a recipe for successful marketing. Deprecate something when its replacement is ready, not a year beforehand. Microsoft has successfully employed this very strategy (FUD) against other companies for years and the Twisted community seems to employ it against themselves. What’s the “right way” to do something in Twisted? Apparently there isn’t one.
There seems to be a couple half-completed examples of a wiki and a blog on the Twisted site. If someone who actually knows the right way to do these in Twisted/Nevow (and could make them more prominent on the site) it would go a lot further than any marketing or hype might. The Twisted community being a little more savvy about outside developer concerns would go a lot further.
Great comment Cliff. Thanks for the sharing!
Cliff – you’re absolutely right. This is a huge problem, and up until recently the problem wasn’t even explained. This page, linked from twistedmatrix.com’s front page, should at least explain the problem a little bit:
http://twistedmatrix.com/trac/.....ithTwisted
For the level of interest in the “one right way” to do web stuff in Twisted, we don’t really have very much manpower. Nobody is getting paid to solve this problem, and all the hyperactive teenagers with too much time on their hands who got Twisted started (myself included) are now working on rather time-consuming jobs at frantic startups, writing software for airlines or online services.
Any chance you know someone with some spare time? There are piles and piles of already-known issues in the Twisted and Divmod (e.g. Nevow) trackers that need working on…
Hi,
So Nevow and Athena are indeed great, and Twisted doesn’t get enough credit, but at this point it’s becoming easier and easier for “mainstream” developers to turn to whatever async network responder layer is built into the environment they’re already invested in. Jumping into Twisted, esp pre-Abe’s book, required not only being familiar with Python, but also seeing enough value in async network programming to put up with the pseudo-CPS-style idioms that Twisted grafts onto the language. For folks who know they need Twisted, it’s a no-brainer, but for a lot of other people the stumbling blocks that Cliff points out prove to be insurmountable.
All of this is just to shed some light on why the name “Comet” and the Bayeux protocol are necessaray: most people really can’t recognize the value until they can recognize the problem. Hopefully the “Comet” moniker will do good things for Nevow and Athena adoption, but part of the battle there is also in helping users to recognize the problems that the framework solves. Documentation, live apps, examples, and a community that believes passionately that they are doing thing the right way will help that a lot, but right now there’s not a lot to hang your hat from if you’re a Nevow/Athena developer.
We’re developing twisted-cometd on top of Twisted Web2 today, and I’ve gotta tell you that I’ll probably be recommending the Jetty or Perl implementations of the server stack because they don’t require being wedded to alphaware or to something that’s clearly EOL’d.
Regards
Alex: AFAIK twisted.web2 is explicitly considered alpha and unstable so why are you using it? Isn’t the web(1) version not enough?
Lawrence: Given the choice between using something that’s been branded as “the future” and something branded as “legacy”, what would you pick?
Alex: in a normal situation I choose “future”, but I’m used to Twisted guys and the web2 so-called-drama and after reading tons of “web2 can and will change” by them I think “legacy”.