Eventlet or gevent or Stackless + Twisted, Pylons, Django and SQL Alchemy
We're using Twisted extensively for apps requiring a great deal of asynchronous io. There are some cases where stuff is cpu bound instead and for that we spawn a pool of processes to do the work and have a system for managing these across multiple servers as well - all done in Twisted. Works great. The problem is that it's hard to bring new team members up to speed. Writing asynchronous code in Twisted requires a near vertical learning curve. It's as if humans just don't think that way naturally.
We're considering a mixed approach perhaps. Maybe keep the xmlrpc server part and process management in Twisted and implement the other stuff in code that at least looks synchronous to some extent while not being as such. Then again I like explicit over implicit so I have to think about this a bit more. Anyway onto greenlets - how well does that stuff work? So there's Stackless and as you can see from my Gallentean avatar I'm well aware of the tremendous success in it's use for CCP's flagship EVE Online game first hand. What about Eventlet or gevent? Well for now only Eventlet works with Twisted. However gevent claims to be faster as it's not a pure python implementation but rather relies upon libevent instead. It also claims to have fewer idiosyncrasies and defects. gevent It's maintained by 1 guy as far as I can tell. This makes me somewhat leery but all great projects start this way so... Then there's PyPy - I haven't even finished reading about that one yet - just saw it in this thread: Drawbacks of Stackless.
So confusing - I'm wondering what the heck to do - sounds like Eventlet is probably the best bet but is it really stable enough? Anyone out there have any experience with it? Should we go with Stackless instead as it's been around and is proven technology - just like Twisted is as well - and they do work together nicely. But still I hate having to have a separate version of Python to do this. what to do....
This somewhat obnoxious blog entry hit the nail on the head for me though: Asynchronous IO for Grownups I don't get the Twisted is being like Java remark as to me Java is typically where you are in the threading mindset but whatever. Nevertheless if that monkey patch thing really works just like that then wow. Just wow!
Answer
You might want to check out:
Eventlet and gevent are not really comparable to Stackless, because Stackless ships with a standard library that is not aware of tasklets. There are implementations of socket for Stackless but there isn't anything as comprehensive as gevent.monkey. CCP does not use bare bones Stackless, it has something called Stackless I/O which AFAIK is windows-only and was never open sourced (?).
Both eventlet and gevent could be made to run on Stackless rather than on greenlet. At some point we even tried to do this as a GSoC project but did not find a student.