I'm having a bit of trouble understanding the differences between these three frameworks:
These three frameworks can be used to run code at the same time but do this in different way using a different amount of threads/processes or codestyle. This is how I'm understanding the differences right now:
The Questions I'm having right now are:
Tornado/Twisted Are able to accept (almost) any amount of sockets because they use asyncronous code and queue the request in the I/O loop.
Are Celery/Gevent able to this? Do they have to spawn a new process/thread to be able to accept a new socket?
I'm trying to figure out which of these technologies is best suited to built a real-time web application.
Gevent uses greenlets instead of threads on an IO loop implicitly, so there is no reactor / IO loop to manually start in the case of Twtisted/Tornado. It also has the ability to monkey patch existing libraries to support it's evented operation, Tornado and Twisted require specific libraries to work with their event loops although you will find many already exist.
Celery is made much more for background processing to offload expensive computations to another process/server.
Processes can share memory but not in the same way that threads do. Threads in CPython suffer from the GIL and generally it is worth not using a threaded solution if you are doing anything CPU intensive.
I'm not sure of Celery's memory requirements but if you are using 1 web process and 1 background process you should be fine even on a 256MB VPS, although more is better if you are supporting many connections.
The number of sockets that can be handled with Tornado/Twisted/Gevent will likely be bounded by the amount and frequency of IO done per socket. Low frequency/low bandwidth sockets are much easier to support a large number of concurrent connections as they will mostly be idle.
Celery will still require some application to listen for sockets and make calls to be processed with the Celery daemon. It supports Gevent as well so you can handle multiple tasks concurrently if required.