What are the benefits of using the Eventlet module in python over the threading module?
Specifically the GreenPool class in Eventlet. I have tested some code to upload large files to S3 as individual pieces of a multipart upload. What I have noticed so far is that when using eventlet the CPU usage is much lower. Just looking for other pros and cons for Eventlet over just using threading. Thanks.
Answer
Basically, Eventlet green threads are to be considered a lightweight analog of OS threads for all practical purposes. Pros:
- cheaper to create in terms of CPU, memory and syscalls (0)
- cheaper to switch; this is especially true in Python 2.x where each thread actively tries to grab GIL which wastes CPU.
Cons:
- important since many green threads operate within one OS thread, when a syscall (e.g. open(2)) in one of them blocks OS thread, all of the green threads are blocked too.
- no SMP (multicpu/multicore); but then with GIL, this is also true for OS threads in Python. With greenlet[1] this restriction is more strict since it's not possible for some C extension to release GIL to allow other green threads to proceed.
You may also find this answer useful: Is a greenthread equal to a "real" thread
[1] "threading" library used by Eventlet https://github.com/python-greenlet/greenlet