Lately I’ve been doing some Python multi-threading to make the best use of some of our amazing server resources. As I was pondering the reasons why one of our 8-core servers reported 83% idle despite 8 threads banging away, I re-discovered the Global Interpreter Lock.
BLECH!
The GIL enforces Python’s requirement that only a single bytecode operation is executed at a time. My nicely coded multi-threaded app was only being executed serially!! Sadly, this seems unlikely to change, even in Python 3000. Last year Guido said:
“Just Say No to the combined evils of locking, deadlocks, lock granularity, livelocks, nondeterminism and race conditions.”
I was brought up to believe that threading was dirty and independent communicating processes were the way to go. But even I realize that this just isn’t practical in these days of GUIs, multi-core processors, and application servers.
Why does the Python community accept the GIL? Is it because most people only use Python as a scripting language? Are there simple workarounds (e.g. not forking, shared memory, or the like) that I’m missing?
31 Responses to “Python’s GIL is EVIL”
-
-
-
Comments are closed.Essay Help
I was brought up to believe that threading was dirty and independent communicating processes were the way to go.
web hosting
How hard would it to be to have the Python SWIG bindings release Python’s GIL on all functions? Could this be done in one place in a .i file, or would each
method need it?
I’m writing a multithreaded server that’s currently in Python and don’t want to
block when it descends into svn’s C code. The networking part is already in C++
and handles connection setup, marshalling, etc, but just gets the GIL when it
calls into Python code. This is all with Ice. I’ll handle locking wiuth
Anonymous
“Fixed” in python 3.2, released today. Fixed in the sense that multi-threaded programs will not be significantly slower than single-threaded programs because of the (new) GIL; but you still cannot expect core utilization beyond 1.