A lot of people complain that Python is slow. But is it really? In my own experience, Python has almost always been “fast enough”.
However, there is that niggling issue of the Global Interpreter Lock or GIL.
Let’s talk about that!
The Global Interpreter Lock
The Python Global Interpreter Lock or GIL, is a mutex or lock that only ever allows the Python interpreter to run in a single thread.
If all you ever do with Python is deal with single threads, then you will never notice any problems. Python will, indeed, be “fast enough” for most things.
Frankly, for tasks that are I/O bound, you will probably find that the GIL doesn’t add much overhead anyway. Python can use the multithreading module, which executes a type of fake thread because no two threads are running concurrently.
Instead, the threads take turns. This works for I/O bound applications (i.e. working with databases, downloading files, etc) because Python has to wait for something to happen.
You can read more about that topic in my article, Python 101 – Creating Multiple Threads.
Why Does the GIL Exist?
The main reason the GIL is here is that Python uses reference counting for memory management.
You can read all about this topic in the following articles:
The GIL prevents deadlocks and makes writing Python code easier Python will take care of the memory management for you.
Efforts to Remove the GIL
One semi-famous effort to remove the GIL from Python was led by Larry Hastings and he dubbed it the “gilectomy”. You can read all about it on GitHub.
PEP 684 is an effort to isolate the GIL into separate sub-interpreters. This feature was proposed several years ago, but is now approved to be added to Python 3.12. It will be interesting to see how this changes things with Python and concurrency.
PEP 703 proposes making the GIL optional in CPython. This PEP, if approved, would add a build flag that you could use to disable the GIL.
What About Multiprocessing?
Python has the multiprocessing module which is a great workaround if you need to better utilize the multiple cores on your machine’s CPU. You would use the multiprocessing module for CPU-bound tasks, such as video transformations, machine learning, etc.
You can read all about the pros and cons of multiprocessing in my article, Python 101 – Creating Multiple Processes.
Other Python Distributions
Alternatively, you could simply NOT use CPython, which is the reference Python version you’ll find on the Python website.
Instead, you could use one of the following:
- Jython (Java-based Python)
- IronPython (.NET-based Python)
- PyPy (RPython)
None of the above variants has a GIL. But they may not be compatible with every package you like to use either. They are worth a look though, if you have hit a wall with Python’s GIL.
Wrapping Up
Regardless, Python remains popular even with the GIL. Python is still crazy useful even if it’s not the fastest programming language on the planet.
For me, being able to write readable, debuggable code quickly is more important than writing the fastest code.
Let me know what you think!