This week we welcome Larry Hastings as our PyDev of the Week! Larry is a core Python developer and long time user of Python. He has a fun talk about Python’s GIL on Youtube that is well worth your time checking out. He is also the mastermind behind the now defunct Radio Free Python podcast. Let’s take a few moments to get to know him better!
Can you tell us a little about yourself (hobbies, education, etc):
I have a Bachelor of Arts in Computer And Information Sciences from the University of California, Santa Cruz. Why it’s a bachelor of arts in computer science has always baffled me–but that’s UCSC for you.
Why did you start using Python?
I started using Python back in the late 90s. Back then I was a professional Windows developer, and at the time Python’s support for Windows was far better than the other scripting languages–Perl on Windows was a distant also-ran to Perl on UNIX. Also, after shipping a decent-sized project on Perl (the web-based signup process for a small ISP), I swore off Perl. Never again! I haven’t touched it since.
I don’t remember specifically my first project–it was so long ago that I honestly don’t remember. I remember the first time I heard about Python, though. My brother Stuart wanted to buy a used boat, and he used Python to scrape a local newspaper’s classified ads to find interesting offers. Naturally I had the same initial reaction to Python everybody does–“that whitespace thing is too weird”–but once I started using the language I quickly got over it.
What other programming languages do you know and which is your favorite?
Python’s the clear favorite. My other main language is C; this shouldn’t be surprising, as I spend a lot of time working on CPython, and those are the two languages it’s written in. It’s rare these days that I code in anything but Python or C.
My first language was BASIC on 8-bits. But my second language was FORTH. My dad and older brothers were all computer guys; although my dad did hardware, my brothers did software. So they were all pretty tech savvy. My dad got me a Commodore 64 and a “Hes FORTH” cartridge, and I was amazed at how fast it was and the neat things you could do with it. FORTH’s “CREATE> DOES” construct remains a uniquely powerful mechanism for defining new syntax. There are some baffling design decisions in FORTH which never made sense to me–until some years later when I tried implementing a FORTH interpreter of my own. Suddenly the reasons behind them became clear. I’ve never shipped my toy FORTH interpreter “Sorta”, however I essentially did the entire 4.0 rewrite of the FORTH variant “FICL”.
Back in 2011 I interviewed Raymond Hettinger for my now-essentially-defunct podcast “Radio Free Python”. Raymond declared that every programmer should learn these three programming langauges: FORTH, Smalltalk, and Prolog. He said “FORTH will change how you program!” And I agree. FORTH certainly teaches the programmer about expressing themselves economically–and there’s never been a language that made refactoring code easier than classic FORTH. I also agree with Raymond when he says that the era of FORTH’s relevance has passed. There’s no longer any problem domain for which using FORTH is the correct answer. FORTH’s heritage from its 1970 design leaves it with some tiresome baggage that just haven’t aged well, and its reputation for “write-only code” is well-deserved.
During my formative years, Pascal was the predominant teaching language, so that’s what I had to use back in my high school and lower-division college programming classes. Happily, once you advanced to the upper-division classes, you also graduated to C. Eventually I used some Scheme and C++ in college. Later I used C++ professionally, as it was the language of choice for Windows development back in the nineties and naughties. I also messed around with writing raw PostScript to run on laser printers–PostScript is essentially a pleasanter, more abstracted variant of FORTH.
What projects are you working on now?
My current project is called “The Gilectomy”. CPython internally has this thing called the “GIL”. That stands for “Global Interpreter Lock”. It’s a single “mutex” lock CPython uses to ensure programs work correctly in multithreaded mode. Counter-intuitively, it also helps performance a great deal–it has essentially no overhead and allows Python to internally make many assumptions. But it also means Python can’t use more than one CPU simultaneously in a single process.
I’m trying to remove the GIL so Python can use multiple cores. Removing your tonsils is a tonsillectomy, right? So removing the GIL is a “gilectomy”. Removing the GIL itself isn’t actually that hard; I did that months ago. But it makes Python many, many times slower, for a number of reasons. The real work here is making Python fast again. Right now it’s very much in a “research” phase, where we’re trying anything we can to do fight that slowdown. We’re not sure it’s going to work! But if we can get it to work, that’ll be tremendously exciting.
I’m also co-chair of the Python Language Summit, a yearly meeting held at PyCon US. That happens in just under a month, and it’s taking up a lot of my Python time right now.
Apart from that, I’m the Release Manager for Python 3.4 and 3.5. This was a lot more work leading up to the 3.4.0 and 3.5.0 Final releases, but as of right now it doesn’t take very much time.
Since I’ve contributed a fair amount of code to CPython, I get a perpetual low-level trickle of bugs sent my way. I’m often added to bugs in the “os” module (internally called the “posix” module) because I’ve done a lot of hacking on it in the past. I also wrote an obscure internal tool used in CPython development called “Argument Clinic”, so I get all the bugs on that. Argument Clinic generates boilerplate code to convert Python values into C values for Python extension modules. It was proposed in PEP 436 if you want to read more.
Which Python libraries are your favorite (core or 3rd party)?
Since I tend to work on Python rather than in Python, I don’t tend to need a lot of external libraries. Python’s “batteries included” approach means that nearly any library I need for basic work is already handy.
In terms of those builtin modules, I like the classics. When I do text processing I like “re”, “shlex”, and “tokenize”, and I get a lot of mileage out of the often-overlooked “partition” and “rpartition” methods on strings. When I do file processing I need “os”, “shutils”, and “hashlib”. I’m used to using “os.path” but have been meaning to switch to the clever, pleasant “pathlib”.
Where do you see Python going as a programming language?
The real answer is: nobody really knows. Python is now roughly 26 years old and has evolved a great deal since its early days. In the last few years development of the language has slowed considerably… and I think that’s a good thing. It’s rare that you have a program that you can’t express in modern Python. And the language is, I feel, reaching a sort of complexity saturation point. I don’t want Python to have much more syntax than it already does. Python 3.5 added keywords for “async def” and “await”, and a new operator (“@”), and I only feel like I really understand the operator. Plus, it seems to me like any new syntax will likely be redundant with existing syntax, violating the Python Zen commandment “There Should Be One (And Preferably Only One) Obvious Way To Do It”.
The counter-argument to that perspective is decorators. When I first saw decorators, and understood how minor an abstraction a decorator actually was, I thought it was totally needless. “You can do @foo with ‘fn = foo(fn)’ after the function!” But history has clearly proved me wrong. I now regard decorators as a pitch-perfect addition to the language. This is exactly why I tend to stay out of discussions about new language features.
Is there anything else you’d like to say?
If you’re not using Python 3, I hope you’ll switch! It’s unquestionably a better language than Python 2, and getting better all the time. I have a lot of fun in Python 3 when I get a chance to write something in it.
Thanks for doing the interview!