This week we welcome David Beazley (@dabeaz) as our PyDev of the Week! David is the author of the Python Essential Reference and the co-author of the Python Cookbook, Third edition. He also has a blog that I enjoyed when I was first learning Python, although I don’t think it’s updated much any more. You might find his Python talks of interest though. Let’s spend some time getting to know him better!
Can you tell us a little about yourself (hobbies, education, etc):
I’ve lived in the Chicago area for the past 17 years where I’ve put down roots with my wife and kids. I’m currently self-employed and spend most of my time working on a variety of Python-related projects including training, consulting, and book writing. Besides coding, I enjoy playing music, working in the shop, and bike riding. I’ve recently been trying to bike outside throughout the Chicago winter in temperatures down to about -25C. There are a lot of things that aren’t quite right about that, but it’s a lot of fun in its own weird way. At the very least, it’s a nice break from staring at a computer screen all day.
Regarding my education, I’ve been playing around with computer programming since about the 6th grade. I studied math as an undergraduate, but eventually went on to earn a Ph.D. in computer science. I also taught various computer science topics for about seven years as a university professor.
Why did you start using Python?
At the time I started using Python (1996), I was working on the problem of making it easier for computational physicists to interact with simulation software running on supercomputers. The standard practice had been to submit non-interactive batch jobs to the system, offload the resulting data many hours later, and then try to wrap your brain around what happened using various data analysis tools running on your desktop. The only problem is that none of this really worked–or certainly didn’t work nearly as efficiently as it could have (often taking weeks of work to do the most simple kind of experiment). Around 1995, I had been experimenting with the idea of incorporating a scripting language directly into our physics software so that users could interact with it more directly. I was familiar with tools such as MATLAB and IDL and had been trying to make something more custom tailored to our specific problem (molecular dynamics). I had even created a home-grown scripting language as well as a code generation tool that later became known as Swig (yes, that Swig). The idea of creating my own programming language didn’t sit so well with my PhD committee and they suggested that I look at some alternatives. So, I had started to explore other options including Tcl, Perl, and Guile. I discovered Python purely by chance after reading a mention of it in an article that Paul Dubois had written in Computers in Physics. As an aside, Paul had been thinking about the exact same kind of problems (making physics software programmable) and was active in the development of many tools that became precursors to NumPy and SciPy.
It didn’t take long to sell me on Python. I loved the simple syntax and the interactive REPL was exactly the kind of thing I wanted to have with our physics software. Not only that, the implementation in C was quite clean and easy to work with. One of my earliest projects involved porting Python to run on the Connection Machine 5 and Cray T3D massively parallel systems. It was kind of insane–you could sit there at the interactive Python prompt like you do now, but whatever you typed would execute simultaneously on 1024 CPUs. The thing that really sold people was that my little Python hack could take problems that had previously required 5-6 hours of work and reduce them down to about 4 seconds. Minds were blown. Keep in mind, this was long before the existence of anything like NumPy, SciPy, Pandas, or any of the advanced data analysis tools that Python programmers take for granted now. The idea that you’d put a slow interpreted language on a supercomputer and interact with your data seemed wrong on so many levels of wrong to those with strong opinions on such matters.
What other programming languages do you know and which is your favorite?
I would consider myself to be pretty fluent in C and assembly programming. I have varying amounts of experience with other languages including C++, Objective C, Java, Perl, Tcl, Scheme, ML, PHP, and Javascript although you’d probably find my head buried in a book or on Stack Overflow if I had to do anything useful in those. I recently found myself messing around with Java because my kids have been bugging me about making Minecraft mods. I can get around, but I don’t think I’ll be seeking employment as a Java programmer anytime in the foreseeable future. I really need to explore a more Pythonic solution for that.
As for a favorite language, that’s a hard one. For day-to-day tasks, Python is definitely my tool of choice. However, if I had to pick an all-time favorite language over my whole career, I think it would probably be assembly language. One of my first jobs involved writing device drivers for a graphics card. I also spent a lot of time buried in assembly language learning how to mod and reverse engineer arcade games on my Apple 2. There is a certain raw simplicity and beauty to assembly language. There are also unusual challenges such as only being able to debug your code through the use of blinking lights or a beeps. It requires a certain combination of perseverance and ingenuity. However, it’s also tremendously satisfying to see your code finally working when you figure it out. You also gain a much deeper awareness of how computers and algorithms work by coding in assembly.
What projects are you working on now?
I’m currently in the early stages of updating the Python Essential Reference book to a new edition. Over the past two years, I’ve also been doing some software development for a startup. Much of that code is pretty typical stuff involving databases, web services, testing, and so forth. I don’t consider myself to be a web programmer so I’ve actually learned quite a few new things by working on that.
Which Python libraries are your favorite (core or 3rd party)?
As a general rule, my favorites tend to be the built-in libraries. It’s hard to point at a favorite, but I think it might be the collections module. I feel like collections is this secret weapon that lets me solve all sorts of tricky data handling problems. If you’re not using it, you’re probably missing out. For pure fun, I also like the multiprocessing.connection submodule. There’s some neat magic in there that can be used to set up authenticated network connections between Python interpreters running on different machines, pass Python objects around between those machines, and do other interesting distributed computing sorts of things.
For 3rd party modules, I’ve been spending most of my recent time using SqlAlchemy, Pandas, and requests. All of those are great. I’ve also spent a fair bit of time using modules related to Redis, ZeroMQ, AWS, and related bits of technology.
Why did you decide to write books about Python?
I got into book writing by refusing to say “no” when asked. I had served as the program chair of the Python conference in 1998 and shortly after the conference, an editor at New Riders publishing (since absorbed into Pearson) contacted me about the possibility of writing a Python book. At that time, there were only a handful of other Python books around. In past programming work, I’d found great use from texts such as the well known Kernighan and Ritche “C Programming Language” book as well as books by W. Richard Stevens on advanced Unix programming. I thought it might be interesting to try and write a Python book in a similar vein. So, this ultimately led to the publication of the Python Essential Reference.
The more recent Python Cookbook (O’Reilly) has a slightly different spin. I’ve been increasingly bothered by all of the excuses and apologies made concerning the use of Python 3. So, for that book, I decided I’d just embrace Python 3 to the greatest extent possible and try to write the most modern Python book I could–without concern for Python 2. There is a certain calculated risk in doing that, but if Python is going to have a bright future, I feel that it should be served by forward-looking books.
Where do you see Python going as a programming language?
Given the general turmoil surrounding Python 2 and 3, this is an interesting question. Having worked with Python 3 for about six years, there are a lot of really great things about it to like. However, it’s also quite different in a few critical areas–and that’s something that people will have to come to terms with if they choose to use it. Although there will certainly be projects that never migrate to Python 2, there’s no reason why you can’t start creating new projects in Python 3 now. In the big picture, the world of programming is always going to be filled with messy and annoying problems—things that Python has always excelled at. For that reason, I don’t really see it going away anytime soon.
What is your take on the current market for Python programmers?
The Python job market is not something that I follow too closely. However, I don’t know any unemployed Python programmers so that’s probably a good sign.
Is there anything else you’d like to say?
Python has always been a language with a sense of practicality and fun–programming with it is supposed to be enjoyable. If not, you’re probably doing something wrong.
The Last 10 PyDevs of the Week