This week we welcome Irit Katriel (@IritKatriel) as our PyDev of the Week! Irit is a CPython Core Dev who works at Microsoft. Irit helped write up PEP 654, which is about “Exception Groups and except*” and was released in Python 3.11 this year. This new feature makes it possible to raise and handle multiple unrelated exceptions simultaneously.
Let’s spend some time getting to know Irit even better!
Can you tell us a little about yourself?
I studied computer science and after graduating in 1995 I worked for Intel at the development centre in Haifa, Israel. I was working on a performance simulator for the Merced processor, which was used by the Microsoft compiler team before the actual chip existed. Then I went back to school and did a PhD on graph algorithms (pen-and-paper proofs, no programming). My favourite paper from that time was about an algorithm to efficiently maintain the topological order of the nodes of a DAG while you insert edges. It was fun to improve on such a fundamental problem.
Post-PhD, I ended up in London through the twists and turns of life. The tech scene in London at the time was very focused on the financial sector, and I worked mostly in banking technology for the next 15 years or so. This was post the financial crash and a lot of things were changing, so there were needs for new tech systems and changes to existing ones. I initially worked in roles where I was close to the business, working on systems that do things like automatically calculate prices of wholesale lots of various currencies. It gave me useful experience, particularly seeing how expensive a bug can be, but I didn’t enjoy that work enough to want to make a career of it. I gradually moved towards more fundamental areas of tech, providing services to other programmers rather than directly supporting a business line.
Why did you start using Python?
In 2016 I landed in a role at Bank of America that brought everything together for me – I was maintaining an in-house python-based language that defines a massive scalable spreadsheet. In other words, a graph that tracks dependencies between calculations to support efficient updates as inputs change. It was a core part of a system that was built in 2010 to comply with the regulatory risk-management requirements of the post-financial-crash era. In this role I was able to leverage my knowledge of graph algorithms, while diving into language design issues. My prior experience as a business-facing programmer helped me understand the working conditions of my users and to have a realistic expectation regarding the changes they can absorb. It was also the first time I did anything major in Python.
At some point, my manager encouraged me to get involved with cpython development itself, which I found very satisfying. In the bank, I had about 4000 direct users, but the work I do on cpython impacts millions of programmers. It is an incredible feeling to know that. Eventually, I became a cpython core developer and later joined the faster-cpython team at Microsoft, to work on cpython full time and focus on performance.
What other programming languages do you know and which is your favorite?
Other than Python I worked mostly in C, C++ and Java. For a while (until about a decade ago) I followed the development of C++ and the boost libraries, and I liked the rigour of the language specification and the processes for getting things into the boost libraries and later the C++ stdlib, while preserving the coherence of the library as a whole. At the same time, C++ was used less and less for new projects (except performance sensitive ones), because programmers were more productive in other languages.
I don’t think I have a favorite language. In most cases it wasn’t up to me which language to use, and the language was not a major component of my decision to accept a job offer. Currently I only write Python and C, and I think this is how it will be for a while because of what I’m doing. If I will have a reason to use another language, I will.
What projects are you working on now?
At the moment I am working on a sequence of incremental changes to the cpython compiler that will improve its internal structure and make it easier to evolve. Translating python code into bytecode involves a number of steps: tokenize the code, transform the tokens into an abstract syntax tree (AST), and finally the compiler which translates the AST into bytecode. We currently have access to the first two steps through the tokenize and ast modules of the standard library, but the third step, compiling an AST to bytecode, is one big C-code black box, and it is quite complicated. It involves a code generation step that traverses the AST and emits an instruction sequence, then an optimization stage that applies a large number of tricks to make the code faster, including analysis of the control-flow-graph. Finally, the assembler resolves jump offsets, emits the final bytecode and constructs additional information for the code object such as the exception table and source location table.
After thirty years of development, the compiler’s code is a bit hard to work with, for two reasons. The first is that the code of these different stages is very coupled and it is sometimes hard to predict the impact that a change can have. The second is that it is not always possible to write unit tests for changes that we want to make, because every test exercises so much code. My goal is to separate the compiler into three or four distinct stages, each of which can be exercised independently from unit tests written in Python. Once we have that, the compiler’s code will be structured better and easier to maintain. It will be more accessible to developers who need to work on it, and it will give us confidence to implement various optimisations.
Is there anything else you’d like to say?
I got involved with open source development at a late stage of my career. The technical experience I brought with me was of course very useful, but there was also a lot to learn about how open source works. The transparency inherent in the work is very refreshing, and I already mentioned the thrill of working on a high-impact project. I hope this will encourage other late-career developers to contribute to open-source projects in general and to cpython in particular. Start by reading the Dev Guide.
Thanks for doing the interview, Irit!