Earlier this week, I wrote a simple post about Python’s Queues and demonstrated how they can be used with a threading pool to download a set of PDFs from the United States Internal Revenue Service’s website. Today I decided to try “porting” that code over to Python’s multiprocessing module. As one of my readers pointed out, Python’s Queues and threads are limited to running on only one core due to the Global Interpreter Lock (GIL) that is a part of Python. The multiprocessing module (and Stackless and several other projects) can run on multiple cores and skirts around the GIL (see the documentation if you’re curious). Anyway, let’s get started. (more…)