PyDev of the Week: Ezio Melotti

This week we welcome Ezio Melotti as our PyDev of the Week! Ezio is a core developer of the Python language. You can get an idea of what he’s been up to via his bitbucket page. Let’s take a few moments to get to know him!

Can you tell us a little about yourself (hobbies, education, etc):

At the moment I live in Finland, where I studied and graduated in Information Technology. While I was graduating I taught a university course on Python programming and now I work remotely as a developer/educator.

I’ve been a CPython core developer since 2009 and I’m the maintainer of the bug tracker and the html package. I spend a good slice of my free time improving Python in different ways, including work on the stdlib, documentation, tests, infrastructure, mentoring for Google Summer of Code, organizing sprints, and doing talks at conferences.

My hobbies include traveling, learning new languages (Chinese and Spanish), and wildlife photography.

Continue reading PyDev of the Week: Ezio Melotti

Python 201 – OrderedDict

Python’s collections module has another great subclass of dict known as OrderedDict. As the name implies, this dictionary keeps track of the order of the keys as they are added. If you create a regular dict, you will note that it is an unordered data collection:

>>> d = {'banana': 3, 'apple':4, 'pear': 1, 'orange': 2}
>>> d
{'apple': 4, 'banana': 3, 'orange': 2, 'pear': 1}

Every time you print it out, the order may be different. There are times when you will need to loop over the keys of your dictionary in a specific order. For example, I have had a use case where I needed the keys sorted so I could loop over them in order. To do that, you can do the following:

>>> keys = d.keys()
>>> keys
dict_keys(['apple', 'orange', 'banana', 'pear'])
>>> keys = sorted(keys)
['apple', 'banana', 'orange', 'pear']
>>> for key in keys:
...     print (key, d[key])
apple 4
banana 3
orange 2
pear 1

Continue reading Python 201 – OrderedDict

Python 201 Kickstarter Campaign Update – New Cover Preview

We’re halfway through the Kickstarter campaign for my new book, Python 201 and we’re a little over halfway to my first stretch goal. If you know anyone who would like to add to their Python skills, be sure to post a link to the campaign to your social website(s) of choice. I also added a couple new reward levels due to input from my backers.

You can now get a T-shirt + a Paperback copy of Python 201 or you can go with the EVERYTHING level of support, which will get you T-Shirt, a paperback of Python 201 and 101, the eBooks and the Python 101 Screencast.

Finally, I got in touch with my illustrator and she sent me a quick preview of the cover of the book which I’d like to share:


Note that this is just a rough version of the cover, but the whole thing should be done some time next week.

Thanks again for your support!

Python 201 – The handy defaultdict

The collections module has another handy tool called defaultdict. The defaultdict is a subclass of Python’s dict that accepts a default_factory as its primary argument. The default_factory is usually a Python type, such as int or list, but you can also use a function or a lambda too. Let’s start by creating a regular Python dictionary that counts the number of times each word is used in a sentence:

sentence = "The red for jumped over the fence and ran to the zoo for food"
words = sentence.split(' ')
reg_dict = {}
for word in words:
    if word in reg_dict:
        reg_dict[word] += 1
        reg_dict[word] = 1

If you run this code, you should see output that is similar to the following:

{'The': 1,
 'and': 1,
 'fence': 1,
 'food': 1,
 'for': 2,
 'jumped': 1,
 'over': 1,
 'ran': 1,
 'red': 1,
 'the': 2,
 'to': 1,
 'zoo': 1}

Now let’s try doing the same thing with defaultdict!

from collections import defaultdict
sentence = "The red for jumped over the fence and ran to the zoo for food"
words = sentence.split(' ')
d = defaultdict(int)
for word in words:
    d[word] += 1

You will notice right away that the code is much simpler. The defaultdict will automatically assign zero as the value to any key it doesn’t already have in it. We add one so it makes more sense and it will also increment if the word appears multiple times in the sentence.

defaultdict(<class 'int'>,
            {'The': 1,
             'and': 1,
             'fence': 1,
             'food': 1,
             'for': 2,
             'jumped': 1,
             'over': 1,
             'ran': 1,
             'red': 1,
             'the': 2,
             'to': 1,
             'zoo': 1})

Now let’s try using a Python list type as our default factory. We’ll start off with a regular dictionary first, as before.

Continue reading Python 201 – The handy defaultdict

Python 201 Book Outline

Over the weekend, I spent some time rearranging ideas for my latest book such that I have have four specific sections of the book. Here they are:

Part I – Intermediate Modules

  • Chapter 1 – The argparse module
  • Chapter 2 – The collections module
  • Chapter 3 – The contextlib module (Context Managers)
  • Chapter 4 – The functools module (Function overloading, caching, etc)
  • Chapter 5 – All about imports
  • Chapter 6 – The importlib module
  • Chapter 7 – The itertools module
  • Chapter 8 – The re module (An Intro to Regex in Python)
  • Chapter 9 – The typing module (Type Hinting)

Part II – Odds and Ends

  • Chapter 10 – generators / iterators
  • Chapter 11 – map, filter, reduce
  • Chapter 12 – unicode
  • Chapter 13 – benchmarking
  • Chapter 14 – encryption
  • Chapter 15 – Connecting to databases

Part III – Web

  • Chapter 16 – Web scraping
  • Chapter 17 – Working with web APIs
  • Chapter 18 – ftplib
  • Chapter 19 – urllib / httplib (client / server)

Part IV – Testing

  • Chapter 20 – Doctest
  • Chapter 21 – unittest
  • Chapter 22 – mock
  • Chapter 23 –

I want to note that these are just the topics that will absolutely be included. I may add others. I will also be adding others should I reach my stretch goal. If you’re interested in getting early access to the book or just want to support the blog, you can do so at my Kickstarter for Python 201!

PyDev of the Week: Massimo DiPierro

This week we welcome Massimo DiPierro (@mdipierro) as our PyDev of the Week! Massimo is the inventor and lead developer of web2py, but he’s also contributed to lots of other projects which you can see on his github profile. He is also the author of the following books: Annotated Algorithms in Python: with Applications in Physics, Biology, and Finance and web2py Complete Reference Manual. Let’s take some time to get to know him better!

Can you tell us a little about yourself (hobbies, education, etc):

I am 44, was born in Italy, and since when I was kid I loved math and science. I have a BS/MS in Physics from the University of Pisa and a Ph.D. in Theoretical Physics from the University of Southampton in UK. This means I can read most of the formulas that Sheldon writes in the Big Bang Theory. Although, while Sheldon works on String Theory, I worked on Lattice Quantum Chromo Dynamics, a computational approach to QCD, the model that describes how quarks bind together to form the majority of matter in the universe. This is what brought me to the US: I moved here to work as postdoctoral researcher at the Fermi National Accelerator Laboratory. Eventually I realized I had a better chance at an academic career in Computing than in Physics and I accepted a job offer from the School of Computing of DePaul University. I spend most of time programming, doing research, and consulting. I enjoy scuba diving, sailing, and skiing but I do not get to do as much of those as I would like.

Continue reading PyDev of the Week: Massimo DiPierro

Python 201: namedtuple

Python’s collections module has specialized container datatypes that can be used to replace Python’s general purpose containers. The one that we’ll be focusing on here is the namedtuple which you can use to replace Python’s tuple. Of course, the namedtuple is not a drop-in replacement as you will soon see. I have seen some programmers use it like a struct. If you haven’t used a language with a struct in it, then that needs a little explanation. A struct is basically a complex data type that groups a list of variables under one name. Let’s look at an example of how to create a namedtuple so you can see how they work:

from collections import namedtuple
Parts = namedtuple('Parts', 'id_num desc cost amount')
auto_parts = Parts(id_num='1234', desc='Ford Engine',
                   cost=1200.00, amount=10)
print auto_parts.id_num

Here we import namedtuple from the collections module. Then we called namedtuple, which will return a new subclass of a tuple but with named fields. So basically we just created a new tuple class. you will note that we have a strange string as our second argument. This is a space delimited list of properties that we want to create.

Now that we have our shiny new class, let’s create an instance of it! As you can see above, we do that as our very next step when we create the auto_parts object. Now we can access the various items in our auto_parts using dot notation because they are now properties of our Parts class.

One of the benefits of using a namedtuple over a regular tuple is that you no longer have to keep track of each item’s index because now each item is named and accessed via a class property. Here’s the difference in code:

>>> auto_parts = ('1234', 'Ford Engine', 1200.00, 10)
>>> auto_parts[2]  # access the cost
>>> id_num, desc, cost, amount = auto_parts
>>> id_num

In the code above, we create a regular tuple and access the cost of the vehicle engine by telling Python the appropriate index we want. Alternatively, we can also extract everything from the tuple using multiple assignment. Personally, I prefer the namedtuple approach just because it fits the mind easier and you can use Python’s dir() method to inspect the tuple and find out its properties. Give that a try and see what happens!

The other day I was looking for a way to convert a Python dictionary into an object and I came across some code that did something like this:

>>> from collections import namedtuple
>>> Parts = {'id_num':'1234', 'desc':'Ford Engine',
             'cost':1200.00, 'amount':10}
>>> parts = namedtuple('Parts', Parts.keys())(**Parts)
>>> parts
Parts(amount=10, cost=1200.0, id_num='1234', desc='Ford Engine')

This is some weird code, so let’s take it a piece at a time. The first line we import namedtuple as before. Next we create a Parts dictionary. So far, so good. Now we’re ready for the weird part. Here we create our namedtuple class and name it ‘Parts’. The second argument is a list of the keys from our dictionary. The last piece is this strange piece of code: (**Parts). The double asterisk means that we are calling our class using keyword arguments, which in this case is our dictionary. We could split this line into two parts to make it a little clearer:

>>> parts = namedtuple('Parts', Parts.keys())
>>> parts
<class '__main__.Parts'>
>>> auto_parts = parts(**Parts)
>>> auto_parts
Parts(amount=10, cost=1200.0, id_num='1234', desc='Ford Engine')

So here we do the same thing as before, except that we create the class first, then we call the class with our dictionary to create an object. The only other piece I want to mention is that namedtuple also accepts a verbose argument and a rename argument. The verbose argument is a flag that will print out class definition right before it’s built if you set it to True. The rename argument is useful if you’re creating your namedtuple from a database or some other system that your program doesn’t control as it will automatically rename the properties for you.

Wrapping Up

Now you know how to use Python’s handy namedtuple. I am already finding multiple uses for it in my code base and I hope you’ll find it helpful in yours. Happy coding!

Related Reading

PyDev of the Week: Hynek Schlawack

This week we welcome Hynek Schlawack (@hynek) as our PyDev of the Week! Hynek is an avid member of the Python community and a fellow of the Python Software Foundation. He has contributed to many projects and writes his own Python blog. Let’s spend some time getting to know him better!


Can you tell us a little about yourself (hobbies, education, etc):

I’ve the former German equivalent of a master’s degree in computer sciences. I try to have as many hobbies away from any screen as possible but I’m not always successful at that. But at the very least, I do quite a bit of sports and traveling to keep my body in shape and my horizon wide.

Continue reading PyDev of the Week: Hynek Schlawack

ANN: Python 201 Book Kickstarter

I am happy to announce my latest project, which is the sequel to my Python 101 book: Python 201 – Intermediate Python. I am launching a Kickstarter campaign to help fund its publication so if you’re interested in supporting, you can do so here:

If you already know the basics of Python and now you want to go to the next level, then this is the book for you! This book is for intermediate level Python programmers only. There won’t be any beginner chapters here.

Note: This book will be covering Python 3

Here are some of the topics covered:

  • generators / iterators
  • Functional idioms (map, filter, reduce).
  • Writing your own context managers.
  • Command-line argument processing
  • collections
  • itertools
  • functools
  • Function Overloading
  • Basics of regular expressions
  • httplib / urllib (client / server)
  • web scraping
  • Basics of Unicode (encoding and codecs)
  • Timing code (benchmarking)
  • Testing (unit tests, doc tests, mock, coverage)