Python 3 – Unpacking Generalizations

Python 3.5 added more support for Unpacking Generalizations in PEP 448. According to the PEP, it added extended usages of the * iterable unpacking operator and ** dictionary unpacking operators to allow unpacking in more positions, an arbitrary number of times, and in additional circumstances. What this means is that we can now make calls to functions with an arbitrary number of unpackings. Let’s take a look at a dict() example:

>>> my_dict = {'1':'one', '2':'two'}
>>> dict(**my_dict, w=6)
{'1': 'one', '2': 'two', 'w': 6}
>>> dict(**my_dict, w='three', **{'4':'four'})
{'1': 'one', '2': 'two', 'w': 'three', '4': 'four'}

Interestingly, if the keys are something other then strings, the unpacking doesn’t work:

>>> my_dict = {1:'one', 2:'two'}
>>> dict(**my_dict)
Traceback (most recent call last):
  File "<pyshell#27>", line 1, in <module>
TypeError: keyword arguments must be strings

Update: One of my readers was quick to point out that the reason this doesn’t work is because I was trying to unpack into a function call (i.e. dict()). If I had done the unpacking using just dict syntax, the integer keys would have worked fine. Here’s what I’m talking about:

>>> {**{1: 'one', 2:'two'}, 3:'three'}
{1: 'one', 2: 'two', 3: 'three'}

One other interesting wrinkle to dict unpacking is that later values will always override earlier ones. There’s a good example in the PEP that demonstrates this:

>>> {**{'x': 2}, 'x': 1}
{'x': 1}

I thought that was pretty neat. You can do the same sort of thing with ChainMap from the collections module, but this is quite a bit simpler.

However this new unpacking also works for tuples and lists. Let’s try combining some items of different types into one list:

>>> my_tuple = (11, 12, 45)
>>> my_list = ['something', 'or', 'other']
>>> my_range = range(5)
>>> combo = [*my_tuple, *my_list, *my_range]
>>> combo
[11, 12, 45, 'something', 'or', 'other', 0, 1, 2, 3, 4]

Before this unpacking change, you would have had do something like this:

>>> combo = list(my_tuple) + list(my_list) + list(my_range)
[11, 12, 45, 'something', 'or', 'other', 0, 1, 2, 3, 4]

I think the new syntax is actually quite handy for these kinds of circumstances. I’ve actually run into this a time or two in Python 2 where this new enhancement would have been quite useful.

Wrapping Up

There are lots of other examples in PEP 448 that are quite interesting to read about and try in Python’s interpreter. I highly recommend checking it out and giving this feature a try. I am hoping to start using some of these features in my new code whenever we finally move to Python 3.

PyDev of the Week: Petr Viktorin

This week our PyDev of the Week is Petr Viktorin (@EnCuKou). Petr is the author of PEP 489 — Multi-phase extension module initialization and teaches Python for the local PyLadies in Czech Republic. You can some of what he’s up to via his Github page or on his website. Let’s take some time to get to know Petr better!

Can you tell us a little about yourself (hobbies, education, etc):


I’m a Python programmer from Brno, Czech Republic. I studied at the Brno University of Technology, and for my master’s I switched to the University of Eastern Finland.

When I’m not programming, I enjoy playing board games with my friends, and sometimes go to an orienteering race (without much success).

Why did you start using Python?

At the university, I did coursework in languages like C, Java, and Lisp, but then I found Python and got hooked. It fit the way I think about programs, abstracted away most of the boring stuff, and makes it easy to keep the code understandable.

After I returned home from the university, I found a community that was starting to form around the language, and that’s probably what keeps me around the language now.

Continue reading PyDev of the Week: Petr Viktorin

Python’s New secrets Module

Python 3.6 added a new module called secrets that is designed “to provide an obvious way to reliably generate cryptographically strong pseudo-random values suitable for managing secrets, such as account authentication, tokens, and similar”. Python’s random module was never designed for cryptographic use but for modeling and simulation. Of course, you could always use the urandom() function from Python’s os module:

>>> import os
>>> os.urandom(8)

But now that we have the secrets module, we can create our own “cryptographically strong pseudo-random values”. Here’s a simple example:

>>> import secrets
>>> import string
>>> characters = string.ascii_letters + string.digits
>>> bad_password = ''.join(secrets.choice(characters) for i in range(8))
>>> bad_password

In this example, we import the secrets and the string modules. Next we create a string of uppercase letters and integers. Finally we use the secrets module’s choice() method to choose characters randomly to generate a bad password. The reason I am calling this a bad password is because we aren’t using mixed case, numbers and symbols in the password. This is actually a fairly decent one compared with what a lot of people use. One of my readers pointed out that another reason this could be considered bad is that the user would probably just write it down on a piece of paper. While that may be true, using dictionary words is typically strongly discouraged so you should learn to use passwords like this or invest in a secure password manager.

Continue reading Python’s New secrets Module

What’s New in Python: Asynchronous Comprehensions / Generators

Python 3.6 added the ability to create Asynchronous Comprehensions and Asynchronous Generators. You can read about asynchronous comprehension in PEP 530 while the asynchronous generators are described in PEP 525. The documentation states that you can now create asynchronous list, set and dict comprehensions and generator expressions. Their example looks like this:

result = [i async for i in aiter() if i % 2]

Basically you just need to add Python’s new async keyword into your expression and call a callable that has implemented __aiter__. Trying to follow this syntax will actually result in a SyntaxError though:

>>> result = [i async for i in range(100) if i % 2]
  File "<stdin>", line 1
    result = [i async for i in range(100) if i % 2]
SyntaxError: invalid syntax

Continue reading What’s New in Python: Asynchronous Comprehensions / Generators

PyDev of the Week: Lukasz Langa

This week we welcome Łukasz Langa (@llanga) as our PyDev of the Week! Łukasz is the author of PEP 443 — Single-dispatch generic functions. He also authors an interesting blog, although it’s not just a programming blog. You can see what Łukasz has been up to over on Github. Let’s take a few moments to get to know him better!

Can you tell us a little about yourself (hobbies, education, etc):

My first name is Łukasz which is an endless source of Unicode joy. I am a committer to Python since 2010, started out as a maintainer for configparser, nowadays working on the type hinting side of things. I like analog synthesizers, bicycles, and the Fallout game series. In my free time I’m helping Facebook embrace Python 3.

Why did you start using Python?

This is a bit embarrassing. It was Autumn 2004, I was studying Computing Science at Poznan University of Technology in Poland. I had trouble with some courses I took, namely Linear Algebra. A friend showed me some scripts he wrote in Ruby and some linear algebra library to check his results when solving homework assignments. I badly needed some reassurance so I got excited about this. Sadly, for some reason Ruby refused to install on my Windows XP box at the time.

As a test was scheduled for the very next day, I started looking for alternatives. I literally typed “ruby alternative” in Google and found Python that way. This installed cleanly and I quickly found a functional linalg library replacement. I can’t remember what it was anymore, this was before NumPy was a thing! Either way, I got hooked.

Continue reading PyDev of the Week: Lukasz Langa

Book Review: Murach’s Python Programming

I review books for the i-programmer website from time to time and they recently gave me a copy of Murach’s Python Programming by Michael Urban and Joel Murach. This book is long at almost 600 pages and it’s pretty expensive for the paperback. For those of you with short attention spans, I give you the Quick Review. For those who would like something a bit more in-depth, you will need to click through to see the rest.

Quick Review

  • Why I picked it up: In this case, because i-programming asked me to
  • Why I finished it: Because this book is well written and fairly interesting
  • I’d give it to: Someone who wants to learn Python

Continue reading Book Review: Murach’s Python Programming

New in Python: Formatted string literals

Python 3.6 added yet another way to do string substitution that they are calling “Formatted String Literals”. You can read all about the concept in PEP 498. I take a bit of umbrage here in that the Zen of Python states that There should be one– and preferable only one –obvious way to do it. Now Python has three ways. Let’s take a walk down memory lane before we talk about the latest way to play with strings.

Ye Olde String Substitution

When Python first started, they followed C++ in the way they did string substitution by using %s, %i and the like. Here are a couple of examples:

>>> The %s fox jumps the %s' % ('quick', 'crevice')
'The quick fox jumps the crevice'
>>> foo = 'The total of your purchase is %.2f' % 10
>>> foo
'The total of your purchase is 10.00'

Continue reading New in Python: Formatted string literals

Pyowa Meeting (Feb 2017)

The local Python users group, Pyowa, was restarted last night after a year and a half hibernation. I hadn’t been to one of their meetings in a few years, but I had decided at the end of 2016 to get it going again. Once the ball was rolling, we got multiple offers of sponsorships for the meeting and ended up going with Pillar Technology for our first meeting of 2017. They call their space The Forge and they provided pizza and refreshments. They are located in downtown Des Moines.

We had around 23 or so people show up including the staff from Pillar. Our speaker was Jesse Horne from Pioneer and he talked about Chalice, a Python Serverless Microframework for AWS (Amazon Web Services). Chalice is kind of a lightweight Flask. Jesse ended up writing a very basic book database web application. While there were some technical issues, the crowd was very supportive and helpful and I thought that the meeting went quite well. There was even a robot wandering around that was remote controlled by one of the employees of Pillar.

The next meeting is currently scheduled for March 7th @ Pioneer. Keep an eye on Pyowa’s website for an update to get the latest information.

PyDev of the Week: Michael Kennedy

This week we welcome Michael Kennedy (@mkennedy) as our PyDev of the Week! Michael is an author and speaker. He is also a Python and MongoDB enthusiast, and an entrepreneur. He is the host of the popular Talk Python To Me podcast as well as the new Python Bytes Podcast. In 2016, he founded the Talk Python Training program. He has an interesting blog that I think you’ll find illuminating. Let’s take some time to get to know Michael better!

Can you tell us a little about yourself (hobbies, education, etc)

Hey everyone. I’m one of the lucky people who got to make his hobby his job (like many of you I suspect). I love programming and technology. I wake up everyday thinking today is even a more amazing day to be a developer than yesterday.

I got into programming in college. First, it was just a minor subject area to fill some requirements for my math degree. Over time I got more and more into programming. It was actually a research project my senior year that forced me to do “real” programming (3D graphics on SGI mainframes in C++ and OpenGL). After a week or two I realized I liked programming more than my actual focus of math.

I ended up getting a master’s degree in Math before I dropped out of the PhD program to pursue a software career. It seemed super risky at the time. Looking back, it was a wise choice.

That’s my education. For hobbies I like active and adventurous things. Throughout highschool I raced motocross and have very fond memories of those times. I don’t race these days.

It may sound odd, but I feel those experiences at the racetrack actually prepared me well for a public role in the software field. It’s nerve wracking and somewhat scary to be on a podcast, give a presentation to 100’s of conference attendees, or deliver a training course on a topic for the first time. But, it’s nothing compared to your first attempt to clear the 40 foot triple jump.

Getting over the hesitation and fear of all of these surprisingly has the same steps. Over prepare, over practice, find the willpower to calm your inner voices, and go for it.

Continue reading PyDev of the Week: Michael Kennedy

Top Ten Articles of 2016

I try to write a retrospective of the previous year every year, usually for the first day of the new year. However due to tons of projects and family medical issues, I ended up pushing it to today. The year 2016 was pretty exciting. I wrote two books. The first was a sequel to my first book, Python 101 which I titled Python 201: Intermediate Python. Then at the end of December 2016 I released my third book, wxPython Cookbook which is my longest book to date.

The blog has done well too. For 2016, we hit 957,763 pageviews. That’s almost one MILLION page views, which is pretty exciting! Let’s take a look at what the top ten articles ended up being this year:

My introductory Reportlab tutorial continues to dominate the list with a second Reportlab tutorial making it onto the very bottom of the list. There are also two logging related articles, so there must be a lot of interest in doing that with Python. The biggest takeaway for me is that 40% of the list is made up of my Python 101 series while the rest is primarily introductory pieces to module in Python or 3rd party libraries (with the exception of item #10).

There is definitely still a lot of interest in introductory articles and in Reportlab. This combined with the feedback I receive from my readers helps me determine what to focus on in 2017. I hope to continue writing useful tutorials on much more of Python’s standard library in addition to the many cool 3rd party modules I keep stumbling across. I am also going to start writing at least one book this year that I hope my readers will enjoy.

I’m looking forward to seeing what else 2017 has to offer. I hope you will join me as we discover this year together!