Tag Archives: Python 3

Python 3: An Intro to f-strings

Python 3.6 added another way to do string interpolation that is called “f-strings” or Formatted string literals (PEP 498). The idea behind f-strings is to make string interpolation simpler. To create an f-string, you just need to prefix the string with the letter “f”. The string itself can be formatted in much the same way that you would with str.format(). In other words, you can have replacement fields in the string that are enclosed with curly braces. Here’s a simple example:

>>> name = 'Mike'
>>> f"Hello {name}"
'Hello Mike'

The Python documentation had a fun example that demonstrates how you can nest the replacement fields. I have modified it a bit to make it simpler though:

>>> total = 45.758
>>> width = 14
>>> precision = 4
>>> f"Your total is {total:{width}.{precision}}"
'Your total is          45.76'

Here we create three variables with the first one being a float and the other two being integers. Then we create our f-string and tell it that we want to put the total variable into our string. But you will note that within the string’s replacement field, we have nested the width and precision variables to format the total itself. In this case, we tell Python that we want the width of the total field to be 14 characters and the precision of the float to be 4, so the result is 45.76, which you will note is rounded up.

The f-string also supports date formatting:

>>> import datetime
>>> today = datetime.datetime.today()
>>> f"{today:%B %d, %Y}"
'March 13, 2018'

I personally like the example given in PEP 498 that actually shows how you can use the date formatting to extract the day of the week from the date:

>>> from datetime import datetime
>>> date = datetime(1992, 7, 4)
>>> f'{date} was on a {date:%A}'
'1992-07-04 00:00:00 was on a Saturday'

You can also use the same variable repeatedly in your f-string:

>>> spam = 'SPAM'
>>> f"Lovely {spam}! Wonderful {spam}!"
'Lovely SPAM! Wonderful SPAM!'

The documentation does note that you have to be careful with f-strings when nesting quotes. For example, you obviously can’t do something like this:

>>> value = 123
>>> f"Your value is "{value}""

This is a syntax error, just like it is when using a regular string. You also cannot use a backslash in your format string directly:

>>> f"newline: {ord('\n')}"
Traceback (most recent call last):
  Python Shell, prompt 29, line 1
Syntax Error: f-string expression part cannot include a backslash: <string>, line 1, pos 0

The documentation points out that you can put the backslash into a variable as a workaround though:

>>> newline = ord('\n')
>>> f"newline: {newline}"
'newline: 10'

In this example, we convert the newline character to its ordinal value. The last item that the documentation mentions is that you can’t use an f-string as a docstring. According to Nick Coghlan, this is because docstrings need to be known at compile time, but an f-string is not resolved until runtime.

Wrapping Up

At this point you should have enough information to start using f-strings in your own code. This is a fun addition to the Python language and while not strictly necessary, I can see it making string interpolation simpler in some aspects. Have fun and happy coding!

Further Reading


Python 3 – Unpacking Generalizations

Python 3.5 added more support for Unpacking Generalizations in PEP 448. According to the PEP, it added extended usages of the * iterable unpacking operator and ** dictionary unpacking operators to allow unpacking in more positions, an arbitrary number of times, and in additional circumstances. What this means is that we can now make calls to functions with an arbitrary number of unpackings. Let’s take a look at a dict() example:

>>> my_dict = {'1':'one', '2':'two'}
>>> dict(**my_dict, w=6)
{'1': 'one', '2': 'two', 'w': 6}
>>> dict(**my_dict, w='three', **{'4':'four'})
{'1': 'one', '2': 'two', 'w': 'three', '4': 'four'}

Interestingly, if the keys are something other then strings, the unpacking doesn’t work:

>>> my_dict = {1:'one', 2:'two'}
>>> dict(**my_dict)
Traceback (most recent call last):
  File "<pyshell#27>", line 1, in <module>
TypeError: keyword arguments must be strings

Update: One of my readers was quick to point out that the reason this doesn’t work is because I was trying to unpack into a function call (i.e. dict()). If I had done the unpacking using just dict syntax, the integer keys would have worked fine. Here’s what I’m talking about:

>>> {**{1: 'one', 2:'two'}, 3:'three'}
{1: 'one', 2: 'two', 3: 'three'}

One other interesting wrinkle to dict unpacking is that later values will always override earlier ones. There’s a good example in the PEP that demonstrates this:

>>> {**{'x': 2}, 'x': 1}
{'x': 1}

I thought that was pretty neat. You can do the same sort of thing with ChainMap from the collections module, but this is quite a bit simpler.

However this new unpacking also works for tuples and lists. Let’s try combining some items of different types into one list:

>>> my_tuple = (11, 12, 45)
>>> my_list = ['something', 'or', 'other']
>>> my_range = range(5)
>>> combo = [*my_tuple, *my_list, *my_range]
>>> combo
[11, 12, 45, 'something', 'or', 'other', 0, 1, 2, 3, 4]

Before this unpacking change, you would have had do something like this:

>>> combo = list(my_tuple) + list(my_list) + list(my_range)
[11, 12, 45, 'something', 'or', 'other', 0, 1, 2, 3, 4]

I think the new syntax is actually quite handy for these kinds of circumstances. I’ve actually run into this a time or two in Python 2 where this new enhancement would have been quite useful.

Wrapping Up

There are lots of other examples in PEP 448 that are quite interesting to read about and try in Python’s interpreter. I highly recommend checking it out and giving this feature a try. I am hoping to start using some of these features in my new code whenever we finally move to Python 3.

Python’s New secrets Module

Python 3.6 added a new module called secrets that is designed “to provide an obvious way to reliably generate cryptographically strong pseudo-random values suitable for managing secrets, such as account authentication, tokens, and similar”. Python’s random module was never designed for cryptographic use but for modeling and simulation. Of course, you could always use the urandom() function from Python’s os module:

>>> import os
>>> os.urandom(8)

But now that we have the secrets module, we can create our own “cryptographically strong pseudo-random values”. Here’s a simple example:

>>> import secrets
>>> import string
>>> characters = string.ascii_letters + string.digits
>>> bad_password = ''.join(secrets.choice(characters) for i in range(8))
>>> bad_password

In this example, we import the secrets and the string modules. Next we create a string of uppercase letters and integers. Finally we use the secrets module’s choice() method to choose characters randomly to generate a bad password. The reason I am calling this a bad password is because we aren’t adding symbols to the password. This is actually a fairly decent one compared with what a lot of people use. One of my readers pointed out that another reason this could be considered bad is that the user would probably just write it down on a piece of paper. While that may be true, using dictionary words is typically strongly discouraged so you should learn to use passwords like this or invest in a secure password manager.

Continue reading Python’s New secrets Module

What’s New in Python: Asynchronous Comprehensions / Generators

Python 3.6 added the ability to create Asynchronous Comprehensions and Asynchronous Generators. You can read about asynchronous comprehension in PEP 530 while the asynchronous generators are described in PEP 525. The documentation states that you can now create asynchronous list, set and dict comprehensions and generator expressions. Their example looks like this:

result = [i async for i in aiter() if i % 2]

Basically you just need to add Python’s new async keyword into your expression and call a callable that has implemented __aiter__. Trying to follow this syntax will actually result in a SyntaxError though:

>>> result = [i async for i in range(100) if i % 2]
  File "<stdin>", line 1
    result = [i async for i in range(100) if i % 2]
SyntaxError: invalid syntax

Continue reading What’s New in Python: Asynchronous Comprehensions / Generators

New in Python: Formatted string literals

Python 3.6 added yet another way to do string substitution that they are calling “Formatted String Literals”. You can read all about the concept in PEP 498. I take a bit of umbrage here in that the Zen of Python states that There should be one– and preferable only one –obvious way to do it. Now Python has three ways. Let’s take a walk down memory lane before we talk about the latest way to play with strings.

Ye Olde String Substitution

When Python first started, they followed C++ in the way they did string substitution by using %s, %i and the like. Here are a couple of examples:

>>> The %s fox jumps the %s' % ('quick', 'crevice')
'The quick fox jumps the crevice'
>>> foo = 'The total of your purchase is %.2f' % 10
>>> foo
'The total of your purchase is 10.00'

Continue reading New in Python: Formatted string literals

New in Python: Syntax for variable annotations

Python 3.6 added another interesting new feature that is known as Syntax for variable annotations. This new feature is outlined in PEP 526. The basic premise of this PEP is take the idea of Type Hinting (PEP 484) to its next logical step, which is basically adding option type definitions to Python variables, including class variables and instance variables. Please note that adding these annotations or definitions does not suddenly make Python a statically typed language. The interpreter still doesn’t care what type the variable is. However, a Python IDE or other utility like pylint could have an annotation checker added to them that could highlight when you use a variable that you have annotated as one type and then used incorrectly by changing its type mid-function.

Let’s look at a simple example so we can see how this works:

# annotate.py
name: str = 'Mike'

Continue reading New in Python: Syntax for variable annotations

New in Python: Underscores in Numeric Literals

Python 3.6 added some interesting new features. The one that we will be looking at in this article comes from PEP 515: Underscores in Numeric Literals. As the name of the PEP implies, this basically gives you the ability to write long numbers with underscores where the comma normally would be. In other words, 1000000 can now be written as 1_000_000. Let’s take a look at some simple examples:

>>> 1_234_567
>>> '{:_}'.format(1234567)

The first example just shows how Python interprets a large number with underscores in it. The second example demonstrates that we can now give Python a string formatter, the “_” (underscore), in place of a comma. The results speak for themselves.

The numeric literals that include underscores behave the same way as normal numeric literals when doing calculations:

>>> 120_000 + 30_000
>>> 120_000 - 30_000

The Python documentation and the PEP also mention that you can use the underscores after any base specifier. Here are a couple of examples taken from the PEP and the documentation:

>>> flags = 0b_0011_1111_0100_1110
>>> flags
>>> 0x_FF_FF_FF_FF
>>> flags = int('0b_1111_0000', 2)
>>> flags

There are some notes about the underscore that need to be mentioned:

  • You can only use one consecutive underscore and it has to be between digits and after any base specifier
  • Leading and trailing underscores are not allowed

This is kind of a fun new feature in Python. While I personally don’t have any use cases for this in my current job, hopefully you will have one at yours.

ANN: The Python by Example Udemy Course

I am happy to announce my first Udemy course on the Python programming language. It is called Python by Example and is a re-branding of my Python 101 Screencast series. I had originally hoped to keep the name, Python 101 on Udemy, but someone else had already taken it by the time I got my course started on there. Regardless, the course is made with the first 35 videos from the Python 101 Screencast. I plan to add the rest later this year.


What You Get

At the time of writing, you will receive 35 lectures or 6.5 hours of content. I have also included chapters from Python 101 that correspond with each lecture.

How to Purchase

To purchase Python by Example, just go to the following link: https://www.udemy.com/python-by-example/?couponCode=py25. The coupon code will give you 25% off.

Python 201: An Intro to importlib

Python provides the importlib package as part of its standard library of modules. Its purpose is to provide the implementation to Python’s import statement (and the __import__() function). In addition, importlib gives the programmer the ability to create their own custom objects (AKA an importer) that can be used in the import process.

What about imp?

There is another module called imp that provides an interface to the mechanisms behind Python’s import statement. This module was deprecated in Python 3.4. It is intended that importlib should be used in its place.

This module is pretty complicated, so we’ll be limiting the scope of this article to the following topics:

  • Dynamic imports
  • Checking is a module can be imported
  • Importing from the source file itself

Let’s get started by looking at dynamic imports!

Continue reading Python 201: An Intro to importlib