Category Archives: Cross-Platform

This article will be about a topic that can be used across platforms, such as Linux, Windows and Mac.

Python 201: namedtuple

Python’s collections module has specialized container datatypes that can be used to replace Python’s general purpose containers. The one that we’ll be focusing on here is the namedtuple which you can use to replace Python’s tuple. Of course, the namedtuple is not a drop-in replacement as you will soon see. I have seen some programmers use it like a struct. If you haven’t used a language with a struct in it, then that needs a little explanation. A struct is basically a complex data type that groups a list of variables under one name. Let’s look at an example of how to create a namedtuple so you can see how they work:

from collections import namedtuple
 
Parts = namedtuple('Parts', 'id_num desc cost amount')
auto_parts = Parts(id_num='1234', desc='Ford Engine',
                   cost=1200.00, amount=10)
print auto_parts.id_num

Here we import namedtuple from the collections module. Then we called namedtuple, which will return a new subclass of a tuple but with named fields. So basically we just created a new tuple class. you will note that we have a strange string as our second argument. This is a space delimited list of properties that we want to create.

Now that we have our shiny new class, let’s create an instance of it! As you can see above, we do that as our very next step when we create the auto_parts object. Now we can access the various items in our auto_parts using dot notation because they are now properties of our Parts class.

One of the benefits of using a namedtuple over a regular tuple is that you no longer have to keep track of each item’s index because now each item is named and accessed via a class property. Here’s the difference in code:

>>> auto_parts = ('1234', 'Ford Engine', 1200.00, 10)
>>> auto_parts[2]  # access the cost
1200.0
>>> id_num, desc, cost, amount = auto_parts
>>> id_num
'1234'

In the code above, we create a regular tuple and access the cost of the vehicle engine by telling Python the appropriate index we want. Alternatively, we can also extract everything from the tuple using multiple assignment. Personally, I prefer the namedtuple approach just because it fits the mind easier and you can use Python’s dir() method to inspect the tuple and find out its properties. Give that a try and see what happens!

The other day I was looking for a way to convert a Python dictionary into an object and I came across some code that did something like this:

>>> from collections import namedtuple
 
>>> Parts = {'id_num':'1234', 'desc':'Ford Engine',
             'cost':1200.00, 'amount':10}
>>> parts = namedtuple('Parts', Parts.keys())(**Parts)
>>> parts
Parts(amount=10, cost=1200.0, id_num='1234', desc='Ford Engine')

This is some weird code, so let’s take it a piece at a time. The first line we import namedtuple as before. Next we create a Parts dictionary. So far, so good. Now we’re ready for the weird part. Here we create our namedtuple class and name it ‘Parts’. The second argument is a list of the keys from our dictionary. The last piece is this strange piece of code: (**Parts). The double asterisk means that we are calling our class using keyword arguments, which in this case is our dictionary. We could split this line into two parts to make it a little clearer:

>>> parts = namedtuple('Parts', Parts.keys())
>>> parts
<class '__main__.Parts'>
>>> auto_parts = parts(**Parts)
>>> auto_parts
Parts(amount=10, cost=1200.0, id_num='1234', desc='Ford Engine')

So here we do the same thing as before, except that we create the class first, then we call the class with our dictionary to create an object. The only other piece I want to mention is that namedtuple also accepts a verbose argument and a rename argument. The verbose argument is a flag that will print out class definition right before it’s built if you set it to True. The rename argument is useful if you’re creating your namedtuple from a database or some other system that your program doesn’t control as it will automatically rename the properties for you.


Wrapping Up

Now you know how to use Python’s handy namedtuple. I am already finding multiple uses for it in my code base and I hope you’ll find it helpful in yours. Happy coding!


Related Reading

Python – The datefinder package

Earlier this week, I came across another fun package called datefinder. The idea behind this package is that it can take any string with dates in it and transform them into a list of Python datetime objects. I would have loved this package at my old job where I did a lot of text file and database query parsing as there were many a time when finding the date and getting it into a format I could easily use was quite the nuisance.

Anyway, to install this handy package all you need to do is this:


pip install datefinder

I should note that when I ran this, it ended up also installing the following packages:

  • PyYAML-3.11
  • dateparser-0.3.2
  • jdatetime-1.7.2
  • python-dateutil-2.4.2
  • pytz-2015.7
  • regex-2016.1.10
  • six-1.10.0
  • umalqurra-0.2

Because of all this extra stuff, you might want to install this package into a virtualenv first. Let’s take a look at some code. Here’s a quick demo I tried:

>>> import datefinder
>>> data = '''Your appointment is on July 14th, 2016. Your bill is due 05/05/2016'''
>>> matches = datefinder.find_dates(data)
>>> for match in matches:
... 	print(match)
... 
2016-07-14 00:00:00
2016-05-05 00:00:00

As you can see, it worked quite well with these two common date formats. Another format that I used to have to support was the fairly typical ISO 8601 date format. Let’s see how datefinder behaves with that.

>>> data = 'Your report is due: 2016-02-04T20:16:26+00:00'
>>> matches = datefinder.find_dates(x)
>>> for i in matches: 
...     print(i)
... 
2016-02-04 00:00:00
2016-02-04 20:16:26

Interestingly, this particular version of the ISO 8601 format causes datefinder to return two matches. The first is just the date while the second has both the date and the time. Anyway, hopefully you’ll find this package useful in your projects. Have fun!

Python 3 – Import from github

The other day, I came across this interesting experimental package called import_from_github_com. The package uses the new import hooks provided in PEP 302 to basically allow you to import a package from github. What the package actually appears to do is install the package and add it to locals. Anyway, you need Python 3.2 or greater, git and pip to use this package.

Once it’s installed, you can do the following:

>>> from github_com.zzzeek import sqlalchemy
Collecting git+https://github.com/zzzeek/sqlalchemy
  Cloning https://github.com/zzzeek/sqlalchemy to /tmp/pip-acfv7t06-build
Installing collected packages: SQLAlchemy
  Running setup.py install for SQLAlchemy ... done
Successfully installed SQLAlchemy-1.1.0b1.dev0
>>> locals()
{'__builtins__': <module 'builtins' (built-in)>, '__spec__': None,
 '__package__': None, '__doc__': None, '__name__': '__main__', 
'sqlalchemy': <module 'sqlalchemy' from '/usr/local/lib/python3.5/site-packages/sqlalchemy/__init__.py'>,
 '__loader__': <class '_frozen_importlib.BuiltinImporter'>}

One important note that isn’t mentioned on the package’s github page is that you need to run Python as an administrator or it won’t be able to install its packages. At least, this was true for me on Xubuntu. Anyway I thought this was a neat little package and demonstrates some of neat little import hooks that you can add to Python 3.

wxPython’s Context Managers

The wxPython toolkit added context managers to its code base a few years ago, but for some reason you don’t see very many examples of their use. In this article, we’ll look at three examples of context managers in wxPython. A wxPython user was the first person to suggest using context managers in wxPython on the wxPython mailing list. We’ll start off by rolling our own context manager and then look at a couple of examples of built-in context managers in wxPython.

Continue reading wxPython’s Context Managers

Python 201 – An Intro to Context Managers

Python came out with a special new keyword several years ago in Python 2.5 that is known as the “with statement”. This new keyword allows a developer to create context managers. But wait! What’s a context manager? They are handy constructs that allow you to set something up and tear something down automatically. For example, you might want to open a file, write a bunch of stuff to it and then close it. This is probably the classic example of a context manager:

with open(path, 'w') as f_obj:
    f_obj.write(some_data)

Back in Python 2.4, you would have to do it the old fashioned way:

f_obj = open(path, 'w')
f_obj.write(some_data)
f_obj.close()

The way this works under the covers is by using some of Python’s magic methods: __enter__ and __exit__. Let’s try creating our own context manager to demonstrate how this all works!

Creating a Context Manager class

Rather than rewrite Python’s open method here, we’ll create a context manager that can create a SQLite database connection and close it when it’s done. Here’s a simple example:

import sqlite3
 
########################################################################
class DataConn:
    """"""
 
    #----------------------------------------------------------------------
    def __init__(self, db_name):
        """Constructor"""
        self.db_name = db_name
 
    #----------------------------------------------------------------------
    def __enter__(self):
        """
        Open the database connection
        """
        self.conn = sqlite3.connect(self.db_name)
        return self.conn
 
    #----------------------------------------------------------------------
    def __exit__(self, exc_type, exc_val, exc_tb):
        """
        Close the connection
        """
        self.conn.close()
 
#----------------------------------------------------------------------
if __name__ == '__main__':
    db = '/home/mdriscoll/test.db'
    with DataConn(db) as conn:
        cursor = conn.cursor()

In the code above, we created a class that takes a path to a SQLite database file. The __enter__ method executes automatically where it creates and returns the database connection object. Now that we have that, we can create a cursor and write to the database or query it. When we exit the with statement, it causes the __exit__ method to execute and that closes the connection.

Let’s try creating a context manager using another method.


Creating a Context Manager using contextlib

Python 2.5 not only added the with statement, but it also added the contextlib module. This allows us to create a context manager using contextlib’s contextmanager function as a decorator. Let’s try creating a context manager that opens and closes a file after all:

from contextlib import contextmanager
 
@contextmanager
def file_open(path):
    try:
        f_obj = open(path, 'w')
        yield f_obj
    except OSError:
        print "We had an error!"
    finally:
        print 'Closing file'
        f_obj.close()
 
#----------------------------------------------------------------------
if __name__ == '__main__':
    with file_open('/home/mdriscoll/test.txt') as fobj:
        fobj.write('Testing context managers')

Here we just import contextmanager from contextlib and decorate our file_open function with it. This allows us to call file_open using Python’s with statement. In our function, we open the file and then yield it out so the calling function can use it. Once the with statement ends, control returns back to the file_open function and it continues with the code following the yield statement. That causes the finally statement to execute, which closes the file. If we happen to have an OSError while working with the file, it gets caught and finally statement still closes the file handler.


Wrapping Up

Context managers are a lot of fun and come in handy all the time. I use them in my automated tests all the time for opening and closing dialogs, for example. Now you should be able to use some of Python’s built-in tools to create your own context managers. Have fun and happy coding!


Related Reading

An Intro to argparse

Have you ever wondered how to process command line arguments in Python? Yeah, there’s a module for that. It’s called argparse, which is a replacement for optparse. In this article, we’ll be taking a whirlwind tour of this helpful module. Let’s start with something simple!

Getting Started

I have always found the simplest way to explain a coding concept is to show some code. So that’s what we’re going to do. Here’s a super simple example that doesn’t do much of anything:

>>> import argparse
>>> parser = argparse.ArgumentParser(
...         description="A simple argument parser",
...         epilog="This is where you might put example usage"
...     )
... 
>>> parser.print_help()
usage: _sandbox.py [-h]
 
A simple argument parser
 
optional arguments:
  -h, --help  show this help message and exit
 
This is where you might put example usage

Here we just import argparse and give it a description and set up a usage section. The idea here is that when you ask the program you are creating for help, it will tell you how to use it. In this case, it prints out a simple description, the default optional arguments (“-h” in this case) and example usage.

Continue reading An Intro to argparse

wxPython: Ensuring Only One Instance Per Frame

The other day, I came across an interesting StackOverflow question where the fellow was trying to figure out how to open a sub-frame only once. Basically he wanted a single instance of the sub-frame (and other sub-frames). After digging around a bit on Google, I found an old thread from the wxPython Google Group that had an interesting approach to doing what was needed.

Basically it required a bit of meta-programming, but it was a fun little exercise that I thought my readers would find interesting. Here’s the code:

Continue reading wxPython: Ensuring Only One Instance Per Frame

Python 101: Episode #10 – Functions

I recently completed Episode #10 of the Python 101 Screencast. It’s about the basics of functions. I hope you like it!

wxPython: Extracting XML from the RichTextCtrl

I recently came across a StackOverflow question where the fellow was asking how to get wxPython’s RichTextCtrl’s XML data so he could save it to a database. I don’t know much about this control, but after a quick Google search, I found an article from 2008 that gave me the information I needed. I took that example and cut it down to the following example:

Continue reading wxPython: Extracting XML from the RichTextCtrl

Python 101 Screencast: Episode #8 – File I/O

Check out my latest video from the Python 101 Screencast. In this episode we learn how to use Python’s file handling capabilities. You will learn how to read, write and append to a file.