Monthly Archives: July 2012

Advanced Python – How to Dynamically Load Modules or Classes

Every now and then you’ll find yourself needing to load modules or classes dynamically. In other words, you’ll want to be able to import a module without knowing ahead of time which one you’re going to import. In this article, we’ll look at two ways to accomplish this feat in Python.

Using the __import__ Magic Method

The easiest way to do this sort of thing is to use the “magic” method, __import__. In fact, if you did a Google search on this topic, that’s probably the first method you’ll find. Here’s the basic methodology:

module = __import__(module_name)
my_class = getattr(module, class_name)
instance = my_class()

Both module_name and class_name have to be strings in the above code. If the class you’re importing requires some parameters passed to it, then you’ll have to add that logic too. Here’s a more fleshed out example to help you understand how this works:

class DynamicImporter:
    def __init__(self, module_name, class_name):
        module = __import__(module_name)
        my_class = getattr(module, class_name)
        instance = my_class()
        print instance
if __name__ == "__main__":
    DynamicImporter("decimal", "Context")

If you run this code, you should see something like the following output to stdout:

Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[DivisionByZero, Overflow, InvalidOperation])

This shows that the code works as expected in that it imports the decimal and returns an instance of the Context class. That was pretty straight-forward. Let’s look at the other method of doing this!

Using Python’s imp Module

Using the imp module is a little bit more complicated. You end up needing to do a little recursive call and you’ll want to wrap everything in exception handlers too. Let’s take a look at the code and then I’ll explain why:

import imp
import sys
def dynamic_importer(name, class_name):
    Dynamically imports modules / classes
        fp, pathname, description = imp.find_module(name)
    except ImportError:
        print "unable to locate module: " + name
        return (None, None)
        example_package = imp.load_module(name, fp, pathname, description)
    except Exception, e:
        print e
        myclass = imp.load_module("%s.%s" % (name, class_name), fp, pathname, description)
        print myclass
    except Exception, e:
        print e
    return example_package, myclass
if __name__ == "__main__":
    module, modClass = dynamic_importer("decimal", "Context")

The imp module comes with a find_module method that will look for the module for you. I wasn’t able to get it to always work reliably though, so I wrapped it in a try/except. For example, it couldn’t find SQLAlchemy at all and when I tried to find wx.Frame, I got it, but not the right one. I don’t know if that latter issue is with wxPython or imp though. Anyway, after imp finds the module, it returns an open file handler, the path to the module that imp found and a description of sorts (see the docs or the PyMOTW article below). Next you’ll need to load the module to get it “imported”. If you want to pull out a class inside the module, then you use load_module again. At this point, you should have the same objects that you had in the first method.

Update: I’ve had one commenter utterly trash this article. So in response, I wanted to clear up a few things. First of all, bare try/excepts are usually bad. I fixed the code, but I see it all the time on blogs, in production code and even in books. Is it a good idea? No. If you know what errors to expect, you should handle them or re-raise the error. Secondly, I’ve been told that “__import__” isn’t a “method”. True enough. It’s actually a function in the internals of Python. However, every double-underscore function in Python that I’ve seen are called “magic methods”. See Foord’s book or this blog that lists a bunch of other resources on the topic. Finally, my IDE (Wingware if anyone cares) adds some silly stuff to classes and functions/methods all by itself, such as empty docstrings or in the __init__ docstring, it will add “Constructor”.

Additional Reading

  • StackOverflow: Python dynamic instantiation from string name of a class in dynamically imported module
  • StackOverflow: How can I import a package using __import__() when the package name is only known at runtime?
  • PyMOTW: imp – Interface to module import mechanism
  • Python official documentation for the imp module

Python 201: List Comprehensions

List comprehensions in Python are very handy. They can also be a little hard to understand when and why you would use them. List comprehensions tend to be harder to read than just using a simple for loop as well. We’ll spend some time looking at how to construct list comprehensions and learn how they can be used. By the end of this article, you should be competent enough to use them confidently. Continue reading Python 201: List Comprehensions

Tkinter: How to Show / Hide a Window

Today we’re going to take a look at Tkinter! I was curious about how one would go about hiding a frame and then re-showing it using Tkinter and I kept finding threads (like this one) that talked about using withdraw() and deiconify() but didn’t really provide any usable code. In wxPython, I did this sort of thing using pubsub. We’ll go over three different versions of how to hide and show the root frame. Continue reading Tkinter: How to Show / Hide a Window

An Intro to Mercurial

Mercurial is a free, distributed source control versioning tool, similar to git or bazaar. Some might even compare it CVS or Subversion (SVN), although those are not distributed versioning systems. The Python programming core development team chose to switch to Mercurial from SVN a couple years ago and many other high profile 3rd party Python projects have too. There are many projects that are using Git as well. In this tutorial, we will go through the basics of using Mercurial. If you need more in depth information, there is a pretty exhaustive guide and an online Mercurial book that should fulfill your needs. Continue reading An Intro to Mercurial

Python PDF Series – An Intro to metaPDF

While researching PDF libraries for Python, I stumbled across another little project called metaPDF. According to its website, metaPDF is a lightweight Python library optimized for metadata extraction and insertion, and it is a fast wrapper over the excellent pyPdf library. It works by quickly searching the last 2048 bytes of the PDF before parsing the xref table, offering a 50-60% performance increase over directly parsing the table line by line. I’m not really sure how useful that will be, but let’s try it out and see what metaPDF can do. Continue reading Python PDF Series – An Intro to metaPDF

Raspithon starts today!

Some teens from around the world decided to learn Python using Raspberry Pi to write a game during a sprint starting today and running through tomorrow. They are taking donations to give to the Raspberry Pi foundation too. You can follow their live stream if you want to. Raspberry Pi is an ARM GNU/Linux box for $25. I thought this was a fun project that my readers might find interesting.

Python 101: Downloading a File with ftplib

There are lots of different ways to download a file from the internet using Python. One popular way is to connect to an FTP server and download your files that way. So that is what we will be looking at in this article. All you need is your standard installation of Python. It includes a library called ftplib, which has all the bits and pieces we need to accomplish this task. Continue reading Python 101: Downloading a File with ftplib

Python: A Simple Step-by-Step SQLite Tutorial

SQLite is a self-contained, server-less, config-free transactional SQL database engine. Python gained the sqlite3 module all the way back in version 2.5 which means that you can create SQLite database with any current Python without downloading any additional dependencies. Mozilla uses SQLite databases for its popular Firefox browser to store bookmarks and other various pieces of information. In this article you will learn the following:

  • How to create a SQLite database
  • How to insert data into a table
  • How to edit the data
  • How to delete the data
  • Basic SQL queries

This article will be similar in function to the recent SQLAlchemy tutorial that appeared on this site earlier this month. If you want to inspect your database visually, you can use the SQLite Manager plugin for Firefox or if you like the command line, you can use SQLite’s command line shell Continue reading Python: A Simple Step-by-Step SQLite Tutorial

Parsing XML and Creating a PDF Invoice with Python

Note: The following post was originally published over on Dzone. I changed the title because I already wrote several XML parsing articles and don’t want my readers to get this one confused with the others.

One of the common tasks I am given in my day job is to take some data format input and parse it to create a report or some other document. Today we’ll look at taking some XML input, parsing it with the Python programming language and then creating a letter in PDF format using Reportlab, a 3rd party package for Python. Let’s say my company receives an order for three items that I need to fulfill. The XML for that could look like the following code: Continue reading Parsing XML and Creating a PDF Invoice with Python