Advanced Python – How to Dynamically Load Modules or Classes

Every now and then you’ll find yourself needing to load modules or classes dynamically. In other words, you’ll want to be able to import a module without knowing ahead of time which one you’re going to import. In this article, we’ll look at two ways to accomplish this feat in Python.

Using the __import__ Magic Method

The easiest way to do this sort of thing is to use the “magic” method, __import__. In fact, if you did a Google search on this topic, that’s probably the first method you’ll find. Here’s the basic methodology:

module = __import__(module_name)
my_class = getattr(module, class_name)
instance = my_class()

Both module_name and class_name have to be strings in the above code. If the class you’re importing requires some parameters passed to it, then you’ll have to add that logic too. Here’s a more fleshed out example to help you understand how this works:

class DynamicImporter:

    def __init__(self, module_name, class_name):
        module = __import__(module_name)
        my_class = getattr(module, class_name)
        instance = my_class()
        print instance
if __name__ == "__main__":
    DynamicImporter("decimal", "Context")

If you run this code, you should see something like the following output to stdout:

Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[DivisionByZero, Overflow, InvalidOperation])

This shows that the code works as expected in that it imports the decimal and returns an instance of the Context class. That was pretty straight-forward. Let’s look at the other method of doing this!

Using Python’s imp Module

Using the imp module is a little bit more complicated. You end up needing to do a little recursive call and you’ll want to wrap everything in exception handlers too. Let’s take a look at the code and then I’ll explain why:

import imp
import sys

def dynamic_importer(name, class_name):
    Dynamically imports modules / classes
        fp, pathname, description = imp.find_module(name)
    except ImportError:
        print "unable to locate module: " + name
        return (None, None)
        example_package = imp.load_module(name, fp, pathname, description)
    except Exception, e:
        print e
        myclass = imp.load_module("%s.%s" % (name, class_name), fp, pathname, description)
        print myclass
    except Exception, e:
        print e
    return example_package, myclass
if __name__ == "__main__":
    module, modClass = dynamic_importer("decimal", "Context")

The imp module comes with a find_module method that will look for the module for you. I wasn’t able to get it to always work reliably though, so I wrapped it in a try/except. For example, it couldn’t find SQLAlchemy at all and when I tried to find wx.Frame, I got it, but not the right one. I don’t know if that latter issue is with wxPython or imp though. Anyway, after imp finds the module, it returns an open file handler, the path to the module that imp found and a description of sorts (see the docs or the PyMOTW article below). Next you’ll need to load the module to get it “imported”. If you want to pull out a class inside the module, then you use load_module again. At this point, you should have the same objects that you had in the first method.

Update: I’ve had one commenter utterly trash this article. So in response, I wanted to clear up a few things. First of all, bare try/excepts are usually bad. I fixed the code, but I see it all the time on blogs, in production code and even in books. Is it a good idea? No. If you know what errors to expect, you should handle them or re-raise the error. Secondly, I’ve been told that “__import__” isn’t a “method”. True enough. It’s actually a function in the internals of Python. However, every double-underscore function in Python that I’ve seen are called “magic methods”. See Foord’s book or this blog that lists a bunch of other resources on the topic. Finally, my IDE (Wingware if anyone cares) adds some silly stuff to classes and functions/methods all by itself, such as empty docstrings or in the __init__ docstring, it will add “Constructor”.

Additional Reading

  • StackOverflow: Python dynamic instantiation from string name of a class in dynamically imported module
  • StackOverflow: How can I import a package using __import__() when the package name is only known at runtime?
  • PyMOTW: imp – Interface to module import mechanism
  • Python official documentation for the imp module

9 thoughts on “Advanced Python – How to Dynamically Load Modules or Classes”

  1. I think it would be a good place to start, but I don’t know how I’d get it to work off the top of my head. The imp module is low-level enough that I think you should be able to do that.

  2. Use importlib! It’s in the stdlib as of 2.7 and a backport is also available on pypi.

  3. Ya, I don’t know either. The ast module would be my next guess. Anyway, thanks for the new know how.

  4. Thanks! I had forgotten about this one as I’ve been stuck in Python 2.6 land for quite a while at my job.

  5. Umesh Bhaskaran

    The second example says call imp.load_module again to load a class name. Does it really work ? No matter what you pass to the name argument it will always import the module not the class. This is what I figured out may be I am wrong.

  6. This is a good writeup. Foo to purest. Puritans screw up everything… Try/excepts are good for many cases especially if you log and follow up for the next iteration… Good stuff and trolls suck.

  7. Pingback: How to import module by variable name and how to name said module – PythonCharm

  8. Pingback: 【已解决】Python中如何动态检测是否安装某个库 – 在路上

Comments are closed.