Python 201 – An Intro to Context Managers

Python came out with a special new keyword several years ago in Python 2.5 that is known as the “with statement”. This new keyword allows a developer to create context managers. But wait! What’s a context manager? They are handy constructs that allow you to set something up and tear something down automatically. For example, you might want to open a file, write a bunch of stuff to it and then close it. This is probably the classic example of a context manager:

with open(path, 'w') as f_obj:

Back in Python 2.4, you would have to do it the old fashioned way:

f_obj = open(path, 'w')

The way this works under the covers is by using some of Python’s magic methods: __enter__ and __exit__. Let’s try creating our own context manager to demonstrate how this all works!

Creating a Context Manager class

Rather than rewrite Python’s open method here, we’ll create a context manager that can create a SQLite database connection and close it when it’s done. Here’s a simple example:

import sqlite3

class DataConn:

    def __init__(self, db_name):
        self.db_name = db_name

    def __enter__(self):
        Open the database connection
        self.conn = sqlite3.connect(self.db_name)
        return self.conn

    def __exit__(self, exc_type, exc_val, exc_tb):
        Close the connection

if __name__ == '__main__':
    db = '/home/mdriscoll/test.db'
    with DataConn(db) as conn:
        cursor = conn.cursor()

In the code above, we created a class that takes a path to a SQLite database file. The __enter__ method executes automatically where it creates and returns the database connection object. Now that we have that, we can create a cursor and write to the database or query it. When we exit the with statement, it causes the __exit__ method to execute and that closes the connection.

Let’s try creating a context manager using another method.

Creating a Context Manager using contextlib

Python 2.5 not only added the with statement, but it also added the contextlib module. This allows us to create a context manager using contextlib’s contextmanager function as a decorator. Let’s try creating a context manager that opens and closes a file after all:

from contextlib import contextmanager

def file_open(path):
        f_obj = open(path, 'w')
        yield f_obj
    except OSError:
        print "We had an error!"
        print 'Closing file'

if __name__ == '__main__':
    with file_open('/home/mdriscoll/test.txt') as fobj:
        fobj.write('Testing context managers')

Here we just import contextmanager from contextlib and decorate our file_open function with it. This allows us to call file_open using Python’s with statement. In our function, we open the file and then yield it out so the calling function can use it. Once the with statement ends, control returns back to the file_open function and it continues with the code following the yield statement. That causes the finally statement to execute, which closes the file. If we happen to have an OSError while working with the file, it gets caught and finally statement still closes the file handler.

Wrapping Up

Context managers are a lot of fun and come in handy all the time. I use them in my automated tests all the time for opening and closing dialogs, for example. Now you should be able to use some of Python’s built-in tools to create your own context managers. Have fun and happy coding!

Related Reading

4 thoughts on “Python 201 – An Intro to Context Managers”

  1. I suspect there’s an error in your DataConn. By explicitly re-raising the exception in the __exit__ method, you’re neglecting to close the connection if an exception was raised—the very point of a context manager!

    The Python language spec ( says that if an exception was raised, and the __exit__ method returns a false value, the exception will continue to propagate. Since “falling off the end” (not executing a return statement) like you do here returns a false value (None), the exception will be reraised automatically. Hence, you can remove the if check entirely and still get the same results—except that the connection will be properly closed.

    If an exception was raised and the __exit__ method returns a true value, it is suppressed.

    In short, most of the time, most people will ignore the three arguments to __exit__, because they do the right thing automatically. Only in the specialized cases that you might want to suppress an exception do you need to pay attention to them.

  2. That’s quite interesting and somewhat contrary to my experience. I went ahead and removed the if check for this example, but I have noticed in some automated tests that we run that if we don’t raise an error, the error is always suppressed and we get a very weird traceback.

  3. Hmm, I know that if the __exit__ method itself raises its own exception, the original exception is lost (you only get the traceback from the new __exit__ exception), but I don’t think I’ve ever seen an original exception get spuriously suppressed when __exit__ returns a false value/nothing.

  4. What I recall seeing is that the exception is appearing to happen inside the with statement, which can be pretty confusing. Makes me wish I still had some of those weird tracebacks

Comments are closed.