Python code testing is something new to me. It’s not required where I work, so I haven’t spent much time looking into it, besides reading a book on the subject and reading a few blogs. However, I decided it was high time I check this out and see what all the excitement is about. In this article, you will learn about Test Driven Development (TDD) with Python using Python’s builtin unittest module. This is actually based on my one experience of TDD and pair programming (thanks Matt and Aaron!). In this article, we will be learning how to score bowling with Python!

Getting Started

I did a search for how to score bowling on the internet. I found a helpful tutorial on About.com that you’re welcome to read too. Once you know the rules, it’s time to write some tests. In case you didn’t know, the idea behind Test Driven Development is that you write the tests BEFORE you write the actual code. In this article, we will write a test, then some code to pass the test. We will iterate back and forth between writing tests and code until we’re done. For this article, we’ll write just three tests. Let’s get started!

The First Test

Our first test will be to test our game object and see if it can calculate the correct total if we roll eleven times and only knock over one pin each time. This should give us a total of eleven.

import unittest
 
########################################################################
class TestBowling(unittest.TestCase):
    """"""
 
    #----------------------------------------------------------------------
    def test_all_ones(self):
        """Constructor"""
        game = Game()
        game.roll(11, 1)
        self.assertEqual(game.score, 11)

This is a pretty simple test. We create a game object and then call its roll method eleven times with a score of one each time. Then we use the assertEqual method from the unittest module to test if the game object’s score is correct (i.e. eleven). The next step is to write the simplest code you can think of to make the test pass. Here’s one example:

########################################################################
class Game:
    """"""
 
    #----------------------------------------------------------------------
    def __init__(self):
        """Constructor"""
        self.score = 0
 
    #----------------------------------------------------------------------
    def roll(self, numOfRolls, pins):
        """"""
        for roll in numOfRolls:
            self.score += pins

For simplicity’s sake, you can just copy and paste that into the same file with your test. We’ll break them into two files for our next test. Anyway, as you can see, our Game class is super simple. All that was needed to pass the test was a score property and a roll method that can update it. If you don’t know how for loops work, then you need to go read a Python tutorial.

Let’s run the test and see if it passes! The easiest way to run the tests is to add the following two lines of code to the bottom of the file:

if __name__ == '__main__':
    unittest.main()

Then just run the Python file via the command line, If you do, you should get something like the following:

E
======================================================================
ERROR: test_all_ones (__main__.TestBowling)
Constructor
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Mike\Documents\My Dropbox\Scripts\Testing\bowling\test_one.py",
 line 27, in test_all_ones
    game.roll(11, 1)
  File "C:\Users\Mike\Documents\My Dropbox\Scripts\Testing\bowling\test_one.py",
 line 15, in roll
    for roll in numOfRolls:
TypeError: 'int' object is not iterable
 
----------------------------------------------------------------------
Ran 1 test in 0.001s
 
FAILED (errors=1)

Oops! We’ve got a mistake in there somewhere. It looks like we’re passing an Integer and then trying to iterate over it. That doesn’t work! We need to change our Game object’s roll method to the following to make it work:

#----------------------------------------------------------------------
def roll(self, numOfRolls, pins):
    """"""
    for roll in range(numOfRolls):
        self.score += pins

If you run the test now, you should get the following:

.
----------------------------------------------------------------------
Ran 1 test in 0.000s
 
OK

Note the “.” because it’s important. That little dot means that one test has run and that it passed. The “OK” at the end clues you into that fact as well. If you study the original output, you’ll notice it leads off with an “E” for error and there’s no dot! Let’s move on to test #2.

The Second Test

For the second test, we’ll test what happens when we get a strike. We’ll need to change the first test to use a list for the number of pins knocked down in each frame though, so we’ll look at both tests here. You’ll probably find this to be a fairly common process where you may need to edit a couple of tests due to fundamental changes in what you’re testing for. Normally this will only happen at the beginning of your coding and you will get better later on such that you shouldn’t need to do this. Since this is my first time doing this, I wasn’t thinking far enough a head. Anyway, let’s take a look at the code:

from game import Game
import unittest
 
########################################################################
class TestBowling(unittest.TestCase):
    """"""
 
    #----------------------------------------------------------------------
    def test_all_ones(self):
        """Constructor"""
        game = Game()
        pins = [1 for i in range(11)]
        game.roll(11, pins)
        self.assertEqual(game.score, 11)
 
    #----------------------------------------------------------------------
    def test_strike(self):
        """
        A strike is 10 + the value of the next two rolls. So in this case
        the first frame will be 10+5+4 or 19 and the second will be
        5+4. The total score would be 19+9 or 28.
        """
        game = Game()
        game.roll(11, [10, 5, 4])
        self.assertEqual(game.score, 28)
 
if __name__ == '__main__':
    unittest.main()

Let’s take a look at our first test and how it changed. Yes, we’re breaking the rules here a bit when it comes to TDD. Feel free to NOT change the first test and see what breaks. In the test_all_ones we pins to equal list comprehension which created a list of eleven ones. Then we passed that to our game object’s roll method along with the number of rolls.

In the second test, we roll a strike in our first roll, a five in our second and a four in our third. Note that we went a head and told it that we were passing in eleven rolls and yet we only pass in three. This means that we need to set the other eight rolls to zeros. Finally, we use our trusty assertEqual method to check if we get the right total. Finally, note that we’re now importing the Game class rather than keeping it with the tests. Now we need to implement the code necessary to pass these two tests. Let’s take a look at one possible solution:

########################################################################
class Game:
    """"""
 
    #----------------------------------------------------------------------
    def __init__(self):
        """Constructor"""
        self.score = 0
        self.pins = [0 for i in range(11)]
 
    #----------------------------------------------------------------------
    def roll(self, numOfRolls, pins):
        """"""
        x = 0
        for pin in pins:
            self.pins[x] = pin
            x += 1
        x = 0
        for roll in range(numOfRolls):
            if self.pins[x] == 10:
                self.score = self.pins[x] + self.pins[x+1] + self.pins[x+2]
            else:
                self.score += self.pins[x]
            x += 1
        print self.score

Right off the bat, you will notice that we have a new class attribute called self.pins that holds the default pin list, which is eleven zeroes. Then in our roll method, we add the correct scores to the correct position in the self.pins list in the first loop. Then in the second loop, we check to see if the pins knocked down equals ten. If it does, we add it and the next two scores to score. Otherwise, we do what we dd before. At the end of the method, we print out the score to check if it’s what we expect. At this point, we’re ready to code up our final test.

The Third (and Final) Test

Our last test will test for the correct score that would result should someone roll a spare. The test is easy, the solution is slightly more difficult. While we’re at it, we’re going to refactor the test code a bit. As usual, we will look at the test first.

from game_v2 import Game
import unittest
 
########################################################################
class TestBowling(unittest.TestCase):
    """"""
 
    #----------------------------------------------------------------------
    def setUp(self):
        """"""
        self.game = Game()
 
    #----------------------------------------------------------------------
    def test_all_ones(self):
        """
        If you don't get a strike or a spare, then you just add up the
        face value of the frame. In this case, each frame is worth
        one point, so the total is eleven.
        """
        pins = [1 for i in range(11)]
        self.game.roll(11, pins)
        self.assertEqual(self.game.score, 11)
 
    #----------------------------------------------------------------------
    def test_spare(self):
        """
        A spare is worth 10, plus the value of your next roll. So in this
        case, the first frame will be 5+5+5 or 15 and the second will be
        5+4 or 9. The total is 15+9, which equals 24,
        """
        self.game.roll(11, [5, 5, 5, 4])
        self.assertEqual(self.game.score, 24)
 
    #----------------------------------------------------------------------
    def test_strike(self):
        """
        A strike is 10 + the value of the next two rolls. So in this case
        the first frame will be 10+5+4 or 19 and the second will be
        5+4. The total score would be 19+9 or 28.
        """
        self.game.roll(11, [10, 5, 4])
        self.assertEqual(self.game.score, 28)
 
if __name__ == '__main__':
    unittest.main()

First off, we added a setUp method that will create a self.game object for us for each test. If we were accessing a database or something like that, we would probably have a tear down method too for closing connections or files or that sort of thing. These are run at the beginning and end of each test respectively should they exist. The test_all_ones and test_strike tests are basically the same except that they are using “self.game” now. The only the new test is test_spare. The docstring explains how spares work and the code is just two lines. Yes, you can figure this out. Let’s look at the code we’ll need to pass these tests:

# game_v2.py
########################################################################
class Game:
    """"""
 
    #----------------------------------------------------------------------
    def __init__(self):
        """Constructor"""
        self.score = 0
        self.pins = [0 for i in range(11)]
 
    #----------------------------------------------------------------------
    def roll(self, numOfRolls, pins):
        """"""
        x = 0
        for pin in pins:
            self.pins[x] = pin
            x += 1
        x = 0
        spare_begin = 0
        spare_end = 2
        for roll in range(numOfRolls):
            spare = sum(self.pins[spare_begin:spare_end])
            if self.pins[x] == 10:
                self.score = self.pins[x] + self.pins[x+1] + self.pins[x+2]
            elif spare == 10:
                self.score = spare + self.pins[x+2]
                x += 1
            else:
                self.score += self.pins[x]
            x += 1
            if x == 11:
                break
            spare_begin += 2
            spare_end += 2
        print self.score

For this part of the puzzle, we add to our conditional statement in our loop. To calculate the spare’s value, we use the spare_begin and spare_end list positions to get the right values from our list and then we sum them up. That’s what the spare variable is for. That may be better placed in the elif, but I’ll leave that for the reader to experiment with. Technically, that’s just the first half of the spare score. The second half are the next two rolls, which is what you’ll find in the calculation in the elif portion of the current code. The rest of the code is the same.

Other Notes

As you may have guessed, there is a whole lot more to the unittest module than what is covered here. There are lots of other asserts you can use to test results with. You can skip tests, run the tests from the command line, use a TestLoader to create a test suite and much, much more. Be sure to read the full documentation when you get the chance as this tutorial barely scratches the surface of this library.

Wrapping Up

I don’t know a whole lot about testing yet, but I learned a lot working on this article and hopefully didn’t botch anything too badly. If I did, let me know and I’ll update the post. Also feel free to let me know what other aspects of testing that I should cover in the future and I’ll take a look. I’d like to tackle some of the other unittest features I didn’t get to in this tutorial along with mock and nose. What would you suggest?

Before PyCon, I was approached by a representative from Packt Publishing to review one of their books. They wanted me to read Daniel Arbuckle’s Python Testing: Beginner’s Guide. I’m not really into testing frameworks or test driven development and thought this would be a good excuse to learn the methodology and see if it was something that was actually valuable or just a lot of hype. The book is only 256 pages long, so I told the Packt contact that it would probably take me a week or so to review. After a week, she seemed kind of anxious that I wasn’t done. Thus, this is a partial review. I don’t like being pressured to review something fast. I want my reviews to be thorough and of the best quality I can make them. You will get a thorough review of the chapters that I did manage to finish. If you want a full review, I’m pretty sure someone else on Python Planet said they were doing one.

I read chapters 1-6 and then skimmed 7-10. The author writes in an engaging manner and the chapters are broken up into sections, which makes it easy to pick up, do a section, and put it back down without losing your place. The primary reason that I was slow to review this book is that it has lots of code examples and it takes quite a while to re-type them all. I hate technical books that are riddled with broken code, so I wanted to make sure that this book’s code and tests worked. If I review a book with lots of code in the future, I will insist on lots of time or an e-book copy so I can just copy and paste the code.

Anyway, the first couple of chapters went by with only minor errors in the text itself. The content was pretty good, but the editors missed simple stuff where the author used “white” instead of “write”. There are silly mistakes like this in most chapters. The real problems don’t happen until chapter 3, which is about creating unit tests with doctest. On page 45, there’s a code example that includes the following line:

self.integrated_error +q= err * delta

Since the author recommends using Python 2.6, I thought that “+q=” might be some kind of new format that I was unaware of. But it’s just a typo. On page 48-50, he has a revised test from earlier in the chapter. The text clearly states that all revisions will be clearly marked. They are not and I was unable to find all the revisions to make the new version work. I didn’t want to re-type the entire thing in again, so I skipped it and decided to download the author’s code bundle from the publisher’s website.

Here is where things really went downhill for me. After getting the code, I noticed that the files and folders were not arranged by chapter, but by some other method. This made it very difficult to find what I was looking for. I eventually did find a chapters sub-folder in the “tests” folder (I think). My plan was to do a diff on that code versus mine. When I opened the book code though, I noticed that the author had included a bunch of mock tests in it. He doesn’t cover mock until chapter 4. I considered stopping here and just telling everyone I knew to avoid this book, but since I got it for free, I decided to soldier on.

Chapter 4 is on the Python Mocker package by Labix. This chapter was quite good and I think I learned a lot from it. The only downer was that the author mentioned that you could use easy_install to install this module and that by so doing, Nose would be ready to run. That was just plain dumb. Other than that, I thought this was a good chapter.

Chapter 5 introduced the reader to Python’s unittest library. I don’t recall if I tested all the code in this chapter or not, but I didn’t note anything that was wrong. I thought this was a good introduction to unittest, especially for a beginner.

For chapter 6, the topic is changed to Nose. As you may have noticed from my descriptions so far, the author is quite gradual in how he went through the topics. He started with doctests, moved to mock, then unittests and then Nose, which is kind of a test runner and enhancer. I had weird issues with Nose when I tried to easy_install it. For some reason, I ended up with the 0.10.1 version which didn’t work with all the commands in this book. Once I got the current version, everything worked better. I should note that I was unable to get the example on page 112 to pass nosetests. I have no idea what the problem is, although it has to do with the Python Mocker module. It may be that I mistyped something, but I couldn’t figure it out.

The author covers Nose’s configuration file briefly and goes over a few command line flags for nosetests. He also covers fixtures and how to use them to enhance your tests. Finally he shows how to use Nose’s pattern matching abilities to run tests that match a certain naming convention. Pretty cool stuff!

I started to skim in earnest after this chapter though. In seven, he does a step-by-step walk-through depicting how to test and create a personal scheduling program. I look forward to reading this section and seeing how complex or simple this is. There is certainly code aplenty in this section and lots of tests (or he may just be re-printing some of the examples in smaller parts for commentary). I have no idea if any of it is functional though.

In chapter 8, the book moves on to Web Testing with Twill. I thought this was an interesting choice since Twill’s website says that it hasn’t been updated in over 3 years. I tested a few of the examples in this chapter and they worked as advertised. Much of this section is made up of explanations of Twill commands though. The last section purportedly shows how to integrate Twill tests with unittest tests. I didn’t do that part though.

Chapter 9 talks about Integration and System testing. I thought the idea of drawing concentric circles in a diagram to figure out how to create integrated units for testing was a good concept. That will probably appeal to the readers who like visuals. Alas, I didn’t really read anything more than the first couple of pages, but it looks like he uses his schedule planner example here as well.

The final chapter entitled “Other Testing Tools and Techniques”. I think this chapter looks fairly interesting, but didn’t get a chance to read it. Looking at the text now, it would seem to briefly cover the following: coverage.py, version control hooks (Bazaar, Mercurial, Darcs, Subversion), and buildbot.

There is also an appendix with Pop Quiz answers.

Overall, I think the chapters that I read had a lot of good information and that is was very well organized. Unfortunately, the good was pretty badly marred by minor grammatical or sentence structure issues and the occasionally larger issue of broken code/tests. If it happens once, then I expect that there probably other code or tests that are broken as well. If this is the case, then I recommend buying it used or wait for a Packt sale of some sort. On the other hand, if the book is pristine after chapter 6…well, I guess that really doesn’t matter since that would mean that over half the book had problems.

Okay. Here’s a better recommendation: Find a Borders or Barnes & Nobles near you that has the book and free wi-fi. Go there, find the book, and read it and mess around with the examples BEFORE you buy it.

Rating Scale: 1 (low) – 5 (high)

Code rating = 3
Writing = 4
Organization = 5

Average Score = 4 (take a point or two off if broken code annoys you)