Book Review: Python Testing

Before PyCon, I was approached by a representative from Packt Publishing to review one of their books. They wanted me to read Daniel Arbuckle’s Python Testing: Beginner’s Guide. I’m not really into testing frameworks or test driven development and thought this would be a good excuse to learn the methodology and see if it was something that was actually valuable or just a lot of hype. The book is only 256 pages long, so I told the Packt contact that it would probably take me a week or so to review. After a week, she seemed kind of anxious that I wasn’t done. Thus, this is a partial review. I don’t like being pressured to review something fast. I want my reviews to be thorough and of the best quality I can make them. You will get a thorough review of the chapters that I did manage to finish. If you want a full review, I’m pretty sure someone else on Python Planet said they were doing one.

I read chapters 1-6 and then skimmed 7-10. The author writes in an engaging manner and the chapters are broken up into sections, which makes it easy to pick up, do a section, and put it back down without losing your place. The primary reason that I was slow to review this book is that it has lots of code examples and it takes quite a while to re-type them all. I hate technical books that are riddled with broken code, so I wanted to make sure that this book’s code and tests worked. If I review a book with lots of code in the future, I will insist on lots of time or an e-book copy so I can just copy and paste the code.

Anyway, the first couple of chapters went by with only minor errors in the text itself. The content was pretty good, but the editors missed simple stuff where the author used “white” instead of “write”. There are silly mistakes like this in most chapters. The real problems don’t happen until chapter 3, which is about creating unit tests with doctest. On page 45, there’s a code example that includes the following line:

[python]
self.integrated_error +q= err * delta


Since the author recommends using Python 2.6, I thought that “+q=” might be some kind of new format that I was unaware of. But it’s just a typo. On page 48-50, he has a revised test from earlier in the chapter. The text clearly states that all revisions will be clearly marked. They are not and I was unable to find all the revisions to make the new version work. I didn’t want to re-type the entire thing in again, so I skipped it and decided to download the author’s code bundle from the publisher’s website.

Here is where things really went downhill for me. After getting the code, I noticed that the files and folders were not arranged by chapter, but by some other method. This made it very difficult to find what I was looking for. I eventually did find a chapters sub-folder in the “tests” folder (I think). My plan was to do a diff on that code versus mine. When I opened the book code though, I noticed that the author had included a bunch of mock tests in it. He doesn’t cover mock until chapter 4. I considered stopping here and just telling everyone I knew to avoid this book, but since I got it for free, I decided to soldier on.

Chapter 4 is on the Python Mocker package by Labix. This chapter was quite good and I think I learned a lot from it. The only downer was that the author mentioned that you could use easy_install to install this module and that by so doing, Nose would be ready to run. That was just plain dumb. Other than that, I thought this was a good chapter.

Chapter 5 introduced the reader to Python’s unittest library. I don’t recall if I tested all the code in this chapter or not, but I didn’t note anything that was wrong. I thought this was a good introduction to unittest, especially for a beginner.

For chapter 6, the topic is changed to Nose. As you may have noticed from my descriptions so far, the author is quite gradual in how he went through the topics. He started with doctests, moved to mock, then unittests and then Nose, which is kind of a test runner and enhancer. I had weird issues with Nose when I tried to easy_install it. For some reason, I ended up with the 0.10.1 version which didn’t work with all the commands in this book. Once I got the current version, everything worked better. I should note that I was unable to get the example on page 112 to pass nosetests. I have no idea what the problem is, although it has to do with the Python Mocker module. It may be that I mistyped something, but I couldn’t figure it out.

The author covers Nose’s configuration file briefly and goes over a few command line flags for nosetests. He also covers fixtures and how to use them to enhance your tests. Finally he shows how to use Nose’s pattern matching abilities to run tests that match a certain naming convention. Pretty cool stuff!

I started to skim in earnest after this chapter though. In seven, he does a step-by-step walk-through depicting how to test and create a personal scheduling program. I look forward to reading this section and seeing how complex or simple this is. There is certainly code aplenty in this section and lots of tests (or he may just be re-printing some of the examples in smaller parts for commentary). I have no idea if any of it is functional though.

In chapter 8, the book moves on to Web Testing with Twill. I thought this was an interesting choice since Twill’s website says that it hasn’t been updated in over 3 years. I tested a few of the examples in this chapter and they worked as advertised. Much of this section is made up of explanations of Twill commands though. The last section purportedly shows how to integrate Twill tests with unittest tests. I didn’t do that part though.

Chapter 9 talks about Integration and System testing. I thought the idea of drawing concentric circles in a diagram to figure out how to create integrated units for testing was a good concept. That will probably appeal to the readers who like visuals. Alas, I didn’t really read anything more than the first couple of pages, but it looks like he uses his schedule planner example here as well.

The final chapter entitled “Other Testing Tools and Techniques”. I think this chapter looks fairly interesting, but didn’t get a chance to read it. Looking at the text now, it would seem to briefly cover the following: coverage.py, version control hooks (Bazaar, Mercurial, Darcs, Subversion), and buildbot.

There is also an appendix with Pop Quiz answers.

Overall, I think the chapters that I read had a lot of good information and that is was very well organized. Unfortunately, the good was pretty badly marred by minor grammatical or sentence structure issues and the occasionally larger issue of broken code/tests. If it happens once, then I expect that there probably other code or tests that are broken as well. If this is the case, then I recommend buying it used or wait for a Packt sale of some sort. On the other hand, if the book is pristine after chapter 6…well, I guess that really doesn’t matter since that would mean that over half the book had problems.

Okay. Here’s a better recommendation: Find a Borders or Barnes & Nobles near you that has the book and free wi-fi. Go there, find the book, and read it and mess around with the examples BEFORE you buy it.

Rating Scale: 1 (low) – 5 (high)

Code rating = 3
Writing = 4
Organization = 5

Average Score = 4 (take a point or two off if broken code annoys you)