Category Archives: Testing

An article on Python testing and possibly debugging

An Intro to coverage.py

Coverage.py is a 3rd party tool for Python that is used for measuring your code coverage. It was originally created by Ned Batchelder. The term “coverage” in programming circles is typically used to describe the effectiveness of your tests and how much of your code is actually covered by tests. You can use coverage.py with Python 2.6 up to the current version of Python 3 as well as with PyPy.

pip install coverage

Now that we have coverage.py installed, we need some code to use it with. Let’s create a module that we’ll call mymath.py Here’s the code:

def add(a, b):
    return a + b
 
 
def subtract(a, b):
    return a - b
 
 
def multiply(a, b):
    return a * b
 
 
def divide(numerator, denominator):
    return float(numerator) / denominator

Now we need a test. Let’s create one that tests the add function. Let’s give our test the following name: test_mymath.py. Go ahead and save it in the same location as you did for the previous module. Then add the following code to our test:

# test_mymath.py
import mymath
import unittest
 
class TestAdd(unittest.TestCase):
    """
    Test the add function from the mymath library
    """
 
    def test_add_integers(self):
        """
        Test that the addition of two integers returns the correct total
        """
        result = mymath.add(1, 2)
        self.assertEqual(result, 3)
 
    def test_add_floats(self):
        """
        Test that the addition of two floats returns the correct result
        """
        result = mymath.add(10.5, 2)
        self.assertEqual(result, 12.5)
 
    def test_add_strings(self):
        """
        Test the addition of two strings returns the two string as one
        concatenated string
        """
        result = mymath.add('abc', 'def')
        self.assertEqual(result, 'abcdef')
 
 
if __name__ == '__main__':
    unittest.main()

Continue reading An Intro to coverage.py

Python 201: An Intro to mock

The unittest module now includes a mock submodule as of Python 3.3. It will allow you to replace portions of the system that you are testing with mock objects as well as make assertions about how they were used. A mock object is used for simulating system resources that aren’t available in your test environment. In other words, you will find times when you want to test some part of your code in isolation from the rest of it or you will need to test some code in isolation from outside services.

Note that if you have a version of Python prior to Python 3, you can download the Mock library and get the same functionality.

Let’s think about why you might want to use mock. One good example is if your application is tied to some kind of third party service, such as Twitter or Facebook. If your application’s test suite goes out and retweets a bunch of items or “likes” a bunch of posts every time its run, then that is probably undesirable behavior since it will be doing that every time the test is run. Another example might be if you had designed a tool for making updates to your database tables easier. Each time the test runs, it will do some updates on the same records every time and could wipe out valuable data.

Instead of doing any of those things, you can use unittest’s mock. It will allow you to mock and stub out those kinds of side-effects so you don’t have to worry about them. Instead of interacting with the third party resources, you will be running your test against a dummy API that matches those resources. The piece that you care about the most is that your application is calling the functions it’s supposed to. You probably don’t care as much if the API itself actually executes. Of course, there are times when you will want to do an end-to-end test that does actually execute the API, but those tests don’t need mocks!

Continue reading Python 201: An Intro to mock

Python 3 Testing: An Intro to unittest

The unittest module is actually a testing framework that was originally inspired by JUnit. It currently supports test automation, the sharing of setup and shutdown code, aggregating tests into collections and the independence of tests from the reporting framework.

The unittest frameworks supports the following concepts:

  • Test Fixture – A fixture is what is used to setup a test so it can be run and also tears down when the test is finished. For example, you might need to create a temporary database before the test can be run and destroy it once the test finishes.
  • Test Case – The test case is your actual test. It will typically check (or assert) that a specific response comes from a specific set of inputs. The unittest frameworks provides a base class called **TestCase** that you can use to create new test cases.
  • Test Suite – The test suite is a collection of test cases, test suites or both.
  • Test Runner – A runner is what controls or orchestrates the running of the tests or suites. It will also provide the outcome to the user (i.e. did they pass or fail). A runner can use a graphical user interface or be a simple text interface.

Continue reading Python 3 Testing: An Intro to unittest

Python: Create Fake Data with Faker

Every once in a while, I run into a situation where I need dummy data to test my code against. If you need to do tests on a new database or table, you will run into the need for dummy data often. I recently came across an interesting package called Faker. Faker’s sole purpose is to create semi-random fake data. Faker can create fake names, addresses, browser user agents, domains, paragraphs and much more. We will spend a few moments in this article demonstrating some of Faker’s capabilities. Continue reading Python: Create Fake Data with Faker

Python Testing with doctest

Python includes a couple of modules for testing in its standard library: doctest and unittest. We will be looking at doctest in this article. The doctest module will search for pieces of text in your code that resemble interactive Python sessions. It will then execute those sessions to verify that they work exactly as written. This means that if you wrote an example in a docstring that showed the output with a trailing space or tab, then the actual output of the function has to have that trailing whitespace too. Most of the time, the docstring is where you will want to put your tests. The following aspects of doctest will be covered:

  • How to run doctest from the terminal
  • How to use doctest inside a module
  • How to run a doctest from a separate file

Let’s get started! Continue reading Python Testing with doctest

pyflakes – the passive checker of Python programs

There are several code analysis tools for Python. The most well known is pylint. Then there’s pychecker and now we’re moving on to pyflakes. The pyflakes project is a part of something known as the Divmod Project. Pyflakes doesn’t actually execute the code it checks, unlike pychecker. Of course, pylint also doesn’t execute the code. Regardless, we’ll take a quick look at it and see how pyflakes works and if it’s better than the competition. Continue reading pyflakes – the passive checker of Python programs

PyLint: Analyzing Python Code

Python code analysis can be a heavy subject, but it can be very helpful in making your programs better. There are several Python code analyzers that you can use to check your code and see if they conform to standards. pylint is probably the most popular. It’s very configurable, customizable and pluggable too. It also checks your code to see if it conforms to PEP8, the official style guide of Python Core and it looks for programming errors too. We’re going to spend a few minutes looking at some of the things you can do with this handy tool. Continue reading PyLint: Analyzing Python Code

Website Automation with Python, Firefox and Selenium

An appropriate alternate title would be: How to control a web page or test your website with Python. Recently, I was given the following assignment:

1) Login to a website
2) click on a toolbar and load a specific search form
3) enter some data into one of the fields on the form and search
4) if found, click another button
5) repeat

Of course, I was supposed to do all this with Python, my favorite programming language. My first thought was to use the Mechanize package, but while I could login to the website using that, I was unable to get it to load the form. Why? Well, unfortunately the aforementioned toolbar was made using javascript and it seemed to be generating the form too. For whatever reason, the Python port doesn’t support loading javascript elements, although I did find out that the project it’s based on has a javascript plugin, so there is hope that it might eventually. Thus, I went looking for another solution and recalled that Selenium might fit the bill. In the end, it worked quite well. Since I won’t be able to show you what I did exactly because it was for an internal project, we’ll be automating Gmail instead. Let’s get cracking! Continue reading Website Automation with Python, Firefox and Selenium

PyCon 2011: TiP BoF

The venerated TiP BoF (Testing in Python “Birds of a Feather”) meeting was held Saturday (3/12/2011) night around 7 p.m. Disney provided free pizza and salads. Someone else (I think) provided some pop. The room was packed with standing room only in the back. While people were eating, Terry Peppers of Leapfrog led the meeting. He told us how the TiP BoF worked and then had one of his employees show us how to do weird hand/arm stretches. If I remember correctly, his name was Feihung Hsu.

After that, the testing-related lightning talks started. The lightning talks are really the main draw of this event, although in years past the alcohol induced many to come. This year, the hotel cracked down on that and there was hardly any liquor to be seen, which was alright by me. I only stayed for two hours, so I’ll just give a run-down of what I saw and heard:

  • There were lots of masturbation and other crude jokes even before we ate anything and they continued through most of the time I was there
  • Peppers started the talks off with one called Snakes on a domain which was about a nagios plugin called NagAconda
  • Next, Disney awarded Jesse Noller with a Disney beer stein that was themed after their animated movie, “Tangled”.
  • Alfredo Deza gave a talk a DSL-testing framework called Konira
  • Following that was a talk on Cram – a mercurial test suite for command line testing. I missed who gave that one. I think it’s this one: https://bitbucket.org/brodie/cram/src
  • Then there was a talk on Lab Coat. They had the speaker wear a lab coat too. I don’t remember who did this one (maybe the author?) or what this project even does…
  • Roman Lisagor gave a talk on Freshen, a clone of Ruby’s Cucumber project. It’s a plugin for nose and supposed to be similar to the lettuce project.
  • Kumar McMillan gave a talk entitled Fudging it with Mock Objects. Yes, it’s another mock library, but this one is based on some project called Mocha (and I think he said he used stuff from Michael Foord’s mock library as well). You can check it out here: http://farmdev.com/projects/fudge/
  • The next talk was Scientific Testing in Python. My notes are bad on this one, but I think it was related to the Bright project (correct me if I’m wrong). The speaker also mentioned something called goathub.com, but as far as I can tell, that doesn’t really exist.
  • Feihung Hsu made another appearance by giving a talk himself. It has this long title: How My comic Book obsession birthed a new functional tool. Basically it was web-scraping project for downloading Japanese manga that had been translated into Chinese using Python. He forked spynner, made it “dumber” and called his fork “Punky Browster”. I don’t think this project is available yet.

To sign up to give lightning talks, they used a convore thread. The front row was made up of hecklers that would heckle the speakers. They seemed to favor strong swearing for the heckling. It could be pretty funny and very crude. I learned about a lot of new projects I had never heard of though. It’s definitely something that I think is worth checking out at least once.

Python 102: An Intro to TDD and unittest

Python code testing is something new to me. It’s not required where I work, so I haven’t spent much time looking into it, besides reading a book on the subject and reading a few blogs. However, I decided it was high time I check this out and see what all the excitement is about. In this article, you will learn about Test Driven Development (TDD) with Python using Python’s builtin unittest module. This is actually based on my one experience of TDD and pair programming (thanks Matt and Aaron!). In this article, we will be learning how to score bowling with Python! Continue reading Python 102: An Intro to TDD and unittest