Matt Brown – Journeys of a software engineer

August 1, 2009

Unit testing Google App Engine Task Queues and urlfetch

Filed under: GAE, Google — Tags: , , , — mattbsoftware @ 5:51 am

My app pulls a web page (urlfetch) and uses the Google App-Engine (GAE) taskqueue API, these were hard to emulate in my tests with the mock lib. This post is about how I solved the urlfetch, and taskqueue problem in unit tests.

I am writing an app that parses a web page and analyzes the data. I needed a better/faster way to iterate my test-debug cycles, than actually hitting my data source page.  Additionally I needed predictable content for my tests.  I am getting to know TDD at the day job (props to Saad-Zilla!!).   This app seemed like a great candidate for Unit Testing. I found the  GAE Unit Test framework and had some great results.   That framework runs all the tests and reports the red/green results at or http://localhost:<port>/test.   However I wasn’t so lucky in the mock dept. I tried a standard mock library, but ran into troubles with the mock library-pymock.

This is how I Unit Test taskqueue and urlfetch in my GAE app. I struggled with the pymock library, so I took advantage of Python’s dynamic language feature.  I simply created new classes that mock’d the objects needed for the test and implemented a simple proxy for urlfetch and the taskqueue.

  1. Get the GAE Unit Test framework from Google Code
  2. Download this zip, and lay it on top of the GAE sample project you download in step #1
  3. Check out proxy classes for TaskQueue (
    1. There is small issue in the taskqueue emulator, since it calls the next task in the list immediately , code after the add to the task queue will not run as it would out of the tests (see the code for more detail).
  4. Check out and urlfetch (
  5. Check out the tests in and
  6. Fix up the sample python code, I have only been using Python for a few months. I think the code could be made more python-like.
  7. red-green-refactor all the live long day 😉

PS: I am only 6 months into python, and that is only when away from the day job. I imagine the zipped code could be a little more Python-like. Please suggest better Python practices.

Create a free website or blog at