Matt Brown – Journeys of a software engineer

August 1, 2009

Unit testing Google App Engine Task Queues and urlfetch

Filed under: GAE, Google — Tags: , , , — mattbsoftware @ 5:51 am

My app pulls a web page (urlfetch) and uses the Google App-Engine (GAE) taskqueue API, these were hard to emulate in my tests with the mock lib. This post is about how I solved the urlfetch, and taskqueue problem in unit tests.

I am writing an app that parses a web page and analyzes the data. I needed a better/faster way to iterate my test-debug cycles, than actually hitting my data source page.  Additionally I needed predictable content for my tests.  I am getting to know TDD at the day job (props to Saad-Zilla!!).   This app seemed like a great candidate for Unit Testing. I found the  GAE Unit Test framework and had some great results.   That framework runs all the tests and reports the red/green results at http://yourapp.appspot.com/test or http://localhost:<port>/test.   However I wasn’t so lucky in the mock dept. I tried a standard mock library, but ran into troubles with the mock library-pymock.

This is how I Unit Test taskqueue and urlfetch in my GAE app. I struggled with the pymock library, so I took advantage of Python’s dynamic language feature.  I simply created new classes that mock’d the objects needed for the test and implemented a simple proxy for urlfetch and the taskqueue.

  1. Get the GAE Unit Test framework from Google Code
  2. Download this zip GAEtest-taskqueue-urlfetch.zip, and lay it on top of the GAE sample project you download in step #1
  3. Check out proxy classes for TaskQueue (TaskQueueProxy_module.py).
    1. There is small issue in the taskqueue emulator, since it calls the next task in the list immediately , code after the add to the task queue will not run as it would out of the tests (see the code for more detail).
  4. Check out and urlfetch (UrlFetchProxy_module.py)
  5. Check out the tests in TaskQueueSampleTest.py and UrlFetchExampleTest.py
  6. Fix up the sample python code, I have only been using Python for a few months. I think the code could be made more python-like.
  7. red-green-refactor all the live long day 😉

PS: I am only 6 months into python, and that is only when away from the day job. I imagine the zipped code could be a little more Python-like. Please suggest better Python practices.

Advertisements

1 Comment »

  1. You could also try Fudge which I’d say is a little more straight forward than PyMock http://farmdev.com/projects/fudge/

    Comment by Kumar McMillan — November 13, 2009 @ 4:51 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: