Unit tests for Google App Engine apps

I recently discovered a problem in my unit tests: I did not have any! Now don't worry -- I'm not climbing on my little soap box now to preach the importance of unit tests. There is enough documentation on the web, I do not need inflate this little article with all the same reasons over and over again. Besides, I would be way too busy wiping egg off my face...

I was a little surprised at how much App Engine code I had written without a single test though. I usually am a firm believer, and my test coverage at work is usually quite decent. Granted, I am mostly a Java guy, but being able to hit F11 in Eclipse to run my tests is no excuse! I went ahead to correct that mistake and ran into a couple of obstacles, like how to simulate the database or the users API, and how to verify the output of my web handlers without parsing html. I figured it out eventually, so I thought I'd share my results.

Since what I'm coding at home is nobody's beeswax, I decided to unit test the guestbook app from the getting started guide instead. By the way, you can download the app and my unit-tests from this link. The archive (in unittest.py) contains a total of four unit tests (two for each handler). I will now take you through one of them; the others follow pretty much the same pattern.

Initial setup

Python comes out of the box with a unittest module, so I did not have to bother making a big decision here. Unfortunately, there were quite a few App Engine specific modules (such as the datastore or users) that had to either be configured or replaced for my tests to work. I decided to use a mocking framework and stub out most google-specific code instead of setting up and initializing a fake datastore, user objects and so on. The following code adjusts the python path and imports the necessary modules:

import unittest
import sys
import os.path  


# Change the following line to reflect wherever your
# app engine installation and the mocker library are
APPENGINE_PATH = '../google_appengine'
MOCKER_PATH = '../mocker-0.10.1'


# Add app-engine related libraries to your path
paths = [
    APPENGINE_PATH,
    os.path.join(APPENGINE_PATH, 'lib', 'django'),
    os.path.join(APPENGINE_PATH, 'lib', 'webob'),
    os.path.join(APPENGINE_PATH, 'lib', 'yaml', 'lib'),
    MOCKER_PATH,
]
for path in paths:
  if not os.path.exists(path): 
    raise 'Path does not exist: %s' % path
sys.path = paths + sys.path
import mocker

Defining and running a test

We just set up our program with all the necessary components for a unit test.
So far so good -- now we are ready to define out test. I am subclassing from MockerTestCase, which provides me with a predefined reference to "self.mocker" (a factory object that controls my fake objects and makes sure that everything gets cleaned up at the end of the test). I create instances of both handlers I am about to test, but the request and response objects I populate them with are mocks. If they call a method that I do not expect, my test will automatically fail. I also replace the users- and templates module (why validating the generated html when I might as well just check the parameters?) and the Greetings datamodel:

import helloworld

class UnitTests(mocker.MockerTestCase):
  """Unit tests for the MainPage and GuestBook handler."""

  def setUp(self):
    self.request = self.mocker.mock()
    self.response = self.mocker.mock()
    self.handler1 = helloworld.MainPage()
    self.handler2 = helloworld.Guestbook()
    self.handler1.request = self.request
    self.handler1.response = self.response
    self.handler2.request = self.request
    self.handler2.response = self.response
    self.Greeting = self.mocker.replace('helloworld.Greeting')
    self.users = self.mocker.replace('google.appengine.api.users')
    self.template = self.mocker.replace('google.appengine.ext.webapp.template')

Now, everything is ready to write the first test. This test validates the behavior of MainPage.get() when a user is logged in (if you are not familiar with that method, check it out here). The easiest way of doing that is to record in our mocking framework each and every call that the application is going to make for our specific test case. The following code simulates a query to the datastore that orders the data and fetches the first 10 results. Note that we do not really return any model objects -- they are not relevant for this particular test. A simple string like Query result for template will do.

  def testGetWhenLoggedIn(self):
    # What should happen in a regular request?
    # First, we create a query on all greetings
    all_query = self.mocker.mock()
    self.Greeting.all()
    self.mocker.result(all_query)
    ordered_query = self.mocker.mock()
    all_query.order('-date')
    self.mocker.result(ordered_query)
    ordered_query.fetch(10)
    self.mocker.result('Query result for template')

We follow exactly the same pattern when the application requests the current user (aren't dynamic languages great?). In this example, we return a mock object instead of a random string. Technically, this is not necessary (or code does not call any methods on the user object), but it demonstrates how we could return a fake object instead if we needed to record more in-depth test behaviour.

    fake_user = self.mocker.mock()
    self.users.get_current_user()
    self.mocker.result(fake_user)
    self.request.uri
    self.mocker.result('fake uri')
    self.users.create_logout_url('fake uri')
    self.mocker.result('fake logout uri')

Now that our handler has loaded all the data, it would usually render a Django template. The following code demonstrates how to test this. Watch out for the checkArgs-function, which is used to verify our expectations regarding our template parameters. At the end, we return a small fake html-page and tell our mocker to verify that this is put into our fake output stream:

    out = self.mocker.mock()
    self.response.out
    self.mocker.result(out)
    self.template.render(mocker.ANY, mocker.ANY)
    def checkArgs(path, params):
      template_values = {
        'greetings': 'Query result for template',
        'url': 'fake logout uri',
        'url_linktext': 'Logout',
        }
      self.assert_(path.endswith('index.html'))
      self.assertEqual(template_values, params)
      return '<html/>'
    self.mocker.call(checkArgs)
    out.write('<html/>')

We have now modelled our expectations: assumed that we are logged into a user, what do we expect our handler to do? Now, all that remains is to switch into replay mode and exercise the function. If the handler behaves different in any unexpected way, our unit test will fail. We also call a prefabricated main method that will detect and run all of our tests.

    # Everything is recorded, so let's go into replay mode :-)
    self.mocker.replay()
    self.handler1.get()

# #####################
# LAUNCH ALL UNIT TESTS
# #####################
if __name__ == "__main__":
  unittest.main()

So far so good?

When you download the full set of unit tests and take a look at one of the later tests, you will find a section where I left a TODO. This is an area where (instead of rendering a Django template) a browser redirect happens. My original code
for that test looked something like this:

    # Last but not least, store the post and redirect
    # TODO: refactor test?
    greeting.put()
    self.handler2.redirect = lambda x: self.assertEquals('/', x)

Again, I forgot my soap box at home, but this is an example where (at least in Java) I would have preferred composition over inheritance. I replace the redirect-method in my handler with a lambda that validates the redirect-url. However, I have no guarantee that this redirect URL is ever called.
I can improve this by having the redirect invoking a mock object:

    # Last but not least, store the post and redirect
    # TODO: refactor test?
    greeting.put()
    mock_redirect = self.mocker.mock()
    self.handler2.redirect = lambda x: mock_redirect.redirect(x)
    mock_redirect.redirect('/')

While this is definitely an improvement, it is still far from ideal. What if somebody changes the handler to additionally call the error()-method? That behaviour would get lost! Even worse, what if someone decides to overwrite the redirect-method in the handler's implementation? In that case, my test might even return false positives, since something in that new redirect method could have been broken.

I am not sure what is the ideal solution. For the moment, I have decided to call the post-method itself on a mock object, passing the mock as the "self"-reference to the Guestbook handler:

  def testPostWhenLoggedOut(self):
    handler3 = self.mocker.mock(helloworld.Guestbook)
    handler3.request
    self.mocker.count(0,10)
    self.mocker.result(self.request)
    handler3.response
    self.mocker.count(0,10)
    self.mocker.result(self.response)
    
    # First, a new greeting is created
    greeting = self.mocker.mock()
    self.Greeting()
    self.mocker.result(greeting)
    
    # The user is None and thus not assigned to the greeting
    self.users.get_current_user()
    self.mocker.result(None)
    
    # Next, the content is fetched from a request parameter
    self.request.get('content')
    self.mocker.result('mock content')
    greeting.content = 'mock content'
    
    # Last but not least, store the post and redirect
    # TODO: (check blog post for more information)
    greeting.put()
    mock_redirect = self.mocker.mock()
    handler3.redirect('/')
    
    # Everything is recorded, so let's go into replay mode :-)
    self.mocker.replay()
    helloworld.Guestbook.post(handler3)

This has the adavantage that the test fails if anything unexpected (like self.error()) is being called. For more complex handlers, however, it also would mean that one would need to record mock behavior for all potential helper methods that the class is actually expected to call.

Is there a better solution? Please post a comment to this article...

Update on 2009/02/01: There are quite a few good other articles on unit tests out there by now :-). If you liked this article, you might also want to check out this one: http://domderrien.blogspot.com/2009/01/automatic-testing-of-gae-applications.html.