Using the "finally" block in Python to write robust applications

This is the first post in my series of three on using XMLRPC to run tests remotely in python (such as javascript and selenium tests in web browsers) and get the results. If that doesn't concern you, this post is probably still relevant; I'd just like to cover the groundwork of making code that is stable and repeatable even in the face of [un]expected problems. Luckily for us, python has a wonderful "finally" block which can be used to properly clean up or "finish" regardless of Bad Things. Let's look at an example of a common problem this can solve:

getLock()
doStuff()
releaseLock()

We need exclusive access to a resource, so we get a lock. We do some stuff, and then release the lock. The problem is that if doStuff raises an exception, the lock never gets released, and your application can be in a broken state. You want to release the lock no matter what. So what you should do is:

getLock()
try:
  doStuff()
finally:
  releaseLock()

Now save a SIGKILL, the lock is going to be released. This is pretty basic, but it is impressive how robust the finally block is. You can "return" in the try block or even "sys.exit()" and the code in the finally block will still be executed.

I recently used this with XMLRPC to safely tell the remote machine to clean up if the local script ran into problems or even got a SIGTERM from a keyboard interrupt. Here's a more elaborate example:

proxy = xmlrpclib.ServerProxy(remoteIP)
try:
  result = proxy.RunTests()
  if result is None:
    sys.exit(1)
  else:
    return result
except:
  sys.exit(2)
finally:
  proxy.CloseFirefox()

The remote machine ("proxy") is running some tests in firefox. While it does this it sets a lock so no one else can run the same tests. If something goes wrong, this lock needs to be reset and firefox needs to be closed so they can run again later. If it gets a result, return it. If it doesn't or something goes wrong, we still clean up but now we can exit with an error code. One of the neatest things about this for me was Ctrl+C'ing the script on my computer and watching the remote machine cleanly quit firefox and release the lock for another process to use.

This is great whenever you need to put something in a temporary state, or change the state after an operation no matter what happens. Think of locks, temporary files, memory usage, or open connections where it is important to close them. Conversely however, make sure you DON'T use an approach like when it isn't appropriate.

for client in clientsToPing[:]:
  try:
    ping(client)
  finally:
    clientsToPing.remove(client)

This is potentially incorrect behavior, because if you failed to ping your client you may want to keep it on the list to try again next time. However, you also may only want to attempt this once and then the above approach would be correct!

In my next post I am going to turn more specifically to remote browser testing and explain how exactly to set up both ends of the connection. After that I'll finish by making a post on using twisted + SSL to retrieve posted results over HTTPS.