Monitoring your Facebook application for regression – Part 1

regression-testing A few days ago there was a post in the Facebook Developers Blog entitled Testing Using The Beta Tier, which is important to read and to understand its implications. The gist of the post is that the developers at Facebook are pretty busy and are constantly modifying the Facebook Platform. They push out updates on a weekly basis and all of us in the Facebook Platform ecosystem are affected by this whether we like it or not (and whether we know it or not). Fortunately, as this post explains, you can get access to the beta version of these updates before they go live to ensure that your application plays well with it. This is a valuable service that Facebook is providing, and it’s important to understand why.

Let’s contrast a Facebook app with a plain old Web Application. Imagine you’ve just finished designing and building your great new Web App. Before deployment you test every possible thing that could go wrong and you feel really confident about its stability and robustness. You’re so confident that it won’t break that you decide to relax and spend the next two weeks in Bora Bora without even a laptop. Things may go wrong, but if your app is really well-written, then all of the problems will be operational problems. Someone tripped over a power cord and knocked out a router, or spilled coffee into a server. Things that are certainly problems, but they don’t need you the programmer to fix them. Go get yourself another drink and don’t worry about your app.

But Facebook development isn’t like that. You still need a good operations team to make sure your app stays online, but the stability and reliability of your Facebook app also depends on the invisible hand of Facebook, something that you have no control over. Before booking your flight to Bora Bora you’ll need to come to grips with the possibility that Facebook could very easily push out an update that changes an API in a way that breaks your application. If/when that happens to you, then you the programmer will need to do something about it to get your app back up and running again, even if it means getting up from the beach. And that’s the important point here: Facebook applications can and will experience regression even when they aren’t being changed because Facebook is always changing.

The first step to dealing with a problem like this is knowing that there is a problem. If you have a general Web App, knowing that it isn’t working isn’t so hard to do. You can simply set up a monitoring system to periodically hit the app and make sure that it responds properly. There are many inexpensive commercial services that make this really easy to do.

But what about Facebook applications? You can monitor the application to ensure it is up and running, but that won’t tell you if Facebook has done something to break it. That’s why Facebook’s Beta Tier is so valuable – you can find and fix potential problems before they get seen by users.

But how can we do this automatically? It would be great if we could set up an automated way to know that the next Facebook update will break your application. In our next post we’ll look at some strategies for how we can do this. In the meantime, leave a comment if you think this would be a good idea, and any thoughts you have about it.