Facebook Canvas applications

To understand why PLT is a bad idea for Canvas applications,  we must go back and understand the Canvas architecture. Users interact with Canvas applications by using a browser connected to the Facebook server which acts as a middleman and requests data from the app. The app may call APIs that interact with the Facebook server, and then ultimately return an FBML file to Facebook which renders it into HTML for return to the users bowser. The problem is that if we only look at PLT by measuring response time at the browser, we have all of this extra stuff being counted in the timing:

• Network latency between the browser and Facebook
• Time required by Facebook to figure out which application must be called, authenticating the request, etc.
• Time to render the FBML into HTML
• Time spent at the Facebook server servicing API calls

These “extraneous” functions will take an unpredictable amount of time depending on, among other things, the current load at the Facebook server. This will make it hard, if not impossible, to positively correlate the timings of two different test runs. Since one of the prime motivators of load testing is to see the effects of changes to the code on performance, metrics like these will ultimatly prove to be of little use.

Facebook Connect applications

Using PLT for Facebook Connect applications is a little bit better. Users interact with Facebook Connect applications directly, and do not need the Facebook server to act as the middleman. The only extraneous functionality being counted in the timing is the time spent at the Facebook Server responding to API calls. If there are many API calls being made, this will cause a bit of randomness in the metrics.

Application Response Times

In contrast to PLT, creating metrics based on the Application Response Time is much more meaningful and useful, although harder to calculate. Application Response Time should calculate the amount of time spent at the application and only the time spent at the application. Unfortunately, in contrast to PLT there’s no simple way to get the Application Response Time – you’ll need to do a little bit of work to get at it.

To calculate this, you’ll need to create a stopwatch function which starts when your application is called, and stops when the application completes the request and returns an FBML file to the caller. In between, the stopwatch needs to be paused when a Facebook API is called. Finally, those of you using PHP as your development platform should check out the microtime() function as it’s much more accurate than the standard time() function. If your application is able to keep up with all of the bookkeeping, you’ll have metrics that can accurately show you the effects on application performance from any changes you make.

For those of you who don’t want to be bothered with keeping track of all of these timing calculations, please note that FriendRunner does all of this for you automatically.

The Graph API.

Very very cool. You need to read up on this, it changes the way that Facebook development happens from here on in. You no longer need to read a whole PHP API. You don’t even need PHP! Everything is based around RESTful APIs, so your favorite language + curl (or just use Java where everything is already included!) is all you’ll need to be a Facebook developer.

The old API will still work, as will FBML, FQL, etc. However any new development should be done in the new style API, it will be much easier.

Check it out – you’ll like it.

What’s an Application Secret for?

The Application Secret has two main purposes:

• Mutual authentication between the Facebook servers and your application
• Ensuring the integrity of the data passed between them

These benefits look a lot like the benefits of using SSL, although SSL adds the additional benefit of data encryption. And yes, SSL is typically authenticated only one way (server to client), but client-side certificates allow that authentication to be mutual. If you’re a little shaky remembering what SSL is all about, you can refresh yourself here.

So why did Facebook engineers choose not to simply use SSL to communicate with applications? Why did they instead create a rather baroque system that relies on Application Secrets?

Why is authentication important?

Clearly, ensuring that any passed data has provable fidelity is very important. But why do we authenticate? Who would try to impersonate the Facebook servers or an application? Let’s take a look at the reasons to authenticate in each direction, and we’ll see why it’s so important.

Authenticating the Facebook Server to the Application

There’s very little value for a hacker to try to convince your application that his server is a part of the authentic Facebook infrastructure. In fact, the only reasonable attack is a Denial-of-Service attack to try to shut your application down (from perhaps a competitive application). If a hacker  who learns the URL of your Canvas application (i.e. the Canvas Callback URL) pretends to be the Facebook server, he could launch loads of requests to your application until it becomes saturated and crashes. Of course, your app can detect this because only the real Facebook server can authenticate itself to your application with your Application Secret.

Aside from this, it’s hard to imagine why anyone would launch an attack pretending to be a Facebook server. Perhaps hackers are more creative and will come up with reasons to do this, but they won’t get past the authentication stage.

Authenticating the Application to Facebook

In contrast, there are many reasons  that a hacker can try to impersonate a popular Facebook application. Without authentication, the hacker can easily run a man-in-the-middle attack by positioning his application between the Facebook servers and the authentic application. Then the hacker can alter any data that the application creates that is destined for the application user’s browser. If the attack and data is malicious, this can damage the user’s computer, as well as the application developer’s reputation if the modified data displeases the application user. Of course, since all the mischief goes on behind the scenes, disconnected from the user’s browser, there’s no simple way for him to know that anything has gone wrong.

Additionally, without authentication, a masquarading application could asynchronously query the Facebook server to get all of the private information that a user has shared with the application. The user opts in to share this information with a known and trusted application – he doesn’t expect it to be available to some hacker who gets it by devious means.

So keeping your Application Secret secret is pretty important. Important enough that Facebook forbids you to share it (Section 3.7 of the Developer Policies). In our next installment, we’ll discuss the dangers of Application Secrets, and how hackers can attempt to get yours.

Hello Goodbye

Today marks the deprecation of the Application Verification Program  (AVP) as announced during the big Platform Roadmap announcement back in October. Why is Facebook doing this, and what does this mean to developers and testers. Let’s have a look:

The AVP was announced a year ago as a way for developers to differentiate their apps. Verification was supposed to show that the apps met a set of criteria to reassure users that the apps provided a good user experience. The theory was that users would flock to these verified apps, Facebook would feel comfortable that these apps provided a consistent user experience, and developers would be rewarded for playing according to the rules with things like prominent placement in the app directory and an increase on notification allocations. So why was this program axed less than a year after announcement? The truth is, it didn’t work as well as planned. When the first wave of verified apps became available in May 2009, bugs prevented some of the promised developer benefits from actually working (e.g. the special green checkmark to denote a verified app did not always show up). Developers also weren’t thrilled with the $375 price that Facebook charged to verify their app, something that really hurt small developers. And finally, did any users really care whether the apps they were using were verified or not, and moreover, did they recognize when they were?

And so today, this program is being put to rest. But that doesn’t mean Facebook is giving up on consistent user experience. They’ve simply taken the criteria from the AVP, strengthened them, and reinvented them as the Developer Principles and Policies . This is not a voluntary program as the AVP was, everyone is supposed to live by these rules. Facebook is reserving the right to censure or cut off developers who don’t. We don’t know the circumstances under which Facebook will do an audit of an application, but one thing is for sure – it won’t be so easy for them.

Understanding why auditing for consistent user experience isn’t easy gets to the central issue that Facebook is trying to solve. Consistent user experience is important. Those of us old enough to remember that awkward time between command line DOS and the advent of Windows will remember the many applications that provided quasi-GUI interfaces on top of a DOS base. None of these applications looked or felt like any other, which made the process of learning to use one long and tedious. Say what you will about Microsoft, but they did users a great service by providing a Framework for consistency with Windows (okay, borrowed from Apple, borrowed from Xerox). Contrast this with my 8-year old son’s Super Mario games which he plays expertly on 3 different platforms with different sets of controls. Not only does he not read a manual, but mocks the idea that a manual even exists. This is the epitome of consistent user experience that Facebook would like all Platform apps to have.

The trouble is, consistency for Facebook apps is elusive. After all, they’re really just web apps, and developers can do pretty much anything they like (a la the pre-Windows GUI days). To compound problems, an application audited today may change how it works tomorrow. Even if the application code doesn’t explicitly change, any changes to its software stack (e.g. the libraries it uses, the application server, the operating system) may alter the way it works.

Developer Principles and Policies: How to stay on Facebook’s good side

Facebook breaks down proper application behavior into two Principles: Be trustworthy, and Create a great user experience. These principles are expanded into ten policies that Facebook says it will enforce. Upon reading the policies, you’ll find that most of them fall into the Be trustworthy category, or as I like to think about it, the “Don’t be a jerk category”. Most of these rules are simply an extension of civilized behavior for any application: Don’t misuse people’s information. Don’t SPAM the network. Don’t engage in illegal activity. Respect copyright. … and lots more. Facebook unfortunately has to list all of these things since when it shuts someone down who engages in one, they’ll need to point to the policy that was violated. Be trustworthy is a necessary evil that Facebook must delineate, and spend lots of time, effort, and money to police. After all, hate speech is really subjective, and cannot be determined through automated processes. It requires real people to make that determination.

On the other hand, Create a great user experience is a much more positive set of policies, mainly focused on Application Integration Points and Application Responses to User Actions. Whereas Windows codified consistent behavior like where the “File” menu was (and what the typical submenus should be) these policies outline the types of things a user should expect (or better yet, not expect) when using your application. Reading the list, one can remember various apps that used some of the techniques outlined before they were outlawed, and the user backlash that they caused. Some of the policies are:

• You must not prompt users to send invitations, requests, generate notifications, or use other Facebook communication channels immediately after a user allows access or returns to your application.

Invitations or notifications should really only be used once you use and like an application. This is a good policy since it spares your friends the application SPAM that was once common. This also signals to users who receive such invitations that their friends who sent it probably really do like the application and there may be some merit in trying it out themselves.

• You must only use one Facebook communication channel in response to a user’s single action.

When a user clicks on some choice that will generate some action, that user should not be surprised when additional actions are also generated.

• You must not prompt users to bookmark your application (e.g., by using a modal window or pop-up dialog). Instead, users must explicitly invoke any bookmark option you provide.

Nothing is worse than a modal dialog that forces you to take some action that you don’t necessarily agree with, but you must do because hey, it’s modal. Sure, you can go back and delete it, but this policy puts to rest this kind of nonsense that some applications used to pull.

• You must use discretion when publishing Stream stories and must not misuse the Stream by publishing an excessive amount of stories on a user’s behalf.

Forcing your friends to confront too much information about you and some application isn’t nice. Having an application use some discretion is good.

• Stream stories must be consistent with our design and be user-focused based on the user’s action that triggered the story. In no case should a Stream story serve primarily as a means to promote or advertise your application.

Time was when any sort of notification was simply an advertisement to your friends to come and check out this cool application. Well, maybe things stayed the same, but don’t be so blatant about it.

Going through the entire list, we begin to notice a common theme. Application consistency in the Facebook world is not about where the “File” menu is, it’s about the behavior of the application when dealing with the generation of privately and publically available information. But what it’s REALLY about is trying to ensure that you don’t look stupid in front of your friends (probably one of the greatest fears out there). It’s about allaying your fears to try out an application, lest that application begins to SPAM your friends who will get annoyed at you. Once we can all try out any application without worrying that bad things will happen, we all (especially Facebook) win.

One other interesting aspect of application consistency is that many of the rules of this principle can be tested in an automated way. Does Facebook have a way to do this automatically, it would sure save them a lot of legwork. How about your development organization? Would an automated tool to test compliance with Developer Principles be of use to you? Leave a comment and let us know!

The simple answer: It protects you, the developer

Think for a moment what would happen if Facebook did not prohibit the use of automated tools to access their servers. Well for one, we’d be able to easily carry out load tests of our applications. This is good and builds value for your application as well as indirectly building value for the entire Facebook Platform.

However, it also opens up a world of possibilities to the vast armies of hackers and spammers who’d use this to their advantage. What could they do? Let’s think:

• They could launch Denial-of-Service attacks and crash your servers. Does your application have competition? Would your competition benefit if your (obviously superior) app was seen as unstable? Let the attacks begin!
• Mischief. Facebook is now viewed as a prime target for all sorts of miscreants. Hardly a week goes by that we don’t hear about another type of attack launched at Facebook users, applications, or Facebook itself. Of course the methods by which most attacks are typically launched is by the use of automation.
• Building up your own numbers. Actually not really attack – you could use a botnet to artificially inflate your own active user numbers. MAUs are used for a variety of reasons (which all ultimately relate to money). If MAU counts also included MAARs (Monthly Active Automated Robots), then all counts become meaningless. the entire system unwinds, and everyone loses.

So let’s say that as a good developer you’d like to protect your application against a Denial-of-Service attack. You install an Intrusion Detection System, but wait - this won’t work in a Facebook environment! Remember that the Facebook Platform architecture is built so that all of the user traffic coming into your application is funneled through one place – the Facebook servers. Any attempts you make to try to distinguish good access from bad access will be foiled by the fact that the bad guys are on the other side of the Facebook servers from you. Your app has no way to distinguish good users from botnet users, it must rely on Facebook to do it for you. This simply pushes the detection of good vs. bad access out to the Facebook servers, so we can ask: “How can Facebook detect a DOS attack on my app?”

Facebook avoids Denial-of-Service attacks by using the same techniques as any other web site. This, of course, will preclude you from running an automated load test since it will simply look like a very basic (not even distributed) Denial-of-Service attack. Instead of trying to distinguish the good load tests from the bad attacks, Facebook simply says “Do not access our servers with automated tools or we’ll cut you off”. They aren’t trying to stifle our ability to get things done, but rather, they’re protecting us from a whole lot of other pain.

In the next part of this article, we’ll discuss some ideas that can be used to load test your Facebook app.

Calvin's dad on load testing

Load testing Facebook applications is really important. Usually the whole point of deploying onto Facebook is to leverage the social graph and go viral. Developers want their applications to go viral, but there’s a nagging concern about what will happen if they do. Is the app as scalable as you think? Are the hosting choices correct? What’s the cost of deploying and crashing?

The way to alleviate these concerns is to load test your app, but how do you do that in a Facebook environment? The simple answer is that it’s very hard due to two issues central to Facebook:

Issue1: You must control enough Facebook users

Testing your application with one or two simultaneous users is easy: simply use your own Facebook account and call up some friends and have them do the same. But running a test with hundreds or even thousands of simultaneous users is much more difficult. Where do you get the users from? That answer isn’t so simple. You can say “I’ll just go into Facebook and define a few hundred ‘phony’ people that I can control”. While this may seem like a good idea, it isn’t:

• It violates the Facebook Terms of Use (section 4.1) which prohibits you from creating phony accounts. When Facebook finds your phony account, they will delete it.
• Real users have friends, become fans of pages, post updates, and hundreds of other things that Facebook allows you to do. This creates the richness of the social graph, and applications often need to take these things into account. Unless you’re willing to spend an inordinate amount of time “humanizing” your phony user army, your tests with them will not be realistic.
• While Facebook does provide the concept of a Test Account to developers to test with, these accounts are very limited in what they can do so are not of very much use for load testing.

Issue 2: Load testing must be automated

Load testing web applications is hard. Traditionally, to test an application you’d deploy it into a “staging area” and then simulate many people using it with a tool such as HP’s LoadRunner or a cloud-based solution such as BrowserMob.

Load testing by it’s very nature must be an automated process. But this is a problem since Facebook applications must be accessed through the Facebook servers (for API support, FBML-to-HTML rendering, etc.)

Load testing the traditional way

This is a problem because it violates Facebook’s Terms of Use (section 3.2) which prohibits the use of accessing the Facebook servers with automated tools. This will force us to rethink our Facebook application load testing strategy.

In Part 2 of article, we’ll discuss why Facebook has a prohibition against automated tools, and why that is ultimately good for you, the developer.

You may be wondering… since Facebook apps are Web apps, why do I need a special place to discuss testing them? Can’t I apply the same tools and techniques for developing and testing web apps to my Facebook apps?

Well, as with many things in life, the answer is yes and no. Sure you can use PHP and your favorite tools to build your app. However, because of the uniqueness of the Facebook Platform, there are special considerations that you need to think about when testing a Facebook app. Perhaps you’ve already encountered some of these special issues:

• How can I perform load testing on my app without defining hundreds, if not thousands of phony Facebook users, and how do I control them all? (…and BTW, defining phony users violates the Facebook Terms of Use Section 4.1)
• If I’m doing performance testing, how do I factor out the time spent at the Facebook servers during API calls and FBML-to-HTML rendering?
• Are there special security issues for social media applications, and how do I test for these?
• How do I tell if if my application is usable?
• How do I test my application when our development schedule is already behind?
• Are there any special issues when testing applications written for Facebook Connect?
• How do I do proper load testing when Facebook doesn’t allow me to hook up automated test tools? (Again, check the Facebook Terms of Use Section 3.2)
• Now that Facebook will proactively apply their Developer Principles, how can I be sure that my app doesn’t violate them?
• How do I test in an environment where the Platform is constantly changing?
• How can I simulate some of the error conditions that Facebook returns when I call API functions so I’ll know my app is robust?

We plan to discuss these, and many other similar topics here. But mostly, we want this to be a resource for you. If you have a question or topic that you’d like discussed, or want to be a part of our ongoing knowledge base of Facebook testing, please let us know at admin@testfacebook.com