Content Negotiation on Mobile considered harmful

This is kind of a follow up to my previous post on using Amazon CloudFront.

This post is to cover conneg, or Content Negotiation.

TL;DR - use HTTP as designed and follow the rules.

Alice tries to access http://example.com/ on her iPhone. She gets back some HTML which references some images.




The markup returned to Alice contains references to 3 images. I'll just look at the first one.



Alice's iPhone made a request for /images/1 and got back a 320x80px PNG image, since we have a clever server-side component which knows about different user-agents and tries to serve the most suitable version of an image for each client.

Along comes Bob. Bob is using a Google Nexus One. He similarly requests our home page and gets back a link to /images/1. When the Nexus One requests that resource though, it gets back a 420x120px PNG, again thanks to our fancy server-side detection.

What does this do to our caching? Well, it stuffs it up completely.

1. We're using a canonical URI for the image - /images/1.
2. We're serving different representations of the image from the same URL.
3. We cannot easily specify good HTTP expiration directives.

Going into point 3 in more detail, we cannot use the Vary header in our response to try to let proxy caches more efficiently. Vary can only specify a request header. This means that something like the User-Agent doesn't work:

1. There are many thousand User-Agent strings in existence.
2. How different are these 2 anyway?
   • Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0_1 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko)
   • Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0_1 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Mobile/7A400

Let's be conservative and say that there are 5000 different User-Agents hitting our site. That means that we could be telling proxy servers to cache 5000 copies of /images/1, rather than the perhaps 7 different sizes that our application might produce.

So alternatives to conneg?

• Use a distinct URI for every bag of bytes that the application can serve. This means that your server-side markup generation needs to be a little smarter, so that you render markup containing /images/1/320x80 and /images/1/480x120 for example (see the point in my CloudFront post about not wanting to use query string parameters for this information).
  
• It's still possible to support the old canonical URLs, either by continuing to perform conneg, or redirecting appropriately.
  
• Rev your URIs so that you can happily set Far Future Expires directives on these resources; i.e. if the image changes, give it a new URI.

I'm still musing on what this means for progressive enhancement. Andrea's post looks like a step in the right direction though.