May 8, 2007
@ 08:29 PM
Comments [5]
REST, Web Platforms and Versioning

In recent months it has become clear to me that the largest Web companies have become quite serious about providing "web platforms"that Web developers can integrate with or build upon. Amazon (http://aws.amazon.com), Google (http://code.google.com), Yahoo (http://developer.yahoo.com) and Microsoft (http://dev.live.com) all provide online destinations where Web developers can learn how to use their Web services or integrate with their AJAX widget platforms. The problem with building a platform is that providing Web services and writing APIs is the easy part. The hard part is hard work of supporting developers who have built on your platform with proper documentation, prompt tech support, and ensuring backwards compatibility when new revisions of the platform are delivered. Today I feel like riffing on that last point; versioning and backwards compatibility.

Maintaining backwards compatibility and being mindful of forwards compatibility is one of the biggest design problems that faces platform vendors. So far, there have been numerous examples of Web companies that provide APIs failing to realize how important stability and backwards compatibility when providing a platform. I've seen numerous examples of breaking changes in APIs providing by Web companies including the Google Adwords API (shutting down old endpoints with new version), TypePad Atom feeds (switching the feed format to a new incompatible version thus breaking feed readers), del.icio.us API (changing the Web service URL and the security protocol) and Amazon Web Services (changing what is added to a shopping cart when an item is added via its Amazon specific ID) among many others. Some of these changes were necessary due to security concerns while others could have been prevented with better up front design and decision making by the platform vendor. However in all of these cases, developers that had taken a bet on the platform had to scramble to check that their applications still worked and patch/redistribute them if they didn't. That is bad. One of the primary goals of a platform vendor is to ensure that the lives of developers on their platform is as easy and smooth as possible. This means releasing a new version of the platform should not cause developers who have taken a bet on the platform to have to rewrite their code.  

In the past I've written lengthy, and fairly complex articles about versioning and XML data formats. These days, I've boiled down my guidance on Web service versioning to one phrase; Cool URIs Don't Change. This simple philosophy is actually all you need to consider when designing your RESTful Web service. You apply it in the following way

  1. The Web service end point (i.e. URL) should not change. Changing the actual service end point is bad (e.g. http://del.icio.us/api/posts/get to https://api.del.icio.us/v1/posts/get) instead you should design your URI space with the knowledge that there will be multiple versions of the protocol.
  2. The types of messages returned by the end point should not change. Thus changing the data format of the service (e.g. changing from RSS to Atom or Atom 0.3 to Atom 1.0) is a bad idea. Thus you should have a strategy for supporting multiple formats and partition your URI space accordingly.
  3. The behavior of the end point should not change. This is the trickiest one to follow but the one that your developers will appreciate the most. New versions of the API should not change the behavior of operations previously defined at that end point. One way to reach this ideal is to partition your URI space in a versioned manner (e.g. http://www.example.com/api/v1/get-stuff in version 1 and  http://www.example.com/api/v2/get-stuff in version 2) and then never change the behavior of the old bits or at the very least keep that behavior around as long as possible.

That's it. That's all you really need to know about versioning and REST Web services. The rest is mainly finicky details that are primarily dependent on the specifics of the platform  and the ease with which the developers on the platform can respond to breaking changes. The harder it is for them to change, the higher the bar should be for breaking changes. For example, for Windows Live Messenger we have a high bar for breaking changes in the protocol that the server uses to talk to the desktop client because we know for a fact that not only does it take a surprisingly long time for the majority of users to switch to a new version of the client, there are users who actually cannot upgrade for a variety of reasons (e.g. new versions of desktop client doesn't work on their OS, corporate users or students who can't install software on the PC without permission, etc). On the other hand, when it comes to the internal SOAP APIs that are used to communicate between the Contacts platform and Windows Live properties such as Windows Live Spaces, Windows Live Hotmail and so on the bar for breaking changes is slightly lower because (i) they are Web applications and the (ii) set of consumers of the API is limited.

Ensuring that a Web platform has a strategy for handling backwards compatibility issues is just as important in the Web 2.0 world as it is on the desktop. Platform vendors who think otherwise are in for a rude awakening if they want adoption of their Web platform by serious developers.


 

Categories: Web Development | XML Web Services
Tracked by:
http://afroginthevalley.wordpress.com/2007/05/09/dare-obasanjo-aka-carnage4life-... [Pingback]
http://devhawk.net/2007/05/10/Morning+Coffee+76.aspx [Pingback]
http://www.dancres.org/blitzblog/2007/05/14/ch-ch-ch-changes/ [Pingback]
http://utlf9fi.net/02/index.html [Pingback]
http://weujmru.net/activities/sitemap1.html [Pingback]
http://weujmru.net/volunteers/sitemap1.html [Pingback]
http://restablog.dreamhosters.com/lessons/sitemap1.html [Pingback]
http://kivablog.com/sitemap1.html [Pingback]
http://kiva.startlogic.com/sitemap1.html [Pingback]
http://lf0s3on.net/artists/sitemap1.html [Pingback]
http://biggest-hosting10.com/~rocata/furniture/index.html [Pingback]
http://host239.hostmonster.com/~blogford/sitemap2.html [Pingback]
http://gator413.hostgator.com/~digital/shoes/sitemap1.html [Pingback]
http://vahq8px.net/ebay/sitemap1.html [Pingback]
http://fastestblog.net/sitemap1.html [Pingback]
http://hrxc1zr.net/activities/sitemap1.html [Pingback]
http://box432.bluehost.com/~zbloginf/sitemap1.html [Pingback]
http://gator442.hostgator.com/~hockteam/lottery/sitemap1.html [Pingback]
« Upcoming RSS Bandit Refresh | Home | Opting out of the XBox Live<->Windows Li... »
Tuesday, 08 May 2007 21:31:57 (GMT Daylight Time, UTC+01:00)
What I have done in the past is this. Make a base url that returns the url to the latest version of your API. So

http://api.yoursite.com/Service/

Returns

http://api.yoursite.com/Service/1.0/

then any subsequent request goes to that base url + the name of the method

Later, when you change the API to 1.1, the main URL will return http://api.yoursite.com/Service/1.1/
but older clients can still use http://api.yoursite.com/Service/1.0/ as long as you make your backend versionable

What you can do is two different virtual directories under your site or something similar and hope your DB is versioned along with
your data layer :)
Steve Novoselac
Tuesday, 08 May 2007 23:06:58 (GMT Daylight Time, UTC+01:00)
"One of the primary goals of a platform vendor is to ensure that the lives of developers on their platform is as easy and smooth as possible. This means releasing a new version of the platform should not cause developers who have taken a bet on the platform to have to rewrite their code."

While I agree with this in theory, in practice I have seen too many instances where breaking apps was the only way to get developers to update their code to close security holes and/or upgrade to a more efficient way of doing things. "Don't fix what isn't broken" could just as easily be viewed as "People won't fix it unless they have to." Of course, breaking apps because you decide you prefer a different URI scheme is obviously a horrible evil, so this rule can only be invoked when it absolutely has to to ensure a system continues to run smoothly and securely.
M. David Peterson
Wednesday, 09 May 2007 08:18:22 (GMT Daylight Time, UTC+01:00)
@M. David: It strikes me that, for a recent API, having a security problem exposed at the API is a tip-off that the abstraction has been gotten wrong. One would hope that some sort of threat model was developed before an API could even reach public beta and the resulting unconstrained adoption. Still, if you are going to break an app by a necessary behavior or interface change, wouldn't you best use versioning to do it? Then there's the question of how to plug the old one with something that blocks the security problem and fails appropriately. I've speculated more than enough. I suppose a real example would serve better. Got one of those? I don't have any handy (that I know of).

@Dare: Well said. Your three constraints are terrific. I'm going to print them out and put them on the wall.
orcmid
Wednesday, 09 May 2007 20:02:27 (GMT Daylight Time, UTC+01:00)
@Orcmid,

Hmmm... Examples. Now that I think it through, my feelings stem directly from the non-REST (and even more so, non-Web/URI) world. I see your point regarding versioning, and its a fair argument. That said (and binding together the non-Web Services world with the typical application API), I can't help but be cautious in regards to applying a "never say never" approach to the world of REST-based Web Services.

One example could be that of providing a transformation service at the end of a URI end point in which someone might attempt to exploit with an XPath injection DOS attack. If just such a DOS attack were invoked, one would have to decide how to lock down the attack. Shut down the service? Change the allowed values of the query string parameters to exclude a particular set of allowed characters? etc.

In such cases, you might very easily find yourself having to break a legitimate use of the URI endpoint to thwart the DOS attack, and as such, potentially break an application as a result.

Beyond "I would just make sure my application was properly designed to avoid such attacks (easy enough to say, but you can't anticipate every possible attack vector)" how would you respond to just such a situation?
M. David Peterson
Wednesday, 09 May 2007 20:43:25 (GMT Daylight Time, UTC+01:00)
One thing I should point out is that I am not suggesting there isn't a better alternative to breaking the app, and instead I'm simply not sure what that might be. It would seem to me that if one does not already exist, a best patterns and practices guide in regards to building REST-based Web Services which provide the most minimal risk of exploitation without being forced to unplug the machine and call it a day would be a handy guide to ensure that things that may not be immediately obvious can be avoided to ensure the best chance at maintaining a URI end-point that doesn't expose your underbelly to the web hacking masses.

Maybe Dare would be willing to write such a guide? ;-)
M. David Peterson
Comments are closed.