One downside of working with experienced people is that, some experienced people tend to be opinionated, and the trouble with opinions is that opinions are, well, instantaneous reactions based on things they learned in the past. You can wake up an opinionated person and ask a question that you know this person has an opinion on, and that person (self included, on occasion) will most likely blurt out that opinion without even understanding the context. Unless careful, it is easy to get trapped into that mindset, and start enjoy being opinionated, and consequently, making or advocating "obvious" choices. Welcome to my soft rant!

Peter Williams recently made a good proposal suggesting content-negotiation as a means to advertise and ask for version support. This is in contrast to using version identifiers in URIs as is commonly done. This is a neat idea. Peter's post prompted some debate and a rebuttal. But I find that this debate is a distraction.

Pundits are redefining architecture yet again.

This is in response to Mike Amundsen's post on hypermedia. I started commenting on his blog, but then realized I have more to say than what I could fit in the comment form.

Mike's questions are, for REST-ful applications that use JSON for representing resources, (a) "where is the hypermedia (links and forms)" and (b) "what standardized clients understand how to render these elements propertly?"

Content negotiation in HTTP 1.1 is not broken. It is insufficient and incomplete, but the way it was defined in RFC 2616 works for what it is intended for. What is broken includes HTML, user agents, and buggy servers. Web services developers should not use the argument that content negotiation is broken and avoid supporting it altogether or support it poorly. I don't know if this was the rationale used by Twitter, but Twitter APIs do not understand content negotiation.

Mike Amundsen is wondering how to express hypermedia in JSON. He rightly identifies that there needs to be a new media type, e.g. application/hypermedia+json. Putting together such a new media type is not hard, but in my view, the real issue is defining a JSON format that can express data/media as well as hypermedia. One possibility is to take the "payload" formats defined by Atom and Atom Publishing Protocol, and translate that into JSON using some simple rules. Here is one, based on example from RFC 5023.

Through Stefan Tilkov's blog, I came across ServiceReg whose goal is to let us "execute RESTful web service calls against Rails applications (and other REST compliant framworks) with a simple URL". The "simple URL" is one that can be submitted via a GET.

When it comes to URI design, I am in the camp that argues that URIs ought to be opaque as far as the representations are concerned. In my view, URIs and representations are two independent axes of web service design, and must be designed to be evolved independently of each other, and that URIs should not be used to infer the structure of representations. Does this mean that URIs should be completely opaque and cryptic, like the following ...

There is a push in some corners to treat URIs as a means to represent hierarchies of objects. In this view, each path segment in the URI either represents either an object or a relationship to another object. For example, if a person object has a firstName, lastName and an address, under this model, the web service would provide the following URIs:

What is the best way to start building RESTful applications? One of the common advice heard is, "nouns are your resources". You walk through your application, identify all the nouns, model them as resources, and try to rely on HTTP verbs to operate on those resources. Use POST to create a resource, PUT to replace (or even create, when the client can name the resource), DELETE to delete the resource, and GET to get the resource. However, the notion that nouns can be used to identify resources is simplistic.

Through del.icio.us/distobj (what a resourceful tag, to find nuggets such as the presentation I am talking about in this pot!), here comes a funny (seriously - not funny at all) presentation on Web Service Architectures by Dennis Sosnoski. This is an old presentation deck, but such views have been presented by several others time and again.

In his Addressing fragments in REST Simon St. Laurent says ...

In response to my post on PATCHing and BATCHing, John Panzer made a very valid comment.

This use case is not unique. I have come across several such use cases, and have seen similar problems being discussed elsewhere in the REST community. My first reaction was that, this use case requires a generic batch server to process these requests on behalf of the client. James Snell made a great start, and let's say, we adopt such a solution for this use case.

Unfortunately this still leaves a number of questions unanswered.

In response to David Peterson's The SimpleDB Epiphany: I Finally GET It... Why RFC 2616 Is To Blame I posted a quick note here, and a few comments on his blog. He asked a few questions, but as I got caught up with my day job, I could not completely elaborate my comment on why idempotency for pipelined requests is a protection and not a limitation. Here is the rest.

Through Mark Baker's SimpleDB did screw up I just found a new rationalization for the un-RESTful nature of Amazon's SimpleDB API by David Peterson at xml.com. For those curious, I have offered a RESTful version here.

David Peterson's rationalization is simple. Since RFC 2616 recommends (a SHOULD) pipelining for idempotent requests, and since Amazon's SimpleDB is "designed from a ground up to support highly parallel applications", SimpleDB ended up being what it is.

Simply hilarious. I hope that it it his interpretation, and not Amazon's.

Here is something very interesting about RESTful batch requests. Whenever the word "batch" is mentioned, I try to probe into the meaning of batch. It turns out it does not always mean the same thing. It could actually mean one of the following three things.

Prompted by Joe Gregorio's How To Do RESTful Partial Updates, there were a lot of discussions over the weekend about partial updates. James Snell just captured the list of reactions in his latest post, and so there is no need to repeat them here. While Joe Gregorio's post is an interesting attempt at encoding the diff into URIs, I would argue that, contrary to what the title of his post suggests, encoding a diff into a URIs is not a generic solution for partial updates. Please note that, while Joe Gregorio's post just focused on Atom, I am extending this to XML in general in this post.

This week there was more activity around supporting batch requests over HTTP. To start with, Bill de hÓra posted a critique on why doing batch processing over PATCH is a stretch. In response James Snell posted an update to his proposal, this time using a new verb BATCH and using a multipart entity body with each part representing an independent HTTP request encoded via the application/http media type. John Panzer of Google posted some of his reactions to this idea. This activity proves that every organization exposing web services over HTTP is facing two common problems:

Over the past few couple of weeks, I came across this question several times. The question is, how can a client submit an arbitrary set of requests to a server in a batch, and get a single response from the server. At first, this seems to be a natural use case to support on the server side. For the client, this would save a lot of work - instead of sending several requests, the client would just send a single request and save on network latency. Late last year, I spent a week prototyping a server to process batch requests. It opened up some really interesting problems to think about. The batch models I have looked have plenty of holes to drive trucks through, particularly when it comes to updating resources.

I just finished reading an article in IEEE Computer titled "Serendipitous Reuse" (pdf here) by Steve Vinoski. There is also some interesting discussion on this article at Steve Vinoski's blog. The central premise of this article is that uniform interfaces such as the those offered by REST encourage reuse, where as specialized interfaces such as those routinely done in various distributed object/interface models such as RPC, CORBA, WSDL etc reduce the chances of reuse. Steve's paper is a good read, and I agree with most of his analysis about why specialized interfaces are complex to build and manage and why reuse does not happen as much as the designers intended. However, the premise that uniform interfaces promote serendipitous reuse is, in my opinion, incomplete. Uniform interfaces do not increase chances of reuse, unless of course, the interface is solving a generic problem, and is adhering to the hypermedia constraint of REST.

The first five days of the new year went by very quickly. I am very excited to say that very soon, I will be launching one of my pet projects publicly. This project is about managing and sharing logs and statistics for recreational cyclists. Over the past few days, I had a chance to learn and play with quite a few things like Ruby on Rails, Garmin browser plugins, YUI, and so on. The project is not quite ready for public consumption but I have most of the key peaces in place and working, and I am planning to share it with a few of my bike buddies over the next week or so to get some feedback. I have another personal goal for this project - that is to test some of my ideas about building RESTful applications. Talking REST is easier than doing REST. I have a few more issues to tackle before I can conclude!

In the midst of all this, my MacBookPro kept crashing over 5-7 times a day. As I installed too many new things recently, it turned out to be difficult to isolate the problem. After a lot of frustration and lost hours, finally, this solution worked. Syslogd was the culprit.

Of the four interface constraints underlying the REST architecture, the constraint that "hypermedia as the engine of application state" is cause for some confusion. As Stu Charlton once noted, it is also the most often ignored constraint in reality. While this constraint does some offer some benefits, I do see practical reasons why this constraint is ignored, particularly for non-userfacing applications, and there are benefits to ignoring it.

It happens again. Over at ebpml.org, JJ Dubray has a couple of related posts about the dark side of REST, and a proof that REST creates strong-coupling. I am not sure what to make of his style of writing, but like good citizen, let me try to make some sense here.

In my view, the answer is simple. We have long been following action-oriented principles to designing distributed software, and action-oriented interfaces, when translated into REST, simply put, suck.

Dimitri Glazkov posted a quick comment on the rest-discuss mailing list stating that the design of Amazon's SimpleDB REST API is just a big welcome to 1999. The technology powering SimpleDB is definitely impressive - no doubt about it. It is a simple-to-use web service to manage structured data in a reliable manner hosted on Amazon's data centers. It does have limitations such as lack of immediate consistency (the "C"" of ACID), and I am sure Amazon will solve such problems eventually. However, as a REST API, it is a disappointment. The API failed (a) to identify resources, and (b) to specify operations on resources in a RESTful way. It uses a single verb GET to create, delete, update, or get data from the store. As Julian Reschke commented, this is worse than SOAPy REST, or using POST for everything as is done with SOAP over HTTP. The SimpleDB API is neither resource oriented nor HTTP friendly.

Having said that, how should such an API be designed in a resource-oriented manner? Here is my take, a version-0.1 of a RESTful SimpleDB.

As I was looking at some design issues around some REST APIs, I had a chance to investigate the HTTP/1.1 Vary header. As it turns out, there are interesting implications around setting or not setting this header or setting this header incorrectly. Here is my notes on the rationale behind the Vary header, and design guidelines.

Is choreography necessary for the enterprise? - this was the question I raised several weeks ago. In response to that post, there were several mixed reactions. Some were positive while some were critical. Over the past week or so, I spent some time reading more background material, particularly a couple of papers by Robert Milner, and on process calculus with the hopes of finding the origins and the rationale behind choreography.

Yesterday I posted a note on No Choreography for REST? asking some questions on the role of web services choreography. Apart from some heated comments I received on this blog, this post was also redditted, where it invited a lot more comments.

Back in the days when SOAP was being worked out, the goal was to build “plumbing that produced and consumed XML-based messages instead of the various binary message formats favored by each technology” (see Tim Ewald's 2002 article on MSDN titled The Argument Against SOAP). In a sense, the goal was to bring in homogeneity across non-homogeneous technologies so that they can talk to each other. Looking back, there is an interesting consequence to this exercice. There are a number of WS-* specifications, some already dead but some alive, that assume that all applications are homogeneously described by WSDL, which IMO, gave rise to esoteric ideas like choreography, orchestration and so on. By its very nature, in general, homogeneity is limiting and hampers creativity. So, is the very motivation behind homogeneity of distributed software via SOAP and WSDL limiting creativity? This thought lead me to do a quick search .

REST does not support Web Services Choreography, and hence REST is unfit for the enterprise - so goes the argument for WS-* over REST. See, for example, the latest one by Jean-Jacques Dubray. But does WS-* need choreography? Does anyone need choreography? Having not cared about web services choreography activities at W3C, I was intrigued about this question, and wanted to find out.

As I was looking at a large number of so-called REST APIs, I have observed one common pattern in some APIs. They are supposed to be RESTful, but smell SOAPy. Here are some examples.

One of the most thought-provoking notes on XML security is Peter Gutmann's piece on Why XML Security is Broken. As I started thinking about security around REST APIs, I happened to stumble upon James Clark's Bytes not infosets which led me to Peter Gutmann's note. This writeup confirmed my experience that web services security is overly complicated and complex to implement and support. It can be made to work, however, and this is where I disagree with Peter Gutmann. His note also explains why web services security is so tightly coupled with the underlying XML processing system (i.e. the web services stack) that choice of using a particular web services stack ties you into using a supported (if there is one) security implementation. This is unlike HTTP-based security where code processing requests and responses is almost always independent of the security layer.

To quote HighScalability on Scaling Twitter,

In other words, expect your users to be as creative as you are (if note more), and let them compete to build interesting applications around your software.

There is a lot of talk about open APIs these days using REST or some such paradigm. Building open APIs for the rest of the world to consume is easy, but attracting a new developer community around the API is hard.

Recently I came across an article at InfoQ that talks about using Using ETags to Reduce Bandwith & Workload with Spring & Hibernate. Unfortunately, the implementation suggested in this article is such that validating the ETag takes as much time (if not more) as generating the resource itself. This is an example of a premature and incorrect optimization. This article barely scratched the surface of the problem with computing caching headers (both expiry related and invalidation related) for dynamic resources.

Recently, there have been a number of blog posts and articles on this topic, and this time around, the REST side is winning, and I find two reasons for this shift. Firstly, HTTP is more ubiquitous than ever. Secondly, WS-* has reached a point where the complexity of specifications and related abstractions overtook the basics. This recent shift reminds me of the history of CORBA, and how it had its glory and then quickly lost it.

Nearly a month afterJSR-311 was announced, I had a chance to read through the proposal, and read others' comments, particularly on the rest-discuss mailing list.

This post is in response to Sameer Tyagi's article on using JAX-WS for RESTful web services posted at the Sun Developer Network. I find a few points stated in this article troubling if not inaccurate - (a) it presents REST in a limited scope thereby showing SOAP in better light, and (b) it misses some key differences between REST and SOAP based web services. Beyond this, the example cited in this article just shows why you SHOULD NOT use JAX-WS for REST style web services. In this post, I would like to discuss the whys behind the two statements I just made.

I just finished a major enhancement to my blog. The idea is show links from the web related to the current entry. To see how this works, just click on the "Similar Entries from Google" link above. This feature uses Yahoo's Term Extraction service to extract keywords from my Movable Type blog entries, and feed these keywords to Google's Search API to fetch related entries. The feature is still experimental, and I need to make a few more refinements.