June 4, 2007
@ 04:47 PM

Last week there was a bunch of discussion in a number of blogs about whether we need an interface definition language (IDL) for RESTful Web services. There were a lot of good posts on this topic but the posts from Don Box and Bobby Woolf which gave me the most food for thought.

In his post entitled WADL, WSDL, XSD, and the Web Don Box wrote

More interesting fodder on Stefan Tilkov's blog, this time on whether RESTafarians need WSDL-like functionality, potentially in the form of WADL.

Several points come to mind.

First, I'm doubtful that WADL will be substantially better than WSDL given the reliance on XSD to describe XML payloads. Yes, some of the cruft WSDL introduces goes away, but that cruft wasn't where the interop probems were lurking.

I have to concur with Don's analysis about XSD being the main cause of interoperability problems in SOAP/WS-* Web services. In a past life, I was the Program Manager responsible for Microsoft's implementations of the W3C's XML Schema Definition Language (aka XSD). The main problem with the technology is that XML developers wanted two fairly different things from a schema language

  1. A grammar for describing and enforcing the contract between producers and consumers of XML documents so that one could, for example, confirm that an XML document received was a valid purchase order or RSS feed.
  2. A way describe strongly typed data such as database tables or serialized objects as XML documents for use in distributed programming or distributed query operations.

In hindsight, this probably should have been two separate efforts. Instead the W3C XML Schema working group tried to satisfy both sets of consistuencies with a single XML schema language.  The resulting technology ended up being ill suited at both tasks.  The limitations placed on it by having to be a type system made it unable to describe common constructs in XML formats such as being able to have elements show up in any order (e.g. in an RSS feed title, description, pubDate, etc. can appear in any order as children of item) or being able to specify co-occurrence constraints (e.g. in an Atom feed a text construct may have XML content or textual content depending on the value of its type attribute).

As a mechanism for describing serialized objects for use in distributed computing scenarios (aka Web services) it caused several interoperability problems due to the impedance mismatch between W3C XML Schema and object oriented programming constructs. The W3C XML schema language had a number of type system constructs such as simple type facet restriction, anonymous types, structural subtyping, namespace based wildcards, identity constraints, and mixed content which simply do not exist in the typical programming language. This lead to interoperability problems because each SOAP stack had its own idiosyncratic way of mapping the various XSD type system constructs to objects in the target platform's programming language and vice versa. Also no two SOAP stacks supported the same set of XSD features. Even within Microsoft, let alone across the industry. There are several SOAP interoperability horror stories on the Web such as the reports from Nelson Minar on Google's problems using SOAP in posts such as Why SOAP Sucks and his ETech 2005 presentation Building a New Web Service at Google. For a while, the major vendors in the SOAP/WS-* space tried to tackle this problem by forming a WS-I XML Schema Profile working group but I don't think that went anywhere primarily because each vendor supported different subsets of XSD so no one could agree on what features to keep and what to leave out.

To cut a long story short, any technology that takes a dependency on XSD is built on a shaky technological foundation. According to the WADL specification there is no requirement that a particular XML schema language is used so it doesn't have to depend on XSD. However besides XSD, there actually isn't any mainstream technology for describing serialized objects as XML. So one has to be invented. There is a good description of what this schema language should look like in James Clark's post Do we need a new kind of schema language? If anyone can fix this problem, it's James Clark.

Ignoring the fact that 80% of the functionality of WADL currently doesn't exist because we either need to use a broken technology (i.e. XSD) or wait for James Clark to finish inventing Type Expressions for Data Interchange (TEDI). What else is wrong with WADL?

In a post entitled WADL: Declared Interfaces for REST? Bobby Woolf writes

Now, in typing rest, my colleague Patrick Mueller contemplates that he "wants some typing [i.e. contracts] in the REST world" and, among other things, discusses WADL (Web Application Description Language). Sadly, he's already gotten some backlash, which he's responded to in not doing REST. So I (and Richard, and others?) think that the major advantage of WSDL over REST is the declared interface. Now some of the REST guys seem to be coming around to this way of thinking and are thinking about declared interfaces for REST. I then wonder if and how REST with declared interfaces would be significantly different from WSDL (w/SOAP).

One thing I've learned about the SOAP/WS-* developer world is that people often pay lip service to certain words even though they use them all the time. For example, the technologies are often called Web services even though the key focus of all the major vendors and customers in this area is reinventing CORBA/DCOM with XML protocols as opposed to building services on the Web. Another word that is often abused in the SOAP/WS-* world is contract. When I think of a contract, I think of some lengthy document drafted by a lawyer that spells out in excruciating detail how two parties interact and what their responsibilities are. When a SOAP/WS-* developer uses the word contract and WSDL interchangeably, this seems incorrect because a WSDL is simply the XML version of OMG IDL. And an IDL is simply a list of API signatures. It doesn't describe expected message exchange patterns, required authentication methods, message traffic limits, quality of service guarantees, or even pre and post conditions for the various method calls. You usually find this information in the documentation and/or in the actual business contract one signed with the Web service provider. A WADL document for the REST Web service will not change this fact.

When a SOAP/WS-* says that he wants a contract, he really means he wants an interface definition language (IDL) so he can point some tool at a URL and get some stubs & skeletons automatically generated. Since this post is already long enough and I have to get to work, it is left as an exercise for the reader as to whether a technological approach borrowed from distributed object technologies like DCE/RPC, DCOM and CORBA meshes with the resource oriented, document-centric and loosely coupled world of RESTful Web services.

PS: Before any of the SOAP/WS-* wonks points this out, I realize that what I've described as a contract can in theory be implemented for SOAP/WS-* based services using a combination of WSDL 2.0 and WS-Policy. Good luck actually finding an implementation in practice that (i) works and (ii) is interoperable across multiple vendor SOAP stacks. 


 

Monday, 04 June 2007 18:10:52 (GMT Daylight Time, UTC+01:00)
Nice one. I am completely aligned with the observation that API signatures are not contracts. I've cringed over this ever since I first saw that corruption in the early indirection of services in the Xerox Network Services back in the day.

I also suspect, as you do, that we will never escape the need for external documentation to complete the definition of the behaviors by which an API is usable, especially in communicating with some stateful entity.

Hmm, this got your heart started this morning, aye? Thanks.
Tuesday, 05 June 2007 02:09:43 (GMT Daylight Time, UTC+01:00)
One of the key constraints of REST is uniform interface. An interface definition language doesn't help when you have only one interface.

That concept is really hard to believe, but the implication is that any IDL is contrary to this RESTful constraint.

A DL like WADL is trying to define and "fix" the service points (URIs) in a web service. But those are exactly the things that should be discovered in a hypermedia system - and should always remain flexible and server-generated to support evolution.

When I say "fix" them, I mean provide _any_ mechanism to generate and distribute client code that depends on them. This always (CORBA, COM, WSDL, XSD code generation) leads at least to a versioning problem.

Instead of an "interface" definition, with all of the implications for defining a processing API, the only solution in a REST-constrained system is to use declarative data formats.

In my blog entry on Signposts (http://johnheintz.blogspot.com/2007/05/does-rest-need-dl.html) I've tried to lay out a simple example of shopping via machine agent. Instead of knowing _any_ of the URIs, a data driven convention is used to point agents towards the right hyperlinks (state transitions) in a more flexible way.

I've (barely) described an HTML micro-format in that post, but the key is just data, not fixed interfaces.
Tuesday, 05 June 2007 16:13:07 (GMT Daylight Time, UTC+01:00)
You're both seem to be confused that XSD or WSDL has something to do with versioning or 'random-pick-of-the-day' contract 'blame' problem.

REST is far more likely to have a versioning problem, not the other way around.

And if you want to pick on WS, REST or anything, it is the implementation that has the versioning problem, not the interface. I bet you did not play long enough with DCE, COM, CORBA or RMI and are a new generation to this fear as well as an idea that uniformity makes you money (example: 'I see a stone on the street', GET diamonds.rest.com ).

You use interfaces all day long, around x million times a day (indirectly or otherwise) and they are of different and wonderful forms. You do not have a machine, input, screen or logon to your system without them either. You do not write documents or write software and type GET/POST all the time do you? First you need a power supply right? Serious, ask yourself, do you have a battery or power supply interface?

Even a simple power supply is not rest-my-case uniform either is it? And it helps to know the form: 3, 2 or heck you can even put a finger in it (re: POST+GET yourself DELETE-d from *.Web :-).

Your work, your family, your tools, your hand, your bank account, your clothes, your friends use them every day, if and you would moan yourselves to oblivion if they were REST-based.

Finally, at least one implementation I am very fond of has a direct mapping to what you both describe as REST and it has been fine since 99. The problem with it is that it is dumb as a brick if you are writing your service with 'look ma self-evolving-script-and-uniform-interfaces-based universe'. But hey without type (or media) information REST is even dumber.

Anyone complaining about typed data and typed messages or giving structural information to data is missing something in all their daily work and their life most likely. There is no evolution described with rest, but with the message that again is typed your micro-format is typed. Or do you read your GIFs as text?

Interfaces is data, data specification better and less vague, and free from browser HTML land and HTTP hopefuls. You would not, again not on IE or other browser, see this display without interfaces or it would look pretty rough. I would not post a random comment (random bit is REST fault), if there was no well-defined form interface ( and could I hack at it).

So what is your implementation behind that interface? Sure, I should not nor do I care.

Nor do you have anything sensible beyond TLS on security with REST, nor do you have anything more than soup of mother of Web 2.0 mania.

Also Web is not everything there is, is it? It is not even well-defined and different 'concept' to many.

Toys like the web companies did back in the day of crash are the reason for it, and you are falling for it all over again. Nothing changed don't you see that? drive.com repeats itself once again.

TCP is the daddy.

And Dare if you change the team from schema to services, it shouldn't influence your view on software construction. Especially if it leads to script kids designs again.

> borrowed from distributed object technologies like DCE/RPC, DCOM and CORBA meshes with the > resource oriented, document-centric and loosely coupled world of RESTful Web services

Completely misses the point, there is nothing borrowed (nothing on prop or rigid endpoint or protocol mapping, proprietery security models, object identity, prop wire format and plenty more), and for the record:

Web Service is REST, and WS, and WSDL, and POX and GIF (ops), and whatever you want it to be: as long as you describe it ( in words, in blog, in documentation or in machine format). Otherwise you get that finger problem up there.

URI describes or guarantees a description of: jack.

Get over it.

GET/POST doesn't even describe the TCP (again TCP) traffic taking over all Web HTTP usage put together either.

Better get over that too.
Defender Type 2
Tuesday, 05 June 2007 18:41:50 (GMT Daylight Time, UTC+01:00)
Defender Type 2,

"You're both seem to be confused that XSD or WSDL has something to do with versioning or 'random-pick-of-the-day' contract 'blame' problem."

I am picking on XSD because I've used it (a lot) and find it to be _too_ constraining. One simple example is that RelaxNG makes it easy to say "these in any order" and XSD doesn't. That simple trick offers a harmless flexibility where no schema is usually needed.

A schema language should _only_ constrain things that _need_ to be constrained, and leave everything else open (for extension, evolution, ...

"REST is far more likely to have a versioning problem, not the other way around."

You're kidding, right? The Web is the most interoperable and evolvable system around. Squid can front any backend server, any browser can navigate any site, dozens of libraries in dozens of languages....

Sure, there are a few exceptions. Contrast that with WS-* and there are mostly interop problems with few successes.

Regarding the rest of your post, could you followup with a 2-sentence summary. I didn't follow it well enough to comment.
Tuesday, 05 June 2007 21:02:52 (GMT Daylight Time, UTC+01:00)
REST is an architectural style hmm, not the entire Web and certainly not Internet (by a mile).

Here is some trivia, Inter + Operate :) Internet, as in Inter + Net, is the distribution mechanism not Web.

As for 'operate' and say Web, eBay does not talk to Amazon without a well-defined interface. Do you think any sensible business likes to adopt some random document exchange without or even lax restrictions?

API is for Application Programming + Interface.

1. Web is an Internet Application.

2. Programming in this paradigm is Data (or getting Paid, doing Work, having real systems around, not demos on top of demos on top of free service on top of proof of concept on top of architecture solved it all).

3. Interface is typed message exchange and patterns.

Therefore, Internet Data Message Exchange and Patterns, see it is very simple :)


We are still reading Web most of the time far less interoperating with it. Question: Why doesn't anyone attempt to get their software to interoperate with a pretty simple form interface on this blog?

Too restrictive?

Are there any other examples from 'these in any order'? If that is all, than one wonders if 'in any order' is a feature or a bug? Question: what does HTML parser do, and what flexibility is there? It is not like HTTP fabric (unrelated but aimed at the parallel crowd) is even suitable for any order is it?

WSDL does not command XML Schema either does it? If not, good that it is the only one adopted, as for evolution because you change the type, you have changed the interface and we better document it or support the old one.

As for evolution, what exactly stops WSDL or XSD document from evolving in contrast to REST? It is still say HTTP and documents and whatever schema language (WS or REST does not stop you there). The old version if required, the new version if required, and best of all whatever else there is, it is supported.

And why does everyone pick on this nonsense of REST vs WS?

Do whatever you prefer, and good luck. Just don't forget HTTP and HTML is not all there is, never was. It is only HTML, its rendering and DOM scripting that makes you comfortable in the 'rest my money making thoughts on web'.

Until you hit a brick wall on how limiting in scalability (yes the HTML interaction with JS), and interoperation with the using solely REST and HTTP is.

In fact, largest by value transacted business on this Planet is not done via HTTP at all. Even the business people are trained to lynch you if you approached it with GET or POST.

It is TCP and many systems that will never see the Web but still use Internet and VPNs and whatever is suitable.

That is what plenty of interop is about, not just Web or some remarkable baker dumping his job search skills while not working in interop land at all.

WS is not an enemy of REST, or something out there meant to 'eliminate it'. That thought is silly, despite people constantly getting off on that fact. What is the argument, that someone is enlightened? Heck even bloated stacks implementing WS, bloated applications running on runtimes and interpreters, do not restrict you out of the REST world either.

Do whatever you like, and be careful of crowd following, WS or REST, Java or CLR, their assumptions, reading someones flavour of the day blog-thought and more.

It is far from reality for everyone, much of it is unrealised IT hype and discussions on the intertwinbudget xml+http. Only Google managed to properly harvest it via : Ads (oh well, what an irony).

I only want tools when Web is appropriate, would not care less about all other cases. And there are plenty of those to live good and actually build exciting apps on, more than writer, email reader, blog writer, blog reader and photo journalism exercises. And that is where XSD can and does help.

When HTTP and Web and Agents and all runtime and XML bloatware grows up (and at this pace it would take another 15 years), perhaps I would open up more, but once burnt.. you are free to take on the risk and time and adoption and listening of pointless pushes of technologies and architecture styles.

Real work gets you involved with interop not just Web, you know making a living, and that wont change fast if ever. Try telling Google to change their NATIVE OS.

My 17 liras
Xatron
Wednesday, 06 June 2007 14:15:46 (GMT Daylight Time, UTC+01:00)
<xsd:complexType name="EntryType">
<xsd:all>
<xsd:element name="title" type="string"/>
<xsd:element name="description" type="string"/>
<xsd:element ref="pubDate" type="xsd:date"/>
</xsd:all>
</xsd:all>

http://www.w3.org/TR/xmlschema-0/#ref18
http://www.zvon.org/xxl/XMLSchemaTutorial/Output/ser_simpleContent_st3.html
Wednesday, 06 June 2007 14:32:34 (GMT Daylight Time, UTC+01:00)
Stian,
Nice try. See http://lists.w3.org/Archives/Public/xmlschema-dev/2002Feb/0193.html
I can't use xs:all to represent (i) content models that can have elements appear more than once (e.g. multiple enclosures on an RSS item) and more importantly (ii) to specify that an unlimited amount of elements from any namespace can show up in any order in my content model (e.g. extension elements such as dc:date, slash:comments, wfw:commentRSS, etc).

PS: The pubDate format in RSS 2.0 is not compatible with xsd:date but that's a trivial nitpick. ;)
Comments are closed.