I read Joel Spolsky's How Microsoft Lost the API War yesterday and found it pleasantly coincidental. Some of the issues Joel brings are questions I've begun asking myself and others at work so I definitely agree with a lot of the sentiments in the article. My main problem with Joel's piece is that it doesn't have a central theme but instead meanders a little and lumps together some related but distinct issues. From where I sit, Joel's article made a few different major & minor points which bear being teased out and talked about separately. The points I found most interesting were

Major Points

1. The primary value of a platform is how productive it makes developers, its ubiquity and how much of a stable environment it provides for them over time. Microsoft's actions in how has released both the .NET Framework and its upcoming plans for Longhorn run counter to this conventional wisdom.
2. Microsoft used to be religious about backwards compatibility, now it no longer is.

Minor Points 

1. The trend in application development is moving to Web applications instead of desktop applications.
2. A lot of developers using the .NET Framework use ASP.NET, client developers haven't yet embraced the .NET Framework.
3. The primary goal of WinFS (making search better)  can be acheived by less intrusive, simpler mechanisms.

So now to dive into his points in more detail.

.NET and Longhorn as Fire & Motion

The primary value of a platform is how productive it makes developers, its ubiquity and how much of a stable environment it provides for them over time. Microsoft's actions in how has released both the .NET Framework and its upcoming plans for Longhorn run counter to this conventional wisdom.

Joel approaches this point from two angles. First of all he rhapsodizes about how the Windows team bends over backwards to achieve backwards compatibility in their APIs even when this means keeping bug compatibility with old versions or adding code to handle specific badly written applications . This means users can migrate applications from OS release to OS release thus widening the pool of applications that can be used per OS. This is in contrast to the actions of competitors like Apple. 

Secondly, he argues that Microsoft is trying to force too many paradigm shifts on developers in too short a time.  First of all, developers have to make the leap from native code (Win32/COM/ASP/ADO) to managed code (ASP.NET/ADO.NET) but now Microsoft has already telegraphed that another paradigm shift is coming in the shape of Longhorn and WinFX. Even if you've made the leap to using the .NET Framework, Microsoft has already stated that technologies in the next release of the .Net Framework (Winforms, ASP.NET Web Services) are already outclassed by technologies in the pipeline (Avalon, Indigo). However to get these later benefits one not only needs to upgrade the development platform but the operating system as well. This second point bothers me a lot and I actually shot a mail to some MSFT VPs about 2 weeks ago raising a similar point with regards to certain upcoming technologies. I expected to get ignored but actually got a reasonable response from Soma with pointers on folks to have followup discussions with. So the folks above are aware of the concerns in this space. Duh!

The only problem I have with Joel's argument in this regard is that I think he connects the dots incorrectly. He agrees that Windows programming was getting too complex and years of cruft  eventually begins to become difficult to manage.  He also thinks the .NET Framework makes developers more productive. So it seems introducing the .NET Framework was the smart thing for Microsoft to do. However he argues that not many people are using it (actually that not many desktop developers are using it) . There are two reasons for this which I know first hand as a developer of a desktop application that runs in the .NET Framework (RSS Bandit)

• The .NET Framework isn't ubiqitous on Windows platforms
• The .NET Framework does not expose enough Windows functionality to build a full fledged Windows application with only managed code.

Both of these issues are why Microsoft is working on WinFX. Again, the elepahant in the living room issue is that it seems that Microsoft's current plans are fix these issues for developing on Longhorn not all supported Windows platforms.

Losing the Backwards Compatibility Religion

Microsoft used to be religious about backwards compatibility, now it no longer is.

Based on my experience as a program Manager for the System.Xml namespace in the .NET Framework I'd say the above statement isn't entirely accurate. Granted, the .NET Framework hasn't been around long enough to acquire a lot of cruft we do already have to be careful about breaking changes. In fact, I'm currently in the process of organizing backing out a change we made in Whidbey to make our W3C XML Schema validation more compliant in a particular case because it broke a number of major XML Web Services on the Web.

However I don't think I've seen anyone go above and beyond to keep bug compatibility in the way Raymond Chen describes in his blog. But then again I don't have insight into what every team working on the .NET

WinFS, Just About Search?

The primary goal of WinFS (making search better)  can be acheived by less intrusive, simpler mechanisms.

I'd actually had a couple of conversations this week with folks related to WinFS, including Mike Deem and Scoble.  We talked about the fact that external perceptions of the whats and whys of WinFS don't really jibe with what's being built. A lot of people think WinFS is about making search better [even a number of Longorn evangelists and marketing folks]. WinFS is really a data storage and data access platform that aims to enable a lot of scenarios, one of which just so happens to be better search. In addition, whether you improve full text search and the indexing service used by the operating system is really orthogonal to WinFS.

The main problem is that what the WinFS designers think WinFS should be, what customers and competitors expect it to be and what has actually been shown to developers in various public Longhorn builds are all different. It makes it hard to talk about what WinFS is or should be when your everyone's mental image of it is slightly different.

Disclaimer: The above statements are my personal opinions and do not reflect the intentions, strategies, plans or opinions of my employer.

A few weeks ago I mentioned I was considering writing a current and future trends in social software, blogging and syndication as part of a Bill Gates "Think Week" paper. Well, it looks like someone beat me to the punch and he already got one as part of the most recent "Think Week". The person who submitted the paper shared BillG's comments which were pretty insightful about some of the issues facing syndication technologies today and some that will loom once their usage becomes more widespread. I have lunch scheduled with the author of the paper so I'll definitely try and exchange some ideas.

In the short term this means I'll probably put any plans of writing such a document on the back burner at least until the end of the summer. Given my current workload at my dayjob (7 hours of meetings tomorrow, didn't get in until about 11PM tonight)  as well as the fact that there is significant work I want to do for the next release of RSS Bandit this probably is for the best anyway.

My submission on Designing Extensible & Version Resilient XML Formats has been accepted to XML 2004. It looks like I'm going to be in Washington D.C. this fall. Currently I'm in the process of writing an article about the topic of my talk which should show up on MSDN and XML.com in the next month or so. Afterwards I plan to submit a revised version of that article as the paper for my talk.

The ongoing conversation between Jeremy Mazner and Jon Udell about the capabilities of WinFS deepen this morning with Jeremy's post Did I misunderstand Udell's argument against WinFS? which was followed up by Jon's post When a journalist blogs. In his post Jon asks

We have standard query languages (XPath, XQuery), and standard ways of writing schemas (XSD, Relax), and applications (Office 2003) that with herculean effort have been adapted to work with these query and schema languages, and free-text search further enhancing all this goodness. Strategically, why not build directly on top of these foundations?

Tactically, why do I want to write code like this:

public class Person
  {
  [XmlAttribute()] public string Title;
  [XmlAttribute()] public string FirstName;
  [XmlAttribute()] public string MiddleName;
  [XmlAttribute()] public string LastName;
  ....

in order to consume data like this?

<People>
  <Person
    DisplayName="Woodgrove Bank"
    IMAddress="Support@woodgrovebank.com"
    UserTile=".\user_tiles\Adventure Works.jpg">
    <EmailAddresses>
        <EmailAddress
            Type="Work"
            Address="mortgage@woodgrovebank.com"/>
        <EmailAddress
            Type="Primary"
            Address="Support@woodgrovebank.com"/>
   </EmailAddresses>

I believe two things to be true. First, we have some great XML-oriented data management technologies. Second, the ambitious goals of WinFS cannot be met solely with those technologies. I'm trying to spell out where the line is being drawn between interop and functionality, and why, and what that will mean for users, developers, and enterprises.

Jon asks several questions and I'll try to answer all the ones I can. The first question about why WinFS doesn't build on XML, XQuery and XSD instead of items, OPath and the WinFS schema language is something that the WinFS folks will have to answer. Of course, Jon could also ask why it doesn't build on RDF, RDQL [or any of the other RDF query languages] and RDF Schema which is a related question that naturally follows from the answer to Jon's question.

The second question is why would one want to program against a Person object when they have a <Person> element. This is question has an easy answer which unfortunately doesn't sit well with me. The fact is that developers prefer programming against objects than they do programming with XML APIs. No XML API in the .NET Framework (XmlReader, XPathNavigator, XmlDocument, etc) comes close to the ease of use of  programming against strongly typed objects in the general case. Addressing this failing [and it is a failing] is directly my responsibility since I'm responsible for core XML APIs in the .NET Framework. Coincidentally, we just had a review with our new general manager yesterday and this same issue came up and he asked what we plan to do about this in future releases. I have some ideas. The main problem with using objects to program against XML is that although objects work well for programming against data-centric XML (rigidly structured tabular data such as an the data in an Excel spreadsheet, a database dump or serialized objects) there is a signficant impedance mismatch when trying to use strongly typed objects to program against document-centric XML (semi-structured data such as a Word document). However the primary scenarios the WinFS folks want to tackle are about rigidly structured data which works fine with using objects as the primary programming model.

Jon says that he is trying to draw the line between interop and functionality. I'm curious as to what he means by interop in this case. The fact that WinFS is based on items, OPath and WinFS schema doesn't mean that WinFS data cannot be exchanged in an interoperable manner (e.g. some form of XML export and import) nor does it mean that non-Microsoft applications cannot interact with WinFS. I should clarify that I have no idea what the WinFS folks consider their primary interop scenarios but I don't think the way WinFS is designed today means it cannot interoperate with other platforms or data models.

I suspect that Jon doesn't really mean interop when says so. I believe he is using the word the same way Java people use it where it really means 'One Language, One Programming Model, One Platform' everywhere instead of being able to communicate between disparate end points. In this case the language is XML and the platform is the XML family of technologies.

In a post entitled My comments on the Infoworld article "Databases flex their XML" Michael Rys writes

Sean McCown wrote this analysis (PDF version) in Apr 2004. In the article, he compares the XML capabilities of the 4 major relational database systems (comparing publicly available versions) both in terms of functionality, ease, flexibility and speed, and adds a sidebar on Yukon. Before I start giving my comments on the article, let me disclose that I talked to Sean during his research for the article and answered his questions on SQL Server 2000 and Yukon. Thus, some of the comments below are just my attempts to make Sean's translation of my answers clearer, because I was not answering his questions clear enough .

Michael then goes on to clarify various points around the terminology used in the article, XQuery and SQL Server. Both Sean's article and Michael's followup are excellent reading for anyone interested in the growing trend of XML-enabled relational databases and how the big 3 relational database vendors stack up.

I was pleasantly surprised today when I logged in to my Yahoo! Mail account and found out they've made good on their promise and now my mailbox size has gone up to 100MB from 6MB. I hope the folks at Hotmail are paying attention and upgrade the measely 2MB of space that they currently allocate to their free users.

For the past few weeks my friend Chris has had an open invitation for me to play Settlers of Catan with him and couple of other guys in the U-district. Today I finally accepted and when I got there it turned out that one of the guys was Evan Martin, one of the devs for LiveJournal. It turns out Evan just graduated from college and this was his last weekend in Seattle before moving to the Bay Area to start work at Google. Once we were introduced he mentioned that he knew of me and in fact that I was the reason he unsubscribed from the atom-syntax mailing list. It seems in one of the early discussions about Atom I wrote something which he felt was a technically valid point but was delivered in a scathing manner (i.e. punctuated with a flame) so he decided to bow out of further discussions about "RSS with different tag names". This reminded me of a comment by Robert Sayre in Joshua's weblog

OTOH, your post was free of insults, hyperbole, and condescension. Dare is usually right when there is an actual technical issue, but we're talking politics

My level of exasperation with a lot of what was going on with the Atom effort made me more scathing than I tend to be in usual email discourse. This is one of the reasons I unsubscribed from the list but it seems I hurt a couple of people's feelings along the way. Sometimes it is easy to forget that the people on the other end of an email thread aren't former denizens of git.talk.flame who relish technical arguments spiced with flame. My apologies to any others that were as significantly affected by my comments.

Anyway, we all (Jag, Chris, Evan and I) played a three hour game of Catan while partaking of some of the nice Bourbon thoughtfully provided by Chris. Evan seems like he would have been a decent guy to talk to about blogging and syndication related technologies. I hope he enjoys his new job at Google.

TiVo calls...

I just found out that Lloyd Banks is about to drop an album, Hunger For More, all I can say is G-G-G-G-G-Unit. Cop that shit.

By the way if you haven't copped Twista's Kamikaze, you should. It's not as gangsta as Adrenaline Rush, instead its more radio friendly, but still off the chain. Almost every track sounds good enough to be a single, definitely all killer no filler. 

I recently read a post by a Jeff Dillon (a Sun employee) entitled .NET and Mono: The libraries were he criticizes the fact that the .NET Framework has Windows specific APIs. Specifically he writes

Where this starts to fall apart is with the .NET and Mono libraries. The Java API writers have always been very careful not to introduce an API which does not make sense on all platforms. This makes Java extremely portable at the cost of not being able to do native system programming in pure Java. With .NET, Microsoft went ahead and wrote all kinds of APIs for accessing the registry, accessing COM objects, changing NTFS file permissions, and other very windows specific tasks. In my mind, this immediately eliminates .NET or Mono from ever being a purely system independent platform.

While I was still digesting his comments and considering a response I read an excellent followup by Miguel De Icaza in his post On .NET and portability where he writes

First lets state the obvious: you can write portable code with C# and .NET (duh). Our C# compiler uses plenty of .NET APIs and works just fine across Linux, Solaris, MacOS and Windows. Scott also pointed to nGallery 1.6.1 Mono-compliance post which has some nice portability rules.
...
It is also a matter of how much your application needs to integrate with the OS. Some applications needs this functionality, and some others do not.

If my choice is between a system that does not let me integrate with the OS easily or a system that does, I personally rather use the later and be responsible for any portability issues myself. That being said, I personally love to write software that takes advantage of the native platform am on, specially on the desktop.

At first I was confused by Jeff's post given that it assumes that the primary goal of the .NET Framework is to create a Write Once Run Anywhere platform. It's been fairly obvious from all the noise coming out of Redmond about WinFX that the primarily goal of the .NET Framework is to be the next generation Windows programming API which replaces Win32. By the way check out the WinFX overview API as JPG or WinFX API Overview as PDF.  Of course, this isn't to say that Microsoft isn't interested in creating an interoperable managed platform which is why there has been ECMA standardization of C#, the Common Language Infrastructure (CLI) and the Base Class Library (BCL). The parts of the .NET Framework that are explicitly intended to be interoperable across platforms are all parts of the ECMA standardization process. That way developers can have their cake and eat it too. A managed API that takes full advantage of their target platform and a subset of this API which is intended to be interoperable and is standardized through the ECMA process.

Now that I think about it I realize that folks like Jeff probably have no idea what is going on in .NET developer circles and assume that the goals of Microsoft with the .NET Framework are the same as that of Sun with Java. That explains why he positions what many see as a flaw of the Java platform as a benefit that Microsoft has erred in not repeating. I guess one man's meat is another man's poison.  

In a recent post entitled 15 Science Street Tim Bray, one of the inventors of XML, writes

Microsoft’s main talking point (I’m guessing here from the public documents) was that their software and format had the advantage that in WordML you can edit documents from arbitrary schemas.

Our pushback on that was that editing arbitrary-schema documents is damn hard and damn expensive and has never been anything more than a niche business.

which seems not to jibe with my experiences. Many businesses have XML formats specific to their target industry (LegalXML, HR-XML, FpML, etc) and many businesses use office productivity suites to create and edit documents. It seems very logical to expect that people would like to use their existing spreadsheet and word processing applications to edit their business documents instead of using XMl editors or specialized tools. More interestingly Tim Bray contradicts his position that editing user-defined schemas is a niche scenario when he writes

As we were winding up, a couple of really smart people (don’t know who they were) put up their hands and asked real good questions. The best was essentially “What would you like to see happen?” After some back and forth, I ended up with “You should have the right to own your own information. It’s your intellectual capital and you worked hard to produce it for your citizens. Sun doesn’t own it, Microsoft doesn’t own it, you own it, and that means it should be living in a nice, long-lived, non-proprietary data format that isn’t anyone’s competitive weapon.”

He took the words right out of my mouth. This is exactly what Microsoft has done with Office 2003 by allowing users to edit documents in XML formats of their choosing. In the letter Bringing the XML Vision to the Desktop with Office 2003 written by Jean Paoli of Microsoft (also a co-inventor of XML) he writes

an even greater and more innovative benefit is the fact that companies can now create their own XML schemas specific to their business, define the structure and type of data that each data element in a document contains and exchange information with customers and business partners more easily. This capability opens up a whole new realm of possibilities, not only for end users, but also for the business itself because now organizations can capture and reuse critical information that in the past has been lost or gone unused. 

Office 2003 is a great step forward in enabling businesses and end users harness the power of XML in typical document interchange scenarios. Arguments about whether you should use Sun's XML format or Microsoft's XML format aren't the point. The point is which tools allow you to use your XML format with the most ease.