The MSN Search team recently silently released http://addins.msn.com which among other things provides the API documentation for Windows Desktop Search that ships with the MSN Search Toolbar. The lowdown on what the API is good for is right at the beginning and is excerpted below

Which Extension Technology to Use?

There are two basic methods for creating add-ins for Desktop Search.

1.     Adding new file types by creating IFilters

• Extend Desktop Search with an IFilter add-in that knows how to “crack” the contents of a new file type in order to index its text and metadata.
• To do this, you need to build and register an object supporting the IFilter interface.
• You can add file-specific icons or context-menu handlers by following this documentation on extending the Windows Explorer file types by creating IContextMenu and IExtractIcon interfaces.

2.     Adding a new store by creating protocol handlers

• Extend Desktop Search so that it can index a new data store, such as the database of an e-mail application.
• To do this, you need to build a protocol handler object supporting the ISearchProtocol interface, along with an IUrlAccessor to pull the items. If the contents of the data store are file types not already indexed by Desktop Search, you may also need to implement one or more IFilters..
• To add icons or context-menu handlers, you need to implement portions of an IShellFolder.

You can also find custom IFilters at http://addins.msn.com including ones for PDF, ZIP, CHM and Mozilla Thunderbird mail formats. If only there were C# wrappers for all the gnarly COM interfaces I'd provide one for indexing and searching the cache file format used by RSS Bandit. Then people would be able to search their feeds directly from desktop search. That would be kinda hot. 

 

It seems both Google and Yahoo! provided interesting news on the personalized search front recently.

Yahoo! MyWeb 2.0 seems to merge the functionality of del.icio.us with the power of Yahoo! search. I can now add tags to my Yahoo! bookmarks, view cached versions of my bookmarked pages and perform searches restricted to the sites in my bookmark list. Even cooler is that I can share my bookmarks with members of my contact list or just make them public. The search feature also allows one to search sites restricted to those shared by others. All they need to do is provide an API and add RSS feeds for this to be a del.icio.us killer.

Google Personalized Search takes a different tack in personalizing search results. Google's approach involves tracking your search history and tracking what search results you cliked on. Then when next you perform searches, Google Personalized Search brings certain results closer to the top based on your search history or previous click behavior.

As for this week's news about MSN Search? Well you can catch what our CEO had to say about us in the ZDNet article Ballmer confident, but admits failings. We definitely have a lot of catching up to do but I don't think the race is over yet.

 

Just when you think we've missed the boat on software development trends on the Web, Microsoft surprises folks. First it was announcing that we will be baking RSS support into the next version of Windows. Now we've announced that we will be shipping a toolkit for building AJAX-style Web applications. In his post about the Atlas Project, Scott Guthrie writes

All of the pieces of AJAX – DHTML, JScript, and XMLHTTP – have been available in Internet Explorer for some time, and Outlook Web Access has used these techniques to deliver a great browser experience since 1998. In ASP.NET 2.0, we have also made it easier to write AJAX-style applications for any browser using asynchronous callbacks, and we use them in several of our built-in controls.

 

Recently, however, the technologies used by AJAX have become broadly available in all browsers, and use of this model for rich web applications has really taken flight. There are a number of high-profile new AJAX-style websites out there today, including a number by Google, as well as sites like A9 and Flickr. Microsoft will also have more sites that use this technology out there soon – check out Start.com and the MSN Virtual Earth project for examples.

 

The popularity of AJAX shows the growing demand for richer user experiences over the web. However, developing and debugging AJAX-style web applications is a very difficult task today. To write a rich web UI, you have to know a great deal of DHTML and JavaScript, and have a strong understanding of all the differences and design details of various browsers. There are very few tools to help your design or build these applications easily. Finally, debugging and testing these applications can be very tricky.

...

For this work, we’ve been working on a new project on our team, codenamed “Atlas”. Our goal is to produce a developer preview release on top of ASP.NET 2.0 for the PDC this September, and then have a website where we can keep updating the core bits, publishing samples, and building an active community around it.

 

Here are some of the pieces of Atlas that we are going to be delivering over time:

 

 

Atlas Client Script Framework

 

The Atlas Client Script Framework is an extensible, object-oriented 100% JavaScript client framework that allows you to easily build AJAX-style browser applications with rich UI and connectivity to web services. With Atlas, you can write web applications that use a lot of DHTML, Javascript, and XMLHTTP, without having to be an expert in any of these technologies.

 

The Atlas Client Script Framework will work on all modern browsers, and with any web server. It also won’t require any client installation at all – to use it, you can simply include references to the right script files in your page.

 

The Atlas Client Script Framework will include the following components:

• An extensible core framework that adds features to JavaScript such as lifetime management, inheritance, multicast event handlers, and interfaces
• A base class library for common features such as rich string manipulation, timers, and running tasks
• A UI framework for attaching dynamic behaviors to HTML in a cross-browser way
• A network stack to simplify server connectivity and access to web services
• A set of controls for rich UI, such as auto-complete textboxes, popup panels, animation, and drag and drop
• A browser compatibility layer to address scripting behavior differences between browsers.

This is excellent news which I know a lot of our UX developers at MSN will be glad to hear. Already Scott Isaacs who's been a key part of the AJAX development we've been doing at MSN has posted his opinions about Atlas in his blog entry entitled My personal thoughts on an AJAX (DHTML) framework..... His post highlights some of the history of AJAX as well as the issues a toolkit like Atlas could solve.

First RSS, now AJAX. All that's left is to see some announcement that we will be shipping a REST toolkit to make this a trifecta of utmost excellence. More nagging I must do...

 

Today I learned that Apple brings podcasts into iTunes which is excellent news. This will definitely push subscribing to music and videos via RSS feeds into the mainstream. I wonder how long it'll take MTV to start providing podcast feeds.

One interesting aspect of the announcement which I didn't see in any of the mainstream media coverage was pointed out to me in Danny Ayers's post Apple - iTunes - Podcasting where he wrote

Apple - iTunes - Podcasting and another RSS 2.0 extension (PDF). There are about a dozen new elements (or “tags” as they quaintly describe them) but they don’t seem to add anything new. I think virtually everything here is either already covered by RSS 2.0 itself, except maybe tweaked to apply to the podcast rather than the item.
They’ve got their own little category taxonomy and this delightful thing:

<itunes :explicit>
This tag should be used to note whether or not your Podcast contains explicit material.
There are 2 possible values for this tag: Yes or No

I wondered at first glance whether this was so you could tell when you were dealing with good data or pure tag soup. However, the word has developed a new meaning:

If you populate this tag with “Yes”, a parental advisory tag will appear next to your Podcast cover art on the iTunes Music Store
This tag is applicable to both Channel & Item elements.

So, in summary it’s a bit of a proprietary thing, released as a fait accompli. Ok if you’re targetting for iTunes, for anything else use Yahoo! Media RSS . I wonder where interop went.

This sounds interesting. So now developers of RSS readers that want to consume podcasts have to know how to consume the RSS 2.0 <enclosure> element, Yahoo!'s extensions to RSS and Apple's extensions to RSS to make sure they cover all the bases. Similarly publishers of podcasts also have to figure out which ones they want to publish as well.

I guess all that's left is for Real Networks and Microsoft to publish their own extensions to RSS for dealing with providing audio and video metadata in RSS feeds to make it all complete. This definitely complicates my plans for adding podcasting support to RSS Bandit. And I thought the RSS 1.0 vs. RSS 2.0 vs. Atom discussions were exciting. Welcome to the world of syndication.

PS: The title of this post is somewhat tongue in cheek. It was inspired by Slashdot's headline over the weekend titled Microsoft To Extend RSS about Microsoft's creation of an RSS module for making syndicating lists work better in RSS. Similar headlines haven't been run about Yahoo! or Apple's extensions to RSS but that's to be expected since we're Microsoft. ;)

 

As the developer of an RSS aggregator I'm glad to see Microsoft's Simple List Extensions for RSS. Many of the aggregator developers I spoke to at Gnomedex this weekend felt the same way. The reason for being happy about these extensions is that they provide a way to fix a number of key feeds that are broken in RSS aggregators today. This includes feeds such as the MSN Music Top 100 Songs feed, iTunes Top 25 Songs feed and Netflix Top 100 Movies feed.

The reasons these feeds appear broken in every aggregator in which I have tried them is covered in a previous post of mine entitled The Netflix Problem: Syndicating Ordered Lists in RSS. For those who don't have time to go back and read the post, the following list summarizes the problems with the feeds

1. When the list changes some items change position, new ones enter the list and old one's leave. An RSS reader doesn't know to remove items that have left the list from the display and in some cases may not know to eliminate duplicates. Eventually you have a garbled list with last week's #25 song and this weeks #25 song and last month's #25 song all in the same view.

2. There is no way to know how to sort the list. Traditionally RSS ggregators sort entries by date which doesn't make sense for an ordered list.  

The RSS extensions provided by Microsoft are meant to solve these problems and improve the current negative user experience of people who subscribe to ordered lists using RSS today.  

To solve the first problem Microsoft has provided the cf:treatAs element with the value "list" to be used as a signal to aggregators that whenever the feed is updated that the previous contents should be dumped or archived and replaced by the new contents of the list. That way we no longer have last week's Top 25 song list comingled with this week's list. The interesting question for me is whether RSS Bandit should always refresh the contents of the list view when a list feed is updated (i.e. the feed always contains the current list) or whether to keep the old version of the list perhaps grouped by date. My instinct is to go with the first option. I know Nick Bradbury also had some concerns about what the right behavior should be for treating lists in FeedDemon.

To solve the second problem Microsoft has provided the cf:sort element which can be used to specify what elements on an item should be used for sorting, whether the field is numeric or textual so we know how to sort it and what the human readable name of the field should be when displayed to the user. I'm not really sure how to support this in RSS Bandit. Having every feed be able to specify what columns to show in the list view complicates the user interface somewhat and requires a degree of flexibility in the code. Changing the code to handle this should be straightforward although it may add some complexity.

On the other hand there are some user interface problems. For one, I'm not sure what should be the default sort field for lists. My gut instinct is to add a "Rank" column to the list of columns RSS Bandit supports by default and have it be a numeric field that is numbered using the document order of the feed. So the first item has rank 1, the second has rank 2, etc. This handles the case where a feed has a cf:treatAs element but has no cf:sort values. This will be needed for feeds such as the Netflix Top 100 feed which doesn't have a field that can be used for sorting. The second problem is how to show the user what columns can be added to a feed. Luckily we already have a column chooser that is configurable per feed in RSS Bandit. However we now have to make the list of columns in that list configurable per feed. This might be confusing to users but I'm not sure what other options we can try.

 

I missed the first few minutes of this talk.

Bob Wyman of PubSub stated he believed Atom was the future of syndication. Other formats would eventually be legacy formats that would be analogous to RTF in the word processing world. They will be supported but rarely chosen for new efforts in the future.

Mark Fletcher of Bloglines then interjected and pleaded with the audience to stop the practice of providing the same feed in multiple formats. Bob Wyman agreed with his plea and also encouraged members of the audience to pick one format and stick to it. Having the same feed in multiple syndication formats confuses end users who are trying to subscribe to the feed and leads to duplicate items showing up in search engines that specialize in syndication formats like PubSub, Feedster or the Bloglines search features.

A member of the audience responded that he used multiple formats because different aggregators support some formats better than others. Bob Wyman replied that bugs in aggregators should result in putting pressure on RSS aggregator developers to fix them instead of causing confusion to end users by spitting multiple versions of the same feed. Bob then advocated using picking Atom since a lot of lessons had been learned via the IETF process to improve the format. Another audience member mentioned that 95% of his syndication traffic was for his RSS feed not his Atom feed so he knows which format is winning in the market place.

A question was raised about whether the admonition to avoid multiple versions of  feed also included sites that have multiple feeds for separate categories of content. The specific example was having a regular feed and a podcast feed.  Bob Wyman thought that this was not a problem. The problem was the same content served in different formats.

The discussion then switched to ads in feeds. Scott Rafer of Feedster said that he agreed with Microsoft's presentation from the previous day that Subscribing is a new paradigm that has come after Browsing and Searching for content. Although we have figured out how to provide ads to support Browse & Search scenarios we are still experimenting with how to provide ads to support the Subscribe scenarios. Some sites like the New York Times uses RSS to draw people to its website by providing excerpts in its feeds. Certain consultants have full text feeds which they view as advertising their services. While others put ads in their feeds. Bob Wyman mentioned that PubSub is waiting to see which mechanism the market settles on for having advertising in feeds before deciding on approach. Bob Wyman added that finding a model for advertising and syndication was imperative so that intermediary services like PubSub, Feedster and Bloglines can continue to exist. An audience member then followed up and asked why these services couldn't survive by providing free services to the general public and charging corporate users instead of resorting to advertising. The response was that both PubSub and Feedster already have corporate customers who they charge for their services but this revenue is not be enough for them to continue providing services to the general public. The Bloglines team considered having fee-based services but discarded the idea because they felt it would be a death-knell for the service given that most service providers on the Web are free not fee-based.

An audience member asked if any of the services would have done anything different two years ago when they started given the knowledge they had now. The answers were that Feedster would have chosen a different back-end architecture, Bloglines would have picked a better name and PubSub would have started a few months to a year sooner.

I asked the speakers what features they felt were either missing in RSS or not being exploited. Mark Fletcher said that he would like to see more usage of the various comment related extensions to RSS which currently aren't supported by Bloglines because they aren't in widespread use. The other speakers mentioned that they will support whatever the market decides is of value.

 

Scott Gatz of Yahoo! started by pointing out that there are myriad uses for RSS. For this reason he felt that we need more flexible user experiences for RSS that map to these various uses. For example, a filmstrip view is more appropriate for reading a feed of photos than a traditional blog and news based user interface typically favored by RSS readers. Yahoo! is definitely thinking about RSS beyond just blogs and news which is why they've been working on Yahoo! Media RSS which is an extension to RSS that makes it better at syndicating digital media content like music and videos. Another aspect of syndication Yahoo! believes is key is the ability to keep people informed about updates independent of where they are or what device they are using. This is one of the reasons Yahoo! purchased the blo.gs service.

Dave Sifry of Technorati stated that he believed the library model of the Web where we talk about documents, directories and so on is outdated. The Web is more like a river or stream of millions of state changes. He then mentioned that some trends to watch that emphasized the changing model of the Web were microformats and tagging.

BEGIN "Talking About Myself in the Third Person"

Steve Gillmor of ZDNet began by pointing out Dare Obasanjo in the audience and saying that Dare was his hero and someone he admired for the work he'd done in the syndication space. Steve then asked why in a recent blog posting Dare had mentioned that he would not support Bloglines proprietary API for synchronizing a user's subscriptions with a desktop RSS reader but then went on to mention that he would support Newsgator Online's proprietary  API. Specifically he wondered why Dare wouldn't work towards a standard instead of supporting proprietary APIs.

At this point Dare joined the three speakers on stage. 

Dare mentioned that from his perspective there were two major problems that confronted users of an RSS reader. The first was that users eventually need to be able to read their subscriptions from multiple computers. This is because many people have multiple computers (e.g. home & work or home & school) where they read news and blogs from. The second problem is that eventually, due to the ease of subscribing to feeds, people eventually succumb to information overload and need a way to see only the most important or interesting content in the feeds to which they are subscribed. This is the "attention problem" that Steve Gillmor is a strong advocate of solving. The issue discussed in Dare's blog post is the former not the latter. The reason for working with the proprietary APIs provided by online RSS readers instead of advocating a standard is that the online RSS readers are the ones in control. At the end of the day, they are the ones that provide the API so they are the ones that have to decide whether they will create a standard or not.  

Dare rejoined the audience after speaking.  

END "Talking About Myself in the Third Person"

Dave Sifry followed up by encouraging cooperation between vendors to solve the various problems facing users. He gave an example of Yahoo! working with Marc Canter on digital media as an example.

Steve Gillmor then asked audience members to raise their hand if they felt that the ability to read their subscriptions from multiple computers was a problem they wanted solved. Most of the audience raised their hands in response.

A member of the audience responded to the show of hands by advocating that people us web based RSS readers like Bloglines. Scott Gatz agreed that using a web based aggregator was the best way to access one's subscriptions from multiple computers. There is some disagreement between members of the audience and the speakers whether there are problems using Bloglines from mobile devices which prevent it from being the solution to this problem.

From the audience, Dave Winer asks Dave Sifry why Technorati invented Attention.Xml instead of reusing OPML. The response was that the problem was beyond just synchronizing the list of feeds the user is subscribed to.

Steve Gillmor ended the session by pointing out that once RSS usage becomes widespread someone will have to solve the problem once and for all.  

 

This was a keynote talk given by Dean Hachamovitch and Amar Gandhi that revealed the the RSS platform that will exist in Longhorn, the level of RSS support in Internet Explorer 7 as well as showed some RSS extensions that Microsoft is proposing.

Dean started by talking about Microsoft's history with syndication. In 1997, there was Active Desktop and channels in IE 4.0 & IE 5.0 which wasn't really successful. We retreated from the world of syndication for a while after that. Then In 2002, Don Box starts blogging on GotDotNet. In 2003, we hired Robert Scoble. In 2004, Channel 9 was launched. Today we have RSS feeds coming out of lots of places from Microsoft. This includes the various feeds on the 14 15 million blogs on MSN Spaces, the 1500 employee blogs on http://blogs.msdn.com and http://blogs.technet.com, 100s of feeds on the Microsoft website and even MSN Search which provides RSS feeds for search results.

Using XML syndication is an evolution in the way people interact with content on the web. The first phase was browsing the Web for content using a web browser. Then came searching the Web for content using search engines. And now we have subscribing content using aggregators. Each step hasn't replaced the latter but instead has enhanced user experience while using the Web.  In Longhorn, Microsoft is betting big on RSS both for end users and for developers in three key ways

1. Throughout Windows various experiences will be RSS-enabled and will be easy for end users to consume
2. An RSS platform will be provided that makes it easy for developers to RSS-enable various scenarios and applications
3. Increasing the number of scenarios that RSS handles by proposing extensions

Amar then demoed the RSS integration in Internet Explorer 7. Whenever Internet Explorer encounters an RSS feed, a button in the browser chrome lights up which indicates that a feed is available. Clicking on the button shows a user friendly version of the feed that provides rich search, filtering and sorting capabilities. The user can then hit a '+' button and subscribe to the feed. Amar then navigated to http://search.msn.com and searched for "Gnomedex 5.0". Once he got to the search results, the RSS button lit up and he subscribed to the search results. This shows how one possible workflow for keeping abreast of news of interest using the RSS capabilities of Internet Explorer 7 and MSN Search.

At this point Amar segued to talk about the Common RSS Feed List. This is a central list of feeds that a user is subscribed to that is accessible to all applications not just Internet Explorer 7. Amar then showed a demo of an altered version of RSS Bandit which used the Common RSS Feed List and could pick up both feeds he'd subscribed to during the previous demo in Internet Explorer 7. I got a shout out from Amar at this point and some applause from the audience for helping with the demo. :)

Dean then started to talk about the power of the enclosure element in RSS 2.0. What is great about it is that it enables one to syndicate all sorts of digital content. One can syndicate  video, music, calendar events, contacts, photos and so on using RSS due to the flexibility of enclosures.

Amar then showed a demo using Outlook 2003 and an RSS feed of the Gnomedex schedule he had created. The RSS feed had an item for each event on the schedule and each item had an iCalendar file as an enclosure. Amar had written a 200 line C# program that subscribed to this feed then inserted the events into his Outlook calendar so he could overlay his personal schedule with the Gnomedex schedule. The point of this demo was to show that RSS isn't just for aggregators subscribing to blogs and news sites.

Finally, Dean talked about syndicating lists of content. Today lots of people syndicate Top 10 lists, ToDo lists, music playlists and so on. However RSS is limited in how it can describe the semantics of a rotating list. Specifically the user experience when the list changes such as when a song in a top 10 list leaves the list or moves to another position is pretty bad. I discussed this very issue in a blog post from a few months ago entitled The Netflix Problem: Syndicating Ordered Lists in RSS.

Microsoft has proposed some extensions to RSS 2.0 that allows RSS readers deal with ordered lists better. A demo was shown that used data from the Amazon Web Services to create an RSS feed of an Amazon wish list (the data was converted to RSS feeds with the help of Jeff Barr). The RSS extensions provided information that enabled the demo application to know which fields to use for sorting and/or grouping the items in the wish list feed.

The Microsoft Simple List Extensions Specification is available on MSDN. In the spirit of the RSS 2.0 specification, the specification is available under the Creative Commons Attribution-ShareAlike License (version 2.5). 

A video was then shown of Lawrence Lessig where he commended Microsoft for using a Creative Commons license.

The following is a paraphrasing of the question & answer session after their talk

Q: What syndication formats are supported?
A: The primary flavors of RSS such as RSS 0.91, RSS 1.0 and RSS 2.0 as well as the most recent version of Atom.

Q: How granular is the common feed list?
A: The Longhorn RSS object model models all the data within the RSS specification including some additional metadata. However it is fairly simple with 3 primary classes.

Q: Will Internet Explorer 7 support podcasting?
A: The RSS platform in Longhorn will support downloading enclosures.

Q: What is the community process for working on the specifications?
A: An email address for providing feedback will be posted on the IE Blog. Robert Scoble also plans to create a wiki page on Channel 9.  

Q: What parts of the presentation are in IE 7 (and thus will show up in Windows XP) and what parts are in Longhorn?
A: The RSS features of Internet Explorer 7 such as autodiscovery and the Common RSS Feed List will work in Windows XP. It is unclear whether other pieces such as the RSS Platform Sync Engine will make it to Windows XP.

Q: What are other Microsoft products such as Windows Media Player doing to take advantage of the RSS platform?
A: The RSS platform team is having conversation with other teams at Microsoft to see how they can take advantage of the platform.

 

I've been doing a lot of thinking and writing (at work) about Web platforms and Web APIs recently. Just yesterday I was talking to some folks about Web APIs and an interesting question came up about the Google Web APIs. Basically we couldn't come up with good reasons for Google to have exposed them or for people to use them in their current form. I left the conversation wondering whether I just wasn't smart enough to see the brilliance of their strategy. Thanks to a post by Alex Bosworth's entitled Search Engine APIS it seems I may not be so dumb after all. He wrote

Recently, the Search engines have been trying to get into the mix, with new pushes to gain developer followers. Google recently launched a new subdomain emphasizing their committment to not being a technology black hole. Less recently, Yahoo launched their own developer site, in an attempt to build 3rd party value around their product.

However so far these 3rd party applications haven't found widespread appeal and had breakout success. This is in large part because of crippling artificial restrictions placed on Google/Yahoo/etc APIs.

It's not to say that applications made on platforms haven't been fairly cool ie: Yahoo News In Pictures, or clustering Yahoo search results.

But these will never progress beyond cool toys, and I'm not sure that Yahoo/Google/etc realize that. Talking to Google engineers, I was informed that Google thinks of their 3rd party API program as a complete failure and only a couple people have made anything vaguely useful from it. The reason why is they have no real committment to making these programs work. For example their terms and conditions are so draconian and laden with legalese there is no motive for developers to work with them.

This confirms some of the suspicions I had about their search engine API. Of course, this isn't to say I think providing programmatic access to search engine results is a bad idea. I just think Google's approach is very weird.

For example, I like the fact that I can get the results of a query on recent news items for terms such as "G-Unit Game beef" as an RSS feed from MSN Search.  Or that I can subscribe to the results of searching for my name on Feedster in my favorite RSS reader. As an end user I get a lot of use out of them and as a developer they are a cinch to work with.

There's a difference between cool apps and useful apps. The people at Google sometimes don't get the difference but the guys at Yahoo! definitely do. All you have to do is compare Yahoo! Maps to Google Maps to figure this out. Similarly I think the APIs provided by Google don't really enable people to actually build useful apps just cool little toys. MSN Search and Yahoo! seem to be on a better track because they enable more useful scenarios with their breadth of services exposed as APIs* as well as the fact that they provide these results as RSS.

What I've actually found surprising is that neither service puts sponsored search results in the results returned. I assume this is because we are still in the early days of search syndication but once it catches on and regular people are subscribing to search results so they can do things like build their own newspaper/dashboard then I'm sure we'll see the ads creep in.

* RSS is an API. Deal with it.

 

Stephen O' Grady has a blog post entitled Gmail Fighting Off Greasemonkey? where he writes

I'm not sure of the reasoning behind it, but it appears that Google may have made some behind the scenes changes to Gmail that disrupted the scipts and extensions I rely on to get the most out of my account. One of the Apache devs noted here that his Greasemonkey enabled persistent searches were no longer functioning, and in the same timeframe I've lost my delete key. It's not just Greasemonkey scripts, however, as my Gmail Notifier extension for Firefox has apparently been disabled as well. While it's Google's perogative to make changes to the service as necessary for maintenance or other reasons, I'm hoping this is not a deliberate play at preventing would-be participants from enhancing the value of Google's service. It's remarkable how much less useful Gmail is to me when I have to log in to see if I have mail, or can't easily delete the many frivolous emails I receive each day (yes, I'm aware that I can use a POP client for this, but I'd rather not).
...
Update: As one reader and a few posters to the GM listserv have noted, one thing that's disrupted a variety of user scripts has been the fact that the domain to Gmail has changed from gmail.google.com to mail.google.com. While simply adding the domains into the GM interface had no effect on my Gmail, a reinstallation of a version of the script with updated domain returned my beloved Delete button. What do developers think of this change with Google's service? Here's one take from the GM list: "I noticed [the domain change] too. Why they can't just leave it alone, I can't understand." To be a bit less harsh, while Google probably had good reasons for making the change, it would have been great to see them be proactive and notify people of the change via their blog or some other mechanism.

I find this hilarious. Greasemonkey scripts work by effectively screen scrapping the website and inserting changes into the HTML. Stephen and others who are upset by Google's change are basically saying that Google should never change the HTML or URL structure of the website ever again because it breaks their scripts. Yeah, right.

Repeat after me, a web page is not an API or a platform.  

Speaking of apps breaking because of GMail domain changes it seems RSS Bandit users were also affected by the GMail domain name change. It looks like we have a problem with forwarding the username and password after being redirected from https://gmail.google.com/gmail/feed/atom to https://mail.google.com/gmail/feed/atom. I'll fix this bug in the next version but in the meantime RSS Bandit users should be able to work around this by changing the URL manually to the new one. My bad.

 

I've been really busy at work and haven't had free time left over to do significant work on RSS Bandit for the past few weeks. I should have some more free time starting this weekend and will begin to start making progress again. In the mean time there have been decisions made which are probably worth sharing.

Synchronization with Bloglines and Newsgator Online
I've mentioned in the past that I'd like the next version of RSS Bandit to be able to synchronize it state with a user's account in Newsgator Online or Bloglines. However I've also mentioned the various limitations of the Bloglines sync API which make it less than ideal as a way to synchronize the state of a desktop RSS reader with an online aggregator. On the flip side, I've had some discussions with folks at Newsgator which imply that I'll be able to get a full synchronization API from them by the time Nightcrawler ships.

My current decision is that I am dropping plans for supporting synchronizing RSS Bandit with Bloglines due to the limitations of its API which is guaranteed to leave users with a poor user experience. Instead the only new synchronization target added in the next release will be Newsgator Online.

Newsgroup support
One interesting problem that came up when integrating newsgroups into the RSS Bandit  user interface was what to use as permalinks for newsgroup posts. Thankfully, Google Groups comes to the rescue here. All newsgroup posts are given a permalink that maps the the Google Groups URL for that message ID. This works fine with public newsgroups that are archived by Google but leads to broken links for internal newsgroups. I think this is a decent trade off.

The screenshot below shows this feature in action

Subscription Wizard
Given that we now provide the ability to subscribe to both newsgroups and RSS/Atom feeds we've decided to revamp the feed subscription mechanism. We'll be adding an "Add Subscription" wizard which gives people the option of adding an RSS or Atom feed, a newsgroup, a web page (which we then scour for feeds) or a search term (which we map to an MSN Search or Feedster search results feed). My initial instinct with the last option was to just go with PubSub but it doesn't seem to have a straightforward mechanism for people to create search feeds programmatically. REST wins again.

Screenshot below 

 

One of the reasons I came to work at MSN is that I believe strongly in the power of software to connect people together. I recently found a post entitled Spaces Friends which shows exactly how strong these connections can become. The post is excerpted below

The Importance of Spaces

 One should never expect the good fortune that has come my way since I was first aware of the MSN Spaces wide beta release last fall.  I first started writing on December 4, 2004, as I recall on a Spaces blog with a different title and editorial theme.  My experience since has been both personally rewarding and life-expanding.

I published (posted) the first poetry I ever showed to anyone in this life here on Crackers.  I’ve made some attempts at humor in some entries.  I have written some serious stuff as well.  All of it is pretty strange coming from an old Electrical Engineer/Physicist/high-tech manager/pilot/geek.  But other guys I’ve known with similar histories have done even more bizarre stuff.

More important than publishing fledging writing has been some remarkable friendships that have developed by magic from initial contacts left in the comments to entries.  The first of these came in late December, 2004, and remains one of my strongest adult friendships, and there is every reason to believe that it is a life-long association even though we may never see each other face to face.  Geographical distance can be a real barrier at times, but one that is breaking down rapidly because of Internet-enabled communication.  Who knows?  We might meet in person someday down the road.  Along the way a few other friendships have developed which span great distances.  Each has been an asset to my life.
...
Now, as I find myself in a situation I simply can’t manage without help, my friends have made a selfless commitment getting me through the initial phases of my cancer treatment.

As my first Spaces friend said the other day, “Isn’t it interesting how God places people in our path at important points?”  It’s more than interesting, B.  It’s going to help save my life.  MSN Spaces has been of the utmost importance to me.  Maybe it will continue to be.

A number of us working on Spaces read Bill's blog and our thoughts definitely go out to him in this time of need. It is really humbling to see how the work we do is changing people's lives.

 

From the press release MSN Continues Investments in Search With the Launch of Local Search we learn

Starting today, consumers in the United States will see a new Local category added to the MSN Search options on MSN.com. When consumers search for local information, they will receive results from city- and region-specific White Pages and Yellow Pages directory information. For example, a local search on “auto mechanics” could bring up listings of nearby mechanics, repair shops and towing companies.

Each local search result is shown as a numbered pin on a corresponding map provided through Microsoft^® MapPoint^® Web Service, and digital aerial images are supplied by TerraServer-USA when available for a given search result. The TerraServer-USA Web site is one of the world’s largest online databases, providing access to a vast data store of maps and aerial photographs of the United States. Originating at the Microsoft Bay Area Research Center, TerraServer is operated by Microsoft as a research project for developing advanced database technology.

The new MSN Local Search functionality is an evolution of the Near Me search feature that debuted on MSN Search in February of this year and allowed consumers to receive search results tailored to a geographic location. Those interested in trying the beta of the new offering should visit http://www.msn.com.

Kudos to the MapPoint/VirtualEarth and MSN Search folks for getting this out the door. My first test was trying a search for "pizza" in my area.  I decided to figure out how to specify my location with more accuracy. Come to think of it I'm not sure how it even figured out to look in Seattle. Anyway I followed the link from the phrase How do I change my location? and saw that I can specify my city, state and zip code in the settings page which is stored in a cookie.

There are two minuses here. The first is that I can't store my exact address. When I'm looking for a pizza place I want to find one that delivers in my area not just one in the same city as me. The second is that I can only store one location which is one better than Google Maps but quite poor when compared to Yahoo! Maps. It looks like I'll be sticking with Yahoo! for now. I will send the MSN Search folks some feedback about needing to add these features.

By the way, a more direct link to the service is http://search.msn.com/local.

 

Joe Wilcox has a post that has me scratching my head today. In his post Even More on New Office File Formats, he writes

Friday's eWeek story about Microsoft XML-based formats certainly raises some questions about how open they really are. Assuming reporter Pater Galli has his facts straight, Microsoft's formats license "is incompatible with the GNU General Public License and will thus prevent many free and open-source software projects from using the formats." Earlier this month, I raised different concerns about the new formats openness.

To reiterate a point I made a few weeks ago: Microsoft's new Office formats are not XML. The company may call them "Microsoft Office Open XML Fromats," but they are XML-based, which is nowhere the same as being XML or open, as has been widely misreported by many blogsites and news outlets.

There are two points I'd like to make here. The first is that "being GPL compatible" isn't a definition of 'open' that I've ever heard anyone make. It isn't even the definition of Open Source or Free Software (as in speech). Heck, even the GNU website has a long list of Open Source licenses that are incompatible with the GPL. You'll notice that this list includes the original BSD license, the Apache license, the Zope license, and the Mozilla public license. I doubt that EWeek will be writing articles about how Apache and Mozilla are not 'open' because they aren't GPL compatible.

Secondly, it's completely unclear to me what distinction Joe Wilcox is making between being XML and being XML-based. The Microsoft Office Open XML formats are XML formats. They are stored on the hard drive as compressed XML files using standard compression techniques that are widely available on most platforms. Compressing an XML file doesn't change the fact that it is XML. Reading his linked posts doesn't provide any insight into whether this is the distinction Joe Wilcox is making or whether there is another. Anyone have any ideas about this?

 

 

I was chatting with Mike Vernal a few weeks ago about the various outcomes we've seen from writing a Bill Gates Thinkweek paper. We came up with the following taxonomy

1. Increased Visibility for the Author: These are usually the papers of the form "Here's a Cool Idea from [my buddies and] me". The most common result for a ThinkWeek paper is for the author to get a virtual pat on the back from BillG and to get their name noticed by their management especially since some product groups have an internal review of these papers before sending them up the stack to the big man himself. 

2. Increased Visibilty for a Product Group: These are usually the papers of the form "Here's a Cool Idea from [my product group]". Sometimes a product group wants to lay out its vision for their product or technology space to the big guy outside of the usual executive review + PowerPoint format and a ThinkWeek paper is good way to do that. A great example is the MapPoint team which submitted the "Virtual Earth" paper recently.

3. The Fire Drill: These are usually of the form "Here's a Cool Opportunity that Microsoft is missing". The best example of this is the "Internet Tidal Wave" memo that Bill Gates sent in response to J Allard's thinkweek paper a decade ago. Many people who send in a ThinkWeek paper are expecting this reaction and in fact I've seen a few that actually have "send Internet Tidal wave-style memo" as one of the recommendations in their paper. Of course, the chances of such reactions to the average ThinkWeek paper are the same as getting a record deal from sending Dr. Dre a copy of your demo tape.

4. Spreading a Meme: Sometimes the ideas in a paper are so powerul it seems that they take a life of their own as they spread from person to person. It is hard to point to any direct action being taken as a result of them but their influence is felt by many. This is my favorite outcome from a ThinkWeek paper.

A recent ThinkWeek paper entitled "10 Crazy Ideas to Shake Up Microsoft" is exactly the last kind of paper mentioned above. The MiniMicrosoft blog mentioned this paper a while ago in a post entitled Think Week, the WSJ, and those Ten Crazy Ideas. According to Mini the 10 Ideas were

1. Schedule Unscheduled Time into Performance Reviews
2. "Break Up" the Company
3. Encourage Loose but Prominent Couplings
4. Exile and Empower Incubation Projects
5. Offer Super-exponential Rewards
6. Offer Different Risk-Reward Compensation Profiles
7. Cut Back on Bureaucracy
8. Review Cost Cutting
9. Reduce Headcount on Large Dev Projects
10. Eliminate Exec Reviews

Why I like the list [and why I assume it's been so popular at work] is that it succintly summarizes the major problems facing folks trying to get stuff done at Microsoft today AND proposes solutions. I suspect these problems exist in every large corporation and are not limited to Microsoft or even software companies.

If you work at Microsoft and you haven't read the paper, you really should. If you work at a medium sized to large corporation and you are a decision maker you should staple that list to a bulletin board in your office and ruminate on its points. 

 

 

I saw Batman Begins this weekend and I'm still trying to decide whether it is the best Batman movie of all time or not. Although there is no villain who gives a performance as commanding as Jack Nicholson's in 1989 Batman movie, Christian Bale is a better Batman than Michael Keaton and Michael Caine plays Alfred better than I imagined possible. Morgan Freeman is also great as Lucius Fox.

The movie is over 2 hours long which is to be expected given that it spends the first 45 minutes or more on the back story leading up to Bruce Wayne deciding to become the Batman. Though the movie is a bit long it makes up for this by hitting all the right notes when it comes to exploring the Batman mythos. My favorite themes from the comic are there from the night and day difference between Bruce Wayne's playboy lifestyle and Batman's ongoing war on crime to Jim Gordon questioning whether the existence of Batman encourages the existence of the more "theatrical" criminals (e.g. Joker, Two Face and the Riddle).

I definitely can't wait to see the next movie in the series.

Score: **** out of *****

 

In today's issue of the Wall Street Journal, Walt Mossberg reviews a number of blog hosting services in the article Taking the Mystery Out of Blog Creation. The article describes the experiences of Walt and his assistant Katie in building their test blogs including  http://spaces.msn.com/members/wmossberg and http://kaboehret.blogspot.com/. He writes

This past week, my assistant Katie Boehret and I tested three of these free blog-creation services to see what they offer. We tried the popular Google-owned service, Blogger.com, as well as Microsoft's new MSN Spaces service, each of which is estimated to host millions of blogs. We also tested Yahoo's Yahoo 360 service, which still is in its test phase. We quickly learned how simple it is to set up a blog, and how addictive they can become.

While using these three sites, we paid careful attention to how each blog-creating service handled four basic tasks: publishing text entries, or "posting" as it is called in blog land; adding photos; publishing links to other Web pages on our blogs; and providing privacy (if desired) online. We also took note of the overall style and formatting options provided on each site.

Our verdict: Microsoft's MSN Spaces did the best job of performing these tasks in a way that was organized and self-explanatory. Yahoo 360 was almost as easy, but it tries to tie in the use of too many other Yahoo services. Blogger.com has a long way to go until it becomes as easy to use as the others.

It's great to see the validation by both our users and reviewers like Walt Mossberg of the design choices we've made in attempting to bring blogging to a more widespread audience.

I'd like to thank all the folks who've been giving us great feedback; reviewers like Walt, various folks who leave comments in my space or Mike's or Scott's, the people who've added stuff to the MSN Spaces wiki, and the good people at work who send mail to the various Spaces mailing lists. You're feedback makes us better. Thanks.

 

A few months ago, Omar Shahine wrote about how various folks in our org came together to get the MSN Mobile Messaging feature to work. This feature allows people to have IM conversations with people using a cell phone via SMS using MSN Messenger. In his post When the work you do is architectural (MSN Mobile Messaging) he talks about the feature and how it was implemented on our various back ends. Some of his post is excerpted below  

So what to do? Enter MSN Messenger to SMS communications (we call this Enhanced mobile messaging). The feature we've been working on for the past year (or longer) was to allow a user of MSN Messenger on a PC to send a message to some one that is not signed into MSN Messenger but has an SMS enabled Mobile device AND to reply to that SMS message and have a real time chat (in otherwords, a two way conversation between MSN Messenger and Mobile phone using SMS as the wire protocol). This last part is important, but to understand it I need to explain one more thing.

For the past few years part of the scenario above has been available through what I will call a hack. Most phones that have SMS also have an email gateway that can take a message sent to a special email address and forward that message to the phone. For example, an email sent to <phone number>@mmode.com will forward that message via SMS to the <phone number> of an ATT Wireless subscriber. However, the user cannot reply to that email enabling a 2 way chat. Furthermore, it breaks the SMS user experience that mobile phone users are used to.

So, to fix this we set out to build all the necessary carrier infrastructure, SMS infrastructure, and build the technology and carrier relationships to ship the ability for users to have a two way conversation from MSN Messenger to a Mobile device that has nothing more than SMS capabilities (practically every singe phone on the planet). Not only that, but we support "Offline messages" so that if a Mobile phone device replies to an SMS from Messenger, and you have signed out of MSN Messenger, the next time you sign in the message will be delivered to you allowing you to continue the conversation.

I was reminded of this today because of his follow up post MSN Mobile Messaging for Cingular, which let me know that my girlfriend's cell phone carrier is now supported. Sweet.

Omar's original post has some interesting details on some of the issues they faced in bringing this feature to the market. This is an awesome feature and I'm definitely going to get a lot of use out of it.

 

So it looks like I'll be attending Gnomedex 5.0 next week. I had expected to sit it out but a chance to attend fell in my lap last week so I took the opportunity. The conference will be in walking distance from my apartment and will be all about blogging & RSS related topics so it is doubleplusgood that I'll get to go.

I exchanged some mail with Nick Bradbury and Chris Pirillo about being part of an aggregator developer get together during the conference so that should be fun too. I'm looking forward to meeting a bunch of folks who I've only exchanged mail with over the past few years.

 

 

I stumbled upon a blog post on Wil Wheaton's blog via Penny Arcade yesterday that I found interesting. It seems there was a recent threat of a strike by the Screen Actors Guild (SAG) on behalf of voice actors in video games which got them improved pay. This seems to have led to somewhat of a backlash from some video game developers. Wil writes

Since I wrote about voice actors the other day, I've been personally attacked, called names, and vilified all over the Internets, often by people whose work I respect and admire.
...
It seems like many developers are angry with SAG because SAG stood up for its members, which is what a union is supposed to do. It just doesn't make any sense to me that SAG is being viewed with such animosity, just for doing its job. Actors represent less than 3% of the total budget on games, so it's incredibly unlikely that if SAG were able to make some residual gains, it would even affect developers' pay. I could be wrong, but I seriously doubt that producers are going to tell developers that they can't afford to pay them, because actors are now taking up 4.5% or 5% of the budget.
...
I don't understand what developers gain by spending energy attacking SAG, when SAG is just doing what its membership expects it to do. As far as I can tell, voice actors and developers have the same ultimate goal, and I just don't get why developers are so angry with SAG for trying to reach that goal. It seems like a lot of developers and gamers are pissed at SAG because SAG has the ability to stand up to our employers and ask for better wages, and from an Art of War standpoint, that is an awfully huge waste of energy. It makes much more sense to me for developers to take that energy and those resources, point it at producers, and take the fight to them. Because, ultimately, getting angry at SAG, or me, or other working actors, isn't going to get developers better contracts or profit-sharing. All it's going to do is take focus away from the people who can make those things happen.

Given that video games seem to have the same hit-centric characteristics of movies and music I can see why there is the school of thought that expects developers to get a percentage of the take if it is a hit. I doubt the status quo will ever change without some organized effort on the part of video game developers.

The comments in response to Wil's post also contain some interesting viewpoints for and against his position. I was actually surprised that there was anything in his post to take offence at but people have a way of surprising me.

 

There's been a bunch of hubbub on certain blogs about MSN Spaces and some of the content filtering that happens on the site due to a recent Financial Times article entitled Microsoft bans 'democracy' for China web users. I've seen a lot of rhetoric about this topic and have avoided commenting on it since it is a sensitive topic that has evoked rather emotional and inflammatory responses from commenters including some Microsoft employees.

I will say two things though. First of all, the behavior of MSN Spaces isn't something that is tied to any recent ventures in the past month or two by MSN in China as the article purports. In December of last year Boing Boing ran a post entitled Chinese editions of MSN Spaces censor political terms which covers the behavior described in the Financial Times article.

The second is that the response to the initial feedback on the "censorship" on MSN Spaces made by Michael Connolly in his post Comments on Content Moderation is still valid. Specifically he wrote

There have been a lot of observations since we launched on how we moderate content on Spaces.   Just so there aren’t any misconceptions floating around, here is exactly what we do, and why.

One of our main goals for Spaces was to create a platform for people to share their thoughts and feelings with their friends and the outside world.  However, we wanted to make Spaces usable by not only the people who are blogging today, but also be approachable by the general internet user, who might not have heard of blogging previously, or been given an opportunity to try it out.

Unfortunately, whenever you create an open platform for people to say whatever they want, and open it up to the wide world (14 languages, in 26 different markets), there is always a handful of people who spoil the party, and post a bunch of inappropriate (and in some cases illegal) stuff. And to make matters worse, what exactly is deemed “appropriate” or not is very subjective, not only from person to person, but from country to country
...
We block a set of specific words from being used in 3 areas: the url you select, the title of your Space, and the title of your blog entry. These three fields are reused and displayed in a variety of areas, like search results, so we thought it would be a little thing we could do to cut down on the obvious cases that would most easily offend.

MC made his post in December of last year and this is still the state of affairs today. I don't know if any official statement will be made in response to the article but I did think it would add some perspective to the various discussions to actually get the facts and as opposed to hearsay.

Quite frankly I've been saddened see the kind of language and rhetoric used in posts like Tim Bray's Microsoft and China to describe the above situation. We have lots of Chinese users who use MSN Spaces to share their lives with friends, family and the rest of the online world. Read their blogs, view their photos and try to see things from their eyes instead of letting the rhetoric blind you to reality.

 

John Montgomery has a post entitled I’m Missing Something Here where he expresses similar sentiments to those expressed by Charles Fitzgerald in the C|Net article Will AJAX help Google Clean Up? on the increased hype around using DHTML and server callbacks (aka AJAX) for building web sites. Both senior Microsoft developer evangelists seemed to be saying "AJAX sucks, use .NET instead". Specifically

John Montgomery: "I’m not sure if AJAX is an amazing transformational technology, or simply the pinnacle of what you can do with Javascript. Nor am I sure that I wouldn’t have been better off writing a ClickOnce Windows Forms app for most of what I was doing."

Charles Fitzgerald: "It's a little depressing that developers are just now wrapping their heads around these things we shipped in the late 20th century. But XAML is in a whole other class. This other stuff is very kludgy, very hard to debug. We've seen some pretty impressive hacks, but if you look at what XAML starts to solve, it's a major, major step up."

The words attributed to Charles Fitzgerald are taken from a news article so they shouldn't be taken as verbatim statements though I assume that C|Net captured his sentiments accurately.

What the Microsoft evangelists are forgetting is that the Web is about reach. AJAX is about attempting to build rich internet applications while preserving the reach and zero deployment costs of the Web. It smacks of completely not understanding the problem space to suggest that sites like http://maps.google.com or http://www.start.com/myw3b should be using Winforms or XAML/Avalon instead of exploring AJAX.

I suspect it is a weird sort of tunnel vision. Spending so much time talking to developers building intranet applications makes some people believe that this is the same as web development. It isn't.

 

Shelley Powers has a few posts that are critical of Microsoft's InfoCard project entitled You Want We Should What? and What do you want from Digital Identity. I'm not really up to speed on all the digital identity and InfoCard discussion you can find in places like Kim Cameron's blog mainly because it bores me to tears. However one thing that struck me when reading Shelley's posts and then reading a few entries from Kim's blog is that it seemed they both were expecting different people to use the InfoCard technology.

I've found it hard to tell whether the identity folks at Microsoft expect InfoCard to mainly be used by Enterprises who need to identify people who communicate across identity domains (e.g. the way Office Communicator is used to communicate with people within an enterprise or folks using Yahoo!, MSN or AOL instant messaging tools) or whether they expect it to be used as some sort of federated single sign on system for various websites. Reading the Microsoft's Vision for an Identity Metasystem whitepaper it seems InfoCard and the "Identity Metasystem" are meant to do this and much more. My spider sense tends to tingle when I see v1 technologies that aim to solve such diverse problems in a broad sweep.

The end of the whitepaper states Passport and MSN plan to implement support for the identity metasystem as an online identity provider for MSN and its partners. Interesting, I may have to start digging into this space since it will eventually impact my day job. 

 

DHTML Lemmings^TM is a faithful recreation of the classic computer game, Lemmings. I remember having lots of fun playing this on my Amiga over a decade ago. This was one of the most innovative games of the '90s.

Too bad clever puzzlers like Lemmings don't seem to be made anymore.

 

The newest version of the MSN Search Toolbar adds tabbed browsing to Internet Explorer. You can check out a screenshot if you're curious to see what its like. Denise from the MSN Search Toolbar team blogged about it in her post Tabbed browsing is here! where she wrote

Tabbed browsing not only makes online shopping easier (and my wallet more broke), it also drastically improves my web search experience! Have you ever gotten annoyed with hitting the back button repeatedly to get back to the original search results?  MSN Search Toolbar gives you the option to open MSN Search results in new background tabs. This is a HUGE time saver because it lets you keep the search results on one tab while links that you click open in the background.

Best of all, Tabbed Browsing with MSN Search Toolbar works in Internet Explorer 5.01 or above! IE7 will also offer tabs but in the meantime this is a great option.

I think it's way cool that the MSN Search Toolbar team has added this functionality to IE. First Trixie and Turnabout, and now tabbed browsing added via the MSN Search Toolbar. It definitely goes to show exactly how extensible IE is as a platform. Of course, I still am interested in seeing what Dean and his merry band have planned for IE7.

 

 

About a year ago, the folks at Sun Microsystems came up with a bunch of benchmarks that showed that XML parsing in Java was much faster than on the .NET Framework. On the XML team at Microsoft we took this as a challenge to do much better in the next version of the .NET Framework. To see how much we improved, you can check out A comparison of XML performance on .NET 2.0 Beta2, .NET 1.1, and Sun Java 1.5 Platforms which is available on MSDN.

In the three test cases, Java 1.5 XML APIs are faster than those in the .NET Framework v1.1 both of which are about half as fast as the XML APIs in .NET Framework v2.0. The source code for the various tests is available so individuals can confirm the results for themselves on their own configurations. 

A lot of the improvements in XML parsing on the .NET Framework are due to the excellent work of Helena Kupkova. She is also the author of the excellent XmlBookMarkReader. Great stuff.

For the XML web services geeks there is also a similar comparison of XML Web Services Performance for .NET 2.0 Beta2, .NET 1.1, Sun JWSDP 1.5 and IBM WebSphere 6.0.

 

If you are a geek, you may have heard of the Firefox extension called GreaseMonkey which lets you to add bits of DHTML ("user scripts") to any web page to change its design or behavior. This basically enables you to remix the Web and either add features to your favorite web sites or fix broken UI design.

Earlier this week, there was a post to the Greasemonkey mailing list pointing out the existence of Turnabout. Below are some excerpts from the Turnabout website.  

What is Turnabout?

Turnabout is an Internet Explorer plugin that runs user scripts of your choice on any website. User scripts are like plugins for websites. They can enhance your web experience in a lot of ways:

• Block pop-ups
• Change text URLs into clickable links
• Add features, like adding custom search links in your Gmail account
• Enlarge text fields that are too small
• …And more!

Essentially, Turnabout does for IE what Greasemonkey does for Firefox .

So where does this leave the other recently announced Greasemonkey for Internet Explorer project, Trixie? Turnabout seems like a better bet for a couple of reasons. First of all, Turnabout doesn't require the .NET Framework like Trixie does. Secondly, Turnabout comes with source code but not with any licensing information which means it is not Open Source. Although Trixie's source code can be easily deciphered with Reflector, that technically is reverse engineering. Finally and most importantly, the developer of Trixie has stopped work on it now that Turnabout exists.

For now I'll be uninstalling Trixe and trying out Turnabout. I'm glad to see that Trixe inspired an even better project to get launched. REMIX the Web. 

 

Adam Bosworth has a blog post entitled AJAX reconsidered that hits right at the heart of some questions I've been asking myself about the renewed interest in using DHTML and server callbacks via XMLHttpRequest to build website applications. Adam writes

I've been thinking about why Ajax is taking off these days and creating great excitement when, at the time we originally built it in 1997 (DHTML) and 1997 (the XML over HTTP control) it had almost no take up.
...
First, the applications that are taking off today in Ajax aren't customer support applications per se. They are more personal applications like mail or maps or schedules which often are used daily. Also people are a lot more familiar with the web and so slowly extending the idiom for things like expand/collapse is a lot less threatening than it was then. Google Maps for example uses panning to move around the map and people seem to love it.

Secondly, the physics didn't work in 1997. A lot of Ajax applications have a lot of script (often 10 or 20,000 lines) and without broadband, the download of this can be extremely painful. With broadband and standard tricks for compressing the script, it is a breeze. Even if you could download this much script in 1997, it ran too slowly. Javascript wasn't fast enough to respond in real time to user actions let alone to fetch some related data over HTTP. But Moore's law has come to its rescue and what was sluggish in 1997 is often lightning quick today.

Finally, in 1997 or even in 1999 there wasn't a practical way to write these applications to run on all browsers. Today, with work, this is doable. It would be nice if the same code ran identically on Firefox, IE, Opera, and Safari, and in fact, even when it does, it doesn't run optimally on all of them requiring some custom coding for each one. This isn't ideal, but it is manageable.

I find the last point particularly interesting. If Web browsers such as Firefox had not cloned Microsoft's proprietary APIs in a way made it easy to write what were formerly IE specific applications in a cross-browser manner then AJAX wouldn't be the hip buzzword du jour. This brings me to Microsoft's next generation of technologies for building rich internet applications; Avalon and XAML.

A few months ago, C|Net ran an article entitled Will AJAX help Google Clean Up? In the article the following statement was attributed to a Microsoft representative

"It's a little depressing that developers are just now wrapping their heads around these things we shipped in the late 20th century," said Charles Fitzgerald, Microsoft's general manager for platform technologies. "But XAML is in a whole other class. This other stuff is very kludgy, very hard to debug. We've seen some pretty impressive hacks, but if you look at what XAML starts to solve, it's a major, major step up."

Based on how adoption of DHTML/AJAX occured over the past few years I suspect that Avalon/XAML will follow a similar path since the initial conditions are similar. If I am correct then even if Avalon/XAML is a superior technology to DHTML/AJAX (which I believe to be the case) it will likely be shunned on the Web due to lack of cross-browser interoperability but may flourish within homogenous intranets. This shunning will continue until suitable clones for the functionality of Avalon/XAML appears for other browsers. In which case as soon as some highly visible pragmatist adopts the technology then it will become acceptable. However it is unclear to me that cloning XAML/Avalon is really feasible especially if the technology is evolved at a regular pace as opposed to being let languish as Microsoft's DHTML/AJAX technologies have been. This would mean that Avalon/XAML would primarily be an intranet technology used by internal businesses applications and some early adopter websites as was the case with DHTML/AJAX. The $64,000 question for me is whether this is a desirable state of affairs for Microsoft and if not what should be done differently to prevent it?

Of course, this is all idle speculation on my part as I procrastinate instead of working on a whitepaper for work. 

Disclaimer: The aforementioned statements are my opinions and do not represent the intentions, thoughts, plans or strategies of my employer.  

 

A few years ago I decided to relocate to Seattle and take a job with Microsoft. One of the things I knew I was going to miss was the Atlanta night life and all the various clubs that played crunk music. As luck would have it, I moved here around the same time that music from the Dirty South started to gain national prominence. I'm still taken aback sometimes when I hear Lil Jon played at the average yuppie infested club or bar in downtown Seattle when back in my college days you had to go to clubs outside downtown Atlanta if you wanted to hear Oomp Camp, Lil Jon, Pastor Troy, Ying Yang Twins or any other crunk local acts.

Of course, not all the good crunk stuff gets radio airplay despite how much Lil Jon is being played these days. Below are some of the songs that have been getting regular play as part of my daily routine. Depending on where you live some of it is mainstream while some of it isn't. Enjoy.

1. Give Me That - Lil Boosie & Webbie (Featuring Bun B)
2. I Smoke I Drank (remix) - Body Head Bangerz (Featuring Youngbloodz)
3. Still Tippin' - Mike Jones (Featuring Slim Thug And Paul Wall)
4. Girlfight - Brooke Valentine (Featuring Lil Jon & Big Boi)
5. Some Cut - Trillville (Featuring Cutty)

If you have some suggestions for new tracks I should check out, post a comment below.

 

Since the recent announcement that the next version of Microsoft Office would move to open XML formats as the default file format in the next version, I've seen some questions raised about why the OpenOffice.org XML formats which were standardized with OASIS weren't used. This point is addressed in a comment by Jean Paoli in the article Microsoft to 'Open' Office File Formats which is excerpted below

"We have legacy here," Jean Paoli, Senior Microsoft XML Architect, told BetaNews. "It is our responsibility to our users to provide a full fidelity format. We didn't see any alternative; believe me we thought about it. Without backward compatibility we would have other problems."

"Yes this is proprietary and not defined by a standards body, but it can be used by and interoperable with others. They don't need Microsoft software to read and write. It is not an open standard but an open format," Paoli explained.

When asked why Microsoft did not use the OASIS (Organization for the Advancement of Structured Information Standards) OpenOffice.org XML file format, Paoli answered, "Sun standardized their own. We could have used a format from others and shoehorned in functionality, but our design needs to be different because we have 400 million legacy users. Moving 400 million users to XML is a complex problem."

There is also somewhat of a double standard at play here. The fact that we are Microsoft means that we will get beaten up by detractors no matter what we do. When Sun announced Java 1.5 5.0 with a feature set that looked a lot like those in C#, I don't remember anyone asking why they continued to invest in their proprietary programming language and platform instead of just using C# and the CLI which have been standardized by both ECMA and the ISO. If Microsoft had modified the OpenOffice.org XML file format so that it was 100% backwards compatible with the previous versions of Microsoft Office it is likely that same people would be yelling "embrace and extend". I'm glad the Office guys went the route they chose instead. Use the right tool for the job instead of trying to turn a screwdriver into a hammer.

It's a really powerful thing that the most popular Office productivity suite on the planet is wholeheartedly embracing open formats and XML. It's unfortunate that some want to mar this announcement with partisan slings and arrows instead of recognizing the goodness that will come from ending the era of closed binary document formats on the desktop.

 

The next version of MSN's experimental RSS reader is available at http://www.start.com/3. You have to solve the questions before getting access to the next version of the site but once that's done you get access to the juicy goodness.

This version of the site does a better job of letting you read posts inline. Another neat addition is the ability to drag and drop modules to arrange them on the page. Kudos to the the Steve Rider, Scott Isaacs and the rest of the folks who've worked on this.

 

Todd Bishop is a reporter on the Seattle Post Intelligencer who has a blog where he posts stories about Microsoft that didn't make it into the paper. In a recent post entitled, Microsoft eyes podcasts he writes

During the revival of the Gillmor Gang audio program over the weekend, podcasting pioneer Adam Curry said he had been trying for some time to persuade Microsoft and Apple to pay attention to the trend and build podcatching features inside Windows Media Player or the iTunes software. He said things changed when Steve Jobs demonstrated the new iTunes podcatching feature last week at the Wall Street Journal's "D: All Things Digital" conference. On the Gillmor Gang, Curry explained what happened next:

"It was like 15 minutes after it showed up in the Wall Street Journal when Microsoft called, saying, 'Hey, how do we get in this?' I don't know a lot about Microsoft. I do see they're a lot hungrier company than they used to be. But every single time you talk to them about anything that's new, or in this case iPodder functionality inside Windows Media Player, the almost standard answer is, 'Yeah we're going to have a lot of that in Longhorn.' That to me means there is this huge steamboat that is very difficult to steer left or right, and it's just harder to get stuff done at Microsoft."

That's just a snippet of an interesting discussion of the topic. You can download the full program via the Gillmor Gang link above.

I had lunch with someone from main campus yesterday and it seems that Adam Curry's perception is also being felt by the folks who work on the product teams. To repeat myself, I hope it doesn't take the stock hitting $10 before some action is taken.

 

I was supposed to post this a month and a half ago but somehow never got around to it. Jon Pincus who organized the session put up his notes on the 20% time session at CFP 2005. It was a pretty interesting discussion, some of which is excerpted below

Some examples that started as 20% projects at Google include GMail, Google Suggests , Firefox prefetching, and various technology pieces that are used throughout Google's code base. At Microsoft, many of the Windows Power Toys had a similar genesis, and attendees thought that Apple's iPod may have had similar origins in a "part-time, on-the-side" project.

One question that came up repeatedly (in various forms) is "how does the company know that people aren't just goofing off?" The short-term answer comes down to trust (that is, the assumption is that most employees will do their best to use this time in ways that give value to the company); in the longer term, contributions from 20% time projects are considered as part of reviews, and so if you've wasted the time, it will come out at that point. [It was pointed out that trust in general reduces "transaction costs", and so can increase efficiency -- so that even if (say) 10% of employees are accomplishing absolutely nothing with this time, it still could be a big win not to have any monitoring overhead which would slow down the other 90%.]

Another question that came up a couple of times was how to manage the impact on schedules -- does this mean that all dates wind up 20% further out than they would otherwise? I don't think there was an explicit answer to this in the meeting, but several people pointed out that a large chunk of many engineers' days at a lot of companies goes to various kinds of not-specifically-product-related tasks (e.g., reading Slashdot); if that "breakage" is diverted into time on a 20% project, then the actual hit is much smaller.

Most of the discussion was devoted to discussing the reasons that this approach appears to work well at Google.

I personally think that creating a framework that encourages developers at a company to work on side projects is very rewarding for both the employees and the company. I don't know how I'd have dealt with some of the issues I had with being a cog in a [relatively] slowly moving corporate wheel if I didn't have RSS Bandit to work on in my previous job.  Now that it is a mature project that has seen hundreds of thousands of downloads since it started, I often end up talking to product teams working with RSS at Microsoft in an adviser role. I got something out of it and Microsoft got something out of it.

One of the questions that kept coming up during the session was how one measured 20% time and made sure employees weren't goofing off. The answer was that it isn't really about taking a day off every week, its more about factoring in the time that the person will spend on their side project into their work schedule. After all, no one has 20% of their time free just before shipping and then there are times when weeks go by with not a lot of work to do. 20% time is more about allowing developer employees to reclaim some of that slack in ways that may or may not benefit the company but would make them more satisfied with their day jobs.

Joe Beda mentioned that the fact that Google was a web application built around web services enabled people to quickly write interesting applications utilizing the data and processing power of their back end. Now that I work on the back end of various services at MSN I can see his point. It would be very interesting to see the kind of applications built if we encouraged people to go wild with the APIs behind applications like MSN Search, MSN Spaces, MSN Messenger, Hotmail, MSN Groups, Start.com, MyMSN and see what they could build in their spare time.  In fact, I've already seen one cool hack built on top of MSN Virtual Earth by some internal folks and it isn't even out yet. This is one of those things I'll definitely be evangelizing at work. Wish me luck. :)

 

About two and half years ago, I was hanging out with several members of the Office team as they gave the details about how Office 2003 would support XML file formats at XML 2002. Now that it's 2005, juicy information like that is now transmitted using blogs.

Brian Jones has a blog post entitled New default XML formats in the next version of Office were he reveals some of the details of XML support in the next version of Office. He writes

  Open XML Formats Overview

To summarize really quickly what’s going on, there will be new XML formats for Word, Excel, and PowerPoint in the next version of Office, and they will be the default for each. Without getting too technical, here are some basic points I think are important:

1. Open Format: These formats use XML and ZIP, and they will be fully documented. Anyone will be able to get the full specs on the formats and there will be a royalty free license for anyone that wants to work with the files.
2. Compressed: Files saved in these new XML formats are less than 50% the size of the equivalent file saved in the binary formats. This is because we take all of the XML parts that make up any given file, and then we ZIP them. We chose ZIP because it’s already widely in use today and we wanted these files to be easy to work with. (ZIP is a great container format. Of course I’m not the only one who thinks so… a number of other applications also use ZIP for their files too.)
3. Robust: Between the usage of XML, ZIP, and good documentation the files get a lot more robust. By compartmentalizing our files into multiple parts within the ZIP, it becomes a lot less likely that an entire file will be corrupted (instead of just individual parts). The files are also a lot easier to work with, so it’s less likely that people working on the files outside of Office will cause corruptions.
4. Backward compatible: There will be updates to Office 2000, XP, and 2003 that will allow those versions to read and write this new format. You don’t have to use the new version of Office to take advantage of these formats. (I think this is really cool. I was a big proponent of doing this work)
5. Binary Format support: You can still use the current binary formats with the new version of Office. In fact, people can easily change to use the binary formats as the default if that’s what they’d rather do.
6. New Extensions: The new formats will use new extensions (.docx, .pptx, .xlsx) so you can tell what format the files you are dealing with are, but to the average end user they’ll still just behave like any other Office file. Double click & it opens in the right application.

...

Whitepapers

The Microsoft Office Open XML Formats: New File Formats for "Office 12"

http://download.microsoft.com/download/c/2/9/c2935f83-1a10-4e4a-a137-c1db829637f5/Office12NewFileFormatsWP.doc

The Microsoft Office Open XML Formats: Preview for Developers

http://download.microsoft.com/download/c/2/9/c2935f83-1a10-4e4a-a137-c1db829637f5/Office12FileFormatDevPreviewWP.doc

This is totally awesome news. I remember asking, back in 2002, why Powerpoint didn't have an XML file format and the answer was that it was due to schedule constraints but it would be fixed in the next version. Not only did the Office guys keep their word but they went above and beyond.

This should make Sam Ruby happy.

 

In a recent post entitled Dare responds re: open and closed systems Tim Ewald writes

Responding to my statement:

Dare was quick to blame WSDL and XSD, noting that POX/REST/AJAX systems don't have this problem. Certainly the customers I talk to who have done POX/REST systems are moving toward WSDL and XSD because they want metadata about services in order to facilitate reuse. Without it, they don't have a clear picture of what their systems are doing - or how to reuse them in other contexts. For a closed system, lack of descriptive metadata is fine. But if you want something open and reusable across apps, it really helps to have it

he called me on my description open and closed systems:

Interesting. So Yahoo! Web Services, the Flickr API, Bloglines API, del.icio.us API, and every RSS feed on the planet are closed systems that aren't open and reusable?

Yes, Dare, you are right. I concede that these very public APIs are open and reusable. I was really talking about systems *inside* a company (which I meant to imply with my reference to our customers). In many cases, developers start using POX over HTTP to build systems. When people want to re-purpose those services, its hard because they don't have a lot of information about what message formats and exchange patterns they support. In many cases there is no documentation for that, other than the code and, in that sense, those systems are closed. XSD and WSDL help open them up by providing metadata about what those services do. In some cases that metadata is also useful for finding services that do interesting things. For many of our customers, that is the reason they are migrating from existing POX over HTTP systems to SOAP-based Web services.

As part of my day job I'm in the position of having to work on both internal and external web services at the same time. I can totally see how internal services can end up not being documented well especially since they usually are written for one or two specific partners whose questions can be answered over the phone or email. In fact, I've been meaning to add code samples to one of our specs for over six months but have never gotten around to it because there's always a higher priority issue to work on.

On the other hand, an external web service could be accessed by dozens to hundreds of developers, many of whom will be unable to contact the creators of the service. In this case,  the onus is on the creators to either provide excellent documentation or use standard interfaces that are well documented elsewhere if they actually want anyone to use their service. 

In both situations, I wouldn't consider an XSD or WSDL file as being sufficient documentation. They are definitely better than nothing but they are a far cry from providing enough metadata for users of a service to determine how to properly interact with a service especially when dealing with operations that have to be performed in a series of steps (e.g. uploading a photo or enclosure to a user's home directory then attaching it to a subsequent blog entry).

If your organization is having problems with poorly documented internal services then the answer is to document them NOT to rely on WSDL & XSD files as documentation. As David Ing mentioned in a response to Tim's post   

- XSD/WSDL helps automate the contract side but doesn't really reduce documentation when compared to POX/H, in fact sometimes they increase the complexity up a notch too high. The tools aren't up to a state where you can hook up and just go - staring at the wire and talking on the phone are still facts of life in enterprise integrations. That's a side affect of the problem domain rather than the technology.

In summary, there is no substitute for good documentation. Anyone who tells you otherwise is most likely trying to sell you something.

 

A number of MSN Spaces users had to deal abusive comments from malicious users impersonating others on their weblogs. This is a problem we are well aware of and are examining various approaches to curtailing.

Recently on I found an entry on Jen's Space entitled Petition where one of our users who's fed up with malicious comments wrote

The following petition was made with regards to suggestions for further upgrades to MSN Spaces as pertains to security.

1. We would like to see the option to enter any URL in your comment removed and replaced with a field showing the URL of the commenter by default. This field should also be inactive and therefore not available for modification or the changing of the default URL. There have been several cases where individuals have left hateful and slanderous comments on other Spaces using the URL of another member to avoid being identified. This sort of thing should be stopped and we believe that the above recommendation will help.

2. As with MSN Messenger, the ability to have your Space public and still being able to block certain members from viewing or commenting on your Space would also be beneficial. There have been several members that went Private to avoid people they did not wish viewing or commenting in their Spaces.
   

If in agreement to the above suggested upgrades to MSN Spaces please sign your names below in the form of a comment so that we can pass this on to the powers that be. I think we can all agree that these two recommendations will help with some of the ridiculous problems that certain members have been forced to deal with.

We've considered both approaches to solving the malicious commenter problem but neither one seems like a complete solution. The main problem with the first request is that it doesn't handle the case where the commenter is a valid Passport user who doesn't have a Space. Unless we prevent people from being able to provide their own URL, name and email address then this solution wouldn't really work. Given that we already get complaints that requiring Passport to post is restrictive, this would seem even more problematic. The second request doesn't really work if the Space is public because all the malicious commenter has to do is sign out and then they have access to the Space even if their Passport account is blocked.

For now, the best solution is to create a private space and add the Passport accounts of the various people you want to access your Space to your permission list. We have excellent documentation about permission settings which should show how to make your space accessible only to specific people in your address book or your MSN Messenger contacts.

In the meantime, rest assured that we are working on solutions to the impersonation problem which allow users to trust that commenters are who they say they are while not limiting the expressivity of people who leave comments in a blog. 

I love the tough problems.