Early this morning I found out that the new CitySearch beta site uses Facebook Connect to allow Facebook users to sign-in to the site and bring their social network with them to CitySearch. Below is a screenshot of the sign-in experience

a user can choose sign-in with their Facebook credentials and if so they can bring their social network and profile info from Facebook to CitySearch and there is the option that some of their activities on CitySearch are republished to Facebook.

Here's what it looks like to view my friends page on CitySearch after signing in with my Facebook credentials

From the screenshot, I have five Facebook friends who have also associated their Facebook profile with CitySearch.

A feature like Facebook Connect is something I've always wanted to see implemented but I've always gotten hung up on the details. The first stumbling block is the chicken and egg problem. Connecting your identity on multiple sites is only valuable if all my friends are doing it as well. From the above screenshot only 5 out of my 472 have linked their accounts and three of them are Facebook employees who work on Facebook Connect. Despite that, I think Facebook could incentivize users to link their accounts if it improves their user experience. For example, CitySearch isn't currently picking up that I live in Seattle even though it pulled in my Facebook profile information.

Another problem I saw with this sort of technology is that sites would be reluctant to tie their user authentication to another site (echoes of the Microsoft Passport is Evil fallout from the turn of the century) especially given all the hype behind decentralized identity solutions like OpenID. I believe Facebook has gotten around this concern in a couple of ways. The first is that Facebook Connect provides concrete benefits of access to a user's mini-feed and thus prime real estate on the user's Facebook profile. Secondly, it allows sites to bootstrap their social networks off of one of the must successful social networking sites in the world. Finally, Facebook's brand doesn't have the "baggage" that other companies that have tried to play in this place have had to struggle against.

Kudos to my friend Mike Vernal and all the other folks at Facebook who worked on Connect. This is just plain brilliant. Much love.

Now Playing: The Game - Nice

Disclaimer: What follows are my personal impressions from investigating the community technology preview version of the Live Framwork (LiveFX). It is not meant to be an official description of the project from Microsoft, you can find that here.

At Microsoft's recent Professional Developer Conference, a new set of Web services called Live Framework (aka LiveFX) was unveiled. As I've spent the past year working on platforms for user experiences in Windows Live, I actually haven't been keeping up to date with what's going on in developer API land when it comes to programming against our services. So I decided to check out the Live Framework website and find out exactly what was announced.

What is it?

Although the main website is somewhat light on details, I eventually gleaned enough information from the Live Framework SDK documentation on MSDN to conclude that LiveFX consists of the following pieces

1. A Resource Model: A set of RESTful APIs for interacting with Live Mesh and Windows Live data.
2. Libraries: A set of libraries for the .NET Framework, Silverlight and Javascript for accessing the REST APIs.
3. The Live Operating Environment: A local web server that implements #1 above so it can be programmed against using #2 above.

The Scary Architecture Diagram

This diagram tries to capture all the ideas in LiveFX in a single image. I found it somewhat overwhelming and after learning more about LiveFX I consider it to be a rather poor way of conveying across its key concepts.  It doesn't help that this diagram is somewhat aspirational given that some key pieces of the diagram are missing from the current technology preview. 

The Resource Model

Live Mesh and Windows Live data is exposed by LiveFX as a set of resources identified by URIs which can be interacted with via the Atom Publishing Protocol (RFC 5023). Relationships between resources are exposed as hyperlinks between resources. The hierarchical data model currently exposed in the CTP is shown in the diagram below taken from the MSDN documentation.

It should be noted that although AtomPub is the primary protocol for interacting with resources in LiveFX, multiple serialization formats can be used to retrieve data from the service including Atom, RSS, Plain Old XML (POX), JSON or even binary XML.

Since LiveFX is a fully compliant implementation of the Atom Publishing Protocol, one can browse to the service document of a user's Mesh or other top level resource and traverse links to various Atom collections and feeds in the hierarchy. Below is a screenshot of the LiveFX resource browser showing the service document for a user's Mesh with the links to various child collections exposed as hyperlinks.

Besides supporting multiple serialization formats, there are a number of other features of LiveFX that separate it from a vanilla implementation of the Atom Publishing Protocol.

• Synchronization via FeedSync: Mesh resources in LiveFX can be synchronized using FeedSync (formerly Simple Sharing Extensions). FeedSync is a family of extensions to RSS and Atom that enables bidirectional synchronization of XML feeds and the resources they reference. However synchronization in LiveFX is based on a client/server model instead of a peer-to-peer model which means that instead of the server subscribing to changes from the client and vice versa, clients subscribe to changes from the server and then inform the server when they make a change. More information about how LiveFX implements FeedSync can be found here.

• Query Model and Linq to REST: LiveFX supports the same URI query parameters for paging, sorting, filtering and inline expansion of linked resources as other Microsoft cloud-based data APIs including ADO.NET Data Services (formerly Astoria), SQL Server Data Services and Windows Azure Storage services. One of the benefits of this consistency is that the ADO.NET client libraries can be used to perform Linq queries over the LiveFX data types using a technology that has been colloquially described as Linq to REST. For example, the following C# Linq query actually executes HTTP GET requests using the $filter parameter under the covers.

  MeshObject GetMeshObjectByTitle(string title)
         {
             MeshObject meshObject = (from mo in mesh.CreateQuery<MeshObject>()
                                      where mo.Resource.Title == title
                                      select mo).FirstOrDefault<MeshObject>();
  
             return meshObject;
         }

  The HTTP request this makes over the wire is

  GET https://user-ctp.windows.net/V0.1/Mesh/MeshObjects/{meshObjectID}/DataFeeds/?$filter=(Title eq ‘WhateverWasPassedIn’)

• Batch Requests via Resource Scripts: LiveFX supports batching using a construct known as resource scripts. Using resource scripts a developer can submit a single request which contains multiple create, retrieve, update and delete operations at once. A resource script consists of a control flow statement which can contain one or more control flow statements, web operation statements, synchronization statements, data flow constructs and data flow statements.  You can find out more about resource scripts by reading the document About Live Framework Resource Scripts on the LiveFX developer site.

• Resource Introspection via OPTIONS: One problem that commonly occurs in REST APIs is determining which operations a resource supports. Some protocols like OpenSocial specify a mechanism where HTTP responses should indicate which parameters are not supported. The problem with this approach is that the client has to first make an HTTP request then have it fail before determining if the feature is supported. LiveFX supports the HTTP OPTIONS verb on every resource. By performing the following HTTP request

  OPTIONS https:// user-ctp.windows.net/V0.1/{UserID}/Mesh/MeshObjects/{meshObjectID}

  a client can retrieve an introspection metadata document which describes what query parameters and serialization formats the resource supports. The $metadata query parameter can also be used to retrieve the introspection metadata document. This enables clients using libraries that don't support making HTTP OPTIONS requests to also be able to retrieve introspection metadata.

• Support for Portable Contacts: By specifying the parameter $type=portable when requesting contact data, the results will be returned in the Portable Contacts schema format as either JSON or Plain Old XML. 

The Libraries

Like most major Web vendors who have exposed REST APIs, Microsoft has provided client libraries to make interacting with the LiveFX service more natural for developers who aren't comfortable programming directly against HTTP. The following client libraries are provided

• A generic AtomPub client library for the .NET Framework. Learn more about programming with it here.
• A .NET Framework library which provides a high-level object model for interacting with the LiveFX service. More details here.
• A Javascript library which provides a high-level object model for interacting with the LiveFX service. Programming samples can be found here.

The Live Operating Environment

The Live Operating Environment refers to two things. The first is the Web platform upon which the LiveFX REST APIs are implemented. The second is a  local cache which runs on your PC or other device which exposes the same REST interface as the LiveFX Web service. This is somewhat similar to Google Gears except that the database is accessed RESTfully instead of via a SQL API.

The intent of the local version of the Live Operating Environment is to enable developers to be able to build apps that target the desktop or the Web without having to change their programming model. All that needs to be altered is changing the base URI from https://user-ctp.windows.net to http://localhost:2048 when accessing LiveFX resources. Everything else works exactly the same.

The Bottom Line

As the title of this blog post states there is a lot of similarity in concept between LiveFX and Google's Data APIs (GData). Like GData, LiveFX provides a consistent set of AtomPub based APIs for accessing resources from a particular vendor's online services along with a set of client libraries that wrap these RESTful APIs. And just like GData, there are Microsoft-specific extensions to the Atom syndication format and custom query parameters for sorting, filtering and paging.  LiveFX also supports batching like GData, however from my perspective adding batching to a Web service seems like an attempt to reinvent distributed transactions. This is a bad idea given the flaws of distributed transactions that are well discussed in Pat Helland's excellent paper Life beyond Distributed Transactions: An Apostate's Opinion. 

A number of LiveFX's additional features such as synchronization and resource introspection which have no analog in GData are fairly interesting and I wouldn't be surprised to see these ideas get further traction in the industry. On the flip side, the client-side Live Operating Environment is a technology whose benefits elude me. I admit it is kind of cool but I can't see its utility.

Now Playing: John Legend - Green Light (feat. Andre 3000)

About a year ago I wrote up a definition of a social operating system in my post The Difference between a Social Network Site, a Social Graph Application and a Social OS which I think is worth revisiting today. In that post I defined a Social OS as

Social Operating System: These are a subset of social networking sites. In fact, the only application in this category today is Facebook.  Before you use your computer, you have to boot your operating system and every interaction with your PC goes through the OS. However instead of interacting directly with the OS, most of the time you interact with applications written on top of the OS. Similarly a Social OS is the primary application you use for interacting with your social circles on the Web. All your social interactions whether they be hanging out, chatting, playing games, watching movies, listening to music, engaging in private gossip or public conversations occurs within this context. This flexibilty is enabled by the fact that the Social OS is a platform that enables one to build various social graph applications on top of it.

In retrospect, the fundamental flaw with this definition is that it encourages services that want to become social operating systems to aspire to become walled gardens. The problem with walled gardens on the Web is that they shortchange users. This is because the Web is about sharing and communicating with people from all over the world while walled gardens are about limiting you to interacting with people (and content) that are part of a particular online service or Web site. Thus walled gardens limit their users.

Jeremy Zawodny had a great post about this entitled There is no Web Operating System (or WebOS) where he wrote

Luckily, two of my coworkers caught on to what I was saying and managed to help put it into context a bit. First off was Matt McAlister (who runs YDN, the group I work in). In The Business of Network Effects he does a good job of explaining how businesses and services in a network are fundamentally different from those which are isolated islands.

Recalling a brief conversation we had a couple weeks ago, he says:

Jeremy Zawodny shed light on this concept for me using building construction analogies.

He noted that my building contractor doesn't exclusively buy Makita or DeWalt or Ryobi tools, though some tools make more sense in bundles. He buys the tool that is best for the job and what he needs.

My contractor doesn't employ plumbers, roofers and electricians himself. Rather he maintains a network of favorite providers who will serve different needs on different jobs.

He provides value to me as an experienced distribution and aggregation point, but I am not exclusively tied to using him for everything I want to do with my house, either.

Similarly, the Internet market is a network of services. The trick to understanding what the business model looks like is figuring out how to open and connect services in ways that add value to the business.

Bingo.

The web is a marketplace of services, just like the "real world" is. Everyone is free to choose from all the available services when building or doing whatever it is they do. The web just happens to be a far more efficient marketplace than the real world for many things. And it happens to run on computers that each need an operating system.

But nobody ever talks about a "Wall Street Operating System" or a "Small Business Operating System" do they? Why not?

Ian Kennedy followed up to Matt's post with The Web as a Loose Federation of Contractors in which he says:

I like Jeremy's illustration - an OS gives you the impression of an integrated stack which leads to strategies which favor things like user lock-in to guarantee performance and consistency of experience. If you think of the web as a loose collections of services that work together on discreet projects, then you start to think of value in other ways such as making your meta-data as portable and accessible as possible so it can be accessed over and over again in many different contexts.

Bingo again.

No matter how popular a particular website becomes it will not be the only service used by its customers. So it follows that no matter how popular a social networking site becomes, it will not be the only social networking service used by its customers or their friends. Thus a true Social Operating System shouldn't be about creating a prettier walled garden than your competitors but instead about making sure you can bring together all of a user's social experiences together regardless of whether they are on your site or on those of a competing service. If I use Twitter and my wife doesn't, I'd like her to know what I'm doing via the service even though she isn't a Twitter user. If my friends use Yelp to recommend restaurants in the area, I'd like to find out about the restaurants even though I'm not a Yelp user. And so on.

With the latest release of Windows Live, we're working towards bringing this vision one step closer to reality. You can read more about it in the official announcement and the accompanying blog post. I guess the statement "life without walls" also applies to Windows Live.

Now Playing: T-Pain - Chopped N Skrewed (Feat. Ludacris)

From the Microsoft press release Microsoft Introduces Updated Windows Live Service we learn

REDMOND, Wash. — Nov. 12, 2008 — Microsoft Corp. today announced the next generation of Windows Live, an integrated set of online services that make it easier and more fun for consumers to communicate and share with the people they care about most. The new generation of Windows Live includes updated experiences for photo sharing, e-mail, instant messaging, as well as integration with multiple third-party sites. The release also includes Windows Live Essentials, free downloadable software that enhances consumers’ Windows experience by helping them simplify and enjoy digital content scattered across their PC, phone and on Web sites. For more information about windows live go to http://www.windowslive.com.

Consumers today are creating online content and sharing it in many places across the Web. To help make is simple for the more than 460 million Windows Live customers to keep their friends up to date, Microsoft is collaborating with leading companies including Flickr, LinkedIn Corp., Pandora Media Inc., Photobucket Inc., Twitter, WordPress and Yelp Inc. to integrate activities on third-party sites into Windows Live through a new profile and What’s New feed. The new Windows Live also gives consumers the added convenience of having a central place to organize and manage information.

It's really exciting to know that hundreds of millions of people will soon be able to take advantage of what we've been thinking about and working on over the past year. I plan to hold my tongue until everyone can play with the new version themselves. For now I'll leave some links and screenshots from various blogs showing our baby off.

From TechCrunch: Sweeping Changes At Live.com: It’s A Social Network!

From Windows Live Wire:Windows Live – Keeping your life in sync

From Mary Jo Foley on All About Microsoft: Windows Live Wave 3: Microsoft’s kinder and simpler consumer services strategy?

From Kara Swisher on BoomTown: Microsoft Officially Facebooks, Oops, Socializes Windows Live Internet Services

Now Playing: Terence Trent D'Arby - Sign Your Name

I remember taking an operating systems class in college and marveling at the fact that operating system design seemed less about elegant engineering and more about [what I viewed at the time as] performance hacks. I saw a similar sentiment recently captured by Eric Florenzano in his post It's Caches All the Way Down where he starts describing how a computer works to a friend and ends up talking about the various layers of caching from CPU registers to L2  caches to RAM and so on.

At the end of his post Eric Florenzano asks the following question I've often heard at work and in developer forums like programming.reddit

That's what struck me. When you come down to it, computers are just a waterfall of different caches, with systems that determine what piece of data goes where and when. For the most part, in user space, we don't care about much of that either. When writing a web application or even a desktop application, we don't much care whether a bit is in the register, the L1 or L2 cache, RAM, or if it's being swapped to disk. We just care that whatever system is managing that data, it's doing the best it can to make our application fast.

But then another thing struck me. Most web developers DO have to worry about the cache. We do it every day. Whether you're using memcached or velocity or some other caching system, everyone's manually moving data from the database to a cache layer, and back again. Sometimes we do clever things to make sure this cached data is correct, but sometimes we do some braindead things. We certainly manage the cache keys ourselves and we come up with systems again and again to do the same thing.

Does this strike anyone else as inconsistent? For practically every cache layer down the chain, a system (whose algorithms are usually the result of years and years of work and testing) automatically determines how to best utilize that cache, yet we do not yet have a good system for doing that with databases. Why do you think that is? Is it truly one of the two hard problems of computer science? Is it impossible to do this automatically? I honestly don't have the answers, but I'm interested if you do.

Eric is simultaneously correct and incorrect in his statements around caching and database layers. Every modern database system has caching mechanisms that are transparent to the developer. For LAMP developers there is the MySQL Query Cache which transparently caches the text of SELECT query and the results so that the next time the query is performed it is fetched from memory. For WISC developers there are the SQL Server's Data and Procedure caches which store query plans and their results to prevent having to repeatedly perform expensive computations or go to disk to fetch recently retrieved data. As with everything in programming, developers can eke more value out of these caches by knowing how they work. For example, using parameterized queries or stored procedures significantly reduces the size of the procedure cache in SQL Server. Tony Rogerson wrote an excellent post where he showed how switching from SQL queries based on string concatenation to parameterized queries can reduce the size of a procedure cache from over 1 Gigabyte to less than 1 Megabyte. This is similar to understanding how garbage collection or memory allocation works teaches developers to favor recycling objects instead of creating new ones and favoring arrays over linked lists to reduce memory fragmentation.

Even though there are caching mechanisms built into relational databases that are transparent to the developer, they typically aren't sufficient for high performance applications with significant amounts of read load. The biggest problem is hardware limitations. A database server will typically have twenty to fifty times more hard drive storage capacity than it has memory. Optimistically this means a database server can cache about 5% to 10% of its entire data in memory before having to go to disk. In comparison, lets look at some statistics from a popular social networking site (i.e. Facebook)

• 48% of active users log-in daily
• 49.9 average pages a day viewed per unique user

How much churn do you think the database cache goes through if half of the entire user base is making data requests every day? This explains why Facebook has over 400 memcached hosts storing over 5 Terabytes of data in-memory instead of simply relying on the query caching features in MySQL. This same consideration applies for the majority of social media and social networking sites on the Web today.

So the problem isn't a lack of transparent caching functionality in relational databases today. The problem is the significant differences in the storage and I/O capacity of memory versus disk in situations where a large percentage of the data set needs to be retrieved regularly.  In such cases, you want to serve as much data from memory as possible which means going beyond the physical memory capacity of your database server and investing in caching servers.

Now Playing: Kanye West - Two Words (feat. Mos Def, Freeway & The Harlem Boys Choir)

At the recent Microsoft Professional Developer's Conference there was a presentation by Anders Hejlsberg about The Future of C# which unveiled some of the upcoming features of C# 4.0. Of all the features announced, the one I found most interesting was the introduction of duck typing via a new static type called dynamic.

For those who aren't familiar with Duck Typing, here is the definition from Wikipedia

In computer programming, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class. The name of the concept refers to the duck test, attributed to James Whitcomb Riley (see History below), which may be phrased as follows:

If it walks like a duck and quacks like a duck, I would call it a duck.

In duck typing one is concerned with just those aspects of an object that are used, rather than with the type of the object itself. For example, in a non-duck-typed language, one can create a function that takes an object of type Duck and calls that object's walk and quack methods. In a duck-typed language, the equivalent function would take an object of any type and call that object's walk and quack methods. If the object does not have the methods that are called then the function signals a run-time error.

This is perfectly illustrated with the following IronPython program

def PrintAt(container, index):
    print "The value at [%s] is %s" % (index, container[index])

if __name__ == "__main__":
    #create dictionary

    table = {1 : "one", 2 : "two",3 : "three"}
    #create sequence

    list = ("apple", "banana", "cantalope")

    PrintAt(table, 1)
    PrintAt(list, 1)

In the above program the PrintAt() simply requires that the container object passed in supports the index operator '[]' and can accept whatever type is passed in as the index. This means I can pass both a sequence (i.e. a list) or a dictionary (i.e. a hash table) to the function and it returns results even though the semantics of using the index operator is very different for lists and dictionaries.

Proponents of static typing have long argued that features like duck typing lead to hard-to-find bugs which are only detected at runtime after the application has failed instead of during development via compiler errors. However there are many situations even in statically typed programming where the flexibility of duck typing would be beneficial. A common example is invoking JSON or SOAP web services and mapping these structures to objects. 

Recently, I had to write some code at work which spoke to a JSON-based Web service and struggled with how to deal with the fact that C# requires me to define the class of an object up front before I can use it in my application. Given the flexible and schema-less nature of JSON, this was a problem. I ended up using the JsonReaderWriterFactory to create an XmlDictionaryReader which maps JSON into an XML document which can then be processed flexibly using XML technologies. Here's what the code looked like

using System;
using System.IO; 
using System.Runtime.Serialization.Json;
using System.Text;
using System.Xml; 

namespace Test
{
    class Program
    {
        static void Main(string[] args)
        {
            string json = @"{ ""firstName"": ""John"", ""lastName"": ""Smith"", ""age"": 21 }";
            var stream = new MemoryStream(ASCIIEncoding.Default.GetBytes(json));
            var reader = JsonReaderWriterFactory.CreateJsonReader(stream, XmlDictionaryReaderQuotas.Max); 

            var doc = new XmlDocument(); 
            doc.Load(reader);
            
            //error handling omitted for brevity
            string firstName = doc.SelectSingleNode("/root/firstName").InnerText; 
            int age          = Int32.Parse(doc.SelectSingleNode("/root/age").InnerText); 
            
            Console.WriteLine("{0} will be {1} next year", firstName, age + 1);
            Console.ReadLine();  
        }
    }
}

It works but the code is definitely not as straightforward as interacting with JSON from Javascript. This is where the dynamic type from C# 4.0 would be very useful. With this type, I could rewrite the above code as follows

using System;
using System.IO; 
using System.Runtime.Serialization.Json;
using System.Text;
using System.Xml; 

namespace Test
{
    class Program
    {
        static void Main(string[] args)
        {
            string json = @"{ ""firstName"": ""John"", ""lastName"": ""Smith"", ""age"": 21 }";
            var stream = new MemoryStream(ASCIIEncoding.Default.GetBytes(json));
            dynamic person = JsonObjectReaderFactory.CreateJsonObjectReader(stream, XmlDictionaryReaderQuotas.Max); 

            Console.WriteLine("{0} will be {1} next year", person.firstName, person.age + 1);
            Console.ReadLine();  
        }
    }
}

In the C# 4.0 version I can declare a person object whose class/type I don't have to define at compile time. Instead the property accesses are converted to dynamic calls to the named properties using reflection by the compiler. So at runtime when my [imaginary] JsonObjectReader dynamically creates the person objects from the input JSON, my application works as expected with a lot less lines of code.

It's amazing how Python-like C# gets each passing year.

FURTHER READING:

• Does C# 3.0 Beat Dynamic Languages at their Own Game?
• Functional Programming in C# 3.0: How Map/Reduce/Filter can Rock your World

The more I learn about OpenID, the more it reminds me of XML in that it is a suboptimal solution to the primary problem people are using it to solve. Recently I've seen OpenID geeks berating large vendors like Microsoft, Yahoo and Google for the way they've incorrectly implemented OpenID. Looking at these situations it has become clear that the problem OpenID solves is actually different from the problem these companies are trying to solve.

What OpenID Gives Us

So what is the problem that OpenID solves? For this I'll go back to one of the original emails on OpenID from it's founder Brad Fitzpatrick way back in 2005. In that email Brad wrote

So LiveJournal for user "bob" would positively assert the follow root URLs as being owned by bob:

http://www.livejournal.com/users/bob/
http://www.livejournal.com/~bob/
http://bob.livejournal.com/

And for those geeks out there with their own domain names (yes, I'm one, and you're one, but we're not the common case), you either run your own identity server, or you use somebody else's that's paranoid. For example, LiveJournal's would only assert off-site URLs which come to us with a rel="me" type of deal (not using XFN) as we currently do,with ljuser_sha1=9233b6f5388d6867a2a7be14d8b4ba53c86cfde2

OpenID is all about confirming that a user "owns" a URI. The expectation is that users have a URI that they have memorized and enter it into sites that support OpenID (i.e. Relying Parties) so that they can be authenticated. This breaks down in a number of situations.

• Users of email services don't have a URI that they own, they have an email address.
• Some users of social networking sites may have a URI but they don't have it memorized. for example I don't know anyone who has memorized the URL to their Facebook profile. Mine is http://www.facebook.com/p/Dare_Obasanjo/500050028 for anyone interested.

As different Web vendors have begun implementing OpenID, they've began to run into situations where OpenID's design breaks down for their users. Unsurprisingly a number of OpenID proponents have described these issues as usability issues as opposed to design problems.

Google's OpenID Woes

The most notorious example of a mainstream vendor colliding with the design mismatch between OpenID and their target domain is Google's implementation of OpenID. In a post entitled Google Abandons Standards, Forks OpenID we have the following excerpt

whatever it is that Google has released support for, it sure as hell isn’t OpenID, as they even so kindly point out in their OpenID developer documentation (that media outlets certainly won’t be reading):

1. The web application asks the end user to log in by offering a set of log-in options, including Google.
2. The user selects the "Sign in with Google" option.
3. The web application sends a "discovery" request to Google to get information on the Google authentication endpoint. This is a departure from the process outlined in OpenID 1.0. [Emphasis added]
4. Google returns an XRDS document, which contains endpoint address.
5. The web application sends a login authentication request to the Google endpoint address.
6. This action redirects the user to a Google Federated Login page.

As Google points out, this isn’t OpenID. This is something that Google cooked up that resembles OpenID masquerading as OpenID since that’s what people want to see – and that’s what Microsoft announced just the day before.

It’s not just a “departure” from OpenID, it’s a whole new standard.

With OpenID, the user memorizes a web URI, and provides it to the sites he or she would like to sign in to. The site then POSTs an OpenID request to that URI where the OpenID backend server proceeds to perform the requested authentication.

In Google’s version of the OpenID “standard,” users would enter their @gmail.com email addresses in the OpenID login box on OpenID-enabled sites, who would then detect that a Google email was entered. The server then requests permission from Google to use the OpenID standard in the first place by POSTing an XML document to Google’s “OpenID” servers. If Google decides it’ll accept the request from the server, it’ll return an XML document back to the site in question that contains a link to the actual OpenID URI for the email account in question.

The key problem Google had with their OpenID implementation is that they want their users to log-in with information they know (i.e. their gmail address) instead of some made up URL that is their Google OpenID identifier. Since only the Über geeks in their user base would know their Google OpenID URL if they went with that approach. With Google's approach meant that only sites that had been hardcoded to support Google's flavor of OpenID would be supported.

Thankfully, the outcry from the OpenID community was heard by the folks at Google and they've reversed their decision. Instead they will support OpenID discovery on gmail.com and so users just have to know that domain name and then the OpenID authentication process kicks off from there. The intent is for Google's OpenID implementation to eventually mirror the functionality of Yahoo's OpenID implementation. 

It Isn't Over Yet

It should be noted that as Google aspires to have an OpenID implementation that is as compliant as Yahoo's, OpenID experts have also criticized Yahoo's implementation. Specifically Sam Ruby in his post OpenId Minus Id Equals Wide Open writes

Martin Atkins: Yahoo!'s OP and now it seems Microsoft’s OP both ignore the value of openid.identity provided to them, and just return an assertion for whatever user’s logged in.

I may ultimately need to black-list such ids.

Looking at live.com instructions:

At any Web site that supports OpenID 2.0, type openid.live-INT.com in the OpenID login box to sign in to that site by means of your Windows Live ID OpenID alias.

If everybody uses the same URI, I can’t tell them apart.  That doesn’t concern me much, but do find it a bit distressing that that’s the recommended usage.

The problem goes back to the original role of OpenID as being a way for a user to claim that they own a URI. As Sam points out, some of the techniques being used to "improve usability" now run counter to the spirit of the original specification. 

Only time will tell if by the time all is said and done, it won't seem like we've been trying to shove a square peg in a round hole all these years.

Now Playing: Kanye West - Heartless

It feels good to realize that I can tell him that he can grow up to be president with a straight face. Congratulations to Team Obama.

Now Playing: Young Jeezy - My President (Feat. Nas)

Disclaimer: What follows are my personal impressions from using the beta version of Windows Azure. It is not meant to be an official description of the project from Microsoft, you can find that here. 

Earlier this week I scored an invite to try out the beta version of Windows Azure which is a new hosted services (aka cloud computing) platform from Microsoft. Since there's been a ridiculous amount of press about the project I was interested in actually trying it out by developing and deploying some code using this platform and sharing my experiences with others.

What is it?

Before talking about a cloud computing platform, it is useful to agree on definitions of the term cloud computing. Tim O'Reilly has an excellent post entitled Web 2.0 and Cloud Computing where he breaks the technologies typically described as cloud computing into three broad categories

1. Utility Computing: In this approach, a vendor provides access to virtual server instances where each instance runs a traditional server operating system such as Linux or Windows Server. Computation and storage resources are metered and the customer can "scale infinitely" by simply creating new server instances. The most popular example of this approach is Amazon EC2.
2. Platform as a Service: In this approach, a vendor abstracts away the notion of accessing traditional LAMP or WISC stacks from their customers and instead provides an environment for running programs written using a particular platform. In addition, data storage is provided via a custom storage layer and API instead of traditional relational database access. The most popular example of this approach is Google App Engine.
3. Cloud-based end user applications: This typically refers to Web-based applications that have previously been provided as desktop or server based applications. Examples include Google Docs, Salesforce and Hotmail. Technically every Web application falls under this category, however the term often isn't used that inclusively.

With these definitions clearly stated it is easier to talk about what Windows Azure is and is not. Windows Azure is currently #2; a Platform as a Service offering. Although there have been numerous references to Amazon's offerings both by Microsoft and bloggers covering the Azure announcements, Windows Azure is not a utility computing offering [as defined above].

There has definitely been some confusion about this as evidenced by Dave Winer's post Microsoft's cloud strategy? and commentary from other sources.

Getting Started

To try out Azure you need to be running Windows Server 2008 or Windows Vista with a bunch of prerequisites you can get from running the Microsoft Web Platform installer. Once you have the various prerequisites installed (SQL Server, IIS 7, .NET Framework 3.5, etc) you should then grab the Windows Azure SDK. Users of Visual Studio will also benefit from grabbing the Windows Azure Tools for Visual Studio.

After this process, you should be able to fire up Visual Studio and see the option to create a Cloud Service if you go to File->New->Project.

Building Cloud-based Applications with Azure

The diagram below taken from the Windows Azure SDK shows the key participants in a typical Windows Azure service

The work units that make up a Windows Azure hosted service can have one of two roles. A Web role is an application that listens for and responds to Web requests while a Worker role is a background processing task which acts autonomously but cannot be accessed over the Web. A Windows Azure application can have multiple instances of Web and Worker roles that make up the service. For example, if I was developing a Web-based RSS reader I would need a worker role for polling feeds and Web role for displaying the UI that the user interacts with. Both Web and Worker roles are .NET applications that can be developed locally and then deployed on Microsoft's servers when they are ready to go.

Azure applications have access to a storage layer that provides the following three storage services

• Blob Storage: This is used for storing binary data. A user account can have one or more containers which in turn can contain one or more blobs of binary data. Containers cannot be nested so one cannot create hierarchical folder structures. However Azure allows applications to work around this by (i) allowing applications to query containers based on substring matching on prefixes and (ii) delimiters such as '\' and other path characters are valid blob names. So I can create blobs with names like 'mypics\wife.jpg' and 'mypics\son.jpg' in the media container and then query for blobs beginning with 'mypics\' thus simulating a folder hierarchy somewhat. 

• Queue Service: This is a straightforward message queuing service. A user account can have one or more queues from which they can add items to the end of each queue and remove items from the front. Items have a maximum time-to-live of 7 days within the queue. When an item is retrieved from the queue, an associated 'pop receipt' is provided. The item is then hidden from other client applications until some interval (by default 30 seconds) has passed after which the item becomes visible. The item can be deleted from the queue during that interval if the pop receipt from when it was retrieved is provided as part of the DELETE operation.  The queue service is valuable as a way for Web roles to talk to Worker roles and vice versa.

• Table Storage: This exposes a subset of the capabilities of the ADO.NET Data Services Framework (aka Astoria). In general, this is a schema-less table based model similar to Google's BigTable and Amazon's SimpleDB. The data model consists of tables and entities (aka rows). Each entity has a primary key made of two parts {PartitionKey, RowKey}, a last modified timestamp and an arbitrary number of user-defined properties. Properties can be one of several primitive types including integer, strings, doubles, long integers, GUIDs, booleans and binary. Like Astoria, the Table Storage service supports performing LINQ queries on rows but only supports the FROM, WHERE and TAKE operators. Other differences from Astoria are that it doesn't support batch operations nor is it possible to retrieve individual properties from an entity without retrieving the entire entity.

These storage services are accessible to any HTTP client and not just Azure applications. 

Deploying Cloud-based Applications with Azure

The following diagram taken from the Windows Azure SDK shows the development lifecycle of an Windows Azure application

The SDK ships with a development fabric which enables you to deploy an Azure an application locally via IIS 7.0 and development storage which uses SQL Server Express as a storage layer which mimics the Windows Azure storage services.

As the diagram shows above, once the application is tested locally it can be deployed entirely or in part on Microsoft's storage and cloud computation services.

The Azure Services Platform: Windows Azure + Microsoft's Family of REST Web Services

In addition to Windows Azure, Microsoft also announced the Azure Services Platform which is a variety of Web APIs and Web Services that can be used in combination with Windows Azure (or by themselves) to build cloud-based applications. Each of these Web services is worthy of its own post (or whitepaper and O'Reilly animal book) but I'll limit myself to one sentence descriptions for now.

• Live Services: A set of REST APIs for consumer-centric data types (e.g. calendar, profile, etc) and scenarios (communication, presence, sync, etc). You can see the set of APIs in the Live Framework poster and keep up with the goings on by following the Live Services blogs.

• Microsoft SQL Services: Relational database in the cloud accessible via REST APIs. You can learn more from the SSDS developer center and keep up with the goings on by following the SQL Server Data Services team blog.

• Microsoft .NET Services: Three fairly different services for now; hosted access control, hosted workflow engine and a service bus in the cloud. Boring enterprise stuff. :) 

• Microsoft Sharepoint Services: I couldn't figure out if anything concrete was announced here or whether stuff was pre-announced (i.e. actual announcement to come at a later date).

• Microsoft Dynamics CRM Services: Ditto.

From the above list, I find the Live Services piece (access to user data in a uniform way) and the SQL Services (hosted storage) most interesting. I will likely revisit them in more depth at a later date.

The Bottom Line

From my perspective, Windows Azure is easiest viewed as a competitor to Google App Engine. As comparisons go, Azure already brings a number of features to the table that aren't even on the Google App Engine road map. The key important feature is the ability to run background tasks instead of just being limited to writing applications that respond to Web requests. This limitation of App Engine means you can't write any application that does any serious background computation like a search engine, email service, or RSS reader on Google App Engine. So Azure can run an entire class of applications that are simply not possible on Google App Engine.

The second key feature is that by supporting the .NET Framework, developers theoretically get a plethora of languages to choose from including Ruby (IronRuby), Python (IronPython), F#, VB.NET and C#. In practice, the Azure SDK only supports creating cloud applications using C# and VB.NET out of the box. However I can't think of any reason why it shouldn't be able to support development with other .NET enabled languages like IronPython. On the flipside, App Engine only supports Python and the timeline for it supporting other languages [and exactly which other languages] is still To Be Determined.

Finally, App Engine has a number of scalability limitations both from a data storage and a query performance perspective. Azure definitely does better than App Engine on a number of these axes. For example, App Engine has a 1MB limit per file while Azure has a 64MB limit on individual blobs and also allows you to split a blob into blocks of 4MB each. Similarly, I've been watching SQL Server Data Services (SSDS) for a while and I haven't seen or heard complaints about query performance.

Azure makes it possible for me to reuse my existing skills as a .NET developer who is savvy with using RESTful APIs to build cloud based applications without having to worry about scalability concerns (e.g. database sharding, replication strategies, server failover, etc). In addition, it puts pressure on competitors to step up to the plate and deliver. However you look at it, this is a massive WIN for Web developers.

The two small things I'd love to see addressed are first class support for IronPython and some clarity on the difference between SSDS and Windows Azure Storage services. Hopefully we can avoid a LINQ to Entities vs. LINQ to SQL-style situation in the future.

Postscript: Food for Thought

It would be interesting to read [or write] further thoughts on the pros and cons of Platform as a Service offerings when compared to Utility Computing offerings. In a previous discussion on my blog there was some consensus that utility computing approaches are more resistant to vendor lock-in than platform as a service approaches since it is easier to find multiple vendors who are providing virtual servers with LAMP/WISC hosting than it will be to find multiple vendors providing the exact same proprietary cloud APIs as Google, Amazon or Microsoft. However it would be informative to look at the topic from more angles, for instance what is the cost/benefit tradeoff of using SimpleDB/BigTable/SSDS for data access instead of MySQL running on multiple virtual hosts? With my paternity leave ending today, I doubt I'll have time to go over these topics in depth but I'd appreciate reading any such analysis.

Now Playing: The Game - Money

UPDATE: This release contained a bug which caused users to lose their feed lists on upgrading from previous versions. Please download v1.8.0.862 from here instead.

I've been talking about it for almost a year and now we've finally shipped it. To celebrate the new release, we've created a new website to capture the awesomeness of the latest version.

Download the installer from here

NEW FEATURES 
- Download manager for viewing pending podcasts/enclosures
- Ability to synchronize feeds with Google Reader, NewsGator Online and the Windows Common Feed List.
- Option to increase or decrease font size in reading pane

BUG FIXES 
- 2153584 FTP storage ignores path, always stores in root directory
- 1987359 System crashes while updating feeds
- 1974485 NewsItem copy content to clipboard only copy the link
- 1971987 Error on start: docs out of order (0 < 6 )
- 1967898 Application hangs on shut down due to search indexing
- 1965832 Exception when refreshing feeds
- 2014927 New feeds not downloaded
- 1933253 Continuously Downloading Enclosures
- 1883614 Javascript in feed content causes exceptions
- 1899299 Oldest unread message shown on alert window on new items
- 1915756 Program crash during close operation
- 1987359 System crashes while updating feeds
- 2096903 Toolbar crashes and replaced with Red X
- 1960767 Crashes on opening due to FIPS policy set on Windows Vista.

Now Playing: Young Jeezy - Vacation