Cory Doctorow has a blog post up on Boing Boing entitled Mark Pilgrim's list of Ubuntu essentials for ex-Mac users where he writes

Mac guru and software developer Mark Pilgrim recently switched to Ubuntu Linux after becoming fed up with proprietary Mac file-formats and the increasing use of DRM technologies in the MacOS. I've been a Mac user since 1984, and have a Mac tattooed on my right bicep. I've probably personally owned 50 Macs, and I've purchased several hundred while working as an IT manager over the years. I'm about to make the same switch, for much the same reasons.

You could probably write an entire Ph.D dissertation on what would motivate someone to tattoo a corporate logo on their arm. Maybe I should buy a Mac just so I can figure out what all the hype is about.

 

It seems the Web API authentication discussion has been sparked up all over the Web by the various announcements of Windows Live ID and the Google Account Authentication for Web apps . In his blog post Google's authentication vs. Microsoft's Live ID Eric Norlin writes

Recent announcements of Google's authentication service have prompted comparisons to Passport, and even gotten to Dick Hardt (of "Identity 2.0" fame) to call it the, "deepening of the identity silo." I'd like to contrast Google's work with Microsoft's recent work around Live ID.

Microsoft's Live ID *is* the old Passport — with a few key changes. Kim Cameron's work around the identity metasystem has driven the concept of InfoCards (now called CardSpace) deep inside of Microsoft. In essence, Kim's idea is that there is a "metasystem" which utilizes WS-Trust to translate tokens, so that all identity systems can interact with each other.

Of extreme importance is the fact that Windows Live ID will support WS-Trust, WS-Federation, CardSpace and ADFS (active directory federation server). This means that A) Windows Live ID can interact with other identity metasystem implementations (Open Source versions, for example); B) that your corporate active directory environment can be federated into Windows Live ID; and C) the closed system that was Passport has now effectively been transformed into an open (standards-based) and transparent system that is Live ID.

Contrast all of this with Google's announcement: create Google account, store user information at Google, get authentication from Google — are we sensing a trend? While Microsoft is now making it easy to interact with other (competing) identity systems, Google is making it nearly impossible. All of which leads one to ask - why?

Perhaps it's because there are now so many old-school Microsoft people at Google? ;)

On a more serious note, I suspect that the Google folks simply didn't think about the federation angle when designing the authentication model for their APIs as opposed to this being some 'evil plot' by Google to create an identity silo.

 

Julien Couvreur has a blog post entitled Web API authentication for mashups where he talks about authentication and Web APIs. This is a topic that is near and dear to my heart since getting this right is very important for the Windows Live developer platform. Julien writes

Authorization techniques:

A number of techniques for controlling access to web APIs are generally used: user authentication cookies, API keys and crossdomain policy files. The problem is that API keys and crossdomain policy files are too restrictive because the service needs to decide which third-parties to let in.

On the other end, access control based on the user authentication cookies are very open to un-planned integration, but also create a huge phishing risk. This is a classic example of the confused deputy problems that appear in principal-based security models.

As a result, most web APIs today don't involve any user data (search, maps, ...) or non sensitive user data.

Yahoo APIs:

Yahoo appears to be tackling the challenge with its announced "browser-based authentication". From the little information I could gather so far, it seems less of an authentication than an authorization system. Unlike cookie based approaches, which give access to any agent presenting user credentials (principal-based security), it appears to follow a capability-based security model, which only grants access if the agent uses the proper "secure handle" or "capability" to call the service. Such capabilities are sufficient to gain access to the service and don't need any additional authentication, they are communicable tokens of authority.

The devil is in the details when talking about authentication, authorization and Web APIs. When I first heard about the Yahoo's proposed authentication model for Web APIs at their ETech 2006 talk entitled Building a Participation Platform: Yahoo! Web Services Past, Present, and Future, I thought it sounded similar to the model used by Passport Windows Live ID. In both approaches instead of applications prompting users for their credentials (username/password combo), the user signs in to the primary service which then returns an opaque token to the target application that identifies the user and gives the application permission to access the user's data. However, having a fine grained access that can give applications access only specific services and can revoke permission given to specific applications seems to be richer than what I've seen offered by  Passport Windows Live ID. This is nice but it's to be seen how easy this will be for users to understand or for applications to manage.

From my perspective there are two primary goals an authentication model for a family of Web APIs must satisfy

1. User credentials are sacred and must be protected at all costs: A security mechanism is only as strong as its weakest link. This means that it is extremely unwise to build an authentication model that has applications built on your APIs to request username/passwords or other credentials from users directly. The last thing you want is for anyone with a copy of Javascript for Dummies to be able to legitimately ask your users for their credentials then store them insecurely. In addition, if users get comfortable with entering their credentials in all sorts of random places then it makes them more susceptible to phishing attacks. This is one of the reasons services like Meebo are worrying to me.

   It should be noted that in certain cases, the information hosted in the service may not be very valuable in which case this tennet can be waived. For example, the NewsGator API expects applications to prompt users for their credentials and then pass those along when interacting with the service. Since the user information hosted in the service is primarily a list of RSS/Atom feeds and their read/unread state, the value to attackers is extremely low and there is little need to build a sophisticated authentication model for this service.

2. Do not discriminate against any platform or any device: In todays world, end users interact with online services using a variety of devices and platforms. Each device and platform has different strengths and limitations but is important in its own right. Online services like email or instant messaging have witnessed the rise of multiple access models from desktop applications running on PCs to applications running on mobile devices, from JavaScript code running clientside in a browser to web service calls being made from one server to another. In many cases, the average user may go back and forth between all these access modes within the course of their normal usage of the service. For example, I check my email using Outlook Web Access, Microsoft Outlook and my Audiovox SMT 5600 during the normal course of my work day. 

Thus far I have not seen any Web API authentication model satisfy both goals. Based on my understanding from the ETech talk, the model proposed by Yahoo! fails to meet the second goal above because it is browser based. Before being accused of bias, I'll also point out that from reading the initial documentation for the Windows Live ID service it also fails to satisfy the second goal because Microsoft has only announced SDKs for server-to-server calls and desktop clients [both of which I assume will only target servers or PCs running varieties of Windows]. 

Providing a comprehensive authentication story for a suite of Web APIs is a hard problem.

 

The Windows Live Custom Domains team has a post on their blog entitled Bye bye beta….Custom Domains v1 has launched which states

We’re leaving the “Beta tag” blanket at home.  That’s right…thanks to your beta testing and feedback; we’ve now officially launched Windows Live Custom Domains.  Our colleagues over in Messenger kicked off the Windows Live launch season last week.  Along with the launch of OneCare and Live Favorites, we're excited to continue the momentum.  Windows Live is about the Web, the way you want it.  Personalization is a key piece here, and let's face it...your identity online is central to that.  Custom Domains enables people to use all of the Windows Live and MSN services they want with an ID that's as unique to them as they want it to be.

For those who aren’t familiar with Custom Domains, we provide free hosted e-mail for your domain.  Let’s say you own the domain name, “wingtiptoys.com.”   With Custom Domains, you get unlimited, free e-mail accounts at that domain.  You can open accounts for sales@wingtiptoys.com, owner@wingtiptoys.com, etc.  Oh wait, did we mention that it’s free?  This isn’t one of those “free during beta” trial offers.  This is free for life. 

New Feature: Open Membership

We’re jazzed about a new feature we’ve added called Open Membership.  How does this work?  Let’s say you run a website called “soccerfan.com.”  Your users love your site and want an e-mail address @soccerfan.com.  Prior to this launch, each user would have to request an e-mail account from the administrator.  Then, the admin would manually approve and create each account.  We’ve made things much easier all around with this launch...With the Open Membership featured enabled, we provide URL links so users can automatically sign-up for an e-mail account @soccerfan.com.  Admins no longer have to burden with the manual creation of email accounts, and users get accounts immediately.

Congratulations to the team, it's good to see more Windows Live services coming out of beta. I really like this service but I'd love to see it expand to cover other Windows Live properties. For example, will I be able to ever use my own custom domain for my Windows Live Space?

 

The Windows Live Local/Virtual Earth team has a blog post entitled Free Phone calls at WLL which states

A new release of Live Local went out over the weekend. Mostly minor bug fixes, but a few new features made it in as well. One of the more interesting is the ability to phone any business for free. Using it is easy - do a business search by name or category and in the result panel will be a 'Call for Free' link next to each business listing. Each pushpin popup on the map will also have the Call for free link. When you click it you specify your phone number  -the system will dial both you and the business and connect you. Once you've made your first call, you can rapid dial businesses without having to re-enter your phone number.

Windows Live Local is definitely my favorite online mapping service today and probably the only Windows Live service I can say is head and shoulders above the competition. Kudos to everyone on the team who have built such a killer service in such a short time.

 

Quentin Clark has posted a new entry entitled Update to the Update on the WinFS team blog that answers some of the questions that have been raised since his post on Friday about the status of WinFS. He writes

Is WinFS dead?
Yes and No. Yes, we are not going to ship WinFS as a separate, monolithic software component. But the answer is also No - the vision remains alive and we are moving the technology forward. A lot of the technology really was database stuff – and we’re putting that into SQL and ADO. But some of the technology, especially the end user value points, are not ready, and we’re going to continue to work on that in incubation. Some or all of these technologies may be used by other Microsoft products going forward.

Does your plan for WinFS have any impact on Windows Vista?
There is no impact on Windows Vista. We announced back in August 2004 that WinFS would not be in Windows Vista.

Will the "Relational Filesystem" ever be in Windows?
Hey – we are very busy finishing Vista, and just aren’t ready to talk about what comes next. The vision for a richer storage in Windows is very much alive.  With the new tools for searching and organizing information in Windows Vista, we are taking a good step towards that vision.  

Why are parts of WinFS going into SQL Server?
We have a vision around data that guides us we call the "Data Platform Vision". We’ve been talking with customers about this for some time, and we have heard consistent positive feedback. It was clear that the integrated storage and automation features of WinFS will help SQL Server deliver on the "Beyond Relational" and "Continuous Availability and Automation" promises of that vision. We decided to focus resources on delivering these technologies to our customers as part of the Data Platform Vision in the near term.

Why did Microsoft announce this now after talking about WinFS at TechEd so recently?
When we were at TechEd, we had not made the decision. Sure, it was under discussion, but we did not have all the information we needed and we had not made the call yet. We did share the news as soon as we had the final word. We could have waited longer to disclose the information and made the change in plans less of a contrast, but we chose to notify people as soon as we could. This is why we used the blog and didn’t fire-up the big MS PR machinery – that takes time.

I commented internally that the response to Quentin's original blog post shows that there has been a discrepancy between what the WinFS team has been working on and what the developer community believes they were delivering. I got to read a draft of this blog post before it went up and it does a better job of stating what has happened with WinFS and even seems to have incorporated some of my feedback. I hope Charles Miller doesn't think this post is also un-blog-like. :)

 

I was reading Charles Miller's post entitled We Come to Bury WinFS... where he wrote

The first thing to strike me about the blog post announcing the end of WinFS as a Vista feature is how totally un-blog-like it is.

Every comment (bar one) got the point. WinFS is dead. Its carcass is being split between SQL Server and ADO.NET, and the relational filesystem that was going to change the way we use computers is no longer just postponed to be shipped after Vista, it’s gone.

The blog post itself, however, is written entirely in marketing-speak. The engineer talks about how super-excited the team is about this "new direction", how encouraging this news is, and leaves the fate of Vista for a final, particularly obfuscatory paragraph. Nary a word is allowed to suggest that the last nail in the coffin for Vista’s most eagerly anticipated feature might be a huge let-down to those people who have been watching it slip further and further down the schedule since its fanfare announcement as a part of Longhorn three years ago.

Did Microsoft forget everything Scoble was supposed to be teaching them, so quickly?

Every now and then, you’ve got to put out a mea culpa. You’ve promised something that turned into something else, or that you changed your mind about, or that you just can’t deliver. In the mass-media world, you do this by spinning the story as positively as you can. The message will be filtered by intermediaries before it reaches the public, and it’s expected the journalists in the middle will get the point, pulling quotes from the positive spin to offset the otherwise negative message.

I agree 100% with Charles Miller's sentiments about the blog posting on WinFS. This seems to be another example a case where Microsoft overpromised and underdelivered failed to deliver but even worse instead of owning up to this, the blog post spins this as being what customers want. From reading the hundred or so comments and trackbacks to Quentin Clark's post it doesn't seem that there are many people who are excited or encouraged that what once was touted as a pillar of Longhorn is now just another checkbox feature in SQL Server. This makes Microsoft look bad to developers because it means that we are either insulting our developer customers by thinking we can pull the wool over their eyes in such a blatant way or even worse that we are completely out of touch with them. Either way, it sucks and I feel like we should apologise to developers and perhaps even the software industry as a whole. Microsoft did offer a mea culpa to developers for the delay between Internet Explorer versions and I think this is another one of those cases where we should do the same.

I feel like I should probably throw in some last thoughts about WinFS, the technology, especially since in my previous post I claim that this decision should have been made a few years ago. Below is a random collection of my thoughts WinFS which can also be gleaned from my various blog posts on the technology over the last few years

1. There was a divergence of opinion in what the team was building and what the people thought the team was building. A common misconception was that WinFS would somehow make search "better" on the desktop in the same way that desktop search tools like Lookout do. I addressed this misconception further in my post Killing the "WinFS is About Making Search Better" Myth from almost two years ago.

2. The project had the biggest example of scope creep I'd ever seen at Microsoft. When WinFS swallowed ObjectSpaces the team decided that instead of just tackling the hard problem of building an object/relational file system for Windows that they also wanted to tackle the hard problem of  building an enterprise-class object to relational mapping technology with the same product in a single release. It also didn't help that key ObjectSpaces folks like Matt Warren, Dinesh Kulkarni and Luca Bolognese ended up joining the C# team to work on LINQ which meant WinFS inherited all of the problems of ObjectSpaces but not necessarily all the folks who had been working on those problems.
   

3. The chicken and the egg problem. One of the key ideas in the WinFS type system for the Windows desktop was that we'd have common file system level types for high level concepts like people/contacts, email messages or photos instead of just opaque bits on disk with some file format specific metadata. To take advantage of this, existing applications would have to be rewritten or new [backwards incompatible] applications would have to be written targetting WinFS. The primary benefits of making this change [besides the improved programming model] were the network effects if lots of applications used these types (e.g. Outlook stored mail identified by a WinFS contact, RSS Bandit stored RSS feeds from that WinFS contact, AOL Instant Messenger stored IM conversation logs using that contact, etc). Even if you got these network effects you then had to deal with the Windows registry problem (i.e. apps stomping over each others data which is one of the main problems with the Windows registry today).
   

4. I never saw good answers to the questions Jon Udell asked in his blog posts Questions about Longhorn, part 1: WinFS and Questions about Longhorn, part 2: WinFS and semantics. Specifically, the world is betting big on open file formats such as XML including parts of Microsoft (e.g. Microsoft Office) so why would anyone want to build aplications targetting a proprietary Windows-only file system that didn't have a good XML story for getting data out of the platform?

I should probably stop now. Even though all the information above is freely available to anyone who reads blogs and can put two and two together, some may object to the above collection of thoughts.
 

Quentin Clark has a blog post entitled WinFS Update where he writes

There are many great technical innovations the WinFS project has created – innovations that go beyond just the WinFS vision but are part of a broader Data Platform Vision the company is pursuing.  The most visible example of this today is the work we are now doing in the next version of ADO.NET for Orcas.  The Entities features we are now building in ADO.NET started as things we were building for the WinFS API.  We got far enough along and were pushed on the general applicability of the work that we made the choice to not have it be just about WinFS but make it more general purpose (as an aside – this stuff is really coming together – super cool). 

Other technical work in the WinFS project is at a similar point – specifically the integration of unstructured data into the relational database, and automation innovations that make the database "just work" with no DBAs – "richer store" work.  It's these storage innovations that have matured to the point where we are ready to start working on including them in our broader database product.  We are choosing now to take the unstructured data support and auto-admin work and deliver it in the next release of MS SQL Server, codenamed Katmai.  This really is a big deal – productizing these innovations into the mainline data products makes a big contribution toward the Data Platform Vision we have been talking about.  Doing this also gives us the right data platform for further innovations. 

These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release.  With most of our effort now working towards productizing mature aspects of the WinFS project into SQL and ADO.NET, we do not need to deliver a separate WinFS offering. 

So that's it, no more WinFS. This is the right decision, albeit two years too late but better late than never. It's sad to think about the projects that got killed or disrupted because of WinFS only for this to happen. In a recent column entitled Taking One for the Team Robert X. Cringley has a quote from Management By Baseball by Jeff Angus which reads "When I worked for a few years at Microsoft Corporation in the early '80s,...no one cared to track and codify past failures as a way to help managers create guidelines of paths to follow and avoid". I hope this doesn't end up happening with the lessons from the WinFS project.

 

Mike Champion has a blog post entitled Why does the world need another XML API? where he writes

One basic question keeps coming up, something like: "We have SAX, DOM, XmlReader/Writer APIs (and the Java people have a bunch more), we have XSLT, we have XQuery ... why do you think we need Yet Another XML API?"
...

• XmlReader / XmlWriter can't go away because XLinq uses them to parse and serialize between XLinq objects and XML text. Also, while we are making XLinq as streaming-friendly as possible (see the XStreamingElement class in the CTP release for a taste of where we are going), we're only aiming at hitting the 80/20 point...
• DOM can't go away because there are important use cases for API-level interoperability, most notably in the browser...DOM doesn't make code truly interoperable across implementations (especially on other languages), but there is enough conceptual similarity that porting is generally not terribly difficult...  
• XSLT definitely won't go away. The Microsoft XML team was promoting XQuery as a "better XSLT than XSLT 2.0" a few years ago (before I came, don't hurt me!), and got set straight by the large and vocal XSLT user community on why this is not going to fly. While it may be true in some abstract way that XQuery or XLinq might logically be able to do everything that XSLT does, as a practical matter it won't...  
  
• XQuery won't go away, at least for its original use case as a database query language.  Microsoft supports a draft of XQuery in SQL Server 2005, contributes to the work of the XQuery working group at W3C, and will continue to invest in finalizing the XQuery Recommendation and implementing it in our DBMS..
  

we believe that the overall LINQ story is going to have a pretty profound impact on data programmability, and we want to make sure that LINQ has a good story for XML...For XML users, I see a few really powerful implications:

• The ability to query data by declaraing the characterics of the result set rather than imperatively navigating through and filtering out all the data...
• The ability to join across diverse data sources, be they XML documents, objects, or DBMS queries
• The ability to "functionally" reshape data within the same language as the application is written.  XSLT pioneered the functional transformation approach to XML processing, but it is difficult for many developers to learn and requires a processing pipeline architecture to combine XSLT transforms with conventional application logic...

This brings back memories of my days on the XML team at Microsoft. We went back and forth a lot about building the "perfect XML API", the one problem we had was that there one too many diverse user bases which had different ideas of what was important to expose in an API. We were always caught between a rock and a hard place when it came to customer requests for fixing our APIs. To some people (e.g. Microsoft Office) XML was a markup format for documents while to others (e.g. Windows Communications Foundation aka Indigo) it was simply a serialization format for programming language objects. Some of our customers were primarily interested in processing XML in a streaming fashion (e.g. Biztalk) while others (e.g. Internet Explorer) always worked on in-memory XML documents. Then there were the teams whose primarily interest was in strongly typed XML (e.g. SQL Server, ADO.NET) since it would be stored in relational database columns.

In trying to solve all of these problems with a single set of APIs, we went down the road of prematurely declaring the death of certain XML APIs and technologies such as the DOM (see Ode to the XML DOM) and XSLT (see XSLT 2.0 Sir? or Why You Won't See XSLT 2.0 or XPath 2.0 in the Next Version of the .NET Framework). At the end of the day we saw the light and we eventually changed our tune by not deprecating the System.Xml.XmlDocument class and by reconsidering whether replacing XSLT with XQuery was the right way forward.

When I was on the team there was a lot of infatuation with XQuery which eventually turned to frustration. There were a number of technical and non-technical reasons for this such as its dependence on W3C XML Schema which significantly complicated its type system and how long the spec was taking to become a standard (over 5 years and counting as I write this post). Since then a bunch of folks who were were enamored with XQuery have taken some of its lessons (e.g. declaritiveness, simple model for XML generation, etc) and integrated it into a mainstream programming environment with the XLinq project. XML geeks everywhere should read Erik Meijer's paper, XLinq: XML Programming Refactored (The Return Of The Monoids), it is a thing of beauty if angle brackets are your thing. And even better, if you are one you are one of those that chants rabid slogans like "XML is the assembly language of Web 2.0", you'll still like XLinq because it provides a easier and richer level of abstraction for working with XML.

Enjoy.

 

One of my coworkers sent me a link to the blog post PhotoBucket Leads Photo Sharing Sites; Flickr at #6 on the HitWise company blog. The highlights of the post are excerpted below

In the SF tech bubble that I live in, most of the talk about photo sites has been centered on Flickr. In fact, you could get the impression from most people I meet that Flickr is the ONLY site at which you can share and store photos. Examination of the category however, shows that Flickr is #6 among the top 10 photo sharing sites, with a market share of 5.95%. Industry standbys like Yahoo! Photos, Webshots Community, and Kodak Gallery currently rank higher than Flickr.

Photobucket dominates the category, with a 44% market share. It surpassed Yahoo! Photos in January, and its share of visits increased by 34% in the four months from February 2006 to May 2006. Flickr, my friends should be happy to note, has also been growing rapidly, increasing 44% in the past four months, and up from a rank of #9 in this category one year ago (week ending 6/18/05). Slide has also taken off this spring, with its visits increasing more than ten fold in the past four months.

In the comments, someone asked about the methodology and why the HitWise marketshare numbers differ significantly from those of other ratings companies like ComScore. The response is that the HitWise numbers are based on page views from sampling 10 million users while ComScore numbers are for unique users. According the the comment ComScore states that both Flickr and Photobucket  get about 16.5 million unique visitors a month while Yahoo! Photos gets about twice that number. Besides the page views versus unique users distinction, another thing that makes this somewhat of an apples to oranges comparison is that the HitWise sample is for Internet users in the USA while ComScore is talking about worldwide usage.

However this is still an instructive set of statistics. For one it shows that even if Flickr does have more people talking about it in the tech blogosphere, Photobucket is generating a lot more page views [mainly through MySpace integration]. Secondly, it shows the value of an integrated suite of social applications [photo sharing, social networking, video sharing, blogging, etc] in engaging users. Flickr may or may not have just as many visitors as Photobucket but it's clear that the average visitor to Photobucket views significantly more photos from the site than the average Flickr visitor. This is primarily due to the fact that Photobucket is one of the top image hosting sites used by MySpace users. This seems to validate the approach of building an integrated social software application like MSN Spaces or Yahoo! 360 instead of a mishmash of narrowly tailored social software applications. Where I think Microsoft and Yahoo! have gone wrong and MySpace has gotten it right is that they rely a lot on an ecosystem of supporting sites for extra features (e.g. image hosting, video hosting, etc) instead of trying to do it all in-house. This enables them to innovate a lot faster since Microsoft and Yahoo! are then competing with multiple companies instead of just one.

 

Tim Berners-Lee has a blog post entitled Net Neutrality: This is serious where he writes

When I invented the Web, I didn't have to ask anyone's permission. Now, hundreds of millions of people are using it freely. I am worried that that is going end in the USA.

I blogged on net neutrality before, and so did a lot of other people. (see e.g. Danny Weitzner, SaveTheInternet.com, etc.) Since then, some telecommunications companies spent a lot of money on public relations and TV ads, and the US House seems to have wavered from the path of preserving net neutrality. There has been some misinformation spread about. So here are some clarifications. ( real video Mpegs to come)

Net neutrality is this:

If I pay to connect to the Net with a certain quality of service, and you pay to connect with that or greater quality of service, then we can communicate at that level.

That's all. Its up to the ISPs to make sure they interoperate so that that happens.

Net Neutrality is NOT asking for the internet for free.

Net Neutrality is NOT saying that one shouldn't pay more money for high quality of service. We always have, and we always will.

I've been depressed by the what has been happening in Washington with regards to Net Neutrality but have hesitated to blog about it since I am clearly biased, I work for a company that is likely to financially benefit if Net Neutrality is the status quo. However that doesn't change the fact that what the telcos are striving would eventually turn the Web into a very different place from what it has been. Read all of Tim Berners-Lee's post and after that  you should read what Lawrence Lessig had to say about this topic in his article for the Washington Post; No Tolls on the Internet.

 

I've counted myself lucky to find a girlfriend who has similar tastes in music to me. Most of my friends don't like equal parts Lil Jon, Metallica, G-Unit and Backstreet Boys but my girlfriend does. In addition, she likes the crunk hip hop clubs just like I do. In fact, the clubs we tend to like going to regularly seem to end up getting closed down for being too crunk (e.g. Mr Lucky, Larry's).

So you can imagine that I was quite surprised by her recent request that we attend a Kenny Chesney concert. So I asked here "Who's Kenny Chesney?" and she replied that he was a country singer. I was like "But you don't like country music" and she replied that actually she did. Since this revelation, you are now just as likely to hear us listening to Settle for a Slowdown or What Hurts the Most as you are to hear Poppin' My Collar or Hustlin' as we drive down the street. How times change.

The concert is this weekend and I'm not sure I have appropriate attire. If one of my coworkers who reads my blog has a cowboy hat that I can borrow, I'd greatly appreciate it.

 

Joe Gregorio has a blog post simply titled Hire Me where he writes

As of 10 O'Clock this morning I am no longer employed; being laid off tends to do that to you. The good news is that I can catch up on all those projects around the house, my backlog of XML.com articles, my editing of the next draft of the Atom Publishing Protocol, etc. I can even start blogging about the industry I was working in and the company I was working for, but only after I fulfill the requirements of my severance package.

As positive as that all sounds I do have a mortgage and a family, and they like to eat, so I need to find gainful employment.

Please hire me.

If you are unable to hire me please do me a favor and link to this entry.

My resume in PDF format. An HTML version will appear shortly.

I'm not a hiring manager which means I can't hire Joe,  so I'm doing the next best thing and linking to his entry and resume. If you work at Microsoft and are interested in a guy who is quite knowlegable about building RESTful services, you'd be hard pressed to find anyone better qualified than Joe Gregorio.

 

I was pretty surprised to find the press release entitled Microsoft Robotics Studio Provides Common Ground for Robotics Innovation via Todd Bishop this morning. It states

PITTSBURGH— June 20, 2006 — Today at RoboBusiness Conference and Exposition 2006, Microsoft Corp. showcased the community technology preview (CTP) of a new Windows^®-based environment for academic, hobbyist and commercial developers to easily create robotic applications for a wide variety of computing platforms. In addition, early adopter companies, universities and research institutes offered demos and provided support for the new Microsoft^® Robotics Studio development platform. The community technology preview of the Microsoft Robotics Studio is available for download at http://msdn.microsoft.com/robotics.

"Microsoft, together with the upcoming LEGO^® MINDSTORMS^® NXT, will help further amplify the impact of robotics,” said Søren Lund, director of LEGO MINDSTORMS at the LEGO Group. “The MINDSTORMS robotics toolset has enjoyed a strong community of users since 1998, and the launch of our next-generation platform includes many built-in features that further the community’s ability to take MINDSTORMS programming out of the box. In combination with Microsoft Robotics Studio, PC users will have a sophisticated tool that will further extend the powerful NXT hardware and software to an even wider range of developers who wish to create advanced applications for their LEGO robots."

At first glance, I thought this was an announcement that Microsoft would be getting into building robots like Asimo but it seems that instead it is Microsoft getting into the business of building development platforms for programming robots. There is a good overview of Microsoft Robotics Studio on MSDN which describes the core pieces of the platform. Interesting, I can now program LEGO Mindstorms using C# and the .NET Framework instead of lower level languages like NQC (Not Quite C) which is quite cool.

A neat bit of trivia is this is the project that George Moore, who now runs the Windows Live developer platform team, came to Windows Live from. It's interesting to see how different ones job roles can be from year to year at Microsoft. 

 

From the press release Launch of Windows Live Messenger Marks Significant Progress for Microsoft’s Windows Live Era we learn

Tomorrow (June 20), Microsoft Corp. releases Windows Live Messenger, available at http://get.live.com/messenger/overview, one of the first of more than 20 new Windows Live services to launch globally over the course of the year. Windows Live Messenger goes beyond the traditional instant messaging (IM) service, enabling people to connect and share, with free PC-to-PC calls and inexpensive calls from a PC to phones around the world, video calling, easy sharing with Sharing Folders, and more.

I was waiting for the download to be available before blogging about it but it looks like that didn't stop Microsoft from publishing a press release. :) Although there is a lot of hyperbole in what is essentially an announcement of a rebranded MSN Messenger, there is some good stuff in this release. The feature list on get.live.com has a good run down of the new features, my favorites being the sharing folders and the various voice calling options. The upcoming ability to communicate with your friends on the Yahoo! Messenger network is also cool but not that interesting to me since I don't have any friends who use Yahoo! Messenger. The feature I can't wait to see turned on is social networking integration between Windows Live Messenger and Windows Live Spaces. This is would be my first major feature showing up in Windows Live Messenger and one I spent a bunch of time on.

We had a recent reorg on my team, and my day job has transitioned from spending about twice as much time working on the services behind MSN Spaces over Windows Live Messenger to the inverse. This means I'll probably be blogging a bit more about WLM and a instant messaging applications.

 

Anil Dash has a blog post entitled Office 2007 is the Bravest Upgrade Ever where he writes

Short and sweet, the Ribbon and new UI in Microsoft Office 2007 is the ballsiest new feature in the history of computer software. I've been using Office 12 for about six months, and not only has it made me more productive, I'm struck by the sheer ambition of the changes in this version.

To clarify the point: Microsoft Office is a bigger business than most of us probably realize. Office generated $11.5 billion in revenue for fiscal year 2005, and it'll exceed that in the current calendar year. But conservatively, you're talking about a billion dollars a month.

Now, most of us who like to prognosticate and pontificate about software like to say things like "It'd be easy to just..." or "It's trivial to add..." but the thing is, most of us aren't betting our entire careers on the little tweaks and changes we'd like to make to our productivity applications. Try making a mistake that jeopardizes a business that makes $250 million a week. I'd figure a 2% error, on the order of $5 million, gets you very, very fired. Maybe they're forgiving and you can make a 10% error, costing $25 million a week. I doubt it. Most of us would lose our nerve about suggesting radical changes if betting wrong meant betting lots of jobs on making the right call. (Nobody ever got fired for making incremental improvements to Office.)

Two of the ballsiest moves I've seen Microsoft make in the past five years were both made by the Office team under the leadership of Steven Sinofsky. The first is the new UI changes in Office 2007. The second has been the movement away from proprietary binary formats and towards open XML formats as the default in Office 2007. In addition both massive changes have been live blogged all the way through by Jensen Harris and Brian Jones respectively which in itself should probably be on the list of pretty cool and risky things that Microsoft has done as well.

Now that Steven Sinofsky now runs both Windows client and Windows Live, I wonder what kind of decisions and product announcements we'll see in the coming months. So far we've been executing on stuff decided on before the big reorg, it'll be interesting to see what direction Windows and Windows Live go in the coming year or two. Definitely interesting times ahead. 

 

Via Mark Baker I found an article in the ACM Queue entitled The Rise and Fall of CORBA by Michi Henning. Lots of good stuff in the article some of which is excerpted below.

the CCM (CORBA Component Model). A specification for CCM was finally published in late 1999 but turned out to be largely a nonevent:

• The specification was large and complex and much of it had never been implemented, not even as a proof of concept. Reading the document made it clear that CCM was technically immature; sections of it were essentially unimplementable or, if they were implementable, did not provide portability.
• No commercial CORBA vendor made a commitment to implement CCM, making it a stillborn child.
• Even if implementations had been available by the time CCM was finally published, it was too late. The horse had already bolted: EJB had become entrenched in the industry to the point where another component technology had no chance of success.

The failure of CCM did little to boost the confidence of CORBA customers, who were still stuck with their complex technology.
...
What steps should we take to end up with a better standards process and better middleware? Seeing that procedural failures are the root cause of technical failures, I suggest at least the following:

1. Standards consortia need iron-clad rules to ensure that they standardize existing best practice.
   
2. No standard should be approved without a reference implementation.
   
3. No standard should be approved without having been used to implement a few projects of realistic complexity.
   
4. Open source innovation usually is subject to a Darwinian selection process.
   
5. To create quality software, the ability to say “no” is usually far more important than the ability to say “yes.”.

The lessons listed above seem rather self evident and obvious yet it s a sad fact of the software industry that the mistakes of CORBA keep getting made all over again. Core XML technologies like W3C XML Schema and XQuery are 'standards' without a reference implementation which invented new features by committee instead of standardizing best practice. At least one of the guidelines is probably unrealistic though. It is hard to require that a standard shouldn't be approved until it has been used to solve a real-world problem since people solving real-world problems typically don't want to be used as guinea pigs.

 

It seems that during the storm of news and blog posts about Robert Scoble and Bill Gates, I missed a couple of newsworthy events involving my employer. Last week Adobe released a press release entitled Adobe and Microsoft which is excerpted below

Adobe has been in discussions with Microsoft for some time, sharing concerns regarding features and functionality in the upcoming releases of Vista (the next version of Microsoft’s operating system) and Microsoft Office. While much of the press coverage to date has centered on disputes over PDF and XPS (a competitive technology to PDF) in MS Office and Vista, the real issue is the protection of open standards.

Adobe is committed to open standards. Adobe publishes the complete PDF specification and makes it available for free, without restrictions, without royalties, to anyone who cares to use it. Because we license the PDF specification so openly, it has become a de facto standard, used by hundreds of independent software vendors worldwide. No other specification is employed on as many hardware platforms, operating systems and applications as PDF. PDF is incorporated into a number of ISO standards, and Adobe encourages developers, independent software vendors and publishers to support and embrace it. While the specification is available publicly, customers expect Adobe to ensure that the format does not become fragmented and that competing implementations of PDF do not undermine what customers have come to expect in terms of reliable viewing and printing of PDF documents across platforms and browsers.

Microsoft has demonstrated a practice of using its monopoly power to undermine cross platform technologies and constrain innovation that threatens its monopolies. Microsoft’s approach has been to “embrace and extend” standards that do not come from Microsoft. Adobe’s concern is that Microsoft will fragment and possibly degrade existing and established standards, including PDF, while using its monopoly power to introduce Microsoft-controlled alternatives – such as XPS. The long-term impact of this kind of behavior is that consumers are ultimately left with fewer choices.

In response, Microsoft released a press release entitled Statement from Microsoft Concerning Adobe which is excerpted below

Microsoft welcomes Adobe’s reaffirmation that PDF is an open standard, made available for free, without restrictions, without royalties, to anyone who cares to use it. Adobe’s frequently stated commitment to PDF as an open standard was the basis upon which Microsoft decided to include support for saving documents in the PDF format in 2007 Microsoft Office system. Customers running Office have clearly said it would be helpful to be able to easily save documents as PDF files. Microsoft expected that Adobe would applaud, rather than object to, our support for the PDF standard in the most widely used productivity software product.

Adobe has now expressed concern that Microsoft would one day “extend” the PDF specifications. Microsoft has not extended the PDF specifications and has no interest in doing so. Our only interest is to support our customers by making it easy to export Office documents in the standard PDF format so they can be viewed in Adobe Reader and printed. Adobe is a participant in the 2007 Office system beta program and can see for itself that Microsoft has not extended the PDF specifications.

To our knowledge Adobe has not imposed limits on how third parties support the PDF format or differentiate their products through extensions. Nevertheless, Microsoft wishes to confirm that it will not extend the PDF specifications published by Adobe. Microsoft hopes that, with this assurance, Adobe will withdraw its objection to including support for PDF in 2007 Office system.

It looks like the ball is now in Adobe's court. I suspect that there is an additional [unstated] concern that the inclusion of PDF support in Microsoft Office reduces the attractiveness of Adobe's suite of products for generating PDF. It would be probably be hard to take that to court, although I've seen  cases which I thought had less merit go against Microsoft. It'll be interesting to see what the response from Adobe will be on this one.

 

The Indigo Windows Communication Foundation folks have decided to let the code do the talking in showing that they are serious about supporting Plain Old XML over HTTP (POX) and RESTful Web services. You can download the WCF RSS Toolkit and try it out yourself. If you are like me, you may not have all the requisite beta bits installed to run it, in which case you can peruse the WCF RSS Toolkit Code Samples which show how straightforward it is to produce RSS or Atom using WCF. 

Sam Ruby has asked whether the WCF RSS Toolkit supports ETags which is really a proxy for asking whether WCF supports manipulating HTTP headers directly. In my conversations with WCF folks like Yasser & Doug, the answer is that although the WCF RSS Toolkit doesn't support ETags that this was due to time constraints than any limitations in WCF. The Windows Communications Foundation will support service enpoints directly interacting with HTTP headers and HTTP methods. So one could build a real-world RSS endpoint or even a WebDAV or Atom Publishing Protocol end point with WCF.

 

The Google Blogoscoped blog has an entry entitled An Inside View From a Google Employee summarizes a very in-depth Something Awful discussion board interview with a developer who claims to work at Google and is currently responsible for the Google Calculator project as his current 20% project. Excerpts from the summary are listed below

• "Nobody keeps track of 20% time with any care whatsoever. It’s assumed that, if a deadline is pressing on your main project, you’ll work on that. If your main project constantly has looming deadlines, it’s time to talk to your manager or your tech lead and tell them that they’re pushing too hard.” Zorba adds that management understands that a programmer can’t be pushed over limits for more than a week at a time.
  
  And: “At Google, the managers and tech leads assume that programmers can manage their own time. If a programmer can’t manage their own time they’re probably not a good fit at Google anyway."

• ZorbaTHut says that Google is mostly C++, Java, and Python (or so he’s been told).

• On how Google goes about staffing a Test Engineer position, Zorba replies: "I don’t know what other teams are like, but on my team everyone owns their own tests and handles their own quality."

• Zorba: "[W]e have one monolithic source control system across the entire company. This lets us link in handy libraries from other projects, and is honestly one of the coolest things about working here – if there’s something common you want, chances are good it’s already been written."

• ZorbaTHut tells us he was assigned on Google Desktop first but didn’t like that much, so he was allowed to switch to working on Google Video. "I actually worked on some neat stuff on Google Video, all of which got cancelled before release. I unfortunately can’t tell you what it was."

• On what kind of info Google employees are allowed to share:
  "If we haven’t announced it publicly, and it’s a project or a coming feature or anything more financially interesting than ’what color are your carpets’, don’t talk about it. End of story".

• "The company’s structure, at least for engineers, is amazingly flat."
  Zorba says the Google hierarchy is just five levels: Programmer - Tech lead - Manager - Department lead - Larry/Sergey/Eric. Google just assumes their workers are competent, Zorba adds.

A lot of this jibes with stuff I've heard from about working at Google from Google employees or second hand from friends of employees. Thus I assume the interviewee is legit and is either an employee or someone with a contact inside the company. If he is an employee, he's probably going to get fired for this if Google's past actions are any indication.

 

The has a blog post entitled Is Meta Better? where he writes

Let me start with Digg. It is a proven model.  It works. When we funded delicious, Digg was about half the users and traffic of delicious. They are not the same service, but the delicious popular page and Digg are both places you can go see what's "hot" on the Internet. Since that time, Digg has grown to 1.3mm unique visitors a month compared to only 350k for delicious. These are Comscore Media Metrix numbers. Clearly Digg has a more mainstream user experience. Further, Digg's audience has doubled in the past three months, so it's growing fast.

But let's put this in perspective. Digg's reach (UVs) puts it 735th on Media Metrix' top 1000 sites list, between upromise and taleo. So in the grand scheme of things Digg isn't necessarily the next big thing on the Internet.

I find it surprising that even with the clout of Yahoo! behind it, del.icio.us sees such a relatively small amount of unique users a month. I guess this goes back to the 53,651 meme. I often see people at work using del.icio.us and Digg as examples that a feature or idea is sound but it is clear that these services aren't really popular enough to draw such conclusions.

By the way, given that RSS Bandit has seen over 250,000 downloads this year it looks like we'll soon be eligible to be purchased by Yahoo!. If so, I promise we'll be cheaper than del.icio.us . ;)

PS: Before I get any weird mail, the last comment is a joke. RSS Bandit is BSD licensed so you don't need to pay me us money to do anything you like with it.

 

Hugh Macloed has a blog post entitled How microsoft lost their canary where he writes

Micorosoft losing both Bill and Robert in the very same week.

In one week, Microsoft lost the two people who best expressed Microsoft; one on the macro-corporate level, one on the micro-grass-roots level.

What does this really say about Microsoft?

To me it says, "Party Over".

To me it says, Microsoft finally has reached the crossroads indicated in the cartoon above, and have opted to take the non-Cluetrain route. They opted to take that route because they have run out of ideas. They're at a time in their corporate life when they need a big idea. And you what? They. Simply. Don't. Have. One.

Hey, it's their company, it's their money, they can do what they like. There's lots of money still there to made, managing one's own demise. General Motors has been doing it for decades. And Madison Avenue, that's pretty much all they do now.

But Robert was the canary in the coal mine. And Microsoft's just lost their canary.

Last week was definitely an interesting week for Microsoft. I agree with Hugh that Microsoft has lost two people that many consider to symbolize the company. I'm not sure I understand what Hugh means by Microsoft has opted to take the non-Cluetrain route but then again the entire Cluetrain trend is something I've never been too knowledgeable about. I do have some opinions on what the loss of Robert Scoble and Bill Gates means to me as a Microsoft employee.

Robert Scoble did three things for Microsoft, only one of which was really his day job. He

1. He was one of the folks working on Channel 9 which has been an important mechanism for Microsoft to soften its image and open a new avenue for conversation with its customers.
2. His weblog acted was an aggregation of the various interesting bits of news about Microsoft. Half of the content of his blog was basically a blog version of Microsoft PressPass, Microsoft Watch and MSDN Blogs rolled into one.
.
3. He had an online persona as a great guy and a passionate technology geek (i.e. the ideal employee) which often made people think "Microsoft can't be all that bad, if this guy works there"

As far as I'm aware, only the first item in the above list was Robert's day job. The other items were just Robert being Robert. Microsoft can easily find someone to replace Robert in the context of item #1 on the list. It may even be a good thing to have some professional media folks running Channel 9 instead of just amateur media geeks. The other two items on the list are things that it's hard to imagine anyone doing in the same way that Robert did. A blog of product announcements can be replicated and although it might be boring reading, people seem to like them, or at least that's the impression I've gotten based on the popularity of "official" Google blog. I personally don't think Microsoft needs one but there probably is enough interest in it that I wouldn't be surprised if one shows up. The third item on the list is one of those things that has incalculable value. It's hard to quantify the value that Microsoft got from this aspect of Robert's blogging but there definitely was a lot of value obtained. There are thousands of Microsoft employees who blog yet somehow Robert Scoble ended up becoming the symbol of blogging @ Microsoft. Do we need another single "model employee" for bloggers to rally around? I don't know. Being the "company mascot" seems like a heavy burden for any single individual to bear especially for a company like Microsoft. The company has lost some points in marketability without Robert Scoble but there are thousands of company's doing just fine without a highly visible employee blogger.

I never got to meet Bill Gates as a full-time Microsoft employee. I did get to ask him questions at one of yearly intern soirree at his house back in my intern days but I've never been at a BillG review. I think Bill Gates leaving the company will be good for him, good for the world and good for the company. I tend to agree with Hugh that Microsoft hasn't had a "big idea" for a while. It used to be "a computer on every desktop" but now that we've almost gotten there it isn't clear what the company stands for as a whole. It isn't just a software company; we do hardware with XBox and other Microsoft hardware devices, television content with MSNBC and online content on MSN. The company doesn't seem to have one big idea because it doesn't have one set of customers. It's hard to see a consistent vision that holds together all the various pieces that are Microsoft. On the other hand, in some of the places where we have had a consistent technical vision, it has been hard for us to actually implement this vision (e.g. Cairo/WinFS, Netdocs and even Hailstorm) for a variety of reasons. I think Microsoft is about due for introducing new perspectives in the technical direction that the company takes. I've met with Ray Ozzie [who'll be taking over from Bill Gates as chief software architect] a few times times and he definitely seems to have a different view of the software landscape than many of the execs at Microsoft. Fresh perspectives are always a good thing. It'll be sad to see Bill Gates step out of his role as Chief Software Architect since he symbolizes so much of the company but I think he'll the world a lot of good with his foundation and he is leaving the company in good hands.

Losing Robert Scoble and Bill Gates sucks but life will go on. This isn't the harbinger of the death of Microsoft, just a signpost that indicates that one era is ending and another beginning.

 

I finally had some free time yesterday to explore the Windows Live Expo API and quickly whipped up a Live.com gadget which browses apartment and condo listings in a zip code of your choice. Given that we might be moving again this summer since a noisy bar opened across the street, this gadget will turn out to be quite handy. Below is a screenshot of the gadget.

You can add it to your Live.com page by going here and clicking .  I didn't really have much problems with the Expo API beyond the fact that it only accepts latitudes and longitudes so I have to use the Yahoo! Geocoding API to convert zip codes to lat/long coordinates before calling the Expo API. The fact that they both used RESTful APIs definitely made writing this mashup a breeze.

I'd love to add a feature to the gadget where I can add an apartment I like to one of my collections in Windows Live Local since I already have a collection of "apartments we should check out" there already. However there are a number of issues which prevent this today. Of course, I know all the right folks to nag to make the changes that would make this possible. :)

 

I saw two related posts on TechCrunch this morning that resonated with some of the issues I face in my day job. The post entitled MySpace Nukes SingleStat.us states

Well, so much for the SingleStat.us experiment that allowed people to find out when someone on MySpace changes their relationship status. MySpace’s friendly lawyers sent a cease and desist letter to David Weekly, the engineer who built SingleStat.us, demanding he take the site down and claiming that his "activities are causing and will continue to cause MySpace substantial and irreparable harm."
...
Something tells me that MySpace might be exaggerating just a bit when they say SingleStat.us is causing them "undue server burden". And if I’ve learned anything from covering companies, it’s that sending a cease and desist letter to a small, one-man startup is generally not going to work out the way you planned.

there is also another post entitled Why is Flickr afraid of Zooomr? which states

Flickr says that users own the the images and tags we enter into their system. Apparently that doesn’t mean they have to make it easy for us to take what we own elsewhere.

When Kristopher Tate, the founder of the feature-rich startup photosharing site Zooomr (see prior coverage), asked Flickr earlier this month for access to their Commercial API, Flickr’s response by email was that "we choose not to support use of the API for sites that are a straight alternative to Flickr." Flickr founder Stewart Butterfield posted to a Flickr forum on Wednesday saying that when it comes to direct competitors like Zooomr, "why should we burn bandwidth and CPU cycles sending stuff directly to their servers?"

The concerns brought up by the MySpace and Flickr folks are concerns that I've had to deal with as our various services such as MSN Spaces and MSN Messenger have grown and we've considered ways to open up these services to third parties via APIs. First of all, despite what Mike Arrington thinks there definitely is a concern when a service shows up whose entire business model is based on polling your website and screenscraping the HTML several times a day. There is a definitley a valid concern if a cottage industry of services springs up around polling MySpace and crawling the various social networks and profiles on the site. Given that there are already necessary evils one has to deal with like search engines and RSS readers, no one who build a successful online service wants to increase the number of web crawlers hitting their site unless absolutely necessary.

However if there are enough of these kinds of requests, it may point to an oppostunity for the website to provide an API. This often ends up building an ecosystem of services and applications aroung the website which ends up benefitting everyone. On the flip side, I often wonder about the balance between making it easy for users to get their data and making it easy for competitors to poach our users. Flickr makes it easy for users to get their data from the service but is hesitant about making it easy for competitors [which includes big players like Google not just startups like Zooomr] from simply bulk copying user information from their service. I think it's a fine line and I've had debates with folks like Mike Torres about where exactly the line is crossed.

In both instances above, the issue isn't the specific incident but the aggregate effect of allowing the behavior to continue.

 

The Windows Live Favorites team has a post on their blog entitled Live Favorites - Beta No More! which states

We, the Live Favorites Team, are happy to announce that as of today Live Favorites is officially released as a V1 product... Beta No more!

 

A little while ago we put into production our final V1 service and web site.  Check it out:

Expanded Market Support

Live Favorites is now available in 37 markets and 11 languages around the world.  We're thrilled to be able to bring the service to so many people!

....

Live Favorites Messenger Tab New Features

We've got some great new features in our Messenger Tab:

• Folder Support - See all of your favorites in folders!  Click on the small Folder icon to switch between Top Favorites and Folder Views.
• Favorite Count - See exactly how many favorites you have stored.
• Add Favorites - You can now add favorites directly from Messenger. Click the down arrow next to "Manage" to manage your favorites, add a favorite, or refresh Messenger's list of favorites

...

Live Favorites Web Site Enhancements

Of course, your main place to access and manage your favorites is still our website at http://favorites.live.com.  We've added a lot of enhancements for this release, check 'em out:

• Speed!  - We've done a lot of work to greatly reduce rendering time on our site.
• Browser Support -  Live Favorites fully supports IE6, IE7 and Firefox.
• Multi-Select - You asked for it! Select multiple favorites and delete or move them with a single click.  Simply hold down the Ctrl key and click on multiple items to select them.
• Favorites Status - We now tell you exactly how many folders and favorites you have
• Easier Sharing - Share your favorites with a single click!  In the details view click "Private" on any favorite to share it, and vice versa!
• UI Enhancements - Resize columns!  Sort by any column!   Improved graphics! List View!  Favorites Preview!

Congrats to Irwin, Lucius, Brion and the rest of the Live Favorites gang for getting v1 of their service out the door

 

The Live.com team have been getting a lot of feedback about how to improve their user experience and it looks like they paid attention to a lot of the critics. The post on the team blog entitled  live.com UI refresh outlines some of the changes they've made including 

 -          Wehave simplified our first run experience to provide a fast andstraight-forward experience for users that are interested in asearch-focused page, as well as the option for users to personalizetheir homepage or learn more about Windows Live services.  

-          Wehave heard that users want a personalized homepage that is easier toset up, so we are also introducing a new customization step to helpusers personalize their Live.com homepage. Users can now choose from 5templates to quickly get started with a few interesting pages rightaway (including a world cup template!)

-          You’ll also note some look and feel improvements with a new logo, improved themes, header and page treatments that make the page more visually appealing.

I know there has been a lot of folks at work have nagged the team about making the start page a simple search box while others want to have a more pre-programmed feel like MSN.com. It looks like the team has found a good compromise. Here are some screenshots of the new start page and one of the templates that can be chosen as a start page. The world cup template below is particularly amusing to me because I've seen a bunch of internal criticism the team has been getting for not doing something to support the world cup.

First run experience

World Cup template

There are almost 50 comments on to the blog post announcing the changes as I write this and most of them are positive. Kudos to Sanaz and company.

 

From the press release entitled Microsoft Announces Plans for July 2008 Transition for Bill Gates

REDMOND, Wash. — June 15, 2006 — Microsoft Corp. today announced that effective July 2008 Bill Gates, chairman, will transition out of a day-to-day role in the company to spend more time on his global health and education work at the Bill & Melinda Gates Foundation. The company announced a two-year transition process to ensure that there is a smooth and orderly transfer of Gates’ daily responsibilities, and said that after July 2008 Gates would continue to serve as the company’s chairman and an advisor on key development projects.

The company announced that Chief Technical Officer Ray Ozzie will immediately assume the title of chief software architect and begin working side by side with Gates on all technical architecture and product oversight responsibilities, to ensure a smooth transition. Similarly, Chief Technical Officer Craig Mundie will immediately take the new title of chief research and strategy officer and will work closely with Gates to assume his responsibility for the company’s research and incubation efforts; Mundie also will partner with general counsel Brad Smith to guide Microsoft’s intellectual property and technology policy efforts.

Wow. It looks like the internal grapevine was right. That explains all the press Ray has been getting as the savior of Microsoft.

 

Mike Arrington has a blog post entitled AOL-Netscape Launches Massive "Digg Killer" where he writes

On Thursday, AOL’s Netscape property will no longer be just another portal - it’s being converted into a Digg-killer. I was briefed on the new site by Jason Calacanis last week. As of tonight, he owns the Netscape property at AOL. The new site will run at beta.netscape.com for now, converting over to the main Netscape.com property soon.

It’s not exactly a Digg clone (home page screenshot here). Submitted stories are voted on in much the same way, and the more votes a story gets the higher it appears in a category home page or on Netscape.com itself. However, the top few spots in each category and on the home page are determined by an "anchor" - essentially an editor choosing from stories moving up the ranks.

There's also some good commentary on this in Greg Linden's post Netscape's News Scrape and Danny Sullivan in his post Netscape Aims To Be Digg 2.0, Slashdot 3.0 With Community News Model. Using the wisdom of the crowds to improve news sites is an good idea and one that has worked for Slashdot and Digg as Danny Sullivan points out. However I agree with Greg Linden that the holy grail is truly personalized news although the Slashdot/Digg/Netscape approach is a decent middle ground.

I was going to send a mail at work about this and encourage Microsoft to do the same until I realized I didn't even know which division of folks to send the mail to. If you read any random Microsoft press release you'll see the following excerpt

MSN is a world leader in delivering compelling programmed content experiences to consumers and online advertising opportunities to businesses worldwide. Windows Live, a new set of personal Internet services and software, is designed to bring together in one place all the relationships, information and interests people care about most

So MSN is for programmed content like your traditional news site and Windows Live is for personalized content. So which of these two divisions would be interested in a half-way approach like Slashdot or Digg? I literally have no idea and I work at Microsoft. So I cancelled the mail and blogged this instead. If I'm confused by this arbitrary distinction between MSN and Windows Live, I can only imagine how much our customers get it. :)

 

Tim Bray has a blog post entitled Blogging Cam: Almost Right where he writes

Via Niall Kennedy (who’s been very good lately): Microsoft cameras that have one-button publishing to MSN Spaces. This is so wrong. I don’t want to see a future in which your camera is LiveJournal-enabled or Facebook-ready. When I get a computer or a mobile device, it’s either Web-ready or not, it doesn’t have to be MSN-enabled or Yahoo-blessed.

I agree 100%. I worked on the APIs for MSN Spaces and this was one of my goals. The APIs we expose should be industry standards (de facto or de jure) and we shouldn't think of APIs as being the purvey of exlusionary deals. We are waiting for some things to pull together such as the the Windows Live ID client SDK before announcing more APIs. In the meantime, you can assume that any public APIs we provide will be available to all at not exlusionary. 

 

The MSN Spaces team blog has a new entry entitled Spaces Updates! which states

Hey Spaces fans, you may have noticed a change in the URL for your Space.  We’re excited to announce that we released late Monday evening (Seattle time.) 

 In addition to the URL Change, we have added support for many additional modules.  Some of these are:

• Horoscope*
• Weather*
• Stock Quotes*
• Quote of the Day*
• Updated Spaces (new Map View!)
• More modules coming soon including Live Gadgets.  Learn more about building a gadget

* Available in these countries for this release:   Australia, United Kingdom, & USA

I have the weather gadget on my space at http://carnage4life.spaces.msn.com. This is a step in the right direction, although ideally I should be able to write my own gadgets to use as modules on my space. That would be killer, except that it introduces a lot of interesting security problems if I could inject mini AJAX applications into my space. A tough problem to solve that would lead to some very cool mashups if figured out. I'd love to be able to add a few of the hundreds of existing gadgets to my space. I also suspect that there'd be a lot more interest in building gadgets when they have a potential audience of tens of millions of people versus to small audience that Live.com has today.

 

From the press release entitled Microsoft Hardware Advances Digital Communications Experiences we learn

Consumer research¹ has revealed that people want to stay connected, and many would use webcams if they were easier to operate and provided better audio and video quality. To counter those frustrations, Microsoft Hardware and the Windows Live team have joined forces to introduce a line of LifeCams starting with the LifeCam VX-6000 and LifeCam VX-3000.

These next-generation webcams provide groundbreaking video and audio quality that opens the door for richer digital communications experiences. Optimized for use with Windows Live™ Messenger, the world’s largest instant messaging network,² LifeCams meet the growing demand for easier, more meaningful connections.
...
Optimized for Windows Live Messenger

LifeCam VX-6000

Click for larger version.

The first two available webcams, the Microsoft LifeCam VX-6000 and Microsoft LifeCam VX-3000, bring a new dimension to Windows Live Messenger and feature exclusive industry firsts that streamline the webcam experience:

•	Windows Live Call Button. Located on the top of each LifeCam, the Windows Live Call Button makes placing a video call a breeze by eliminating the usual multiple steps. Just one touch brings up the Buddy Picker, a tool that shows users only current online buddies. They simply select their contact’s name and they are on their way to making a video call.
•	LifeCam Dashboard. Built right into the Windows Live Messenger window for easy access during video calling, the LifeCam Dashboard provides simple access to the controls people need most, including pan, tilt and zoom. Now users’ attention stays where it should be — on their video conversation.
•	One-touch blogging. Windows Live Spaces is one of the fastest-growing blog communities in the world, with more than 50 million individual Spaces. Now, users can post High Definition LifeCam pictures directly to their Windows Live Space blog with one click from within the LifeCam window.

This is another one of the product teams I've gotten to work with in recent months. I'm sure you can guess which of the listed bits of Windows Live integration I worked on. By the way, if you are a hardware vendor or into tweaking your hardware you might be interested in http://dev.live.com/hardware/. Building mashups with Windows Live services isn't going to be limited to Web apps, we expect hardware devices to get in on the game as well.

NOTE: MSN Spaces isn't Windows Live Spaces. Yet.

 

That's what I get for not getting financial advice from a professional. :)

 

This week is TechEd 2006, Microsoft's primary conference for IT professionals and developers. There'll be a bunch of announcements about Microsoft products over the next few days but with Robert Scoble on his way out I'm not sure where we are supposed to get our info.

Anyway, on Sunday there was a keynote given by Ray Ozzie which I thought was interesting and is excerpted below

For those of us close to IT, and who have been close to IT for many years, this is a jarring reversal from the days when we saw the latest innovations in computing and communications at places like NCC and COMDEX. In those days, the enterprise requirements for large-scale transaction systems, and the public sector requirements for large-scale scientific computing drove creation of the world's most advanced data centers. Enterprises were showcases for vendors' most sophisticated and scalable technologies.

Today some of the world's most advanced data centers are those designed to directly serve consumers out on the Internet. For example, last month there were about 130 million people who used Windows Live Spaces, another 230 million used our Messenger IM service. More than 250 million people used Windows Hotmail service, hundreds of millions of active, unique users each month. Clearly, building systems at this scale is different than building software for enterprise servers, which are designed to serve thousands or tens of thousands of concurrent users.

It's estimated that just among Microsoft and Yahoo and Google, there are well over 1 million servers racked up in data centers, located around the globe, serving trillions of e-mails, and IMs, and searches, housing many, many petabytes of storage, serving 1 billion Internet users. And the investment continues, you don't have to stray far from our Redmond headquarters to see.
...
At times of disruption like this there are always extremists. Twenty-five years ago, at the beginning of the PC revolution, some predicted the death of the mainframe, because of the PC. Now there are extremists who believe that every application will be accessed through a browser, and that everything will move to this computing cloud, that your enterprise data center will go away, that you'll trust third parties with your business information, and systems.

Microsoft is taking a very pragmatic approach; a seamless, blended, client-server-service approach. We want to make sure that you can easily transition client and server-based applications to services, or vice-versa. Our services won't be disconnected from existing applications, but instead are going to be designed to complement and extend our Windows and Office platforms to the Internet.

Under the name Live, we'll provide a blend of desktop software, server-based software, and our own enterprise service offering, and our partners' offerings, enabling you to make the right tradeoffs that make the most sense for your business. One notable example of this client-server-service synergy can be found in our approach to information management and search. Our goal is to provide the people within your organization a simplified, unified way of getting at the information that they need, no matter where it resides.

There are two themes I like here. The first is that it seems Ray Ozzie agrees with the My Website is Bigger Than Your Enterprise meme. The funny thing is that even though our CTO gets it, sometimes it is hard to explain to some of the folks working on server products at Microsoft. There is a big difference in the complexity and scale requirements to build a system like Hotmail or MSN Spaces versus building an Exchange or a SharePoint. There are lessons to learn on here, both on the part of vendors like Microsoft as well as customers of enterprise software who want to utilize the lessons that mega-scale online services have learned the hard way. The second theme is that there is a continuum of software experiences that spans both desktop applications and Web applications. I don't believe that Web applications will or should replace desktop applications. On the flip side, I think that desktop applications that don't harness the power of the network (the Web or the intranet) will begin to look archaic in a few years. Ray Ozzie seems to totally get this which is good for Microsoft. Maybe in a few years we can get Steve Gillmor to stop calling it Office Dead. :)

PS: From Trevin's TechEd update it looks like our MSN vs. Windows Live "branding strategy" is just as confusing to end users as I expected. He wrote

Booth fraffic[sic] was pretty light today, and the most frequent question was "What is Windows Live?".  People's guesses at what Windows Live was were all over the map -- some thought it was related to Live Communicator, others thought it was just the www.live.com homepage, while others took wild stabs in the dark.  Gerard commented that one person thought it might be related to our Education/Learning divison.

 

After talking to about 25 customers, it was abundantly clear that customers have no idea at all what Windows Live is, or how it relates to Windows or MSN.  This explained why there was so little traffic to our booth -- of the people that stopped by, they almost did it by accident.

Trevin is confident that our customers will soon have some of their confusion cleared. I on the other hand am not so sure. I'm sure once the major MSN properties like Hotmail, MSN Messenger and MSN Spaces are rebranded there will be more awareness of 'Windows Live' by customers. However I suspect the confusion around the difference between 'MSN' and 'Windows Live' will continue for quite a while. Maybe some of the marketing folks who fixed the weirdness of having both WinFX and the .NET Framework as dueling brands will be reorged into our division and can fix this foolishness.

 

Nick Carr has a blog post entitled Have Face Will Travel where he writes

So Microsoft's self-styled human face is now some other company's human face. This must be the first corporate human face transplant ever attempted. Will it take? Or will the new body reject the used puss? And what does it say about this whole human face business when a person proclaims himself to be a company's human face and then, when a better offer comes along, tears himself from the old noggin and stitches himself to the new one? That seems a little untoward to me. If I were in a punny mood, I just might call it a mugging.

A company should probably be a little nervous about letting some blogger set up shop as its human face. The earnings the blogger pulls in through the attention economy may accrue more to his own bottom line than the firm's.

I've been thinking about this a little over the past few days. The problem with having visible employees is that when they inevitably leave there is a potential negative PR hit. I started thinking about this when I read Gretchen Ledgard's post entitled closure about the emotional and career impact of having a highly visible corporate blog. I remember thinking that Microsoft's recruiting department had lost some of its shine when Gretchen and Zoe left. Seeing Robert Scoble combating the negative spin on his leaving Microsoft in his post Correcting the Record about Microsoft made similar thoughts come to mind.

What Nick Carr points out is a valid point, however I question the claim that the credit a company accrues due to visible employees accrues more to the employees than the company. It's definitely true that the employees get something out of it. Robert's clout as the 'human face' of Microsoft has gotten him as the cover story of magazines like The Economist and eventually has made him so famous it's been surprising (at least to me) that he'd want to stay at Microsoft as a mere evangelist when he could be off trading his brand for a better gig. On the other hand, there are thousands of people who've changed their impression of the company based on bloggers like Gretchen and Robert. This has directly impacted the hiring situation for the company [given the number of resumes that came in via Gretchen & Zoe's JobsBlog alone] as well as improved customer satisfaction for a lot of product groups.

At the end of the day, who wants to work at the kind of company that is worried that it's employees will become too popular and may get stolen away? Microsoft isn't that kind of company and that's one of the reasons it is a great place to work. If a company is so worried about employees becoming too visible then it probably has deeper personnel problems than just worrying about losing a few bloggers.

 

Recently one of the developers on my team asked what the difference was between a program manager and a product manager at Microsoft. It seemed to him that the roles were redundant. Before answering I answered, I suggested we look up both job descriptions at the Microsoft Career site. This is what we found

Product Management @ Microsoft: Product Managers help chart the course of those new products and services as they emerge. They work with the development team to determine product functionality, strategize product positioning, and drive the product launch. After introduction, the Product Management team monitors customer and partner response to the products and refines product strategy to ensure the best possible user experience.

Program Management @ Microsoft: Working across multiple groups with marketing and sales personnel on the customer end, program managers translate customer requirements into product features and create functional specifications. On the implementation end, they prioritize and deliver on those features, working closely with key technical resources, such as software development, testing, documentation, localization, tech support, and more.

Product Management tends to be about answering questions such as why a product should be built and what features should be added to existing products. Program Management is more about answering questions such as when features should be delivered and how they should be exposed to customers. Depending on the division at Microsoft program managers may have more or less influence than product managers on answering the questions about what features are built. 

Here are some examples from my day job that highlights the difference between the job roles. Product Management answers questions like what are the most popular MSN Spaces or MSN Messenger features in the U.S., UK & China and who are the top competitors in each market. Similarly, product management may also answer questions like 'what is the financial impact of focusing the next release of MSN Spaces to be more competitive with photo sharing sites like Flickr versus making it more competitive with social networking sites like MySpace. Program managers work on actually designing the software that ends up being built based on input from product management. There are User Experience (UX) PMs who work on the actual user interface design and the workflow of how features should work and then there are platform PMs [like me] who work on what the logical schema for various concepts in the application should be and design the APIs through which various front end & back end systems can interact with the feature(s).

In Windows Live we also have a difference between Product Planner and Product Manager. I actually have no insight into what the difference in job roles is here and wouldn't mind some education of my own.

 

On Friday, S. "Soma" Somasegar posted an entry in his blog entitled .NET Framework 3.0 where he wrote

When speaking to developers about WinFX one question that repeatedly comes up is, “WinFX sounds great, but what happens to .NET?” .NET Framework has becomes the most successful developer platform in the world.  Developers know and love .NET.

The .NET Framework has always been at the core of WinFX, but the WinFX brand didn’t convey this.  The WinFX brand helped us introduce the incredible innovations in terms of Windows Presentation Foundation (WPF), Windows Communication Foundation (WCF), Windows Workflow Foundation (WF) and the newly christened Windows CardSpace (WCS) formerly known under the codename “InfoCard.”  The brand also created an unnatural discontinuity between previous versions of our framework and the current version.

With this in mind we have decided to rename WinFX to the .NET Framework 3.0.  .NET Framework 3.0 aptly identifies the technology for exactly what it is – the next version of our developer framework.
...
The .NET Framework 3.0 will still ship with Windows Vista, and will be available down-level for Windows XP and Windows Server 2003 as planned.  This change doesn’t affect in any way the ship schedules of either Windows Vista or the .NET Framework 3.0 itself.

The good news to me isn't that Microsoft is fixing the branding confusion around having both WinFX and the .NET Framework. It is cool but what is more interesting is that developers can count on having Avalon (WPF) and Indigo (WCF) on every Windows Vista computer. As someone who's written an application based on the .NET Framework, it totally sucks that I still can't assume that every modern PC running the most up-to-date version of the operating system has the .NET Framework installed. It's finally gotten to the point where .NET Framework v1.1 has mass market penetration but we are on .NET Framework v2.0 and right now [based on my server logs] it looks like for every 1 person who is using v2.0 of the .NET Framework to access my site, there are 10 people on v1.1 of the .NET Framework. It's going to take at least a few more years for that ratio to get any better.

I've been wondering whether there is any point looking at Avalon given that it at this rate it could be four or five years before it has enough mass market penetration to be worth targetting exclusively. With the .NET Framework 3.0 shipping with every Windows Vista PC, the adoption rate should be a lot more rapid than what we've seen for previous versions of the .NET Framework. So now the next question for me is whether LINQ (aka C# 3.0) is expected to ship as part of the .NET Framework 3.0? If so, this would be the most interesting development for Windows developers I've heard all year.

This is probably old news to a bunch of folks but it is news to me.

 

Robert Scoble has posted an entry entitled Correcting the Record about Microsoft where he confirms statements by Dave Winer and Chris Pirillo that he is leaving Microsoft for PodTech.

This seems like a good move for Robert. He's always seemed more at home with his role as an amateur technology journalist than as the unofficial spokesman for Microsoft. Congrats to the PodTech folks for making a great hire and good luck to Robert on his new adventure. He definitely will be missed over here in the B0rg Cube.

 

I have a Dell laptop which was recently purchased and thus as a consequence of the Google<->Dell deal comes with a bunch of Google utilities installed such as the Google Toolbar and Google Desktop. In addition, it had http://www.google.com/ig/dell as the default home page. One thing I've noticed is every once in a while when I type a URL in the browser address bar, instead of going the web page I get search ads instead. Below is a screenshot of what happened when I typed http://www.apartments.com/ in the address bar recently.

Weird huh?

 

Windows Live Dev is now live. Wondering what it is? Check out the answer at What is Dev?

Windows Live Dev is your one-stop shop for the Windows Live Platform, including information on getting started with Windows Live services, latest documentation and APIs, samples, access to community areas and relevant blogs, and announcements of future releases and innovations.

Windows Live Dev is a new site and will be growing over time, adding more content and features. Please, let us know what you think of the site and what you’d like to see in the future. Post to our “Chatter” forum and start a conversation about what you’d like to see. We’ll see you there.

About the Windows Live Platform

The Windows Live Platform puts a deeper level of control into developers' hands by offering access to the core services and data through open, easily accessible APIs. Now you can build applications and mashups that combine your innovation with the power of Windows Live services and social relationships.

This is awesome. This is the third developer website I've been a part of getting started at Microsoft (http://msdn.microsoft.com/xml and http://msdn.microsoft.com/msn are the others) and yet I can never get over how great it feels to see something go from an idea on a whiteboard to reality. I wanted the URL of the site to be http://developer.live.com but Brian (our VP) suggested http://dev.live.com which definitely has a better ring to it. Check it out and let us know what you think. 

I'll be doing the group blog thing at http://dev.live.com/spaces. Expect interesting new API announcements in the coming months and perhaps even a peep or two about the rumored support for gadgets coming to Windows Live Spaces.

 

Tim O'Reilly ran a series of blog posts a few months ago on the O'Reilly Radar blog entitled "Database War Stories" where he had various folks from Web companies talk about their issues scaling databases. I thought the series of posts had an interesting collection of anecdotes and thus I'm posting this here so I have a handy link to the posts as well as the highlights from each entry.

1. Web 2.0 and Databases Part 1: Second Life: Like everybody else, we started with One Database All Hail The Central Database, and have subsequently been forced into clustering. However, we've eschewed any of the general purpose cluster technologies (mysql cluster, various replication schemes) in favor of explicit data partitioning. So, we still have a central db that keeps track of where to find what data (per-user, for instance), and N additional dbs that do the heavy lifting. Our feeling is that this is ultimately far more scalable than black-box clustering.

2. Database War Stories #2: bloglines and memeorandum: Bloglines has several data stores, only a couple of which are managed by "traditional" database tools (which in our case is Sleepycat). User information, including email address, password, and subscription data, is stored in one database. Feed information, including the name of the feed, description of the feed, and the various URLs associated with feed, are stored in another database. The vast majority of data within Bloglines however, the 1.4 billion blog posts we've archived since we went on-line, are stored in a data storage system that we wrote ourselves. This system is based on flat files that are replicated across multiple machines, somewhat like the system outlined in the Google File System paper, but much more specific to just our application. To round things out, we make extensive use of memcached to try to keep as much data in memory as possible to keep performance as snappy as possible.

3. Database War Stories #3: Flickr: tags are an interesting one. lots of the 'web 2.0' feature set doesn't fit well with traditional normalised db schema design. denormalization (or heavy caching) is the only way to generate a tag cloud in milliseconds for hundereds of millions of tags. you can cache stuff that's slow to generate, but if it's so expensive to generate that you can't ever regenerate that view without pegging a whole database server then it's not going to work (or you need dedicated servers to generate those views - some of our data views are calculated offline by dedicated processing clusters which save the results into mysql).

4. Database War Stories #4: NASA World Wind: Flat files are used for quick response on the client side, while on the server side, SQL databases store both imagery (and soon to come, vector files.) However, he admits that "using file stores, especially when a large number of files are present (millions) has proven to be fairly inconsistent across multiple OS and hardware platforms."

5. Database War Stories #5: craigslist: databases are good at doing some of the heavy lifting, go sort this, give me some of that, but if your database gets hot you are in a world of trouble so make sure can cache stuff up front. Protect your db!

   you can only go so deep with master -> slave configuration at some point you're gonna need to break your data over several clusters. Craigslist will do this with our classified data sometime this year.

   Do Not expect FullText indexing to work on a very large table.

6. Database War Stories #6: O'Reilly Research: The lessons:
   

• the need to pay attention to how data is organized to address performance issues, to make the data understandable, to make queries reliable (i.e., getting consistent results), and to identify data quality issues.
  
• when you have a lot of data, partitioning, usually by time, can make the data usable. Be thoughtful about your partitions; you may find its best to make asymmetrical partitions that reflect how users most access the data. Also, if you don't write automated scripts to maintain your partitions, performance can deteriorate over time.

7. Database War Stories #7: Google File System and BigTable: Jeff wrote back briefly about BigTable: "Interesting discussion. I don't have much to add. I've been working with a number of other people here at Google on building a large-scale storage system for structured and semi-structured data called BigTable. It's designed to scale to hundreds or thousands of machines, and to make it easy to add more machines the system and automatically start taking advantage of those resources without any reconfiguration. We don't have anything published about it yet, but there's a public talk about BigTable that I gave at University of Washington last November available on the web (try some searches for bigtable or view the talk)."

8. Database War Stories #8: Findory and Amazon: On Findory, our traffic and crawl is much smaller than sites like Bloglines, but, even at our size, the system needs to be carefully architected to be able to rapidly serve up fully personalized pages for each user that change immediately after each new article is read. Our read-only databases are flat files -- Berkeley DB to be specific -- and are replicated out using our own replication management tools to our webservers. This strategy gives us extremely fast access from the local filesystem. We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load. Our much smaller read-write data set, which includes information like each user's reading history, is stored in MySQL. MySQL MyISAM works very well for this type of non-critical data since speed is the primary concern and more sophisticated transactional support is not important.

9. Database War Stories #9 (finis): Brian Aker of MySQL Responds: Brian Aker of MySQL sent me a few email comments about this whole "war stories" thread, which I reproduce here. Highlight -- he says: "Reading through the comments you got on your blog entry, these users are hitting on the same design patterns. There are very common design patterns for how to scale a database, and few sites really turn out to be all that original. Everyone arrives at certain truths, flat files with multiple dimensions don't scale, you will need to partition your data in some manner, and in the end caching is a requirement."

   I agree about the common design patterns, but I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.

I found most of the stories to be interesting especially the one from the Flickr folks. Based on some early thinking I did around tagging related scenarios for MSN Spaces I'd long since assumed that you'd have to throw out everything you learned in database class at school to build anything truly performant. It's good to see that confirmed by more experienced folks.

I'd have loved to share some of the data we have around the storage infrastructure that handles over 2.5 billion photos for MSN Spaces and over 400 million contact lists with over 8 billion contacts for Hotmail and MSN Messenger. Too bad the series is over. Of course, I probably wouldn't have gotten the OK from PR to share the info anyway. :)

 

The MSN Spaces team's blog has a few entries about one of the projects they've been working on in collaboration with our team. From the posts URL Changes and More info on the URL Changes we learn

All MSN Spaces Users:

Please note that your MSN Space's URL will change on June 5 8, 2006.  As part of investments in the improvement of MSN Spaces, it we will be migrating all of the URLs from http://spaces.msn.com/<NAME> to http://<NAME>.spaces.msn.com.   (For instance, instead of http://spaces.msn.com/thespacecraft/  you will now see http://thespacecraft.spaces.msn.com.)

 On and after June 5th 8th, all viewers and users going to the "old" URL will be automatically redirected to the new URL.
...
Spaces has grown very quickly into one of the Web’s mega services. So quickly in fact that we just passed the 100MM user mark and have had to do some architectural changes to ensure that Spaces can be deployed in multiple data centers. We needed to deliver a system that allows for Spaces to be distributed across multiple data centers without requiring a URL that included the data center name. How unkewl would that be? Can you imagine telling your friends and family that the URL to your space was http://cluster25.dc1.spaces.msn.com/gphipps?

So we have developed a DNS (Domain Name System) based solution that allows us to redirect requests to the right data center and allows us to keep a better looking URL. Moving the Space name into the domain name is a requirement of that.
...
Doing the rearchitecture work and making the move to Live Spaces was not possible for a number of technical reasons. This is why we can’t move straight to the spaces.live.com name. However, we believe that when we do move to Live Spaces that will be the last time we have to change the URL. This really isn’t something we decided to do lightly. We have had to make a ton of tradeoffs from both a technical perspective and the impact to our users.

Converting a service as large as MSN Spaces and it's associated services from a single data center to be able to be deployed in multiple data centers has been a significant undertaking. One unfortunate side effect is that we've had to alter the URL structure of MSN Spaces. Doubly unfortunate is that the URL structure will change again with the switch to Windows Live Spaces.

Although these changes suck, they are necessary to ensure that we can continue to handle the explosive growth of the service across the world as well as pump out crazy new features. Thanks to all our users who have to bear with these changes. 

 

Samir from the Windows Live Expo team has posted an entry entitled Attention Developers: Expo API RTW where he writes

The first version of the Expo API went live this week.  The API gives developers read access to the Expo classifieds listings database.  All our listings are geo-tagged, so there are some cool posibilities for mashups with some of those mapping APIs.

 

The API docs are published on MSDN and you can find them here.  You can get started by signing up for an Application key at http://expo.live.com/myapikeys.aspx.

 

If you've got a slick working demo using our APIs, please let us know at expoapi@microsoft.com so we can link to it.  The nuttier and more creative, the better!  Also, make sure you post a link from this blog post.

I took a gander at the API before I shipped and there are a number of things I like about it. The first thing I like is that the API has both a RESTful interface and a SOAP interface. Even better is that the RESTful interface is powered by RSS with a couple of extensions thrown in. This API and others like it that we have planned is one of the reasons I've been thinking about Best Practices for Extending RSS and Atom. On the other hand, I don't like that the API takes latitude/longitudes instead of addresses and zip codes. This means if I want to write an app that uses the Expo API, I need to also use a geocoding API. This may make it easier to integrate Windows Live Expo into Map-based mashups though. I've asked Samir about also accepting addresses and zip code support in the API but I suspect the team will need to get the request from a couple more folks before they take it seriously. :(

I've promised the Expo folks that I'll write a Live.com gadget using the API which I hope to get started on soon. Given that I've also promised the folks at O'Reilly's XML.com an article on building Live.com gadgets by next week, I better get cracking. It's a good thing I have a copious amount of free time. ;)

 

Joe Wilcox of Jupiter Research has a blog post entitled Google My Spreadsheet where he talks about Google's recently announced Web-based spreadsheet application. He writes

So, Google is doing a spreadsheet. On the Web. With collaboration. And presumably--if and when released--Google Spreadsheet will be available for free. I predict there will be crisis meetings at Microsoft today. I totally agree with Colleague David Card that "Google is just playing with Microsoft's (hive) mind. Scaring the troops. Sleight-of-handing the managers."

Perhaps the real Google competitive threat isn't any product or products, but the information vendor's ability to rustle Microsoft corporate paranoia. To get Microsoft chasing Google phantoms, and in the process get Microsoft's corporate mind off its core business. News media will be gaga with Google competing with Microsoft stories--two juggernauts set to collide. Yeah, right. I'd be shocked if Google ever released a Web browser, operating system or desktop productivity suite. Those markets aren't core to Google's business, contrary to speculation among news sites and bloggers.

As for the spreadsheet, which isn't really available beyond select testers, what's it really going to do? David is right, "Consumers don't use spreadsheets. No thinking IT manager would sign off on replacing Excel with a Web-based spreadsheet." Google's target market, if there really is one, appears to be casual consumer and small business users of spreadsheets--people making lists. OK, that competes with Microsoft how? So soccer mom or jill high schooler can work together with other people from the same based Web-based spreadsheet. Microsoft shouldn't really sweat that, although Microsoft might want to worry about what Google might do with extending search's utility.

I agree 100% with Joe Wilcox's analysis here. This seems more like a move by Google to punk Microsoft into diverting focus from core efforts than a product category that is well thought out. I thought at the recent Google Press Day, Eric Schmidt mentioned that they have not been doing a good job of following the 70/20/10 principle (70 percent focus on core business, 20 percent on related businesses and 10 percent on new businesses).

If I was a Google investor, I'd be worried about the fact that their search engine relevance is detoriorating (Google Search for "msdn system.string" doesn't find this page in the top 10 results) and they are wasting resources fragmenting their focus in so many myriad ways. As a competitor, it makes me smile. ;) 

Update: From the Google blog post entitled It's nice to share it looks like this is an offerings from the creators of XL2Web who were acquired by Google a year ago. Another example of innovation by acquisition by Google? Interesting.

 

Guy Kawasaki has a blog post entitled The Top Sixteen Lies of CEOs which has the following item on the list

2. “It’s like a startup around here.” This could mean that the place lacks adult supervision; capital is running out; the product is behind schedule; investors have given up, and employees are paid below market rates. Sure, it could alternatively mean that the company is energized, entrepreneurial, making meaning, and kicking butt, but just be sure to double check.

One thing that amuses me at work is that every once in a while I see some internal team on a recruitment drive and one of the selling points is that "working for us is like working at a startup". The seems pretty amusing on the face of it given that most of the characteristics one associates with working at a startup are negative as Guy suggests. Paul Graham pointed out in his essay How to Make Wealth that the main attraction of working at a startup is that you can get really rich working at it. That makes getting underpaid, overworked and in constant fear of your competitors worth it at the end.

Unless your team can guarantee folks a huge chunk of change if your product is successful then working there is not like working at a startup. There are lots of ways to describe how cool your workplace is without resorting to the flawed comparison to working at a startup.   

Thanks for listening.
 

Seeing Jon Udell's post about having difficulty with the Google PR team with regards to discussing the Google GData API reminded me that I needed to write down some of my thoughts on extending RSS and Atom based on looking at GData. There are basically three approaches one can take when deciding to extend an XML syndication format such as RSS or Atom

1. Add extension elements in a different namespace: This is the traditional approach to extending RSS and it involves adding new elements as children of the item or atom:entry element which carry application/context specific data beyond that provided by the RSS/Atom elements. Microsoft's Simple Sharing Extensions, Apple's iTunes RSS extensions, Yahoo's Media RSS extensions and Google's GData common elements all follow this model.
   

2. Provide links to alternate documents/formats as payload: This approach involves providing links to additional data or metadata from an item in the feed. Podcasting is the canonical example of this technique. One argument for this approach is that instead of coming up with extension elements that replicate existing file formats, one should simply embed links to files in the appropriate formats. This argument has been used in various discussions on syndicating calendar information (i.e. iCalendar payloads) and contact lists (i.e. vCard payloads). See James Snell's post Notes: Atom and the Google Data API for more on this topic.
   

3. Embed microformats in [X]HTML content: A microformat is structured data embedded within another markup language (typically HTML/XHTML). This allows one to represent both human-readable data and machine-readable data in a single document. The Structured Blogging initiative is an example of this technique.
   

All three approaches have their pros and cons. Option #1 is problematic because it encourages a proliferation of duplicative extensions and may lead to fragmenting the embedded data into multiple unrelated elements instead of a single document/format. Option #2 requires RSS/Atom clients to either build parsers for non-syndication formats or rely on external libraries for consuming information in the feed. The problem with Option #3 above is that it introduces a dependency on an HTML/XHTML parser for extracting the embedded data from the content of the feed.

From my experience with RSS Bandit, I have a preference for Option #1 although there is a certain architectural purity with Option #2 which appeals to me. What do the XML syndication geeks in the audience think about this?

 

The Windows Live Mail Desktop team have a blog post entitled Better Together with Active Search where they talk about a new feature currently being called "Active Search". The post is excerpted below

Much of what you need to get done online – from planning your next vacation to remembering to buy flowers for your mom on her birthday – is piling up in your inbox, just waiting for you to take action, usually by looking something up on the web.

With this in mind, we’ve designed Active Search to make it easier for you to act on anything that piques your interest while reading your email.That’s why we show you key search terms we find in a message and provide a search box right underneath, so you can quickly search for terms of your own.

We also show search results and sponsored links right inline, so you can see what’s related to your message on the web, without having to open a new browser window. Of course, if you come across something really interesting, just click More results… and we’ll open a new window with a full set of search results for you to dive into.

Because we only look for relevant keywords in the current email message or RSS article you happen to be viewing in your inbox, there are times when we just can’t find anything relevant enough to show you. So we don’t – we just show a search box ready for you to enter search terms you happen to come up with while reading the message.

I got a demo of this feature from Bubba in the cafeteria a few weeks ago and it seemed pretty interesting. It reminds me of text ads in GMail but for a desktop application and a few other key differences. What I'd love to know is whether there is a plan to make some of this stuff available as APIs for non-Windows Live applications. I wouldn't mind being able to integrate search ads into RSS Bandit to offset some of our hosting costs.

 

Mark Cuban has blog post entitled Why I think ClickFraud is far greater than imagined where he lays out some anecdotal reasons why he thinks the sky is falling for Pay-Per-Click (PPC) advertising due to click fraud. He writes

Now i have no idea how much money is being lost to click fraud. All i know is that when the black hat hackers see easy money, they take it. I also know that they are greedy and a jealous bunch. The more they see the more they take, so you can pretty well bet that the amount of click fraud is going up by the minute.

And no amount of IP repetition algorithms are going to stop them.

Again, this is all opinion. No investigative reporting going on. Yet.

I have no hard numbers on how much is being "lost" to click fraud but there are a number of reasons why I'm skeptical about how much attention people pay to scare mongering about click fraud and it's effects on companies like Google when 'the market corrects'.

Reason number one is that despite how much people may complain about PPC advertising, it works a lot better than the alternatives. Mark Cuban actually just wrote a blog post a few days ago entitled A quick letter to the Newspaper and Magazine Industries where he complains about how expensive traditional advertising is compared to the returns. If you were trying to drive traffic to your website and had a million dollars to spend would you spend it on newspaper ads, television ads or AdSense/Adwords ads? As long as the return on investment (ROI) for PPC ads is higher than other forms of advertising, I suspect advertisers will consider the money lost to click fraud as acceptable losses. This is no different from retail stores which have to accept a certain amount of loss from shop lifting and customer returns yet still remain profitable.

Another reason I'm skeptical about fear mongering around click fraud is that this isn't the first time technology has made it easy to subvert a market yet those markets still exist. My employer (Microsoft) has built one of the most profitable businesses in the world selling products that can be copied by anyone with a CD burner and a computer. In college, I don't think I knew many people who paid for their copies of Microsoft Office yet that business still manages to bring in billions of dollars in profit a year. Then there are other more recent markets like ring tones. This has emerged as a multi-billion dollar industry in the past few years even though it is quite possible for people to get ringtones for free on their cell phones without much hassle. And then there's the iTunes Music Store. 

I'd be unsurprised if there is a larger than commonly assumed amount of click fraud going on out there. No more surprised than I'd be to find out that software/music piracy rings exist, insurance fraud is about 15 %- 30% of insurance claims or that shoplifting costs retailers billions of dollars every year. I don't see people ringing the death knell of WalMart or Safeco because of these grim statistics. So the next time you see someone claiming that the sky is falling because of click fraud, remember that fraud exists in every aspect of human commerce and the Web isn't going to change that. 

 

While I was in the cafeteria with Mike Vernal this afternoon I bumped into some members of the Windows Desktop Search team. They mentioned that they'd heard that I'd decided to go with Lucene.NET for the search feature of RSS Bandit instead of utilizing WDS. Much to my surprise they were quite supportive of my decision and agreed that Lucene.NET is a better solution for my particular problem than relying on WDS. In addition, they brought an experienced perspective to a question that Torsten and I had begun to ask ourselves. The question was how to deal with languages other than English.

When building a search index, the indexer has to know what the stop words it shouldn't index are (e.g. a, an, the) as well as have some knowledge about word boundaries. Where things get tricky is that a user can receive content in multiple languages, you may receive email in Japanese from some friends and English from others. Similarly you could subscribe to some feeds in French and others in Chinese. Our original thinking was that we would have to figure out the language of each feed and build a separate search index for each language. This approach seemed error prone for a number of reasons

1. Many feeds don't provide information about what language they are in
   
2. People tend to mix different languages in their speech and writing. Spanglish anyone?
   

The Windows Desktop Search folks advised that instead of building a complicated solution that wasn't likely to work correctly in the general case, we should consider simply choosing the indexer based on the locale/language of the Operating System. This is already what we do today to determine what language to display in the UI and we have considered allowing users to change the UI language in future which would also affect the search indexer [if we chose this approach]. This assumes that people read feeds primarily in the same language that they chose for their operating system. This seems like a valid assumption but I'd like to hear from RSS Bandit users if this is indeed the case. 

If you use the search features of RSS Bandit, I'd appreciate getting your feedback on this issue.