I’ve made some more progress in integrating the Facebook news feed into the next version of RSS Bandit currently codenamed Colossus. This weekend I completed the addition of support for viewing and replying to comments in the news feed. So here are some screenshots of current comment workflow for interacting with Facebook comments

Fig 1: Viewing the comments in response to a funny status update from Anil Dash 

Fig 2: Responding to the comment by pressing "Ctrl + R" or right-clicking and selecting Post Reply.

Fig 3: The news feed on Facebook with the comment posted from RSS Bandit

The second major change coming in the Colossus release is the adoption of the design elements from the Microsoft Office fluent user interface such as the ribbon, contextual tabs, galleries and live preview. To prepare for this change, we’re first building a prototype of the redesigned user interface and once we’re happy with it we will start refactoring the RSS Bandit application to enable swapping out our existing menus and taskbars with the new interface.

Here’s where we are in the design prototype for next release. Let me know what you think in the comments.

In the past week or so, two of the biggest perception problems preventing proliferation of OpenID as the de facto standard for decentralized identity on the Web have been addressed. The first perception problem is around the issue of usability. I remember attending the Social Graph Foo Camp last year and chatting with a Yahoo! employee about why they hadn’t become an Open ID relying party (i.e. enable people to login to Yahoo! account with OpenIDs). The response was that they had concerns about the usability of OpenID causing reducing the number of successful log-ins given that it takes the user off the Yahoo! sign-in page to an often confusing and poorly designed page created by a third party.

Last year’s launch and eventually success of Facebook Connect showed developers that it is possible to build a delegated identity workflow that isn’t as intimidating and counterproductive as the experience typically associated with delegated identity systems like OpenID. On May 14th, Google announced that a similar experience has now been successfully designed and implemented for OpenID in the Google Code blog post titled Google OpenID API - taking the next steps which states

We are happy to announce today two new enhancements to our API - introducing a new popup style UI for our user facing approval page, and extending our Attribute Exchange support to include first and last name, country and preferred language.

The new popup style UI, which implements the

OpenID User Interface Extension Specification, is designed to streamline the federated login experience for users. Specifically, it's designed to ensure that the context of the Relying Party website is always available and visible, even in the extreme case where a confused user closes the Google approval window. JanRain, a provider of OpenID solutions, is an early adopter of the new API, and already offers it as part of their RPX product. As demonstrated by UserVoice using JanRain's RPX, the initial step on the sign-in page of the Relying Party website is identical to that of the "full page" version, and does not require any changes in the Relying Party UI.

Once the user selects to sign in using his or her Google Account, the Google approval page is displayed. However, it does not replace the Relying Party's page in the main browser window. Instead it is displayed as a popup window on top of it. We have updated our Open Source project to include a complete Relying Party example, providing code for both the back-end (in Java) and front-end (javascript) components.

Once the user approves the request, the popup page closes, and the user is signed in to the Relying Party website.

The aforementioned OpenID User Interface Extension allows the relying party to request that the OpenID provider authenticate the user via a “pop up” instead of through navigating to their page and then redirecting the user back to the relying party’s site. Thus claim that OpenID usability harms the login experience is now effectively addressed and I expect to see more OpenID providers and relying parties adopt this new popup style experience as part of the authentication process.

The second biggest perception blocker is the one asked in articles like Is OpenID Being Exploited By The Big Internet Companies? which points out that no large web companies actually support OpenID as a way to login to their primary services. The implication being that companies are interested in using OpenID as a way to spread their reach across the web including becoming identity providers for other companies but don’t want others to do the same to them.

That was true until earlier this week when Luke Shepard announced Facebook Supports OpenID for Automatic Login. Specifically,

Now, users can register for Facebook using their Gmail accounts. This is a quicker, more streamlined way for new users to register for the site, find their friends, and start exploring.

Existing and new users can now link their Facebook accounts with their Gmail accounts or with accounts from those OpenID providers that support automatic login. Once a user links his or her account with a Gmail address or an OpenID URL, logs in to that account, then goes to Facebook, that user will already be logged in to Facebook.

In tests we've run, we've noticed that first-time users who register on the site with OpenID are more likely to become active Facebook users. They get up and running after registering even faster than before, find their friends easily, and quickly engage on the site.

This makes Facebook the first major web company to truly embrace OpenID as a way to enable users to sign up and login to the site using credentials from a third party (a competitor even). The fact that they also state that contrary to popular perception this actually improves the level of engagement of those users is also a big deal.

Given both of these events, I expect that we’ll see a number of more prominent sites adopting OpenID as they now clearly have nothing to lose and a lot to gain by doing so. This will turn out to be a great thing for users of the web and will bring us closer to the nirvana that is true interoperability across the social networking and social media sites on the web.

After all the hype, I got around to taking Wolfram Alpha for a spin last night due to being unable to sleep after a weird Doctor Manhattan themed nightmare. The experience of using the site is very impressive and there is a great walkthrough of the power of the site in the Wolfram Alpha screencast which I encourage people to watch if you are interested in learning about a new breed of search engine.

There have been a ton of articles calling Wolfram Alpha a "Google Killer" but after using the site for a few hours although I find it fascinating, I question how much of a threat the site is to Google either as a way to satisfy the typical questions people ask Web search engines or a threat to Google’s search advertising cash cow. You can get a sense for the kinds of queries that Wolfram Alpha handles amazingly well from the list below

• life expectancy 40 year old male UK
• GOOG YHOO
• nathan name
• president of nigeria 2004
• weather seattle christmas day 2008
• microsoft
• ethnic groups nigeria

As you can tell from the above list, Wolfram Alpha is like having a search engine over the kind of data you’d see in the CIA's World Factbook or Time Almanac. There really isn’t anything like it on the Web today. However it isn’t really a competitor to traditional web search engines who for the most part are still focused on finding web pages despite the various advancements in answering a subset of queries with direct answers instead of links to web pages such as Google's OneBox results and Live Search’s instant answers feature.

From my perspective, the threat to search engines like Google isn’t Wolfram Alpha but the trend it represents. That trend is the renaissance of the vertical search engine. Earlier this year, I was putting together a panel at the MIX ‘09 conference and needed to invite the panelists from a pool of people who I’d either heard about or knew of professionally but had never contacted directly. How did I find out how to contact these people?  Even though all of them had blogs, there wasn’t a consistent way to track down contact information. So I looked them up on Facebook and sent each of them a private message. Mission accomplished. Unbeknownst to me, Facebook had become my “people” search engine”.

Here’s another story. Last year I worked on the most satisfying software release of my career, Windows Live (wave 3). After the launch I wanted to find out what people were saying about the product so I did a Twitter search for Windows Live and posted the results. While I wasn’t paying attention, Twitter had become my “what are people saying about <insert brand here>” search engine.

This trend of search engines dedicated to specific scenarios and contexts that can’t be answered well by Web search is the trend that traditional search engines should watch carefully.

I can imagine Wolfram Alpha eventually growing to satisfy a lot of the sorts of queries I go to Wikipedia today to get answers to and doing so in a more authoritative manner. In that case, it would become my “facts and trivia” search engine. However there are currently too many gaps in its knowledge of commercial products (e.g. search for “ipod” results in a coming soon notice) and people (e.g. the Jim Carrey entry is amazingly brief yet still manages to have a factually inaccuracy) to make it a true replacement for wikipedia. That said, the service shows great promise and it will be interesting watching as it evolves. 

Biz Stone, Twitter’s , recently wrote in a blog post entitled The Replies Kerfuffle that

We removed a setting that 3% of all accounts had ever touched but for those folks it was beloved.
…
97% of all accounts were not affected at all by this change—the default setting is that you only see replies by people you follow to people you follow. For the 3% who wanted to see replies to people they don't follow, we cannot turn this setting back on in its original form for technical reasons and we won't rebuild it exactly the same for product design reasons.
…
Even though only 3% of all Twitter accounts ever changed this setting away from the default, it was causing a strain and impacting other parts of the system. Every time someone wrote a reply Twitter had to check and see what each of their followers' reply setting was and then manifest that tweet accordingly in their timeline—this was the most expensive work the database was doing and it was causing other features to degrade which lead to SMS delays, inconsistencies in following, fluctuations in direct message counts, and more. Ideally, we would redesign and rebuild this feature but there was no time, hence the sudden deploy.

As someone whose day job is working on a system for distributing a user’s updates and activities to their social network in real-time across Web and desktop applications, I’m always interested in reading about the implementation choices of others who have built similar systems. However when it comes to Twitter I tend to be more confused than enlightened whenever something is revealed about their architecture.

So let’s look at what we know. When Ashton Kutcher posts an update on Twitter such as

it has to be delivered to all 1.75 million of his followers. On the other hand, when Ashton Kutcher posts an update directed to one of his celebrity friends such as

then Twitter needs to decide how to deliver it based on the Replies settings of users.

One option would to check each of the 1.75 million followers of aplusk’s setting to decide whether they need have @replies restricted to only people they are following. Since this will be true for 97% of his followers (i.e. 1.7 million people) then there would need to be a 1.7 million checks to see if the intended recipient are also friends of John Mayer before  delivering the message to each of them. On the other hand, it would be pretty straightforward to deliver the message to the 3% of users who want to see all replies. Now this seems to be what Biz Stone is describing as how Twitter works but in that case the default setting should be more expensive than the feature that is only used by a minority of their user base.

In that case I’d expect Twitter to argue that the feature they want to remove for engineering reasons is filtering out some of the tweets you see based on whether you are a follower of the person the message is directed to not the other way around.

What have I missed here? 

Update: A comment on Hacker News put me on track to what I probably missed in analyzing this problem. In the above example, if the default case was the only case they had to support then all they have to do to determine who should receive Ashton Kutcher’s reply to John Meyer is perform an intersection of both user’s follower lists. Given that both lists need to be in memory for the system to be anywhere near responsive, performing the intersection isn’t as expensive as it sounds.

However with the fact that 3% of users will want to have received the update even though they aren’t John Mayer’s friends means Twitter needs to do a second pass over whoever was not found in both follower lists and check what their @reply delivery settings are. In the above example, even if every follower of John Mayer was a follower of Ashton Kutcher, it would still require 750,000 settings checks. Given that it sounds like they keep this setting in their database instead of in some sort of cache, it is no surprise that this is a feature they’d like to eliminate. 

Joshua Porter has an excellent post titled Are you building an everyday app? (the LinkedIn problem)  where he writes

In a recent interview, LinkedIn CEO Reid Hoffman describes moving away from day to day to a more strategic role in the company he founded:

I want to be able to sink my mind around a couple of problems and work through them. For example, many professionals still don’t understand how LinkedIn can be valuable on a daily or weekly basis”

Another way you could phrase this is: “people don’t use LinkedIn everyday…we need to figure out how to change that”.

The fact is that LinkedIn, in its current incarnation, is not an everyday app. An everyday app is one that is used every day (or most days) by its users.
...
In general, most people think they’re building an everyday app, but they’re not. When the actual use patterns are discovered, most apps will be used every few days or less. Designers have to ask themselves a very hard question: “How often are people really going to use our web application?”. The answer is important…it will even help drive design decisions. Whether or not you have an everyday app affects the entire design of what you’re building, including the screens, notifications, and frequency of the service. For example, only everyday apps really need to use real-time technology to update streams. If you find out that you’re not building an everyday app, you probably don’t need to invest in making it real-time. But…you might invest in a notifications system that can alert users to when something very interesting happens.

You don’t have to be an everyday app to be successful. Netflix, for example, is not an everyday app. It’s an every-few-days app. Most people go back every few days to update their queue. There is really no need to go back more often.

Many developers of social software applications on the Web believe they’ve built an everyday app but they actually haven’t. One thing I’ve learned in almost five years of working on social software applications at Microsoft is that simply having the features of an everyday app doesn’t translate to people using the application every day. The best way to think about this is that no application starts off as an everyday app. Very rarely does an application show up that is so amazing that people start using it everyday right off the bat. Instead there is a transition where either users transition from occasional to frequent users while the application stays static or the application itself transitions to catering to a more engaged user base relative to where it was in previous years.

An example of the former is Twitter, the site hasn’t really changed much since I started using it about a year and a half ago. However it wasn’t until the right set of factors came together such as getting enough people I was following, adding the Twitter app to Facebook & Windows Live to update my friends on those services and getting Twitter applications for my desktop & phone did I transition to becoming an everyday user. Twitter’s main problem is that not every user eventually hits this sweet spot which is why you read articles like Twitter Quitters Post Roadblock to Long-Term Growth which points out that retention rate over a one month period hovers between 30% – 40% for new users. This need to make the application instantly useful to users is what prompts features like the Suggested Users List whose purpose is to give new users content worth coming back to every day instead of the “Trying out Twitter for the first time” style posts that they probably see from most of their friends who are also kicking the tires on the service they heard about on Oprah.

Having features that are useful every day like a constantly updating activity stream doesn’t mean people will use the site every day. For the users that cross the hump, it does. The challenge is how to get users to cross that hump.

A site that has done a good job of motivating its users to check it out on a daily basis and adjusting its features as its user base has become more engaged is Facebook. One of Facebook’s most annoying features for a long time was the fact that notifications on new messages in the service didn't actually contain the message. I suspect the purpose of doing this was to drive users back to the site so that they would then catch up on all they’d missed such as invitations and content in the news feed after they were done answering the message. Although this feature is annoying, it was definitively effective given anecdotal feedback from various people I talked to at the time. After a certain point, Facebook’s user engagement grew to the point where sending messages without the content to drive users back to the site wasn’t worth it relative to the decreased customer satisfaction.

Another great example from Facebook is the transition from the old news feed to the new stream. In March of 2009, the news feed was transformed into a real-time stream and almost two months later the real-time stream now updates live without having to refresh the page. The previous functionality of the news feed was relegated to an alternate highlights stream as shown below 

From reading the blog posts about the changes, the problem the switch from a highlights-centric news feed to a real-time stream is addressing is the fact that the highlights-based feed is stale and doesn’t provide enough value for users who’ve not just become every day users but are now every hour users. Not only do over 100 million of its users login every day but with 90 million users generating 90 billion page views in the month of March 2009 that implies the average page view for a Facebook user is over once per hour.

And when you think about it, the introduction of the original news feed in 2006 was a successful attempt to go from being an occasionally updated rolodex for a large chunk of their users into a social utility to keep up with what’s going on in the lives of family, friends and coworkers. The switch to a real-time stream is how Facebook is addressing the fact that they have slowly become an every hour app instead of just an everyday app for their users.

A number of services I use online could learn from how Facebook has evolved their experience over time increase the engagement of their user base by making sometimes small and sometimes huge changes to the user experience to encourage people to make the service a regular part of their lives.