Last week, Douglas Bowman posted a screed against making web design based strictly on usage data. In a post entitled Goodbye Google, he wrote

With every new design decision, critics cry foul. Without conviction, doubt creeps in. Instincts fail. “Is this the right move?” When a company is filled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative effects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions.

Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such minuscule design decisions. There are more exciting design problems in this world to tackle.

I can’t fault Google for this reliance on data. And I can’t exactly point to financial failure or a shrinking number of users to prove it has done anything wrong. Billions of shareholder dollars are at stake. The company has millions of users around the world to please. That’s no easy task. Google has momentum, and its leadership found a path that works very well.

One thing I love about building web-based software is that there is the unique ability to try out different designs and test them in front of thousands to millions of users without incurring a massive cost. Experimentation practices such as A/B testing and Multivariate testing enable web designers to measure the impact of their designs on the usability of a site on actual users instead of having to resort to theoretical arguments about the quality of the design or waiting until after they've shipped to find out the new design is a mistake.

Experimentation is most useful when you have a clear goal or workflow the design is trying to achieve and you are worried that a design change may impact that goal. A great example of this is how shopping cart recommendations were shipped at Amazon which is recalled in a great story told by Greg Linden in his post Early Amazon: Shopping cart recommendations excerpted below

The idea of recommending items at checkout is nothing new. Grocery stories put candy and other impulse buys in the checkout lanes. Hardware stores put small tools and gadgets near the register. But here we had an opportunity to personalize impulse buys. It is as if the rack near the checkout lane peered into your grocery cart and magically rearranged the candy based on what you are buying.Health food in your cart? Let's bubble that organic dark chocolate bar to the top of the impulse buys. Steaks and soda? Get those snack-sized potato chip bags up there right away.

I hacked up a prototype. On a test site, I modified the Amazon.com shopping cart page to recommend other items you might enjoy adding to your cart. Looked pretty good to me. I started showing it around.While the reaction was positive, there was some concern. In particular, a marketing senior vice-president was dead set against it. His main objection was that it might distract people away from checking out -- it is true that it is much easier and more common to see customers abandon their cart at the register in online retail -- and he rallied others to his cause.

At this point, I was told I was forbidden to work on this any further. I was told Amazon was not ready to launch this feature. It should have stopped there. Instead, I prepared the feature for an online test. I believed in shopping cart recommendations. I wanted to measure the sales impact. I heard the SVP was angry when he discovered I was pushing out a test. But, even for top executives, it was hard to block a test. Measurement is good. The only good argument against testing would be that the negative impact might be so severe that Amazon couldn't afford it, a difficult claim to make. The test rolled out.

The results were clear. Not only did it win, but the feature won by such a wide margin that not having it live was costing Amazon a noticeable chunk of change. With new urgency, shopping cart recommendations launched.

This is a great example of using data to validate a design change instead of relying on gut feel. However one thing that is often overlooked is that the changes still have to be well-designed. Shopping cart recommendations feature on Amazon is designed in such a way that it doesn't break you out of the checkout flow. See below for a screenshot of the current shopping cart recommendation flow on Amazon

On the above page, it is always very clear how to complete the checkout AND the process of adding an item to the cart is a one click process that keeps you on the same page. Sadly, a lot of sites have tried to implement similar features but often end up causing users to abandon shopping carts because the design encourages users to break their flow as part of the checkout process.

One of the places experimentation falls down is when it is used to measure the impact of aesthetic changes to the site when these changes aren't part of a particular workflow (e.g. changing the company logo). Another problem with experimentation is that it may encourage focusing on metrics that are easily measurable to the detriment of other aspects of the user experience. For example, Google's famous holiday logos were a way to show of the fun, light-hearted aspect of their brand. Doing A/B testing on whether people do more searches with or without the holiday logos on the page would miss the point. Similarly, sometimes even if A/B testing does show that a design impacts particular workflows it sometimes is worth it if the message behind the design benefits the brand. For example, take this story from Valleywag "I'm feeling lucky" button costs Google $110 million per year

Google cofounder Sergey Brin told public radio's Marketplace that around one percent of all Google searches go through the "I'm Feeling Lucky" button. Because the button takes users directly to the top search result, Google doesn't get to show search ads on one percent of all its searches. That costs the company around $110 million in annual revenue, according to Rapt's Tom Chavez. So why does Google keep such a costly button around?

"It's possible to become too dry, too corporate, too much about making money. I think what's delightful about 'I'm Feeling Lucky' is that it reminds you there are real people here," Google exec Marissa Mayer explained

~~~

Last night, I stumbled on a design change in Twitter that I suspect wouldn't have been rolled out if it had gone through experimentation first. On the Twitter blog, Biz Stone writes Replies Are Now Mentions 

We're updating the Replies feature and referring to it instead as Mentions. In your Twitter sidebar you'll now see your own @username tab. When you click that tab, you'll see a list of all tweets referencing your account with the @username convention anywhere in the tweet—instead of only at the beginning which is how it used to work. So for me it would be all mentions of @biz. For developers, this update will also be included in the APIs.

Compare the above sidebar with the previous one below and which do you think will be more intuitive for new users to understand?

This would be a great candidate to test because the metric is straightforward; compare clicks on the replies tab by new users using the old version as the control and the new version as the test. Then again, maybe they did A/B test it which is why the "@username" text is used instead of "Mentions" which is even more unintuitive. :)

Now Playing: Jamie Foxx - Blame It (remix) (feat. T-Pain & Rosco)

Over the past two weeks I participated in panels at both the SXSW and MIX 09 on the growing trend of provide streams of user activities on social sites and aggregating these activities from multiple services into a single experience. Aggregating activities from multiple sites into a single service for the purpose of creating a activity stream is fairly commonplace today and was popularized by Friendfeed. This functionality now exists on many social networking sites and related services including Facebook, Yahoo! Profile and the Windows Live Profile. 

In general, the model is to receive or retrieve user updates from a social media site like Flickr and make these updates available on the user's profile on the target social network and share it with the user's friends via an activity stream (or news feed) on the site. The diagram below attempts to capture this many-to-many relationship as it occurs today using some well known services as examples.

The bidirectional arrows are meant to indicate that the relationship can be push-based where the content-based social media site notifies the target social network of new updates from the user or pull-based where the social network polls the site on a regular basis seeking new updates from the target user.

There are two problems that sites have to deal with in this model

1. Content sites like Flickr have to either deal with being polled unnecessarily millions of times a day by social networks seeking photo updates from their users.  There is the money quote from last year that FriendFeed polled Flickr 2.7 million times a day to retrieve a total of less than 7,000 updates. Even if they move to a publish-subscribe model it would mean not only having to track which users are of interest to which social network but also targeting APIs on different social networks that are radically different (aka the beautiful f-ing snowflake API problem).

2. Social aggregation services like Friendfeed and Windows Live have to target dozens of sites each with a different APIs or schemas. Even in the case where the content sites support RSS or Atom, they often use radically different schemas for representing the same data. 

The approach I've been advocating along with others in the industry is that we need to adopt standards for activity streams in a way that reduces the complexity of this many-to-many conversation that is currently going on between social sites.

While I was at SXSW, I met one of the folks from Gnip who is advocating an alternate approach. He argued that even with activity stream standards we've only addressed part of the problem. Such standards may mean that FriendFeed gets to reuse their Flickr code to poll Smugmug with little to no changes but it doesn't change the fact that they poll these sites millions of times a day to get a few thousand updates.

Gnip has built a model where content sites publish updates to Gnip and then social networking sites can then choose to either poll Gnip or receive updates from Gnip when the update matches one of the rules they have created (e.g. notify us if you get a digg vote from Carnage4Life). The following diagram captures how Gnip works.

The benefit of this model to content sites like Flickr is that they no longer have to worry about being polled millions of times a day by social aggregation services. The benefit to social networking sites is that they now get a consistent format for data from the social media sites they care about and can choose to either pull the data or have it pushed to them.

The main problem I see with this model is that it sets Gnip up to be this central point of failure and I'd personally rather deal interact directly with the content services directly instead of inject a middle man into the process. However I can see how their approach would be attractive to many sites who might be buckling under the load of being constantly polled and to social aggregation sites that are tired of hand coding adapters for each new social media sites they want to integrate with. 

What do you think of Gnip's service and the problem space in general?

Now Playing: Eamon – F**k It (I Don't Want You Back)

On Thursday the 19^th of March there was a panel on activity feeds like you find on Twitter & Facebook and social aggregation services like Friendfeed. The panelists were Kevin Marks (@kevinmarks) of Google, John McCrea (@johnmccrea) of Plaxo, Monica Keller (@ciberch) of MySpace, Luke Shepard (@lukeshepard) of Facebook and Marc Canter (@marccanter4real) of Broadband Mechanics. Yours truly was the moderator.

Although the turnout wasn't massive given it wasn't the run of the mill content for MIX 09, the audience was very engaged and we had almost 45 minutes of Q&A until we ran out of time. You can find the video here and also embedded below if you have Silverlight installed.

Now Playing: Jodeci - My Heart Belongs to You

Facebook's latest redesign which has been clearly inspired Twitter's real-time stream of status updates has had a ton of detractors from all corners. One of the biggest places where the outcry has centralized is the Facebook Layout vote application which currently has had over a million votes from Face book users with over 94% against the new changes and almost 600,000 comments, most of which seem to be negative if the hundred or so I read were a representative sample.

One thing I've wondered is how the folks at Facebook are taking this feedback. On the one hand, people don't like changes and the more disruptive the change the more they fight it. It's almost comical to go back and read the Time magazine article about the backlash against the news feed from back in 2006 given how much the feature has not only ended up defining Facebook but how significantly it has impacted the social software landscape at large.  On the other hand, sometimes people have a good reason to protest such as the outcry against the privacy destroying Facebook Beacon which eventually inspired a mea culpa from Zuckerburg himself.

Owen Thomas from Valleywag has an article entitled Even Facebook Employees Hate the Redesign which contains the following excerpt

The feedback on Facebook's new look, which emphasizes a stream of Twitter-like status updates, is almost universally, howlingly negative. Why isn't CEO Mark Zuckerberg listening to users? Because he doesn't have to, he's told employees.

A tipster tells us that Zuckerberg sent an email to Facebook staff reacting to criticism of the changes: "He said something like 'the most disruptive companies don't listen to their customers.'" Another tipster who has seen the email says Zuckerberg implied that companies were "stupid" for "listening to their customers." The anti-customer diktat has many Facebook employees up in arms, we hear.

When your application becomes an integral part of your customers lives and identities, it is almost expected that they protest any major changes to the user experience. The problem is that you may eventually become jaded about negative feedback because you assume that for the most part the protests are simply people's natural resistance to change.

I tend to agree that disruptive companies don't listen to their customers in the narrow sense of using them as a barometer to decide what products or features to build. Customer feature requests aren't the source of input that would spawn a Netflix in a world that had Blockbuster & Hollywood video. Such disruptive products are spawned from understanding the customers better than they understand themselves. If you had simply "listened" to Blockbuster's customers you'd think the best way to compete with them would be to have cheaper late fees or a bigger selection in your store. Netflix actually went a step further and understood the underlying customer problems (e.g. even going to a video store is a hassle which is why you end up with late fees in the first place) and created a product that was truly disruptive.

Using that model of "disruptive companies" the question then is whether the new Facebook is an example of understanding your customers better than they understand themselves or is truly a mistake? For my take on the answer I'll first point out a comment on Valleywag on the redesign

Here's the problem with the redesign. Twitter is a micro-blog. The 140 character Livejournal.

Facebook is not a blog. In its old form it was a really great PHONEBOOK. A phonebook that not only updated your acquaintance's (most FB friends are not friends) contact info, but also gave you a general summary of their life. It was a big picture kind of thing: Where they are, who they're dating, what school or job they have, and how to contact them. It was never about "sharing" your daily thoughts on how great your panini was or omg gossip girl is back! The livejournal twit-blog crap is messing up the phonebook interface.

This is the crux of the problem with the Facebook redesign. The expectations around how user relationships were created on Twitter are totally different from how they were on Facebook. On Twitter, users explicitly decide as part of following someone that they want all of the person's tweets in their stream. In fact, this is the only feature of the relationship on Twitter. On Facebook, you have relationships with people that attempt to mirror your real life so you have your boss, coworkers, school friends and acquaintances all trying to be part of your social graph because FB is really a kind of "rolodex" in the sky.

The fact that you got a news feed was kind of a side effect of filling out your virtual rolodex but it was cool because you got the highlights of what were going on in the lives of your friends and family. There is a legitimate problem that you weren't getting the full gist of everything your 120 contacts (average number of Facebook friends) were doing online but it would clearly lead to information overload to get up to the minute updates about the breakfast habits of some guy who sat next to you in middle school.

Somewhere along the line, it seems the folks at Facebook didn't internalize this fundamental difference in the social context that differentiates user to user relationships on Twitter versus Facebook. This to me is a big mistake.

Now Playing: Goodie Mob - They Don't Dance No Mo'

I still dream about being back in the Nigerian military school which I attended two decades ago. Wow, two decades.

Wow, even trippier is that the school has a web site.

Now Playing: Supremes - Stop! In The Name Of Love

Our mystery panelist has been unveiled.

Standards for Aggregating Activity Feeds and Social Aggregation Services MIX09-T28F

Thursday March 19 |2:30 PM-3:45 PM | Lando 4201

By: Marc Canter, Monica Keller, Kevin Marks, John McCrea, Dare Obasanjo, Luke Shepard Tags: Services

Come hear a broad panel discussion about aggregating social feeds and services from leading people and companies in this rapidly evolving area including Dare Obasanjo from Microsoft as panel moderator, John McCrea from Plaxo, Kevin Marks from Google, Luke Shepard from Facebook, Marc Canter from Broadband Mechanics, and Monica Keller from MySpace.

You can read my previous post to learn more about what you can expect will be discussed at the panel.

Now Playing: Metallica - ...And Justice For All

Details of my upcoming panel at Microsoft's MIX 09 conference are below.

Standards for Aggregating Activity Feeds and Social Aggregation Services MIX09-T28F

Thursday March 19 |2:30 PM-3:45 PM | Lando 4204 (might be changing to 4201)

By: Marc Canter, Monica Keller, Kevin Marks, John McCrea, Dare Obasanjo Tags: Services

Come hear a broad panel discussion about aggregating social feeds and services from leading people and companies in this rapidly evolving area including Dare Obasanjo from Microsoft as panel moderator, Kevin Marks from Google, Monica Keller from MySpace, Marc Canter from Broadband Mechanics, and John McCrea from Plaxo.

News feeds and activity streams as they pertain to social networks is a pretty hot topic given the rise of services like Twitter and Facebook along with their attendant ecosystems. As more and more services provide activity streams as APIs and we see them showing up across different web sites such as FriendFeed and on the desktop (e.g. Seesmic for Facebook, Tweetdeck's new Facebook integration, etc), it is a great time to talk about whether we need standards in this space and what they should look like.

There is also interesting food for thought as to whether we have reached the state of the art in this space or whether there is still more innovation to be seen and also what form it could potentially take. Most importantly, as social networks start propagating activity across each other (e.g. using Twitter to update Facebook status, sharing Flickr activities with my Windows Live Messenger buddies, etc) is this the beginning of the much heralded dream of social network interoperability or are we still far away from making this a reality?

PS: We might have a surprise fifth panelist. I'm still working on the details and will update this post if the person can make it.

Now Playing: Mystikal - Unpredictable

Yesterday I was part of the Feed Me: Bite Size Info for a Hungry Internet panel at the SXSW conference. I've been interested to see what the key take away would be for people that attended the panel and it seems the votes are in. The BBC report on the panel is titled Social networks 'are new e-mail' which is extremely similar to the coverage in the VentureBeat report titled Is the social stream the new email?

Although the "X will kill Y" spin is always good for spicing up a technology story there is an interesting trend that is worth watching. For many people, email serves multiple purposes. It is a way to have conversations and also a way for them to share information. However activity streams such as the Facebook news feed, Twitter and the what's new feed on Windows Live are creating new ways for people to share things about themselves or that they find interesting with others. Joe Kraus of Google described this as the shift from active to passive sharing.

These days I'm more likely to post an interesting link by sharing it on Twitter and have it filter out to my social networks on Facebook and Windows Live than I am to share it via email and risk spamming a bunch of my friends and coworkers. As more people embrace social networking, the trend of using email for certain types of sharing will likely decline. It is already interesting to note that Perez Hilton's #1 Traffic Source Is Facebook and it would be interesting to see if this is a trend that crosses more categories of sites (e.g. I wonder if YouTube gets more traffic from email services or from social networks). 

Where things get interesting is in trying to bridge the gap between active and passive sharing. Sending an email about your latest vacation pics may be too intrusive but you also may not want to just relegate them to being shown in your friends' feeds to be gone from sight if your friends don't log-in in time. David Sacks of Yammer mentions Twitter's @reply feature as one way to bridge this gap. On Twitter, you can push content out into the ether not caring particularly if your friends read it or not but when you do it is just a special character away to send a directed message to them. It will be interesting to see if other services figure out a feature that provides the same functionality as Twitter's @reply and is similarly lightweight.

PS: Food for thought, can you imagine the 25 random things meme on Facebook spreading as effectively if it was an email chain letter?

Now Playing: Disturbing Tha Peace - Growing Pains (Do It Again)

These are my notes from the The Search for a More Social Web talk by Dave Morin.

I had expected this talk to be about lessons learned by Facebook along the way as they've built the site. It actually was more of a product launch keynote and overview of recent launches.

There was a brief preamble about the history of communications from carrier pigeons and the postal service to telegrams and computers. Computers have become increasingly social from when they were first connected into a major computer network as ARPANET, followed by the creation of the World Wide Web, then email [ed note - I think he has his chronology wrong with this one] and chat. However social communications via computers hasn't really come into its own until recently.

Facebook has been working on the social graph for a few years. With Facebook, people for the first time are sharing their real identities and personal content on the Web in a big way. As time has gone by the nature of Facebook's social graph has changed. What was once a graph of relationships between college kids is now a graph that include  coworkers, friends, families and even celebrities.

There are three key pillars in the current Facebook experience

1. The stream (which used to be called the news feed): A chronologically sorted list of what people in your social network are saying, sharing and doing.
2. Social everywhere: With Facebook Connect they have now made it possible to bring social to every site on the Web.
3. The social graph: The graph of connections between users which has been expanded to enable users to connect to everything they care about. The recently announced changes to Facebook Pages now enables brands to participate as nodes in the graph and show up in the stream. Examples of such profiles around celebrities and brands on Facebook include CNN, Barack Obama and U2.

At this point, Dave Morin brought on Gary Vaynerchuck from Wine Library TV to talk about his experiences using the new, improved Facebook Pages to promote his brand and connect with his fans. The first comment Gary made was being glad that he no longer had to put under with the 5000 friend limit if he wanted to be able to participate fully in Facebook. He considers the changes to have turned Facebook into a word-of-mouth marketing machine on steroids. When he posts a video, it shows up in real time in the news feeds of his thousands of fans who can then redistribute it to their social networks if they like or dislike it with the click of a button [ed note – I really love that Facebook explicitly separate the notion of re-share and "like" instead of what FriendFeed does]. With Facebook he has the ability to note only broadcast to his customers but also listen to them by reading the hundreds of comments they leave in response to his videos. He considers this an important paradigm shift in how brands communicate with users and considers Facebook to be the most powerful social media marketing tool around today.

Dave then gave an overview of the four key aspects of the redesigned homepage shown below.

Filters enable you to control what content you see in your stream. The stream is a real-time stream of changes from your social network. The publisher allows you to share your thoughts or interesting content. The highlights section functions like the old news feed by showing you content from people you interact with the most or other notable content from your stream so you don't miss it.

The talk then switched focus to Facebook Connect. Facebook Connect enables all 175 million users of the site to take their Facebook experience to a host of partner sites. There are currently 6000 websites that have implemented Facebook Connect including TechCrunch, Vimeo, Meetup, Geni and Joost. All of these services have mentioned increased engagement from their user base from deploying Facebook Connect. TechCrunch has been getting higher quality comments from people who use the integration since it is attached to their real names. When you combine Joost with Boxee, it is now possible to see social to television by seeing what your friends are watching from your TV. There is also Aardvark social search engine which also will be integrating Facebook Connect [ed note – not clear this last one has actually shipped].

Facebook Connect has not only been used to bring social to the web but also to the desktop as well. Two key examples are Xobni integrating with Facebook Connect to bring social to your desktop email client and Apple iPhoto adding the ability to upload photos directly to Facebook. At this point Dave brought on Loic Le Meur of Seesmic who announced a new desktop client built on Adobe Air for interacting with Facebook now available at http://www.seesmic.com/facebook . You can find out more about this on Loic's blog post about the launch at Seesmic Launches the First Facebook Desktop Client Available Today. Screenshot of the application below. It only does status updates for now but is expected to show richer media types in forthcoming releases.

The question then is what are the next steps for Facebook with regards to their platform. In 2006, they shipped their first set of APIs. In 2007, they shipped the Facebook platform which now has over 50,000 applications. 2008 was the year of Facebook Connect. In 2009, they have done a couple of things thus far. They have opened up APIs to status, notes, links and videos. They are now active in community standards like OpenID and ActivityStreams where they are ably represented by Luke Shepard. They have also contributed to open source projects like memcached.

one more thing…

Dave announced Facebook Connect for the iPhone. He then brought on a number of CEOs of various iPhone application companies to talk about upcoming or just released applications for the iPhone that will integrate with Facebook. The CEO of Playfish which makes three of the top ten games on the Facebook platform and has over 60 million users was the first to speak. They are debuting Who has the biggest brain? for the iPhone. You can challenge your friends on Facebook and see where you rank with them directly from the iPhone. He was followed by the CEO of Social Gaming Network (SGN) which is debuting at least two games with Facebook integration. The first is Agency Wars where users can create a secret agent character then recruit or assassinate their Facebook friends. They are also the makers of iBowl which will be updated with the ability to see when your Facebook friends are online and challenge them to a game of bowling. One of the co-founders of Tapulous was next to talk about their next product Tap Tap Revenge 2 which is shipping with not only 250 new songs but also will use Facebook Connect so that you can challenge your Facebook friends to games in a new split-screen mode. Although they have their own social network it pales in comparison to the highly connected, 175 million user strong social graph on Facebook. Patrick O'Donell from Urbanspoon also spoke about their Facebook integration. Urbanspoon has 1 million restaurant reviews and 4 million iPhone users who have used the "shake" feature of the application over 200 million times. The key integration with Facebook is that restaurant reviews entered via your phone will now show up on your profile in Facebook and in your friends' stream. Joe Greenstein of Flixster was the final iPhone application developer to talk about their Facebook integration. They have 3.4 million users of their application on the iPhone and will now gives those users the ability to integrate their Facebook identity with the application. There are more iPhone applications expected to ship with Facebook integration in the coming weeks including applications from Zynga, Loopt, CitySearch, MTV, Citizen Sports and more.

More details on the Facebook Connect for iPhone announcement can be found on the Facebook official blog post entitled Facebook Connect for iPhone: Friends Now Included.

Now Playing: Dr. Dre - Let Me Ride (remix)

I'll be attending the SXSW interactive conference this weekend and wanted to share my schedule for any of my blog readers who want to meet up and chat after one of the sessions I'll either be attending or participating in.

SATURDAY

10:00 am ·  Is Privacy Dead or Just Very Confused?  (danah boyd on privacy & social networks)

11:30 am ·  The Search for a More Social Web (Dave Morin from Facebook talking about the industry's search to make the Web more social. The kind of talk I'd have expected to be given by Kevin Marks)

2:00 pm ·  Opening Remarks: Tony Hsieh (the founder of Zappos.com talks about their legendary reputation as the kinds of customer service)

3:30 pm ·  Feed Me: Bite Size Info for a Hungry Internet (your opportunity to heckle me on a panel – Facebook, FriendFeed, Yammer and Windows Live folks on the future of feeds in social networks)

5:00 pm · <nothing in this time slot interests me>

6:30 pm ·  Salon: Friendship is Dead (I always love discussions on whether social networks have devalued the notion of friends or made some of these ties stronger. wonder if they'll talk about Cameron Marlow's research on Maintained relationships in Facebook)

SUNDAY

10:00 am · <nothing in this time slot is in my areas of interest>

11:30 am ·  OpenID, OAuth, Data Portability and the Enterprise (Joseph Smarr and Kaliya Hamlin are well known for getting identity, authentication and authorization concerns on the Web so it will be interesting to hear their perspectives on what lessons can transfer to the enterprise, if any)

2:00 pm · <I'll spend this time getting my affairs in order since I go straight to the airport after my panel>

3:30 pm ·  Post Standards: Creating Open Source Specs (another opportunity to heckle me on a panel – two people from Microsoft on a panel on "open source" specs !?!)

I'll try to write up the sessions I attend while on the plane and will post them next week if I get around to it.

Now Playing: Geto Boys - My Mind's Playin Tricks On Me

In reading Facebook's recent announcements about upcoming changes to their homepage I've thought it weird that they are being so blatant about their fear of Twitter and altering the dynamic of their site so radically. Like John Battelle said, Facebook Shows "What's On Its Mind" – Twitter.

It seems pundits and social software nerds like myself aren't the only ones that think this. Below are a sampling of comments that confronted me on my Facebook homepage in the user replies to the Facebook post Welcome to your New Home Page.

As I've mentioned before, inline comments in feeds can detract a whole lot more than people think. This is one thing I love about Twitter's model. Just because I follow a popular user on Twitter doesn't mean I have to succumb to the flood of less than stellar replies from the rest of their followers.

Now Playing: DJ Green Lantern - We All Die Someday (Eminem/Lloyd Banks/Obie Trice/50 Cent)

After logging in, be sure to visit all the options under Configuration in the Admin Menu Bar above. There are 26 themes to choose from, and you can also create your own.

Below is a chart of home prices in my zipcode taken from Zillow 

We bought our house around the peak of that chart. According to Zillow our home seems to have lost about $50,000 in value since we bought it. That seems high on the surface but I know of at least one house in our neighborhood that just sold for $60,000 less than what the owners paid for it around the same time we bought our house.

I don't expect that the recently announced home owner rescue plan by the Obama administration (which is covered in a great Q & A in the New York Times blog) will have any effect on us given that we can afford our house and aren't in dire financial straits. Unless I end up one of the 3,600 waiting for the other shoe to drop and can't find a new job. At least I'm no longer an H-1B so I don't have to worry about needing to leave the country within a week or two if laid off.

I expect house prices to drop even further before we hit the bottom. This is a rational expectation when you look at the following chart

None of this would be a concern if we plan to live here for the next 20 – 30 years. However I have a horrible daily commute and as a new dad I'm not enamored with the schools in the area. 

So I punched some numbers into Pay Or Go: Walk away from your mortgage calculator and the result was a recommendation to walk away if we don't expect the house to appreciate back to the price we paid for it in the next 5 – 7 years. Given the historical chart above, I don't.

Articles like Silicon Valley 'underwater' homeowners: Should I stay or go? point out that the biggest consequence of walking away is having a blemish on your credit score for up to seven years. This implies to me that if moving to a neighborhood whose schools I feel better about is important then it makes sense to take the credit hit now, rent a place and put away the cost savings over the next seven years so we can have a great down payment when we want to move to that house in the great school district when Nathan will be about 7 years old.

On the one hand, I feel like I'm shirking some financial responsibilities even thinking about this but on the other hand I want to do what's best for my family. What do you guys think?

Now Playing: Bob Marley & The Wailers - Redemption Song

…just different requirements and constraints.

The top story on programming.reddit this morning is an example of how differently people can view the same incident and how it relates to making design decisions when trying to ship a software project.

Almost a decade ago, Joel Spolsky wrote an article entitled Two Stories which contrasted his time working at Microsoft with his time working at Juno (a popular ISP from back in the day). Below is the relevant excerpt from Joel's article which talks about his time developing the macro language that was to be used by Excel.

My first assignment at my first job was working at Microsoft, where I was told to come up with a new macro language strategy for Excel. Pretty soon, I had the first draft of the "Excel Basic" spec (which later evolved into Visual Basic for Applications, but that's another story). Somehow, this mysterious group of people at Microsoft called the "Application Architecture" group got wind of my spec, which must have concerned them, because for some reason they thought that they were in charge of things like macro language strategies, and they asked to see my spec.
…
So I proceeded to ignore them as diplomatically as possible.

This seemed to piss off a guy named Greg Whitten who headed up the App Architecture group. Now, Greg was something like Microsoft employee number 6. He had been around forever; nobody could quite point to anything he had done but apparently he had lunch with Bill Gates a lot and GW-BASIC was named after him. Greg called a BIG MEETING and proceeded to complain about how the Excel team (meaning me) was screwing up the macro strategy. We pressured him to come up with some specific reasons but his arguments just weren't convincing. I thought it was nice that here I was, a new hire pipsqueak right out of college, arguing with employee number 6 and apparently winning the argument. (Can you imagine that happening at a Grey Flannel Suit company?) My programming team, headed by Ben Waldman (now a VP at Microsoft) backed me up completely, which was all that really mattered, because the programming team wrote the code and thus had the final say on how things got done.

Five years later, John Foust tracked down Greg Whitten to get his take on the same incident and then republished Greg's private response on the classic computing mailing list. The relevant excerpt from Greg Whitten's mail is below

On the Joel Spolsky subject he was a basically ignorant junior employee who left Microsoft after a short number of years. His short sighted decisions to take the VB macro language in Excel in its own directions caused 6 other major applications that were doing BASIC macro languages to diverge and not be able to share any macro programs between the applications. He made other similarly stupid decisions like creating a custom programming interface for BASIC in Excel instead of sharing a common interface as strongly recommended. The applications group spent 30 man-years integrating custom interfaces for each application with the Office 95 applications. In Office 98 they tossed it all and went back to my original suggestion which only took 1.5 man-years to develop and provided better commonality and learning between the applications.

It is instructive to compare both approaches to solving the same problem. Joel seems to be a believer in the philosophy that shipping is a feature which means you execute on your project by limiting dependencies while making sure you are building specific solutions to customers problems. Greg was big picture guy who wanted to ensure that all of the various BASIC macro languages being developed by different groups at Microsoft were compatible and could developers could easily port applications between them. They both have a point, on the one hand the designer of the Excel macro language shouldn't have to be limited by the constraints of building macros for Outlook but on the other hand anyone building common tooling or infrastructure for macros in Office would prefer that there weren't significant differences in the various macro languages in the product suite. So you end up with two smart people with contradictory goals and it is thus unsurprising that each thinks the other is ignorant.

The unfortunate thing about this entire incident is that it would have been a great learning experience for Joel if he had stayed on in Excel to see some of the consequences of his design decisions and then be in a position to consider whether he'd made the right tradeoffs in the first place. Of course, this is pretty commonplace when it comes to large software platforms where people can spend 3 – 5 years working on a single release which in combination with an average job tenure of 4 years in the U.S. (probably less in fast paced the software industry) means that many people never learn from their mistakes or improve their skills over time.

~~

As a program manager at Microsoft, I'm often in the same predicament as Joel was when he started working at Excel. Although the same challenge exists at every big software company, Microsoft is unique in its breadth of software offerings which means that almost every technology choice or design decision you make is an opportunity to pay some sort of strategy tax. The key thing I always keep in mind is that shipping is a feature and building software that delights our customers should have the highest priority. Being able to articulate how your choices reflect both of these points during every phase of the project is part of being a good program manager. Joel was able to do that, which is why he got to ship his project how he saw fit.

Now Playing: Korn - Freak On A Leash

Yesterday Mark Zuckerburg announced a number of interesting upcoming changes to Facebook in his post Improving Your Ability to Share and Connect which are excerpted below

What's New Today
Starting today, we are announcing new profiles for public figures and organizations. Once called Pages, these new profiles will now begin looking and functioning just like user profiles. Just as you connect with friends on Facebook, you can now connect and communicate with celebrities, musicians, politicians and organizations. These folks will now be able to share status updates, videos, photos or anything else they want, in the same way your friends can already. You'll be able to keep up with all of their activity in your News Feed. This means that you can find out that Oprah is reading a book backstage before a show, CNN posted a breaking story or U2 is working on a new song, just as you would see that your friend uploaded new photos from her trip to Europe.

We're also going to make some changes to the home page. The new home page will let you see everything that's shared by your friends and connections as it happens. It will also provide you more control by letting you choose exactly who you see among the people and things you are connected to. You can decide you no longer want to get updates from your old friend from high school who you rarely talk to, or you can filter the stream to only see updates about your family members. And now, if you want, you can read what President Obama is saying on the same page as your best friend. You can find out what it is your mother, your high school classmate or President Obama are doing, thinking and sharing right now just by logging into Facebook.

We'll begin rolling out the new home page next week, so please check out our home page tour to see the new design and let us know what you think about it. This is an exciting move for us and we have more coming, so keep an eye on the blog for more updates about upcoming products.

One thing you can say about Facebook, is that they are quick to adopt new features from hot startups once its been shown that those features have legs. Most recently, it has borrowed ideas such as the ability to post inline comments or indicate you like an item in our news feed from FriendFeed and with yesterday's announcement it looks like the company agrees that Assymetrical follow is a core Web 2.0 pattern as popularized by Twitter.

Facebook initially decided to segregate celebrities and brands from regular users with their Pages feature. Given the success that MySpace has had both from a user experience and monetization perspective from integrating brands into their social graph and treating them equally as users, I'm surprised that it took Facebook this long and that when they did it seemed this was more inspired by Twitter's success than MySpace's.

I expect this to significantly change the dynamic of Facebook especially since many people have already indicated they are fans of various brands on the site. So there are many people who will be exposed to this functionality right off the bat. What will happen when the site I used to visit to see what's going on with members of my extended family, former coworkers and high school friends becomes just as likely to be filled with marketing messages from Windows Mobile and 50 Cent because I've indicated interest in these brands?

Already, my news feed now has stuff like

which repeats a lot of the mistakes of the FriendFeed user interface such as too much space being dedicated to content I may be marginally interested in and putting people I have no interest in right in my face.

Twitter doesn't have these problems due to their core design being different. People only get 140 characters to make their point so I get more content on the page instead of half the screen being dedicated to a single update. Also I don't see @replies from people I'm not following so "me too" comments in response to content from popular users doesn't clutter my experience unless it comes from mutual friends and even then I can disable that feature.

As Facebook continues to grow, the legacy of their existing features will make it harder and harder for them to adapt to new trends. As someone who works on bringing new social features to decade old applications with pre-existing usage patterns and hundreds of millions of users as part of his day job, I know first hand how difficult things are going to get for them.

Now Playing: Bone Thugs 'N Harmony - Da Introduction

Yahoo! has thrown their hat in the ring and joined the hottest new trend on the Web, shipping your very own "Connect" technology. What is a "Connect" technology? David Recordon does a good job of summarizing the common characteristics of this new category of technology offerings in his post Anatomy of "Connect" where he wrote

a straw man definition of a "Connect" application:

1. Profile: Everything having to do with identity, account management and profile information ranging from sign in to sign out on the site I'm connecting with.
2. Relationships: Think social graph. Answers the question of who do I know on the site I've connected with and how I can invite others.
3. Content: Stuff. All of my posts, photos, bookmarks, video, links, etc that I've created on the site I've connected with.
4. Activity: Poked, bought, shared, posted, watched, loved, etc. All of the actions that things like the Activity Streams project are starting to take on.

In my mind, the Goals of all of these "Connect" applications are focused on helping people discover new content, people they already know as well as new people with similar interests.

Facebook launched this trend with their announcement of Facebook Connect which offered an alternative to OpenID by offering a smoother user experience and the opportunity for sites to also leverage their news feed as a viral distribution mechanism. According to the Yahoo! Developer blog there is now a similar offering from Yahoo! announced in their post Extend Your Publishing Reach with JS-Kit + Yahoo! Updates which states

Here's how it works: Integration with the Social Directory API allows JS-Kit to display a user’s Yahoo! nickname and avatar picture on the site. Integration with the Updates API allows JS-Kit to publish an item to the Yahoo! Updates feed when a user adds a comment to a web site powered by JS-Kit. At all times, your users remain in control of their data by leveraging OAuth to broker data access between Yahoo! and JS-Kit.

Yahoo! Updates allow publishers (and publishing partners like JS-Kit), to syndicate user-generated actions (ratings, reviews, comments, favorites, and uploads) to Yahoo!'s massive global distribution network. In the coming months, as Updates are implemented across Yahoo!, publishers will enjoy referral traffic back to their sites from across the Yahoo! Network (more than 500M+ monthly unique visitors).

The more of these "Connect" offerings that are announced the more they seem like introducing a new problem instead of creating a solution. Back in the day, it was easy to argue that having a button on your site is more palatable than the OpenID login box prompting users to enter some obscure URL. However when I go to places like Sun's website and see a comment box with the following

I wonder if this is progress. There are even more perverse cases like StackOverflow whose current login page looks like

Of course, this is just StackOverflow's attempt to make the OpenID sign-in process easier for users instead of having them remember some obscure URL. Still I wonder about the paradox of choice style experience that is being thrown in front of users. Whenever I go to Hacker News I actually forget which of my various Facebook/Windows Live/Google/Yahoo IDs I've previously associated with the site via ClickPass. It actually would have been easier and less confusing for me to just create a regular account on the site like did with Reddit which would also enable cookies to work so I wouldn't have to sign-in every single time.

I also wonder about the unnecessary duplicative efforts it will take sites to integrate all of these different "Connect" offerings as well especially when you consider how radically different APIs like Facebook's friends.get are from Yahoo's Contacts API. I'm typically against premature standardization but when you have almost half a dozen offerings that effectively do the same thing but with radically different APIs it may be time for standards to start showing up.

Now Playing: Domino - Sweet Potatoe Pie

Earlier this year I was approached about writing a book on cloud computing topics by an editor for one of the big computer book publishers. Given that between my day job and having an infant I barely have time to keep this blog updated, I had to turn down the offer. However I did spend some time taking a second look at various cloud computing platforms like Amazon Web Services and Google App Engine then trying to put myself into the mindset of a potential customer as a way to figure out the target audience for the book. Below are the two categories of people I surmised would be interested in spending their hard earned cash on a book about cloud computing platforms

1. Enterprise developers looking to cut costs of running their own IT infrastructure by porting existing apps or writing new apps. 
2. Web developers looking to build new applications who are interested in leveraging a high performance infrastructure without having to build their own.

As I pondered this list it occurred to me that neither of these groups is well served by Google App Engine.

Given the current economy, an attractive thing to enterprises will be reducing the operating costs of their current internal applications as well as eliminating significant capital expenditure on new applications. The promise of cloud computing is that they can get both. The cloud computing vendor manages the cloud so you no longer need the ongoing expense of your own IT staff to maintain servers. You also don't need to make significant up-front payments to buy servers and software if you can pay as you go on someone else's cloud instead. Google App Engine fails the test as a way to port existing applications because it is a proprietary application platform that is incompatible with pre-existing application platforms. This same incompatibility rears its head when you look at App Engine simply as a way for enterprises to do new development. App Engine is based on Python, which if you look at the State of the Computer Book Market 2008, part 4 -- The Languages is not a terribly popular programming language. In today's world, enterprise development still means Java or .NET development which means enterprises will favor a platform where they can reuse their existing skills and technology expertise. Google App Engine isn't it.

So how about Web developers? In my classification, I broke up Web developers who'd be interested in cloud computing into hobbyists (like myself when writing a Twitter search engine on Windows Azure) and professionals (like myself when working on the platform that powers Hotmail's recently launched social features). Hobbyists either don't spend money or spend relatively little so I discounted them as a target audience of interest. The professional Web developers interested in cloud computing would be those who are considering Server or Web hosting but have concerns about scaling up if their service gets successful. After all, it seems like every week you are either reading about scaling hurdles that startup developers have faced as their applications become successful whether it is Bret Taylor's recent post How FriendFeed uses MySQL to store schema-less data or Jeff Atwood's Deadlocked! which talks about how he had to learn more about SQL Server's locking strategy as StackOverflow.com became more popular. The fact that Google App Engine is limited to only Python meaning that it is unavailable to developers using WISC platforms and only a subset of developers using LAMP can participate on the platform. Furthermore, there are key limitations in the platform that make it infeasible to build a full scale application. For example, Bret Taylor mentions that a consequence of having a denormalized database they need to run a background "Cleaner" process which fixes up data references and makes their database consistence. The App Engine DataStore API requires applications to store data in a denormalized way but there is no facility to run background processes to clean up the data as FriendFeed and most other large scale services which use database sharding often do. According to a recent blog post the Google App Engine roadmap has been updated so at least this limitation will be addressed. Other limitations for Python developers are that they can't use all of their existing knowledge of Python libraries since it only supports a subset of Python libraries. Database developers may be relieved that a lot of database management tasks no longer exist but may be chagrined once they see the restrictions on queries they can perform and hear some of the horror stories about query performance. At the end of the day, it just didn't seem to me that there were many professional Web developers who would put up with all of this pain over just going with AWS or dedicated hosting.

That said, Google App Engine does address the long tail of developers which I guess is nothing to sneeze at. Maybe it will see some success from targeting the low end in the same way that AdSense targeted the long tail of advertisers and is now the powerhouse of search advertising. Maybe. I doubt it though.

Now Playing: Aerosmith - Livin' On the Edge