The most interesting news from Facebook’s F8 last week was the announcement of App Links. If you are unfamiliar with the announcement, watch the 1 minute video embedded below which does a great job of setting up the sales pitch. Using App Links, mobile app developers can put markup in their web pages that indicate how to launch that page in their application on Android, iOS and Windows Phone. For example, clicking on a link to a FourSquare check-in from the news feed will launch the FourSquare app on your phone and will open that specific location or event.  .

The interesting question is why is Facebook doing this? It boils down to the fact that Facebook is an advertising company which makes the majority of its revenue from those ads asking you to install Candy Crush and Clash of Clans in your news feed.

Facebook’s pattern at this point is well known. They give you something of value for free (traffic) and once you get hooked they dial it down until you have to pay. The world is littered with the ashes of various companies who were once media darlings because Facebook gave them a bunch of free traffic from liberal news feed algorithms and then turned off the spigot. Just ask Viddy, all those social readers, Zynga, or read that hilarious break up letter from those guys at Eat24.

Publishers who use app links will likely get a boost in the news feed algorithm likely under the pretext that they provide a better user experience to consumers. Early success stories will cause lots of developers to create app links and then get hooked on the traffic they get from Facebook. Eventually your traffic will start dropping and any complaints will be met with an elaborate mathematical formula which explains why your content isn’t that hot on Facebook anymore. But don’t worry, you can fix all that by buying ads.  

It’s obvious, devious and I love it. Especially since it does actually move the user experience of the mobile web forward even if the end goal is to make Facebook tons of money.

The other thing I give Facebook props for is holding a mirror up to the major search engines to see how silly we were being. Bing supports standards for app linking but it's only for Windows & Windows Phone apps. Google supports the same and again it only works for Android apps. Facebook is trying to say it doesn’t matter if you are on the web, Windows Phone, Android or iOS, links in the news feed should open in the native app on that platform. Google and Bing’s search engines on the other hand only supported the same when searching on the OSes from their parent companies. #strategytax

Hopefully Facebook’s move will bring more inclusiveness across the board from many online platform providers not just search engines. For example, I would love it if email providers also supported app links as well.

Now Playing: DJ Snake & Lil Jon – Turn Down For What

 

Danny Sullivan wrote an interesting blog post this morning titled Google’s Broken Promises & Who’s Running The Search Engine? whose central thesis is that Google now does a number of things it once described as “evil” when it comes to how search results and ads work in Google Search. Given that I now work in Bing Ads, this is a fairly interesting topic to me and one I now have some degree of industry knowledge about.

Promises Are Like Pie Crust, Easy to Make and Easy to Break

Danny Sullivan categorizes two big broken promises in his article, one in 2012 and one from 2013. The 2012 broken promise is excerpted below

The first broken promise came last year, when Google took the unprecedented step of turning one of its search products, Google Product Search, into a pure ad product called Google Shopping.
…
Google Shopping is a different creature. No one gets listed unless they pay. It’s as if the Wall Street Journal decided one day that it would only cover news stories if news makers paid for inclusion. No pay; no coverage. It’s not perfect metaphor. Paid inclusion doesn’t guarantee you’ll rank better or get favorable stories. But you don’t even get a chance to appear unless you shell out cold hard cash.

What Was Evil In 2004, Embraced In 2012

Shopping search engines have long had paid inclusion programs, but not Google. Google once felt so strongly that this was a bad practice that when it went public in 2004, it called paid inclusion evil, producing listings that would be of poor relevancy and biased. The company wrote, in part:

Because we do not charge merchants for inclusion in [Google Shopping], our users can browse product categories or conduct product searches with confidence that the results we provide are relevant and unbiased.

There is a similar Google then versus Google now perspective when looking at the second broken promise related to banner ads in the search results page.

“There will be no banner ads on the Google homepage or web search results pages,” Google promised in December 2005, on its main blog, to reassure consumers concerned that its new partnership with AOL would somehow change the service. Eight years later, Google’s testing big banner ads like these:

These excerpts could almost be a cautionary tale to idealistic young companies about making absolute statements and the staking one’s brand on these statements without thinking about the future. However that isn’t the point of this post.

I decided to write this post because Danny Sullivan’s article starts starts out quite strongly by pointing to this misalignment between Google’s past statements and their current behavior but then peters out. The rest of the article is spent studying Google’s org chart trying to figure out which individual to blame for these changes as well as trying to come up with a rationalization for these moves in the context of making search better for consumers. As an industry watcher the rationale for these moves is quite straightforward and has been a natural progression for years.

The Love of Money is the Root of all Evil

Any analysis of business decisions Google makes in the arena of search, is remiss if it fails to mention Google makes the majority of its revenue and profits from ads running on Google sites. As an example, Google made $9.39 billion last quarter from ads running on its sites whereas of the $3.15 billion it made from ads running on other people’s websites it paid those people $2.97 billion. Combining that with the fact that its $12.5 billion acquisition of Motorola has so far produced nothing but financial losses, there is a lot of pressure for Google to make as much money as possible from ads running on its sites specifically in Google Search results pages. 

When it comes to search engine advertising, the money is primarily in queries with “commercial intent”. This is a fancy way of saying that the person who is performing the search is planning to spend money. Advertisers are willing to pay several dollars to a search engine each time a customer clicks on their ad when the search term has commercial intent. In fact, companies are willing to spend up to $40 – $50 each time a user clicks on an ad if the user is searching for a life insurance policy or a home loan.

Over time both search engines and advertisers have figured out exactly where the money is and how to extract the most value from each other. Google has slowly been making changes to their search engine that implies that for queries with commercial intent they always want a cut of the action. This is why if you perform a search today that has commercial intent, there are an order of magnitude (i.e. ten times) as many links to ads as there are unpaid search engine results. For example, take a look at this screenshot of a query for “northface jackets” on Google.

There are two links on this page that are unpaid search results and eighteen links where Google gets paid if you click on them. Given that context, it is no surprise that Google eventually realized it was competing with itself by having a “free” shopping search engine. This explains the broken promise in 2012 related to paid inclusion.

Now if you take a close look at the above screenshot, you’ll notice that The North Face is actually the top advertiser on this page. This means that despite the fact that the user was specifically looking for the North Face brand products, the company has to still compete with other advertisers by paying people for clicks to their website from Google search results. Brand advertisers hate this. A lot. Not only did they spend a lot of money and effort to become a well-known brand but now they still end up paying when this brand recognition pays off and people explicitly are looking for them on Google.

This leads us to the second broken promise, banner ads in search results. What Google is trying to do is to appease brand advertisers by letting them “take over” the search engine results page in cases where the user is quite clearly searching for their brand. Treating this is a giant billboard that reinforces their brand as opposed to scrabbling with other advertisers for a user which they already consider theirs is a more amenable pitch. This explains the broken promise of 2013.

I expect to see more aggressive commercialization of the search results page given Google’s seeming lack of interest and inability to diversify their sources of income. Doing this while preserving the customer experience will be the #1 challenge of their search engine and other similarly advertising focused web products in 2014 and beyond.

Now Playing: Jay-Z – F*ckwithmeyouknowigotit (featuring Rick Ross)

 

The big news in tech circles is this is the week Google Reader died. There has been a lot of collective shock about this since Google Reader was a staple of the sort of technology news omnivore and early adopter that was once considered to be the core fan base for Google. Despite Google’s protestations it’s quite clear that there was ample usage of the service with sites like Buzzfeed providing evidence that up to its last day it was driving over a million referrals a day. As someone who wrote a feed reader that synced with Google Reader which has been downloaded over 1.5 million times and is credited in Atom format specification it is sad to see such a core part of the RSS ecosystem come tumbling down.

The biggest surprise for many is how far Google seems to have strayed from its core mission as epitomized by this comment on Hacker News

ivank 1 day ago | link | parent

Organizing the World's Information... and setting it on fire 8 years later.

Like many companies before it, the company has done a great job of separating its public persona from the gritty reality of its business practices. Google’s business mission is to organize all the world’s information and use that to sell advertising. The great thing about this business model is that for a long time it has been a win-win for every one involved in the ecosystem. Consumers get great web services and products that are expensive to build and run for free such as Google Search, Gmail, Chrome and Android. Advertisers get to target consumers in new ways which have a better return on investment than they traditionally have been able to get from print and other media. While Google has minted hundreds of millionaires with the profits they’ve gotten from creating this ecosystem.

Unfortunately, Google has also hit the same problems that successful companies before it have had to face. The laws of large numbers mean that for Google to continue to be a great business for its shareholders (which includes its employees and executives) then it needs to find ways to eke even more money out of its customer base (advertisers) by giving them even more product (consumers with rich demographic and behavioral information) or by growing into new markets. Companies like Apple and Microsoft have faced this problem by growing new multi-billion dollar businesses. Apple went from a computer company to the iPod company and now it’s the iPhone company which is transitioning to becoming the iPad company. Microsoft started off as the BASIC company, transitioned to being the Windows & Office company and today has several billion dollar businesses from game consoles to database software. Google has decided to face this challenge by doubling down on their advertising business which means trying to suck in even more data via acquisitions like Waze and ITA as well as extracting more revenue from advertisers by removing consumer targeting options.

For the most part Google has diverted tech press scrutiny from this ongoing attempt to monopolize data sources and hover up personal information about users for later resale using misdirection. A few years ago the Google playbook was to release some technology branded as “open” such as OpenSocial which aimed to paint Facebook as “evil”, open sourced Android which paints Apple as “evil” or Google Chrome which was released with an excellent comic book explaining why it was so open. How could one complain about Google with a straight face when they were giving all of these free services to consumers and “open” technologies to the industry? No one would buy it. This was the strategy of the old Google under Eric Schmidt and it worked beautifully.

It has been an interesting hallmark of Google under Larry Page that it doesn’t play these games anymore. Pursuing Google’s business strategies at the expense of its public image as the good guy of the technology industry is now par for the course. I was taken aback when Google announced that it was going to fork Webkit since this was a blatant political move which many technology savvy geeks would see through. For days afterwards, the article A Short Translation from Bullshit to English of Selected Portions of the Google Chrome Blink Developer FAQ was a viral hit among developers on Twitter with excerpts such as

1.2 Is this new browser engine going to fragment the web platform's compatibility more?

Yes.

We intend to distract people from this obvious problem by continually implying that our as-yet unwritten replacement is somehow much better and more sophisticated than the rendering engine that until yesterday was more than good enough to permit us to achieve total dominance of the Windows desktop browsing market in less than two years.

This strategy has worked extremely well for Netscape, Microsoft, Apple and us in previous iterations of the browser wars, and we firmly believe that everyone in this industry was born yesterday and they will not recognise this for the total bullshit it so clearly is.

1.10 Is this going to be open source?

Not really.

While you can certainly read the source code, we're fully aware that actually tracking and understanding a modern HTML renderer is extremely difficult. In addition, the first changes we will make are intended specifically to break compatibility with WebKit, so the only organisation with sufficient resources to track our changes will no longer be able to do so.

In practice, this allows us to call the project "open" while simultaneously ensuring Google will be the only effective contributor to the Chrome and Blink source now and in the future. We've had enormous success co-opting the language of open source in the past to imply our products are better, and we aim to continue with that strategy.

So what does all of this have to do with Google Reader? As Marco Arment points out in his excellent post titled Lockdown, Google is in a battle with Facebook and Twitter to suck up user demographic and behavioral data around social signals. Google Reader is literally the opposite of that strategy since it is far too decentralized when compared to the model pioneered by Facebook Pages and Twitter.

Google Reader has been living on borrowed time since Facebook and Twitter became prominent. The only thing that has changed in 2013 is that Google’s management doesn’t think it’s worth it to throw a bone to millions of geeks and early adopters by keeping the lights running on the service with its existing skeleton crew. This new Google doesn’t care if geeks and early adopters just see it as another money hungry corporation that only focuses on the bottom line. Larry Page doesn’t give a shit.

Welcome to the new Google.

Now Playing: J.Cole - Can't Get Enough (featuring Trey Songz)

 

In a past life, I worked on the social news feed for a number of Microsoft products including the Messenger Social feed in Hotmail & Messenger and most recently the People app in Windows 8. When I worked on these products, we strongly believed in the integrity of the user experience and so never considered the social feed as a canvas for showing users ads.

Thus I read a pair of recent posts by Dalton Caldwell, founder of App.net, with some interest. Dalton wrote about the recent moves that both Twitter and Facebook are making to ensure that the social feeds on their sites become a great canvas for showing ads.

In his post Understanding Like-gate Dalton writes

The best ad is indistinguishable from content

We can expect to see Facebook deemphasizing traditional advertising units in favor of promoted news stories in your stream. The reason is that the very best advertising is content. Blurring the lines between advertising and content is one of the most ambitious goals a marketer could have.

Bringing earnings expectations into this, the key to Facebook “fixing” their mobile advertising problem is not to create a new ad-unit that performs better on mobile. Rather, it is for them to sell the placement of stories in the omnipresent single column newsfeed. If they are able to nail end-to-end promoted stories system, then their current monetization issues on mobile disappear.

In his post Twitter is pivoting Dalton writes

Predicting the future

In this paradigm, Twitter’s business model is to help brands “amplify their reach”. A brand participating in Twitter can certainly distribute their content for free and get free organic traffic, but if they want to increase their reach, they need to pay.

It’s no accident that this sounds exactly like the emerging Facebook business model. As discussed in that link, algorithmically filtered primary feeds are vastly easier to advertise against vs unfiltered feeds. The issue for Twitter is that Facebook already has a far larger userbase which is already trained to read an algorithmically filtered feed.

In a twist, I wouldn’t have predicted a few years ago it is now a regular occurrence for both users of Facebook and Twitter to see ads in their feeds. Twitter has gone as far as effectively crippling new Twitter apps to ensure that every Twitter user gets an ads-heavy unfiltered Twitter experience. The reason for this is straightforward. Both companies have sky high expectations from investors as evidenced by Facebook's $100 billion valuation it has failed to meet and Twitter's $8 - $10 billion valuation on $100 million in revenues. The challenge for both services is that investors are expecting Google-like returns on investment but neither of these companies have a Google-like business model.

The problem with ad supported online businesses is that for the most part their business models suck. In a traditional business, if you focus on building a great product or service that provides an excellent customer experience then you will be making money hand over fist. In most ad supported online businesses, your business is selling your product’s audience as opposed to the product itself. That means if you want to make more money you have to pimp out your audience often in disrespectful and intrusive ways to eke out that extra dollar.

The one place where this is different is online search (i.e. Google’s primary business). In the web search, the ads aren’t just indistinguishable from content but in the most lucrative cases, the ads are better than the content. As an example take a look at these searches

• "buy nokia lumia" on Bing Search
• "buy nokia lumia" on Google Search

Since we may get different results, I’m including a screenshot below

There are 8 ads in this screenshot and 2 search results. However instead of being irritated as I would be if the ratio of ads to content was 4:1 in a YouTube video or Facebook feed, the ads are actually more relevant than the organic search results. This is the holy grail that Twitter and Facebook are trying to achieve.

As Dalton points out, Facebook has already socialized its users to the notion that brands will post commercial messages to the feed. In addition, brands have grown so entitled to it then when asked to pay for them since they are ads, they get outraged. However Facebook has been boiling this particular frog for a while. Facebook encourages advertisers to create Pages and users to like Pages so that they can stay connected to the brands they care about. Content in your feed from people and brands you don’t follow snuck in under the aegis of showing you content your friends interacted with. Finally not only has Facebook had promoted posts for brands for a while, they now also allow users to promote their personal posts to friends for $7 a pop.

Without really thinking about it much, we’re halfway to a future where a significant percentage of the content of your Facebook feed is paid. Since the posts go through the same ranking algorithm as your regular feed, they are more likely to be relevant to you than the traditional ad products that Facebook and other online properties are known for today. When the goal is to be entertained, do you really care if that viral video of the day being shared via a friend is a paid impression or not? 

Twitter is playing catch up here but if they don’t, the flop that was Facebook’s IPO will look tame in comparison.

Now Playing: Macklemore & Ryan Lewis - Can't Hold Us Feat. Ray Dalton 

 

I’ve been joking with Omar that Google+ is the new FriendFeed. I recently posted this on Google+ and was asked to explain what I meant since Google+ doesn’t support importing of content from other services which was the key feature FriendFeed. The reason I say this is that Google+ fulfills the same need that FriendFeed when it first came out.

Here’s an excerpt from a post by Robert Scoble in 2008 about FriendFeed titled Loving my FriendFeed

I love my FriendFeed. Here’s a list of top bloggers who are using the service. Why do I love it? It’s one place you can find all my stuff and, even, comment on it. It’s amazing the discussions that a 140-character “Tweet” on Twitter can generate. I subscribe to a ton of people on FriendFeed and notice that often the conversations after a Twitter message will be 1000x longer (and generally more interesting) than the Twitter itself.

In my previous post I asked what problem Google+ solves and the answer is above. Google+, like FriendFeed before it, gives people a place to subscribe to and participate in conversations around content produced by people they are interested in.

Why Twitter Doesn’t Solve This Problem

Twitter relationships have been described as a public interest graph. Specifically, Twitter is a way to keep on top of people and content you find interesting whether it is tech news sites, bloggers, celebrities, government officials and even people you know. However there are a number of key gaps in the Twitter user experience which FriendFeed fixed and Twitter still hasn’t even though people have been complaining about them for years.

The first problem is that is really difficult to have conversations on Twitter. Here’s an excerpt from a TechCrunch post made in 2008 titled Actual Conversations On Twitter Not Possible Until Twitter Lets Us which explains the problem

One of the big complaints about Twitter is that conversations are hard to follow. Users can write a response to a Twitter message (or anything else), but the easy way to do this is to add an @[username] tag to the Twitter, which refers back to the original Twitter user. But by then that original user has often moved on to other subjects, and it becomes impossible to follow the conversation.
…
The fact is that Twitter purposefully doesn’t want users to be able to track conversations. The content begins and ends with a discreet Twitter message, up to 140 characters long. Competitor Friendfeed does a nice job of tracking conversations by letting users reply to actual messages, not just users. Twitter, for whatever reason (possibly to keep things simple), just doesn’t want that. And until they do, nothing is going to change.

The ability to have actual comment threads about a status update as opposed to disconnected @replies is a more satisfying experience for many users. As Mike Arrington stated above, the challenge for Twitter is that this would change the dynamics of the service in ways that take away some of the character of the service.

The second problem is that Twitter doesn’t give a public way to indicate that a piece of content is interesting without also sharing it. Specifically, there is no analog to Facebook’s “I like this” within the stream (not to be confused with the like button social plugin). Twitter has favorites but it’s actually meant to be a way to bookmark posts not to tell people you like the status update. There are now sites like Favstar.fm which have garnered a sizable user base by giving people a way to get “I like this” style functionality from Twitter and see how many people have favorited a tweet.

Both of these problems are fixed by Google+ and it is unsurprising that the same sorts of people who loved FriendFeed are not only on Google+ but are its most popular users. The question is whether Twitter will fix these problems with their experience given that this has made people pine for alternate services. Given that they didn’t try to address these when FriendFeed was at the height of its hype curve, it seems unlikely they will unless they see declines in their more mainstream user base.

Why Facebook Doesn’t Solve This Problem

Facebook relationships are an attempt to mirror our offline relationships online. The problem with this is captured in Paul Adams’ excellent slideshow The Real Life Social Network v2

The problem with Facebook is that people you may find interesting (i.e. your interest graph) or that find you interesting are not necessarily people you want to sharing the same space as your family, friends and even coworkers. A good example of this problem are the following suggestions I saw when I logged into Facebook this morning.

Alexia Tsotsis and Steven Levy are both journalists who work for TechCrunch and Wired respectively. Although I find the articles they write interesting, I don’t want to have them be on the receiving end my mobile phone videos of my son playing in the park or my check-ins from places around Seattle nor do I want to be subjected to their similar personal updates.

The combination of asymmetric following (people can subscribe to my updates without my accepting a friend request) and the ability to place people into groups (i.e. Circles) which can then be used to provide limited visibility to various updates is how Google+ solves this problem for various interest graphs. Neither of these features exists in Facebook today and while I suspect they will add the latter especially since Paul Adams now works there, it is harder to imagine seeing asymmetric follow ever showing up on Facebook outside of Pages.

Where That Leaves Us

I expect that both Twitter and Facebook will lose some chunk of people’s time to Google+. However Twitter is more vulnerable than Facebook, because Facebook has been fairly resistant the rise of the “interest graph” by building features like Facebook Pages which allows people to follow their interests in the same stream as updates from people they care about offline. For example, it is interesting to note that the most popular user on Twitter is Lady Gaga with 11.5 million followers but on the other hand her Facebook fan page has 40 million fans. Secondly, there really isn’t a gap Google+ fills with regards to communicating and staying in touch with the people one cares about offline via a social network.

On the other hand, Google+ is more in the same product space as Twitter being interest graph related which can be seen by the usage patterns of its early adopters. It’s also telling to read comments from Google+ readers on how much less time they now spend on Facebook and Twitter.

Now Playing: Frank Ocean - Novacane

 

Two months ago Nelson Minar wrote a post entitled Stop making social networks, Facebook won where he argues that websites should just treat Facebook as the one true social graph instead of trying to build their own. I agree a lot with what Nelson wrote which some people tell me conflicts with my argument that There will be many social graphs. I thought that the best way to illustrate this seeming contradictory thinking is by comparing two sites that the media considers competitors to Facebook in different ways; Twitter and FourSquare.

How Twitter and FourSquare position themselves against Facebook

Recently, Twitter’s VP of business and corporate development spoke at Nokia World 2010 where he proclaimed that Twitter is NOT a social network. Below are some excerpts from ReadWriteWeb’s coverage of his talk.

says Thau: Twitter is for news. Twitter is for content. Twitter is for information.
…
So Twitter Is "News"?

Yes, says Thau. Twitter is changing the very nature of news today. Journalists are sending their stories to Twitter and some are even publishing directly to Twitter. It's also allowing everyday users to become journalists themselves by providing them with a simple mechanism to break news.

"The guy who saw a plane land on the Hudson River right in front of him didn't think to send an email," says Thau. "He tweeted it."

Thau also wanted to assure Twitter users it's OK if you think you're not interesting enough to have your own Twitter account. Don't apologize if you don't tweet - just come to Twitter and consume content instead. After all, plenty of people already do just that.

The key thing here is that Twitter is arguing that the primary relationships on the site are not “social”. They are producer<->consumer where the product is news and other information content.

Dennis Crowley of FourSquare made more direct comparisons between his service and Facebook in an interview with TechCrunch.

On why the world needs more than one social graph

Our social graph is more representative of the people that you meet in the real world. I am starting to believe, if you asked me a year ago, Why would you ever need more than one social graph? You need representation of a couple of them. Between the three, Facebook is literally everyone I’ve ever shaken hands with at a conference or kissed on the cheek at Easter. Twitter seems to be everyone I am entertained by or I wish to meet some day. Foursquare seems to be everyone I run into on a regular basis. All three of those social graphs are powerful in their own.

The FourSquare argument is that services that create new social graphs that are tied to a specific social context can continue to exist and grow in a world where social networking is dominated by Facebook’s website and Facebook Connect.

 

Facebook’s trajectory: Adding a social element to every online activity

Before analyzing the wisdom of the approaches that FourSquare and Twitter have taken to differentiate their offerings from Facebook, it is a good idea to have an idea of Facebook’s ultimate strategy. This isn’t hard since Zuckerberg and other Facebook regularly share this with TechCrunch. Below is an excerpt from an article titled Zuckerberg: Facebook Photos Used 5 Or 6 Times More Than Competitors — Combined which describes their long term strategy

He noted that when they launched the product, they didn’t have all of the features that their competitors did. For example, they didn’t have high-resolution photos and you couldn’t print them. But one thing they did have was the social element — and this changed everything.

“Those features by themselves were more important than anything else combined,” Zuckerberg said of the social elements of Facebook Photos. He then dropped the competitor bomb. “The photo product that we have is maybe five or six times more used than every other product on the web — combined,” Zuckerberg stated.
…
And it was clear from both Zuckerberg and CTO Bret Taylor’s talk at the event that photos to them was the harbinger of things that eventually came — and will still come.

Taylor noted that he had been “brainwashed by Silicon Valley” before he saw and understood the power of Facebook Photos (he was likely working at Google at the time). He had been thinking like an engineer about the best way to organize photos on the web. But he quickly realized that “the best possible organization of photos is around people,” Taylor said.

“There are ten other industries waiting to have this type of disruption,” Taylor said noting the travel industry, e-commerce, and music as a few of them. Earlier, Zuckerberg agreed. Because of the social element, “every single vertical will be transformed.“

Facebook’s social graph is a graph of people I know or have met. Facebook’s fundamental strategy is to build a product and platform where key online activities are improved by adding the social element of people you know. Where Facebook has been dominant is when the activity is one that already related to interacting with people you know. Facebook has beaten geek favorites like Gmail, Flickr and Delicious as the way regular people share private messages, photos and links online. This is both due to the powerful network effects of a social networking product and the fact that their graph maps 100% to the people one typically wants to indulge in those activities with.

Facebook has had less success with products where their graph doesn’t correspond well with the target activity. The best examples of this are Facebook Marketplace versus eBay/Craig's List or Facebook Questions versus Yahoo! Answers. In both of these comparisons, Facebook isn’t even on the radar of the market leader. This is because activities such as buying someone’s old junk really need a wider net than just your friends, family and coworkers.

This is where the positioning and focus of Twitter as a news service as opposed to a social network puts them in a good place in comparison to Facebook. Twitter is where I go to get entertainment and news about the topics I’m interested in from subject matter experts. These subject matter experts (in many cases bloggers, minor & major celebrities) are not people I know nor  have I met. This is distinct from my Facebook social graph but has some overlap depending on how much of a subject matter expert I am myself. On the other hand, FourSquare is a place where I go to share my location with people I know or have met. This set of people is almost always a subset of the people in my Facebook social graph. The only value additions you get from FourSquare are the game mechanics and deals (not anymore). FourSquare has unfortunately reached the point where the only practical difference between using it versus Facebook Places is that I get to be mayor of my local Gymboree and collect two dozen video game style achievements. Personally I’ve already grown bored with the game mechanics and suspect that targeting the console gaming demographic guarantees it will be a niche service at best.

The bottom line is that if the primary focus of your product is that it connects people with their friends, family and others they know around a particular activity then you need to be able to answer the question as to how your product can compete in a world where your service is a feature of Facebook or of an app on its platform.

Now Playing: Far East Movement - Like a G6

 

I was reading Pandas and Lobsters: Why Google Cannot Build Social Applications... and came across the following statements

Now, consider the Four Horsemen of Hotness in 2010: Facebook, Quora, Foursquare, and Twitter. Think deeply about why none of these four could have been developed inside Google.
...
Quora is restaurant that serves huge quantities of bacn and toast. Quora is a dozen people running dozens of experiments in how to optimally use bacn to get people to return to Quora, and how to use toast to keep them there. Bacn is email you want but not right now, and Quora has 40 flavors of it that you can order. Quora's main use of Bacn is to sizzle with something delicious (a new answer to a question you follow, a new Facebook friend has been caught in the Quora lobster trap, etc.) to entice you to come back to Quora. Then, once you're there, the toast starts popping. Quora shifts the content to things you care about and hides things you don't care about in real-time, and subtly pops up notifications while you're playing, to entice you to keep sticking around and clicking around.

Although I’m a regular user of Foursquare, Twitter and Facebook and consider myself to be fairly up on what’s going on in the social media space, I’d never used Quora when I read the article. Given that I’ve seen hype about it in various corners I decided to create an account and give the service a try. Below are some of my impressions

What is Quora?

The easiest way to think about Quora is that Quora is to Yahoo! Answers as Facebook is to MySpace. It is a Q&A site where users utilize their real names often linked to Facebook profiles as opposed to pseudonyms. It avoids game mechanics such as high score leaderboards and user badges that services like Stackoverflow and Windows Live QnA instead relying on people getting kudos from their peers in the form of endorsements and votes on their answers to motivate users to answer questions.

Why is Quora is so Hot

Quora seems to have started off as an invitation-only service which allowed them to cherry pick the original users to meet a particular demographic (i.e. Silicon Valley geek) and also carefully manage the initial culture of the site. What they’ve created is a place where members of the Silicon Valley technorati and wannabes can post questions and expect them to be answered by members of the tech elite including insiders at various tech companies. Quora is the kind of place where questions like What are the scaling issues to keep in mind while developing a social network feed? is answered by one of the people who built the original Facebook news feed and a random opinion like Should Mark Zuckerberg step down as CEO of Facebook and find a more seasoned replacement? gets a response from Blake Ross. There aren’t many places online where a question like What were the 4 or 5 key decisions that Larry Page and Sergey Brin made in the early days of Google? can get a serious answer let alone a well researched history lesson as well as some actual insights.

This site is pure gold for technology bloggers, journalists and Web startup geeks. It is unsurprising that these sorts of people would consider Quora to be the greatest thing since sliced bread.

Quora is a Community Site not a Communications Tool

One of the things I find weird about lumping Quora into the same grouping as Facebook, Foursquare and Twitter is that unlike those sites it is not a communication tool. Facebook created a new communication channel between friends, acquaintances and family members that sits somewhere between brings together the functionality of email and IM along with the feed. Twitter created a lighter weight way to consume and create content for brands and people you find interesting as compared to blogging. Foursquare is about broadcasting your location to interested parties.

Quora on the other hand seems to have more in common with mail lists and discussion forums. Specifically, it is more like Metafilter, Digg and Reddit than it is like the aforementioned sites. This is a service that will live and die based on the culture of its community and is very dependent on "power users” who altruistically provide lots of value to the site in exchange for respect from their peers. The challenge for Quora is that it will be difficult to keep its current culture as it grows bigger. Will Facebook and Google insiders still be showing up in various question threads if the site grows to be as big as Yahoo! Answers with the same breadth of audience and volume of content? I can’t imagine that happening.

I also can’t imagine being able to segregate audiences like you can on communications services. Twitter has communities of mommy bloggers, tech bloggers, fans of various celebrities, sports fans, etc which operate independently of each other and really only are noticed by others every once in a while due to the trending topics feature. The same goes for Facebook and Foursquare. Quora will not be able isolate the various demographics from each other without changing the nature of the site. However they will have to figure that out once the current crop of users start logging in and seeing "how is babby formed" style questions because the site has taken off.

Then again, we might get lucky and the site never take off with the masses which may not be good for the VCs that have invested in it but would be for the community that has formed there.

Now Playing: Young Buck - Welcome to the South (featuring Lil Flip & David Banner)

 

In a move that was telegraphed by Fred Wilson’s (Twitter investor) post titled The Twitter Platform's Inflection Point where he criticized Twitter platform developers for “filling holes” in Twitter’s user experience, the Twitter team have indicated they will start providing official Twitter clients for various mobile platforms. There have been announcements of an official Blackberry client and the purchase of Tweetie so it can become the official iPhone client. The latter was announced in the blog post Twitter for iPhone excerpted below

Careful analysis of the Twitter user experience in the iTunes AppStore revealed massive room for improvement. People are looking for an app from Twitter, and they're not finding one. So, they get confused and give up. It's important that we optimize for user benefit and create an awesome experience.

We're thrilled to announce that we've entered into an agreement with Atebits (aka Loren Brichter) to acquire Tweetie, a leading iPhone Twitter client.

This has led to some anger on the part of Twitter client developers with some of the more colorful reactions being the creation of the Twitter Destroyed My Market Segment T-shirt and a somewhat off-color image that is making the rounds as representative of what Twitter means by “filling holes”.

As an end user and someone who works on web platforms, none of this is really surprising. Geeks consider having to wade through half a dozen Twitter clients before finding one that works for them a feature even though paradox of choice means that most people are actually happier with less choices not more. This is made worse by the fact that in the mobile world, this may mean paying for multiple apps until you find one that you’re happy with.

Any web company that cares about their customers will want to ensure that their experience is as simple and as pleasant as possible. Trusting your primary mobile experience to the generosity and talents of 3rd party developers means you are not responsible for the primary way many people will access your service. This loss of control isn’t great especially when the design direction you want to take your service in may not line up with what developers are doing in their apps. Then there’s the fact that forcing your users to make purchasing decisions before they can use your site conveniently on their phone isn’t a great first time experience either. 

I expect mobile clients are just the beginning. There are lots of flaws in the Twitter user experience that are due to Twitter’s reliance on “hole fillers” that I expect they’ll start to fill. The fact that I ever have to go to http://bit.ly as part of my Twitter workflow is a bug. URL shorteners really have no reason to exist in the majority of use cases except when Twitter is sending an SMS message. Sites that exist simply as image hosting services for Twitter like Twitpic and YFrog also seem extremely superflous especially when you consider that since only power users know about them not every Twitter user is figuring out how to use the service for image sharing. I expect this will eventually become a native feature of Twitter as well. Once Twitter controls the primary mobile clients for accessing their service, it’ll actually be easier for them to make these changes since they don’t have to worry about whether 3rd party apps will support Twitter image hosting vs. Twitpic versus rolling their own ghetto solution.

The situation is made particularly tough for 3rd party developers due to Twitter’s lack of a business model as Chris Dixon points out in his post Twitter and third-party Twitter developers

Normally, when third parties try to predict whether their products will be subsumed by a platform, the question boils down to whether their products will be strategic to the platform. When the platform has an established business model, this analysis is fairly straightforward (for example, here is my strategic analysis of Google’s platform).  If you make games for the iPhone, you are pretty certain Apple will take their 30% cut and leave you alone. Similarly, if you are a content website relying on SEO and Google Adsense you can be pretty confident Google will leave you alone. Until Twitter has a successful business model, they can’t have a consistent strategy and third parties should expect erratic behavior and even complete and sudden shifts in strategy.

So what might Twitter’s business model eventually be?  I expect that Twitter search will monetize poorly because most searches on Twitter don’t have purchasing intent.  Twitter’s move into mobile clients and hints about a more engaging website suggest they may be trying to mimic Facebook’s display ad model.

The hard question then is what opportunities will be left for developers on Twitter’s platform once the low hanging fruit has been picked by the company. Here I agree with frequent comments by Dave Winer and Robert Scoble, that there needs to be more metadata attached to tweets so that different data aggregation and search scenarios can be built which satisfy thousands of niches. I especially like what Dave Winer wrote in his post How Twitter can kill the Twitter-killers where he stated

Suppose Twitter wants to make their offering much more competitive and at the same time much more attractive to developers. Sure, as Fred Wilson telegraphed, some developers are going to get rolled over, esp those who camped out on the natural evolutionary path of the platform vendor. But there are lots of things Twitter Corp can do to create more opportunities for developers, ones that expand the scope of the platform and make it possible for a thousand flowers to bloom, a thousand valuable non-trivial flowers.

The largest single thing Twitter could do is open tweet-level metadata. If I want to write an app for dogs who tweet, let me add a "field" to a tweet called isDog, a boolean, that tells me that the author of the tweet is a dog. That way the dog food company who has a Twitter presence can learn that the tweet is from a dog, from the guy who's developing a special Twitter client just for dogs, even though Twitter itself has no knowledge of the special needs of dogs. We can also add a field for breed and age (in dog years of course). Coat type. Toy preference. A link to his or her owner. Are there children in the household?

I probably wouldn’t have used the tweeting dog example but the idea is sound. Location is an example of metadata that is added to tweets which can be used for interesting applications on top of the core news feed experience as shown by Twittervision and Bing's Twitter Maps. I think there’s an opportunity to build interesting things in this space especially if developers can invent new types of metadata without relying on Twitter to first bless new fields like they’ve had to do with location (although their current implementation is still inadequate in my opinion).

Over the next few months, Twitter will likely continue to encroach on territory which was once assumed to belong to 3rd party developers. The question is whether Twitter will replace these opportunities they’ve taken away with new opportunities or instead if they’ve simply used developers as a means to an end and now they are no longer useful?

Now Playing: Notorious B.I.G. - One More Chance

 

From the Google Wave Federation architecture white paper 

Google Wave is a new communication and collaboration platform based on hosted documents (called waves) supporting concurrent modifications and low-latency updates. This platform enables people to communicate and work together in new, convenient and effective ways. We will offer these benefits to users of wave.google.com and we also want to share them with everyone else by making waves an open platform that everybody can share. We welcome others to run wave servers and become wave providers, for themselves or as services for their users, and to "federate" waves, that is, to share waves with each other and with wave.google.com. In this way users from different wave providers can communicate and collaborate using shared waves. We are introducing the Google Wave Federation Protocol for federating waves between wave providers on the Internet.

From a Buzz post by Dewitt Clinton, a Google employee

The best way to get a sense of where the Buzz API is heading is to take a look at http://code.google.com/apis/buzz/. You'll notice that the "coming soon" section mentions a ton of protocols—Activity Streams, Atom, AtomPub, MediaRSS, WebFinger, PubSubHubbub, Salmon, OAuth, XFN, etc.

What it doesn't talk much about is Google. That's because the goal isn't Google specific at all. The idea is that someday, any host on the web should be able to implement these open protocols and send messages back and forth in real time with users from any network, without any one company in the middle. The web contains the social graph, the protocols are standard web protocols, the messages can contain whatever crazy stuff people think to put in them. Google Buzz will be just another node (a very good node, I hope) among many peers. Users of any two systems should be able to send updates back and forth, federate comments, share photos, send @replies, etc., without needing Google in the middle and without using a Google-specific protocol or format.

From Mark Sigal’s post Google Buzz: Is it Project, Product or Platform?

I think that it's great that Google is iterating Gmail (read Tim O'Reilly's excellent write-up on it here), and actually improving an existing product, versus rolling out a knock-off of something that is already in the market.

Nonetheless. I am confused. I thought that Google Wave was destined to be the new Gmail, but after yesterday's announcement, I am left wondering if Gmail is, instead, the new Google Wave.

---

Since the saying goes that people in glass houses shouldn’t throw stones, I won’t make any comment besides sharing these links with you.

Now Playing: 50 Cent - Crime Wave

 

The New York Times has an article titled How Google Decides to Pull the Plug which talks about the rationale behind the rash of abandoned and cancelled projects that have come out of the Big G in recent months. The article contains the following interesting excerpts

GOOGLE recently set the blogosphere abuzz by announcing that it was pulling the plug on several products.

The victims included Lively, a virtual world that was Google’s answer to Second Life; Dodgeball, a cellphone service aimed at young bar-hoppers who wanted to let their friends know where they were hanging out; Catalog Search, which scanned paper product catalogs so they could be searched online; and Notebook, a simple tool that allowed people to take notes on Web sites they had visited.

Google also said it would stop actively developing Jaiku, a microblogging service similar to Twitter, and instead turn it over to its users as an open-source project they could tinker with as they wished.
…
All of the shuttered projects failed several of Google’s key tests for continued incubation: They were not especially popular with customers; they had difficulty attracting Google employees to develop them; they didn’t solve a big enough problem; or they failed to achieve internal performance targets known as “objectives and key results.”

You’d think that Google, a highly profitable engineer’s playground, would keep supporting quirky side projects as long as someone wanted to work on them. The company, which is best known to consumers for its search engine, is famous in business for promoting innovation by letting engineers devote 20 percent of their time to projects outside their main responsibilities.

But in this difficult economy, even Google is paying more attention to costs.
…
Lively, Google’s entry into three-dimensional virtual worlds, was publicly unveiled last July. Four months later, when the company decided to close it, only 10,000 people had logged into the service over the previous seven days. That was well below the targets set by Google’s quarterly project review process, and far behind Second Life from Linden Lab, which had about half a million users in a similar period.

“We didn’t see that passionate hockey-stick growth in the user base,” said Bradley Horowitz, Google’s vice president for product management. Management decided that the half-dozen people working on Lively could be more productive elsewhere.

It will be interesting to see what the long term effects of these changes in perspective will be on Google's culture around launching new products out of 20% time projects and the veneration of side projects at the Googleplex. One expected change is that employees will be more conservative around what 20% projects they choose to work on so that they don't end up wasting their time on the next Google Page Creator or Google Web Accelerator which is received with initial hype but quickly discontinued because it doesn't see "hockey stick growth in user base".

You can already see this happening somewhat by looking at how many side projects are being shipped as part of Gmail labs. Checking out the list of Gmail labs offerings I see over 30 big and small features that have been added to Gmail as side projects from various individuals and teams at Google. It seems on the surface that a lot of Google employees are betting on tying their side projects to a huge, successful product like Gmail which isn't in danger of being cancelled instead of working on new projects or helping out smaller projects like Dodgeball and Jaiku which eventually got cancelled due to lack of interest.

This expectation that a new Google product will need massive adoption to justify its investment or be cancelled within four months, as was the case with Google Lively, will be a significant dampener new product launches. Reading Paul Buchheit's post on the early days of Gmail I wonder how much time he'd have invested in the project if he was told that Google would cancel the project if it's user base growth wasn't competitive with market leaders like Yahoo! Mail and Hotmail's within four months.

I suspect that the part of Google's DNA which spurs innovation from within is being killed. Then again when you look at Google's long list of acquisitions you realize that a lot of their most successful projects outside of search including Google Maps, Blogger and YouTube were the results of acquisitions. So maybe this culture of internal innovation was already on the way out and the current economic downturn has merely sealed its fate.

Now Playing: Metallica - Fade To Black

 

Facebook made two interesting announcements this week in the area of making their platform more open. The first announcement was made by Mike Shroepfer in the developer blog post Next Steps in Openness where he writes

Enabling social information to flow through the Web is one of the core goals of Facebook. In the two months since Facebook Connect became generally available, over 4,000 sites and desktop applications have gone live with the service. Users can now log into sites across the Web using their Facebook account, bring their identity and friends with them, and share information and experiences using the same features as they would with applications on Facebook.

As we've launched and built Facebook Connect, we've been participants in OpenID efforts. One of our user experience experts, Julie Zhuo, presented at the UX Summit in October. Several of our engineers have been participating in meetups, and one of them ran as a community member for a board seat. We're happy to announce today that we are formalizing our support of the OpenID Foundation by officially joining the board.

At first glance it seems contradictory for Facebook to join the OpenID foundation given that many people view Facebook Connect to be a competitor to OpenID as a single sign-on solution albeit a centralized one. The confusion from pundits around this announcement from Facebook is summarized in the Ars Technica article Facebook's "next steps in openness" raises questions which openly questions why the company needs to join the OpenID foundation when Facebook Connect seems to be enjoying broader deployment across the Web.

The reason seems quite straightforward to me. Right now, Facebook has created a situation where web sites that want to enable their users log-in with credentials from third party sites need to implement two solutions; an OpenID consumer and Facebook Connect. This places a burden on web sites which now either have to chose to adopt one solution over the other or create a user experience problem by offering users both options. It's bad enough that there is now a market for companies who wrap both solutions in a single package like JanRain and Clickpass. I wouldn't be surprised if Facebook has been getting an earful from some of their larger Facebook Connect partners about being a competing offering to OpenID.

That said, I doubt that a generic solution like OpenID will ever beat the ease of use and value added functionality that Facebook has created with Facebook Connect. By supporting OpenID, Facebook gets to silence the sites who simply want single sign-on solution that is built on standards so they can implement one solution to get access to users of Facebook, Google, Yahoo!, Windows Live, MySpace, and AOL's services among others. However anyone who wants the richer integration that comes with integrating with the Facebook mini-feed and live feed will still need to adopt the proprietary Facebook Connect.

The second interesting announcement about Facebook opening up comes from Chris Putnam in the post titled Opening Up Facebook Status, Notes, Links, and Video to Facebook Platform which is excerpted below

We've seen increasing engagement with over 15 million users updating their status each day and sharing over 24 million links per month. We wanted to make sure this content and the ability to share this content was available through our standard APIs.

Specifically, your applications can now directly access all of a user's status, links, and notes via new methods and FQL calls. Your application will have access to any status, notes, or links from the active user or their friends that are currently visible to the active user. In addition, we're opening new APIs for you to post links, create notes, or upload videos for the current user, and we've made setting a user's status easier.

We're pretty excited to see what kinds of ideas you can come up with to help users create and share more content.

Like Facebook Connect, this is a further step away from the walled garden approach that characterized the Facebook platform of 2007. In 2007, the goal of Facebook's platform efforts was to tell websites to re-implement themselves as Facebook apps (e.g. iLike website vs. iLike Facebook app). With Facebook Connect and the changes to their APIs announced above, Facebook is now acknowledging that their users use other websites and integration with those sites doesn't require these sites to become sharecroppers on the digital farmland that is the Facebook canvas page.

Twitter has shown the value of having open APIs for link sharing and status updates which isn't lost on Facebook given their failed attempt to acquire Twitter. Sharing status updates is a powerful way for humans to connect in social contexts and given Facebook's goal to be the world's social operating system we can expect them to make more moves in this space. It's particularly interesting to see how people use status updates as a way to communicate and share experiences during news events such as President Obama's inauguration. Facebook saw this first hand with their integration of Facebook's status updates with the CNN website where Randi Zuckerburg wrote

Over 2 million status updates were posted through the CNN.com Live Facebook feed with 4,000 status updates per minute on average and a huge spike of 8,500 updates when President Obama began his speech. We're thrilled that you and so many others chose to actively participate as history was being made—by sharing the experience with the people that matter most to you.

Similar surges were seen on Twitter as evidenced by the chart below. The thing that is interesting to note is that while status updates increased during the most exciting part of the inauguration other forms of Web traffic declined at the same time.

Similar traffic drop on Last.FM seen on the right

Google saw a similar drop as well

Was it because everybody went to Twitter?

Charts taken from the post Understanding Web Operations Culture - the Graph & Data Obsession on the O'Reilly Rader blog.

UPDATE: I've been playing around with the APIs announced and it looks like there actually isn't an API to get your friends' status updates included in the announcement. The status.get method only returns the status updates posted by the currently user. The closest I could get was trying the query below via fql.query but it never returned results

SELECT uid, message FROM status WHERE uid IN (SELECT uid2 FROM friend WHERE uid1=$userid)

So much for the claims that this is a Twitter killer.

Now Playing: Eminem, Dr. Dre & 50 Cent - Crack a Bottle

 

This morning I was reading Om Malik's post entitled Google & The Big Ideas where he writes

Earlier today, when I read about Google launching a mobile version of Tasks. I was amazed by the attention being focused on essentially a to-do list website. And while it wasn’t worth a story, I shared my feelings via I sent Twitter. My tweet read:

I think google has no big ideas. this morning they announced a to-do-list. FGS. [For God Sake] Remember the Milk MUCH better.

...
Matt’s comments and the responses both on this blog and FriendFeed resulted in some thoughts about what constitutes a big idea, where Google is right and where it is light. Instead of responding on FriendFeed, I decided to share my thoughts with you, hoping that we can have a larger conversation about Google and the big ideas.

For me startups and products such as Skype, Flickr and YouTube represent big ideas. Why? Because they not only redefine our notions about certain technologies, but they also change our behavior and cause massive disruption. For instance, Skype redefined our relationship with our phone and in the process put disrupting the telecom industry. Flickr made a largely way web into a dynamic, thriving social community. Today even our friends at Wall Street Journal have comments, and the New York Times is hoping to use LinkedIn to foster a community. YouTube made us rethink television, by making it dead simple not only to consume video but also to broadcast video.

Similarly, Google’s Search changed how we consumed information. Instead of going to destinations, we now consume information by just finding it. What made this “big idea” even more disruptive — Google’s use of data analytics to offer highly focused advertising messages to marry search queries. (Of course, Google wasn’t the one to think of this big idea, but that’s a whole another story.)

The problem with "Big Ideas" is that sometimes people get obsessed with how game changing or disruptive their product is instead of focusing on making it the best at what it does.

I remember a few years ago when every other article out of the technology press was about how innovative every product Google created was. For a while, this created a scramble amongst various Web companies to show that they could be just as innovative as Google. Yahoo! tried to do this by buying innovation in the form of every hot startup they could get their hands on (Flickr, del.icio.us, Jumpcut, Webjay, Konfabulator, Upcoming, etc) which only led to spreading itself too thin as eloquently captured in Brad Garlinghouse's Peanut Butter Manifesto memo.

On the other hand, there are lots of products that have been unoriginal ideas now seen as game changing because of how these ideas have been executed. One example is the iPod + iTunes which started of as yet another MP3 player with a proprietary music player but has grown to be a cultural force and the number one retailer of music in the United States. When Facebook launched it plowed ground that had already been covered by MySpace, Orkut and Friendster yet today it is the number one social networking site in the world. In both cases, these have been very focused products that have been careful to grow their feature set in a limited way but have done an excellent job in providing a very user-centric experience. This isn't to say that there aren't any innovations in Facebook or in iTunes+iPod but instead that they are examples of products that have been game changing not by being a "big idea" but instead by being full of lots of good ideas.

Being original or innovative is nice but even better is building a product that your customers love. Never forget that.

Now Playing: Justin Timberlake - What Goes Around.../...Comes Around Interlude

 

A few days ago, the top news story on Techmeme was the fact that Google launched Google Friend Connect and Facebook announced the final version of Facebook Connect within minutes of each other. Reading up on both announcements it seems interesting to note how most of the coverage is about who will win the race to dominate the Web as opposed to what end user value is actually being created by these technologies.

It was somewhat depressing until I read Dave Winer's brilliant post Soon it will be time to start over, again which contains the following excerpt

We're now reaching the end of a cycle, we're seeing feature wars. That's what's going on between Facebook and Google, both perfectly timing the rollouts of their developer proposition to coincide with the others' -- on the very same day! I don't even have to look at them and I am sure that they're too complicated. Because I've been around this loop so many times. The solution to the problem these guys are supposedly working on won't come in this generation, it can only come when people start over. They are too mired in the complexities of the past to solve this one. Both companies are getting ready to shrink. It's the last gasp of this generation of technology. 

But the next one can't be far away now. It will be exhilirating!!

Remember how great Google was when it first appeared?

Remember how great Netscape was, and before that Apple, and I know you guys won't like this, but Microsoft offered us some great new places to play. I remember finding out that their OS address space in 1981 was 640K. That was a lot to guy who was spending huge amounts of time trying to cram a 256K app into 48K.

The trick in each cycle is to fight complexity, so the growth can keep going. But you can't keep it out, engineers like complexity, not just because it provides them job security, also because they really just like it. But once the stack gets too arcane, the next generation throws their hands up and says "We're not going to deal with that mess." 

We're almost there now.

The value of Facebook Connect to Facebook is obvious. They get to become a centralized identity provider for the Web including the benefit of tracking every single time one of their users logs-in on a partner which lets them build an even better advertising profile of their users. Similarly the value to the customers of the sites adopting it seem clear at first. Below are the claimed benefits of Facebook Connect to users from my initial perusal

1. One login, multiple sites. No need to create a new account on partner sites.
2. Account information such as profile picture, location and other fields on the partner site can be prepopulated from Facebook
3. Bring your social graph with you to partner sites. 
4. Let your friends on Facebook know what you are doing on partner sites. Updates show up on your profile but do not go in your friends' news feeds (they go in their live feed instead). 

Where things get interesting is that none of these benefits require a proprietary and centralized approach like Facebook has done. If Facebook implemented OpenID and OpenID attribute exchange, they could have given their users the benefits of #1 and #2 using widely adopted industry standards.  For #3, there is the burgeoning Portable Contacts effort to define a set of standard APIs for accessing the social graph that supports the key data portability principles around this information. As for broadcasting your updates from one site to another, FriendFeed has shown how that can be done using standard technologies like RSS, Atom and XMPP. 

Ignoring the fact that Facebook Connect is a proprietary and centralized approach instead of being based on open standards, there are still other points worthy of debate. When trying out sites like CitySearch beta with Facebook Connect, the experience is that I am connected with all of my Facebook friends who also use CitySearch. There is the genuine question of whether users really want to use one friends' list across every site regardless of context (e.g. interacting with the exact same people on LinkedIn, MySpace and XBox Live) or whether they want to have universal access to any of their friends lists and bridge them when necessary?

Yesterday on Twitter, I mentioned that Facebook Connect is the wrong direction to go on the Web for the reasons above. I also called Google Friend Connect a misguided "me too" effort for trying to copy Facebook's strategy and glomming an AJAX widget play on top of it. Kevin Marks, an evangelist at Google challenged by statement with the following response

@Carnage4Life the problem for users is re-entering data and restating friends for each app. For developers its becoming social without this

If that is truly the problem, how does the technology in the video below solve the problem any better than the combination of OpenID and Portable Contacts?

As with OpenSocial, Google has fallen in love with its role as a spoiler when it comes to Facebook's platform efforts without stopping to think whether it actually makes sense to be aping Facebook's strategies in the first place. Monkey see, monkey do.

This will be the death of them if they aren't careful. Once you become a follower and define yourself by reacting to others actions, it is hard to step back into a leadership role both in the industry and even within one's corporate culture.

Now Playing: Britney Spears - Circus

 

My RSS reader is buzzing with a lot hype around Facebook's Connect this morning. The lead story seems to be the New York Times article entitled Facebook Aims to Extend Its Reach Across the Web which announces that a number of popular sites are about to adopt the technology. The article is excerpted below

Facebook Connect, as the company’s new feature is called, allows its members to log onto other Web sites using their Facebook identification and see their friends’ activities on those sites. Like Beacon, the controversial advertising program that Facebook introduced and then withdrew last year after it raised a hullabaloo over privacy, Connect also gives members the opportunity to broadcast their actions on those sites to their friends on Facebook.

In the next few weeks, a number of prominent Web sites will weave this service into their pages, including those of the Discovery Channel and The San Francisco Chronicle, the social news site Digg, the genealogy network Geni and the online video hub Hulu.
…
MySpace, Yahoo and Google have all announced similar programs this year, using common standards that will allow other Web sites to reduce the work needed to embrace each identity system. Facebook, which is using its own data-sharing technology, is slightly ahead of its rivals.

This set of partners is definitely higher profile than the last list of Facebook Connect adopters and yet I still have to wonder how this is eventually going to shake out. Even with this set of partners there are still two big hurdles Facebook has to surmount. The first is just getting users to connect their identities on different sites with their Facebook identity. Just having the ability to connect a Digg account and a Facebook account doesn't mean users will adopt the feature. I assume this is why the Facebook Beacon automatically linked a user's account on a partner site to their Facebook account in the first place. How this is presented to users on participating sites will be key to its adoption and this is mostly out of Facebook's control.

The other challenge that Facebook Connect will face is how to prevent it from tarred with the same "centralized identity service" brush that Microsoft's Passport got tarred with at the turn of the century. Back in the year 2000, Joel Spolsky wrote Does Issuing Passports Make Microsoft a Country? which began as follows

Am I the only one who is terrified about Microsoft Passport? It seems to me like a fairly blatant attempt to build the world's largest, richest consumer database, and then make fabulous profits mining it. It's a terrifying threat to everyone's personal privacy and it will make today's "cookies" seem positively tame by comparison. The scariest thing is that Microsoft is advertising Passport as if it were a benefit to consumers, and people seem to be falling for it! By the time you've read this article, I can guarantee that I'll scare you into turning off your Hotmail account and staying away from MSN web sites.

These sentiments never went away and by 2005 Microsoft had lost some of its most prominent Passport partner sites. The service has since been rebranded Windows Live ID and is now primarily a unified identity system for Microsoft sites as opposed to being a single-sign on service for the entire Web. It may be that Microsoft was ahead of its time (as I've argued it was with Hailstorm and other initiatives) but the arguments against centralized identity systems have seemed pretty convincing in the past. In addition, I suspect that developers will start asking questions when they realize that they have to support one proprietary technology for Facebook Connect, something different for MySpace Data Availability and yet another for Google Friend Connect. How many sign-in buttons will end up adorning these sites? http://beta.citysearch.com already has two sign-in links, will that expand to four as they move to support signing in with your MySpace account and your Google Friend Connect-enabled account? Or will services decide to pick a social network to side with to the exclusion of all others? It's beginning to remind me of the high definition DVD format wars and not in a good way.

Interesting times indeed.

Now Playing: Kanye West - Welcome To Heartbreak (feat. Kid Cudi)

 

John McCrea of Plaxo has written a cleverly titled guest post on TechCrunchIT, Facebook Connect and OpenID Relationship Status: “It’s Complicated”, where he makes the argument that Facebook Connect is a competing technology to OpenID but the situation is complicated by Facebook developers engaging in discussions with the OpenID. He writes

You see, it’s been about a month since the first implementation of Facebook Connect was spotted in the wild over at CBS’s celebrity gossip site, TheInsider.com. Want to sign up for the site? Click a single button. A little Facebook window pops up to confirm that you want to connect via your Facebook account. One more click – and you’re done. You’ve got a new account, a mini profile with your Facebook photo, and access to that subset of your Facebook friends who have also connected their accounts to TheInsider. Oh, and you can have your activities on TheInsider flow into your Facebook news feed automatically. All that, without having to create and remember a new username/password pair for the site. Why, it’s just like the vision for OpenID and the Open Stack – except without a single open building block under the hood!
...
After the intros, Allen Tom of Yahoo, who organized the event, turned the first session over Max Engel of MySpace, who in turn suggested an alternative – why not let Facebook’s Julie Zhuo kick it off instead? And for the next hour, Julie took us through the details of Facebook Connect and the decisions they had to make along the way to get the user interface and user experience just right. It was not just a presentation; it was a very active and engaged discussion, with questions popping up from all over the room. Julie and the rest of the Facebook team were engaged and eager to share what they had learned.

What the heck is going on here? Is Facebook preparing to go the next step of open, switching from the FB stack to the Open Stack? Only time will tell. But one thing is clear: Facebook Connect is the best thing ever for OpenID (and the rest of the Open Stack). Why? Because Facebook has set a high bar with Facebook Connect that is inspiring everyone in the open movement to work harder and faster to bring up the quality of the UI/UX for OpenID and the Open Stack.

There are a number of points worth discussing from the above excerpt. The first is the implication that OpenID is an equivalent technology to Facebook Connect. This is clearly not the case. OpenID just allows you to delegate to act of authenticating a user to another website so the user doesn't need to create credentials (i.e. username + password) on your site. OpenID alone doesn't get you the user's profile data nor does it allow you to pull in the authenticated user's social graph from the other site or publish activities to their activity feed. For that, you would need other other "Open brand" technologies like OpenID Attribute Exchange, Portable Contacts and OpenSocial. So it is fairer to describe the contest as Facebook Connect vs. OpenID + OpenID Attribute Exchange + Portable Contacts + OpenSocial.

The question then is who should we root for? At the end of the day, I don't think it makes a ton of sense for websites to have to target umpteen different APIs that do the same thing instead of targeting one standard implemented by multiple services. Specifically, it seems ridiculous to me that TheInsider.com will have to code against Facebook Connect to integrate Facebook accounts into their site but code against something else if they want to integrate MySpace accounts and yet another API if they want to integrate LinkedIn accounts and so on. This is an area that is crying out for standardization.

Unfortunately, the key company providing thought leadership in this area is Facebook and for now they are building their solution with proprietary technologies instead of de jure or de facto ("Open brand") standards. This is unsurprising given that it takes three or four different specs in varying states of completeness created by different audiences deliver the scenarios they are currently interested in. What is encouraging is that Facebook developers are working with OpenID implementers by sharing their knowledge. However OpenID isn't the only technology needed to satisfy this scenario and I wonder if Facebook will be similarly engaged with the folks working on Portable Contacts and OpenSocial.

Facebook Connect is a step in the right direction when it comes to bringing the vision of social network interoperability to fruition. The key question is whether we will see effective open standards emerge that will target the same scenarios [which eventually even Facebook could adopt] or whether competitors will offer their own proprietary alternatives? So far it sounds like the latter is happening which means unnecessary reinvention of the wheel for sites that want to support "connecting" with multiple social networking sites.

PS: If OpenID phishing is a concern now when the user is redirected to the ID provider's site to login, it seems Facebook Connect is even worse since all it does is provide a pop over. I wonder if this is because the Facebook folks think the phishing concerns are overblown.

Now Playing: 2Pac - Mind Made Up (feat. Daz, Method Man & Redman)

 

John Battelle has a blog post entitled When Doesn't It Pay To Pay Attention To Search Quality? which contains the following statement and screenshot

the top result is the best result - and it's a paid link.

In the past I've talked about Google's strategy tax which is the conflict between increasing the relevance of their search results and increasing the relevance of their search ads. The more relevant Google's "organic" results are the less likely users are to click on their ads which means the less money the company makes. This effectively puts a cap on how good Google's search quality can get especially given the company's obsessive measurements of every small change they make to get the most bang for the buck.

When I first wrote about this, the conclusion from some quarters was that this inherent conflict of interest would eventually be Google's undoing since there were search innovations that they would either be slow to adopt or put on the back burner so as not to harm their cash cow. However John Battelle's post puts another spin on the issue. As long as people find what they want it doesn't matter if the result is "organic" or an ad.

As Jeremy Zawodny noted in his post The Truth about Web Navigation 

Oh, here's a bonus tip: normal people can't tell the difference between AdSense style ads and all the other links on most web sites. And almost the same number don't know what "sponsored results" on the Search Results Page are either. It's just a page of links to them. They click the ones that look like they'll get them what they want. It's that simple.

even more interesting is the comment by Marshall Kirkpatrick in response to Jeremy's post

The part of your post about AdWords reminds me of a survey I read awhile ago. Some tiny percentage of users were able to tell the difference between paid and natural search results, then once away from the computer almost all of them when asked said that the best ways to make it clear would be: putting paid links in a colored box, putting them in a different section of the page and putting the words "sponsored links" near them!! lol

What this means in practice is that the relevance of Google's ads in relation to the search term will increase in comparison to the relevance of the organic search results for that term. John Battelle has shown one example of this in his blog post. Over time this trend will get more pronounced. The problem for Google's competitors is that this doesn't necessarily mean that their search experience will get worse over time since their ad relevance will likely make up for any deficiencies in their organic results (at least for commercial queries – where the money is). What competitors will have to learn to exploit is Google's tendency to push users to Adwords results by making their organic results satisfactory instead of great.  

For example, consider the following search results page which my wife just got while looking for an acupuncturist in Bellevue, Washington

The interesting thing about the organic results is that it is relevant but very cluttered thus leading to the paradox of choice. On the other hand, the sponsored links give you a name and a description of the person's qualifications in their first result. Which result do you think my wife clicked?

Now why do you think Google ended up going with this format for commercial/business-based search results?  The average Web user is a satisficer and will look for the clearest and simplest result which is in the ads. However the geeks and power users (i.e. people who don't click on ads) are often maximizers when it comes to search results and are thus served by the organic results.

The question for you is whether you'd consider this a weakness or a strength of Google Search?

Now Playing: Missy Elliott - Hit Em Wit Da Hee (remix) (feat. Lil Kim & Mocha)

 

For a hot, pre-IPO startup I'm surprised that Facebook seems to be going through an exodus of co-founders (how many do they have?) and well-respected key employees. Just this year alone we've seen the following exits

• Adam D'Angelo (former CTO and co-founder)
• Matt Cohler (former VP of Product Management and one of the first five emploees of the company)
• Ben Ling (former Director of Platform Product Marketing and a high profile Google hire)
• Jeff Hammerbacher (former lead of Facebook's "Data" team)
• Dustin Moskovitz (co-founder)

If you combine the list above with the New York Times article, The Facebooker Who Friended Obama which states

Mr. [Chris] Hughes, 24, was one of four founders of Facebook. In early 2007, he left the company to work in Chicago on Senator Obama’s new-media campaign. Leaving behind his company at such a critical time would appear to require some cognitive dissonance: political campaigns, after all, are built on handshakes and persuasion, not computer servers, and Mr. Hughes has watched, sometimes ruefully, as Facebook has marketed new products that he helped develop.

That's three departures from people named as co-founders of Facebook. Of course, it isn't unusual for co-founders to leave a business they helped start even if the business is on the path to being super successful (see Paul Allen) but it is still an interesting trend nonetheless.

Now Playing: T.I. - My Life, Your Entertainment (Ft. Usher)

 

A few weeks ago Google released the beta of Google Chrome, a new Web browser based on WebKit. Since then there has been a lot of interesting hype and backlash against the hype about Chrome. Two great examples of the hype and the corresponding backlash are Mike Arrington's Meet Chrome, Google’s Windows Killer and Ted Dziuba's article Chrome-fed Googasm bares tech pundit futility in response.

The best way to think about Google Chrome is to understand how Google thinks about the Web. Nick Carr has a post entitled The Omnigoogle which does a great job of capturing a sentiment I've seen expressed by every Google employee I've ever talked to from senior people like Sergey Brin and Vint Cerf to front line folks Dewitt Clinton and Kevin Marks. Nick Carr writes

But while Google is an unusual company in many ways, when you boil down its business strategy, you find that it’s not quite as mysterious as it seems. The way Google makes money is straightforward: It brokers and publishes advertisements through digital media. More than 99 percent of its sales have come from the fees it charges advertisers for using its network to get their messages out on the Internet.

Google’s protean appearance is not a reflection of its core business. Rather, it stems from the vast number of complements to its core business. Complements are, to put it simply, any products or services that tend be consumed together. Think hot dogs and mustard, or houses and mortgages. For Google, literally everything that happens on the Internet is a complement to its main business. The more things that people and companies do online, the more ads they see and the more money Google makes. In addition, as Internet activity increases, Google collects more data on consumers’ needs and behavior and can tailor its ads more precisely, strengthening its competitive advantage and further increasing its income. As more and more products and services are delivered digitally over computer networks — entertainment, news, software programs, financial transactions — Google’s range of complements expands into ever more industry sectors. That's why cute little Google has morphed into The Omnigoogle.

Because the sales of complementary products rise in tandem, a company has a strong strategic interest in reducing the cost and expanding the availability of the complements to its core product. It’s not too much of an exaggeration to say that a company would like all complements to be given away. If hot dogs became freebies, mustard sales would skyrocket. It’s this natural drive to reduce the cost of complements that, more than anything else, explains Google’s strategy.

This boils down to the corporate ideology that "anything that is good for the Web is good for Google". This means Google is in favor of anything that increases the breadth of the Web which explains why it is investing in O3b networks in an effort intended to bring the Web to 3 billion people in emerging markets. The more people there are using the Web, the more people there are viewing ads on Google's services and on pages of sites that use AdSense and DoubleClick ads. This also means that Google is in favor of moving as much media consumption as possible to the Web. This explains why purchasing YouTube was so important. In addition to purchasing the number one video site on the Web, Google also ensured that it would be on the front line of defending video on the Web given that YouTube was in the cross hairs of various corporate content owners. This focus on expanding the breadth of the Web also explains why they have purchased startups like Zenter, Upstartle and 2Web Technologies to create a Google office suite in an attempt to unseat the current breed of desktop based office productivity software. It explains why they created Gmail as a way to make Web-based email as satisfying or even more satisfying than desktop mail experiences especially when compared to other Webmail offerings at the time. This ideology also explains why the company invests in Android and so on..

The media has tried to make it seem like Google spits out a bunch of random, unfocused projects without much thought besides "shipping something cool". However this is far from the case. Google is the most successful company on the Web and it believes that its fortunes are directly tied to the increased usage and evolution of the Web. This means Google has a strong incentive to improve the capabilities of the Web as a delivery vehicle for user experiences. Google had telegraphed their intent to take a more direct role in the evolution of Web technologies in a few ways. For one, the company hired Ian Hickson who had been rallying browser vendors to start improving Web technologies like HTML via the Web Hypertext Applications Technology Working Group (WHAT WG). His success in these efforts since joining Google has led to HTML 5 becoming an official W3C effort. Secondly, Google also heavily supported Firefox both by hiring developers who worked on Firefox full time and via a search affiliate program that brings in millions for the Mozilla corporation [Ed note – Google has a similar deal with Opera]. However the relationship with Firefox clearly was not evolving the Web at a pace that Google found satisfactory as evidenced by the creation of Google Gears a product which Google evangelists have positioned as a bleeding edge HTML 5 implementation even though it implements capabilities not mentioned in HTML 5. 

However even with having a seat at the table in defining HTML 5 and being a significant sponsor of the second most popular Web browser, Google still did not have a direct way to push the evolution of the Web directly to users. They were still dependent on the pace of innovation of incumbent browser vendors or figuring out how to distribute a browser plug-in by convincing companies like MySpace to take a dependency on it. This was clearly an uphill battle. Thus creating their own Web browser was inevitable.

So why is this significant? It isn't because "Google Chrome is going to replace Windows" or some other such silliness. As it stands now, Google Chrome is a Windows based application whose most interesting features exist in other browsers. A Web browser cannot replace an operating system any more than an automobile can replace an Interstate highway. The significant end user innovation in Google Chrome is that it is bundled with Google Gears. This means that Google Chrome has a mechanism for delivering richer experiences to end users out of the box. Google can now use this as a carrot and a stick approach to convincing browser vendors to do what it wants. Google can make its sites work better together with Chrome + Gears (e.g. YouTube Uploader using Gears) which could lead to lost browser market share for competing browser vendors if this becomes a widespread practice among Google's offerings. Even if Google never does this, the implied threat is now out there.

Chrome will likely force Google's competitors to up their game with regards to adopting newer Web standards and features just to stay competitive. This is similar to what Google did with online mapping and Web mail, and what the Opera browser has been doing by pioneering features like "pr0n mode" and tabbed browsing. So even if Google loses because Chrome doesn't get massively popular, Google still wins because the user experience for browsing the Web has been improved.  And at the end of the day, if more people are using the Web because the user experience is better across the board that's just fine for Google. The same way the fact that all online mapping experiences and Web mail experiences have improved across the board is also good for Google.

Now Playing: Metallica - The Judas Kiss

 

Update: A blog post on the official Google blog entitled A fresh take on the browser states that the comic book went out early. It was supposed to show up tomorrow in tandem with the launch of the beta of Google Chrome which will be available in over 100 countries.

Phillipp Lessen was sent an announcement in the form of a comic book which gives the details on an upcoming Open Source browser project from Google. He gives the details and links to scanned images of the comic in his post Google Chrome, Google’s Browser Project. His site seems to have been running slow so he took down links to the scans. I managed to grab them and have uploaded them to a folder on Windows Live SkyDrive. The link is embedded below and the comic can be accessed directly from here.

The key features of Chrome according to the comic are

• Based on WebKit
• Each browser tab gets it's own process and Javascript execution is actually multithreaded instead of single threaded as it is in most modern browsers. This actually solves lots of memory problems with modern browsers since it reduces memory fragmentation and mitigates the impact of memory leaks since all memory is reclaimed when the bad Website's browser tab is closed.
• Task manager shows how much resources each tab is using so you can tie resource usage to individual Web pages.
• Created Javascript VM from scratch which has clever optimizations like just-in-time compilation and incremental garbage collection.
• Each tab has it's own URL box and can effectively be considered it's own browser Window.
• OmniBox URL bar is similar to AwesomeBar in Firefox 3 and the smart address bar in IE 8.
• Instead of "about:blank" the default homepage shows our nine most visited Web pages and four most used search engines
• Has an incognito mode where no browser history is saved and cookies are wiped out when the browser is closed. Some people have affectionately dubbed such features pr0n mode. Amusingly, the comic uses the same "planning a surprise birthday party" scenario that the Internet Explorer team used when describing a similar feature in IE 8. 
• Pop ups are not modal and scoped to the tab which they were spawned from. However they can be promoted to becoming their own tab.
• There is a "streamlined" mode where the URL box and browser toolbar are hidden so only the Web page is visible.
• Web pages are sandboxed so that if the user hits a malware site it cannot access the user's computer and perform drive-by downloads or installations.
• The sandbox model is broken by browser plugins (e.g. Flash) but this is mitigated by having the plugin execute in it's own separate process that is different from that of the browser's rendering engine.
• The browser will continuously phone home to Google to get a list of known malware sites so that the user is warned when they are visited.
• Will ship with Google Gears built-in

My initial thoughts on this are that this is a smart move on Google's part. Google depends on the usage and advancement of the Web for its success. However how quickly the Web is advanced is in the hands of browser vendors which probably doesn't sit well with them which is why they created Gears in the first place and hired the guy driving HTML 5. Chrome now gives them an even larger say in which way the Web goes.

As for their relationship with Firefox, it may be mutually beneficial financially since Mozilla gets paid while Google gets to be the search default in a popular browser but it doesn't mean Google can dictate their technical direction. With Chrome, has a way to force browser vendors to move the Web forward on their terms even if it is just by getting users to demand the features that exist in Chrome.

PS: Am I the only one that thinks that Google is beginning to fight too many wars on too many fronts. Android (Apple), OpenSocial (Facebook), Knol (Wikipedia), Lively (IMVU/SecondLife), Chrome (IE/Firefox) and that's just in the past year.

Now Playing: Boys Like Girls - Thunder

 

According to Werner Vogels's blog post entitled Amazon EBS - Elastic Block Store has launched, it seems that my friends at Amazon have plugged a gaping hole in their cloud computing platform story. Werner writes

Back in the days when we made the architectural decision to virtualize the internal Amazon infrastructure one of the first steps we took was a deep analysis of the way that storage was used by the internal Amazon services. We had to make sure that the infrastructure storage solutions we were going to develop would be highly effective for developers by addressing the most common patterns first. That analysis led us to three top patterns:

1. Key-Value storage. The majority of the Amazon storage patterns were based on primary key access leading to single value or object. This pattern led to the development of Amazon S3.
2. Simple Structured Data storage. A second large category of storage patterns were satisfied by access to simple query interface into structured datasets. Fast indexing allows high-speed lookups over large dataset. This pattern led to the development of Amazon SimpleDB. A common pattern we see is that secondary keys to objects stored in Amazon S3 are stored in SimpleDB, where lookups result in sets of S3 (primary) keys.
3. Block storage. The remaining bucket holds a variety of storage patterns ranging special file systems such as ZFS to applications managing their own block storage (e.g. cache servers) to relational databases. This category is served by Amazon EBS which provides the fundamental building block for implementing a variety of storage patterns.

What I like about Werner's post is that it shows that Amazon had a clear vision and strategy around providing hosted cloud services and has been steadily executing on it.

S3 handled what I've typically heard described as "blob storage". A typical Web application typically has media files and other resources (images, CSS stylesheets, scripts, video files, etc) that is simply accessed by name/path. However a lot of these resources also have metadata (e.g. a video file on YouTube has metadata about it's rating, who uploaded it, number of views, etc) which need to be stored as well. This need for queryable, schematized storage is where SimpleDB comes in. EC2 provides a virtual server that can be used for computation complete with a local file system instance which isn't persistent if the virtual server goes down for any reason. With SimpleDB and S3 you have the building blocks to build a large class of "Web 2.0" style applications when you throw in the computational capabilities provided by EC2.

However neither S3 nor SimpleDB provides a solution for a developer who simply wants the typical LAMP or WISC developer experience of building a database driven Web application or for applications that may have custom storage needs that don't fit neatly into the buckets of blob storage or schematized storage. Without access to a persistent filesystem, developers on Amazon's cloud computing platform have had to come up with sophisticated solutions involving backing data up manually from EC2 to S3 to get the desired experience.

EBS is the final piece in the puzzle that had prevented Amazon's cloud computing platform from being comparable to traditional hosting solutions. With EBS Amazon is now superior to most traditional hosting solutions from a developer usability perspective as well as cost. Google App Engine now looks like a plaything in comparison. In fact, you could build GAE on top of Amazon's cloud computing platform now that the EBS has solved persistent custom storage problem. It will be interesting to see if higher level cloud computing platforms such as App Engine start getting built on top of Amazon's cloud computing platform. Simply porting GAE wholesale would be an interesting academic exercise and a fun hacking project. 

Now Playing: T.I. - Whatever You Like

 

It is a fairly well known fact in the business community that the majority of mergers and acquisitions are a failure when it comes to increasing shareholder value, benefiting customers or any of the other metrics that are used to judge the "success" of an acquisition. Whenever you read a news story about some startup being acquired or two large companies merging, there is a greater than 50% chance that the resulting product or company will be of less value to customers and shareholders than if the deal had never happened.

When it comes to software company acquisitions, there are additional factors working against success that go beyond the typical laundry list of reasons that are given for why M&As usually result in failure. With technology company acquisitions not only are there a minefield of people and financial issues that have to be dealt with, there is also the real problem of what to do about technology mismatch that often exists across different companies.

Whenever a large software company acquires a startup, the first order of business is often an attempt to move the startup's application onto the larger company's technology infrastructure so that it can get benefits of "economies of scale" or some other buzzword that is typically a euphemism for "we bought you so now you're our bitches" that is not grounded in business realities. This often requires application rewrites that have the unfortunate consequence of causing the shipped application to stagnate as all efforts are poured into recreating the same application using a different technology. In addition, the founders of the startup typically get frustrated with what they [rightfully] deem as a pointless exercise and eventually move on to greener pastures. There are a number of examples of this that have occurred in the "Web 2.0" space as shown below

From Fred Wilson's post We Need A New Path To Liquidity

So if you can't take a company public, how do you get out? M&A has been the primary answer in the web/tech sector for the past eight years. And it's been a great period to sell companies. We've sold three in the past couple years out of our Union Square Ventures portfolio, delicious, FeedBurner, and TACODA, to Yahoo!, Google, and AOL, respectively. Were we happy to take their money? Yes. Were we happy with the outcome? Yes. Were they good buys for their new owners? On the face of it, yes.

But if you look deeper, I wonder. Delicious grew nicely for a while under Yahoo!'s ownership but recently the user base has fallen off pretty dramatically. I double checked this chart in compete and alexa and they all show the dropoff.

Well, what about FeedBurner? Clearly Google has done a good job with that acquisition. Well I am not sure. I don't see any integration between Adwords and FeedBurner yet. I can't buy FeedBurner inventory through Google's text ad interface. I honestly don't see any additional money flowing to me, the publisher of the feed, since the Google acquisition. There's no way to know what the rate of signup by publishers has been since the acquisition, but I wonder if it's increased much.

And TACODA? I know that TACODA had an incredible fourth quarter post the acquisition by AOL, blowing way past the numbers we were projecting in our annual budget. But in the first quarter, AOL fired Curt Viebranz, TACODA's CEO, and many of the top members of the TACODA team are now gone from AOL. Another acquisition messed up.

But who am I to complain? We got paid right? So sit down and shut up.

From Joshua Schachter's comment on "How Yahoo dropped the del.icio.us ball with a pointless 3 year rewrite (from mod_perl to PHP)"

The writer is accidentally correct - we were told that it had to be in PHP to get ops support. Curiously the PHP part didn't take that much time - the majority of the "business logic" is in C++ which took forever and ever to write. I think the open question now is whether the remaining team will be able to innovate or be stuck in complicated codebase hell. 

From Dennis Howlett's article Google Sites - spoiled by usability issues

After 16 months at Google developer’s hands, the outcome is substandard. This is such a pity. In its JotSpot incarnation, it was far from perfect but that didn’t matter because JotSpot was shedding light on a new way of collaborating. Since passing into Google’s hands, the guts have been ripped out and then re-assembled with as much Google ’stuff’ as they could cram in but rushed to completion.

At the very least, Google should get rid of the gadgets addition facility and rework it. Otherwise, I sense the SMBs at which it is aimed will find the service a turn off.

Google has a real chance to differentiate itself from Microsoft - which is clearly what it wants to do, while adding significant numbers of users to its Google Apps offering. It won’t do it this way because despite all the gripes around Microsoft products, the fact is Microsoft offers a more polished experience. Until Google truly understands this, it will find it difficult to adequately compete. In the meantime, offerings like Wetpaint and Ning have little to fear.

From Ryan Paul's article Jaiku users flee to Twitter as a result of Google's neglect

Unfortunately, Google has allowed Jaiku to languish and is now suffering a backlash from frustrated users who are beginning to mass-migrate to Twitter, a competing microblogging service. Jaiku's external feed servers, which are used by third-party Jaiku client applications, have been down frequently during the past week, often returning 504 gateway errors or nothing at all. During the brief stints when the feed servers are operational, they have been extremely slow and often out of sync with the actual content—typically lagging by between four and 13 hours. These problems have been noted by many users and several third-party Jaiku client application developers who discussed the problem with Ars. Users also complain that Jaiku's IM bots and the third-party Jaiku Facebook interface are exhibiting problems as well.

When Google announced the acquisition, the company promised new features within a few months, but we have seen no evidence of any development at all. Registration is still closed and new users can only join the site by receiving an invitation from Google. The Jaiku developers have been completely and totally silent since the announcement of the takeover, and the official Jaiku blog—which used to have several messages a month—has had no new posts at all. The Jaiku Team feed has also not received any posts from Jaiku developers since the acquisition.

The stories are the same except that some of the names are different. A startup gets bought and immediately stops innovating because all their development time is being spent porting the code to a new platform. During that time newer, more agile competitors show up and eat their lunch. Why I find this to be such a conundrum is that when you buy a technology startup, you are primarily buying three things

• customers
• employees
• technology

However the standard operating procedure during Web software acquisitions is to discard the technology and consequentially tick off the employees who made the product a success in the first place thus creating an exodus. The application rewrite plus employee exodus leads to product stagnation which eventually leads to lots of pissed off customers. Thus the entire value from the acquisition is effectively thrown away.

This is the default situation when it comes to acquisitions in the software industry. For every successful acquisition like Google + YouTube there are two or three that are more like Google + Dodgeball. So if there is a startup whose product you love that you hear is being acquired by a one of the large Web companies, be happy for the founders and be sad for yourself because the product you love is likely going to become a neglected bride.

Disclaimer: I'm an employee of a large software company that has displayed similarly counterproductive tactics when acquiring startups. Although no examples are provided in the post above, I'm sure some can be found from it's history of acquisitions.

Now Playing: Dire Straits - Money For Nothing

 

Earlier today I read two posts that were practically mirror opposites of each other. The first was Paul Graham's essay, The Pooled-Risk Company Management Company, where he makes a case against founding a company that you intend to run as a successful business over the long term and instead building a company you that you can sell to a public company so you can move on and enjoy your money. His argument is excerpted below

At this year's startup school, David Heinemeier Hansson gave a talk in which he suggested that startup founders should do things the old fashioned way. Instead of hoping to get rich by building a valuable company and then selling stock in a "liquidity event," founders should start companies that make money and live off the revenues. Sounds like a good plan. Let's think about the optimal way to do this.

One disadvantage of living off the revenues of your company is that you have to keep running it. And as anyone who runs their own business can tell you, that requires your complete attention. You can't just start a business and check out once things are going well, or they stop going well surprisingly fast.

The main economic motives of startup founders seem to be freedom and security. They want enough money that (a) they don't have to worry about running out of money and (b) they can spend their time how they want. Running your own business offers neither. You certainly don't have freedom: no boss is so demanding. Nor do you have security, because if you stop paying attention to the company, its revenues go away, and with them your income.

The best case, for most people, would be if you could hire someone to manage the company for you once you'd grown it to a certain size.
...
If such pooled-risk company management companies existed, signing up with one would seem the ideal plan for most people following the route David advocated. Good news: they do exist. What I've just described is an acquisition by a public company.

Austin Wiltshire has a counterpoint to Paul Graham's article entitled New hire cannon fodder where he decries the practice of "exploiting" junior developers so that a couple of fat cats with money can get even richer. Parts of Austin's counter argument are excerpted below

Why these large firms, and now even places like YCombinator continually think the best way to move forward in software is to hire as many gullible young naive programmers as possible and work them to death is beyond me.  It’s pretty well known that 80 hour work weeks and inexperience is a guarantee to continually make the same damn mistakes over and over again.  It’s also an open question as to why new hires let these companies take advantage of them so badly.  Paul Graham had a start up, he begged for angel investing, and his life should show you - what does he do now?  Well he learned from his experience that designing and building is for chumps, to make the big bucks and sit on your ass you become an angel investor.

Kids will work for pennies.  You can continue to fill their heads with dreams of having the next big idea, even though they are carrying all the risk for you.  Junior developers, whether entrepreneurs or otherwise, are being asked to give up their 20’s, probably the best, most energetic years of their lives, to have a chance at making a dent in someone else’s bottom line.  (Make note, the one exception here I’ve seen is 37 Signals )
…
But have we never stopped to think who truly is benefiting from all these hours?  Do we get paid more?  No.  In fact, because many of us are salaried, we’re effectively paid less.  Are we compensated with faster promotions?  Possibly - but don’t forget about that silicon ceiling.  The only person who knows how many hours you’re putting in is probably just the guy above you - but he makes sure to show just how productive his department is (via your hard work) to everyone.  He will always get the spoils.  Who will end up really getting the spoils out of any of YCombinator’s work?  Paul Graham.

Both arguments have their merits and there are also parts I disagree with on both sides. Austin is right that YCombinator takes advantage of the naivety of youth. However when you are in your 20s with no serious attachments (like a mortgage, a family or even a sick relative) it doesn't sound like a bad idea to make hay while the sun shines. If you can sacrifice some time in your youth for a chance at a better life for yourself and your future spouse/kids/girlfriend/family/etc in a few years, is it wrong to treat that as an opportunity? Especially if you'll be working in an energetic environment surrounded by likeminded souls all believing that you are building cool stuff? Additionally, if the startup doesn't work out [which it most likely won't] the experience will still turn out to be useful when you decide to get a regular job at some BigCo even if it is just realizing how good you have it to no longer have to work 80 hour weeks while eating Top Ramen for breakfast and dinner any more.

From that perspective I don't think Austin is right to completely rail against the startup lifestyle. However I totally agree with the general theme of Austin's post that working ridiculous hours is dumb. It should be common knowledge that sleep deprivation impairs brain function and may even lead to psychiatric disorders. The code you check-in during your 14th hour at your desk will not be as good as what you checked in during your 4th. If you really have to work that much, work on the weekends instead of spending over 12 hours sitting in front of your IDE. Even then, busting your butt to that extent only makes sense if you not only get to share the risks of failure but also the rewards of success as well. This means you better be a co-founder or someone with equity and not just some poor sap on salary.

My issue with Paul Graham's essay and the investment style of YCombinator is that I it sells startup founders short. Paul recently wrote an essay entitled Cities and Ambition where he had this beautiful quote about the kind of "peer pressure" the Silicon Valley area exerts on startup founders

When you ask what message a city sends, you sometimes get surprising answers. As much as they respect brains in Silicon Valley, the message the Valley sends is: you should be more powerful.

That's not quite the same message New York sends. Power matters in New York too of course, but New York is pretty impressed by a billion dollars even if you merely inherited it. In Silicon Valley no one would care except a few real estate agents. What matters in Silicon Valley is how much effect you have on the world. The reason people there care about Larry and Sergey is not their wealth but the fact that they control Google, which affects practically everyone.

Read the above quote again and let its message sink in. The great thing about software is how you can literally take nothing (i.e. a blank computer screen) and build something that changes the world. Bill and Paul did it with Microsoft. Larry and Sergey have done it with Google. Jerry and David did it with Yahoo!, and some might say Mark Zuckerberg is doing it with Facebook.

Are any of those companies YCombinator-style, built-to-flip companies? Nope.

I strongly believe in the idea behind the mantra "Change the World or Go Home". Unlike anything that has come before it, the combination of software and the World Wide Web has the potential to connect people and empower them in more ways than humanity has never seen. And it is possible to become immensely rich while moving humanity forward with the software that you create.

So if you have decided to found a startup, why decide to spend your youth building some "me too" application that conforms to all the current "Web 2.0" fads in the desperate hope that you can convince some BigCo to buy you out? That sounds like such a waste. Change the world or go home.

Now Playing: Wu-Tang Clan - Triumph

 

For several months Nick Carr has pointed out that Wikipedia ranks highly in the search results for a number of common topics in Google's search engine. In his post entitled Googlepedia Nick Carr speculated on why Google would see this trend as a threat in a paragraph which is excerpted below

I'm guessing that serving as the front door for a vast ad-less info-moshpit outfitted with open source search tools is not exactly the future that Google has in mind for itself. Enter Knol.

Clearly Nick Carr wasn't the only one that realized that Google was slowly turning into a Wikipedia redirector. Google wants to be the #1 source for information or at least be serving ads on the #1 sites on the Internet in specific area. Wikipedia was slowly eroding the company's effectivenes at achieving both goals. So it is unsurprising that Google has launched Knol and is trying to entice authors away from Wikipedia by offering them a chance to get paid.

What is surprising is that Google is tipping it's search results to favor Knol. Or at least that is the conclusion of several search engine optimization (SEO) experts and also jibes with my experiences.

Danny Sullivan wrote in his post The Day After: Looking At How Well Knol Pages Rank On Google that

We've been assured that just because content sits on Google's Knol site, it won't gain any ranking authority from being part of the Knol domain. OK, so a day after Knol has launched, how's that holding up? I found 1/3 of the pages listed on the Knol home page that I tested ranked in the top results.
…
I was surprised to see a post covering how Knol's How to Backpack was already hitting the number three spot on Google. Really? I mean, how many links could this page have gotten already? As it turns out, quite a few. And more important, it's featured on the Knol home page, which itself is probably one of the most important links. While Knol uses nofollow on individual knols to prevent link credit from flowing out, it's not used on the home page -- so home page credit can flow to individual knols featured on it.
…
here's a test knol I made yesterday -- Firefox Plugins For SEO & SEM -- which ranks 28 for firefox plugins for seo. I never linked to it from my article about knol. I don't think it made the Knol home page. I can see only three links pointing at it, and only one of those links uses anchor text relevant to what the page is ranking for. And it's in the top 30 results?

Look, I know that being ranked 28 is pretty much near invisible in terms of traffic you'll get from search engines. But then again, to go from nowhere to the 28th top page in Google out of 755,000 matches? I'm sorry -- don't tell me that being in Knol doesn't give your page some authority.

Aaron Wall noticed something even more insidious in his post entitled Google Knol - Google's Latest Attack on Copyright where he notices that if Google notices duplicate content then it favors the content on Knol over a site that has existed for years and has decent PageRank. His post is excerpted below

Another Knol Test

Maybe we are being a bit biased and/or are rushing to judgement? Maybe a more scientific effort would compare how Knol content ranks to other content when it is essentially duplicate content? I did not want to mention that I was testing that when I created my SEO Basics Knol, but the content was essentially a duplicate of my Work.com Guide to Learning SEO (that was also syndicated to Business.com). Even Google shows this directly on the Knol page

Google Knows its Duplicate Content

Is Google the Most Authoritative Publisher?

Given that Google knows that Business.com is a many year old high authority directory and that the Business.com page with my content on it is a PageRank 5, which does Google prefer to rank? Searching for a string of text on the page I found that the Knol page ranks in the search results.

If I override some of Google's duplicate content filters (by adding &filter=0 to the search string) then I see that 2 copies of the Knol page outrank the Business.com page that was filtered out earlier.

Interesting.

Following Danny's example, I also tried running some searches for terms that appear on the Knol homepage and seeing how they did in Google's search. Here's the screenshot of the results of searching for "buttermilk pancakes"

Not bad for a page that has existed on the Web for less than two weeks.

Google is clearly favoring Knol content over content from older, more highly linked sites on the Web. I won't bother with the question of whether Google is doing this on purpose or whether this is some innocent mistake. The important question is "What are they going to do about it now that we've found out?"

Now Playing: One Republic - Stop and Stare
 

I've been using the redesigned Facebook profile and homepage for the past few days and thought it would be useful to write up my impressions on the changes. Facebook is now the the world's most popular social networking site and one of the ways they've gotten there is by being very focused on listening to their users and improving their user experienced based on this feedback. Below are screenshots of the old and new versions of the pages and a discussion of which elements are changed and the user scenarios the changes are meant to improve.

Homepage Redesign

OLD HOME PAGE:

NEW HOME PAGE:

The key changes and their likely justifications are as follows

• Entry points for creating content are now at the top of the news feed. One of the key features driving user engagement on Facebook is the News Feed. This lets a user know what is going on with their social network as soon as they logon to the site. In a typical example of network effects at work, one person creates some content by uploading a photo or sharing a link and hundreds of people on their friend list benefit by having content to view in their News Feed. If any of the friends responds to the content this again benefits hundreds of people and so on.  The problem with the old home page was that a user sees their friends uploading photos and sharing links and may want to do so as well but there is no easy way for her to figure out how to do the same thing without having to go two or three clicks away from the home page. The entry points at the top of the feed will encourage more "impulse" content creation.

• Left sidebar is gone. There were three groups of items in the left nav; a search box, the list of a user's most frequently accessed applications and an advertisement. The key problem is that the ad is in a bottom corner of the feed. This makes it easy for users to mentally segregate that part of the screen from their vision and either never look there or completely ignore it. Removing that visual ghetto and moving ads to being inline with the feed makes it more likely that users will look at the ad. Ah, but now you need more room to show the ad (all the space isn't needed for news feed stories). So the other elements of the left nave are moved, the search box to the header and the list of most accessed applications to the sidebar on the right. Now you have enough room to stretch out the News Feed's visible area and advertisers can reuse their horizontal banner ads on Facebook even though this makes the existing feed content now look awkward. This is one place where monetization trumped usability.

• Comments now shown inline for News Feed items with comments (not visible in screen shot). This may be the feature that made Mike Arrington decide to call the new redesign the FriendFeedization of Facebook. Sites like FriendFeed have proven that showing the comments on an item in the feed inline gives users more content to view in their feeds and increases the likelihood of engagement since the user may want to join the conversation.

Profile Redesign

OLD PROFILE:

NEW PROFILE:

The key changes and their likely justifications are as follows

• The profile now has tabbed model for navigation. This is a massive improvement for a number of reasons. The most important one is that in the old profile, there is a lot of content below the fold. My old profile page is EIGHT pages when printed as opposed to TWO pages when the new profile page is printed. Moving to a tabbed model (i) improves page load times and (ii) increases number of page views and hence ad impressions.

• The Mini-Feed and the Wall have been merged. The intent here is to give more visibility to the Wall which in the old model was below the fold. The "guest book" or wall is an important part of the interaction model in social networking sites (see danah boyd's Friendster lost steam. Is MySpace just a fad? essay) and Facebook was de-emphasizing theirs in the old model.

• Entry points for creating content are at the top of the profile page. Done for the same reason as on the Home page. You want to give users lots of entry points for adding content to the site so that they can kick off network effects by generating content which in turn generates tasty page views.

• Left sidebar is gone. Again the left sidebar is gone and the advertisement is moved closer to the content, and away from the visual ghetto that is the bottom left of the screen. Search box and most accessed applications are now in the header as well. The intent here is also to improve the likelihood that users will view and react to the ads.

Now Playing: Da Back Wudz - I Don't Like The Look Of It (Oompa)

 

For the past few years, the technology press has been eulogizing desktop and server-based software while proclaiming that the era of Software as a Service (SaaS) is now upon us. According to the lessons of the Innovator's Dilemma the cheaper and more flexible SaaS solutions will eventually replace traditional installed software and the current crop of software vendors will turn out to be dinosaurs in a world that belongs to the warm blooded mammals who have conquered cloud based services.

So it seems the answer is obvious, software vendors should rush to provide Web-based services and extricate themselves from their "legacy" shrinkwrapped software business before it is too late. What could possibly go wrong with this plan? 

Sarah Lacy wrote an informative article for Business Week about the problems facing software vendors who have rushed into the world of SaaS. The Business Week article is entitled On-Demand Computing: A Brutal Slog and contains the following excerpt

On-demand represented a welcome break from the traditional way of doing things in the 1990s, when swaggering, elephant hunter-style salesmen would drive up in their gleaming BMWs to close massive orders in the waning days of the quarter. It was a time when representatives of Oracle (ORCL), Siebel, Sybase (SY), PeopleSoft, BEA Systems, or SAP (SAP) would extol the latest enterprise software revolution, be it for management of inventory, supply chain, customer relationships, or some other area of business. Then there were the billions of dollars spent on consultants to make it all work together—you couldn't just rip everything out and start over if it didn't. There was too much invested already, and chances are the alternatives weren't much better.

Funny thing about the Web, though. It's just as good at displacing revenue as it is in generating sources of it. Just ask the music industry or, ahem, print media. Think Robin Hood, taking riches from the elite and distributing them to everyone else, including the customers who get to keep more of their money and the upstarts that can more easily build competing alternatives.

But are these upstarts viable? On-demand software has turned out to be a brutal slog. Software sold "as a service" over the Web doesn't sell itself, even when it's cheaper and actually works. Each sale closed by these new Web-based software companies has a much smaller price tag. And vendors are continually tweaking their software, fixing bugs, and pushing out incremental improvements. Great news for the user, but the software makers miss out on the once-lucrative massive upgrade every few years and seemingly endless maintenance fees for supporting old versions of the software.
…
Nowhere was this more clear than on Oracle's most recent earnings call (BusinessWeek.com, 6/26/08). Why isn't Oracle a bigger player in on-demand software? It doesn't want to be, Ellison told the analysts and investors. "We've been in this business 10 years, and we've only now turned a profit," he said. "The last thing we want to do is have a very large business that's not profitable and drags our margins down." No, Ellison would rather enjoy the bounty of an acquisition spree that handed Oracle a bevy of software companies, hordes of customers, and associated maintenance fees that trickle straight to the bottom line.
…
SAP isn't having much more luck with Business by Design, its foray into the on-demand world, I'm told. SAP said for years it would never get into the on-demand game. Then when it sensed a potential threat from NetSuite, SAP decided to embrace on-demand. Results have been less than stellar so far. "SAP thought customers would go to a Web site, configure it themselves, and found the first hundred or so implementations required a lot of time and a lot of tremendous costs," Richardson says. "Small businesses are calling for support, calling SAP because they don't have IT departments. SAP is spending a lot of resources to configure and troubleshoot the problem."

In some ways, SaaS vendors have been misled by the consumer Web and have failed to realize that they still need to spend money on sales and support when servicing business customers. Just because Google doesn't advertise it's search features and Yahoo! Mail doesn't seem to have a huge support staff that hand holds customers as it uses their product doesn't mean that SaaS vendors can expect to cut their sales and support calls. The dynamics of running a free, advertising based service aimed at end users is completely different from running a service where you expect to charge business tens of thousands to hundreds of thousands to use your product.

In traditional business software development you have three major cycles with their own attendant costs; you have to write the software, you have to market the software and then you have to support the software. Becoming a SaaS vendor does not eliminate any of these costs. Instead it adds new costs and complexities such as managing data centers and worrying about hackers. In addition, thanks to free advertising based consumer services and the fact that companies like Google that have subsidized their SaaS offerings using their monopoly profits in other areas, business customers expect Web-based software to be cheaper than its desktop or server-based alternatives. Talk about being stuck between a rock and a hard place as a vendor.

Finally, software vendors that have existing ecosystems of partners that benefit from supporting and enhancing their shrinkwrapped products also have to worry about where these partners fit in a SaaS world. For an example of the kinds of problems these vendors now face, below is an excerpt from a rant by Vladimer Mazek, a system administrator at ExchangeDefender, entitled Houston… we have a problem which he wrote after attending one of Microsoft's partner conferences

Lack of Partner Direction: By far the biggest disappointment of the show. All of Microsoft’s executives failed to clearly communicate the partnership benefits. That is why partners pack the keynotes, to find a way to partner up with Microsoft. If you want to gloat about how fabulous you are and talk about exciting commission schedules as a brand recommender and a sales agent you might want to go work for Mary Kay. This is the biggest quagmire for Microsoft – it’s competitors are more agile because they do not have to work with partners to go to market. Infrastructure solutions are easy enough to offer and both Google and Apple and Amazon are beating Microsoft to the market, with far simpler and less convoluted solutions. How can Microsoft compete with its partners in a solution ecosystem that doesn’t require partners to begin with?

This is another example of the kind of problems established software vendors will have to solve as they try to ride the Software as a Service wave instead of being flattened by it.  Truly successful SaaS vendors will eventually have to deliver platforms that can sustain a healthy partner ecosystems to succeed in the long term. We have seen this in the consumer space with the Facebook platform and in the enterprise space with SalesForce.com's AppExchange. Here is one area where the upstarts that don't have a preexisting shrinkwrap software businesses can turn a disadvantage (lack of an established partner ecosystem) into an advantage since it is easier to start from scratch than to retool.

The bottom line is that creating a Web-based version of a popular desktop or server-based product is just part of the battle if you plan to play in the enterprise space. You will have to deal with the sales and support that go with selling to businesses as well as all the other headaches of shipping "cloud based services" which don't exist in the shrinkwrap software world. After you get that figured out, you will want to consider how you can leverage various ISVs and startups to enhance the stickiness of your service and turn it into a platform before one of your competitor's does. 

I suspect we still have a few years before any of the above happens. In the meantime we will see lots more software companies complaining about the paradox of embracing the Web when it clearly cannibalizes their other revenue streams and is less lucrative than what they've been accustomed to seeing. Interesting times indeed.

Now Playing: Flobots - Handlebars

 

About a week ago, the Facebook Data team quietly released the Cassandra Project on Google Code. The Cassandra project has been described as a cross between Google's BigTable and Amazon's Dynamo storage systems. An overview of the project is available in the SIGMOD presentation on Cassandra available at SlideShare. A summary of the salient aspects of the project follows.

The problem Cassandra is aimed at solving is one that plagues social networking sites or any other service that has lots of relationships between users and their data. In such services, data often needs to be denormalized to prevent having to do lots of joins when performing queries. However this means the system needs to deal with the increased write traffic due to denormalization. At this point if you're using a relational database, you realize you're pretty much breaking every major rule of relational database design. Google tackled this problem by coming up with BigTable. Facebook has followed their lead by developing Cassandra which they admit is inspired by BigTable. 

The Cassandra data model is fairly straightforward. The entire system is a giant table with lots of rows. Each row is identified by a unique key. Each row has a column family, which can be thought of as the schema for the row. A column family can contain thousands of columns which are a tuple of {name, value, timestamp} and/or super columns which are a tuple of {name, column+} where column+ means one or more columns. This is very similar to the data model behind Google's BigTable.

As I mentioned earlier, denormalized data means you have to be able to handle a lot more writes than you would if storing data in a normalized relational database. Cassandra has several optimizations to make writes cheaper. When a write operation occurs, it doesn't immediately cause a write to the disk. Instead the record is updated in memory and the write operation is added to the commit log. Periodically the list of pending writes is processed and write operations are flushed to disk. As part of the flushing process the set of pending writes is analyzed and redundant writes eliminated. Additionally, the writes are sorted so that the disk is written to sequentially thus significantly improving seek time on the hard drive and reducing the impact of random writes to the system. How important is improving seek time when accessing data on a hard drive? It can make the difference between taking hours versus days to flush a hundred gigabytes of writes to a disk. Disk is the new tape.

Cassandra is described as "always writable" which means that a write operation always returns success even if it fails internally to the system. This is similar to the model exposed by Amazon's Dynamo which has an eventual consistency model.  From what I've read, it isn't clear how writes operations that occur during an internal failure are reconciled and exposed to users of the system. I'm sure someone with more knowledge can chime in in the comments.

At first glance, this is a very nice addition to the world of Open Source software by the Facebook team. Kudos.

Found via James Hamilton.

PS: Is it me or is this the second significant instance of Facebook Open Sourcing a key infrastructure component "inspired" by Google internals?

Now Playing: Ray J - Gifts

 

Matt Asay of C|Net has an article entitled Facebook adopts the CPAL poison pill where he writes

Instead, by choosing CPAL, Facebook has ensured that it can be open source without anyone actually using its source. Was that the intent?

As OStatic explains, CPAL requires display of an attribution notice on derivative works. This practice, which effectively requires downstream code to carry the original developer(s)' logo, came to be known as "badgeware." It was approved by the OSI but continues to be viewed with suspicion within the open-source community.

I've written before about how most open-source licenses don't apply themselves well to the networked economy. Only the OSL, AGPL, and CPAL contemplate web-based services. It's not surprising that Facebook opted for one of these licenses, but I am surprised it chose the one least likely to lead to developers actually modifying the Facebook platform.

If the point was to protect the Facebook platform from competition (i.e., derivative works), Facebook chose a good license. If it was to encourage development, it chose the wrong license.

But if the purpose was to prevent modifications of the platform, why bother open sourcing it at all?

I've seen more than one person repeat the sentiment in the above article which leaves me completely perplexed. With fbOpen Facebook has allowed anyone who is interested to run Facebook applications and participate in what is currently the most popular & vibrant social network widget ecosystem in the world.

I can think of lots of good reasons for not wanting to adopt fbOpen. Maybe the code is in PHP and you are a Ruby On Rails shop. Or maybe it conflicts with your company's grand strategy of painting Facebook as the devil and you the heroes of openness (*cough* Google *cough*). However I can't see how requiring that you mention somewhere on your site that your social network's widget platform is powered by the Facebook developer platform is some sort of onerous POISON PILL which prevents you from using it. In the old days, companies used to charge you for the right to say your application is compatible with theirs, heck, Apple still does. So it seems pretty wacky for someone to call Facebook out for letting people use their code and encouraging them to use the Facebook brand in describing their product. Shoot!

The premise of the entire article is pretty ridiculous, it's like calling the BSD License a poison pill license because of the advertising clause. This isn't to say there aren't real issues with an advertising clause as pointed out in the GNU foundation's article The BSD License Problem. However as far as I'm aware,  adopters of fbOpen don't have to worry about being obligated to display dozens powered by X messages because every bit of code they depend on requires that it is similarly advertised. So that argument is moot in this case.

Crazy article but I've come to expect that from Matt Asay's writing.

Now Playing: Eminem & D12 - Sh*t On You

 

Recently the folks behind Twitter came clean on the architecture behind the service and it is quite clear that the entire service is being held together by chewing gum and baling wire. Only three MySQL database servers for a service that has the I/O requirements of Twitter? Consider how that compares to other Web 2.0 sites that have come clean with their database numbers; Facebook has 1800, Flickr has 166, even Wikipedia has 20. Talk about bringing a knife to a gunfight.

Given the fact that Twitter has had scaling issues for over a year it is surprising that not only has it taken so long for them to figure out that they need a re-architecture but more importantly they decided that having a developer/sys admin manage fail over and traffic spikes by hand was cheaper to the business than buying more hardware and a few weeks of coding. 

A popular social networking that focuses on features instead of performance while upstart competitors are waiting in the wings? Sounds like a familiar song doesn't it? This entire episode reminds me of a story I read in the New York Times a few years ago titled The Wallflower at the Web Party which contains the following familiar sounding excerpts

But the board also lost sight of the task at hand, according to Kent Lindstrom, an early investor in Friendster and one of its first employees. As Friendster became more popular, its overwhelmed Web site became slower. Things would become so bad that a Friendster Web page took as long as 40 seconds to download. Yet, from where Mr. Lindstrom sat, technical difficulties proved too pedestrian for a board of this pedigree. The performance problems would come up, but the board devoted most of its time to talking about potential competitors and new features, such as the possibility of adding Internet phone services, or so-called voice over Internet protocol, or VoIP, to the site.
...
In retrospect, Mr. Lindstrom said, the company needed to devote all of its resources to fixing its technological problems. But such are the appetites of companies fixated on growing into multibillion-dollar behemoths. They seek to run even before they can walk.

“Friendster was so focused on becoming the next Google,” Professor Piskorski said, “that they weren’t focused on fixing the more mundane problems standing in the way of them becoming the next Google.”
...
“We completely failed to execute,” Mr. Doerr said. “Everything boiled down to our inability to improve performance.”

People said about Friendster the same thing they say about Twitter, we travel in tribes - people won't switch to Pownce or Jaiku because all their friends use Twitter. Well Friendster thought the same thing until MySpace showed up and now we have Facebook doing the same to them.

It is a very vulnerable time for Twitter and a savvy competitor could take advantage of that by adding a few features while courting the right set of influential users to jumpstart an exodus. The folks at FriendFeed could be that competitor but I suspect they won't. The Bret & Paul have already boxed their service into being an early adopter's play thing when there's actually interesting mainstream potential for their service. They'd totally put paid to their dreams of being a household brand if they end up simply being a Twitter knock off even if they could end up outplaying Evan and Biz at the game they invented.

Now Playing: Bob Marley - Redemption Song

 

Disclaimer: This post does not reflect the opinions, thoughts, strategies or future intentions of my employer. These are solely my personal opinions. If you are seeking official position statements from Microsoft, please go here.

Recently there were three vaporware announcements by Facebook, Google and MySpace each describing a way for other web sites to integrate the user profiles and friends lists from these popular social networking sites. Given that I'm a big fan of social networking sites and interoperability between them, this seemed like an interesting set of announcements. So I decided to take a look at these announcements especially given the timing of them.  

What Do They Have in Common?

Marc Canter does a good job of describing the underlying theme behind all three announcements in his post I do not compromise where he writes

three announcements that happened within a week of each other: MySpace’s Data Availability, Facebook’s Connect and Google’s Friend Connect - ALL THREE had fundamentally the same strategy!

They’re all keeping their member’s data on their servers, while sending out tentacles to mesh in with as many outside sites as they can. These tentacles may be widgets, apps or iFrames - but its all the same strategy.

Basically all three announcements argue that instead of trying to build social networking into their services from scratch, Web sites should instead outsource their social graphs and "social features" such as user profiles, friends lists and media sharing from the large social networking sites like Facebook, MySpace and Orkut.

This isn't a new pitch, Facebook has been singing the same song since they announced the beta of the Facebook Platform in August 2006 and Google has been sending Kevin Marks to every conference they can find to give his Social Cloud presentation which makes the same pitch. The new wrinkle to this time worn tale is that Google and Facebook [along with MySpace] are no longer just pitching using REST APIs for integration but are now preaching "no coding required" integration via widgets. 

Now that we know the meat of all three announcements we can go over the little specifics that have leaked out about each forthcoming product thus far.

Facebook Connect

Dave Morin gave the first official statement about Facebook Connect news in his blog post Announcing Facebook Connect where he wrote

Trusted Authentication
Users will be able to connect their Facebook account with any partner website using a trusted authentication method. Whether at login, or anywhere else a developer would like to add social context, the user will be able to authenticate and connect their account in a trusted environment. The user will have total control of the permissions granted.

Real Identity
Facebook users represent themselves with their real names and real identities. With Facebook Connect, users can bring their real identity information with them wherever they go on the Web, including: basic profile information, profile picture, name, friends, photos, events, groups, and more.

Friends Access
Users count on Facebook to stay connected to their friends and family. With Facebook Connect, users can take their friends with them wherever they go on the Web. Developers will be able to add rich social context to their websites. Developers will even be able to dynamically show which of their Facebook friends already have accounts on their sites.

Dynamic Privacy
As a user moves around the open Web, their privacy settings will follow, ensuring that users' information and privacy rules are always up-to-date. For example, if a user changes their profile picture, or removes a friend connection, this will be automatically updated in the external website.

The key features to note are (i) a user can associate their Facebook account with their account on a 3rd party site which means  (ii) the user's profile and media shared on Facebook can now be exposed on the 3rd party site and (iii) the users friends' on Facebook who have also associated their Facebook account with their account on the 3rd party site will show up as the user's friends on the site. 

The "dynamic privacy" claim seems pretty vague if not downright empty. All that is stated above is that the user's changes on Facebook are instantly reflected on 3rd party sites. Duh. Does that need to be called out as a feature?

Google Friend Connect

On the Google Friend Connect page there is the following video

The key features to note are (i) a user can associate their Facebook account OpenID with their account on a 3rd party site which means  (ii) the user's profile and media shared on Facebook account a small set of social networking site can now be exposed on the 3rd party site and (iii) the users friends' on Facebook the small set of social network sites who have also associated their Facebook account OpenID using Google Friend Connect to connect their account on the 3rd party site will show up as the user's friends on the site (iv) the user's activities on the 3rd party site are broadcast in her friends' news feeds.

One interesting thing about Google Friend Connect's use of OpenID is that it allows me to associate multiple social network profiles to a single account which may not even be from a social networking site (e.g. using my AOL or Y! email to sign-in but associating it with my Facebook profile & friend list).

Google Friend Connect seems to be powered by Google OpenSocial which is Google's attempt to commoditize the functionality of the Facebook platform by making it easy for any social networking site to roll its own Facebook-style platform by using Google's standard set of REST APIs, Javascript libraries and/or hosting services. In the above video, it is mentioned that Web sites which adopt Google Friend Connect will not only be able to obtain user profile and friend list widgets from Google but also OpenSocial widgets written by 3rd party developers. However since Facebook announced the JavaScript Client Library for Facebook API way back in January they already have the technology in place to offer something similar to Web site owners if this capability becomes in demand.  More important will be the set of functionality that comes "out of the box" so to speak since a developer community won't form until Google Friend Connect gains traction.

By the way, it turns out that Facebook has banned Google from interacting with their user data using Google Friend Connect since it violates their terms of service. My assumption is that the problem is Google Friend Connect works by building an OpenSocial wrapper on top of the Facebook API and then exposing it to other web sites as widgets and to OpenSocial gadget developers via APIs. Thus Google is pretty much proxying the Facebook social graph to other sites and developers which takes control of safeguarding/policing access to this user data out of Facebook's hands. Not good for Facebook. 

MySpace Data Availability

The only details on the Web about MySpace's Data Availability seems to be second hand data from tech bloggers who were either strategically leaked some details/screenshots or took part in a press release conference call. The best source I found was Mike Arrington's TechCrunch post entitled MySpace Embraces DataPortability, Partners With Yahoo, Ebay And Twitter which contains the following excerpt

MySpace is announcing a broad ranging embrace of data portability standards today, along with data sharing partnerships with Yahoo, Ebay, Twitter and their own Photobucket subsidiary. The new project is being called MySpace “Data Availability” and is an example, MySpace says, of their dedication to playing nice with the rest of the Internet.

A mockup of how the data sharing will look in action with Twitter is shown above. MySpace is essentially making key user data, including (1) Publicly available basic profile information, (2) MySpace photos, (3) MySpaceTV videos, and (4) friend networks, available to partners via their (previousy internal) RESTful API, along with user authentication via OAuth.

The key goal is to allow users to maintain key personal data at sites like MySpace and not have it be locked up in an island. Previously users could turn much of this data into widgets and add them to third party sites. But that doesn’t bridge the gap between independent, autonomous websites, MySpace says. Every site remains an island.

But with Data Availability, partners will be able to access MySpace user data, combine it with their own, and present it on their sites outside of the normal widget framework. Friends lists can be syncronized, for example. Or Twitter may use the data to recommend other Twitter users who are your MySpace friends.

The key difference between MySpace's announcement and those of Facebook & Google is that MySpace has more ground to cover. Since Facebook & Google already have REST APIs that support a delegated authentication model, MySpace is pretty much playing catch up here.

In fact, on careful rereading it seems MySpace's announcement isn't like the others since the only concrete technology announced above is a REST API that uses a user-centric delegated authentication model which is something both Google and Facebook have had for years (see GData/OpenSocial and Facebook REST API).

Given my assumption that MySpace is not announcing anything new to the industry, the rest of this post will focus on Google Friend Connect and Facebook Connect.  

The Chicken & Egg Problem

When it comes to social networking, it is all about network effects. A social networking feature or site is only interesting to me if my friends are using it as well.

The argument that a site is better off using a user's social graph from a big social networking site like Facebook instead of building their own social network features only makes sense if (i) there is enough overlap in the user's friends list on Facebook and that on the site AND (ii) the user's friends on the site who are also his friends on Facebook can be discovered by the user. The latter is the tough part and one I haven't seen a good way of bridging without resorting to anti-patterns (i.e. pull the email addresses of all of the user's friends from Facebook and then cross-reference with the email addresses of the sites users). This anti-pattern works when you are getting the email addresses the user entered by hand from some Webmail address book (e.g. Hotmail, Gmail, Y! mail, etc).

However since Google and Facebook are going with a no-code solution, the only way to tell which of my Facebook friends also use the 3rd site is if they have also opted-in to linking their account on the site with their Facebook profile. This significantly weakens the network effects of the feature compared to the find your friends on your favorite "Web 2.0" site which a lot of sites have used to grow their user base by screen scraping Webmail address books then cross referencing it with their user databases.

How Does this Relate to Data Portability and Social Network Interoperability?

Short answer; it doesn't.

Long answer; the first thing to do is to make sure you understand what is meant by Data Portability and Social Network Interoperability. The difference between Data Portability and Social Network Interoperability is the difference between being able to export your email inbox and address book from Gmail into Outlook or vice versa (portable) and being able to send an email from a Gmail address to someone using Outlook or Hotmail (interoperable).

So do these new widget initiatives help portability? Nope. Widgets give developers less options for obtaining and interacting with the user data than APIs. With Facebook's REST API, I know how to get my friends list with profile data into Outlook and my Windows Mobile phone via OutSync. I would actually lose that functionality if it was only exposed via a widget. The one thing they do is lower the bar for integration by people who don't know how to code.

Well, how about interoperability? The idea of social network interoperability is that instead of being a bunch of walled gardens and data silos, social networking sites can talk to each other the same way email services and [some] IM services can talk to each other today. The "Use our data silo instead of building your own" pitch may reduce the number of data silos but it doesn't change the fact that the Facebooks and MySpaces of the world are still fundamentally data silos when it comes to the social graph. That is what we have to change. Instead we keep getting distracted along the way by shiny widgets.

PS: The blog hiatus is over. It was fun while it lasted. ;)

Now Playing: Fugees (Refugee Camp) - Killing Me Softly

 

 

Now playing: DRS - Sickness

 

News of the layoffs at Yahoo! has now hit the presses. A couple of the folks who've been indicated as laid off are people I know are great employees either via professional interaction or by reputation. The list of people who fit this bill so far are Susan Mernitt, Bradley Horowitz,  Salim Ismail and Randy Farmer.  Salim used to run Yahoo's "incubation" wing so this is a pretty big loss. It is particularly interesting that he volunteered to leave the company which may be a coincidence or could imply that some of the other news about Yahoo! has motivated some employees to seek employment elsewhere. It will be interesting to see how this plays out in the coming months.

Randy Farmer is also a surprise given that he pretty much confirmed that he was working on Jerry Yang's secret plan for a Yahoo comeback which included

• Rethinking the Yahoo homepage
• Consolidating Yahoo's plethora of social networks
• Opening up Yahoo to third parties with a consistent platform similar to Facebook's
• Revamping Yahoo's network infrastructure

If Yahoo! is reducing headcount by letting go of folks working on various next generation projects, this can't bode well for the future of the company given that success on the Web depends on constant innovation.

PS: To any ex-Yahoo's out there, if the kind of problems described in this post sound interesting to you, we're always hiring. Give me a holler. :) 

 

From the press release entitled Microsoft Proposes Acquisition of Yahoo! for $31 per Share we learn

REDMOND, Wash. — Feb. 1, 2008 — Microsoft Corp. (NASDAQ:MSFT) today announced that it has made a proposal to the Yahoo! Inc. (NASDAQ:YHOO) Board of Directors to acquire all the outstanding shares of Yahoo! common stock for per share consideration of $31 representing a total equity value of approximately $44.6 billion. Microsoft’s proposal would allow the Yahoo! shareholders to elect to receive cash or a fixed number of shares of Microsoft common stock, with the total consideration payable to Yahoo! shareholders consisting of one-half cash and one-half Microsoft common stock. The offer represents a 62 percent premium above the closing price of Yahoo! common stock on Jan. 31, 2008.

“We have great respect for Yahoo!, and together we can offer an increasingly exciting set of solutions for consumers, publishers and advertisers while becoming better positioned to compete in the online services market,” said Steve Ballmer, chief executive officer of Microsoft. “We believe our combination will deliver superior value to our respective shareholders and better choice and innovation to our customers and industry partners.”

“Our lives, our businesses, and even our society have been progressively transformed by the Web, and Yahoo! has played a pioneering role by building compelling, high-scale services and infrastructure,” said Ray Ozzie, chief software architect at Microsoft. “The combination of these two great teams would enable us to jointly deliver a broad range of new experiences to our customers that neither of us would have achieved on our own.”

WOW. Just...wow.

There's a conference call with Ray Ozzie, Steve Ballmer, Chris Liddell and Kevin Johnson in about half an hour to discuss this. This is the first time I've considered listening in on one of those.

 

Two seemingly unrelated posts flew by my aggregator this morning. The first was Robert Scoble’s post The shy Mark Zuckerberg, founder of Facebook where he talks about meeting the Facebook founder. During their conversation, Zuckerburg admits they made mistakes with their implementation of Facebook Beacon and will be coming out with an improved version soon.

The second post is from the Facebook developer blog and it is the announcement of the JavaScript Client Library for Facebook API which states

This JavaScript client library allows you to make Facebook API calls from any web site and makes it easy to create Ajax Facebook applications. Since the library does not require any server-side code on your server, you can now create a Facebook application that can be hosted on any web site that serves static HTML.

Although the pundits have been going ape shit over this on Techmeme this is an unsurprising announcement given the precedent of Facebook Beacon. With that announcement they provided a mechanism for a limited set of partners to integrate with their feed.publishTemplatizedAction API using a Javascript client library. Exposing the rest of their API using similar techniques was just a matter of time.

What was surprising to me when reading the developer documentation for the Facebook Javascript client library is the following notice to developers

Before calling any other Facebook API method, you must first create an FB.ApiClient object and invoke its requireLogin() method. Once requireLogin() completes, it invokes the callback function. At that time, you can start calling other API methods. There are two way to call them: normal mode and batched mode.

So unlike the original implementation of Beacon, the Facebook developers aren’t automatically associating your Facebook account with the 3^rd party site then letting them party on your data. Instead, it looks like the user will be prompted to login before the Website can start interacting with their data on Facebook or giving Facebook data about the user.

This is a much better approach than Beacon and remedies the primary complaint from my Facebook Beacon is Unfixable post from last month.

Of course, I haven’t tested it yet to validate whether this works as advertised. If you get around to testing it before I do, let me know if it works the way the documentation implies in the comments.

 

Ari Steinberg who works on the Facebook developer platform has a blog post entitled New Rules for News Feed which states

As part of the user experience improvements we announced yesterday, we're changing the rules for how Feed stories can be published with the feed.publishTemplatizedAction API method. The new policy moving forward will be that this function should only be used to publish actions actively taken by the "actor" mentioned in the story. As an example, feed stories that say things like "John has received a present" are no longer acceptable. The product motivation behind this change is that Feed is a place to publish highly relevant stories about user activity, rather than passive stories promoting an application.

To foster this intended behavior, we are changing the way the function works: the "actor_id" parameter will be ignored. Instead the session_key used to generate the feed story will be used as the actor.
…
In order to ensure a high quality experience for users, starting 9am Pacific time Tuesday 22 January we may contact you, or in severe cases initiate an enforcement action, if your stories are not complying with the new policy, especially if the volume of non-complying stories is high.

If you are not a developer using the Facebook platform, it may be unclear what exactly this announcement means to end users or applications that utilize Facebook’s APIs.

To understand the impact of the Facebook announcement, it would be useful to first talk about the malicious behavior that Facebook is trying to curb. Today, an application can call feed.publishTemplatizedAction and publish a story to the user’s Mini-feed (list of all the user’s actions) which will also show up in the News Feed of the users friends. Unfortunately some Facebook applications have been publishing stories that don’t really correspond to a user taking an action. For example, when a user installs the Flixster application, Flixster not only publishes a story to all of the user’s friends saying the user has installed the application but also publishes a story to the friends of each of the user’s friends that also have Flixster installed. This means my friends get updates such as

being sent to my friends when I wasn’t actually doing anything with the Flixster application. I don’t know about you but this seems like a rather insiduous way for an application to spread “virally”.

Facebook’s attempt to curb such application spam is to require that an application have a session key that identifies the logged in user when publishing the story which implies that the user is actually using the application from within Facebook when the story is published. The problem with this remedy is that it totally breaks applications that publish stories to the Facebook News Feed when the user isn’t on the site. For example, since I have the Twitter application installed on Facebook, my Facebook friends get an update sent to their News Feeds whenever I post something new on Twitter.

The problem for Facebook is that by limiting a valid usage of the API, they may have closed off a spam vector but have also closed off a valuable integration point for third party developers and for their users.

PS: There might be an infinite session key loophole to the above restriction which I’m sure Facebook will close off if apps start abusing it as well.

Now playing: Silkk the Shocker - It Ain't My Fault

 

I’ve read a number of stories this week that highlight that interoperability between social networking sites will be a “top ask” in 2008 (as we say at Microsoft). Earlier this week I read the Wired article Should Web Giants Let Startups Use the Information They Have About You? which does a good job of telling both sides of the story when it comes to startups screen scraping importing user data such as social graphs (i.e. friend and contact lists) from more successful sites as a way to bootstrap their social networks. The Wired article is a good read if you want to hear all sides of the story when it comes to the issue of sharing user social data between sites.

Yesterday, I saw Social Network Aggregation, Killer App in 2008? which points out the problem that users often belong to multiple social networks at once and that bridging between them is key. However I disagree with the premise that this points to need for a “Social Network Aggregator” category of applications. I personally believe that the list of 20 or so Social Network Aggregators on Mashable are all companies that would cease to exist if the industry got off it’s behind and worked towards actual interoperability between social networking sites.

Today, I saw saw Facebook disabled Robert Scoble’s account. After reading Robert’s account of the incident, I completely agree with Facebook.

Why Robert Scoble is Wrong and Facebook is Right

Here’s what Robert Scoble wrote about the incident

My account has been “disabled” for breaking Facebook’s Terms of Use. I was running a script that got them to keep me from accessing my account
…
I am working with a company to move my social graph to other places and that isn’t allowable under Facebook’s terms of service. Here’s the email I received:

+++++

Hello,

Our systems indicate that you’ve been highly active on Facebook lately and viewing pages at a quick enough rate that we suspect you may be running an automated script. This kind of Activity would be a violation of our Terms of Use and potentially of federal and state laws.

As a result, your account has been disabled. Please reply to this email with a description of your recent activity on Facebook. In addition, please confirm with us that in the future you will not scrape or otherwise attempt to obtain in any manner information from our website except as permitted by our Terms of Use, and that you will immediately delete and not use in any manner any such information you may have previously obtained.

The first thing to note is that Facebook allows you to extract your social graph data from their site using the Facebook platform. In fact, right now whenever I get an email from someone on my Facebook friend list in Outlook or I get a phone call from them, I see the picture from their Facebook profile. I did this using OutSync which is an application that utilizes the Facebook platform to merge data from my contacts in Outlook/Exchange with my Facebook contacts.

So if Facebook allows you to extract information about your Facebook friends via their APIs, why would Robert Scoble need to run a screen scraping script? The fact is that the information returned by the Facebook API about a user contains no contact information (no email address, no IM screen names, no telephone numbers, no street address). Thus if you are trying to “grow virally” by spamming the Facebook friend list of one of your new users about the benefits of your brand new Web 2.0 site then you have to screen scrape Facebook.  However there is the additional wrinkle that unlike address books in Web email applications Robert Scoble did not enter any of this contact information about his friends. With this in mind, it is hard for Robert Scoble to argue that the data is “his” to extract from Facebook. In addition, as a Facebook user I consider it a feature that Facebook makes it hard for my personal data to be harvested in this way. Secondly, since Robert’s script was screen scraping it means that it had to hit the site five thousand times (once for each of his contacts) to fetch all of Robert’s friends personally idenitifiable information (PII).  Given that eBay won a court injunction against Bidder’s Edge for running 100,000 queries a day, it isn’t hard to imagine that the kind of screen scraping script that Robert is using would be considered malicious even by a court of law.

I should note that Facebook is being a bit hypocritical here since they do screen scrape other sites to get the email addresses of the contacts of new users. This is why I’ve called them the Social Graph Roach Motel in the recent past. 

O’Reilly Social Graph FOO Camp

This past weekend I got an email from Tim O'Reilly, David Recordon, and Scott Kveton inviting me to a Friends of O’Reilly Camp (aka FOO Camp) dedicated to “social graph” problems. I’m still trying to figure out if I can make it based on my schedule and whether I’m really the best person to be representing Microsoft at such an event given that I’m a technical person and “social graph problems” for the most part are not technical issues.

Regardless of whether I am able to attend or not, there were some topics I wanted to recommend should be added to a list of “red herring” topics that shouldn’t be discussed until the important issues have been hashed out.

• Google OpenSocial: This was an example of unfortunate branding. Google should really have called this “Google OpenWidgets” or “Google Gadgets for your Domain” since the goal was competing with Facebook’s widget platform not actually opening up social networks. Since widget platforms aren’t a “social graph problem” it doesn’t seem fruitful the spend time discussing this when there are bigger fish to fry.

• Social Network Portability: When startups talk about “social network portability” it’s usually a euphemism for collecting a person’s username and password for another site, retrieving their contact/friend list and spamming those people about their hot new Web 2.0 startup. As a user of the Web, making it easier to receive spam from startups isn’t something I think should be done let alone a “problem” that needs solving. I understand that lots of people will disagree with this [even at Microsoft] but I’m convinced that this is not the real problem facing the majority of users of social networking sites on the the Web today.  

What I Want When It Comes to Social Network Interoperability

Having I’ve said what I don’t think is important to discuss when it comes to “social graph problems”, it would be rude not to provide an example fof what I think would be fruitful discussion. I wrote the problem I think we should be solving as an industry a while back in a post entitled A Proposal for Social Network Interoperability via OpenID which is excerpted below

I have a Facebook profile while my fiancée wife has a MySpace profile. Since I’m now an active user of Facebook, I’d like her to be able to be part of my activities on the site such as being able to view my photos, read my wall posts and leave wall posts of her own. I could ask her to create a Facebook account, but I already asked her to create a profile on Windows Live Spaces so we could be friends on that service and quite frankly I don’t think she’ll find it reasonable if I keep asking her to jump from social network to social network because I happen to try out a lot of these services as part of my day job. So how can this problem be solved in the general case? 

This is a genuine user problem which the established players have little incentive to fix. The data portability folks want to make it easy for you to jump from service to service. I want to make it easy for users of one service to talk to people on another service. Can you imagine if email interoperability was achieved by making it easy for Gmail users to export their contacts to Yahoo! mail instead of it being that Gmail users can send email to Yahoo! Mail users and vice versa?

Think about that.

Now playing: DJ Drama - The Art Of Storytellin' Part 4 (Feat. Outkast And Marsha Ambrosius)

 

In the past three months, I’ve seen three moves by Google that highlight that not only is their strategic vision becoming more questionable but their engineering talent has also begun to show signs of being seriously deficient

• STRIKE 1: OpenSocial – OpenSocial Coalition Faltering
• STRIKE 2: Android – Glitches Bug Google's Android Software
• STRIKE 3: Knol – Google Takes Aim at Wikipedia, Is Now Officially a Media Company

This year is going to be a watershed year for the company. They are eroding a lot of the geek cred they’ve built up over the past decade. That will be hard to regain once it is lost.

In the meantime, I’ve noticed an uptick in the quiet smart folks you don’t see heralded in blogs turning down offers from Google and picking another horse when the job offers come knocking. Watershed year, indeed.

Now playing: John Legend - I Can Change (feat. Snoop Doggy Dogg)

 

Yesterday I read about the Opening up Facebook Platform Architecture. My initial thoughts are that Facebook has done what Google claimed to have done but didn't with Open Social. Facebook seems to have provided detailed specs on how to build an interoperable widget platform unlike Google who unleashed a bunch of half baked REST API specs with no details about the "widget" aspect of the platform unless you are building an Orkut application.

As I've thought about this over the past few weeks, building a widget platform that is competitive with Facebook's is hard work. Remember all those stories about OpenSocial apps being hacked in 45 minutes or less? The problem was that sites like Plaxo Pulse and Ning simply didn't think through all the ramifications of building a widget platform and bumped up against the kind of "security 101" issues that widget platforms like Netvibes, iGoogle and Live.com gadgets solved years ago.  I started to wonder exactly how many of these social networking sites will be able to keep up with the capabilities and features of platforms like Facebook's and Orkut's when such development is outside their core competency.

In fact let's take a quote from the TechCrunch story First OpenSocial Application Hacked Within 45 Minutes 

theharmonyguy says he’s successfully hacked Facebook applications too, including the Superpoke app, but that it is more difficult:

Facebook apps are not quite this easy. The main issue I’ve found with Facebook apps is being able to access people’s app-related history; for instance, until recently, I could access the SuperPoke action feed for any user. (I could also SuperPoke any user; not sure if they’ve fixed that one. Finally, I can access all the SuperPoke actions - they haven’t fixed that one, but it’s more just for fun.) There are other apps where, last I checked, that was still an issue ( e.g. viewing anyone’s Graffiti posts).

But the way Facebook setup their platform, it’s tons harder to actually imitate a user and change profile info like this. I’m sure this kind of issue could be easily solved by some verification code on RockYou’s part, but it’s not inherent in the platform - unlike Facebook. I could do a lot more like this on FB if Facebook hadn’t set things up the way they did.

At that point I ask myself, how useful is it to have the specs for the platform if you aren't l337 enough to implement it yourself? [Update: It looks like Google is well aware of this problem and has launched an Apache project called Shindig which is meant to be an Open Source widget platform that implements the Open Social APIs. This obviously indicates that Google realizes the specs are worthless and instead shipping a reusable widget platform is the way to go. It’s interesting to note that with this move Google is attempting to be a software vendor, advertising partner and competitor to the Web’s social networking sites. That must lead to some confusing internal meetings. ]

For now, Facebook has definitely outplayed Google here. The most interesting part of the Facebook announcement to me is

Now we also want to share the benefits of our work by enabling other social sites to use our platform architecture as a model. In fact, we’ll even license the Facebook Platform methods and tags to other platforms. Of course, Facebook Platform will continue to evolve, but by enabling other social sites to use what we’ve learned, everyone wins -- users get a better experience around the web, developers get access to new audiences, and social sites get more applications.

it looks like Facebook plans to assert their Intellectual Property rights on anyone who clones their platform. This is one of the reasons I've found Open Social to be worrisome abuse of the term "open". Like Facebook, Google shipped specs for a proprietary platform whose copyrights, patents, etc belong to them. Any company that implements Open Social or even GData which it is built upon is using Google's intellectual property.

What's to stop Google from asserting these intellectual property rights the way Facebook is doing today? What exactly is "open" about it that makes it any less proprietary than what Facebook just announced?

 

Om Malik has a blog post entitled Zuckerberg’s Mea Culpa, Not Enough where he writes

Frankly, I am myself getting sick and tired of repeating myself about the all-important “information transmission from partner sites” aspect of Beacon. That question remains unanswered in Zuckerberg’s blog post, which upon second read is rather scant on actual privacy information. Here is what he writes:

If you select that you don’t want to share some Beacon actions or if you turn off Beacon, then Facebook won’t store those actions even when partners send them to Facebook.”

So essentially he’s saying the information transmitted won’t be stored but will perhaps be interpreted. Will this happen in real time? If that is the case, then the advertising “optimization” that results from “transmissions” is going to continue. Right!

If they were making massive changes, one would have seen options like “Don’t allow any web sites to send stories to Facebook” or “Don’t track my actions outside of Facebook” in this image below.

This is the part of Facebook's Beacon service that I consider to be unfixable which probably needs to be stated more explicitly given comments like those by Sam Ruby in his post Little Details.

The fundamental design of Facebook Beacon is that a Web site publishes information about my transactions to Facebook without my permission and then Facebook tells me what happened after the fact. This is fundamentally Broken As Designed (B.A.D.).

I read Mark Zuckerburg's Thoughts on Beacon last week and looked at the new privacy controls. Nowhere is the fundamental problem addressed.

Nothing Mark Zuckerburg wrote changes the fact that when I rent a movie from Blockbuster Online, information about the transaction is published to Facebook regardless of whether I am a Facebook user or not.  The only change Zuckerburg has announced is that I can opt out of getting nagged to have the information spammed to my friends via the News Feed. One could argue that this isn't Facebook's problem. After all, when SixApart implemented support for Facebook Beacon they didn't decide that they'd blindly publish all activities from users of TypePad to Facebook. Instead they have an opt-in model on their site which preserves their users' privacy by not revealing information to Mark Zuckerburg's company without their permission. On the flip side the Blockbuster decided to publish information about all of their customers' video rental transaction history  to Mark Zuckerburg and company, without their explicit permission, even though this violates federal law. As a Blockbuster customer, the only way around this is to stop using Blockbuster's service.

So who is to blame here? Facebook for designing a system that assumes that 3rd parties publishing private user data to them without the user's consent is OK as the default or Facebook affiliates who care so little of their customer's privacy that they give it away to Facebook in return for "viral" references to their services (aka spam)?

Now playing: Akon - Ghetto (Green Lantern remix) (feat. Notorious B.I.G. & 2Pac)

 

I often tell people at work that turning an application into a platform is a balancing act, not only do you have to please the developers on your platform BUT you also have to please the users of your application as well.

I recently joined the This has got to stop group on Facebook. If you don't use Facebook, the front page of the group is shown in the screenshot below.

I've seen a bunch of tech folks blog about being overwhelmed by Facebook app spam like Tim Bray in his post Facebook Rules and Doc Searls in Too much face(book) time. However I assumed that the average college or high school student who used the site didn't feel that way. Looks like I was wrong.

The folks at Facebook could fix this problem easily but it would eliminate a lot of the "viralness" that has been hyped about the platform. Personally, I think applications on the site have gotten to the point where the costs have begun to outweigh the benefits. The only way to tip the balance back is to rein them in otherwise it won't be long until the clean and minimal vs. cluttered and messy aesthetics stop working in their favor in comparisons with MySpace. When that happens there will be an opportunity for someone else to do the same thing to them.

On an unrelated note,  the MoveOn.org sponsored group about Facebook Beacon has 74,000 members which is less than half of the size of the This has got to stop group.  This is despite the fact that MoveOn.org has had national media attention focused on that topic. I guess it goes to show that just because a story gets a lot of hype in blogs and the press doesn't mean that it is the most important problem facing the people it actually affects.

Now playing: Jay-Z - Ignorant Shit

 

Danny Sullivan has an article in Advertising Age entitled Forget Facebook. Search Ads Are the Revolution he writes

Facebook unleashed SocialAds this month, calling it the beginning of a revolutionary, hundred-year era in advertising that will see the end of untargeted messages in mass media. If the revolution is upon us, allow me to submit the lowly search ad as the true revolutionary. For unlike social ads and most other types of advertising, search is something people want rather than something that gets in the way.  
…
The trusted referral is indeed a holy grail, and Facebook will offer a new way to build word-of-mouth. But how did that friend find the sweetener in the first place? What comes first -- word-of-mouth or the egg? At some point, a new product has to hatch, and those old-school brand-building channels probably will always play a crucial role. Search offers a key way for new products to emerge and be spread around. People turn to search for solutions -- ways to enjoy coffee without the calories or local coffeehouses to try. If you're not visible in search, perhaps you won't generate word-of-mouth as easily, if at all.

Search isn't revolutionary for aiding word-of-mouth, however. It's revolutionary for not "getting into" or in the way of anything. People turn to search when they have particular desires and need particular solutions.

I agree with Danny that the search advertising like AdWords is revolutionary while word-of-mouth advertising platforms like Facebook’s SocialAds are evolutionary.  With search ads, for the first time in the history of advertising people can find advertising when they are looking for it and otherwise it stays out of their way. When I search for digital camera or zune 80 it is quite likely that I’m making a purchasing decision so showing me ads related to buying these devices makes sense. On the other hand, when I search for foreach C# or XmlTextReader.NodeType I don’t get irrelevant ads shoved in my face. That level of match making between given consumers and advertisers is unprecedented. 

However this doesn’t mean that there isn’t something to be said for brand advertising and word of mouth. For search advertising to work, I actually have to have been looking for something in the first place. A lot of advertising today is intended to create the desire for a product in the first place not help you make an informed choice.  For example, I saw the movie Enchanted last weekend. I found out about the movie from TV ads and thought what I saw looked funny. My wife also came to the same conclusion from watching similar ads and then we decided to see the movie. After seeing the movie, I thought it was great and rated the movie in the Flixster Facebook application which sent out the following notification to my “friends”

a few days later, one of my co-workers said she saw the movie on the strength of my recommendation and other factors.

This story is a small case study in the effectiveness of traditional “media-based” advertising coupled with the power of word-of-mouth marketing using social networking sites. For now, search ads simply cannot provide a similar level of return value for such advertisers. Although search engines like Google have tried to encourage this behavior, people don’t typically perform searches like movies 98052 then decide what movies to see that weekend based on the search results page. This means that for certain classes of products, traditional advertising techniques in combination with word-of-mouth techniques like Facebook’s social ads are extremely valuable.

However at the end of the day, it is extremely unlikely that improved word-of-mouth techniques will be as impactful to the long tail of advertisers as search ads have been or ever will be. 

Now playing: Tear Da Club Up Thugs - Triple Six Clubhouse

 

Earlier this week I wrote a blog post which pointed out that the two major privacy and user experience problems with Facebook Beacon where that it (i) linked a user's Facebook account with an account on another site without the users permission and (ii) there was no way for a user to completely opt out of being tracked by the system.  Since then Facebook has announced some changes which TechCrunch named Facebook Beacon 2.0. The changes are excerpted below

Notification

Facebook users will see a notification in the lower right corner of the screen after transacting with a Beacon Affiliate. Options include “No Thanks” that will immediately stop the transaction from being published. Alternatively closing or ignoring the warning won’t immediately publish the story, but it will be put in a queue

Second Warning

Presuming you’ve ignored or closed the first notification, Facebook warns users again the next time they visit their home page. A new box reminds you that an activity has been sent to Facebook. Like the first notification you can choose to not publish the activity by hitting remove, or you can choose to publish it by hitting ok.

...

Opt Out
Found via the “External Websites” section of the Facebook Privacy page, this allows users to permanently opt in or out of Beacon notifications, or if you’re not sure be notified. The downside is that there is no global option to opt out of every Beacon affiliated program; it has to be set per program. Better this than nothing I suppose.

The interesting thing to note is that neither of the significant problems with Beacon have been fixed. After the changes were announced there was a post on the CA Security Advisory blog titled Facebook's Misrepresentation of Beacon's Threat to Privacy: Tracking users who opt out or are not logged in which pointed out that the complaining about purchase history getting into the news feed of your friends is a red herring, the real problem is that once a site signs up as a Facebook affiliate they begin to share every significant action you take on the site with Facebook without your permission. 

Which is worse, your friends knowing that you rented Prison Girls or Facebook finding that out without your permission and sharing that with their business partners, without your permission? Aren't there laws against this kind of invasion of privacy? I guess there are (see 18 U.S.C. § 2710)

I wonder who'll be first to sue Facebook and Blockbuster? 

Anyway, back to the title of this blog post. The problem with Facebook Beacon is that it is designed in a way that makes it easy for Facebook Beacon affiliates to integrate into their sites at the cost of user's privacy. From Jay Goldman's excellent post where he Deconstructed the Facebook Beacon Javascript we learn

Beacon from 10,000 Feet

That basically wraps up our tour of how Beacon does what it does. It's a fairly long explanation, so here's a quick summary:

1. The partner site page includes the beacon.js file, sets a <meta> tag with a name, and then calls Facebook.publish_action.            
2. Facebook.publish_action builds a query_params object and then passes it to Facebook._send_request.            
3. Facebook._send_request dynamically generates an <iframe>which loads the URL http://www.facebook.com/beacon/auth_iframe.php and passes the query_params. At this point, Facebook now knows about the news feed item whether you choose to publish it or not. 

When you read this you realize just how insidious the problem actually is. Facebook isn't simply learning about every action taken by Facebook users on affiliate sites, it is learning about every action taken by every user of these affiliate sites regardless of whether they are Facebook users or not.

At first I assumed that the affiliates sites would call some sort of IsFacebookUser() API and then decide whether to send the action or not. Of course, this is still broken since the affiliate site has told Facebook that you are a user of the site, and depending on the return value of the hypothetical function the affiliate in turn learns that you are a Facebook user.

But no, it is actually worse than that. The affiliate sites are pretty much dumping their entire customer database into Facebook's lap, FOR FREE and without their customers permission. What. The. Fuck.

The icing on the cake is the following excerpt from the Facebook Beacon page

Stories of a user's engagement with your site may be displayed in his or her profile and in News Feed. These stories will act as a word-of-mouth promotion for your business and may be seen by friends who are also likely to be interested in your product. You can increase the number of friends who see these stories with Facebook Social Ads.

So after giving Facebook millions of dollars in customer intelligence for free in exchange for spamming their users, Facebook doesn't even guarantee their affiliates that the spam will even get sent. Instead these sites have to pay Facebook to "increase the chances" that they get some return for the free customer intelligence they just gave Facebook.

This reminds me of the story of Tom Sawyer tricking people into paying him to paint a fence he was supposed to paint as part of his chores.

At the end of the day, Facebook can't fix the privacy problems I mentioned in my previous post in a way that completely preserves their users privacy without completely changing the design and implementation of Facebook Beacon. Until then, we'll likely see more misdirection, more red herrings and more violations of user privacy to make a quick buck. 

 

Recently I’ve read a number of negative posts about the Facebook Beacon which highlight how easy it is for a company to completely misjudge the privacy implications and ramifications of certain features in social software applications.

Charlene Li, a Principal Analyst at Forrester Research who specializing in social software trends and marketing, writes in her blog post Close encounter with Facebook Beacon  

I put a lot of trust in sites like Facebook to do the right thing when it comes to privacy. After all, the only stuff that gets out into the public is the stuff that I actually put in. Until now.

Earlier this week, I bought a coffee table on Overstock.com. When I next logged into Facebook and saw this at the top of my newsfeed:

I was pretty surprised to see this, because I received no notification while I was on Overstock.com that they had the Facebook Beacon installed on the site. If they had, I would have turned it off.

I used my personal email address to buy the coffee table, so I was puzzled why and how this "personal" activity was being associated with my "public" Facebook profile.

David Treadwell, a corporate vice president of Windows Live, writes in his blog post entitled Blockbuster, you're fired

Yesterday evening, I decided to add a few movies to my Blockbuster queue. Upon adding movies, I was surprised to see toasts from Facebook showing up on the Blockbuster site indicating that something was being added to my Facebook news feed. When I finished adding movies, I went to Facebook to see what was going on. I was then quite surprised to learn that Blockbuster and Facebook were conspiring to broadcast my movie selections to my Facebook friends.

I am not normally uptight about privacy issues, but you guys really crossed the line on this one:

• I had never told either Blockbuster or Facebook that you should share my movie selections with friends.
• Neither of you asked me if you could take this action. You just went ahead and did it, assuming that I would not mind.
• This sharing of information about me without my informed consent about the mechanism of sharing is absolutely unacceptable to me.

You can find similar complaints all over the Web from similarly Web savvy folks who you typically don’t see griping about privacy issues. In all of the complaints raised, the underlying theme is that Facebook violated the principle of putting the user in control of their user experience.

As someone who works on a competing service I have to give the folks on Facebook credit for shipping the Facebook Beacon so quickly. I assumed something like that was still about six months away from being on their radar. I do give them poor marks when it comes to how this feature has been rolled out. There are several problems with how this feature has been rolled out when it comes to how it affects their users.

1. Linking identities and data sharing without user permission: One of the thinks people have found creepy about this feature is that they are automatically discovered to be Facebook users on sites that they have not told they use Facebook. In Charlene’s case, she actually uses different email addresses to log in on both sites which must have seemed even doubly weird to her at first. As Ethan Zuckerman points out in his post Facebook changes the norms for web purchasing and privacy this completely upturns user expectations of how privacy on the Web works especially when it comes to cookies.  

   It's a genuine concern that Facebook has opened a Pandora's box when you consider what could happen if it is deemed socially acceptable for Web sites to use cookies to actively identify users across sites as opposed to the passive way it is done today. I’m sure the folks at Google would be excited about this since thanks to AdSense and DoubleClick, they  probably have cookies on every computer on the Web that has cookies turned enabled in the Web browser. Today it’s Facebook, tomorrow Amazon and eBay are posting your purchase history to every OpenSocial enabled web site courtesy of the cookies from these sites or from Google ads on your machine.

2. No global opt-out: There is no way to turn off this feature. The best you get is that when a site tries to publish an update to your news feed and mini-feed, you get an entry for the site added to your Privacy Settings for External Websites page on Facebook. I guess it never occured to Mark Zuckerburg and Justin Rosenstein that not sharing my purchase history with Facebook is a valid privacy option. Why do I have to police this list and refer back to it every couple of days to figure out if some new Web site is now publishing my private data to Facebook without my permission? 

   I expect that kind of myopia and hubris from the Googles and Microsofts of the world not Facebook. Wow, the honeymoon lasted shorter than I expected.

I suspect that Facebook will loathe fixing both issues. The first issue can’t really be solved by having partner sites provide an opt-in mechanism because there is the valid concern that (i) people won’t opt-in to the feature and (ii) the experience and messaging will vary too much from site to site for users to have a consistent set of expectations. This then points to Facebook having an opt-in page for partner sites that is part of the Facebook settings page for this feature but that may start getting away from the add 3 lines of code to reach millions of users sales pitch which they have going. Adding a global opt-out button is also similarly fraught with down side for Facebook.

At this point, they’ll have to do something. I’ll be impressed if they address both issues. Anything less is simply not good enough.

PS: The technically inclined folks in the audience should take a look at Jay Goldman’s excellent Deconstruction of the Facebook Beacon Javascript. Found via Sam Ruby.

Now playing: Eightball & MJG - Relax & Take Notes (feat. Project Pat & Notorious B.I.G.)

 

I've been pondering the implications of Facebook's SocialAds announcement and it has created some interesting lines of thought. The moment the pin dropped was when Dave Winer linked to one of his old posts that contains the following money quote

that's when the whole idea of advertising will go poof, will disappear. If it's perfectly targeted, it isn't advertising, it's information. Information is welcome, advertising is offensive.

If you think about it, the reason Google makes so much money from search advertising is because the ads are particularly relevant when a user is seeking information or a trusted opinion as part of the process to make a commercial decision. If I'm searching for "iPod" or "car insurance" then it is quite likely that ads selling me these products are relevant to my search and are actually considered to be information instead of intrusive advertising. 

Where Google's model breaks down is that a large amount of the advertising out there is intended to make you want to buy crap that you weren't even interested in until you saw the ads. In addition, trusted recommendations are a powerful way to convince customers to make purchases they were otherwise not considering. Former Amazon employee Greg Linden has written blog posts that indicate that 20% - 35% of Amazon's sales comes from recommendations like "people who like 50 Cent also like G-Unit". Given that Amazon made over 10 billion dollars in revenue last year (see financials), this means that $2 billion to $3.5 billion of that revenue is based on what Facebook is calling "social" ads.

So what does all this have to do with the title of my blog post? Glad you asked. Recently Yaron and I were chatting about the virtues of the Facebook platform. He argued that the fact that applications are encouraged to keep their data within their own silos (e.g. Flixster isn't supposed to be mucking with my iLike data and vice versa) prevents everyone [including Facebook] from benefiting from all this profile data being created from alternate sources. I argued that seeing the complexities introduced by having multiple applications being able to write to the same data store (e.g. the Windows registry) it's a lot better for users and app developers if they don't have to worry that some half baked app written by some drunken college kid is going to hose their Scrabulous scores or corrupt all their movie ratings. 

However what this means is that some of the juiciest data to serve "social" ads against within Facebook (i.e. movies and music) is not in Facebook's databases but in the databases of the developers of Facebook applications like Slide, iLike and Flixster. Considering the following entry that shows up in my friends news feeds after I performed an action in iLike ,

This entry could be improved with "social" ads in a way that is informative and valuable to my friends while also providing financial value to the application developer. For instance, would you consider the following changes to that entry to be advertising or information?

Flixster does an even worse job than iLike in making the actions they show in my news feed to be both useful and monetizable. Here's the kind of stuff that shows up in my news feed from Flixster

I don't know about you but I consider this spam. In fact, it is also misleading since what it really means is that someone on my friends list (Steve Gordon) has also installed the Flixster application on their profile. However what if the application actually published some of my movie ratings into the news feed with more context such as

People keep asking how Facebook application developers will make money. From where I'm sitting, this looks like a freaking gold mine. The problem seems to be that these applications either haven't yet figured out how lucrative a position they're in or are still in the audience acquisition phase until they flip to the highest bidder.

If Mark Zuckerburg has any strategic bone in his body, he'd snap up these companies before a hostile competitor like Google or Fox Interactive Media does. I'd put money on it that people are slowly realizing this all over Silicon Valley. 

What do you think?

 

I've seen a couple of recent articles talking about how Facebook has turned on it's platform developers with it's most recent announcements. Fortune magazine has an article today entitled Fear Among Facebook Developers which states

Zuckerberg wouldn’t deny it. On stage at the Web 2.0 conference in October in San Francisco, he acknowledged that his company reserves the right to build anything it wants and compete with any of its independent developers, but that the company intends to do this fairly. “We need to make sure we have the flexibility to do what we need as the platform grows—to be flexible enough able to add in the next big component [like the News Feed],” he said.

Yesterday Erick Schonfeld wrote an article on TechCrunch entitled iLike vs. Facebook: The Battle For The Music Artist that contains the following excerpt

Instead, Facebook is treating music artists just like any other brands, which can also set up their own Facebook pages, collect fans, and market to them directly. Yet, when it comes to music artists, one of Facebook’s most popular application developers, iLike, is doing the exact same thing.
...
So if you are a music artist, you now have to make a decision: Do you go with the iLike page as your main Facebook page (and take advantage of the nearly 10 million members who use the iLike app), or do you go with your own advertiser page on Facebook? Case in point: the new Facebook page for 50 Cent (shown left) had only three fans when it first went up just after midnight, compared to 1.2 million fans on his iLike page on Facebook.

This is a tale as old as the hills. Software platforms evolve and often this means incorporating features that were once considered as "features to be provided by others" as core parts of the platform. There are thousands of examples of application developers adding value to a platform that eventually became features of the platform due to popular demand. Whether it is adding a TCP/IP stack to the operating system, tabbed browsing to a Web browser or adding persistent searches to a Web mail application, it's all the same story. It is hard to argue that it isn't better for users such functionality to be a native part of the platform or underlying application, however it often leaves the platform developers in a lurch.

If the application developer cannot find a new way to add value to the platform then their usefulness to users comes to an end. This doesn't make it a slam dunk that once the platform vendor sees the value added by an application on it's platform, that things will eventually go sour for the application. There are many examples of vendors trying to compete with an application on their platform only to concede defeat and then try to acquire the company; PhotoBucket's acquisition by MySpace and Oracle's attempt to acquire BEA are two recent examples. [Editors note - I suspect that iLike vs. Facebook will end up going the same route as well]. In other cases, entry into the application space by the platform vendor helps to validate the market and draws more visibility to it from users.  

At the end of the day, articles like the ones I've mentioned above serve to prove that Facebook has actually built a viable and successful platform given that it is following the patterns of other successful platforms from the past several decades of the software industry.

 

Mitch Radcliffe has a blog post on ZDNet entitled Google: Does it have to be all FUD all the time? where he writes

Fear, uncertainty and doubt (FUD–see Wikipedia) is how IBM tried to retain its dominance, what Microsoft used to cement its monopoly and, now, I suggest we review recent Google news and wonder:
...
Basically, four of the last five press releases from Google have amounted to “me and my friends are going to…” beat a major competitor or rule a marketplace based on pre-announcements without a great deal of substance or products that can be seen and used today. Google sounds more like Microsoft circa 1988-to-1992, when it was launching consortia right and left to block competitors without delivering much, or any, real product. Microsoft still does this, but it doesn’t enjoy the credibility (or, better, the credulousness) that greet Google announcements.

It seems I'm not the only one that noticed this trend. I guess the Features Not Products initiative morphed into a "FUD Not Products" initiative somewhere along the line. :)

Or maybe they just hit a critical mass of ex-Microsoft employees from the old days.

PS: For bonus points, read Dave Winer's Why Google launched OpenSocial.

 

Disclaimer: This post does not reflect the opinions, thoughts, strategies or future intentions of my employer. These are solely my personal opinions. If you are seeking official position statements from Microsoft, please go here.

One of the Google folks working on OpenSocial sent me a message via Facebook asking what I thought about the technical details of the recent announcements. Since my day job is working on social networking platforms for Web properties at Microsoft and I'm deeply interested in RESTful protocols, this is something I definitely have some thoughts about. Below is what started off as a private message but ended up being long enough to be it's own blog post.

First Impressions

In reading the OpenSocial API documentation it seems clear that is intended to be the functional equivalent of the Facebook platform. Instead of the Facebook users and friends APIs, we get the OpenSocial People and Friends Data API. Instead of the Facebook feed API, we get the OpenSocial Activities API. Instead of the Facebook Data Store API, we get the OpenSocial Persistence Data API. Instead of FQL as a friendly alternative to the various REST APIs we get a JavaScript object model.  

In general, I personally prefer the Facebook platform to OpenSocial. This is due to three reasons

• There is no alternative to the deep integration into the Web site's user experience that is facilitated with FBML.  
• I prefer idiomatic XML to tunnelling data through Atom feeds in ways that [in my opinion] add unnecessary cruft.
• The Facebook APIs encourage developers to build social and item relationship graphs within their application while the OpenSocial seems only concerned with developers stuffing data in key/value pairs.

The Javascript API

At first I assumed the OpenSocial JavaScript API would provide similar functionality to FBML given the large number of sound bites quoting Google employees stating that instead of "proprietary markup" you could use "standard JavaScript" to build OpenSocial applications. However it seems the JavaScript API is simply a wrapper on top of the various REST APIs. I'm sure there's some comment one could make questioning if REST APIs are so simple why do developers feel the need to hide them behind object models?

Given the varying features and user interface choices in social networking sites, it is unsurprising that there is no rich mechanism specified for adding entry points to the application into the container sites user interface. However it is surprising that no user interface hooks are specified at all. This is surprising given that there are some common metaphors in social networking sites (e.g. a profile page, a friends list, etc) which can be interacted with in a standard way.  It is also shocking that Google attacked Facebook's use of "proprietary markup" only to not even ship an equivalent feature.

The People and Friends Data API 

The People and Friends Data API is used to retrieve information about a user or the user's friends as an Atom feed. Each user is represented as an atom:entry which is a PersonKind (which should not be confused with an Atom person construct). It is expected that the URL structure for accessing people and friends feeds will be of the form  http://<domain>/feeds/people/<userid> and http://<domain>/feeds/people/<userid>/friends respectively.

Compare the following response to a request for a user's information using OpenSocial with the equivalent Facebook API call response.

GET http://orkut.com/feeds/people/14358878523263729569

<entry xmlns='http://www.w3.org/2005/Atom' xmlns:georss='http://www.georss.org/georss' xmlns:gd='http://schemas.google.com/g/2005'>
  <id>http://sandbox.orkut.com:80/feeds/people/14358878523263729569</id>
  <updated>2007-10-28T14:01:29.948-07:00</updated>
  <title>Elizabeth Bennet</title>
  <link rel='thumbnail' type='image/*' href='http://img1.orkut.com/images/small/1193601584/115566312.jpg'/>
  <link rel='alternate' type='text/html' href='http://orkut.com/Profile.aspx?uid=17583631990196664929'/>
  <link rel='self' type='application/atom+xml' href='http://sandbox.orkut.com:80/feeds/people/14358878523263729569'/>
  <georss:where>
    <gml:Point xmlns:gml='http://www.opengis.net/gml'>
      <gml:pos>51.668674 -0.066235</gml:pos>
    </gml:Point>
  </georss:where>
  <gd:extendedProperty name='lang' value='en-US'/>
  <gd:postalAddress/>
</entry>

Below is the what the above information would look like if returned by Facebook's users.getInfo method

GET

<users_getInfo_response xmlns="http://api.facebook.com/1.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://api.facebook.com/1.0/ http://api.facebook.com/1.0/facebook.xsd" list="true">
  <user>
    <uid>14358878523263729569</uid>
     <current_location>
      <city>Palo Alto</city>
      <state>CA</state>
      <country>United States</country>
      <zip>94303</zip>
    </current_location>
    <first_name>Elizabeth</first_name>
    <is_app_user>1</is_app_user>
    <has_added_app>1</has_added_app>
    <pic>http://photos-055.facebook.com/ip007/profile3/1271/65/s8055_39735.jpg</pic>
    </user>
</users_getInfo_response>

I've already mentioned that I prefer idiomatic XML to tunnelling data through Atom feeds. Comparing the readability of both examples should explain why.

The Activities Data API 

A number of social networking sites now provide a feature which enables users to see the recent activities of members of their social network in an activity stream. The Facebook news feed, Orkut's updates from your friends, and the Windows Live Spaces what's new page are all examples of this feature. The OpenSocial Activities Data API provides a mechanism for OpenSocial applications to access and update this activity stream as an Atom feed. All of the users activities or all activities from a specific application can be accessed using URIs of the form  http://<domain>/activities/feeds/activities/user/<userID> and http://<domain>/activities/feeds/activities/user/<userID>/source/<sourceID> respectively.  

Currently there is no reference documentation on this API. My assumption is that since Orkut is the only OpenSocial site that supports this feature, it is difficult to produce a spec that will work for other services without it being a verbatim description of Orkut's implementation.

There are some notes on how Orkut attempts to prevents applications from spamming a user's activity stream. For one, applications are only allowed to update the activity stream for their source directly instead of the activity stream for the user. I assume that Google applies some filter to the union of all the source specific activity streams before generating the user's activity feed to eliminate spam. Secondly, applications are monitored to see if they post too many messages to the activity stream or if they post promotional messages instead of the user's activities to the stream. All of this makes it seem difficult to see how one could specify the behavior of this API and feature set reliably for a diverse set of social networking sites.

The Persistence Data API 

The OpenSocial Persistence API allows applications to store and retrieve key<->value pairs that are either user-specific or are global to the application. An example of the former is a listing of company name and stock ticker pairs while an example of the latter is a user's stock portfolio. The feed of global key<->value pairs for an application can be accessed at a URL of the form http://<domain>/feeds/apps/<appID>/persistence/global for the entire feed and http://<domain>/feeds/apps/<appID>/persistence/global/<key> if seeking a particular key<->value pair. User-specific key<->value pairs are available at the URL of the form http://<domain>/feeds/apps/<appID>/persistence/<userID>/instance/<instanceID>.

This is probably the least interesting aspect of the API. A simple persistence API like this is useful for applications with simple storage needs that need to store user preferences or simple textual data that is needed by the application. However you aren't going to use this as the data storage platform for applications like iLike, Flixster or Scrabulous.

However I will add that an Atom feed seems like a horrible representation for a list of key<->value pairs. It's so bad that the documentation doesn't provide an example of such a feed.

Hosting OpenSocial Applications

The documentation on hosting OpenSocial applications implies that any site that can host Google gadgets can also host OpenSocial applications. In practice, it means that any site that you can place a <script> element on can point to a gadget and thus render it. Whether the application will actually work will depend on whether the hosting service has actually implemented the OpenSocial Service Provider Interface (SPI).

Unfortunately, the documentation on implementing the OpenSocial SPI is missing in action. From the Google site

To host OpenSocial apps, your website must support the SPI side of the OpenSocial APIs. Usually your SPI will connect to your own social network, so that an OpenSocial app added to your website automatically uses your site's data. However, it is possible to use data from another social network as well, should you prefer. Soon, we will provide a development kit with documentation and code to better support OpenSocial websites, along with a sample sandbox which implements the OpenSocial SPI using in-memory storage. The SPI implements:

• Adding and removing friends
• Adding and removing apps
• Storing activities
• Retrieving activity streams for self and friends
• Storing and retrieving per-app and per-app-per-user data

The OpenSocial website development kit will include full SPI documentation. It will provide open source reference implementations for both client and server components.

I assume that the meat of the OpenSocial SPI is documentation is just more detailed rules about how to implement the REST APIs described above. The interesting bits will likely be the reference implementations of the API which will likely become the de facto standard implementations instead of encouraging dozens of buggy incompatible versions of the OpenSocial API to bloom.   

Conclusion

In general I believe that any effort to standardize the widget/gadget APIs exposed by various social networking sites and AJAX homepages (e.g. iGoogle, Netvibes, Live.com, etc) is a good thing. Niall Kennedy has an excellent series of articles on Web Widget formats and Web Widget update technologies that shows how diverse and disparate the technologies that developers have to learn and utilize when they want to build widgets for various sites. Given that Web widgets are now a known quantity, the time is ripe for some standardization.

That said, there are a number of things that give me cause to pause with regards to OpenSocial

1. A common practice in the software industry today is to prefix "Open" to the name of your technology which automatically gives it an aura of goodness while attempting to paint competing technologies as being evil and "closed". Examples include OpenDocument, OpenID, OpenXML, OAuth, etc. In this case, OpenSocial is being positioned as an "open" alternative to the Facebook platform.  However as bloggers like Shelley Powers, Danny Ayers and Russell Beattie have pointed out, there isn't much "open" about OpenSocial. Russell Beattie asks in his post Where the hell is the Container API?

   Would people be jumping on this bandwagon so readily if it was Microsoft unilaterally coming up with an API, holding secret meetings geared towards undercutting the market leader, and then making sure that only those anointed partners get a head start on launch day by making sure a key part of the API isn't released - even in alpha. (It obviously exists already, all the partners have that spec and even sample code, I'm sure. The rest of us don't get access yet, until the GOOG says otherwise).

   Let's say we ignore that the process for creating the technology was not "open" nor have key aspects of the technology even been unveiled [which makes this more of a FUD announcement to take the wind out of Facebook's sails than an actual technology announcement], is the technology itself open? Shelley Powers points out her post Terms that

   Perhaps the world will read the terms of use of the API, and realize this is not an open API; this is a free API, owned and controlled by one company only: Google. Hopefully, the world will remember another time when Google offered a free API and then pulled it. Maybe the world will also take a deeper look and realize that the functionality is dependent on Google hosted technology, which has its own terms of service (including adding ads at the discretion of Google), and that building an OpenSocial application ties Google into your application, and Google into every social networking site that buys into the Dream.

   Google has announced a technology platform that is every bit as proprietary as Facebook's. The only difference is that they've cut deals with some companies to utilize their proprietary platform while Facebook's platform is only for use on the Facebook site. If Zuckerburg announces next week that the Facebook platform is freely implementable by any 3rd party Web site, where does that leave OpenSocial? After all, the Facebook platform is actually a proven, working system with complete documentation instead of the incomplete rush job that OpenSocial clearly is right now.

   There are all sorts of forums for proposing and discussing open Web technologies including the IETF, W3C, OASIS and even ECMA. Until all of the underlying technologies in OpenSocial have been handed over to one or more of these standards bodies, this is a case of the proprietary pot calling the proprietary kettle black.

2. One of the things that comes along with OpenSocial is that Google has now proposed GData as the standard protocol for interacting with social graphs on the Web. This is something that I've been worried about for a while and I've written a couple of blog posts to address this topic because it is not clear that the Atom Publishing Protocol upon which GData is based works well outside it's original purpise of editing blog posts and the like. I'm not the only one that feels this way.

   Danny Ayers wrote in his post Open? Social?

   However the People Data API is cruel and unusual. It first stretches Atom until it creaks with "each entry in the People or Friends feed is a PersonKind"; then gives a further tug  (a person's name is represented using atom:title) then extends it even more (a person's email is gd:email) and finally mops up all the blood, sweat and dribble:

   Key value parameters - gd:extendedProperty - "As different social networks and other sources of People data have many different named fields, this provides a way for them to be passed on generally. Agreeing on common naming conventions is to be decided in future."

   Got to admire the attempt, but (to mix the metaphorical namespaces) silk purses don't make very good sow's ears either.

   In addition, AtomPub geek extraordinairre, Tim Bray wrote in his blog post entitled Web3S

   If you decide you totally can’t model your world as collections of entries populated with hyperlinks to express relationships, well then I guess APP’s not for you. And at the level of engineering intuition, I have to say that a monster online address book does feel different at a deep level from most online “publications” (I thought that was why we had LDAP... but I repeat myself).

   Now that we have AtomPub/GData as a de facto standard protocol for accessing various kinds of non-microcontent data on the Web as a reality, I'm done debating its suitability for the task since the horse has already left the barn. However I will continue to ask when will GData be RFC 5023 compliant?

3. At the end of the day, the most disappointing thing about OpenSocial is that it doesn't really further the conversation about actual interoperability across social networking sites. If I use Orkut, I still need a MySpace account to interact with my friends on that site. Some people have claimed that OpenSocial will enable routing around such lock-in via applications like iLike and Flixster which have their own social networks and thus could build cross-site social networking services since they will be hosted on multiple social networking sites. However the tough part of this problem is how a hosted application knows that carnage4life@windowslivespaces is the same user as DareObasanjo@Facebook? It seems OpenSocial completely punts on satisfying this scenario even though it wouldn't be hard to add this as a requirement of the system. I guess the various applications can create their own user account systems and then do the cross-site social network bridging that way, which sucks because it will be a lot of duplicative work and will require users to create even more accounts with various services.

   Given that the big widget vendors like iLike, Slide and RockYou already have their users creating accounts on their sites that can be tied back to which social networking site the user utilizes their widgets on, this might be a moot point. Wouldn't it be mad cool if the Top Friends Facebook application could also show your top friends from MySpace or Orkut? I suspect the valuation of various widget companies will be revised upwards in the coming months.

4. There is no mention of a user-centric application authorization model. Specifically, there is no discussion of how users grant and revoke permission to access their personal data to various OpenSocial applications. Regular readers of my blog are familiar with my mantra of putting the user in control which is why I've been so enthusiastic about OAuth. Although there is some mention of Google's Authentication for Web Application in the documentation, this seems specific to Google's implementation of OpenSocial hosting and it is unclear to me that we should expect that this is the same model that will be utilized by MySpace, Bebo, TypePad or any of the other social networking sites that have promised to implement OpenSocial. On the other hand, Facebook has a well thought out applications permission model and I would have thought it would be quite easy to simply reverse engineer that and add it to the OpenSocial spec than to simply punt on this problem.

Despite these misgivings, I think this is a step in the right direction. Web widget and social graph APIs need to be standardized across the Web.

PS: I've subscribed to the Google OpenSocial blog. So far there have only been posts by clueless marketing types but I'm sure interesting technical information that addresses some of the points above will be forthcoming.

 

In a post entitled Checkmate? MySpace, Bebo and SixApart To Join Google OpenSocial (confirmed) Mike Arrington writes

Google may have just come out of nowhere and checkmated Facebook in the social networking power struggle.

…

Update (12:30 PST): On a press call with Google now. This was embargoed for 5:30 pm PST but they’ve moved the time up to 12:30 PST (now). Press release will go out later this evening. My notes:

On the call, Google CEO Eric Schmidt said “we’ve been working with MySpace for more than a year in secret on this” (likely corresponding to their advertising deal announced a year ago).

MySpace says their new platform efforts will be entirely focused on OpenSocial.

The press release names Engage.com, Friendster, hi5, Hyves, imeem, LinkedIn, Ning, Oracle, orkut, Plaxo, Salesforce.com, Six Apart, Tianji, Viadeo, and XING as current OpenSocial partners.

We’re seeing a Flixster application on MySpace now through the OpenSocial APIs. Flixster says it took them less than a day to create this. I’ll add screen shots below.

Here’s the big question - Will Facebook now be forced to join OpenSocial? Google says they are talking to “everyone.” This is a major strategic decision for Facebook, and they may have little choice but to join this coalition.

Bebo has also joined OpenSocial.

I'm confused as to how Mike Arrington considers this a checkmate by Google. At the end of the day, this announcement is simply that folks like Slide and RockYou don't have to maintain multiple code bases for their widgets on various popular social networking sites. In addition, it brings the widget/gadget platform on these sites to a similar level to the Facebook platform. Of course, it won’t be on the same level unless it meets all the criteria from my post on how developers should evaluate the MySpace platform. Which is unlikely since besides MySpace, none of those sites have the userbase or engagement of Facebook users nor does any of them have the same kind of viral properties in distributing applications that Facebook platform has built-in. 

At the end of the day, will we see widget developers like the folks at iLike, Slide or Scrabulous leave the Facebook platform because of these announcements? Unlikely.

Will we see a mass migration from Facebook to MySpace or Orkut because you can now add Flixster or Scrabulous to your profile on these sites? Probably not.

So how is this a checkmate again?

OpenSocial simply keeps Facebook’s competitors in the game. It is more like a successful kingside castle than a checkmate.

Now playing: Backstreet Boys - Incomplete

 

In thinking about the Google OpenSocial Announcement I realized how much some of Google's recent moves remind me of Microsoft of old [for some undeclared definition of old]. We have

• Google is now a two trick pony with two self reinforcing monopolies (Search + AdSense/Adwords) which is reminiscent of the self reinforcing nature of Microsoft's Windows + Office.

• A scared overreaction to the growing hype of a startup that is eclipsing it in coolness. See OpenSocial reaction to Facebook
  

• Several high level and very visible defections to said startup from people who were highly regarded there. Seems like just yesterday we were reading similar articles and blog posts about MSFT -> Google employee defections.
  

• Said startup may be attacking Google's core monopoly in ways that were unforeseen but make sense in hindsight. See Facebook's SocialAds announcements.
  

• Creation of de jure and de facto industry standards which are pitched as being great for customers but seem suspiciously like attempts at Fire & Motion. Compare WS-* to AtomPub + GData + OAuth + OpenSocial + ???.

 

There’s nothing like a successful company with a near monopoly to force the software industry to come up with standards. Or in this case, as in many others, force it’s competitors to band together and call what they are doing the standard because more than one vendor supports it.

From TechCrunch’s article Details Revealed: Google OpenSocial(To Launch Thursday we learn

Google wants to create an easy way for developers to create an application that works on all social networks. And if they pull it off, they’ll be in the center, controlling the network.

What They’re Launching

OpenSocial is a set of three common APIs, defined by Google with input from partners, that allow developers to access core functions and information at social networks:

• Profile Information (user data)
• Friends Information (social graph)
• Activities (things that happen, News Feed type stuff)

Hosts agree to accept the API calls and return appropriate data. Google won’t try to provide universal API coverage for special use cases, instead focusing on the most common uses. Specialized functions/data can be accessed from the hosts directly via their own APIs.

Unlike Facebook, OpenSocial does not have its own markup language (Facebook requires use of FBML for security reasons, but it also makes code unusable outside of Facebook). Instead, developers use normal javascript and html (and can embed Flash elements). The benefit of the Google approach is that developers can use much of their existing front end code and simply tailor it slightly for OpenSocial, so creating applications is even easier than on Facebook.

Similar details are available from folks like Om Malik and Marc Andreesen. 

This is a brilliant move. I’ve blogged on multiple occassions that the disparate widget platforms in social networking sites is a burden for widget developers and will lead to a “winner takes all” situation because no one wants to support umpteen different platforms. If enough momentum gains around OpenSocial, then three things will happen

• Widget developers will start to favor coding to OpenSocial because it supports multiple sites as well as targeting the Facebook platform  
• Eventually Facebook platform developers will start asking Zuckerburg and company to support OpenSocial so they only need to worry about one code base (kinda, it won’t be that easy)  
• Other companies with proprietary widget platforms or plans to create one will bow down to the tide and adopt OpenSocial

Of course, this requires a popular social networking site with a wide audience (e.g. MySpace) to adopt the platform before we see this kind of traction.

However this is the only thing Google could have done that makes any sense. Building a clone of the Facebook platform like some social networking sites planned would have been dumb because that would be the tail wagging the dog. Similarly building a competing proprietary platform would also have been dumb due to the winner takes all problem I mentioned earlier.

This is the only move that has a chance of actually giving their anti-Facebook platform a chance of being successful.

I wonder how my coworkers in Windows Live are going to take this news?

Now playing: 50 Cent - All Of Me (Feat. Mary J Blige) (Prod by Jake One)

 

Web companies are primarily media companies. Their job is to assemble audiences and then sell ads to their captive audiences. This is what the newspaper, television and magazine industries did before them but for some reason Web companies like Google love to claim that they aren’t media companies.  The irony is that Google stumbled on the most lucrative media property of all time (Web search results) as well as the most lucrative way of monetizing it (auction based keyword sales).

Web 1.0 was about gathering a bunch of eyeballs and then entertaining them with a bunch of content you created. Web 2.0 is all about gathering a bunch of eyeballs then having them entertain themselves with content they’ve created (also known as UGC). The most important thing hasn’t changed, it’s about gathering eyeballs.

This takes us to social networking sites like MySpace and Facebook which have taken the act of gathering eyeballs and then having your users entertain themselves with the help of random developers and elevated it to a fine art.  Things get interesting when you realize that a lot of these “random developers” have built compelling experiences and have gathered a significant audience in their own right on the backs of these social networking sites.

In a blog post entitled Scrabulous Jason Antonelli writes

I've become addicted to Scrabulous on Facebook…I'm on my 3rd game of Scrabble with my mom (yes I won two games so far) - and it usually starts with a phone call where we play for a bit and chat about various mom/son things.  My mom lives 3000+ miles away -- she's in Wilmington, NC and I'm in Seattle.  After a while when one of us needs to go do something else - we'll go ahead and play it throughout the day - maybe each making a 1-2 moves a day.

This is so much fun (though I wonder how much it would be if I didn't have a 100% win rate) and it clearly is bringing social networks into the mainstream.  The fact that this game is hosted on the Facebook platform (although it does look like it has a standalone site as well) gave two major wins to Facebook.  First I actually convinced by mom (I'm 33 so you can only guess her age!) to signup for Facebook - the first person I got to sign up to any social networking application even including the one I've worked on.  And second, both my mom and I log in to Facebook multiple times a day.

I’m similarly hooked on Scrabulous on Facebook and estimate that 75% of my page views to *.facebook.com are actually to the Scrabulous application page. I’m definitely see more ads hosted within Scrabulous than I see those hosted within Facebook*. Which leads me to wonder especially when I notice that Scrabulous now sports an ads by Google logo. I specifically wondered about the reach of the most popular widget makers and was surprised when I took a look at the Slide.com About Page which informed me that  

Slide is the largest personal media network in the world, reaching more than 134 million unique global viewers each month and 30 percent of the U.S. Internet audience. We help people express themselves and tell stories through personalized photos and videos created on Slide.com and viewed anywhere on the web or desktop.

Slide widgets — including Slideshows, Guestbooks, SkinFlix and FunPix — are popular on top social networking and blog platforms, including MySpace, Facebook, Bebo, Hi5, Friendster, Tagged, Piczo and Blogger. Slide is also the leading application developer on Facebook with more than 63 million applications installed, including SuperPoke and Top Friends, the most active application by more than four times that of any other 3rd party developer.

Whoa. I suspect that once MySpace ships their platform which relaxes the rules on advertising within widgets, companies like Slide will start to look like very interesting acquisition targets. Or maybe there’s enough money to be made generating the majority of the page views on the most popular social networking sites in the world, that there’ll be no incentive to cash out. Interesting times ahead.

* I do think I’ve clicked more on Facebook hosted ads than those hosted within Scrabulous so that may not be a useful metric.

Now playing: Fabolous - This Is My Party

 

Disclaimer: Although I work on the What’s New feed in Windows Live Spaces this should not be considered an announcement or precursor to an announcement of upcoming features of any Windows Live service.

I spend a lot of my time these days thinking about digital lifestyle aggregators such as Facebook and FriendFeed. One of the things I wonder about is how to make them more relevant to users as a way to stay connected to each other without seeming confusing, overwhelming or just plain spammy.

For instance, I look at the Facebook News Feed as the first significant implementation of this concept to hit the mainstream and I try to see what we can learn from their mistakes and where there is room for improvement. Below are two mistakes and one place I see room for improvement in the news feed as currently implemented by Facebook. The screenshot below is provided as a reference point.

Wall Posts

I’m now quite convinced that having wall posts show up in the news feed is a mistake. In general, Facebook already indulges in bad design by having a Wall-to-Wall posting which means that you can be viewing a friend’s wall and may only see one half of the conversation. So there is always a confusing loss of context when reading a wall on Facebook. This loss of context is exacerbated by adding wall posts to the news feed since now we not only have to deal with hearing one side of a conversation. Instead a user logs in and is confronted with a statement from the middle of a conversation, clicks through and only sees half the conversation, tries to click through to that and may not have access to both user’s walls.

More than once I’ve logged into Facebook and been confronted with wall posts that would have been embarassing to the posters if they realized that their banter on some person’s guestbook wall was being broadcast out of context to all their co-workers, their manager and even their VP/CxO via the news feed.

Application Installations

Although I don’t agree with Kara Swisher for criticizing Facebook applications as being mostly trivial time wasters instead of professional tools, I do agree with her that most apps on the site aren’t of value. This means that using up my screen real estate to tell me that a buddy has installed the Pink Ribbon application or the ProductPulse application is spam almost every single time you do it. One should also consider that Facebook limits the amount of updates from your friends they show in your news feed to ensure a good mix of updates. I suspect most users would gladly trade the slots taken up by notifications of application installs for more personally relevant updates from their social network.

The notifications about application installs showing up in the news feed is nice for developers but I question it’s value to users. Especially when you consider that Facebook applications already have ways to spread virally via application requests which has unfortunately led to Facebook Application Fatigue by their users.

Groups and Events

Today, a lot of groups on Facebook are there primarily as a way to declare affiliation as opposed to being an active community of users such as you’d find on sites like MSN Groups or Yahoo! Groups. I joined groups like I Dont care How Comfortable Crocs Are, You Look Like A Dumbass, I Am Fluent in Sarcasm, and If Wikipedia Says It, It Must Be True because I thought joining them would look funny on my friend’s news feeds as opposed to wanting to be part of these groups either as a lurker or as a regular discussion participant.

One question is why Facebook uses the news feed to drive user to user traffic but not user to group traffic besides the “Dare has joined People Who Always Have To Spell Their Names For Other People” which encourages people to join the group but doesn’t encourage them to participate in the group. It may be that they don’t want users creating online groups within the site like you find in services like MSN Groups or Yahoo! Groups or it could just be that their platform can’t support that scale of activity. I wonder…

The same questions apply for events as well. It would be cool if after I was invited to an event, I also got news feed updates via my news feed later on telling me if my friends were attending or that the event was getting lots of attendees which may influence my attendance. Again, you have to wonder why such obvious enhancements haven’t made it into their service.

Now playing: Three 6 Mafia - Late Nite Tip

 

Erick Schonfeld from TechCrunch writes in his post Windows Live SkyDrive Doubles Storage to 1GB, Still Can’t Keep Up With Gmail that

Microsoft doubled the online storage consumers can get for free in Windows Live SkyDrive. It’s hard to get excited about that when Gmail is already giving me 2.9 GB of storage, with more on the way—4GB by the end of the month, and 6GB by early January, according to one estimate.

You’d think that someone who works as a pseudo-journalist on a popular technology website would be able to tell the difference between an email service and a file storage service. You’d think he’d want to compare apples to apples and compare GMail’s 2.9 GB of Storage with Windows Live Hotmail’s 5 Gigabytes of Storage or compare the capabuilities of Microsoft’s SkyDrive with Google’s GDrive. 

Except Google hasn’t figured out how to ship GDrive for over 5 years so it would be an apples to vaporware comparison. 

Much love to my SkyDrive peeps on their new release. The champagne and ice cream yesterday was much appreciated. You can learn more about their release from the post Updates to Windows Live SkyDrive! on their team blog.  

Now playing: Foo Fighters - My Hero

 

Mini-Microsoft has a blog post up to let us know that his Facebook account was cancelled. In the comments he clarifies he wasn’t specifically targetted and this is just part of the Facebook terms of service. He writes

For those who probably will never see this Facebook help-topic, this is what I've been directed to:

http://www.facebook.com/help.php?page=45

The only relevant text that I can find:

"Facebook does not allow users to register with fake names, to impersonate any person or entity, or to falsely state or otherwise misrepresent themselves or their affiliations."

I imagine they only do something when someone complains vs. being constantly policing things. And someone out there (scanning the crowd of exceptionally good looking people who visit here) must have taken it upon themselves to complain.

I didn’t realize that if I don’t provide 100% accurate data about myself (thus making identity theft easier) I could get my account banned from Facebook.

I can understand why they want to encourage people to use real names since they want to be the kind of place that have users like “Dare Obasanjo” and “Robert Scoble” not ‘carnage4life’ and ‘scobleizer’ since the former implies a more personal experience.

However it seems dumb to be trying to replicate Friendster’s mistake by killing off every account that didn’t conform to their standards. There are ways to encourage such behavior without being jerks as they’ve clearly been in this case.

Now playing: Dem Franchize Boyz - Oh I Think They Like Me (remix) (feat. Jermaine Dupri, Da Brat & Lil Bow Wow)

 

Yesterday morning, I tossed out a hastily written post entitled It Must Be a Fun Time to Work on Microsoft Office which seems to have been misread by some folks based on some of the comments I’ve seen on my blog and in other places. So further exposition of some of the points in that post seems necessary.

First of all, there’s the question of who I was calling stupid when talking about the following announcements

• Google announcing the launch of Presently, their Web-based Powerpoint clone. Interestingly enough, one would have expected presentation software to be the most obvious application to move to the Web first instead of the last.
• Yahoo! announcing the purchase of Zimbra, a developer of a Web-based office productivity and collaboration suite.
• Microsoft announcing the it would integrate Web-based storage and collaboration into it’s desktop office productivity suite.
• IBM announcing that it would ship it’s own branded version of an Open Source clone of Microsoft’s desktop productivity suite.

Given that three of these announcements are about embracing the Web and the last one is about building disconnected desktop software, I assumed it was obvious who was jumping on a dying paradigm while the rest of the industry has already moved towards the next generation. To put this another way, James Robertson’s readers were right that I was talking about IBM.

There is something I did want to call out about James Robertson’s post. He wrote

People have moved on to the 80% solution that is the web UI, because the other advantages outweigh that loss of "richness".

I don’t believe that statement when it comes to office productivity software. I believe that the advantages of leveraging the Web are clear. From my perspective

1. universal access to my data from any device or platform 
2. enabling collaboration with “zero install” requirements on collaborators

are clear advantages that Web-based office productivity software has over disconnected desktop software.

It should be noted that neither of these advantages requires that the user interface is Web-based or that it is rich (i.e. AJAX or Flash if it is Web-based). Both of these things help but they aren’t a hard requirement.

What is important is universal access to my data via the Web. The reason I don’t have an iPhone is because I’m hooked on my Windows Mobile device because of the rich integration it has with my work email, calendar and tasks list. The applications on my phone aren’t Web-based, they are the equivalent of “desktop applications” for my phone. Secondly, I didn’t have to install them because they were already on my phone [actually I did have to install Oxios ToDo List but that’s only because the out-of-the-box task list synchronization in Windows Mobile 5 was less than optimal for my needs].

I used to think that having a Web-based interface was also inevitable but that position softened once I realized that you’ll need offline support which means building support for local storage + synchronization into the application (e.g. Google Reader's offline mode) to truly hit the 80/20 point for most people given how popular laptops are these days. However once you’ve built that platform, the same storage and synchronization engine could be used by a desktop application as well.

In that case, either way I get what I want. So desktop vs. Web-based UI doesn’t matter since they both have to stretch themselves to meet my needs. But it is probably a shorter jump to Web-enable the desktop applications than it is to offline-enable the Web applications.  

Now playing: Playa Fly - Feel Me

 

This is one of those posts I started before I went on my honeymoon and never got around to finishing. There are lots of interesting things happening in the world of office productivity software these days. Here are four announcements from the past three weeks that show just how things are heating up in this space, especially if you agree with Steve Gillmor that Office is Dead ^{*(see footnote)}.

From the article Google Expands Online Software Suite 

MOUNTAIN VIEW, Calif. (AP) — Google Inc. has expanded its online suite of office software to include a business presentation tool similar to Microsoft Corp.'s popular PowerPoint, adding the latest twist in a high-stakes rivalry.
…
Google's software suite already included word processing, spreadsheet and calendar management programs. Microsoft has been reaping huge profits from similar applications for years.

Unlike Google's applications, Microsoft's programs are usually installed directly on the hard drives of computers.

From the article I.B.M. to Offer Office Software Free in Challenge to Microsoft’s Line

I.B.M. plans to mount its most ambitious challenge in years to Microsoft’s dominance of personal computer software, by offering free programs for word processing, spreadsheets and presentations.

Steven A. Mills, senior vice president of I.B.M.’s software group, said the programs promote an open-source document format.

The company is announcing the desktop software, called I.B.M. Lotus Symphony, at an event today in New York. The programs will be available as free downloads from the I.B.M. Web site.

From the blog post Yahoo scoops up Zimbra for $350 million

Yahoo has been on an acquisition binge late, but mostly to expand its advertising business. Now Yahoo is buying its way deeper into the applications business with the acquisition of Zimbra for a reported $350 million, mostly in cash. Zimbra developed a leading edge, Web 2.0 open source messaging and collaboration software suite, with email, calendar, document processing and a spreadsheet.

and finally, from the press release Microsoft Charts Its Software Services Strategy and Road Map for Businesses

 Today Microsoft also unveiled the following:

• Microsoft® Office Live Workspace, a new Web-based feature of Microsoft Office that lets people access their documents online and share their work with others

…

Office Live Workspace: New Web Functionality for Microsoft Office

Office Live Workspace is among the first entries in the new wave of online services. Available at no charge, Office Live Workspace lets people do the following:

• Access documents anywhere. Users can organize documents and projects for work, school and home online, and work on them from almost any computer — even one not connected to the company or school network. They can save more than 1,000 Microsoft Office documents to one place online and access them via the Web.

• Share with others. Users can work collaboratively on a project with others in a password-protected, invitation-only online workspace, helping to eliminate version-control challenges when e-mailing drafts to multiple people. Collaborators who don’t have a desktop version of Microsoft Office software can still view and comment on the document in a browser.
  

As you can see one of these four announcements is not like the others. Since it isn’t fair to pick on the stupid, I’ll let you figure out which company is jumping on a dying paradigm while the rest of the industry has already moved towards the next generation.  The Web is no longer the future of computing, computing is now about the Web.

* I do. Disconnected desktop software needs to go the way of the dodo.

Now playing: Prince - Sign 'O' the Times

 

I scored an invite to FriendFeed and after trying out the service, I have to say it is both disappointing and encouraging at the same time. It is disappointing because one would expect folks like Bret Taylor and Paul Buchheit who helped launch Google Maps, Gmail and AdSense while at Google to come up with something more innovative than a knock-off of Plaxo Pulse and Google’s SocialStream which are themselves knock-offs of the Facebook News feed.

On the other hand, this is encouraging because it is another example of how the digital lifestyle aggregator is no longer just a far out idea being tossed around on Marc Canter’s blog but instead has become a legitimate product category.  

So what exactly is FriendFeed? The site enables users to associate themselves with the various user generated content (UGC) sites which they use regularly that publish RSS feeds or provide open APIs and then this is turned into the equivalent of a Facebook Mini Feed for the user. You can get a good idea of it by viewing my page at http://friendfeed.com/carnage4life which aggregates the recent activities from my profiles on reddit, digg, and youtube.

The “innovation” with FriendFeed is that instead of asking you to provide the URLs of your RSS feeds, the site figures out your RSS feed from your username on the target service. See the screenshot below for this in action

Of course, this same “innovation” exists in Plaxo Pulse so this isn’t mindblowing. If anything, FriendFeed is currently a less feature rich version of Plaxo Pulse.

I personally doubt that this site will catch on because it suffers from the same chicken and egg problem that face all social networking sites that depend on network effects. And if it does catch on, given that there is zero barrier to entry in the feature-set they provide, I wouldn’t be surprised to see Facebook and a host of other services roll this into their feature set. I expect that News Feed style pages will eventually show up in a majority of social sites, in much the same way that practically every website these days has a friend’s list and encourages user generated content. It’s just going to be another feature when it comes to making a website, kinda like using tabs for navigation.

I’m sure Marc Canter finds this validation of his vision quite amusing.

Now playing: Puddle of Mudd - Control

 

Last month there was a press release published by Sophos, an IT  security company, with the tantalzing title Sophos Facebook ID probe shows 41% of users happy to reveal all to potential identity thieves which reports the following

 The Sophos Facebook ID Probe involved creating a fabricated Facebook profile before sending out friend requests* to individuals chosen at random from across the globe.
...

Sophos Facebook ID Probe findings:

• 87 of the 200 Facebook users contacted responded to Freddi, with 82 leaking personal information (41% of those approached)
• 72% of respondents divulged one or more email address
• 84% of respondents listed their full date of birth
• 87% of respondents provided details about their education or workplace
• 78% of respondents listed their current address or location
• 23% of respondents listed their current phone number
• 26% of respondents provided their instant messaging screenname

In the majority of cases, Freddi was able to gain access to respondents' photos of family and friends, information about likes/dislikes, hobbies, employer details and other personal facts. In addition, many users also disclosed the names of their spouses or partners, several included their complete résumés, while one user even divulged his mother's maiden name - information often requested by websites in order to retrieve account details.

This is another example of how Facebook needs to be better at managing multiple social contexts. Right now, there is no way for me to alter my privacy settings to prevent people who I’ve added to my “friends list” from seeing my personal information. The thing is my “friends list” comprised of more than just friends. It is comprised of co-workers, people who work at the same company, people I went to high school with, and close personal friends. There’s also the category of “people who read my blog or use RSS Bandit” that I generally tend to decline friend requests from. I don’t mind some of these people being able to access my personal information (e.g. cell phone number, email address, birthday, etc) but clearly I also don’t want every random person who reads my blog that wants to be my “friend” on Facebook to have access to this information. 

Is there a better way to do this? Below are screenshots of the permissions model we came up with for Profiles on MSN Spaces when I worked on the feature juxtaposed with the Profile permissions options on Facebook.

Facebook

 

 Windows Live Spaces

Straightforward isn’t it? I suspect that the problem here is that the folks at Facebook are refusing to acknowledge that their user base is changing now that they’ve opened up. As danah boyd writes in her post SNS visibility norms (a response to Scoble) 

Facebook differentiated itself by being private, often irritatingly so. Hell, in the beginning Harvard kids couldn't interact with their friends at Yale, but that quickly changed. Teens and their parents worship Facebook for its privacy structures, often not realizing that joining the "Los Angeles" network is not exactly private. For college students and high school students, the school and location network are really meaningful and totally viable structural boundaries for sociability. Yet, the 25+ crowd doesn't really live in the same network boundaries. I'm constantly shifting between LA and SF as my city network. When I interview teens, 80%+ of their FB network is from their high school. Only 8% of my network is from Berkeley and the largest network (San Francisco) only comprises 17% of my network. Networks don't work for highly-mobile 25+ crowd because they don't live in pre-defined networks. (For once, I'm an example!)
...
I don't really understand why Facebook decided to make public search opt-out. OK, I do get it, but I don't like it. Those who want to be PUBLIC are more likely to change settings than those who chose Facebook for its perceived privacy. Why did Facebook go from default-to-privacy-protection to default-to-exposure? I guess I know the answer to this... it's all about philosophy.

The first excerpt illustrates the point well. Facebook worked well as a social tool in the rigid social contexts of high school and college but completely breaks down when you’re all grown up.  Of course, the Facebook folks know this is an issue for some of their users. However it may be a “problem” that they consider to be By Design and not a bug.

The second excerpt is there because I’m surprised that danah is unsure about why Facebook profiles will now appear in search results. There are a lot of people for whom their social network profile is their primary or only online presence. Even for me, besides my blog(s), my Facebook profile is the only online identity Web which I keep updated regularly. It totally makes sense for Facebook to capitalize on this by making it so that everytime you search for a person whose primary presence is on their site, you get an ad to join their service [since only the fact that the person has a Facebook profile is exposed]. In addition, if you want to contact the person directly, you’re a lot better off joining Facebook and sending the person a private message than posting a comment on their blog [if they have one] or hoping that they’ve exposed their email address somewhere on the Web that isn’t their profile.

Update: The ability to expose a Limited Profile does render moot a lot of the points I just raised above. However making it a separate option from the privacy settings for the profile and incorrectly stating that your friends can always see your contact information makes it less likely to be used by users who are concerned about their privacy. Another example of a design flaw that is likely considered to be By Design according to the Facebook team.

Now playing: Metallica - The Unforgiven

 

Yesterday the Wall Street Journal had an article entitled Why So Many Want to Create Facebook Applications which gives an overview of the burst of activity surrounding the three month old Facebook platform. If a gold rush, complete with dedicated VC funds targetting widget developers, around building embedded applications in a social networking site sounds weirdly familar to you, that’s because it is. This time last year people were saying the same thing about building MySpace widgets. The conventional wisdom at the time was that sites like YouTube (acquired for $1.65 billion) and PhotoBucket (acquired for $250 million) rose in popularity due to their MySpace widget strategy.

So, why would developers who’ve witnessed the success of companies developing MySpace widgets rush to target a competing social networking site that has less users and requires more code to integrate with the site? The answer is that MySpace made the mistake of thinking that they were a distribution channel instead of a platform. If you are a distribution channel, you hold all the cards. Without you, they have no customers. On the other hand, if you are a platform vendor you realize that it is a symbiotic relationship and you have to make people building on your platform successful because of [not in spite of] your efforts.

Here are the three classic mistakes the folks at MySpace made which made it possible for Facebook to steal their thunder and their widget developers.

1. Actively Resent the Success of Developers on Your Platform: If you are a platform vendor, you want developers building on your platform to be successful. In contrast, MySpace’s executives publicly griped about the success of sites like YouTube and PhotoBucket that were “driven off the back of MySpace” and bragged about building competing services which would become as popular as them since “60%-70% of their traffice came from MySpace”. In a sign that things may have gotten out of hand is when MySpace blocked PhotoBucket widgets only to acquire the site a month later, indicating that this was an aggresive negotiation tactic intended to scare off potential buyers.  

2. Limit the Revenue Opportunities of Developers on Your Platform: MySpace created all sorts of restrictions to make it difficult for widget developers to actually make money directly from the site. For one, they blocked any widget that contained advertising even though advertising is the primary way to make money on the Web. Secondly, they restricted the options widgets had in linking back to the widget developers website thus driving users to where they could actually show them ads. Instead of trying to create a win<->win situation for widget developers (MySpace gets free features thus more engagement from their users, widget developers get ad revenue and traffic) the company tipped the balance excessively in their favor with little upside for widget developers.  

3. Do Not Invest in Your Platform: For a company that depends so much on developers building tiny applications that integrate into their site, it’s quite amazing that MySpace does not provide any APIs at all. Nor do they provide a structured way for their users to find, locate and install widgets. It turns out that Fox Interactive Media (MySpace’s parent company) did build a widget platform and gallery but due to internal politics these services are not integrated. In fact, one could say that MySpace has done as little as possible to making developing widgets for their platform a pleasant experience for developers or their users. 

This is pretty much the story of all successful technology platforms that fall out of favor. If you do not invest in your platform, it will become obsolete. If people are always scared that you will cut off their air supply out of jealousy, they’ll bolt the first chance they get. And if people can’t make money building on your platform, then there is no reason for them to be there in the first place. Don’t make the same mistakes.

Now playing: 50 Cent - Many Men (Wish Death)

 

Now playing: Pastor Troy - Vica Versa (feat. Peter The Disciple)

 

The Facebook developer blog has a post entitled Change is Coming which details some of the changes they've made to the platform to handle malicious applications including

Requests

We will be deprecating the notifications.sendRequest API method. In its place, we will provide a standard invitation tool that allows users to select which friends they would like to send a request to. We are working hard on multiple versions of this tool to fit into different contexts. The tool will not have a "select all" button, but we hope it enables us to increase the maximum number of requests that can be sent out by a user. The standardized UI will hopefully make it easier for users to understand exactly what they are doing, and will save you the trouble of building it yourself.

Notifications

Soon we will be removing email functionality from notifications.send, though the API function itself will remain active. In the future, we may provide another way to contact users who have added your app, as we know that is important. Deceptive and misleading notifications will continue to be a focus for us, and we will continue to block applications which behave badly and we will continue to iterate on our automated spam detection tools. You will also see us working on ways to automatically block deceptive notifications.

It looks like some but not all of the most egregious behavior is being targetted which is good. Specifically, I  wonder what is meant by deprecating the notifications.sendRequest API. When I think of API deprecation, I think of @deprecated in Java and Obsolete in C#, neither of which prevent the API from being used.

One of my biggest gripes with the site is the number of “friend requests” I get from applications with no way to opt out of getting these requests. However it doesn’t seem that this has been eliminated. Instead an API is being replaced with a UI component but the API isn’t even going away. I hope there is a follow up post where they describe the opt-out options they’ve added to the site so users can opt-out of getting so many unsolicited requests.

Now playing: Big Pun - Punish Me

 

Robert Scoble has a blog post up entitled Why Mahalo, TechMeme, and Facebook are going to kick Google’s butt in four years where he argues that search based on social graphs (e.g. your Facebook relationships) or generated by humans (e.g. Mahalo) will eventually trump Google's algorithms. I'm not sure I'd predict the demise of Google but I do agree that the social graph can be used to improve search and other aspects of the Internet experience, in fact I agree so much that was the topic of my second ThinkWeek paper which I submitted earlier this year (Microsoft folks can find it here).

However I don’t think Google’s main threat from sites like Facebook is that they may one day build social graph powered search that beats Google’s algorithms. Instead it is that these sites are in direct conflict with Google’s mission to

organize the world's information and make it universally accessible and useful.

because they create lots of valuable content that Google can not access. Google has branched out of Web search into desktop search, enterprise search, Web-based email and enterprise application hosting all to fulfill this mission.

The problem that Google faces with Facebook is pointed out quite well in Jason Kottke’s post Facebook vs. AOL, redux where he writes

Think of it this way. Facebook is an intranet for you and your friends that just happens to be accessible without a VPN. If you're not a Facebook user, you can't do anything with the site...nearly everything published by their users is private. Google doesn't index any user-created information on Facebook.²

and in Jeff Atwood's post Avoiding Walled Gardens on the Internet which contains the following excerpt

I occasionally get requests to join private social networking sites, like LinkedIn or Facebook. I always politely decline…public services on the web, such as blogs, twitter, flickr, and so forth, are what we should invest our time in. And because it's public, we can leverage the immense power of internet search to tie it all-- and each other-- together.

What Jason and Jeff are inadvertantly pointing out is that once you join Facebook, you immediately start getting less value out of Google’s search engine. This is a problem that Google cannot let continue indefinitely if they plan to stay relevant as the Web’s #1 search engine.

What is also interesting is that thanks to efforts of Google employees like Mark Lucovsky, I can use Google search from within Facebook but without divine intervention I can’t get Facebook content from Google’s search engine. If I was an exec at Google, I’d worry a lot more about the growing trend of users creating Web content where it cannot be accessed by Google than all the “me too” efforts coming out of competitors like Microsoft and Yahoo!.

The way you get disrupted is by focusing on competitors who are just like you instead of actually watching the marketplace. I wonder how Google will react when they eventually realize how deep this problem runs?

Now playing: Metallica - Welcome Home (Sanitarium)

 

This morning there were a number of news stories about collaboration between Windows Live and Bebo. These news stories didn’t tell the whole story. Articles such as C|Net’s Bebo's new instant messaging is Microsoft-flavored and TechCrunch’s Windows Live Messaging Comes to Bebo give the impression that the announcement was about instant messaging. However there was much more to the announcement. The agreement between Windows Live and Bebo spans two areas; social network portability and interop between Web-based IM and Windows Live Messenger. 

1. Social Network Portability: As I’ve mentioned before a common practice among social networking sites is to ask users for their log-in credentials for their email accounts so that the social networking sites can screen scrape the HTML for the address book and import the user’s contact list into the social networking site. There are a number of problems with this approach, the main one being that the user is simply moving data from one silo to another without being able to get their contact list back from the social network and into their email client. There’s also the problem that this approach makes users more susceptible to phishing since it encourages them to enter their log-in credentials on random sites.  Finally, the user isn’t in control of how much data is pulled from their address book by the social network or how often it is pulled.

   The agreement between Windows Live and Bebo enables users to utilize a single contact list across both sites. Their friends in Bebo will be available as their contacts in Windows Live and vice versa. This integration will be facilitated by the Windows Live Contacts API which implements a user-centric access control model where the user grants applications permission to access and otherwise manipulate their contact list.

2. Web-based IM and Windows Live Messenger interoperability: Users of Bebo that are also Windows Live Messenger users can opt in to getting notifications from Bebo as alerts in their desktop IM client. In addition, these users can add an “IM Me” button to their profile which allows people browsing their profile on the Web to initiate an IM conversation with them using a Microsoft-provided Web IM widget on the Bebo website which communicates with the Windows Live Messenger client on the profile owner’s desktop.

   The above scenarios were demoed at this year's MIX '07 conference during the session Broaden Your Market with Windows Live. The current plan is for the APIs for interacting with the Windows Live Messenger service and the IM widgets that can be embedded within a non-Microsoft website that power this scenario to be available via http://dev.live.com in the near future.

At the end of the day, it is all about putting users in control. We don’t believe that a user’s social graph should be trapped in a roach motel of our creation. Instead users should be able to export their contact lists from our service on their own terms and should be able to grow their social graph within Windows Live without having to exclusively use our services.

It’s your data, not ours. If you want it, you can have it. Hopefully, the rest of the industry comes around to this sort of thinking sooner rather than later.

Stay tuned, there’s more to come.

Now playing: Gucci Mane - So Icy (feat. Young Jeezy)

 

Brad Fitzpatrick, the founder of LiveJournal, who recently left Six Apart for Google has published notes on what he's going to be working on moving forward. It is an interesting read entitled Brad's Thoughts on the Social Graph which contains the following excerpts

Currently if you're a new site that needs the social graph (e.g. dopplr.com) to provide one fun & useful feature (e.g. where are your friends traveling and when?), then you face a much bigger problem then just implementing your main feature. You also have to have usernames, passwords (or hopefully you use OpenID instead), a way to invite friends, add/remove friends, and the list goes on. So generally you have to ask for email addresses too, requiring you to send out address verification emails, etc. Then lost username/password emails. etc, etc. If I had to declare the problem statement succinctly, it'd be: People are getting sick of registering and re-declaring their friends on every site., but also: Developing "Social Applications" is too much work.

Facebook's answer seems to be that the world should just all be Facebook apps.
...
Goals:
1. Ultimately make the social graph a community asset, utilizing the data from all the different sites, but not depending on any company or organization as "the" central graph owner. 

1. Establish a non-profit and open source software (with copyrights held by the non-profit) which collects, merges, and redistributes the graphs from all other social network sites into one global aggregated graph. This is then made available to other sites (or users) via both public APIs (for small/casual users) and downloadable data dumps, with an update stream / APIs, to get iterative updates to the graph (for larger users)

...
Non-Goals:

1. The goal is not to replace Facebook. In fact, most people I've talked to love Facebook, just want a bit more of their already-public data to be more easily accessible, and want to mitigate site owners' fears about any single data/platform lock-in. Early talks with Facebook about participating in this project have been incredibly promising. 

It seems to me that Facebook is the new Microsoft in that there are now a significant amount of people who are either upset at the level of "lock-in" they have created or are just plain jealous of their "wealth" who have created dedicated efforts to break their hegemony. It'll be interesting watching this play out.

From my perspective, I'm skeptical of a lot of the talk about social network portability because the conversation rarely seems to be user centric. Usually it's creators of competing services who are angry about "lock-in" because they can't get a new user's contacts from another service and spam them to gain "viral growth" for their service. As for the various claims of social network overload only the power users and geeks who join a new social network service a month (WTF is Dopplr?) have this problem.

A real social network is a community and users don't change communities at the drop of a hat. What I find more interesting is being able to bridge these communities instead of worrying about the 1% of users who hop from community to community like crack addled humming birds skipping from flower to flower.

I'll put it this way, when it comes to email which is more important? The ability to send emails to people regardless of what email service or mail client they use or the ability to import your contact list from one free email service into another when you switch service providers?

 

Disclaimer: This blog post does not reflect future product announcements, technical strategy or advice from my employer. Disregard this disclaimer at your own risk.

In my previous post Some Thoughts on Open Social Networks, I gave my perspective on various definitions of "open social network" in response to the Wired article Slap in the Facebook: It's Time for Social Networks to Open Up. However there was one aspect of the article that I overlooked when I first read it. The first page of the article ends with the following exhortation.

We would like to place an open call to the web-programming community to solve this problem. We need a new framework based on open standards. Think of it as a structure that links individual sites and makes explicit social relationships, a way of defining micro social networks within the larger network of the web.

This is a problem that interests me personally. I have a Facebook profile while my fiancée has a MySpace profile. Since I’m now an active user of Facebook, I’d like her to be able to be part of my activities on the site such as being able to view my photos, read my wall posts and leave wall posts of her own. I could ask her to create a Facebook account, but I already asked her to create a profile on Windows Live Spaces so we could be friends on that service and quite frankly I don’t think she’ll find it reasonable if I keep asking her to jump from social network to social network because I happen to try out a lot of these services as part of my day job. So how can this problem be solved in the general case?

OpenID to the Rescue

This is exactly the kind of problem that OpenID was designed to solve.  The first thing to do is to make sure we all have the same general understanding of how OpenID works. It's basically the same model as Microsoft Passport Windows Live ID, Google Account Authentication for Web-Based Applications and Yahoo! Browser Based Authentication. A website redirects you to your identity provider, you authenticate yourself (i.e. login) on your identity providers site and then are redirected back to the referring site along with your authentication ticket. The ticket contains some information about you that can be used to uniquely identify you as well as some user data that may be of interest to the referring site (e.g. username).

So how does this help us? Let’s say MySpace was an OpenID provider which is a fancy way of saying that I can use my MySpace account to login to any site that accepts OpenIDs . And now let’s say Facebook was a site that accepted OpenIDs  as an identification scheme. This means that I could add my fiancée to the access control list of people who could view and interact with my profile on Facebook by using the URL to her MySpace profile as my identifier for her.  So when she tries to access my profile for the first time, she is directed to the Facebook login page where she has the option of logging in with her MySpace credentials. When she chooses this option she is directed to the MySpace login page. After logging into MySpace with proper credentials, she is redirected back to Facebook  and gets a pseudo-account on the service which allows her to participate in the site without having to go through an account creation process.

Now that the user has a pseudo-account on Facebook, wouldn’t it be nice if when someone clicked on them they got to see a Facebook profile? This is where OpenID Attribute Exchange can be put to use. You could define a set of required and optional attributes that are exchanged as part of social network interop using OpenID. So we can insert an extra step [which is may be hidden from the user] after the user is redirected to Facebook after logging into MySpace where the user’s profile information is requested. Here is an example of the kind of request that could be made by Facebook after a successful log-in attempt by a MySpace user.

openid.ns.ax=http://openid.net/srv/ax/1.0
openid.ax.type.fullname=http://example.com/openid/sn_schema/fullname
openid.ax.type.gender=http://example.com/openid/sn_schema/gender
openid.ax.type.relationship_status=http://example.com/openid/sn_schema/relationship_status
openid.ax.type.location=http://example.com/openid/sn_schema/location
openid.ax.type.looking_for=http://example.com/openid/sn_schema/looking_for
openid.ax.type.fav_music=http://example.com/openid/sn_schema/fav_music
openid.ax.count.fav_music=3
openid.ax.required=fullname,gender,location
openid.ax.if_available=relationship_status,looking_for,fav_music

which could return the following results

openid.ns.ax=http://openid.net/srv/ax/1.0
openid.ax.type.fullname=http://example.com/openid/sn_schema/fullname
openid.ax.type.gender=http://example.com/openid/sn_schema/gender
openid.ax.type.relationship_status=http://example.com/openid/sn_schema/relationship_status
openid.ax.type.location=http://example.com/openid/sn_schema/location
openid.ax.type.looking_for=http://example.com/openid/sn_schema/looking_for
openid.ax.type.fav_music=http://example.com/openid/sn_schema/fav_music
openid.ax.value.fullname=Jenna
openid.ax.value.gender=F
openid.ax.value.relationship_status=Single
openid.ax.value.location=Seattle, WA, United States
openid.ax.value.looking_for=Friends
openid.ax.value.fav_music=hiphop,country,pop
openid.ax.update_url=http://www.myspace.com/url_to_send_changes_made_to_profile

With the information returned by MySpace, one can now populate a place holder Facebook profile for the user.

Why This Will Never Happen

The question at the tip of your tongue is probably “If we can do this with OpenID today, how come I haven’t heard of anyone doing this yet?”.  As usual when it comes to interoperability, the primary reasons for lack of interoperability are business related and not technical.  When you look at the long list of Open ID providers, you may be notice that there is no similar long list of sites that accept OpenID  credentials. In fact, there is no such list of sites readily available because the number of them is an embarassing fraction of the number of sites that act as Open ID providers. Why this discrepancy?

If you look around, you’ll notice that the major online services such as Yahoo! via BBAuth, Microsoft via Passport Windows Live ID, and AOL via OpenID all provide ways for third party sites to accept user credentials from their sites. This increases the value of having an account on these services because it means now that I have a Microsoft Passport Windows Live ID I not only can log-in to various Microsoft properties across MSN and Windows Live but also non-Microsoft sites like Expedia. This increases the likelihood that I’ll get an account with the service which makes it more likely that I’ll be a regular user of the service which means $$$. On the other hand, accepting OpenIDs does the exact opposite. It actually reduces the incentive to create an account on the site which reduces the likelihood I’ll be a regular user of the site and less $$$. Why do you think there is no OpenID link on the AOL sign-in page even though the company is quick to brag about creating 63 million OpenIDs?

Why would Facebook implement a feature that reduced their user growth via network effects? Why would MySpace make it easy for sites to extract user profile information from their service? Because openness is great? Yeah…right.

Openness isn’t why Facebook is currently being valued at $6 billion nor is it why MySpace is currently expected to pull in about half a billion in revenue this year. These companies are doing just great being walled gardens and thanks to network effects, they will probably continue to do so unless something really disruptive happens.   

PS: Marc Canter asks if I can attend the Data Sharing Summit between Sept. 7th – 8th. I’m not sure I can since my wedding + honeymoon is next month. Consider this my contribution to the conversation if I don’t make it.

Now playing: Wu-Tang Clan - Can It Be All So Simple

 

A few weeks ago, one of our execs at work asked me to think about "open" social networks. Since my day job is working on the social networking platform that underlies Windows Live Spaces and other Windows Live properties, it makes sense that if anyone at Microsoft is thinking about making our social networks "open" it should be me. However I quickly hit a snag. After some quick reading around, I realized that there isn't really a common definition of what it means for a social networking service to be "open". Instead, it seems we have a collection of pet peeves that various aggrieved parties like to blame on lack of openness. For example, read the Wired article Slap in the Facebook: It's Time for Social Networks to Open Up and compare it to this post on Read/Write Web entitled PeopleAggregator and Open Social Network Systems. Both articles are about "open" social networks yet they focus on completely different things. Below are my opinions on the various definitions of "open" in the context of social networking sites

1. Content Hosted on the Site Not Viewable By the General Public and not Indexed by Search Engines:  As a user of Facebook, I consider this a feature not a bug. I've mentioned in previous blog postings that I don't think it is a great idea that all the stuff being published by teenagers and college students on the Web today will be held against them for the rest of their lives. Especially since using search engines to do quick background searches on potential hires and dates is now commonplace. Personally, I've had several negative experiences posting personal content to the public Web including

   1. fresh of out of college, I posted a blog post about almost hooking up with some girl at a nightclub and a heated email discussion I had with someone at work. It was extremely awkward to have both topics come up in conversations with fellow coworkers over the next few days because they'd read my blog.
   2. a few months ago I posted some pictures from a recent trip to Nigeria and this ignited a firestorm of over a hundred angry comments filled with abuse and threats to myself and my family because some Nigerians were upset that the president of Nigeria has servants domestic staff. I eventually made the pictures non-public on Flickr after conferring with my family members in Nigeria.
      
   3. around the same time I posted some pictures of my fiancée and I on my Windows Live Space and each picture now has a derogatory comment attached to it.

   At this point I've given up on posting personal pictures or diary like postings on the public Web. Facebook is now where I share pictures.

   When we first launched Windows Live Spaces, there was a lot of concern across the division when people realized that a significant portion of our user base was teenage girls who used the site to post personal details about themselves including pictures of themselves and friends. At the end we decided, like Facebook, that the default accessibility for content created by our teenage users (i.e. if they declare their age in their profile) would be for it to only be visible to people in their social network (i.e. Windows Live Messenger buddies and people in their Windows Live Spaces friends list). I think it is actually pretty slick that on Facebook, you can also create access control lists with entries like "anyone who's proved they work at Microsoft". 

2. Inability to Export My Content from the Social Network: This is something that geeks complain about especially since they tend to join new social networking sites on a new basis but for the most part there isn't a lot of end user demand for this kind of functionality based on my experience working closely with the folks behind Windows Live Spaces and keeping an eye on feedback about other social networking sites. There are two main reasons for this, the first is that there is little value of having the content that is unique to the social network site outside of the service. For example, my friends list on Facebook is only useful in the context of that site. The only use for it outside the service would be for a way to bootstrap a new friends list by spamming all my friends on Facebook to tell them to join the new site.  Secondly, danah boyd has pointed out in her research that many young users of social networking sites consider their profiles to be ephemeral, to them not being able to just port your profile from MySpace to Facebook isn't a big deal because you're starting over anyway. For working professionals, things are a little different since they may have created content that has value outside the service (e.g. work-related blog postings related to their field of endeavor) so allowing data export in that context actually does serve a legitimate user need. 
   

3. Full APIs for Extracting and Creating Content on the Social Network: With the growth in popularity and valuations of social networking sites, some companies have come to the conclusion that the there is an opportunity for making money by becoming meta-social network sites which aggregate a user's profiles and content from multiple social networking sites. There are literally dozens of Social Network Profile aggregators today and it is hard to imagine social networking sites viewing them as anything other than leeches trying to steal their page views by treating them as dumb storage systems. This is another reason why most social network services primarily focus on building widget platforms or APIs that enable you to create content or applications hosted within the site but don't give many ways to programmatically get content out.  

   Counter examples to this kind of thinking are Flickr and YouTube which both provide lots of ways to get content in and out of their service yet became two of the fastest growing and most admired websites in their respective categories. It is clear that a well-thought out API strategy that drives people to your site while not restricting your users combined with a great user experience on your website is a winning combination. Unfortunately, it's easier said than done.
   

4. Being able to Interact with People from Different Social Networks from Your Preferred Social Network: I'm on Facebook and my fiancée is on MySpace. Wouldn't it be great if we could friend each other and send private messages without both being on the same service?

   It is likely that there is a lot of unvoiced demand for this functionality but it likely won't happen anytime soon for business reasons not technical ones. I suspect that the concept of "social network interop" will eventually mirror the current situation in the instant messaging world today.

   • We'll have two or three dominant social networking services with varying popularity in different global markets with a few local markets being dominated by local products.
   • There'll be little incentive for a dominant player to want to interoperate with smaller players. If interop happens it will be between players that are roughly the same size or have around the same market strength.
     
   • A small percentage of power users will use services that aggregate their profiles across social networks to get the benefits of social network interoperability. The dominant social networking sites will likely ignore these services unless they start getting too popular.
     
   • Corporate customers may be able to cut special deals so that their usage of public social networking services does interoperate with  whatever technology they use internally.
     

   Since I've assumed that some level of interoperability across social networking sites is inevitable, the question then is what is this functionality and what would the API/protocols look like? Good question.
   

 

Yesterday, I was chatting with a former co-worker about Mary Jo Foley's article Could a startup beat Microsoft and Google to market with a ‘cloud OS’? and I pointed out that it was hard to make sense of the story because she seemed to be conflating multiple concepts then calling all of them a "cloud OS". It seems she isn’t the only one who throws around muddy definitions of this term as evidenced by C|Net articles like Cloud OS still pie in the sky and blog posts from Microsoft employees like Windows Cloud! What Would It Look Like!? 

I have no idea what Microsoft is actually working on in this space and even if I did I couldn't talk about it anyway. However I do think it is a good idea for people to have a clear idea of what they are talking about when the throw around terms like "cloud OS" or "cloud platform" so we don't end up with another useless term like SOA which means a different thing to each person who talks about it. Below are the three main ideas people often identify as a "Web OS", "cloud OS" or "cloud platform" and examples of companies executing on that vision.

WIMP Desktop Environment Implemented as a Rich Internet Application (The YouOS Strategy)

Porting the windows, icons, menu and pointer (WIMP) user interface which has defined desktop computing for the last three decades to the Web is seen by many logical extension of the desktop operating system. This is a throwback to the Oracle's network computer of the late 1990s where the expectation is that the average PC is not much more than a dumb terminal with enough horse power to handle the display requirements and computational needs of whatever rich internet application platform is needed to make this work.

A great example of a product in this space is YouOS. This seems to be the definition idea of a "cloud os" that is used by Ina Fried in the C|Net article Cloud OS still pie in the sky.

My thoughts on YouOS and applications like it were posted a year ago, my opinion hasn't changed since then.

Platform for Building Web-based Applications (The Amazon Strategy)

When you look at presentations on scaling popular websites like YouTube, Twitter, Flickr, eBay, etc it seems everyone keeps hitting the same problems and reinventing the same wheels. They all start of using LAMP thinking that’s the main platform decision they have to make. Then they eventually add on memcached or something similar to reduce disk I/O. After that, they may start to hit the limits of the capability of relational database management systems and may start taking data out of their databases, denormalizing them or simply repartition/reshard them as they add new machines or clusters. Then they realize that they now have dozens of machines in their data center when they started with one or two and managing them (i.e. patches, upgrades, hard disk crashes, dynamically adding new machines to the cluster, etc) becomes a problem.

Now what if someone who’d already built a massively scalable website and now had amassed a bunch of technologies and expertise at solving these problems decided to rent out access to their platform to startups and businesses who didn’t want to deal with a lot of the costs and complexities of building a popular Website beyond deciding whether to go with LAMP or WISC? That’s what Amazon has done with Amazon Web Services such as EC2 ,S3, SQS and the upcoming Dynamo.  

The same way a desktop operating system provides an abstraction over the complexity of interacting directly with the hardware is the way Amazon’s “cloud operating system” insulates Web developers from a lot of the concerns that currently plague Web development outside of actually writing the application code and dealing with support calls from their customers.

My thoughts on Amazon’s Web Services strategy remain the same. I think this is the future of Web platforms but there is still a long way to go for it to be attractive to today’s startup or business.

NOTE: Some people have commented that it is weird for an online retailer to get into this business. This belies a lack of knowledge of the company’s history. Amazon has always been about gaining expertise at some part of the Web retailer value chain then opening that up to others as a platform. Previous examples include the Amazon Honor System which treats their payment system as a platforn, Fulfillment by Amazon which treats their warehousing and product shipping system as a platform, zShops allows you to sell your products on their Website as well as more traditional co-branding deals where other sites reused their e-commerce platform such as Borders.com.

Web-based Applications and APIs for Integrating with Them (The Google Strategy)

Similar to Amazon, Google has created a rich set of tools and expertise at building and managing large scale websites. Unlike Amazon, Google has not indicated an interest in renting out these technologies and expertise to startups and businesses. Instead Google has focused on using their platform to give them a competitive advantage in the time to market, scalability and capabilities of their end user applications. Consider the following… 

If I use GMail for e-mail, Google Docs & Spreadsheets for my business documents, Google Calendar for my schedule, Google Talk for talking to my friends, Google search to find things on my desktop or on the Web and iGoogle as my start page when I get on the computer then it could be argued that for all intents and purposes my primary operating system was Google not Windows. Since every useful application eventually becomes a platform, Google’s Web-based applications are no exception. There is now a massive list of APIs for interacting and integrating with Google’s applications which make it easier to get data into Google’s services (e.g. the various GData APIs) or to spread the reach of Google’s services to sites they don’t control  (e.g. widgets like the Google AJAX Search API and the Google Maps API).

In his blog post GooOS, the Google Operating System Jason Kottke argues that the combination of Google’s various Web applications and APIs [especially if they include an office suite] plus some desktop and mobile entry points into their services is effectively a Google operating system. Considering Google’s recent Web office leanings and its bundling deals with Dell and Apple, it seems Jason Kottke was particularly prescient given that he wrote his blog post in 2004.

Now playing: Fabolous - Do The Damn Thing (feat. Young Jeezy)

 

Jerff Atwood has a blog post entitled Google's Number One UI Mistake where he writes

Google's user interface minimalism is admirable. But there's one part of their homepage UI, downloaded millions of times per day, that leaves me scratching my head:

Does anyone actually use the "I'm Feeling Lucky" button? I've been an avid Google user since 2000; I use it somewhere between dozens and hundreds of times per day. But I can count on one hand the number of times I've clicked on the "I'm Feeling Lucky" button.
...
I urge us to Omit Needless Buttons. I hope the "I'm Feeling Lucky" button isn't considered a sacred cow at Google. Removing it would be one small step for Google, but a giant collective improvement in the default search user interface for users around the world.

A number of Jeff’s readers chimed in with the reason why Jeff is mistaken about this being a user interface “mistake” on Google’s part.

Robert Cooper wrote the following as part of his response

Google actually brought that up when they were user testing the UI a while back. (I can't remember where I read the story) But the crux of it was, the users actually LIKE having that button there, and rated the search interface lower when it wasn't there.

Geoff Wilson also validates the idea that this is on purpose in his response which is excerpted below

I remember listening to a google staffer on a podcast a while ago, and the reason the "I feel lucky" button stays is not its click through rates, but the message it sends about google's corporate culture.

Apparently, users felt Google was more human by having something quirky like that on the front page. It doesn't have lots of people click on it, but it encouraged more people to click on the main "Google Search" button.

and here’s one more comment excerpt, this time from Max

They can't remove that button now, it's part of their branding, part of their identity. They would be no more likely to stop updating their logo with colorful playful holiday-appropriate images (the shock of all that wasted bandwidth as proxy servers everywhere have to recache it!) than they would to remove that button.

Needless to say, the connection that branding builds between the user and the service/company is important. This button and the feeling of playfulness that it reflects on google even moreso. The few extra bytes of page size and the small extra cognitive load is more than made up for the positive feelings it engenders in users.

 Building a connection with the users of your software is important. There’s nothing like a little playfulness and humor to make your company and your software seem a lot more friendly to your customers.

Now playing: Timbaland - The Way I Are (Extended mix) (feat. Keri Hilson, Sebastian & D.O.E)

 

Yesterday I got offered an opportunity to interview Vint Cerf just before he gave his talk entitled Tracking the Internet into the 21st Century (link is to video of the talk) at Google's Kirkland offices. I got to ask the questions I blogged about yesterday and also learned about some of Vint Cerf's interests in Nigeria. Below are the questions I asked and his paraphrased answers to my questions.

Yesterday I got offered an opportunity to interview Vint Cerf just before he gave his talk entitled Tracking the Internet into the 21st Century (link is to video of the talk) at Google's Kirkland offices. I got to ask the questions I blogged about yesterday and also learned about some of Vint Cerf's interests in Nigeria. Below are the questions I asked and his paraphrased answers to my questions.

Q: Why did he decide to leave MCI, a company steeped in networking technology, to join Google, an advertising and search engine company, as Chief Internet Evangelist? 

A: The job title was not his doing. Larry, Sergey and Eric told him they wanted him to continue his efforts in encouraging the growth of the Internet around the world and thought the title "Chief Internet Evangelist" best fit this position. There are 6.5 billion people on the planet today yet there are only 1 billion people on the Internet. Google would like to see the other 5 billion people on the Internet because the more people there are using the Internet, the better it is for Google. This is why the company needed a "Chief Internet Evangelist".

Vint Cerf spends a significant portion of his time encouraging implementations of the Internet. He travels all over the world meeting with senior government officials (presidents, ministers of information, etc) to recommend Internet friendly policies, discourage the rise of monopolistic or closed networks and encourage domestic/foreign investments in fledgling markets where Internet usage hasn't taken off. For example, he is working with some charitable entities to donate solar powered Internet cafes to businesses in Nigeria to encourage the usage of the Internet in remote or underprivileged parts of the country.

One aspect of the Internet's growth which he didn't pay much attention to at first but does now is Internet enabled mobile phones. It is estimated that there will be 3 billion people with mobile phones by the end of the year. That is 3 billion people who could all be connected to the Internet, if Internet connectivity became ubiqitous on mobile devices within a few years.

Looking back at the past few years, it is clear that adding more users to the Internet increases the quantity and diversity of information on the Web. This trend has been hastened by the rise of the consumer as producer. We now have people who would be traditionally considered to be consumers producing content on blogs, video sharing sites like YouTube, and creating social networks on sites like Orkut and Facebook. Another interesting trend is the rise of virtual worlds like World of Warcraft and Second Life. In these worlds users are creating interesting economic and sociological experiments with fascinating consequences (e.g. gold farming in China). In fact, some college professors are encouraging their students to join these sites to test out economic and sociological theories in ways that simply weren't feasible in the past. An interesting idea would be to see if we could create virtual objects which were associated with and could influence objects in the real world. For example, a virtual university where the electron microscopes and telescope actually displayed image data from electron microscopes and telescopes in the real world. Maybe as an optimization we could cache large amounts of the astronomical data so multiple instances of the virtual telescope could be used at once but only rarely would the physical telescope have to be used so there wasn't resource contention. Given that Google already has already started partnering with NASA to store and process large amounts of astronomical data this may be something that the company could be interested in trying out in the future.

Q: He has spoken out on Google's behalf in favor of net neutrality. However there seem to be many different definitions of Net Neutrality, some of which imply that having different tiers for Quality of Service is OK and some of which don't, which definition is Google in favor of and why?

A: Google didn't start the network neutrality debate, AT&T's CEO Ed Whitacre did when he claimed that companies like Google are getting a "free ride" on his network. This seems backwards to Vint Cerf since AT&T's customers pay broadband fees so they can access any site on the Internet. Expecting companies to pay AT&T's for access to its paying customers who are already paying for access to the Internet is old school "Telephone Think" that harkens back to the monopoly days of Ma Bell.

The philosophy of the Internet comes from a completely different roots. The philosophy was pretty much "Here are the specs, if you can figure out how to implement our protocols and can connect to our network then it's all good". This open philosophy is what enabled the growth of the Internet and eventually led to commercial entities [including telcos like AT&T] to become part of the network.

Vint Cerf and Google's definition of network neutrality has these five basic pillars

1. users should be able to reach any service connected to the network
2. users should be able to run any application and connect to the network (of course, this doesn't apply to applications that violate the law)
3. it is OK to charge for higher speed connections to the network.
4. operators should not discriminate against services a user is trying to access by varying the user's QoS or access charges when accessing that service.
5. Discrimination against a type of service (e.g. all video traffic has different QoS) is OK but singling out specific sites is not.

A number of ISPs already break these rules yet are not upfront with users that they are not getting a full Internet experience. Some claim that these rules limit the ability of ISPs to prevent denial of service attacks, fight spam and perform other activities that protect their networks. Google believes that such protections can still be enforced but should be done at the application layer and not by discriminating against packets. As for ISPs that believe this limits their ability to provide value added services [such as video sharing] the response is that competition should be based on providing innovative services instead of by artificially limiting the capabilities of your competitors because you control the network pipes.

Google wants the Internet to be an open environment which allows for innovation. They believe this is important to the Internet's growth.

Q: Google just pledged to spending up to $4.6 billion to license the 700MHz wireless spectrum in what the company has described as the most significant auction of wireless spectrum in history by the U.S. federal government. Why is this auction so significant and what kind of services can we expect from Google if it wins the auction?

A: [Editor's Note: Why this auction is significant is summarized quite well in David Stone's post Vint Cerf and the 700MHz Spectrum]
Google's primary goal is to increase the openness of Internet-connected networks around the world. This is why they've committed at least $4.6 billion to licensing the 700MHz wireless spectrum.

It isn't quite clear what business model Google will use the 700MHz spectrum for if they win the auction. Whatever they end up deciding, it will honor the four principles of open platforms they have espoused with regards to wireless networks. It is quite likely that leasing out this capacity is one of the business models Google will try out. However due to the propagation characteristics of the 700MHz band, it is likely that different business models will have to apply in rural versus urban environments.

Q: Net neutrality gets a lot of press, however there are other issues facing the Internet as well. What keeps him up at night besides net neutrality? Botnets? Government censorship of the Internet? Concerns that we’ll never upgrade from the current version of the Internet since it is already so entrenched around the world?

A: The rise of botnets, domain name security and the problems related to handling internationalized domain names (IDNs) are at the top of the list of problems facing the Internet that concern Vint Cerf. The IDN problem is particularly pernicious because not only did we have to figure out how to support non-ASCII characters in a system that was never designed to support them but once a way was found, the IDN homograph attack was born which promptly reverse most of the gains.

Switching to IPv6 is also an issue facing the Internet that we will have to deal with sooner than most people expect. Some have predicted that at the current rate of allocation by ICAAN we will run out of IPv4 addresses by 2011. At that point, it will start to look a lot more attractive to switch to IPv6. There may be workarounds such as people leasing some of the blocks they've been allocated to other parties but this leads to interesting problems for routers since the routing tables will be screwed up and will have to be tampered with to adjust to these shenanigans. Given that pretty much all the major operating systems (Vista, Mac OS X, *nix, etc) and networking equipment manufacturers (e.g. Juniper, Cisco) support IPv6, it's really up to the ISPs and they likely won't make any moves without customer demand. Unfortunately for them, things are liably to get ugly in the next five years or so and they may have to change their minds.

 

One of the side effects of working for a large, successful, multinational corporation is that you tend to lose your sense of perspective. For example, take this post from the Official Google blog entitled Cookies: expiring sooner to improve privacy which states

We are committed to an ongoing process to improve our privacy practices, and have recently taken a closer look at the question of cookie privacy. How long should a web site "remember" cookie information in its logs after a user's visit? And when should a cookie expire on your computer? Cookie privacy is both a server and a client issue.

On the server side, we recently announced that we will anonymize our search server logs — including IP addresses and cookie ID numbers — after 18 months.
...
In the coming months, Google will start issuing our users cookies that will be set to auto-expire after 2 years, while auto-renewing the cookies of active users during this time period. In other words, users who do not return to Google will have their cookies auto-expire after 2 years. Regular Google users will have their cookies auto-renew, so that their preferences are not lost. And, as always, all users will still be able to control their cookies at any time via their browsers.

What’s is interesting in this post is that Google has sidestepped the actual privacy issue that has many people concerned about the amount of knowledge the company has about Internet users. Numerous bloggers such as Nelson Minar, Shelley Powers and John Dowdell have already pointed how this change doesn't actually change the status quo. In today’s world, Google knows more about most Internet users than their spouse. Thanks to the magic of HTTP cookies Google knows remembers...

• What searches you perform on Google.com
• What blogs you read if they are hosted on Blogspot, use Feedburner for their RSS feeds or are in your subscription list in Google Reader
• What websites you visit regularly via AdSense ads or usage of Google Analytics on these sites 
• Your communication history if you communicate with people via GMail or GTalk
• Your schedule via Google Calendar
• Your business plans via Google Docs & Spreadsheets
• Who the people in your social network are if you use Orkut
• Your income if you host AdSense ads
• Your purchase history on sites that use Google Checkout
• Your address and your favorite locations if you use Google Maps
• What online videos you watch on YouTube

You pretty much can't use the Web without running into a Google cookie. So it seems somewhat facetious for Google to claim that if you can avoid using the Internet for two years then they'll forget everything they are storing about you. Oops, actually they don't even claim that. They simply claim that they’ll stop associating your old data with your current usage, if you manage to avoid hitting a Google cookie for two years. 

If Google really wanted to address people's privacy concerns they’d blog about how they plan to use and protect all the data they are collecting about Internet users from all of their services instead of making ineffective token gestures that are specific to one service.      

Now playing: Lil Boosie & Webbie - Wipe Me Down (feat. Foxx)

 

Yesterday I saw a blog post by John Battelle where he said he turned down a chance to be part of discussion on a television show about whether Facebook is the next Google. This seems like a pretty absurd assertion on the face of it and the reason is snuggled somewhere in the middle of Robert Scoble's post Why Facebook, why now? where he writes

Everything I do in Facebook is about interacting with people. For instance, at the top of my Facebook inbox right now is Ryan Coomer. The advertising next to him says “Try Forex Trading Today.” There is absolutely NO connection between who Ryan is and the advertising that’s put next to him.

Imagine if advertisers could “buy people.” I just clicked on Ryan’s profile, hes into Running and Golf. Why don’t ads for running and golf gear get put onto his profile? Wouldn’t that make sense? He’s also a software developer. Where’s the Visual Studio advertisement? He’s into video games. Where’s the Halo 3 advertisement?

Translation: Facebook needs an advertising platform and it needs one in the worst way. I’m not going to even look at the ads until the ads are tied to the people on Facebook. Facebook knows what we’re into, put ads for those things onto our profiles and messages.

Robert Scoble is both right and wrong. He is right that Facebook needs an advertising platform but he is wrong about how it should be used. If Facebook has detailed demographic data about all its users then it makes the most sense to show the user what they are most interested in and not what the person they are currently interacting with on the site is most interested in. That's counter productive. I hate running and golf, I like G-Unit and alcopops. Which ads does it make more sense to show me if I'm browsing Ryan Coomer's profile?

Anyway, that's besides the point. Let's go back and look at what made Google such a compelling investment when they had their IPO. The company had

1. An extremely popular website that served a key need for its users. 
   
2. A way to make a metric ton of money from their popularity via search ads.

#1 is obvious but #2 needs further clarification. Search advertising created a paradigm shift in the advertising market in several ways. For one, it was the first time in the history of advertising that advertisers had a way to conclusively calculate the ROI of their advertising campaigns compared to other media (TV, newspapers, radio, sponsorships, etc). More importantly, it allowed advertisers to target people when they were in the process of making a buying related decision (i.e. doing research). For the most part advertising is often intrusive, irrelevant and often just plain annoying. On the other hand, ads shown when you are doing research for a commercial transaction are exactly what you want. It is quite telling when you look at highest paying search terms and see that it is for terms like "mortgage refinance", "dui" and "consolidate loans". When searching for those terms I want to see ads, in fact the more ads the better.  What is most interesting is that even though the majority of searches on the Web are non-commercial, this was still a goldmine which could only get more lucrative as more people got on the Web and advertisers eventually realized that their dollars needed to follow the eyeballs and search advertising was actually an easier expense to justify than all their other ad campaigns.

Now we know what made Google such an attractive investment back in the day which has now turned itself into a major force in the software industry and on the Web. So what does Facebook have?

1. An extremely popular website that served a key need for its users. 
   
2. The hope that way will be able to make a metric ton of money from their popularity via highly targeted ads based on all the rich demographic data they have about their users.

The initial thrust of #2 is that Facebook hasn't figured out how to make money yet. More importantly, if they do figure out how to build an ad platform it will likely be a system that is more complex than Google's Adwords and AdSense combined. When you think about it Adwords is pretty simple, allow people to bid on search terms then show these people's ads when people use those search terms. AdSense is similarly straightforward, use term extraction to convert the content on the page to the equivalent of search terms and show ads from the people who bid on ads. Of course, the devil is in the details and is mainly about optimizing how much money can be extracted from their advertisers.

On the other hand, if you want to show display ads based on demographic data you'll also need to build something very similar to a recommendation system as well as everything else I've described because you want to know that if I like wearing G-Unit clothing that I probably wouldn't mind seeing ads for Ecko Unlimited wear either or that if I like music by Young Jeezy that I probably would like music from T.I. or that if I've read Freakonomics I probably wouldn't mind reading The Tipping Point.

Now let's assume that the developers at Facebook figure out how to build an advertising platform that is more complex than Google's Adwords and AdSense combined. There is still the question of whether the "spray and pray" approach that is the hallmark of display advertising [even if you are sure that the ads will be about the topics the user is interested in]  is a more lucrative model than showing people ads when they are actually looking for something via search ads. I personally don't think it will be but I'm no expert and I'm definitely no fortune teller.

I suspect that the folks at Facebook will eventually realize how hard a problem it is to monetize their users in a way that justifies the exorbitant valuation that has now been placed on them. When that time comes, I wouldn't be surprised if they find some sucker to buy them based on the promise that they present as opposed to the value that they actually bring. Almost like when Yahoo! paid $5 billion for Broadcast.com which gave us the Mark Cuban we all know and love but also ended up as one of the top five worst billion dollar acquisitions of all time in the Internet space.

 

 

In my previous post I mentioned the various problems with relying on incubation teams to bring innovation into a product or organization. The obvious follow up question is that if carving off some subset of your team to work on the "next big thing" while the rest of your employees work on the boring bread and butter product(s) that pay the bills doesn't work, how do you revitalize an organizations products and make them innovative?

My advice is to look at companies within your industry that are considered innovative and see what you can learn from them. One such company is Google which is widely considered to be the most innovative company on Earth by many in the software industry. A number of Google's competitors have several internal groups whose job is to "incubate ideas" and foster innovation yet it seems that Google is the company most associated with innovation in the online space. For example, Yahoo! has Brickhouse and Yahoo! Research while Microsoft has Microsoft Research, Live Labs, Search Labs, and Windows Live Core among others. 

Below are some of the ways technology companies can follow their example without having to resort to some of their more eccentric practices like free food prepared by gourmet chefs and on-site massages, dry cleaning and oil changes to motivate your employees.

1. Everyone is Responsible for Innovation: There are several ways Google has created a culture where every technical employee feels that innovation is expected of them. First, there is the strong preference for people who have a track record of producing original ideas such as Ph.D's [who are required to produce original research which advances the state of the art as part of their thesis] and founders of Open Source projects (e.g. Spencer Kimball (GIMP), Aaron Boodman (Greasemonkey), and Guido Van Rossum (Python)). Secondly, employees are strongly encouraged but not required to spend 20% of their time on projects of their own design which are intended to benefit the company and/or its customers. Not only does this give employees an outlet for their creativity in a productive way, working on multiple projects at once gives developers a broader world view which makes it less likely that they will develop tunnel vision with regards to their primary project. Finally, Google has a single code base for all of their projects and developers are strongly encouraged to fix bugs or add features to any Google product they want even if they are not on the product team. This attitude encourages the cross pollination of ideas across the company and encourages members of the various product teams to keep an open mind about ideas from outside their particular box.
   

2. Good Ideas Often Come from Outside your Box: A lot of people in the software industry often criticize Microsoft for its practice of innovation through acquisition and have compiled lists of Microsoft's innovations that were actually acquisitions but the fact is that the road to success lies in being able to spot good ideas whether they come from within your company or without. Google is no exception to this rule as the following table of acquisitions and the Google products they resulted in shows

   Acquired Company/Product	Google Product
   Applied Semantics	Google AdSense
   Kaltix	Google Personalized Search
   Keyhole Corp.	Google Earth
   Where2	Google Maps
   ZipDash	Google Ride Finder
   2Web Technologies	Google Spreadsheets
   Upstartle	Google Docs
   Urchin Software Corporation + Measure Map	Google Analytics
   Zenter + Tonic Systems	Unreleased Google Web-based Presentation application

   As you can see from the above list, a lot of Google's much lauded products were actually the products of acquisitions as opposed to the results of internal incubation. Being able to conquer the NIH mentality is important if one wants to ensure that the products that exhibit the best ideas are produced by your company because quite often they won't originate from your company.  

3. Force Competition to Face the Innovator's Dilemma: One reason that a number of Google's products are considered innovative is that they challenge a number of pre-existing notions about software in certain categories. For example, when Gmail [a product of an engineer's 20% time spent on side projects] was first launched it was a shock to see a free Web-based email service qive users 1 gigabyte of free storage. A key reason that this was a shock was because most free Web-based email services gave users less than a hundredth of that amount of storage. This was because the business model for free email was primarily to give users a crappy user experience (2MB of storage, obnoxious advertising, etc) and then charge them for upgrading to a decent experience. Thus there was little incentive for the major players in the free email business to give free users lots of storage or a rich online experience because isn't how the business worked. Another example, that is likely to be a classic case study of the innovator's dilemma in the years to come is Google Docs & Spreadsheets vs. Microsoft Office. From the Wikipedia article on disruptive technology

   In low-end disruption, the disruptor is focused initially on serving the least profitable customer, who is happy with a good enough product. This type of customer is not willing to pay premium for enhancements in product functionality. Once the disruptor has gained foot hold in this customer segment, it seeks to improve its profit margin. To get higher profit margins, the disruptor needs to enter the segment where the customer is willing to pay a little more for higher quality. To ensure this quality in its product, the disruptor needs to innovate. The incumbent will not do much to retain its share in a not so profitable segment, and will move up-market and focus on its more attractive customers. After a number of such encounters, the incumbent is squeezed into smaller markets than it was previously serving. And then finally the disruptive technology meets the demands of the most profitable segment and drives the established company out of the market.

   As someone who now maintains several wedding lists in collaboration with his future spouse I can say without a doubt that universal access to our files from any computer without my fiancèe or I having to install or purchase any software is head and shoulders beyond the solution provided by traditional desktop productivity suites. In addition, it is quite clear that Google will move to address the gaps in time (see Google Gears) so we are likely on the cusp of a multi-billion dollar software category undergo upheaval in the next few years.

   The main lesson here is Change the Game. Do not play by the rules that favor your competitors.

The key thing for people wanting to learn from Google's practices isn't to follow each of Google's specific policies but instead to understand the philosophy behind their practices then apply those philosophies in your specific context.

 

A couple of years ago, I wrote a blog post entitled Social Software: Finding Beauty in Walled Gardens where I riffed on the benefits of being able to tell the software applications you use regularly "here are the people I know, these are the ones I trust, etc". At the time I assumed that it would be one of the big Web companies such as Google, Yahoo!, or Microsoft that would build the killer social software platform that was powered by this unified view of your social connections. I was wrong. Facebook has beaten everyone to doing it first. There are a lot of user scenarios on the Web that can be improved if the applications we were using know who our friends, family and co-workers were knew without us having to explicitly tell them. Below are a couple of online services where access to a user's social network has made Facebook better at performing certain Web tasks than the traditional market leaders. 

NOTE: This started off as three different blog posts in my writing queue but after reading ridiculous overhype like ex-Google employees publicly decamping from their former employer because 'Facebook is the Google of yesterday, the Microsoft of long ago' I decided to scale back my writing about the service and merge all my thoughts into a single post. 

Displacing Email for Personal Communication

Gervase Markham, an employee of the Mozilla Foundation, recently wrote in his blog post entitled The Proprietarisation of Email

However, I also think we need to be aware of current attempts to make email closed and proprietary.

What am I talking about, I hear you ask? No-one's resurrected the idea of a spam-free email walled garden recently. Companies who tout their own secure mail protocols come and go and no-one notes their passing. The volume of legitimate email sent continues to grow. What's the worry?

I'm talking about the messaging systems built into sites like Facebook and LinkedIn. On several occasions recently, friends have chosen to get back in touch with me via one of these rather than by email. Another friend recently finished a conversation with a third party by saying "Facebook me"; when I asked her why she didn't just use email, she said "Oh, Facebook is so much easier".

And she's right. There's no spam, no risk of viruses or phishing, and you have a ready-made address book that you don't have to maintain. You can even do common mass email types like "Everyone, come to this event" using a much richer interface. Or other people can see what you say if you "write on their wall". In that light, the facts that the compose interface sucks even more than normal webmail, and that you don't have export access to a store of your own messages, don't seem quite so important.

After I read this post, I reflected on my casual use of the user to user messaging feature on Facebook and realized that even though I've only used it a handful of times, I've used it to communicate with friends and family a lot more than I have used either of my personal email addresses in the past three months. In fact, there are a bunch of friends and family whose email addresses I don't know that I've only communicated with online through Facebook. That's pretty wild. The fact that I don't get spam or random messages from people I don't know is also a nice plus and something a lot of other social network sites could learn from.

So one consequence of Facebook being used heavily by people in my real-life social network is that it is now more likely to be my interface for communicating with people I know personally than email. I suspect that if they ever add an instant messaging component to the site, it could significantly change the demographics of the top instant messaging applications.

Changing the Nature of Software Discovery and Distribution

I wrote about this yesterday in my post Marc Andreessen: The GoDaddy 2.0 Business Model but I think the ramifications of this are significant enough that it bears repeating. The viral software distribution model is probably one of the biggest innovations in the Facebook platform. Whenever my friends add an application to their dashboard I get a message in my news feed informing with a link to try out the application. I've tried out a couple of applications this way and it seems like a very novel and viral way to distribute applications. For one thing, it definitely a better way to discover new Facebook applications than browsing the application directory. Secondly, it also means that the best software is found a lot more quickly. The iLike folks have a blog post entitled Holy cow... 6mm users and growing 300k/day! they show a graph that indicates that iLike on Facebook has grown faster in its first few weeks than a number of popular services that grew quite quickly in their day including Skype, Hotmail, Kazaa and ICQ. 6 million new users in less than a month? 300,000 new users a day? Wow.

Although there are a number of issues to work out before transferring this idea to other contexts, I believe that this is a very compelling to approach how new software is discovered and distributed.  I would love it if I my friends and family got a notification whenever I discovered a useful Firefox add-on or a great Sidebar gadget and vice versa. I wouldn't be surprised if this concept starts showing up in other places very soon.  

Facebook Marketplace: A Craigslist Killer

Recently my former apartment was put up for rent after I broke the lease as part of the process of moving into a house. I had assumed that it would be listed in the local paper and apartment finding sites like Apartments.com. However I was surprised to find out from the property manager that they only listed apartments on Craig's List because it wasn't worth it to list anywhere else anymore. It seemed that somewhere along the line, the critical mass of apartment hunters had moved to using Craig's List for finding apartments instead of the local paper.

Since then I've used Craig's List and I was very dissatisfied with the experience. Besides the prehistoric user interface, I had to kiss a lot of frogs before finding a prince. I called about a ten people based on their listing and could only reach about half of them. Of those one person said he'd call back and didn't, another said he'd deliver to my place and then switched off his phone after I called to ask why he was late (eventually never showed) while yet another promised to show up then called back to cancel because his wife didn't want him leaving the house on a weekend. I guess it should be unsurprising how untrustworthy and flaky a bunch of the people listing goods and services for sale on Craig's List are since it doesn't cost anything to create a listing.

Now imagine if I could get goods and services only from people I know, people they know or from a somewhat trusted circle (e.g. people who work for the same employer) wouldn't that lead to a better user experience than what I had to deal with on Craig's List? In fact, this was the motivation behind Microsoft's Windows Live Expo which is billed as a "social marketplace". However the problem with marketplaces is that you need a critical mass of buyers and sellers for them to thrive. Enter Facebook Marketplace. 

Of course, this isn't a slam dunk for Facebook and in fact right now there are ten times as many items listed for sale in the Seattle area on Windows Live Expo than on Facebook Marketplace (5210 vs. 520). Even more interesting is that a number of listings on Facebook Marketplace actually link back to listings on Craig's List which implies that people aren't taking it seriously as a listing service yet.

Conclusion

From the above, it is clear that there is a lot of opportunity for Facebook to dominate and change a number of online markets beyond just social networking sites. However it is not a done deal. The company is estimated to have about 200 employees and that isn't a lot in the grand scheme of things. There is already evidence that they have been overwhelmed by the response to their platform when you see some of the complaints from developers about insufficient support and poor platform documentation.

In addition, it seems the core Facebook application is not seeing enough attention when you consider that there is some fairly obvious functionality that doesn't exist. Specifically, it is quite surprising that Facebook doesn't take advantage of the wisdom of the crowds for powering local recommendations. If I was a college freshman, new employee or some other recently transplanted individual it would be cool for me to plug into my social network to find out where to go for the best chinese food, pizza, or nightclubs in the area.

I suspect that the speculation on blogs like Paul Kedrosky's is right and we'll see Facebook try to raise a bunch of money to fuel growth within the next 12 - 18 months. To reach its full potential, the company needs a lot more resources than it currently has.

 

If you don't read Stevey Yegge's blog, you should. You can consider him to be the new school version of Joel Spolsky especially now that most of Joel's writing is about what's going on at Fog Creek software and random rants about applications he's using. However you should be warned that Stevey writes long posts full of metaphors which often border on allegory.

Consider his most recent post That Old MarshMallow Maze Spell which is an interesting read but full of obfuscation. I actually haven't finished it since it is rather longer than I tend to devote to a single blog post. I've been trying to track down summaries of the post and the best I've gotten so far are some comments about the post on reddit which seem to imply that the allegory is about being burned out due to some death march project at his current employer.

I'm as down with schadenfreude as the next guy but a death march project seems wildly contradictory to Stevey's previous post Good Agile, Bad Agile where he wrote

The basic idea behind project management is that you drive a project to completion. It's an overt process, a shepherding: by dint of leadership, and organization, and sheer force of will, you cause something to happen that wouldn't otherwise have happened on its own.
Project management comes in many flavors, from lightweight to heavyweight, but all flavors share the property that they are external forces acting on an organization.

At Google, projects launch because it's the least-energy state for the system.
...
Anyway, I claimed that launching projects is the natural state that Google's internal ecosystem tends towards, and it's because they pump so much energy into pointing people in that direction. All your needs are taken care of so that you can focus, and as I've described, there are lots of incentives for focusing on things that Google likes.

So launches become an emergent property of the system.

This eliminates the need for a bunch of standard project management ideas and methods: all the ones concerned with dealing with slackers, calling bluffs on estimates, forcing people to come to consensus on shared design issues, and so on. You don't need "war team meetings," and you don't need status reports. You don't need them because people are already incented to do the right things and to work together well.

So, did anyone else get anything else out of Stevey's post besides "even at Google we have death marches that suck the soul out of you"? After all, I just kinda assumed that came with the territory.

 

Robert Scoble breaks the news that Google brings developers offline with "Gears" where he writes

Right now in Sydney, Australia, the first of 10 Google Developer days are starting up and the audience there is hearing about several new initiatives. The most important of which is “Google Gears,” an open source project that will bring offline capabilities to Web Applications — aimed at developers
...
Regarding Gears. It works on Macs, Windows, Linux on IE, Firefox, Opera. Enables versioned offline storage. Extension to HTML/JavaScript.

They are showing me a demo of the new Google Reader using the new Gears plugin. After you load the Gears plugin you get a new icon at the top of your Reader window which enables offline capabilities of Google Reader. They showed how Google Reader then downloaded 2,000 feed items. They took the browser offline and it continued to work great.
...
Gears supports using Adobe’s Apollo and Flash and should support other technologies including Microsoft’s Silverlight.

Gears will be submitted to a standards organization eventually, they said, but want to make sure the technology is rock solid first.

Am I the only one wondering what took them so long? I remember chatting about this in mid-2005 with Joshua Allen, we were both pretty sure it we would see it happen within a year. I guess a year and a half isn't so bad. :)

The bit about standardizing the technology is a nice touch, not that it matters. What matters is that "it doesn't work offline" is no longer a valid criticism for Google's family of Microsoft office knock offs (i.e. Google Docs & Spreadsheets) or any other AJAX/Flash application that competes with a desktop application. Running it through a standards body wouldn't make a significant difference one way or the other to the adoption of the technology for use with Google apps. It may prevent other developers from adopting the technology but I doubt many developers would look this gift horse in the mouth, after all it is freaking offline support for Web apps. 

Welcome to the future.

 

Recently the mainstream media has been running profiles on people who have built businesses worth hundreds of millions of dollars by buying lots of Internet domain names then filling them with Google ads. Last week, CNN Money ran an article entitled The man who owns the Internet which contained the following excerpt

When Ham wants a domain, he leans over and quietly instructs an associate to bid on his behalf. He likes wedding names, so his guy lifts the white paddle and snags Weddingcatering.com for $10,000. Greeting.com is not nearly as good as the plural Greetings.com, but Ham grabs it anyway, for $350,000. Ham is a devout Christian, and he spends $31,000 to add Christianrock.com to his collection, which already includes God.com and Satan.com. When it's all over, Ham strolls to the table near the exit and writes a check for $650,000. It's a cheap afternoon.
...
Trained as a family doctor, he put off medicine after discovering the riches of the Web. Since 2000 he has quietly cobbled together a portfolio of some 300,000 domains that, combined with several other ventures, generate an estimated $70 million a year in revenue. (Like all his financial details, Ham would neither confirm nor deny this figure.)
...
And what few people know is that he's also the man behind the domain world's latest scheme: profiting from traffic generated by the millions of people who mistakenly type ".cm" instead of ".com" at the end of a domain name. Try it with almost any name you can think of -- Beer.cm, Newyorktimes.cm, even Anyname.cm -- and you'll land on a page called Agoga.com, a site filled with ads served up by Yahoo

The New York Times has a profile on another multimillion dollar company in the same business in today's article entitled Millions of Addresses and Thousands of Sites, All Leading to One which contains the following excerpts

What Internet business has raised $120 million in financing in the last year, owns 725,000 Web sites, and has as its chief executive the former head of Primedia and International Data Group? If you guessed NameMedia, a privately held owner and developer of Web sites based in Waltham, Mass., you take the prize.
...
“What we’ve wanted to do, quietly, is amass the largest real estate position on the Internet, which we feel we have,” Mr. Conlin said. Some of those properties, he said, are the equivalent of “oceanfront” sites, or high-value addresses like Photography.com or DailyHoroscope.com that NameMedia will populate with relevant editorial content. Those who type in any of NameMedia’s other 6,000 or so photography-related Internet addresses, like photographyproducts.com, will land on Photography.com.
...
So far the company’s strategy is paying off, Mr. Conlin said, with company revenue doubling last year, to $60 million.

Companies like this are bad for the Internet for several reasons. For one, they artificially reduce the pool of domain which has resulted in legitimate domains having to choose name that are either awful misspellings or sound like they were stolen from Star Wars. Secondly, a lot of these sites tend to clog up search results especially when they have generic domain names and a couple thousand sites all linking or redirecting back to one domain. Finally, the fact that these companies are making so much money in a manner that is user-hostile and ethically questionable encourages more such businesses which prey on naive Internet users to be formed.

What I've found most shocking about this trend is that the big Web advertising companies like Google go out of their way to court these businesses. In fact, Google has a service called Google AdSense for Domains [with the tastefully chosen URL http://www.google.com/domainpark] which caters exclusively to these kinds of sites.

One of the things I've disliked about the rush towards advertising based business models on the Web is that if unchecked it leads to user-hostile behavior in the quest to satisfy the bottom line. The recent flap over Google and Dell installing the equivalent of spyware on new PCs to show users ads when they make a typo when browsing the Web is an example of this negative trend. Now it turns out that Google is in bed with domain name squatters. These are all examples of Google's Strategy Tax, the fact that they make their money from ads compromises their integrity when there is a conflict between between doing what's best for users and doing what's best for advertisers.

Do no evil. It's now Search, Ads and Apps

 

From the Microsoft press release entitled Microsoft to Acquire aQuantive, Inc we learn

REDMOND, Wash. — May 18, 2007 — Microsoft Corp. today announced it will acquire aQuantive, Inc., for $66.50 per share in an all-cash transaction valued at approximately $6 billion. This deal expands upon the Company’s previously outlined vision to provide the advertising industry with a world class, Internet-wide advertising platform, as well as a set of tools and services that help its constituents generate the highest possible return on their advertising investments.
...
The aQuantive acquisition enables Microsoft to strengthen relationships with advertisers, agencies and publishers by enhancing the Company’s world-class advertising platforms and services beyond its current capabilities to serve MSN. The acquisition also provides Microsoft increased depth in building and supporting next generation advertising solutions and environments such as cross media planning, video-on-demand and IPTV.
...
aQuantive, which has approximately 2600 employees, will continue to operate from its Seattle headquarters as part of Microsoft’s Online Services Business. The combination of Microsoft and aQuantive takes the Company’s advertising platform to the next level in its ability to serve Microsoft’s first party audience assets like MSN, Windows Live, Xbox Live, and Office Live, as well as for third party publishers and applications such as Facebook and Activision game titles.

The first thing I thought when I heard this news is that it is quite telling that Microsoft's biggest acquisition ever is for a Web advertising company. The second is that it seems that Google's overpaying for market share disease ($3.1 billion for DoubleClick and $1.65 billion for YouTube) is contagious. And finally, Microsoft now has offices in downtown Seattle. Oh. Yeah. :)

I worked with some of the Avenue A/Razorfish folks as part of the Social Networking feature in Windows Live Spaces. They seemed like good folks. I'm glad Microsoft has decided to open the pocketbook instead of just rolling over when it comes to buy versus build. Sometimes you really just have to suck it up and buy.

Excerpts from the conference call at TechCrunch.

 

Robert Scoble has a blog post entitled Google to Yahoo and Microsoft: the $1.65 billion was worth it which contains the following excerpt

Ahh, now you all understand what I meant when I said YouTube is a moat, not a revenue generator. By putting YouTube results into Google’s main engine Google ensures it will have better searches than Yahoo and Microsoft (who were, truth be told, getting damn close to matching Google’s quality). And it does it in a way that Yahoo and Microsoft will not be willing to match. Seriously, can you see an executive at Microsoft advocating putting YouTube videos into Microsoft’s search results? I can’t.
...
Anyway, Google just distanced themselves from Yahoo and Microsoft. And they just provided a way to monetize YouTube videos.

I love Google’s strategy. It continues to mess with Microsoft’s strategy. Microsoft still treats each team as something that must make money. Google doesn’t do that. They didn’t care one bit that YouTube didn’t have any revenues. They knew that there’s other ways to make money off of YouTube than to force YouTube to monetize on its own.

Interesting analysis, too bad it doesn't pass muster when you look at the facts. So let's do that

1. Google didn't need to spend $1.65B on YouTube to integrate their search results. An existence proof of this are video search startups like Blinkx and Dabble that index video from practically every major video source on the Web including YouTube and definitely didn't spend billions doing so. Secondly, Google has already announced that they'll index videos from sites they don't own. Does Robert believe they plan to buy every site whose video content they'll index?

2. If Google doesn't care about monetizing YouTube why did the company transfer Shashi Seth to YouTube to work on monetization and what about the ads within videos they prototyped a few days ago?

A better analysis of Google Universal Search is that it is the ultimate manifestation of the Features, Not Products initiative. Google had too many search verticals with no way for users to find them (I used to do Google searches for Google Music Search before I could use it) and now they've remedied that in one fell swoop. Just look at what  their users had to deal with before they changed the search engine results page.

In case you were wondering the even more link goes here

 

Danny Sullivan over at Search Engine Land reports on the Google Searchology Day in his entry Google Searchology Day: Recap Of Announcements where he writes about one of the new announcements called "Universal Search" which is excerpted below.

Universal Search: Do a "regular" search and books, news, video, image and local results blended into one single page. It will still mostly be web pages that dominate, but the verticals will be more noticeable. See our very long article detailing this, Google 2.0: Google Universal Search.

So I tried the query bon jovi always and got the following results page.

Notice how there is no tab for music search results even though they are showed inline as integrated results? This seems like a bug given that it works as expected for other queries like virginia tech (which shows a tab for "news" which is also shown inline) and britney spears (which has tabs for six categories including music)  Interesting...

 

A few years ago Joel Spolsky wrote a widely quoted blog post which stated

A very senior Microsoft developer who moved to Google told me that Google works and thinks at a higher level of abstraction than Microsoft. "Google uses Bayesian filtering the way Microsoft uses the if statement," he said. That's true. Google also uses full-text-search-of-the-entire-Internet the way Microsoft uses little tables that list what error IDs correspond to which help text. Look at how Google does spell checking: it's not based on dictionaries; it's based on word usage statistics of the entire Internet, which is why Google knows how to correct my name, misspelled, and Microsoft Word doesn't.

This morning I fired up my favorite RSS reader and saw the danah boyd's entry entitled quality of Google searches? where she pointed out

It's also annoying that they've stopped correcting my atrocious spelling. I mean, it's all fine and well that lots of people in the blogosphere can't spell in exactly the same way that i can't spell, but the #1 type of search i do everyday is spell check. I throw something god-awful like Cziskentmihalyi into the engine knowing that it'll return Csikszentmihalyi. This still works quite well for names but it's stopped working for lots of regular words that i just can't spell to save my life. How pathetic is it that i've started opening up Word for the little red squigglies instead of relying on search? Or maybe both practices are weird...

This seems like a predictable problem. There are lots of commonly misspelled words and in many online communities people have simply given up on correct spelling (heck, I've now grown used to the fact that computer geeks have decided that the correct spelling of ridiculous is rediculous). Thus it is quite likely that a frequently misspelled word eventually occurs so much in the wild that it is considered a valid word. Maybe Google needs some if statements in their code after all, instead of blindly trusting the popularity contest that is Bayesian analysis. :)

The interesting thing is that one could argue that if a particular spelling of a word becomes popular then it automatically is "correct" since that is how the English language has evolved over time anyway.

 

If like me your head is spinning from all the announcements of proprietary "AJAX killers" out there then you probably were still too busy trying to digest Silverlight to even have heard that Sun's announced it's own "AJAX killer" and Flash competitor named Java FX.

Don Park has a good post which serves as a good starting point to grok the meat of this announcement titled Inside JavaFX where he writes

So what is JavaFX? This is what it looks like:

import javafx.ui.*;

Frame {
  title: "Hello World JavaFX"
  width: 20
  height: 50
  content: Label {
    text: "Hello World"
  }
  visible: true
}

As you can see, JavaFX script (*.fx) is basically JSON-like version of XAML which can be either compiled into Java bytecode or run as is using a java-based JavaFX player. Since JavaFX is just java, it can run as applets in browsers, midlets on mobiles, and applications on WebStart.

Sam Ruby has a good follow up to Don's post titled JavaFX Script where he describes some of the innovations and shortcomings of the platform he has noticed in his initial observations.

It looks like Web developers are getting spoiled for choice this summer; Sun vs. Microsoft vs. Adobe vs. Web Standards.

 

By now most people interested in technology news have heard the story initially reported by the New York Post in the article Bill's Hard Drive which claimed that Microsoft was planning to purchase Yahoo! and the subsequent rebuttal of this news in the Wall Street Journal's article Microsoft, Yahoo Discussed Deal. Since I work at Microsoft I'm not supposed to comment on rumors like this so I'll just point to a couple of opinions.

Charlene Li, an analyst at Forrester Research, wrote in her blog post entitled Why Microsoft + Yahoo! makes sense – and why it won’t work that

But there is one major reason why I don’t think Microsoft executives have the stomach for any sort of brand rationalization -- the continued dual branding of Windows Live and MSN. Each time I have a conversation with Microsoft about Windows Live, I get a different explanation of what it is and how it fits with MSN. If the company can’t event figure out its branding strategy with existing properties, I don’t hold out much faith that they could do so with a premium brand like Yahoo!

Robert Scoble writes in his post No “Microhoo”

I’ll tell ya one thing, though. It sure made for interesting conversation with a Yahoo employee I met tonight. We were starting to draw up where value would be built and where it’d be destroyed. There were a lot of places it would be destroyed and if two guys drinking beer can figure that out in half an hour, I’m sure that caused smarter people than us to put on the brakes. For instance one of Yahoo’s biggest properties is its email service. But that’s one of Microsoft’s biggest properties too (aka Hotmail). Then you look at finance sites. Microsoft is doing pretty well there with Microsoft Money (I met an employee who works there too and he said both Microsoft and Yahoo are way ahead of Google in finance traffic). Then you look at mapping. Again, they are pretty strong competitors there. Search? Who’s search technology would be thrown away? Advertising technology? Both Microsoft and Yahoo are pretty close there. Flickr? Clear value creation in an acquisition cause Microsoft doesn’t have anything like it. Same with Del.icio.us, Upcoming.org, Yahoo Answers, MyBlogLog. Portal? Yahoo is clear winner in brand name, but that’s just cause Microsoft has done an awful job in rebranding everything as “Live.”

As for me, two phrases popped into my head when I heard the rumor; hail mary play and post merger layoffs. I guess word association is mother...

In related news, Mike Arrington reported yesterday that Yahoo To Shut Down Yahoo Photos In Favor Of Flickr. This is a shockingly disruptive move that will definitely piss off users given how different the features and target audience of both sites are. Why would a Web savvy, user-centric company like Yahoo! do something that will so obviously cause bad blood between them and their users? How about all the employees working on Yahoo! Photos who busted their butts to make it one of the Web's most popular photo sites? The answer is simple. Yahoo! has redundant and competing products which they could no longer rationalize.

Now go back and reread Robert Scoble's post linked above then consider what a YHOO-MSFT merger would look like again.  

 

Terry Semel, Yahoo! CEO, can't seem to get a break these days. It seems everyone has assumed his days are numbered and he will be ousted for "incompetence". I've seen everyone get into the act from the technology press like C|Net and TechCrunch to more laid back technology pundits like Jason Fried in his post I'd rather be Microsoft than Yahoo.

All of these calls for Terry Semel's head remind me of a book I've been planning to read, The Halo Effect: ... and the Eight Other Business Delusions That Deceive Managers which describes The Halo Effect as it applies to business. So what exactly is this effect? Well according to Wikipedia

The halo effect refers to a cognitive bias whereby the perception of a particular trait is influenced by the perception of the former traits in a sequence of interpretations. The halo effect is involved in Kelley's implicit personality theory, where the first traits we recognize in other people then influence the interpretation and perception of latter ones (because of our expectations). Attractive people are often judged as having a more desirable personality and more skills than someone of average appearance. Celebrities are used to endorse products that they have no expertise in evaluating.

That may have been a bit difficult to follow, so here's an excerpt from David Wolfe's review of the book that explains the Halo Effect in business

Rosenzweig argues that media is no less error-prone in divining the reasons for a company's performance than gurus who write books on company performance.

He uses two prominent case histories to support the latter claim. Cisco and ABB. He recalls for readers how before the dotcom bubble collapse Cisco's John Chambers was widely regarded as the world's best manager and Cisco itself as without surpass in its organizational structure and corporate culture. Beyond that, according to the common view expressed in media, no company operated with greater customer centricity.

Within months of the dotcom collapse the same media that had virtually canonized Chambers were ripping his reputation as a great CEO to shreds. Cisco's was criticized for lack of attention to customer needs. This was alleged to have played a major role in its downturn in revenues and consequent precipitous loss in stock value. Writers told how the company's organization was fragmented and its culture grossly defective.

Rosenzweig tells a similar story of the Swiss-Swedish power company ABB. From being one of Europe's most highly regarded company to being a favorite whipping boy in media within not very many months, ABB turned out to be another example of what Rosenzweig calls the Halo Effect.

As Rosenzweig explains it, the Halo Effect refers to the aura surrounding a company and its leadership that promotes gross generalizations about its nature. When a company is outperforming, most everyone assumes everything about the company is exemplary. When the same company is underperforming, most everyone assumes everything about the company is defective.
Rosenzweig reminds us that externalities accounted mainly for Cisco's and ABB's decline in fortune. Cisco's downturn for example was inextricably linked to its Internet customers who flamed out in droves when the dotcom bubble burst. No amount of executive brilliance could have stayed the fiscal injury inflected by the dotcom bust.

What does this have to do with Yahoo! and Terry Semel? Glad you asked. See the blog post from ValleyWag entitled Fred Vogelstein's Amnesia which contains the following excerpts

How Yahoo blew it. Semel has been Yahoo's CEO for nearly six years, yet he has never acquired an intuitive sense of the company's plumbing. Semel's vaunted dealmaking skills seem to have deserted him. At Yahoo, the marketers rule, and at Google the engineers rule. And for that, Yahoo is finally paying the price. [Fred Vogelstein, writing in Wired Magazine, January 2007]

By figuring out how to make brand advertising work online, Terry Semel is on the verge of creating the 21st century's first media giant. So far, Semel has put together one of the web's hottest winning streaks... It turns out that the riddle of how to succeed online isn't so tough after all. Semel is taking everything he learned in his analog past and marrying it to what he can see in the digital future. [Fred Vogelstein, writing in Fortune Magazine, August 2005]

Same leadership style, same company. Yet when you tweak the externalities the CEO goes from being "on the verge of creating the 21st century's first media giant" to being someone whose "vaunted dealmaking skills seem to have deserted him". From where I sit, it is unclear that Terry Semel is actually doing a bad job, it just seems that his company isn't doing as well as Google which may or may not have anything to do with him.

I wonder how many years it will take until 20% time, free lunches and all the other things that people hype up about Google are used as examples of why the company has issues when some new upstart shows up on the scene?

 

Mike Arrington of TechCrunch has a blog post entitled Can PhotoBucket Survive Without MySpace? where he writes

There was a lot of fingerpointing, denials, and “he said, she said” going on today as everyone digested the news that MySpace had blocked PhotoBucket’s 40 million members from embedding videos into their MySpace pages.

From my perspective this looks like MySpace just found an excuse to send a big middle finger to the largest independent widget company in the hope of disrupting their ongoing acquisition talks. Om Malik sees things differently and thinks Photobucket practically asked for this blockade. Robert Scoble calls Photobucket and services like it “parasitic.” Nick Carr says this is all basically inevitable, regardless of who’s to blame.
...
And many MySpace/Photobucket users will simply leave MySpace and go to one of its many competitors rather than lose the ability to embed their Photobucket media. Re-creating a profile at another social network takes a lot less time than re-uploading hours of video. In the end, Photobucket could prove to be stickier than MySpace.

I agree with Om Malik that PhotoBucket clearly violated MySpace's terms of service by having ads in videos embedded on  http://www.myspace.com. I also think that the advice that Mike Arrington gives in his post will lead PhotoBucket down a bad road if they take it. The key question is whether the lock-in from a social network site like MySpace (where all your friends are) is more significant than the lock-in from having my media stored in a particular photo hosting or video hosting site. If MySpace has only blocked new embeds from PhotoBucket then I'm willing to bet that it is more likely that users will simply pick a new media hosting provider (e.g. YouTube for videos, Flickr for photos) than that they'll switch to Facebook or Windows Live Spaces because they are too tied to PhotoBucket.

If I were PhotoBucket, I'd work with MySpace and either (i) agree on how MySpace gets a revshare of PhotoBucket ads shown on their site or (ii) make it easy for MySpace to filter out the embeds with ads (which are a minority) and allow other embedded media from PhotoBucket pass through. Considering that they are trying to flip their startup, the PhotoBucket crew would be wise to avoid going to war with the biggest traffic driver to their site.

 

The combination of my messed up internal clock and the fact that it seems there is now a wireless LAN at the villa means that I am up blogging when I should be sleeping. The Web geek blog buzz today is the announcement contained in the blog post entitled Map-making: So easy a caveman could do it on the Official Google blog which states

That's why we're announcing My Maps, a new feature that makes it quick and easy to create your own custom Google Maps just by pointing and clicking. You can add placemarks, draw lines and shapes, and embed text, photos and videos -- all using a simple drag and drop interface. Your map automatically gets a public URL that you can share with your friends and family, or you can also publish your map for inclusion in Google Maps search results. We'll continue to show organic local search results with red pushpins; user-generated results will have blue pushpins. The user-created results include KML as well as maps made through My Maps.

To give you a better idea of what kind of maps you can make, here are some examples that Googlers created after we released the feature internally. (We ran a contest and gave a Nintendo Wii to the best map-maker.)

• America's Highway: Oral Histories of Route 66: A glimpse into life on the legendary (and now-decommissioned) highway, and what drivers will find there today
• Around Japan in 28 Days: Catalogs a trip across Japan that can be done in 3 weeks

As usual the reactions from the blog pundits are equal parts surprising and unsurprising to me. The unsurprising bit is that I didn't find anyone who compared this to the collections feature of MSN Virtual Earth Windows Live Local Live Search Maps Live Maps which can be viewed at http://collections.live.com. I'm sure when the "Web 2.0" pundits eventually discover we have this feature it will be claimed we copied it from Google. :)

On the other hand what I did find surprising were blog posts like Google Launches MyMaps - Platial Gets Screwed and My Maps at Google: Is Google Doing a Microsoft? from Pete Cashmore and Paul Kedrosky which complained that Google was killing "social mapping" startups like Platial and Frappr with this move. Ignoring that "social mapping" seems like a silly product category in the first place, I wonder what exactly is wrong with this move. Some startups point out consumer demand for certain features from online mapping sites (i.e. missing features) and the consumer mapping sites add the features.

Is the expectation that companies shouldn't be able to improve their products once startups start trying to build a business out of the fact that their product is missing features? I've never understood that reasoning. Paul Kedrosky specifically calls this "pulling a Microsoft". I wonder...do users really want their computers to ship without a Web browser or a media player? Do users really want to go to one site to get maps and another to add annotations to these maps? Adding features to products is good for users and we shouldn't be trying to artificially limit products because some startups are trying to build a business out of a feature. If startups like  Platial and Frappr are actually adding value then they'll survive. If they don't, then they probably didn't have much of a business to begin with.  

PS: I understand that the philosophy of anti-trust law in Europe is about preventing unfair competition whereas in the U.S. it is about preventing harm to consumers. Thus depending on where you are from these questions will strike a different chord with you.

 

I just arrived at London Heathrow and can look forward to another 9 hours or so until my flight to Nigeria. In the meantime, I've found complimentary Web access in the business class lounge so it looks like I won't be bored. I am a little worried about keyloggers and spyware on this computer given how easy it was for me to install Firefox on it. Here are a couple of quick thoughts I had on the way related to links I've seen over the past 24 hours

• Shelley Powers has blog post on newly announced vulnerabilities in popular AJAX libraries. It seems this is another cross site request forgery (XSRF) issue similar to the one found in GMail a few months ago. I remember blogging about mitigations to this issue in my post The GMail Security Flaw and Canary Values when the flaw was announced. I suspect that XSRF will be the new 'buffer overflow' and 'SQL injection attack' of the Web, I expect we'll see a lot more of these kinds of attacks over the next few years before the technology and technologists will catch up with basic safety practices.

• Thomas Hawk and Mike Arrington are continuing with their attempts to stir up an online lynch mob to hang Amazon for shutting a site that was screenscraping and hotlinking to their content. Arrington's post is entitled Amazon’s War on Statsaholic while Hawk's is So it Looks Like Amazon.com Is Going to Be a Jackass and Sue Statsaholic. Nothing beats the blogosphere when it comes to misplaced righteous indignation. A couple of my readers pointed out that the issue is that Amazon initially supported the service then changed their mind. Ignoring the fact that in this 'Stay the Course' world there is nothing wrong with changing your mind, let's look at the facts.

  1. Some geek builds a snazzy AJAX UI on top of screenscraped Alexa data named Alexaholic and sends announcements to Web 2.0 gatekeepers who give it a thumbs up.
  2. Developer on Amazon's blog gives a hat tip to Alexaholic.
  3. Amazon business folks get involved and realize some guy is building a competitor to their service with their own proprietary data and server resources then shut him down.

  As a geek, I think Alexaholic is cool. as someone who actually works on large scale Web services and knows that these services cost millions of dollars to build and maintain, I know why the last thing you want is some competing service built on screenscraping your data then showing its own ads on top of it. Heck, even Google has realized that APIs that allow competitors to build competing services on your data + resources while circumventing your ads are a bad idea so why is it wrong for Amazon to do the same thing?

  This reminds me of Don MacAskill's post on Enterprise Linux where he points out that he saves money on enterprise Linux for his business by buying rebranded/repackaged versions of RedHat Linux from RedHat's competitors like Oracle's Unbreakable Linux and  CentOS because they undercut Red Hat on price on their own software. Is that really the world Arrington and Hawk want to force "Web 2.0" companies into? Well, screw that. More power to Amazon.

  PS: Amazon is probably wise to let it go against Alexaholic mainly because it hurts the messaging around their Web Platform story. Morally they are in the right but bad press never cares who's right, does it?

• Thanks for all the book suggestions. I ended up going with Ender's Game and Freakonomics from the suggestions. I also got three Discworld books, Anansi Boys and Dune. I'm halfway through the Neil Gaiman book and although it isn't as good as American Gods it is still an enjoyable read albeit a little slow to start off which I guess was on purpose.
  

• It looks like there is a new version of Live Maps according to the team blog. This product has now officially gone through more names than I've had ex-girlfriends. It's been named MSN Virtual Earth, Windows Live Local, Windows Live Maps, Live Search Maps and now Live Maps. It's sad that we are intent on screwing one of the coolest products we are shipping these days in this way. :(

  Anyway, check it out at http://maps.live.com

• This is another summer that promises a bunch of movies from cartoons and comics that rocked my childhood. Fantastic Four: Rise of the Sliver Surfer, Spider-Man 3 and Transformers all look hot. The trailer for Transformers was initially disappointing because it looked like there were too many humans and not enough robots but the newly released pics from the movie make it look like my fears were unfounded. Sweet.

• I saw Forest Whitaker the last time I travelled to Nigeria in the Virgin Atlantic upper class lounge. I wonder if I'll meet any celebrities this time around? I doubt it though given that British Airways is a step down from travelling on Virgin Atlantic. Damn that Richard Branson.

I'm hungry and need to get back to my book. Holla at y'all later.
 

Mike Arrington has a blog post on TechCrunch entitled Amazon’s War on Statsaholic where he writes

Statsaholic (formerly Alexaholic) launched a year ago and provided much easier access to Alexa traffic data than the Alexa site itself. Statsaholic also had other features Alexa didn’t offer, like embeddable graphs and data smoothing. Others agreed, and soon started linking to Statsaholic instead of Alexa when doing traffic comparisons. At one point, Alexa was the no. 3 search result on Google for “Alexa.”

Statsaholic was not using the Alexa web service to get the data, because Alexa doesn’t offer the graph data via their web service. Amazon, which owns Alexa, could have complained or simply shut them down when it launched, but they didn’t. They actually complimented the service in a post on the Alexa blog last April.
...
What bothers me about the situation is that Amazon sat on it for a year, complimenting the service along the way (and copying it). Then, just when the service started getting really popular, they took drastic measures to shut it down.

I'm totally perplexed by Arrington's position here. Statsaholic is screenscraping Alexa and building a business on top of that. It seems like a pretty open and shut issue to me. The fact that Amazon didn't bother prosecuting them until they got a lot of traffic just points out that there is little point harassing folks who are abusing your service unless they are consuming a lot of your resources or are taking money out of your pocket. It seems Statsaholic was doing both. 

You'd think a smart startup founder would know better than to build a business model on hotlinking and bandwidth theft. You'd also expect a Web savvy dude like Mike Arrington to know better than blame the victim in such situations. Next thing you know, he'll be flaming websites that block hotlinking to their images via htaccess. Weird. 

PS: Someone in the comments wondered how Mike Arrington would feel if someone created a mashup that showed all of the TechCrunch content minus the obnoxious ads (e.g. http://techcrunchminusads.com). I wonder if Mike would sue if the site started stealing a bunch of his traffic  since it wouldn't load so many ads thus being faster and perhaps included more useful info (e.g. crosslinking TechCrunch posts with PodTech interviews)? 

 

The Yahoo! Developer blog has an entry entitled Introducing the Yahoo! Mail Web Service which states

While we are certainly proud of the success of Yahoo! Mail, today we are announcing how we are going beyond the boundaries of the Yahoo! network and enabling developers to build new tools or applications around Yahoo! Mail. We are thrilled to announce the open availability of the Yahoo! Mail Web Service, web service for Yahoo! Mail (accessible via SOAP or JSON-RPC) that we previewed to Yahoo! Hack Day attendees. With the Yahoo! Mail Web Service, you can connect to the core mail platform to perform typical mailbox tasks for premium users such as list messages and folders, and compose and send messages (you can also build mail preview tools for free users with limited Web Service functionality). In other words, developers outside of Yahoo! can now build mail tools or applications on the same infrastructure we use to build the highly-scaled Yahoo! Mail service that serves nearly 250 million Yahoo! Mail users today -- users who might want to help you make some cash with your application.
...
The Yahoo! Mail Web Service is a big release for Yahoo! and the Internet, and it's only the beginning of what you'll be seeing from Yahoo!. Jump into our code samples for Java, .NET, PHP, Perl and Python, and build your dream mail app today, then be sure to give us feedback on your experience so we can continue to make the API even better. Be sure to leverage the online Yahoo! Mail Web Service support group where you can get help from the Yahoo! Mail Web Service team and your fellow developers. We can't wait to see what applications you will build when you add your imagination to the platform. Maybe you want to build an application that backs up Yahoo! mail targeted at a large number of Yahoo! users, or maybe you just want to add a niche feature that makes Yahoo! Mail better for your mom. For inspiration, we've gathered a few applications:

• Yahoo! Mail Search (Mail Plus)
• Flickr Postcard
• Yahoo! Mail with Flickr Image Association

First things first, this is an unprecedented and very welcome move on the part of Yahoo! This is another example of why I think of the three major Web portals, Yahoo! has done the best job of turning their services into a developer platform. I like the fact that the Web service is exposed over multiple protocols and the code samples are in multiple programming languages that run the gamut from enterprise developer fare (Java/.NET) to Web geek fare (Perl/Python/PHP). Mad props to the developer platform folks at Yahoo!, good work.

With that out of the way, there is some stuff that has me scratching my head after taking a look at the Yahoo! Mail Web Service User Guide and API Reference. The first thing that is weird is that although Yahoo! provides SOAP and RESTful JSON web services for accessing one's mail, I still can't get POP access to my Yahoo! mail without shelling out $20/year. After all, GMail has POP access for free and users of the free versions of Windows Live Hotmail can get POP-like access if they use Windows Live Mail desktop although they are restricted to using one mail client.

So I decided to see if the Yahoo! Mail Web Services provides a way around this restriction but found out from the section on "Determining the Users Account Capabilities" that

The Yahoo! Mail Web Service limits the functionality available to free accounts. Premium accounts have no such limits. First call the GetUserData method to get the user’s account type from the web service.

Account Types

●	Premium Users

Premium users may access any part of the web service. There are no restrictions.

●	Non-Premium Users

Non-Premium users are restricted to the following APIs:

ListFolders	ListMessages	DeleteMessages
GetUserData	BatchExecute	CreateFolder
FetchExternalMail	RemoveFolder	RenameFolder
MoveMessages	EmptyFolder

Calls to other APIs will return an error.

So it looks like I actually can't build an application that can be used to read the actual mail messages from my Yahoo! Mail account with the API unless I'm a premium user. Otherwise, all I can do is list the messages but not actually get their content. That makes the APIs a lot less cool than I initially thought. 

Like Sam Ruby and Paul Downey I initially wondered about the choice of exposing a SOAP API but then realized that it may be that they already use SOAP internally so this wasn't that much work for them in that case. I also wondered about the lack of a RESTful non-SOAP XML interface as well but after looking at the complex object models I can see why they went with data formats that are pretty much serialized object models (i.e. Javascript OBJECT Notation & Simple OBJECT Access Protocol) instead of expecting developers to write a bunch of gnarly XML parsing code for processing over a dozen different response formats from the 23 methods in the Yahoo! Mail Web Service.

I suspect that Yahoo! won't get as much traction as they expect with the API until they remove some of the restrictions on non-premium accounts. Even then it does look like there is enough to build Windows and Windows Live gadgets for Yahoo! Mail that show your messages. Except that there is no way to read the mail contents and not even a way to link to the message in way that sends the user to Yahoo! Mail to read its contents. I bet if Yahoo! fixed the latter and perhaps had a rev-share with people who drove clicks back to the Web-based mail client, things would get very interesting. I wonder if Jeremy is listening?

 

The Australian iTWire has a rather biased and misinformed article entitled Official: Microsoft ‘bribes’ companies to use Live Search which contains the following excerpt

Microsoft’s new program is called “Microsoft Service Credits for Web Search” and has been unveiled by John Batelle’s ‘SearchBlog’. The money on offer is significant, especially when multiplied across thousands of PCs. The deal means that companies can earn between US $2 and US $10 per computer on an annual basis, plus a US $25,000 “enrollment credit” which is a nice big wad of cash that will likely need a large-ish, strong and sturdy brown paper bag to hold securely while being passed under the table.  

For companies that have thousands of computers, this could translate into anywhere from US $100,000 to $200,000 per year, which is money that could be put to good use in the IT department or elsewhere in the company.
...
Former Microsoft employee and blogger Robert Scoble who served as the online face of Microsoft during his three years at the company is not impressed with Microsoft’s moves in deciding to offer companies money to use search.  His arguments are quite valid and boil down to Microsoft really needing to create better products, rather than needing to pay companies to get more traction for Windows Live. After all, Scoble isn’t the first to observe that Google doesn’t need to pay anyone to use its search services – people use them voluntarily because of the quality of the results

The amount of bias in this article is pretty amazing considering that Microsoft is primarily reacting to industry practices created by the Google [which have also been adopted by Yahoo!]. Let me count the ways Google bribes companies and individuals to use their search engine

1. Google pays AdSense publishers for each user they convince to install Firefox with the Google Toolbar installed. Details are in the documentation for the AdSense Referrals Feature. Speculation on Slashdot was that they pay $1 per user who switches to Firefox + Google Toolbar.

2. Google paid Dell $1 billion dollars to ensure that Google products are preinstalled in all the computers they sell and the default search engine/home page is set to Google. Details of this deal were even published in iTWire.

3. Google paid Adobe an undisclosed amount to bundle Google Toolbar [which converts your default search engine in your browser to theirs] with all Adobe products.

4. Google entered a distribution deal with Sun Microsystems to bundle Google Toolbar [which converts your default search engine in your browser to theirs] with all new installations of the Java runtime.

5. Google products which converts your default search engine in your browser to theirs are bundled with the Winzip archiving utility. Financial details of the deal were undisclosed.

6. Google is the default search engine for both the Opera and Firefox browsers. Both vendors get a cut of the search revenue generated from user searches which runs in the millions of dollars.

I could go on but my girlfriend just told me it's time for breakfast and I'm already in trouble for blogging on a Sunday morning. However the above links should be enough to debunk the inaccurate statements in the iTWire article. I guess iTWire's "journalism" is further validation of the saying that you should never let the facts get in the way of a good flame.

 

A number of blogs I read have been talking about Amazon's S3 service a lot recently. I've seen posts from Jeff Atwood, Shelley Powers and most recently Dave Winer. I find it interesting that S3 is turning into a classic long tail service that works for both startups who are spending hundreds of thousands to millions of dollars a year to service millions of users (like Smugmug) to bloggers who need some additional hosting for their cat pictures. One reason I find this interesting is that it is unclear to me S3 is a business that will be profitable in the long term by itself.

My initial assumption was that S3 was a way for Amazon to turn a lemons into lemonade with regards to bandwidth costs. Big companies like Amazon are usually billed for bandwidth using 95th percentile billing, which is explained below

With 95th percentile billing, buyers are billed each month at a fixed price multiplied by the peak traffic level, regardless of how much is used the rest of the time. Thus with the same nominal price, the effective price is higher for buyers with burstier traffic patterns.

So my assumption was that S3 allows Amazon to make money from bandwidth they were already being charged for and not using. As for storage, my guess is that they are either making a miniscule amount of profit or at cost. Where this gets tricky is that, if S3 gets popular enough then all of a sudden it no longer is a way to make money from bandwidth they are being billed for but aren't using but instead impacts their actual bandwidth costs which then changes the profit equation for the service. Without any data on Amazon's cost structure it is unclear whether this would make the service unprofitable or whether this is already factored into their pricing.

On the other hand, Amazon's Elastic Compute Cloud (EC2) isn't something I've seen a lot of bloggers rave about. However it seems to be the service that shows that Amazon is making a big play to be the world's operating system in the sky as opposed to dabbling in providing some of its internal services to external folks as a cost savings measure. With EC2 you can create a bunch of virtual servers in their system and load it up with an Amazon Machine Image (AMI). An AMI is basically a server operating system and the platform components you need on it. Typical AMIs are an instance of a LAMP system (Linux/Apache/MySQL/PHP/Perl/Python) although I did see one AMI that was an instance of Windows 2003 server. You can create as many or as few server instances as you need and are billed just for what you need.

I suspect that the combination of EC2 and S3  is intended to be very attractive to startups. Instead of spending hundreds of thousands of dollars building out clusters of servers, you just pay as you go when you get your monthly bill. There are only two problems with this strategy that I can see. The first is that, if I was building the next Digg, Flickr or del.icio.us I'm not sure I'd want to place myself completely at the mercy of Amazon especially since there doesn't seem to be any SLA published on the site. According to the CEO of Smugmug in his post Amazon S3: Outages, slowdowns, and problems they've had four major problems with S3 in the past year which has made them rely less on the service for critical needs. The second issue is that VC money is really, really, really easy to come by these days judging from the kind of companies that get profiled on TechCrunch and Mashable. If the latter should change, it isn’t hard to imagine dozens of enterprising folks with a couple of thousand dollars in their pockets deciding to go with S3  + EC2 instead of seeking VC funding. But for now, I doubt that this will be the case.  

What I suspect is that without some catalyst (e.g. the next YouTube is built on S3  + EC2)these services will not reach their full potential. This would be unfortunate because I think in much the same way we moved from everyone rolling their own software to shrinkwrapped software, we will need to move to shrinkwrapped Web platforms in the future instead of everyone running their own ad-hoc cluster of Windows or LAMP servers and solving the same problems that others have solved thousands of times already.

I wonder if Amazon has considered tapping the long tail by going up against GoDaddy's hosting services with S3  + EC2. They have the major pieces already although it seems that their prices would need to go down to compete with what GoDaddy charges for bandwidth although I suspect that Amazon's quality of service would be better.

 

Today on the Facebook blog I spotted a post entitled FQL which contains the following excerpt

Two and a half months ago, a few of us were hanging out in the Facebook TV room, laying on the Fatboys and geeking out about how to move forward with the API for the Facebook Platform. We had a beta version that was fully functional, but we kept wishing that the interface were cleaner, more concise, and more consistent. Suddenly it occurred to me – this problem had been solved over 30 years earlier by database developers who came up with SQL – the Structured Query Language. What if we could use the same time-tested interface as the way for developers to access Facebook's data?
...
This isn't a simple problem – with millions of users and billions of friend connections, photos, tags, etc., Facebook's data doesn't exactly fit into your average database. And, even if it did, we still have to carefully apply all of those complicated privacy rules. Facebook Query Language would have to take those SQL-style queries from developers, figure out what data they're actually looking for, figure out if they're allowed to actually see the data, figure out where the data is stored, and then finally go and get the data to return back to the developer. I knew building FQL would be hard, but that's why I couldn't wait to do it.

This is one of those things I used to think was a great idea when I was on the XML team at Microsoft. Instead of exposing your data using APIs, why not expose your data as XML then allow people to perform XQuery operations over the data. In reality, this often isn't really feasible because you don't want people performing arbitrary queries over your data store that may request data too much data (SELECT * FROM blog_posts) or are expensive computationally.

Looking at the FQL developers guide it states that a typical queries look like

SELECT name, pic FROM user WHERE uid=211031 OR uid=4801660

SELECT name, affiliations FROM user
WHERE uid IN (SELECT uid2 FROM friend WHERE uid1=211031)
      AND "Facebook" IN affiliations.name AND uid < 10

SELECT src, caption, 1+2*3/4, caption, 10*(20 + 1) FROM photo
WHERE pid IN (SELECT pid FROM photo_tag WHERE subject=211031) AND
      pid IN (SELECT pid FROM photo_tag WHERE subject=204686) AND
      caption

and return results as XML. I've assumed that what is supported is a simple subset of SQL, perhaps written with Lex & Yacc or ANTLR but it still seems somewhat problematic to move away from the constrained interface of an API and provide access via a query language. It is definitely a lot cooler and more consistent to work with a query language than an API though. Later on when I have some free time, I'll see if I can deduce the grammer for FQL by trying out queries in the Facebook API test console. It looks like there goes one of my evenings this week.

Nice work.

 

From the Microsoft press release Microsoft Demonstrates Further Commitment to Healthcare Market With Planned Acquisition of Web Search Company we learn

NEW ORLEANS — Feb. 26, 2007 — Microsoft Corp. today announced that it has agreed to acquire Medstory Inc., a privately held company based in Foster City, Calif., that develops intelligent Web search technology specifically for health information. The acquisition represents a strategic move for Microsoft in the consumer health search arena and signals a long-term commitment toward the development of a broader consumer health strategy. Medstory employees will join the Health Solutions Group, a recently formed division at Microsoft that will manage product development and delivery. Financial terms were not disclosed, as part of the agreement between the organizations.

This reminds me of the post Thoughts on health care, continued from Google's Adam Bosworth which stated

As I indicated in my post last week, I've been interested in the issue of health care and health information for a while. I just spoke at a conference about some of the challenges in the health care system that we at Google want to tackle. The conference, called Connecting Americans to Their Health Care, is a gathering focused on how consumers are transforming health care through the use of personal health technologies.

This speech will give you some insight into the problems that we believe need our attention.

It is also interesting that Adam Bosworth had been billed with the title Architect, Google Health for a while. I'd once heard that the the market for medical related keywords is one of the most lucrative for search engines which may explain the interest. However if you look at the list of most expensive adwords it would seem that building a vertical search engine targetted at debt consolidation is the real goldmine. :)

 

I'm almost caught up on my blog reading since getting back from vacation and I've spotted a couple of items I'd have blogged responses to if I was around. Since I don't have the time to write full blog posts on each of these items, here are links to the posts and brief outlines on what I thought about them

• Harish Mallipeddi has a blog post entitled Measuring efficiency of tagging with Entropy links to the paper Understanding Navigability of Social Tagging Systems by Ed Chi and Todd Mytkowicz of Xerox Parc which excerpts the key findings from the paper. One result of their research which seems obvious in hindsight and shows one of the issues that social software has to deal with as its community of users grows was

  The way he does that is to measure entropy (yup that same old same old Claude Shannon’s information theory which you learned in one of the CS courses) of entities like documents (D), users (U) and tags (T). His research group crawled the entire del.icio.us archive and then calculated the entropies. Here’s what they found:

  • H(D|T) specifies the social navigation efficiency. How efficient is it for us to specify a set of tags to find a set of specific documents? We found that in del.icio.us that it is getting less and less efficient.

  This makes sense when you think about it. Let's say the first set of users of del.icio.us came from a homogenous software development background and started applying the tag "xml" to mean items about the eXtensible Markup Language. Later on as the community grew, a number of gamers joined the site and they now use the tag "xml" to refer to items about the game X-Men Legends. Now if you are one of the original geek users of the site, the URL http://del.icio.us/tag/xml no longer is just about markup languages but also about video games. To actually find items strictly about the eXtensible Markup Language you may have to add other tags as refinements such as http://del.icio.us/tag/xml+programming.

  What this means is that to the oldest users of the site, the quality of the tagging system will seem to degrade over time even though this is a natural consequence of growth and diversifying its user base. Of course, this is only a problem if a lot of people use del.icio.us to find all items about a topic (i.e. browsing by tags) as opposed to just storing their individual bookmarks or subscribing to the bookmarks of people they know and trust.

• It seems Google announced some sort of Microsoft Office killer last week. You can read Don Dodge's Why Microsoft will not fall into the Innovators Dilemma and Robert Scoble's Microsoft has no innovator’s dillema? for two conflicting opinions on how this affects Microsoft. Personally, I think I've overdosed on the amount of times I've read the words innovator's dilemma in association with this announcement while catching up on email and blogs. What is funny about this situation is that almost everyone I've seen who throws the term around doesn't seem to have read the book. It is quite interesting to see Don Dodge write sentences like

  Microsoft will do everything possible to preserve these businesses while transitioning to the new Live strategy.

  and then follow that up with "No Innovators Dilemma here" without seeing the obvious contradiction in his words. Lots of  doublethink at work it seems.

  A side effect of reading this set of blog posts is that I found Don Dodge's Innovate or Imitate...Fame or Fortune? which praises being a fast follower as being more valuable than being an innovator. I've found that a lot of people at Microsoft point to past and recent successes such as XBox, Microsoft Office and Internet Explorer as proof that being a "fast follower" is the best strategy for Microsoft. There are three key problems with this kind of thinking

  1. It assumes your competitors are incompetent. This may have worked in the old days but with competitors like Google and Apple Inc, it isn't the case anymore.
  2. It requires that you have an ace up your sleeve that significantly one ups the competitors when you ship your knock off (e.g. integrating disparate applications into an Office Suite and pricing it lower than competitors, integrating product into the operating system, integrating a rich and social online experience into what was previously a solitary experience etc).
  3. It ignores the fact that "first mover advantage" is actually true for applications that have network effects which is definitely the case for social software which a lot of software has become today.

• The "diversity in conferences" recurring debate was kicked off again by a blog post by Jason Kottke entitled Gender Diversity at Web Conferences which encouraged the interesting responses from folks like Eric Meyer, Anil Dash and Shelley Powers. They are all good posts with stuff I agree and disagree with in them but I wasn't moved to write until I read the post Why are smart people still stuck on gender and skin-color blinders? by Tantek Çelik where he wrote

  Why is it that gender (and less often race, nay, skin-color, see below) are the only physical characteristics that lots of otherwise smart people appear to chime in support for diversity of?

  E.g. as long as we are trying for greater diversity in superficial physical characteristics (superficial because what do such characteristics have to do with the stated directly relevant criteria of "technical expertise, speaking skills, professional stature, brand appropriateness, and marketability" - though perhaps I can see a tenuous link with "rainbow" marketing), why not ask about other such characteristics?

  Where are all the green-eyed folks?

  Where are all the folks with facial tattoos?

  Where are all the redheads?

  Where are the speakers with non-ear facial piercings?

  Surely such speakers would help with "hipness" marketing.

  I found this post to be disingenious and wondered how anybody could downplay the gender and racial bias in the "Web 2.0" technology conference scene by equating it to a preference for green eyed speakers. So I decided to throw in my $0.02 on this topic...again.

  After the last ETech, I realized I was seeing the same faces and hearing the same things over and over again. More importantly, I noticed that the demographics of the speaker lists for these conferences don't match the software industry as a whole let alone the users who we are supposed to be building the software for.

  There were lots of little bits of ignorance by the speakers and audience which added up in a way that rubbed me wrong. For example, at the 2005  Web 2.0 conference a lot of people were ignorant of Skype except as 'that startup that got a bunch of money from eBay'. Given that there are a significant amount of foreigners in the U.S. software industry who use Skype to keep in touch with folks back home, it was surprising to see so much ignorance about it at a supposedly leading edge technology conference. The same thing goes for how suprised people were by how teenagers used the Web and computers. Additionally, there are just as many women using social software such as photo sharing, instant messaging, social networking, etc as men yet you rarely see their perspectives presented at any of these conferences. 

  When I think of diversity, I expect diversity of perspectives. People's perspectives are often shaped by their background and experiences. When you have a conference about an industry which is filled with people of diverse backgrounds building software for people of diverse backgrounds, it is a disservice to have the conversation and perspectives be homogenous. The software industry isn't just young white males in their mid-20s to mid-30s nor is that the primary demographic of Web users.

  Personally, I've gotten tired of attending conferences where we heard more about technologies and sites that the homogenous demographic of young to middle aged, white, male computer geeks find interesting (e.g. del.icio.us and tagging) and less about what Web users actually use regularly or find interesting (hint: it isn't del.icio.us and it sure as fuck isn't tagging).

 

Twice this week, I've been impressed by how some rant I made in my blog turned into an implemented feature in Web software that hundreds of thousands of people use. The first incident comes from my blog post Why Feedburner Doesn't Count Outlook 2007 Subscribers where I wrote about the fact that FeedBurner doesn't track subscribers to my feed who're using Outlook 2007 because it uses the same user agent string as Internet Explorer 7. Thus I was pleasantly surprised when I logged into FeedBurner and saw the following

It seems that while I was on vacation the folks at FeedBurner decided to implement a solution to the problem I pointed out even though it isn't their fault. Nice.

The second incident comes in response to my post MSN SoapBox in Public Beta where I mentioned that neither Google Reader nor Bloglines would display videos from MSN Soapbox embedded in a blog post. Yesterday Mihai Parparita who works on Google Reader let me know that they added support for that while I was on vacation. That means if you are reading this in Google Reader you should see a video on the next line

Thanks to the Google Reader team for implementing this so quickly.

 

Nick Carr has a blog post entitled Googlegate in North Carolina where he writes

North Carolina's Senate Finance Committee is hastily arranging hearings for next week on the state's use of tax incentives to lure businesses, as public outrage mounts over disclosures that Google was granted as much as a quarter billion dollars in secret tax breaks for a plant expected to employ approximately 200 workers. There's no word yet on whether any Google officials will be asked to testify.
...
The Googlegate controversy is unlikely to abate any time soon. Troubling new details of the secret deal-making continue to emerge. Today's Charlotte Observer features a long article describing how public officials leaned on some local residents to sell their homes to make way for the Google plant. The mayor of the town of Lenoir, Davis Barlow, and the county commissioner, Tim Sanders, were among the officials who, according to the paper, went "door-to-door on behalf of the Internet giant Google. In some cases, officials returned to homes four or five times. Barlow and Sanders effectively used the personal touch, avoiding a drawn-out public debate that Google was secretly telling them would scuttle the deal. That personal touch enabled some residents to feel comfortable in selling their property."

This reminds me of a comment I once heard about why the deal makers at GOOG are such hardball players. It goes back to the Google Founders' Awards which were intended to be a way to significantly reward people who add value to the company's bottom line.  Since this award is worth millions of dollars, there is a lot of stiff competition and I'd heard that it ended up the sales folks, acquisitions experts and other deal makers who end up as the primary contenders for the award.

I guess it makes sense, which other job functions can say that they directly save or benefit the company a quarter of a billion dollars on the bottom line? Not the lead developer of Google Calendar or the PM who wrote the spec for Google Base, that's for sure. :)

Unfortunately, when you put millions of dollars in incentives in front of your employees you shouldn't be surprised if they start cutting ethical corners to make things happen. Even CEOs and CFOs aren't immune from this which is why we have Sarbanes Oxley today.

 

It's unfortunate that when people become successful, there are always the jealous waiting on the wings to tear them down and exaggerate every flaw. Today was one of those days for Flickr. You can tell things are fishy when the people complaining loudest about a company aren't its users but its competitors. We have

1. There's Thomas Hawk who writes There's Some Mighty Pissed Off Flickr Members Right Now

   [I am CEO of Zooomr]

   Flickr needs to reverse the asinine decisions made today to force people to merge their accounts with Yahoo and to place new limits on your contacts and tags.
   
   Here are some of the more interesting comments pulled over the past few hours from two flickr forums where they are taking a beating over this right now.
   
   People are pissed.

2. And There's Don MacAskill who doesn't just stop at gloating stoking the flames reporting on the problems at a competitor's service. In his post The Dark Side of the Flickr Acquisition where he writes

   You asked for it, you got it: SmugMug is offering 50% off to all Flickr refugees. Just sign up for our free trial using the coupon code flickr and if you like what you see, you’ll get 50% off your first year.

   We’re getting some email from ‘Old Skool’ Flickr users asking us if they can get a discount because Yahoo’s making some changes they don’t like. Thomas Hawk has more coverage over on his blog, you can read the Flickr Forums for more reactions, and even check out the Flick Off group (aka the Flickr Accounts Mass Suicide Countdown group).

I looked at the announced Flickr changes and didn't see anything particularly shocking. For performance/architectural reasons, they are limiting people to having 3000 buddies and 75 tags. That sucks but it isn't the end of the world. If anything, it just points out the architectural limits of Flickr's backend to competitors but those limits should be fine for most users. The second issue seems to be that you'll soon need a Yahoo! ID to access Flickr. As Anil Dash states in his post I am okay with my Yahoo sign-in, people have known this for months so it shouldn't be a surprise and they've encouraging people to get a Yahoo! login on the sign-in page for a while now.

I wouldn't be surprised if a lot of the "aggrieved users" are actually astroturfers from competing Web 2.0 photo sharing sites.

 

I like the concept of online Q&A sites and I have to say that I've been quite impressed at how successful Yahoo! has been with Yahoo! Answers. Not only did they build a good end user experience but they followed up with heavy cross promotion on their other services, TV ads and getting lots of real-world celebrities to use the service. My favorite questions asked by real-world celebrities thus far

Based on your own family's experience, what do you think we should do to improve health care in America? asked by Hillary Clinton (U.S. Senator and Presidential Candidate)

What should we do to free our planet from terrorism? asked by Dr. APJ Abdul Kalam (President of India)

That's pretty freaking cool. Kudos to the Yahoo! Answers for being able to pull off such a great promotion and build such a successful service in such a short time. 

 

Interesting, it seems Flickr have formalized the notion of partitioning tags into namespaces with their introduction of Machine Tags which are described as

# What are machine tags?

Machine tags are tags that use a special syntax to define extra information
about a tag.

Machine tags have a namespace, a predicate and a value. The namespace defines a class or a facet that a tag belongs to ('geo', 'flickr', etc.) The predicate is name of the property for a namespace ('latitude', 'user', etc.) The value is, well, the value.

Like tags, there are no rules for machine tags beyond the syntax to specify the parts of a machine tag. For example, you could tag a photo with :

* flickr:user=straup

* flora:tree=coniferous

* medium:paint=oil

The XML geek in me can't help but squint at the term "namespaces" and wonder how they plan to avoid naming collisions in a global namespace (e.g. if multiple people choose the same name for a namespace they create) . I guess this is no different from people using the same word to tag an item while meaning totally different things (e.g. "apple", "glass", "light", etc) and folksonomies like Flickr seem to handle this just fine.

Creating facets in tags like this isn't new, del.icio.us has had this for a while and it it looks like a good way to create ways to create hidden tags that the system can use for performing special operations without it being in the user's face.

Now that the two granddaddies of tagging both provide this functionality, I wonder how long it takes for machine tags to wind it's way through all the tagging systems in the various copycat Web 2.0 sites on the Web.

 

In response to my recent post entitled ODF vs. OOXML on Wikipedia one of my readers pointed out

Well, many of Weir's points are not about OOXML being a "second", and therefore unnecessary, standard. Many of them, I think, are about how crappy the standard actually is.

Since I don't regularly read Rob Weir's blog this was interesting to me. I wondered why someone who identifies himself as working for IBM on various ODF technical topics would be spending a lot of his time attacking a related standard as opposed to talking about the technology he worked. I assumed my reader was mistaken and decided to subscribe to his feed and see how many of his recent posts were about OOXML. Below is a screenshot of what his feed looks like when I subscribed to it in RSS Bandit a few minutes ago

Of his 24 most recent posts, 16 of them are explicitly about OOXML while 7 of them are about ODF.

Interesting. I wonder why a senior technical guy at IBM is spending more time attacking a technology whose proponents have claimed is not competitive with it instead of talking about the technology he works on? Reading the blogs of Microsoft folks like Raymond Chen, Jensen Harris or Brian Jones you don't see them dedicating two thirds of their blog postings to bash rival products or technologies.

From my perspective as an outsider in this debate it seems to me that OOXML is an overspecified description of an open XML document format that is backwards compatible with the billions of documents produced in Microsoft Office formats over the past decade. On the other hand, ODF is an open XML document format that aims to be a generic format for storing business documents that isn't tied to any one product which still needs some work to do in beefing up the specification in certain areas if interoperability is key.

In an ideal world both of these efforts would be trying to learn from each other. However it seems that for whatever reasons IBM has decided that it would rather that Microsoft failed at its attempt to open up the XML formats behind the most popular office productivity software in the world. How this is a good thing for Microsoft's customers or IBM's is lost on me.

Having a family member who is in politics, I've learned that whenever you see what seems like a religious fundamentalism there usually is a quest for money and/or power behind it. Reading articles such as Reader Beware as ODF News Coverage Increases it seems clear that IBM has a lot of money riding on being first to market with ODF-enabled products while simultaneously encouraging governments to only mandate ODF. The fly in the ointment is that the requirement of most governments is that the document format is open, not that it is ODF. Which explains IBM's unfortunate FUD campaign. 

Usually, I wouldn't care about something like this since this is Big Business and Politics 101, but there was something that Rick Jellife wrote in his post An interesting offer: get paid to contribute to Wikipedia which is excerpted below

So I think there are distinguishing features for OOXML, and one of the more political issues is do we want to encourage and reward MS for taking the step of opening up their file formats, at last?

The last thing I'd personally hate is for this experience to have soured Microsoft from opening up its technologies so I thought I'd throw my hat in the ring at least this once.

PS: It's pretty impressive that a Google search for "ooxml" pulls up a bunch of negative blog posts and the wikipedia article as the first couple of hits. It seems the folks on the Microsoft Office team need to do some SEO to fix that pronto.

 

By now it's common news that Google has been hit by what seems like half a dozen or more cross site scripting security flaws in the past month. If you missed the news, you can read blog posts like More Google security failures and Wow, more Google XSS problems which contain links to some of the stories of recent exploits. The bugs in those blog posts aren't exhaustive, I've seen some blog posts about exploits that don't seem to have hit the mainstream tech blogs such as the one mentioned in the blog post Pending Members - Google Groups XSS Bug [Part 2].

Anyway, the fact that Google is having problems with XSS issues isn't terribly interesting and should be an expected part of the growing pains as they go from a service that doesn't store any user data to one that aims to be the repository of all their user's data. That requires an entirely different approach to security. What I did find interesting was a blog post on the Google Blogoscoped blog entitled On Google Security which stated

Today, it almost seems as if every single product team in the Googleplex has the “power” to accidentally introduce a Google Account risk with an HTML injection hole, or another kind of cross-site scripting issue. An exotic Blogger bug was able to reveal your Google Docs, even if you’re not blogging with Blogger – an improbable Google Base bug was able to reveal your personalized homepage, even when you’ve never worked with Google Base**. I would argue: these things happen, individual developers and developer teams make errors. It’s impossible not to. There are ways to automatically test against HTML injections, but such tools too need to be handled by humans.

The real problem, and solution, might be on the higher level of the system architecture – the way Google integrates its services and handles cookie data. Right now, the Google Office product partly resembles a mighty convenient & long chain... a chain which is only as strong as its weakest link. Is this a trade-off we’ll just have to make with future web apps, or are there ways to improve on the situation... either by users, or those building browsers, or those developing web apps?

Those who ignore history are doomed to repeat it. None of the problems listed are unique to Google. Any portal that provides multiple services that require the user to login is vulnerable to these problems. This includes competing portals like Yahoo!, MSN and AOL. All of these services have had to encounter and protect users against the very same problems Google is having difficulty dealing with today.

It is likely that with time, Google will stumble upon the same set of best practices that are common knowledge amongst its portal competitors who have been in the game a lot longer. Thinking that this is a problem that affects "the future of Web apps" ignores the history of the Web. 

In the meantime, if you are a Web developer at Google, I'd suggest reading Chapter 12 of Writing Secure Code by Michael Howard. After that, take a look at You know about XSS. How about XSRF/CSRF? which happens to use a Google service as an example of Cross Site Request Forgery attack (XSRF).

That which doesn't kill us only makes us stronger. ;)

 

Danny Sullivan over at Search Engine Land has a post entitled comScore: Google Wins Again & IE7 Doesn't Stop Microsoft's Slide where he writes

It's that time again -- search popular stats for last month are coming out. Actually, Hitwise sent me their figures earlier this month but I'm diving in with the comScore figures that just came out. The main real news is despite the Internet Explorer 7 launch, Microsoft's Live continues to show a drop in usage.

What is puzzling to me is that people thought that the release of IE 7 would cause a increase in search share for Microsoft's search engine and a decline in competitors. The fact is that built-in search boxes within the browser encourage people to treat search as a feature of the browser instead of a site they visit. That means that the defaults built into the browser/operating system are important. But what exactly is the default search engine on most PCs running IE 7? I don't have any hard numbers but here's some data from my post about this entitled Competing with Google is Like the War in Iraq which stated

The combination of the proliferation of search toolbars and a new generation of Web browsers with built-in search boxes (e.g. IE 7 and Firefox) have reduced the need for users to actually go to websites to perform a search. This means that it is now very important to be the search engine that is used when a user enters a search directly from their browser. Guess which search engine is the one used by your browser if you

1. Are you a user of the Firefox browser?
2. Are you a user of the Opera browser?
3. Are you a user of IE 7 and have installed Adobe Acrobat?
4. Are you a user of IE 7 and have installed the Java runtime?
5. Are you a user of IE 7 and have installed the WinZip archive utility?
6. Are you using a newly purchased Dell computer?
7. Are you a user of the Google Toolbar?

Yes, the answer is Google in every case. So even if you are an Internet n00b who hasn't made up their mind about which search engine to choose, there is a large chance that the default search engine you end up using thanks to recent innovations in IE 7 and Firefox will be Google.

If anything, browsers like Firefox and IE 7 make it harder for users to switch from Google not easier because it gets them away from the notion of visiting websites to perform searches and instead they just accept whatever default the browser provides.

 

My sister is paying me a surprise visit this weekend and I decided to look on the Web for ideas for what we could do together this weekend. My initial thoughts were that we'd go to the movies and perhaps check out the Bodies: The Exhibition. I wouldn't to see if I could get a better suggestion on the Web.

My first instinct was to try Seattle - City Search but had to give up when I realized the only events listed for today were either announcements of what DJs would be at local clubs tonight or announcements sales at local stores. Another thing that bugged me is how few ratings there were for events or locations on City Search. This reminds me of a blog post on Search Engine Land entitled Local And The Paradox of Participation which came to a set of incorrect conclusions about a poll that claimed that people are equally likely to post a positive or negative review of an event or location. The incorrect conclusion was that it is a myth that few people are likely to post reviews. Given that locations and events that are attended by thousands of people tend to only have dozens of reviews on almost every review site I've ever seen seems to make this a fact not a myth. The poll only seems to imply that people are willing to share their opinion if prompted which is totally different from someone attending a nightclub or concert then feeling compelled to visit one of umpteen review sites to share their opinion. What is surprising to me is that there doesn't seem to even be a small community of die hard reviewers on City Search which is unlike most review sites I've seen. Just compare Amazon or IMDB which both seem to have a number of reviewers who are on top of certain categories of products.

Anyway, what does this have to do with Google? Well, I went to Rich Skrenta's much vaunted starting point of the Intenet and tried some queries such as "local events", "seattle events" and "events in seattle" with pathetic results. The only useful links in the search results page led me to a couple of event search engines (e.g. NWsource, Upcoming) that were pathetically underpopulated with event information. None of them even had a listing for Bodies: The Exhibition. Lame. 

I tried Google Local which turned out to be redirect to their mapping site. Shouldn't a local search engine be able to find events in my local area? Double lame.

Before you bother pointing it out, I realize that other search engines don't do a much better job either. This seems to point to an opportunity to add a lot of value in what must be a very lucrative search market. I'm surprised that Yahoo! hasn't figured out how to do more with their purchase of Upcoming.org. Then again Yahoo! hasn't figured what to do with any of the Web 2.0 startups they've purchased so maybe that is expecting too much. Maybe Google will purchase Eventful.com and fix this fairly big hole in their search offerings. Somehow I doubt it. .

 

The bigger Google grows, the more it seems that like every large company it's products now have to pay a strategy tax which may end up shortchanging their users. If you are unfamiliar with the term, you should read Dave Winer's excellent essay Strategy Tax which is excerpted below

Ben explained that sometimes products developed inside a company such as Microsoft have to accept constraints that go against competitiveness, or might displease users, in order to further the cause of another product. I recognized the concept but had never heard the term.

An example. Consider a company that develops both a Web browser and a word processor. The product team for the browser might get lots of input from users saying "We'd like a better editor in the browser." It would be natural to give the users what they want. So they put the feature in their project plan, but when it comes up for review, the CEO shoots them down. "They should use the word processor," he says.

Another example. An electronics company makes a portable music player, and also owns a recording studio. The team working on the music player knows that there's a big market for players that work with a format that's popular with computer users, but lacks sophisticated copy protection. They project huge sales. But when the product comes up for review, the CEO shoots them down. "We'll lose sales of our music," he says.

Before I read this essay I thought this was a problem unique to Microsoft and also thought that I was being astute in observing the trend at the company when in truth the term was part of the cultural knowledge of the company while I was still in programming diapers. Over time, it has become clear to me that this is a problem that affects any business endeavor where different product units either rely on each other or indirectly compete with each other [assuming that the company isn't dysfunctional enough to have products that directly compete against each other]. Below are three examples of how the strategy tax is affecting Google, all of which are observations by other bloggers which I've noticed myself but refrained from mentioning since I work for a competitor and it would have come off as sour grapes.

Disincentive to Improve Search Due to Ad Business

In his post Good Luck Jimmy, Dave Winer writes

Google is repeating the pattern of the previous generation of search engines (Alta Vista, Infoseek) were doing when Google zigged to their zag, so successfully. Today, Google is fattening up and spreading out, going after Microsoft in productivity apps, chasing the TV networks with YouTube. Etc etc. Today search is only one of the things Google is doing, and it may not be the most important thing.

Today Google's profits come from ads, and that business gives them a reason to keep search weak. They want you to do a lot of searching to find what you're looking for -- and the stuff they find for you for free is competing with the stuff they make money on. So Google actually has a disincentive to make search better.

A few months ago, I used to get into regular debates with Matt Augustine who argued that the fact that companies like Google make so much money from search advertising seems like a bug in the system. Matt would argue that if search engines were really that good at finding what we want, we would never have to click on the results they had been paid to show us unless we were deceived into doing so.

This seems to put a 'glass ceiling' on how good the search engine can be because you don't want people to stop clicking on ads when you make billions of dollars a year from them doing so.

Promoting Google Services at the Expense of the Integrity of Search Results and it's Advertisers

Blake Ross has a blog post entitled Tip: Trust is hard to gain, easy to lose where he writes

But Google lost me today, and it didn’t take much:

Google is now displaying “tips” that point searchers to Google Calendar, Blogger and Picasa for any search phrase that includes “calendar” (e.g. Yahoo calendar), “blog” and “photo sharing,” respectively. This is clearly bad for competitors, and it’s also a bad sign for Google. But I generally support anything that benefits users, including monopolistic packaging. I believe, for instance, that shipping Internet Explorer with Windows was a good move. So why are tips bad for users?
...
The tips are different—and bad for users—because the services they recommend are not the best in their class. If Google wants to make it faster and easier for users to manage events, create a blog or share photos, it could do what it does when you search GOOG: link to the best services. To prevent Google from being the gatekeeper, the company could identify the services algorithmically. But if that sounds familiar, perhaps that’s because Google already works that way. After all, Google is predicated on the idea that the democratic structure of the Web will push the cream to the top. Search for “photo sharing” and you should already get the highest quality services. According to Google, Picasa is not one of them.
...
While advertisers compete to be first in a string of lookalike ads that are often shunted to the side, Google now determines the precise position and appearance of ads tips that are not subject to any of the same rules. Its ads get icons while others don’t, and if you think that’s small potatoes, you are not an advertiser: images boost clickthrough. Google can make a Picasa ad say “Easier to use than Kodak,” but Kodak cannot create an ad that reads “Easier to use than Picasa.” And the kicker: neither the highest quality ads nor the highest quality search results can replace these tips.

The "strategy tax" here is being paid by the search engine and advertising groups at Google. To carry along Google services that Blake points out are not best in class, Google is foregoing ad dollars from a number of lucrative keywords and causing distrust in the search engine by the very power users upon whose backs it rose to fame in the first place. Google used to brag about how unlike other search engines, they don't use misleading ads that people can confuse for search results. However I tend to agree with the last statement in Blake's post

Perhaps the most nefarious aspect of this feature is how it operates within our collective blind spots. Advertisers are happy that Google no longer invades the canonical Ad Results. Technology purists continue to see untainted Search Results. But does my mother make that distinction? How much does a result have to look like a Result to cross the line?

Indeed.

Artificially Promoting it's Products in Search Results

From a comment highlighted in the post Google's Silent Monopoly Redux (Google Responds - Issues Public Statement) which states

But type in "maps". Google is again first. Ahead of Mapquest. Ahead of Yahoo maps. Yahoo also has backlinks out the ying yang. So why is it third? And mapquest has been around forever.. I'm sure there are more links to mapquest.com than to the maps.google.com URL, simply because the millions of web pages that linked their directions to Mapquest from 1996 to 2004 didn't all rush out and change all their links to Google maps in February of 2005 (when it was released), even if Google's is a better product.

Next, try "mail". Despite the fact that Yahoo mail has been around forever, and has all sorts of links, and that Hotmail has also been around forever, Gmail still manages to come up first.

And the most interesting thing about this particular keyword? The word "mail" by itself doesn't even appear on the gmail.google.com page! The words gmail, webmail, and email appear. But not "mail". At least on the Yahoo page, the word "mail" does indeed appear. Yet Google still manages to rank ahead of Yahoo.

Finally, try "answers". Yes, answers.google.com comes up second, rather than first. But answers.yahoo.com comes in third! Is the Google Answers site really getting that many more links than Yahoo's? Especially in light of the fact that Google recently decided to kill it, because almost no one was using it, while Yahoo's usage (and therefore also linkage, no doubt) are skyrocketing?

This claim was actually the most interesting to me since Google is very adamant about the integrity of their search results and claims we don’t accept payment for inclusion in our index, nor do we manipulate search results by hand. I tried a number of these queries myself and was pretty shocked by the results especially when it came to "mail". Here are some screenshots that illustrate the point

1. Search results for "mail" on Google

2. Number of links to gmail.google.com (also the same as gmail.com) according to Google

3. Number of links to mail.yahoo.com according to Google

It's hard to imagine any objective metric that should make Gmail show up ahead of Yahoo! Mail in a search for the word "mail". Of course, this doesn't mean that Google is tampering with search results "by hand". Their algorithm can simply have allowances to rank sites in their domain or linked from their domain higher without having to actually sully their hands by tweaking individual results by hand. Still, if Google is how the world finds information and we are increasingly being pointed to information that financially benefits Google, doesn't that taint the much vaunted claim of the integrity of their search results even if it is being done in an automated manner?