Robert Scoble has a post entitled Search for Toshiba music player demonstrates search engine weakness where he complains about relevance of search results returned by popular web search engines. He writes
Think search is done? OK, try this one. Search for: Toshiba Gigabeat Did you find a Toshiba site? All I see is a lot of intermediaries. I interviewed a few members of the MSN Search team last week and I gave them hell about this. When I'm writing I want to link directly to the manufacturer's main Web site about their product. Why? Because that's the most authoritative place to go. But I keep having trouble finding manufacturer's sites on Google, MSN, and Yahoo. Relevancy ratings on search engines still suck. Let's just be honest about it as an industry. Can the search researchers find a better algorithm? I sure hope so. Here, compare for yourself. If you're looking for the Toshiba site, did you find what you're looking for when you do searches Google ? On Yahoo ? On MSN ? Here's the right answer: http://www.gigabeat.toshiba.com . Did you find it with any of the above searches? I didn't.
Think search is done? OK, try this one. Search for:
Toshiba Gigabeat
Did you find a Toshiba site? All I see is a lot of intermediaries.
I interviewed a few members of the MSN Search team last week and I gave them hell about this. When I'm writing I want to link directly to the manufacturer's main Web site about their product. Why? Because that's the most authoritative place to go.
But I keep having trouble finding manufacturer's sites on Google, MSN, and Yahoo.
Relevancy ratings on search engines still suck. Let's just be honest about it as an industry.
Can the search researchers find a better algorithm? I sure hope so.
Here, compare for yourself. If you're looking for the Toshiba site, did you find what you're looking for when you do searches Google ? On Yahoo ? On MSN ?
Here's the right answer: http://www.gigabeat.toshiba.com . Did you find it with any of the above searches? I didn't.
The [incorrect] assumption in Robert Scoble's post is that the most relevant website for a person searching for information on a piece of electronic equipment is the manufacturer's website. Personally, if I'm considering buying an MP3 player or other electronic equipment I'm interested in (i) reviews of the product and (ii) places where I can buy it. In both cases, I'd be surpised if the manufacturer's website would be the best place to get either.
Relevancy of search results often depends on context. This is one of the reasons why the talk on Vertical Search and A9.com at ETech 2005 resonated so strongly with me. The relevancy of search results sometimes depends on what I want to do with the results. A9.com tries to solve this by allowing users to customize the search engines they use when they come to the site. Google has attempted to solve this by mixing in both traditional web search results with vertical results inline. For example, searching for MSFT on Google returns traditional search results and a stock quote. Also searching for "Seattle, WA" on Google returns traditional web search results and a map. And finally, searching for "Toshiba Gigabeat" on Google returns traditional web search reults and a list of places where you can buy one.
Even with these efforts, it is unlikely any of them would solve the problem Scoble had as well as if he just used less ambiguous searches. For example, a better test of relevance is which search engine gives the manufacturer's website for the search for "toshiba gigabeat website".
I found the results interesting and somewhat surprising. There definitely is a ways to go in web search.