Faroo Employs P2P Technology to Search and Index the Web(startupmeme.com)
startupmeme.com
Faroo Employs P2P Technology to Search and Index the Web
http://startupmeme.com/2007/05/14/faroo-employees-p2p-technology-to-search-and-index-the-web/
http://startupmeme.com/2007/05/14/faroo-employees-p2p-technology-to-search-and-index-the-web/
One is speed: a search may touch on many 'shards' that then have to be combined and ranked somewhere. That's hard enough to do when you have 100K machines linked with 100Mbps up and down.
The second is honesty: search ranking is a high-value target. It's ripe for a takeover by zombies, even if each node's keyspace is assigned by some trusted authority.
The third is coordinating a web-wide crawler: how to avoid floods, duplicate work, spammy submissions, etc.
The fourth is most important: the system has to work much better than Google for people to care enough to use it. Simply being P2P (i.e. cheaper for the operator) is not enough.