Ask HN: What sites, e.g. experts-exchange.com, would you remove from Google?
In Google's results, what useless sites do you notice coming up again and again?
38 comments
Its funny you mention experts-exchange. Whenever I see that as a search result, its usually a relief. I have yet to see an experts-exchange page that did not have the information I was looking for on it, even though they do make it rather difficult to find (You have to scroll all the way to the bottom to see the actual answer).
Compared to the huge number of real spam sites that usually come up when search for technology related questions, like all of those mailing-list archives/mirrors, and the huge number of pages that ripped old forum content, experts-exchange is amazing.
I can't really think of any pages I would want removed. I would like to see better pages for a lot of products, but that seems to have more to do with the manufacturers simply not creating the pages than with Google. Search results for physical goods seem to be fairly lacking, its always hard to find information about them and not just stores selling them.
Compared to the huge number of real spam sites that usually come up when search for technology related questions, like all of those mailing-list archives/mirrors, and the huge number of pages that ripped old forum content, experts-exchange is amazing.
I can't really think of any pages I would want removed. I would like to see better pages for a lot of products, but that seems to have more to do with the manufacturers simply not creating the pages than with Google. Search results for physical goods seem to be fairly lacking, its always hard to find information about them and not just stores selling them.
The thing with experts exchange is you have to pay to see the answers, right? (unless I have missed something).
Because they are all over google for most error related searches it is frustrating to click without looking and find a useless page (taunting you almost :P). It hides all the free useful pages with the same answers on :(
I wouldnt remove it from Google though - and I agree, it is better than all the spam link sites.
Because they are all over google for most error related searches it is frustrating to click without looking and find a useless page (taunting you almost :P). It hides all the free useful pages with the same answers on :(
I wouldnt remove it from Google though - and I agree, it is better than all the spam link sites.
It implies that you need to pay to see the answers even though they are presented there on the page. The first block of "answers" is deliberately mangled to imply you need to pay to see them, and then there is a block of advertisements, but below that is the same answers presented in plain text.
Very shifty indeed.
Very shifty indeed.
I have an account there that I created years ago, and just login if I want to read an answer. I never paid for anything on the site.
It used to be free to register, so unless they've changed something, all you need to do is create an account.
It used to be free to register, so unless they've changed something, all you need to do is create an account.
You only need to sign in if you are NOT referred by Google.
I think a lot of people don't realize the answers are at the bottom, so they click on the link, see the request to register/pay to see the posts, and immediately back up, cursing the pseudo-paywall.
I've seen people say this before - how do you mean "at the bottom". Every page I have seen highlights the "accepted answer" in green and does ask you to pay to access it..... am I missing a workaround?
Their ploy changes sometimes, but the answer is always there on the page somewhere.
http://www.experts-exchange.com/Apple/Operating_Systems/OS_X...
If you scroll down a bit it says "answer is available to premium members". But the page would not rank on Google if it didn't have actual content. Scroll all the way to the very bottom of the page. (APPLE-DOWNARROW on Mac) Bingo.
http://www.experts-exchange.com/Apple/Operating_Systems/OS_X...
If you scroll down a bit it says "answer is available to premium members". But the page would not rank on Google if it didn't have actual content. Scroll all the way to the very bottom of the page. (APPLE-DOWNARROW on Mac) Bingo.
No, I scroll all the way down and don't see anything. Do you need to be logged in? Even if you only have to be logged in without paying, it's still annoying as heck.
It only works for certain referring pages. Google being one of them. Direct access, or a HN referral will not show the answers.
I never knew that, had to sign up for HN just so I could thank you for that info, now google search when EE shows up I don't have to skip those.
aha that's the crucial bit of info I had never heard before :) cheers works for me now.
Hmm what browser do you use?
Firefox and Chrome on Windows XP dont appear to have content hidden anywhere.
This is what I see on the example you used. http://screencast.com/t/mJLb2rsD Any ideas where I am missing it? (checked the source too)
Firefox and Chrome on Windows XP dont appear to have content hidden anywhere.
This is what I see on the example you used. http://screencast.com/t/mJLb2rsD Any ideas where I am missing it? (checked the source too)
The price comparison sites that don't contain reviews but do manage to come high up in a Google search for 'X reviews'.
swik.net - For some reason I see it's "search" pages every day in search results. Put two protocol names together, or a language + protocol and you're almost guaranteed to get it in the second place :/
scribd - They hide your search phrases somewhere in the page, so that you can think you've found exactly what you were looking for. But all you get is some random, slightly related book that doesn't help you at all.
scribd - They hide your search phrases somewhere in the page, so that you can think you've found exactly what you were looking for. But all you get is some random, slightly related book that doesn't help you at all.
Seems like it wouldn't be too hard to whip up an extension (or just a greasemonkey script) to append "-site:blah.com" entries to all google searches, according to a user defined list.
In fact, I'd be surprised if such an extension didn't already exist (I'm not interested enough to go look...)
In fact, I'd be surprised if such an extension didn't already exist (I'm not interested enough to go look...)
The "Customize Google" extension does this
1. Sites which are useless for everyone e.g.: splogs, etc.
2. Some sites could be useful for a few, while being completely useless for the rest i.e. letting users remove certain sites from their results only
3. Like the ad-block software, people could subscribe to well-defined popular lists to get started.
4. I would also segment by verticals. E.g.:when you do a people search, you could bucket results into:
a.) sites where the user, him/her self has entered the info - linkedin, facebook type websites
b.) sites where a community has created a profile - wikipedia
c.) sites that inteliigently crawl other sites and present the info -- lead411 , pipl, zoominfo, spoke, etc are useless to me
2. Some sites could be useful for a few, while being completely useless for the rest i.e. letting users remove certain sites from their results only
3. Like the ad-block software, people could subscribe to well-defined popular lists to get started.
4. I would also segment by verticals. E.g.:when you do a people search, you could bucket results into:
a.) sites where the user, him/her self has entered the info - linkedin, facebook type websites
b.) sites where a community has created a profile - wikipedia
c.) sites that inteliigently crawl other sites and present the info -- lead411 , pipl, zoominfo, spoke, etc are useless to me
Here in Argentina, if you search in Spanish, todoar.com.ar comes up in the top 10 for almost everything. I'll hand it to the owners, they know how to do SEO. The problem is there's no useful content on any of their ad-ridden pages.
All the stores that have been spidered
(amazon.com etc)
Googles own sites (youtube.com, images.google.com, etc)
All the cloaked sites (sites that will redirect you after landing, or that show you different content than they showed the search engine)
All the spammers (you've already mentioned experts xchg, but there are many many more)
Anything that points to or contains the word sedoparking
All the crappy blogs
I'm sure that google could do with several billion pages less than they index right now and their quality would shoot up.
Googles own sites (youtube.com, images.google.com, etc)
All the cloaked sites (sites that will redirect you after landing, or that show you different content than they showed the search engine)
All the spammers (you've already mentioned experts xchg, but there are many many more)
Anything that points to or contains the word sedoparking
All the crappy blogs
I'm sure that google could do with several billion pages less than they index right now and their quality would shoot up.
I would argue that indexing a bunch of crap actually _improves_ the quality of Google's search.
Just like trying to filter e-mail spam, the larger your spam corpus, the easier it is to differentiate between valid content and junk.
Just like trying to filter e-mail spam, the larger your spam corpus, the easier it is to differentiate between valid content and junk.
If they did a better job on the differentiation then I would agree with you.
All the stores that have been spidered (amazon.com etc)
Googles own sites (youtube.com, images.google.com, etc)
I'm glad this is a hypothetical question because I find both of the above very useful. I often suffix "amazon" or "youtube" on to a search - saves me actually having to go to those sites to perform a search in most cases.
I'm glad this is a hypothetical question because I find both of the above very useful. I often suffix "amazon" or "youtube" on to a search - saves me actually having to go to those sites to perform a search in most cases.
site:amazon.com
that works just fine and it would keep the rest of the search results clean.
that works just fine and it would keep the rest of the search results clean.
Doesn't quite roll off the fingers as well as amazon or wikipedia though. Anyway, those entries are as valuable as anything else. If I'm searching for the title of a book, why shouldn't stores that sell the book come up?
It sounds like some people in this thread would prefer non-profit communist Google, rather than a proper search engine.
It sounds like some people in this thread would prefer non-profit communist Google, rather than a proper search engine.
I actually wrote myself a greasemonkey script to hide results from w3schools once upon a time. I'll try to dig it up.
Their docs are almost always missing the one piece of information I actually want (usually a function definition) and they nearly always rank very highly for such queries.
Their docs are almost always missing the one piece of information I actually want (usually a function definition) and they nearly always rank very highly for such queries.
If Google won't filter the garbage out of the results why doesn't someone build something that will, for example, instantly remove any webpage that hasn't been updated since 1999 etc.etc. Off topic yes, but I find most of the sites that come up, well, useless.
I have my own 'custom' search engine that filters out those annoying sites. You can use it here: http://www.google.com/coop/cse?cx=015394218516616598090:kmqd...
I've often thought that if you removed any site that uses Google Adsense you'd get much better results.
I wish I had known the experts-exchange "hack" before...
I have been compiling a list of useless sites to share with the world. You will need CustomizeGoogle for Firefox or your own userscript in the browser of your choice. Come to think of it, I might even create some userscripts/plugins for other browsers to go with my useless website list.
If you'd like to contribute your own lists or userscripts, let me know.
If you'd like to contribute your own lists or userscripts, let me know.
Here is an example from my list, I call this the "site info" category.
http://*.aboutus.org/* http://www.quantcast.com/* http://www.websiteoutlook.com/* http://www.xomreviews.com/* http://www.statbrain.com/* http://www.alexa.com/data/* http://searchanalytics.compete.com/* http://builtwith.com/?* http://*.compete.com/* http://www.who.is/*
http://*.aboutus.org/* http://www.quantcast.com/* http://www.websiteoutlook.com/* http://www.xomreviews.com/* http://www.statbrain.com/* http://www.alexa.com/data/* http://searchanalytics.compete.com/* http://builtwith.com/?* http://*.compete.com/* http://www.who.is/*