Author here. Thanks for submitting this project! I made this because I thought it would be funny if publisher websites had SciHub links that look like they belong on the website [0]. I didn't know about the bookmarklet when I started this. Maybe I should have used that instead. Oh well.
I'm currently waiting for Mozilla to accept this into the add on store. If that passes, I will submit it to Chrome as well.
I think a lot of this is due to Google both owning search and the ads on the websites (AdSense). There’s an incentive for them to prioritize click farms (and other sites filled with their ads). I think in general there may be a correlation between the number of ads on a site and its usefulness to me, which is inverse to its usefulness to google.
I’m curious what would happen if those products were split up into 2 separate companies.
NP :) It's exciting to have people using & commenting on something I build.
> I will look into fine tuning it for my needs. It is an interesting approach to a very annoying problem.
If a premium version were available with a customizable whitelist, would you pay for that? The API is around $5 / 1000 searches so it would cost about the same.
Good idea! Would you pay for a premium version where you can customize the whitelist (and additional features) if it were available? Bing charges around $5 per 1000 searches, so I guess it would cost about the same. (Google's API is limited to 10K searches / day, even when paid)
Yes! Maybe I'll build a premium customizable version out of it. Do you think that'd be useful? Bing charges around $5 per 1000 searches, so the premium version would be around the same (with caching covering hosting, maybe).
> If only I could get NTS to whitelist my domain name
There is a form [0] on the about page that allows people to suggest websites to add :)
> p.s. OP this is amazing! Would love an article explaining any backstory and details on how you made this (or setup / configured it).
Thanks! I think this is gonna be disappointing from an engineering perspective, and certainly not article worthy :) As further explained in my other comments, the website is basically a wrapper around google programmable search [1] where I whitelisted a set of sites I found useful personally, plus some suggestions from other users. It's really easy to set up.
As to why, I will quote some other comments of mine:
"I built this website a couple of months ago because I was annoyed by how hard it was to find useful things on Google."
"to find things more easily while programming or studying (I study biology, cs and ai; and philosophy in my free time, so expect the best results for queries related to those subjects). ... When I'm not doing those things, I just use Google or DDG because they have better results for day-to-day queries."
Here's a pastebin of the list I made yesterday [0] in a format that allows uploading to your own instance of programmable search. I have updated a few things since (like removing w3schools :)).
The site uses a whitelist of URLs to (attempt to) keep results relevant to science and programming. In the context in which I'm using this search engine, I have no interest in (reviews on) 3 hole punches. (That's not to say I never do, but in that case I'd use Google, Reddit, etc.) The fact that results don't show up here means that they also won't show up when I'm not looking for them, which is 100% of the time when I'm using this search engine. That's a plus for me personally.
Best case would be to have relevant results in a single search engine, but that's not what I intended when building this site.
Ads are added automatically by Google. The whole thing is little more than a wrapper around the 'Programmable Search Element Control API' which is an HTML element you can just insert into any site, like an iframe. Unfortunately this is the only way to make Programmable Search available at scale as the API is restricted to either 10 sites or 10K queries / day, even when paid!
There is a paid version for the HTML plugin, but that would leak the API key and so it wouldn't work as a business.
There is an option to get a share of the revenue generated by a search engine. Maybe it's time for me to figure out how that works.
I was thinking of making a hosted, ad free, customizable version where people upload their own keys. Not sure if people would like that.
As a side-note, it's super easy to remove ads with 1 line of CSS, but I wasn't sure how Google would feel about that so it's not in the online version. TamperMonkey is an extension that allows people to insert their own CSS on different websites. Hmm.
(obviously: this is subjective, so what's significant to me may not be to you.)
Honestly I just created this search engine for myself to find things more easily while programming or studying (I study biology, cs and ai; and philosophy in my free time, so expect results the best results for queries related to those subjects). I think those subjects also appeal to the HN audience, that's why I shared it here. When I'm not doing those things, I just use Google or DDG because they have better results for day-to-day queries.
That being said, I'm definitely interested in helping improve other people's search as well (reason I'm posting at all), so let me know if you have suggestions for sites to add!
As explained in my other comment, this website is a wrapper around google programmable search. The actual searching happens on Google servers, and I can see why people have problems with that. The code you see on the website is the same as the repo, though. It's actually hosted by GitHub! You can verify this by opening the web inspector in any browser or looking at the `.github.io` portion of the URL.