That's really interesting to know, thanks. I think for my purposes a lot of high-quality information isn't paid - it's academic, policy, and nonprofit sources mainly. I feel like you could make a lot of improvement for my purposes just by:
- Downweighting commercial sources and upweighting academic, government, and nonprofit sources.
- Using some measure of quality, maybe even something simple like length + reading age?
- Building big whitelists of quality sites and blacklists of low-quality sites, as picked by human curators and users
Beyond this, perhaps users could plug in sources that are particularly useful for them, including ones they have subscriptions to. I've thought it would be handy to have search results include ebooks, papers, and notes on my computer, for example.
But maybe the userbase of people who need high quality, non-subscription info is too small to get this started.
I'm a consultant/researcher advising charitable foundations. It means I need to research all sorts of different topics depending on what area the foundation is interested in. e.g. one project might be on humanitarian aid, another on educational technology, another on infectious disease. So I don't need a search engine in a particular niche, I need a search engine that is good at finding high quality information in multiple niches. By high quality information I mean a mixture of academic articles, expert blog posts, podcasts with experts, policy reports, books, expert tweets, quality journalism etc.
I think there are other jobs that probably face this kind of problem, for example: policymakers, management consultants, journalists, nonfiction writers, some kinds of investors. They all need to rapidly learn things in topics outside of their expertise.
I haven't found other search engines much better, although I haven't yet done a systematic test. One thing that puts me off them is it seems from their marketing copy that most alternative search engines like Duck Duck Go are focusing entirely on being a privacy-preserving alternative. I want one that is focused on quality (and customisability), not just privacy.
I tried to install Searx so that I could at least get results from a variety of engines. But I got stuck in some dependency yakshaving so gave up.
I and/or the company I work for would. Google is awful for finding the kind of high quality, niche information I need in my job. It just returns lowest common denominator rubbish.
My route in was pretty idiosyncratic, and I think that's true of many people in these kinds of jobs. A more standard way to transition into this kind of research/consulting/policy work from tech might be go for jobs in technology or innovation policy. If you're in the UK I can point out some orgs that do that kind of work but in any country there will be various think tanks, policy consultancies, etc.
I worked as a web developer for a couple of years after teaching myself ruby and javascript. I got into programming because I found it intellectually interesting, but unfortunately commercial programming is mostly mindnumbing. I got bored of creating CRUD apps over and over again for boring business applications. I considered upping my CS and maths skills to get more interesting jobs, but decided to pursue more of a nonprofit strategy/research path instead.
I now work for a small consultancy company doing research and designing funding programmes for charitable foundations. It's great because I get to do lots of research, writing, and thinking and have a positive impact on the world. No regrets, I'm very glad to have made the transition.
Still follow HN out of mild addiction and because there are interesting articles.
- Downweighting commercial sources and upweighting academic, government, and nonprofit sources.
- Using some measure of quality, maybe even something simple like length + reading age?
- Building big whitelists of quality sites and blacklists of low-quality sites, as picked by human curators and users
Beyond this, perhaps users could plug in sources that are particularly useful for them, including ones they have subscriptions to. I've thought it would be handy to have search results include ebooks, papers, and notes on my computer, for example.
But maybe the userbase of people who need high quality, non-subscription info is too small to get this started.