I originally took all domains crawled by CommonCrawl and made my own private tool for exploring the public web.
I've used it to get some sense/intuition for what's being made by others. It's helped me avoid proposing the 100th agentic marketing SaaS, when brainstorming with friends :)
So thought others might see the fun in exploring websites this way.
I've tried my best to circumvent the Cookie popups for the pictures, and have removed most parked domains, access denied, bad gateway etc.
I've also filtered out subdomains, as there's too much noise.
Lmk if there's anything else about it you'd like to know.
Only the frontend and two basic pg tables have been made specifically for this project. The rest have been part of other hobby projects, e.g. parsing CC indexes (and WET data), taking screenshots, saving and keeping it all updated etc.
0 comments
—
Show HN: hot or not for .ai websites · HackerTrans
So thought others might see the fun in exploring websites this way. I've tried my best to circumvent the Cookie popups for the pictures, and have removed most parked domains, access denied, bad gateway etc. I've also filtered out subdomains, as there's too much noise.
Lmk if there's anything else about it you'd like to know.
Only the frontend and two basic pg tables have been made specifically for this project. The rest have been part of other hobby projects, e.g. parsing CC indexes (and WET data), taking screenshots, saving and keeping it all updated etc.