They made 11B$ last year. It has incredible amount of ads. If you haven't noticed, then that means they did a great job. (tip: look for the custom logo pins in the map. Its printing money)
Unless you start a war against scrapers, you don't need to worry about that as I'll always find a way to scrape your site as long as its valuable to 'me'. Even if it requires Real browser + OCR :)
Crul looks nice, though, you cannot imagine how many startups that I've seen failed doing a very similar thing as Crul. Wouldn't rely on it.
The problem is complex: Humans generating messy pages
These days, I'm not even using Go for scraping that much, as the webpage changes makes me crazy and JS code evaluation is a lifesaver, so I moved to Typescript+Playwright. (Crawlee framework is cool, while not strictly necessary).
Its been 8+ years since i started scraping. I even wrote a popular Go web scraping framework previously: (https://github.com/geziyor/geziyor).
My favorite stack as of 2023: TypeScript+Playwright+Crawlee(Optional)
If you're serious in scraping, you should learn javascript, thus, playwright should be good.
Note: There are niche cases where lower-level language would be required (C++, Go etc), but probably only <%5
But its too expensive to become practical with the OpenAI API. Also, demo is cool until you see the real-world webpages, then you'll realize that this only works less than %50 of webpages.
I don't understand why people still spend time on jailbreaks of the proprietary models, while they can easily use uncensored open-source models these days. I feel like its kind of waste of time.