HackerTrans
TopNewTrendsCommentsPastAskShowJobs

xlr8_track

6 karmajoined vor 18 Tagen

Submissions

Show HN: Focal Harvest – CLI research pipeline to search, scrape and notify

github.com
9 points·by xlr8_track·vor 6 Tagen·2 comments

comments

xlr8_track
·vor 5 Tagen·discuss
Thanks for the star and the question. Most tools that do search and summarisation either need custom selectors per site or pull in a full 200MB headless browser just to handle JS rendering. Focal Harvest does not. It runs on a cheap VPS, has no database, and works without any API key (If you want, you can provide an API key though).

The whole pipeline is plain Python with no browser dependency. JS-heavy pages are handled by extracting `__NEXT_DATA__`/`__NUXT_DATA__` JSON directly from the raw html, and wayback machine covers the rest. Everything saves as markdown and JSON files, which you can open by double-clicking. I've updated the repo now so that you can see the logs too.
xlr8_track
·vor 6 Tagen·discuss
[flagged]
xlr8_track
·vor 16 Tagen·discuss
Yes, many people like myself are willing to contribute. It'll take the load off you and give you time to work on other features or projects.
xlr8_track
·vor 16 Tagen·discuss
Awesome, how do I contribute to this? A gihub link or smthg?
xlr8_track
·vor 16 Tagen·discuss
Amazing Idea, now I don't have to worry about screenshots being up to date with the repo.
xlr8_track
·vor 18 Tagen·discuss
Tried it and absolutely love this. Is there any way I can contribute to this? Like a GitHub link or something?