Here's some extra implementation stuff I didn't include above. Most of my dev friends seem the most interested in this part.
The whole thing essentially runs on 1 VM with Playwright, Postgres and a handful of systemd timers. It's several nightly batch jobs (hourly to disperse a growing user-base across many hours, hopefully ;)).
The part that took forever to get right was the workflow for determining if a job was actually live on the company's ATS system. There is SO much garbage on the main job boards. The naive version checks the HTTP status, but a dead Workday or aggregator listing happily returns 200. So I'd get confident LLMs excited to link me to a "this position has expired" page... passing garbage is obviously never ok for this to work. That's why I had to go "real browser". It plays nice in the marketing but that gating is a necessity that the whole thing is kind of built around. It waits for the page to actually render, uses LLM intelligence to read the whole body and fails if it can't confirm the job is real. It's kind of awesome.
Here's some extra implementation stuff I didn't include above. Most of my dev friends seem the most interested in this part.
The whole thing essentially runs on 1 VM with Playwright, Postgres and a handful of systemd timers. It's several nightly batch jobs (hourly to disperse a growing user-base across many hours, hopefully ;)).
The part that took forever to get right was the workflow for determining if a job was actually live on the company's ATS system. There is SO much garbage on the main job boards. The naive version checks the HTTP status, but a dead Workday or aggregator listing happily returns 200. So I'd get confident LLMs excited to link me to a "this position has expired" page... passing garbage is obviously never ok for this to work. That's why I had to go "real browser". It plays nice in the marketing but that gating is a necessity that the whole thing is kind of built around. It waits for the page to actually render, uses LLM intelligence to read the whole body and fails if it can't confirm the job is real. It's kind of awesome.
I think this is only going to become more relevant. I'm personally a $200/mo Claude Maxer and I know that the usage I'm getting on Opus 4.8 Max and (until they yoked it out from under me) Fable 5 is way, way more than what I'm paying them. At some point, this will turn usage-based and I will be hammered on it and probably forced to look at self-hosting. I think while the caps are there, even at $200, it's honestly not too bad if you're coding value into the market, but as soon as those caps come off for retail AI users, we're all going to have some tough choices to make.
This is an interesting problem to take on. I think basically all teams that are adopting AI heavily right now are trying to solve this problem for themselves. I've been building something to fill this gap for my Technical Operations team. This gives me the advantage of handling sensitive information how I see fit in our documentation and creates a single source of all truth that cascades out to our other documentation.
So my question would be: what's the pitch for teams that have the engineering bandwidth to roll something custom out themselves? I think teams that are early adopters for heavy agentic AI use are also typically pretty security conscious. We do heavy sec reviews before we even consider another vendor in the space. I think, for us, that puts the barrier to adopting this over building and tweaking internally just too high.
And ultimately, the thing that would keep us from reaching for a third-party app for this gap is exactly the concern you flag at the end. I wouldn't want my company's full context (and it would have to be very full for this to be effective) sitting in someone else's single-point data layer. A breach could level our entire business.
Is self-hostable high up on the road map? Personally I think this idea lives and dies by it.
The whole thing essentially runs on 1 VM with Playwright, Postgres and a handful of systemd timers. It's several nightly batch jobs (hourly to disperse a growing user-base across many hours, hopefully ;)).
The part that took forever to get right was the workflow for determining if a job was actually live on the company's ATS system. There is SO much garbage on the main job boards. The naive version checks the HTTP status, but a dead Workday or aggregator listing happily returns 200. So I'd get confident LLMs excited to link me to a "this position has expired" page... passing garbage is obviously never ok for this to work. That's why I had to go "real browser". It plays nice in the marketing but that gating is a necessity that the whole thing is kind of built around. It waits for the page to actually render, uses LLM intelligence to read the whole body and fails if it can't confirm the job is real. It's kind of awesome.
Happy to go deeper.