HackerTrans
TopNewTrendsCommentsPastAskShowJobs

larsmosr

no profile record

Submissions

Show HN: I built an SDK that scrambles HTML so scrapers get garbage

obscrd.dev
16 points·by larsmosr·4 miesiące temu·38 comments

comments

larsmosr
·4 miesiące temu·discuss
That’s so interesting! I know a few of the individuals and really am curious on how they interact/work together. Cool stuff
larsmosr
·4 miesiące temu·discuss
You project a lot in your life
larsmosr
·4 miesiące temu·discuss
Content protection and accessibility are in direct conflict with most approaches. Working on making the a11y layer something I handle properly rather than just hoping CSS ordering is enough.
larsmosr
·4 miesiące temu·discuss
The goal is raising the cost from "curl + cheerio" to "write a custom decoder per site." Most scrapers move on to easier targets.
larsmosr
·4 miesiące temu·discuss
The breadcrumb approach right now is simple invisible markers, not paraphrase-resistant watermarking. You're right that semantic watermarking that survives LLM rephrasing is the harder and more interesting problem. It's on the radar but not in scope for v1.
larsmosr
·4 miesiące temu·discuss
Full user-agent string: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.3;

Official OpenAI documentation: https://platform.openai.com/docs/gptbot
larsmosr
·4 miesiące temu·discuss
Thanks, what phone/browser? I'll fix that.
larsmosr
·4 miesiące temu·discuss
For content you want public, no.
larsmosr
·4 miesiące temu·discuss
The TalkBack issue is useful feedback, thank you. I tested with NVDA and VoiceOver but not TalkBack on Android. If light mode is reading individual words instead of a continuous block that's a real bug I want to fix.

On the broader point, I hear you, but I think there's a middle ground. Not all content is public knowledge. Some of it is premium, proprietary, or behind a paywall. The people publishing it should get to decide whether it becomes free training data.
larsmosr
·4 miesiące temu·discuss
For public knowledge sites this would be the wrong tool entirely. The use case is more like paywalled articles, proprietary product data, or premium content that companies paid to create and don't want scraped into a competitor's training set. obscrd is opt-in per component, not a whole-site lockdown.
larsmosr
·4 miesiące temu·discuss
The bar for reverse engineering dropped to "paste the HTML into Claude and ask it to decode." That's partly why the v2 roadmap moves toward techniques where the readable text never exists in the DOM at all. Static obfuscation patterns need to keep evolving or they become a one-prompt solve.
larsmosr
·4 miesiące temu·discuss
You are more than welcome to do so. Please keep in mind the realistic goal is raising the cost of scraping. Most bots use simple HTTP requests, and we make that useless.
larsmosr
·4 miesiące temu·discuss
Well the information is not to hide, quiet the opposite haha. There is a Demo page
larsmosr
·4 miesiące temu·discuss
You are not wrong. But the use case I keep seeing is companies with proprietary content they spent real money creating, who don't want it showing up in someone else's training data for free. It's less about bot hunting and more about content owners having a choice.
larsmosr
·4 miesiące temu·discuss
Font remapping is actually on the v2 roadmap. The reason v1 uses CSS ordering instead is it preserves screen reader access. Tradeoff is it's reversible (as another commenter just showed). Font remapping is stronger but breaks assistive tech. Solving both is the hard problem.
larsmosr
·4 miesiące temu·discuss
Yep, that works. The data-o attributes are readable in the DOM so you can reverse it with custom code. That's in the threat model. The goal is raising the cost from "curl + cheerio" to "write a custom decoder per site." Most scrapers move on to easier targets.
larsmosr
·4 miesiące temu·discuss
Copy-paste breaking is intentional for protected content but it's opt-in per component, not whole-site.

On the AI docs concern, fair point. To answer directly: I've confirmed the obfuscation defeats any scraper reading raw HTML via HTTP requests. Whether GPTBot or ClaudeBot use headless browsers internally, I honestly don't know. The README threat model lists headless browsers under "what it does NOT stop" for that reason.
larsmosr
·4 miesiące temu·discuss
Fair concern. obscrd actually preserves screen reader access. CSS flexbox order is a visual reordering property, so assistive tech follows the visual order and reads the text correctly. Contact components use sr-only spans with clean text and aria-hidden on the obfuscated layer. We target WCAG 2.2 AA compliance.

Happy to have a11y experts poke at it and point out gaps.
larsmosr
·4 miesiące temu·discuss
[dead]
larsmosr
·4 miesiące temu·discuss
I wanted to check it out but I get Application error on "Try for free"