HackerTrans
トップ新着トレンドコメント過去質問紹介求人

larsmosr

no profile record

投稿

Show HN: I built an SDK that scrambles HTML so scrapers get garbage

obscrd.dev
16 ポイント·投稿者 larsmosr·4 か月前·38 コメント

コメント

larsmosr
·4 か月前·議論
That’s so interesting! I know a few of the individuals and really am curious on how they interact/work together. Cool stuff
larsmosr
·4 か月前·議論
You project a lot in your life
larsmosr
·4 か月前·議論
Content protection and accessibility are in direct conflict with most approaches. Working on making the a11y layer something I handle properly rather than just hoping CSS ordering is enough.
larsmosr
·4 か月前·議論
The goal is raising the cost from "curl + cheerio" to "write a custom decoder per site." Most scrapers move on to easier targets.
larsmosr
·4 か月前·議論
The breadcrumb approach right now is simple invisible markers, not paraphrase-resistant watermarking. You're right that semantic watermarking that survives LLM rephrasing is the harder and more interesting problem. It's on the radar but not in scope for v1.
larsmosr
·4 か月前·議論
Full user-agent string: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.3;

Official OpenAI documentation: https://platform.openai.com/docs/gptbot
larsmosr
·4 か月前·議論
Thanks, what phone/browser? I'll fix that.
larsmosr
·4 か月前·議論
For content you want public, no.
larsmosr
·4 か月前·議論
The TalkBack issue is useful feedback, thank you. I tested with NVDA and VoiceOver but not TalkBack on Android. If light mode is reading individual words instead of a continuous block that's a real bug I want to fix.

On the broader point, I hear you, but I think there's a middle ground. Not all content is public knowledge. Some of it is premium, proprietary, or behind a paywall. The people publishing it should get to decide whether it becomes free training data.
larsmosr
·4 か月前·議論
For public knowledge sites this would be the wrong tool entirely. The use case is more like paywalled articles, proprietary product data, or premium content that companies paid to create and don't want scraped into a competitor's training set. obscrd is opt-in per component, not a whole-site lockdown.
larsmosr
·4 か月前·議論
The bar for reverse engineering dropped to "paste the HTML into Claude and ask it to decode." That's partly why the v2 roadmap moves toward techniques where the readable text never exists in the DOM at all. Static obfuscation patterns need to keep evolving or they become a one-prompt solve.
larsmosr
·4 か月前·議論
You are more than welcome to do so. Please keep in mind the realistic goal is raising the cost of scraping. Most bots use simple HTTP requests, and we make that useless.
larsmosr
·4 か月前·議論
Well the information is not to hide, quiet the opposite haha. There is a Demo page
larsmosr
·4 か月前·議論
You are not wrong. But the use case I keep seeing is companies with proprietary content they spent real money creating, who don't want it showing up in someone else's training data for free. It's less about bot hunting and more about content owners having a choice.
larsmosr
·4 か月前·議論
Font remapping is actually on the v2 roadmap. The reason v1 uses CSS ordering instead is it preserves screen reader access. Tradeoff is it's reversible (as another commenter just showed). Font remapping is stronger but breaks assistive tech. Solving both is the hard problem.
larsmosr
·4 か月前·議論
Yep, that works. The data-o attributes are readable in the DOM so you can reverse it with custom code. That's in the threat model. The goal is raising the cost from "curl + cheerio" to "write a custom decoder per site." Most scrapers move on to easier targets.
larsmosr
·4 か月前·議論
Copy-paste breaking is intentional for protected content but it's opt-in per component, not whole-site.

On the AI docs concern, fair point. To answer directly: I've confirmed the obfuscation defeats any scraper reading raw HTML via HTTP requests. Whether GPTBot or ClaudeBot use headless browsers internally, I honestly don't know. The README threat model lists headless browsers under "what it does NOT stop" for that reason.
larsmosr
·4 か月前·議論
Fair concern. obscrd actually preserves screen reader access. CSS flexbox order is a visual reordering property, so assistive tech follows the visual order and reads the text correctly. Contact components use sr-only spans with clean text and aria-hidden on the obfuscated layer. We target WCAG 2.2 AA compliance.

Happy to have a11y experts poke at it and point out gaps.
larsmosr
·4 か月前·議論
[dead]
larsmosr
·4 か月前·議論
I wanted to check it out but I get Application error on "Try for free"