HN 표시: 정규식 및 Stephen King을 사용하여 AI 슬롭 감지(github.com)
github.com
Show HN: Detecting AI slop with regex and Stephen King
https://github.com/guy-lifshitz/tacheles
4 comments
because a % based detector doesn't tell you what is wrong or why it fails, it is not actionable
forgot to mention - with mine you can tech your AI your rhythm and style of writing
If it's really slop shouldn't it be obvious, thus negating the need for a detector?
For about a year I've had the same argument with people - I can tell what LLM Claude or GPT wrote a piece of text. Not "is it AI", but which model. Nobody believed me, so I stopped arguing and wrote the checks down.
The part I find interesting isn't that AI text is detectable. It's that the two big model families give themselves away in separate ways.
Claude leaves rhythm fingerprints. The structure is the tell: the "it's not X, it's Y" reframe, a short bold punchline on its own line, em-dashes everywhere, triads (three items, three clauses, three sentences).
GPT leaves vocabulary fingerprints. The words are the tell: "let's dive in", "in today's fast-paced world", "whether you're a beginner or a pro", delve, tapestry, robust, leverage. And almost no em-dashes.
Run the same prompt through both and you can flag each with a separate rule set. Claude almost never trips the GPT vocab rules, and GPT almost never trips the Claude rhythm rules. That's the thing that made me build the tool instead of complaining about my feed.
What i've built is a deterministic linter. Basically a regex plus some statistics. Runs on your laptop, offline, no API key, nothing leaves the machine. I didn't want to ship my drafts to a service to find out they're mid.
I give no percentage, or other badge of a kind. The "87% AI" detectors are IMHO a dead end. You get a verdict you can't act on and that flags plenty of human text anyway. I didn't want a verdict. I wanted the actual words highlighted so I can decide what to cut. So yes, it's a linter, not a judge.
The English rules aren't mine. I turned my favorite Stephen King's "On Writing" into checks: kill the adverbs, prefer the active voice, the second draft is the first draft minus ten percent. Those are the opinionated bits.
More than one script. English is where the rules are real, but the engine also handles Cyrillic and a basic right-to-left proof of concept for Hebrew, so the tokenizer and the highlighting don't fall apart on RTL. Every language has its own slop.
Limitations, since someone will for sure raise them - these are rules, they are gameable by nature, anyone who reads them can write around them. The goal is to cut the bull, not winning a prize. The Claude/GPT split is empirical, not a law, models change and the fingerprints will drift, so the rules are plain text files on purpose. And it flags a lot of human writing too, because a lot of human writing is also slop.
And yes, the launch post for this tripped its own linter on every slop word I used to describe it. That's the point. It shows you, you decide.
Happy to answer anything. I'm curious whether the Claude-vs-GPT split holds up on other people's text, or whether I've overfit to my own texts.
The part I find interesting isn't that AI text is detectable. It's that the two big model families give themselves away in separate ways.
Claude leaves rhythm fingerprints. The structure is the tell: the "it's not X, it's Y" reframe, a short bold punchline on its own line, em-dashes everywhere, triads (three items, three clauses, three sentences).
GPT leaves vocabulary fingerprints. The words are the tell: "let's dive in", "in today's fast-paced world", "whether you're a beginner or a pro", delve, tapestry, robust, leverage. And almost no em-dashes.
Run the same prompt through both and you can flag each with a separate rule set. Claude almost never trips the GPT vocab rules, and GPT almost never trips the Claude rhythm rules. That's the thing that made me build the tool instead of complaining about my feed.
What i've built is a deterministic linter. Basically a regex plus some statistics. Runs on your laptop, offline, no API key, nothing leaves the machine. I didn't want to ship my drafts to a service to find out they're mid.
I give no percentage, or other badge of a kind. The "87% AI" detectors are IMHO a dead end. You get a verdict you can't act on and that flags plenty of human text anyway. I didn't want a verdict. I wanted the actual words highlighted so I can decide what to cut. So yes, it's a linter, not a judge.
The English rules aren't mine. I turned my favorite Stephen King's "On Writing" into checks: kill the adverbs, prefer the active voice, the second draft is the first draft minus ten percent. Those are the opinionated bits.
More than one script. English is where the rules are real, but the engine also handles Cyrillic and a basic right-to-left proof of concept for Hebrew, so the tokenizer and the highlighting don't fall apart on RTL. Every language has its own slop.
Limitations, since someone will for sure raise them - these are rules, they are gameable by nature, anyone who reads them can write around them. The goal is to cut the bull, not winning a prize. The Claude/GPT split is empirical, not a law, models change and the fingerprints will drift, so the rules are plain text files on purpose. And it flags a lot of human writing too, because a lot of human writing is also slop.
And yes, the launch post for this tripped its own linter on every slop word I used to describe it. That's the point. It shows you, you decide.
Happy to answer anything. I'm curious whether the Claude-vs-GPT split holds up on other people's text, or whether I've overfit to my own texts.