Bad characters: imperceptible Natural Language Processing attacks [pdf](arxiv.org)
arxiv.org
Bad characters: imperceptible Natural Language Processing attacks [pdf]
https://arxiv.org/abs/2106.09898
6 comments
A computer vision model could solve the homoglyph issue for the parser. Even humans are subject to this attack vector, though.
I was kind of excited reading the headline, but this seems mostly to be a pretty straight forward application of existing homoglyph attacks that have been common for decades at this point (albeit maybe not common against these types of programs previously).
I thought it was going to be adding imperceptible changes that allow you to choose the interpretation (sort of like the visual adversarial examples) not just substitute a homoglyph and the binary classifier no longer works.
Don't get me wrong, this is still important work, i just got a bit too excited.
I thought it was going to be adding imperceptible changes that allow you to choose the interpretation (sort of like the visual adversarial examples) not just substitute a homoglyph and the binary classifier no longer works.
Don't get me wrong, this is still important work, i just got a bit too excited.
It seems like most of these imperceptible changes could be addressed by something like ascii folding (https://www.elastic.co/guide/en/elasticsearch/reference/curr...) but this might not apply for non-english use cases.
If you're interested in adversarial NLP, I also recommend reading this blog post on adversarial attacks on GPT2 with universal triggers (e.g. adding "nobody" as prefix for all inputs causes all entailments to be predicted as contradiction).
If you're interested in adversarial NLP, I also recommend reading this blog post on adversarial attacks on GPT2 with universal triggers (e.g. adding "nobody" as prefix for all inputs causes all entailments to be predicted as contradiction).
Which browser add-ons and bookmarklets exist that help me conveniently obfuscate messages so as to avoid automated surveillance?
I guess you could use one of the utils here to make your text weird: (the type of text that causes issues of the type in the article)
https://lingojam.com/
https://lingojam.com/
Models aren’t designed to understand Unicode. It’s the tokenizers job to chop that up and feed it to the model properly. What this person found is a step missing from the preprocessing pipelines of these models, or, more likely, something that should be part of the steps that come way before this data gets anywhere near an ML model. So, I think saying this is an adversarial attack on the machine learning model it’s self is a bit disingenuous, because for the input they were given, they did pretty well.
A similar example to this for a computer vision model would be something like inserting some kind of malformed payload in the headers of an image. The data is already bad long before it gets to the model.