Defenders have threat modeling on their side. With access to source code and design docs, configs, infra, actual requirements and ability to redesign / choose the architecture and dependencies for the job, etc - there's a lot that actually gives defending side an advantage.
I'm quite optimistic about AI ultimately making systems more secure and well protected, shifting the overall balance towards the defenders.
The real world use cases for LLM poisoning is to attack places where those models are used via API on the backend, for data classification and fuzzy logic tasks (like a security incident prioritization in a SOC environment). There are no thumbs down buttons in the API and usually there's the opposite – promise of not using the customer data for training purposes.
CrowdStrike does this trick where it replaces the file (being transferred over a network socket) with zeroes if it matches the malware signature. Assuming that these are the malware signature files themselves, a match wouldn't be surprising.
It's heartwarming to believe in the "family" narrative some companies promote, but it's important to remain pragmatic. This narrative is just a motivational tool to encourage employees to go above and beyond their compensated duties. It's not a binding contract though.
In reality, the power dynamic inherently favors the employer. Once an employee has invested their time and energy, the company holds all the leverage. There's little incentive for the company to uphold their end of this unspoken "deal."
Leadership changes, company priorities shift, and the "family" narrative can quickly fade when faced with financial realities or strategic decisions.
I might be misunderstanding, but from Andrew's Linkedin it looks like he wasn't a security researcher at MS, he was actually the person responsible for translating between security researchers and the upper management:
> Evangelize security services, practices, products, both internally and externally.
> Leading technical conversations around strategy, policy and processes with FINSEC and DoD/IC executive staff.
For the MS size entities, the risk calculation is way more complicated. The 1:1 between cost of mitigation vs cost of exploitation only applies to opportunistic attacks, really. At the level where APTs get involved, the data / access might be so valuable that they'd gladly outspend blue team's budget by a factor of 10-100.
Well, because as a security person I can only evaluate his actions from the point of security. Evaluating actions of MS business leadership is beyond my expertise.
I highly doubt that the senior leadership would willingly accept this kind of liability. But you need to put it into right terms for them to understand. Politics play important role at that level as well. There are ways of putting additional pressure on the c-suite, such as making sure certain keywords are used in writing, triggering input from legal or forcing stakeholders to formally sign off on a presented risk.
Without insight knowledge, it's impossible to figure out what went wrong here, so I'm not assigning blame to the whistleblower, just commenting that way too often techies fail to communicate risks effectively.
I work in infosec, and this sounds like a communication failure on the whistleblower's part.
Contrary to what many people believe, the profits should be prioritized over security for the most companies, that's only natural (after all, they don't generate any profits themselves, typically). The key is finding the right balance for this tradeoff.
Business leaders are the ones that are responsible for figuring out the acceptable risk level. They already deal with that every day, so it's nonsensical to claim they aren't capable of understanding risk. InfoSec's role for the most part is being a good translator, by identifying the technical issues (vulnerabilities, threats, missing best practices) that go beyond the acceptable risk profile and to present these findings to the business stakeholders, using the language they understand.
Either the guy wasn't convincing enough, or he failed to figure out the things business cares about & present the identified risk in these terms.
At this point you should just leave this dumpster fire of an organization and find a more reasonable place to work. I can't relate to the people who keep inventing atrocious workarounds ignoring the problem that they work in a hostile work environment.
I work in security and can't relate to banning Python & replacing it with Microsoft crap either.
No, there is no confusion here at all (for a Python developer). I would consider it a code smell though as the whole problem is completely avoidable by better naming.
In my experience Bing chat and phind are useless. But perplexity.ai and GPT4 are amazing. GPT-3.5 and Cloude-instant (available through poe.com) are cool as well, even though they got significantly dumbed down recently, presumably to lower the maintenance costs.
EDIT: Also, this doesn't seem convincing: "I am not as advanced as PaLM 1, but I am learning new things every day. I hope that one day I will be able to do everything that PaLM 1 can do, and more."
ChatGPT was the same last year, but since ClosedAI added some kind of magic (fine-tuning or just embeddings auto-injection) so that models can somewhat describe themselves.
Not even close. I'd prefer local WizardLM/Vicuna to this Artificial Idiot.
The funniest to me is that one of the examples they suggest you try is asking questions about google/jax repo, which kinda suggest that Bart can index source code from GitHub. Well, it fails even at listing directory structure and completely hallucinates all following answers!
I'm not sure why Google would bother releasing Bart in this state. It's made me lose respect for their AI competency.
I'm quite optimistic about AI ultimately making systems more secure and well protected, shifting the overall balance towards the defenders.