Defenders have threat modeling on their side. With access to source code and design docs, configs, infra, actual requirements and ability to redesign / choose the architecture and dependencies for the job, etc - there's a lot that actually gives defending side an advantage.
I'm quite optimistic about AI ultimately making systems more secure and well protected, shifting the overall balance towards the defenders.
The real world use cases for LLM poisoning is to attack places where those models are used via API on the backend, for data classification and fuzzy logic tasks (like a security incident prioritization in a SOC environment). There are no thumbs down buttons in the API and usually there's the opposite – promise of not using the customer data for training purposes.
CrowdStrike does this trick where it replaces the file (being transferred over a network socket) with zeroes if it matches the malware signature. Assuming that these are the malware signature files themselves, a match wouldn't be surprising.
It's heartwarming to believe in the "family" narrative some companies promote, but it's important to remain pragmatic. This narrative is just a motivational tool to encourage employees to go above and beyond their compensated duties. It's not a binding contract though.
In reality, the power dynamic inherently favors the employer. Once an employee has invested their time and energy, the company holds all the leverage. There's little incentive for the company to uphold their end of this unspoken "deal."
Leadership changes, company priorities shift, and the "family" narrative can quickly fade when faced with financial realities or strategic decisions.
I might be misunderstanding, but from Andrew's Linkedin it looks like he wasn't a security researcher at MS, he was actually the person responsible for translating between security researchers and the upper management:
> Evangelize security services, practices, products, both internally and externally.
> Leading technical conversations around strategy, policy and processes with FINSEC and DoD/IC executive staff.
For the MS size entities, the risk calculation is way more complicated. The 1:1 between cost of mitigation vs cost of exploitation only applies to opportunistic attacks, really. At the level where APTs get involved, the data / access might be so valuable that they'd gladly outspend blue team's budget by a factor of 10-100.
Well, because as a security person I can only evaluate his actions from the point of security. Evaluating actions of MS business leadership is beyond my expertise.
I highly doubt that the senior leadership would willingly accept this kind of liability. But you need to put it into right terms for them to understand. Politics play important role at that level as well. There are ways of putting additional pressure on the c-suite, such as making sure certain keywords are used in writing, triggering input from legal or forcing stakeholders to formally sign off on a presented risk.
Without insight knowledge, it's impossible to figure out what went wrong here, so I'm not assigning blame to the whistleblower, just commenting that way too often techies fail to communicate risks effectively.
I work in infosec, and this sounds like a communication failure on the whistleblower's part.
Contrary to what many people believe, the profits should be prioritized over security for the most companies, that's only natural (after all, they don't generate any profits themselves, typically). The key is finding the right balance for this tradeoff.
Business leaders are the ones that are responsible for figuring out the acceptable risk level. They already deal with that every day, so it's nonsensical to claim they aren't capable of understanding risk. InfoSec's role for the most part is being a good translator, by identifying the technical issues (vulnerabilities, threats, missing best practices) that go beyond the acceptable risk profile and to present these findings to the business stakeholders, using the language they understand.
Either the guy wasn't convincing enough, or he failed to figure out the things business cares about & present the identified risk in these terms.
At this point you should just leave this dumpster fire of an organization and find a more reasonable place to work. I can't relate to the people who keep inventing atrocious workarounds ignoring the problem that they work in a hostile work environment.
I work in security and can't relate to banning Python & replacing it with Microsoft crap either.
I'm quite optimistic about AI ultimately making systems more secure and well protected, shifting the overall balance towards the defenders.