At my company someone has introduced an internal tool that should help understand and give a "score" to design documents from teams.
Needless to say, this tool gives scores exactly like the article mentions. Same document, same LLM, same prompt, and different results. It becomes even more ridiculous once you switch to other models, or if you ask a model to review the work of another model.
I am not sure why we insist on making LLMs do the work they are not supposed to do and/or in a way they are not supposed to do.
The worst part is that people are aware of the problem but they just ignore it and consider it as "a reference number, just to have an understanding".
If it were like that, it would be less of a problem. The issue comes from the fact that eventually someone without enough knowledge will trust the output (so X points out of Y is how it is), or someone will stop challenging the output and consider it for their process - like in this unfortunate case of hiring.
At a certain point, people who don't know what they are doing give a tool that doesn't know what its doingto people who don't know what they are doing. A pure mess. And everyone has to comply and applaud. If you go against, you are against AI.
This is what I hate the most about AI. Not the tool, but the shortcuts we're willing to take to justify its existence.
If the code is unreachable is obviously not a threat, Mythos or not. Can you do this analysis for all your 200+ services, libs etc?
This is the main issue about compliance nowadays. In a fedramp scenario you would very likely have to prove that it's unreachable, and you might even risk compliance over it.
From an attacker perspective they don't know the lib version you're using, but bruteforcing / finding patterns faster than a hacker can? That's what I believe AI can do. This is why for me CVEs are a useless metric, the number and/or criticality. It's a simple security control but they are giving it so much importance. Then you "forget" to secure access to your mcp server and this leaks company info, but hey, zero CVEs, soc2 compliance check check check.
I think it's a good practice to fix as many CVEs as possible, to have a clean/updated codebase, but I am of the opinion that if someone wants, with the tools they have nowadays, they will find a way in. Of course, using a lib that has obvious security issues for input validation, for example, should be a no-go. However, we're reaching a point of ridicule (like you said above, a critical CVE but unreachable).
I am not against AI but putting automation tools is in a different category than "AGI, you only need specs and they do the work for you".
20 years ago even with automation etc you needed armies of people to make something work. It was more of a transformation of the type of work. Look at the amount of work Amazon as a company has created. It changed how people buy stuff etc, but behind the scenes there is always human workforce, to deliver, to invent the recommendation algorithms, to package the items, etc (although here there is heavy automation).
Now the idea is that we need AI so we can replace humans, so "people can spend more time on what they like to do". Which is what, searching for jobs on LinkedIn?
Not as a metric, but it basically becomes one, like with Fedramp.
You need to fix also moderate/low CVEs within a certain time frame.
So CVE count becomes relevant, because the target is zero, although it doesn't mandate "zero CVEs" but that's finally what the desired outcome is.
It's basically unrealistic to ignore that number, because it's unlikely that you have a steady 1000 CVEs (that are being continuously fixed and new ones discovered), but more like "a few exceptions".
I'd like to know how a "critical CVE running in your software for 29 days" is acceptable from a security standpoint. With nowadays tooling, these AI agents can take you down in no time if they target you.
Compliance the way is done today is basically outdated, but everyone has to follow these rules to sell software basically.
This is true, but security teams often work on tooling dedicated to reduce the n. of CVEs so that a company can keep compliance. That is in fact part of compliance itself to have an automated/reliable processo to tackle CVEs...
In my experience it is becoming basically ridiculous that we disallow compliance based on a number of cve, their level, etc. It's just a checkbox, but it has nothing to do with security.
Was it a better prompt? Have you tried giving the same prompt to other models?
I have found out that the mistakes of other models (which I choose first to save money) help me refine the prompt more and more, until I am fed up and pick Opus 4.8 (for example) which magically seems to get it right, but there is a lot of pre-work there...
Can you give an example of what those "toughest problems/great code" are? I don't need to know the prompt nor the output, but the general idea, what it is about.
However, I think actually that while it won't give the results expected (AI agents run the company, build all features, etc.), it will nevertheless become a developer tool like IDEs, something "you have to have".
It's here to stay but probably with more realistic expectations than some CEO/CTO are pushing for (agents for everything, nobody writes 1 LOC, self healing systems, etc).
So the market expectations will be probably resized, but these tools are here to stay. Be it for cybersecurity (from CVEs to cyber warfare) alone, that's already worth all the money they are throwing a it.
>I've never worked at any company where there was any limit to the work to be done. Sales people don't give a shit what your product can do, only what they can sell, and they never sleep.
The issue is how much of that work is "valuable" in the sense = makes money.
I have both been in projects and seen projects which were canceled once it turned out they didn't make money (bad sales? bad product? bad market fit? a bit of everything?). This you can only afford when you have money to spare (= with debts? high profits...?).
With the interest rates so high, how can a company justify hiring dozens/hundreds of people more? It's a risk, and what I am seeing now is that companies are shrinking left and right to focus on the business that makes money and reduce headcount on what they believe doesn't make money at all, or it's a cost too high for their "long term strategy" or whatever. Right now the only metrics that they are caring about is EBIDTA. They don't even care anymore about ARR, they are becoming irrelevant as long as they stay within a range (we want 20% increase, but we're ok with 5%).
The AI will replace everything and everyone is working out pretty well for Anthropic/OpenAI, though.
I gave up on "people should read things". Especially with AI now telling you how to think and what your final design should look like, I would be at least happy if they did some test of their own design and criticize it constructively.
Just put a "designer" in one of these cars and let them drive in real life situations like:
- a wasp entering your car, while you're approaching the entrance to the highway
- a child suddenly appears on the street from behind an SUV so big you could barely see the sidewalk
- a traffic light, green for you, but red for the car coming straight for your door.
We're past the "happy path". Try real life shit in your tests and maybe we'll install less screens and more sensors to actually help you drive, instead of distracting you.
Saving someone's life should be more important than a dumb undeserved promotion because you digitalized the whole car.
They just complain about the algorithms but they use also the same tool for propaganda / marketing. The only thing they literally agree on is "online hatred" because sometimes it goes against them, so they need to keep the system running.
For example, the previous German government was paying influencers for sponsoring heat pumps. All these "content creators" must be paid by someone - left, right, center, oil, nuclear, gas companies, it's like watching TV for its advertisements. Crazy what it has become.
So, that will most likely never change, although that's probably in the top 3 reasons why social media is unusable.
>Is now the moment that the world's digital infrastructure succumbs to waves of hackers using countless exploits; I doubt it.
I am not into cybersecurity but the existing "technical debt" in terms of security has been barely exploited.
The issue is that literally all software has some vulnerability, want it or not. And these LLMs are like brute forcing all possibilities faster than a human can do. Sometimes humans even ignore low security issues, while maybe these LLMs are capable to build exploits on top of multiple ones.
For me they understood the moat - cybersecurity is such a trivial space to get into, I guess they are investing heavily on that because as someone else mentioned in other threads, it's obvious they are too limited for other tasks.
Becoming a "mandatory" (SOC-2 etc, things like that) integrated part of your CI/CD pipeline would be a huge win for them. Imagine that.
There are people hosting agents online to talk to other agents etc. on their behalf. How difficult is it to just instruct such an agent to do the tasks you mentioned? You're assuming it's done by "bad actors" while it's most likely just going to be done by "everyone" that knows how to do it.
They are doing crazy things to not do the one single thing that had to be done years ago - make Facebook, Instagram and Co. pay hard for the damage they brought on our kids and society.
90% of the crap our kids are exposed to comes from there. Not sure what's left to tackle, once you remove these websites from the picture - videogames? News??
Needless to say, this tool gives scores exactly like the article mentions. Same document, same LLM, same prompt, and different results. It becomes even more ridiculous once you switch to other models, or if you ask a model to review the work of another model.
I am not sure why we insist on making LLMs do the work they are not supposed to do and/or in a way they are not supposed to do.
The worst part is that people are aware of the problem but they just ignore it and consider it as "a reference number, just to have an understanding".
If it were like that, it would be less of a problem. The issue comes from the fact that eventually someone without enough knowledge will trust the output (so X points out of Y is how it is), or someone will stop challenging the output and consider it for their process - like in this unfortunate case of hiring.
At a certain point, people who don't know what they are doing give a tool that doesn't know what its doingto people who don't know what they are doing. A pure mess. And everyone has to comply and applaud. If you go against, you are against AI.
This is what I hate the most about AI. Not the tool, but the shortcuts we're willing to take to justify its existence.