That entirely depends upon the cost of preventing them. At no cost, sure.
But nothing is ever free. So what is the cost.
<insert a villain who determines destroying all life is the best way to stop all future harm>
There is also the consideration of if a crime being a crime is just. Consider crimes that use to be on the book that we now consider horrible things to have outlawed or even crimes that are still on the books but not enforced because we reject them.
For example, some places us to make it a crime for kids to play pinball. Is preventing kids from playing pinball, even if it came at no cost, an unquestionably good thing?
For a sufficiently bad crime, at sufficiently low costs, preventing it is good. But those two factors are both very big questions, directly challenging the notion of "unquestionably good".
>Many developers are too focused on delivery value compared to maintenance cost.
The way this is worded feels like it leaves the blame on developers. Aren't these developers focused on exactly what they are being judged by? Shouldn't we say it is the management who is too focused on delivery value compared to maintenance cost? Is it the developer's job to guide management or the manager's job to request guidance, assuming such guidance is needed?
We need to make sure the responsibility to resolve this problem falls on those with the power to act on it, and in this, developers tend to be receiving far more responsibility to fix than power to fix.
Are court decisions universally recognized? If a court rules someone true in one area, couldn't it still be considered slander/libel in another area if another court rules it as not being true?
It is a bit like saying convicted murderer vs murderer. One is claiming the conviction for murder happened (which generally correlates to the murder happening but doesn't openly claim such) while the other directly claims the murder happens. If the conviction is later overturned, does the second claim open one up to more libability for a false claim?
I think it might be built on something more than deterministic systems. Some property that is a subset of deterministic, so all your argument still apply, but merely being deterministic is not good enough.
LLMs are what made me start considering this. Imagine a company using an LLM that was fully deterministic. All RNG was either removed or seeded in such a way that the same input (so many the seed counts as part of the input) gave the exact same output. Fully deterministic.
But such an LLM, with a slight drift in input, could still produce very different outputs. This isn't being non-deterministic, but more than the change in outputs does not naturally follow from the input. I'm thinking like how 2 double pendulums can (but not always do) greatly diverge given a very small change in their input.
So in light of that I've begun to call this new property non-chaotic. So Enterprise depends on non-chaotic systems, which are a subset of deterministic systems, and then wrangling the chaotic elements they cannot remove as much as possible.
The follow question I now have is if all LLMs are inherently chaotic, or if it is possible to have a non-chaotic LLM.
>The parents job is to keep their kids out of trouble.
I challenge that anyone believes this, and for my evidence, I would submit all the age based laws that protect children regardless of what parents do.
We have already, long ago, decided that it is the government's job to protect children, at least in cases where parents fail to do an adequate job. That's why I don't see this ending any other way. The march to total domination by the side of the government might be slow, but they already won the war around a century ago (exact timeline for laws protecting children in place of parents is a very long topic and does differ country to country, I recall hearing some places still even let kids buy alcohol if they say it is for their parents to consume).
Never really like this because it feels some arbitrary as to what counts as making a claim. Human language is filled with implications, and those implications are filled with implied claims. Yet burden or proof seems mostly used when someone makes one of those implied claims explicit, say by explicitly rejecting one.
"We should do X."
"X is not possible."
There was an implied claim that X is possible and an explicit claim that X is not possible, but often it feels that in such a scenario, it is the second individual who is deemed having a burden of proof.
This is made all the worse that there is rarely a single implied claim, but more a group of related implied claims that are also weighted. Like some combination of "data centers don't destroy the environment" and "or maybe they do, but only in amounts of destruction that are tolerable compared to human behaviors and worth the outputs gained" and even "they modify the environment, but it doesn't count as destruction, just change". And even worse (yet again), different readers of the conversation will find themselves placing different weights on these different interpretations making it near impossible to agree on what the original claim is. Thus the first person who makes an explicit claim gets all the attention which ignores all the messy implied human communications preceding it.
But why does it do so inconsistently, and sometimes even forgetting to swap back to English when it comes time to do 'normal' output? It also seems recent, as when I was using deepseek even a week ago this was very rare compared to what I was seeing yesterday. I had to start including a line asking it to stay to English because I can only speak/read English.
How so? Everyone would still have their skills to provide goods and services and everyone would still have wants for other's goods and services, so an economy would still run. AI can shift the economy but it doesn't lock the entire population out of the economy. It can lock out any one group because everyone else gets the good/services of that group for cheaper from the AI, but if everyone else can't afford the AI, if the AI locks everyone out, then they trade between themselves instead. And that is the sort of 'worst case possible' outcome, not even what is likely to happen as the AI makes some things much cheaper.
The argument is more "I want to do good thing X, but it will also cause bad thing Y." followed by "Wait, bad thing Y is going to happen anyways, so I might as well do good thing X so we get both X and Y instead of just X."
Viewed this way, the idea is that given the world will have bad thing Y regardless, the one impact of your choice is if good thing X exists or not, and it is better to create good thing X.
Where it becomes an issue is that there is no clear X or Y. There are many different but very related bad things, so if the one you would add is actually better or worse than what is already out there, or maybe it'll exist both ways but you make it more popular, and very subjective things to judge, so different people look at the same outcome and some agree that bad thing Y would have existed anyways and others say that no, this is a new bad thing Z that wouldn't have existed anyways.
>From where I sit it seems reasonable for Anthropic to not want their product used to create malware
Yes, I think there is a PR component to this that is often left out of this discussions.
Making a claim of 3 star service, without providing 3 star service evidence, isn't any different than a claim like they don't clean the dishes enough. Sure, you didn't write out the exact words, but by submitting the review, you still submitted such a claim, and so it remains a question of if you provided evidence to prove it, which gets into how deep the legal system wants the evidence verified (is a picture enough, or do you need to provide evidence the picture is of the store at the date and time relevant to your review).
One of the major concerns with prototyping, at least in my experience and based on the general vibe I've felt from others over the years, is that clients will generally do some variation of "You have a working prototype? So how many days until it is prod ready? No more than a few weeks, right?" and that will be their expectation. AI makes prototyping easier, but makes this specific drawback much worse. Expectations are going to be misaligned, leading to many disappointments, and likely some number of developers taking undue negative outcomes. I'm not even sure if it will normalize to reasonable expectations, given that this tension never seemed resolved even before AI was in the mix.
Does the model need to offer new testable hypothesis if it provides a way of explaining existing results that current models can't?
If it is competing against another model that does both that and offers new testable hypothesis (which experiments match), the other model is the clear winner. But lacking that, if no other model explains all existing data, is new testability really necessary when it is the only model that currently explains all existing tests?
That said, aren't most of theoretical models only contenders for such, as in they haven't been expanded to actually explain all testing results, only that, as far as they have been expanded, there are no contradictions yet? So they need physicists to expand them, but if the model is wrong, the effort might largely be wasted, and we have some models that there is disdain for not because they contradict existing experiments, but because they have eaten too many careers without showing value in return?
Didn't the original authors end up leaking this before OpenAI fixed it? They gave them a chance, but then had to decide between staying fully silent or publishing the details despite malicious actors learning about it before it was fixed or leaving users in the dark. They chose it was better to warn users and inform malicious actors despite it not being fixed.
>This vulnerability was responsibly disclosed to OpenAI. Despite multiple follow-ups, we received no communication beyond an automated reply to our initial disclosure. OpenAI's documentation fails to describe sensitive capabilities granted to the model (e.g., running privileged scripts) or risks of model manipulation via indirect prompt injection, instead focusing solely on functional limitations and data-handling concerns. As such, we are publishing our findings to enable informed decision-making regarding the risk surface.
That very last sentence was considered justification of putting this knowledge into the wild when OpenAI refused to fix it. So, if we consider it justified with a delay, then we are saying it is acceptable (it is "responsible") to give the information to malicious actors as long as you tried to warn the right party first.
Compare that to two alternatives. Alternative 1 is never disclosing it to the public until fixed. Saying it is never acceptable to let malicious actors know until it is no longer a concern, even though this will mean users are kept in the dark about the risk.
Alternative 2 is to reduce that timeline to 0. Say that users are immediately warned, despite the risks of making it known to bad actors.
So if we are saying the current delay is acceptable, but both a longer and a shorter delay are unacceptable, then why is that? What justifies the current delay, what makes that the responsible one, rather than a shorter or longer window?
>I can't imagine something being "more responsible" for the public than privately notifying the owning party to give them time to fix the issue, before notifying the rest of the world (including malicious actors) about it.
What about ensure they have fixed it, and only considering it responsible to disclose it when fixed (alternative 1)? If it is never fixed, then the bug is never disclosed, because it is not acceptable to tell malicious actors how to exploit a vulnerability? Even evidence of use wouldn't be justified, as publishing this makes all malicious actors aware of it rather than just a subset of them.
And if you disagree and think some window is reasonable, then apply that argument to a slightly shorter window and repeat until either the argument hits some built in limit or reaches a window of 0.
Isn't this a double plus good phrase? What makes this more responsible? Reasoning about first order effects of different disclosure models? But what if someone uses higher order reasoning and critical thinking to reach a conclusion that other disclosure models are better for the average user and the long term health of the industry, even if they are worse in any individual case. A difference in the security culture incentivized by different disclosure patterns. Why does this one win the name of responsible while other alternatives, which have never been proven to be worse, are automatically marked as irresponsible?
Reminds me a bit of the concept of identity theft, as a way to say that even though the bank (or other creditor) was the one who had money taken from them, it is actually the random person not involved in the transaction who is the victim and has to hold the debt until the issue is resolved.
Nothing of the level of rocket failure, but I've tracked down issues where you are never sure of the cause. You keep the doubt and let it drive you. You aren't as much sure of a theory, as you have the theory you most want to disprove and keep failing to do so. The more you fail to disprove a given theory while other people with their own personal 'targets' do end up disproving them, the more you can report that the theory is the reasonable conclusion. But you never given up the idea of looking to disprove it. Eventually others join you and work to disprove your theory. As the group continues to fail to disprove it, it becomes the officially stated cause unless someone can provide evidence otherwise.
Sometimes I'll have one that I'm stuck on for a month before finally disproving it, and it is an interesting feeling. There is some level of happiness I succeeded at my goal, but it is very bittersweet because it normally was my last working theory and now I'm simply lost until I can formulate a new one. Sometimes disappointment in myself that I might've missed some easy way to disprove it for so long, but other times the way to disprove it was sufficiently hard enough that I just accept it is what it is.
Consider indie games. If there are 10 of them and 5 are great, you don't need any filter. You look through 5 great and 5 not so great games and end up with 5 great ones.
Now go to a world where indie games explode. But only 1 in every 100 are great. There are now 100,000 games, but most qualify as very low quality. There are now 1,000 great games (and a few of these might be the perfect game you dreamt of), but if you don't have a filter and are buried under 10s of thousands of horrible games, things feel worse.
With a filter, you now live in a world where you can easily find most of those great games with only a few lower quality ones showing up. So as long as the filters that exist, whatever they might be, can handle it, more is better even if quality drops.
Unless the quality extremely fast, say my previous example of 100,000 games but only 1 in a million was a great game. I think this level of quality drop is extremely unlikely. Instead, I suspect the real problem is if the filters can keep up, because they depend upon human effort, so it is possible to hit a point where they are overwhelmed and stop functioning properly. That's when things get worse. As long as the filters hold, more building leads to better outcomes even with a drop in quality.
One other question I had but wasn't sure if it would leave my previous post too unfocused is "aren't we a bit too early to determine in our current government systems are really the most effective?" This is something that will be decided by political scientists far removed from the current societies who can see how our current societies evolve.
Isn't this still assuming we can even determine what is true or false?
Newtonian physics is false, but it works well enough we teach it in college. But our best models of physics are currently in disagreement, so can we even say they are true? Given the replication crisis, especially in social sciences, how many of peer reviewed findings can be called true? Even experimental results can be false (consider studies that found FTL neutrinos, which were rejected as an error in the experiment, and which was eventually confirmed but it took quite a lot of work and in a softer field than physics with a claim less absurd than FTL, would have likely long been accepted as a true finding).
Even in math, basic statements aren't really true or false, but more a question of "given these axioms, can we prove or disprove it" noting that we have different systems with different axioms. If we are talking basic sets, most people are using naive set theory which is inherently contradictory, which means that notions like true or false probably can't be considered well defined.
If it was so clearly ineffective, why does it get challenged more often and replaced? Existing corporations aren't likely to change, but new startups and work owned coops exist, so why don't they compete?
Maybe ranking it on a scale of best to worse is too simplistic a view, and there are reasons this develops. Maybe it is the best option when there is a good leader, thus such structures dominate, much as a government ran by philosopher kings are better. But this only lasts as long as a wise rule is in charge, and it reverts back to a norm, and eventually, due to pure time and chance, enough bad leaders come on board that slowly dismantle the giants, but this happens at a time scale we don't particularly notice due to how much inertia large corporations can have (before we even get into the less pleasant issues like regulatory capture).
Think the difference between AI saying "This paragraph seemed muddled and lacks a clear point. Consider rewriting it." vs "Here, I rewrote this paragraph to focus it more on bridging the previous and next paragraphs."
The problem with this as a metric is that it is loosely defined so it becomes quite easy for a person to twist it to justify almost any level of AI usage as "well, it is still more effort than <X>".
But nothing is ever free. So what is the cost.
<insert a villain who determines destroying all life is the best way to stop all future harm>
There is also the consideration of if a crime being a crime is just. Consider crimes that use to be on the book that we now consider horrible things to have outlawed or even crimes that are still on the books but not enforced because we reject them.
For example, some places us to make it a crime for kids to play pinball. Is preventing kids from playing pinball, even if it came at no cost, an unquestionably good thing?
For a sufficiently bad crime, at sufficiently low costs, preventing it is good. But those two factors are both very big questions, directly challenging the notion of "unquestionably good".