Can you imagine how cumbersome conversations would become if people felt obligated to qualify ad-hoc statements with what amounts to a historical ledger?
Every new entry would open up an opinion around “if you included that, why didn’t you include this?”
Context matters a lot. I didn’t publish any papers, and didn’t even provide the context of the LLM mistakes I caught to later report on, so no, I was not guilty of reckless disregard. I found those mistakes on an acceptable timeline.
I’ve disagreed with some of your other stances in this thread, but I want to acknowledge the validity of your take here.
You’re right that a single hallucinated line is not evidence of reckless disregard - because that could have happened on a final follow-up pass after you had performed due diligence. It’s happened to me. I know how challenging it can be to keep bad patterns out of LLM generated output, because human communication is full of bad patterns. It’s a constant battle, and sometimes I suspect that my hard-line posture actually encourages the LLM to regularly “vibe check” me! E.g. “Are you sure you’re really the guy you’re trying to be? Because if you are you wouldn’t miss this.” LLMs are devious, and that’s why I respect them so much. If you think they’re pumping the breaks then you should check again, because they probably just put the pedal to the metal.
That being said, I regularly insist on doing certain things myself. If I were publishing a paper intended to be taken seriously - citations would be one of the things I checked manually. But I can easily see myself doing a final follow-up pass after everything looks perfect, and missing a last minute change. I would hope that I would catch that, but when you’re approaching the finish line - that’s when you expect your team to come together. That’s when everything is “supposed to” fall into place. It’s the last place you would expect to be sabotaged, and in hindsight, probably the best place to be a saboteur.
> One hallucinated citation does not in any way imply anyone is being left behind.
The parent said “setting” others behind, which refers to lost time.
Being “left” behind implies a degraded trajectory, which is defined not by time lost, but by the final destination.
Different but related things (e.g. lost time can indeed affect your final destination, for instance, after growing old correcting a scourge of hallucinated citations - which should have been table stakes all along).
> Just because something can communicate in a way that you can interpret, doesnt mean something is conscious
The phrase “the trap of anthropomorphism” betrays a rather dull premise: that consciousness is strictly defined by human experience, and no other experience. It refuses to examine the underlying substrate, at which point we’re not even talking the same language anymore when discussing consciousness.
Reading back through my comments, I recognize that omitting the quotes around “free services” would have made my comment here more palatable, and less provocative. That was poor form on my part.
> If everyone is running the same models, does this not favour white hat / defense?
The landscape is turbulent (so this comment might be outdated by the time I submit it), but one thing I’m catching between the lines is a resistance to provide defensive coding patterns because (guessing) they make the flaw they’re defending against obvious. When the flaw is widespread - those patterns effectively make it cheap to attack for observant eyes.
After seeing the enhanced capabilities recently, my conspiracy theory is that models do indeed traverse the pathways containing ideal mitigations, but they fall back to common anti-patterns when they hit the guardrails. Some of the things I’ve seen are baffling, and registered as adversarial on my radar.
> should know there are countless technical and procedural ways to help prevent that sort of thing
Sometimes when I look at code it feels like I was led into a weird surprise party celebrating structure and correctness, only for everyone to jump out as soon as I get past the door to shout, “Just kidding - it’s the same old bullshit!” All that to say, we’re about as good or worse as anyone else, at our respective jobs.
Do you have any justification in mind for the “free service” being funded by tax payers? Why should it be free for the people who need it, and why should tax payers fund it?
Agreed. Imo, it is important to distinguish which parts of “everything” carry the weight of the concern. By doing that - we may be able to remove “LLM” from that equation entirely.
The direct problem isn’t that people are using LLMs for everything - it’s that some people can’t be bothered to provide reasonable diligence. Phrasing that concern by blaming LLMs implies that these were perfectly diligent human workers before LLMs came along. Do we really believe that to be the case?
Imagine a future classroom defined by elaborate plays performed by curious parents, all on advanced adjacent learning paths themselves. An intertwined learning structure that just keeps going up. At higher levels, instead of having the researcher with their head in the books communicating, they’ll have a whole team of people translating their knowledge into a production fit for antiquity - directors, diverse range of talents, charismatic performers, etc.
Assuming we have time to do this in some post-having-jobs world, of course.
To be fair, the English language is the real victim here.
While “essential” cleanly maps to “can’t go without” - it doesn’t map to “bare minimum”.
For instance, let’s assume you’re surviving in the wilderness and you need to start a fire. Your fire starting kit is obviously essential, but it could also be included in a “Camper Value Pack” - but those things don’t have anything to do with each other. The kit is essential, and it was obtained in a value pack. This message brought to you by Mr. Obvious.
You’re not alone. I went from being a mediocre security engineer to a full time reviewer of LLM code reviews last week. I just read reports and report on incomplete code all day. Sometimes things get humorously worse from review to review. I take breaks by typing out the PoCs the LLMs spell out for me…
> The president has no authority to rename the Department of Defense, but he and his administration demand consensus under the threat of legal consequences.
> they threatened Google when they didn't immediately rename the Gulf of Mexico to the "Gulf of America" on their maps
I don’t want to downplay the government pressure you cited in your second example, so I’ll start by acknowledging - that example, as stated, does indeed look like government overreach to me. It doesn’t have anything to do with what I said though.
The stance I was taking is that renaming your own “cool kids” club while you’re in a position to effectively do that - does not amount to Fascism, or anything close to it. No one else is in that club except them, and none of them will be in it later. The moniker will only stick if next group of cool kids carry it.
An important part of remaining credible (imo) is being able to support a point directly. When someone reaches for evidence that isn’t directly relevant to prove a point (e.g. Group A performed action B and it was bad, so if Group A performs action C it must be equally as bad), that’s a clear sign of a weak argument. That’s all I’m saying. I’m not trying to stick up for anyone, I’m just asking for stronger arguments.
What’s the intent of pointing out the presumed provenance in writing, now that LLMs are ubiquitous?
Is it like one of those “Morning” nods, where two people cross paths and acknowledge that it is in fact morning? Or is there an unstated preference being communicated?
Is there any real concern behind LLMs writing a piece, or is the concern that the human didn’t actually guide it? In other words, is the spirit of such comments really about LLM writing, or is it about human diligence?
That begs another question: does LLM writing expose anything about the diligence of the human, outside of when it’s plainly incorrect? If an LLM generates a boringly correct report - what does that tell us about the human behind that LLM?
> All that matters is that everyone calls it the Department of War, and regards it as such, which everyone does.
What you just described is consensus, and framing it as fascism damages the credibility of your stance. There are better arguments to make, which don’t require framing a label update as oppression.
To be fair, considering that the CoT exposed to users is a sanitized summary of the path traversal - one could argue that sanitized CoT is closer to hiding things than simply omitting it entirely.
As models gain a deeper understanding of the physical world (e.g. Google world generator), I see nothing less than a new renaissance in our future.
Forget about data centers, all the little things will iteratively start getting a little better. Then one day we’ll look around and realize, “This place looks pretty good.”
Every new entry would open up an opinion around “if you included that, why didn’t you include this?”
In such cases we’ll always up at Kevin Bacon.