I love the idea, and I developed something similar of myself in the past (https://github.com/golddranks/kobuta), but... this reeks of slop. With Rust code, edition="2021" is a dead giveaway.
Why aren't the role tags preprocessed algorithmically/deterministically and then fed in as one-hot-encoded vectors alongside the semantic word embeddings? I'd imagine that it would be easier to train to _stay_ in the role an not confuse it, if the current role marker is explicitly set as a part of each input token, and not just implied by some past token. Plus a input separate from the word embedding would be unforgeable.
I am currently quitting a company of 10 years of employment. And I keep hearing how everything's shit. Btw. I'm located in Tokyo where it isn't as bad, apparently, but...
Let's see. My plan:
- Have my own company and start looking for customers. (Rust consulting)
- Keep looking for job opportunities, but don't succumb for shit jobs.
It might be that I'm too hopeful, but you can't know unless you try.
Anyway, I may join the "everything is shit" crowd in half a year if nothing pans out, but until then, I'm hopeful.
Indeed, I realized I had my timeline a bit off after posting that. Of course, we have Lisp ca. 1960. (Pascal appeared 1970, but I don't think that's widely considered memory-safe. ML 1973, and that never got a widespread industry use.)
What I mean is: we had memory safe system-implementation languages in wide spread production use only after/around the times of the publication of that memo; importantly, Java.
We had memory-safe experimental programming languages, and scripting languages before that. And of course, around those times, hardware was fast enough that you could start implementing systems with scripting languages (Perl, Python, Ruby, JavaScript)
And the bit I want to correct is, of course, the point is if they are actually used. In that sense, I'd correct the introduction of Rust around 2023-2026 in actual, wide-spread use.
I disagree. I think that after 1992, we got memory safe languages that brought a meaningful improvement to the status quo. And after 2015, we've got low-level memory safe languages (Rust, as the major example. There are others, more experimental.)
The average programmer doesn't get better – if anything, we might be getting worse, because the tools allow us to, and the capitalist reality doesn't optimize for great programs or programmers but for more money.
But, at least, the tools are way better than in 1992, and I think we, as a collective profession, have learned a thing or two.
I think it was way too easy to guess corretly based on exluding obviously incorrect choises and then going with vibes.
There were many words I couldn't have explain the meaning of at all, if I wouldn't have had the options, but having the options made it easy. I wouldn't count those correct answers as a part of my vocabulary (even passive), even if I could answer with relative confidence.
This is interesting, as Ralf's articles always are. I wonder if/how will this concept be formalized. Rust is slowly but surely moving in good direction with regards to formal model of the language.
On the name "Respectify": it immediately reminded me of Linus Torvald's famous quote "respect should be earned". That quote, in its literal form, strikes a chord with me. While I share his sentiment towards respect, I think that lacking respect towards any individual shouldn't entitle you to be an asshole – but that's something that Linus has historically been from time to time. In that context, the quote sounds like a sorry excuse.
In my opinion, the toxicity of communication shouldn't be framed in terms of respect, but in terms of "basic human decency". To me, using the word "respect" sounds like the right to non-toxic communications should be earned. I'd rather have that as the baseline, which is a value that I expect you to share.
Science works by scientist having a model of reality and then testing that model against reality, gathering evidence that fits or doesn't fit the model, evaluating how well the model corresponds to reality.
If there is a widely accepted model in the archaeological community, and the new data contradicts it, the wording "than archaeologists thought" seems plausible enough.
Of course, depending on the model, the model itself might admit regimes of "non-applicability", or have some measure of confidence... If archeologists have large uncertainty whether human ancestors made tools 500,000 years back or not, then they shouldn't be surprised upon finding evidence that the ancestors did.
I don't know any specifics about this case, just arguing that that kind of wording by itself is not always wrong by default.
Yeah, you're right, and the snark might be warranted. I should consider it the same as my stupid (but cute) robot vacuum cleaner that goes at random directions but gets the job done.
The thing that differentiates LLM's from my stupid but cute vacuum cleaner, is that the (at least OpenAI's) AI model is cocksure and wrong, which is infinitely more infuriating than being a bit clueless and wrong.
Just a supplementary fact: I'm in the beneficial position, against the AI, that in a case where it's hard to provide that automatic feedback loop, I can run and test the code at my discretion, whereas the AI model can't.
Yet. Most of my criticism is not after running the code, but after _reading_ the code. It wrote code. I read it. And I am not happy with it. No even need to run it, it's shit at glance.
I feel like I'm taking crazy pills. The article starts with:
> you give it a simple task. You’re impressed. So you give it a large task. You’re even more impressed.
That has _never_ been the story for me. I've tried, and I've got some good pointers and hints where to go and what to try, a result of LLM's extensive if shallow reading, but in the sense of concrete problem solving or code/script writing, I'm _always_ disappointed. I've never gotten satisfactory code/script result from them without a tremendous amount of pushback, "do this part again with ...", do that, don't do that.
Maybe I'm just a crank with too many preferences. But I hardly believe so. The minimum requirement should be for the code to work. It often doesn't. Feedback helps, right. But if you've got a problem where a simple, contained feedback loop isn't that easy to build, the only source of feedback is yourself. And that's when you are exposed to the stupidity of current AI models.