orbital-decay

6,035 karmajoined قبل 9 سنوات

Don't blindly believe any comment that looks competent; use your damn sense.

comments

قبل 17 ساعةdiscuss

I'm just pointing out that the moral high ground the author thinks he stands on looks more like the Mariana Trench. If he's saying that the US oligarchs stole something from the entire humanity, giving it to another US entity can't unsteal it. "Public" in this article is pure mockery, like those Soviet mental gymnastics about "people" aka proletariat and peasantry owning everything, while in practice elites did. It's not meaningfully different from OpenAI's charter talking about the benefit of the entire humanity and then colluding with the government to restrict the access to this and that part of it, because how else it could be.

Eight billion people couldn't care less what kind of the US elites own their* stuff. If the author really thinks these companies owed something to the entire humanity then the article should start with "open weights" or "compensation for the entire humanity" or something like that, not a transfer of ownership inside the US.

*according to the article

قبل 23 ساعةdiscuss

>Artificial intelligence has been built by robbing the collective work of humanity.

Conclusion of the author: "we" should own and gatekeep it, as in the US government, of course. The rest of humanity which has been "robbed of its collective work" is ignored.

Korean war was way more extreme. NK has been bombed into an absolute moonscape, yet never surrendered.

It's pretty clear you've never experimented with it. Creative writing demands everything the model can do and more, and most problems are still unsolved. It's extremely heavy reasoning-wise, more so than coding (check e.g. Engram paper for some insights), but also needs good scattered retrieval, careful subjective training for prose quality, character, and human likeness, a ton of facts baked in, and much much more. Mode collapse is not solved. No LLM does creative writing well but historically only the absolute largest models were able to do write anything complex more or less convincingly and were creative enough.

The real question is not how many "weights" the human brain has (neurons+synapses may or may not translate into "weights", and brain might be also inefficient for what it is), but rather how much evolutionary and social "compute" was necessary to pack everything into that capacity.

Or a lot better efficiency.

أول أمسdiscuss

https://hcker.news/?ai=exclude

There's the "small web" filter too, if you want this.

أول أمسdiscuss

Commenter comments before reading.

The entire article literally starts by referring to Google coding guidelines and other well known opinions as arguments against unsigned, then tries to provide a rebuttal.

قبل 3 أيامdiscuss

Depending on who "we" are, "we" are contributing to its stability and don't give a shit about people.

قبل 3 أيامdiscuss

The text can be legible, but the meaning the model assigns to these words can be subtly different, and you have no way to tell. This is evident when trying to make most modern reasoning models follow a fixed CoT plan by filling a form with placeholders, they're extremely stubborn because they simply don't understand your words in it! They learned their own language in their CoT. Sometimes they're controlled for readability during training but even then they find a way to circumvent this, for example Gemini 3.0 had perfectly readable raw CoT but was barely able to follow such a plan (3.5 Flash is way better at following, they clearly improved it).

قبل 3 أيامdiscuss

As it always is with these articles, that has nothing to do with non-determinism the author is talking about. Model's input is in natural language which isn't formally defined, unlike Ragel's input. This makes it open to interpretation by the model that isn't trained the same way as you, has very limited cognitive capabilities, and must generate something in very limited time by design, even if the result is incorrect. This also makes it not related to determinism in any way. You can make model outputs deterministic, but this won't solve your problem because it's not about determinism. Words have meaning.

Claude or any other model just translates your natural language instructions into formally defined tool calls. You cannot replace this layer with a formal tool like Ragel. You can write code for Ragel directly, in which case the responsibility for this is yours and not Claude's. (duh)

>What about Claude? Well, my instructions say in all caps: DO NOT PARSE ANYTHING MANUALLY, EVER. (...) It tries anyway

This needs a self-verification loop. It still won't guarantee that model's interpretation will match yours, but it will improve the accuracy. Almost every model will know that it went off the rails upon checking what it's trying to do. Harness has to provide the loopback for this, because the transformer architecture doesn't.

قبل 3 أيامdiscuss

When in doubt, look for adjacent localities and do what the others are already doing. OSM is chaotic and (usually) driven by consensus that can be slightly different in specific communities.

قبل 4 أيامdiscuss

>The reality is none of the companies want to do these things. (...) They're an unfortunate response to (...)

This reasoning can be used to justify pretty arbitrary behavior, don't you think? Intent is irrelevant for malice, incentives are enough.

قبل 4 أيامdiscuss

I used to host pretty popular servers for several games (mostly the PvP survival kind), built a community around them, and developed custom admin software. It's hard to say I haven't experienced this first-hand, I fully understand the reasons for this to exist. Yet I 100% agree with the author, this went way too far long ago. This will NOT stop until the personal computers are forcibly locked into a fully attested hardware chain, for everyone not just gamers. The control over your personal computer will be eventually taken from you even if you never played a game in your entire life. And then cheats will simply move outside the machine entirely. No entertainment is worth it.

Starting from a certain point, the solution for cheating is better game design, more human involvement, and community building, not adding more restraints which will be circumvented in any case.

قبل 4 أيامdiscuss

I'm sure Anthropic of all companies don't do that, since using mechinterp as a training target will make the the result uninterpretable.

قبل 5 أيامdiscuss

I'm saying that the models have to be trained better for them to be considered creative. Looking at how many low hanging fruits there are in AI right now, I'd say it's possible given enough time and slow enough adoption. Looking at how little the AI labs and everyone else are interested in those low-hanging fruits and how much they're focused on politics, hype, and safety religion, I'm not optimistic.

قبل 5 أيامdiscuss

I'm saying extrapolation as a mathematical concept is orthogonal to the non-rigorous definition of creativity.

قبل 5 أيامdiscuss

>The output of a GPT is an interpolation (an estimation of new data points inside the range of known data) rather than extrapolation (estimations outside that range).

That's a common meme but it's the opposite of true. Everything big models, not just transformers, mathematically do is extrapolation in the feature space, almost never interpolation. They're perfectly able of combining the ideas, although of course this ability declines once they're off the distribution, just like in humans. The model is creative and its output is transformative, however it only makes sense if you define creativity in a pure manner, based on novelty for itself.

However most people use entirely different definitions of creativity, something like "surprise me in a way that still makes sense to me". This includes "me", a side observer, and depends on what the side observer considers novel. Hence the main reason for the lack of "creativity" in big models is not some hand-waved "regression to the mean" or "interpolation", but the fact that they're still insufficiently intelligent compared to a human. Current models simply don't have enough fidelity to understand humans and think as deep, that's why humans think their output isn't sufficiently novel for them! Think about this, insanely smart aliens would probably dismiss humans as non-creative too.

The contributing factor is also the lack of semantic diversity in current models. Also known as mode collapse, but the name is a bit of misnomer and describes a technicality, not the resulting phenomenon. This indirectly affects creativity as it's usually understood, because the models generate and repeat the same thing in response to the same thing, which is the opposite of novel in layman's definition. That's part of where the slop comes from. Mode collapse has many causes, e.g. post-training with current algorithms. It's likely fixable but AI shops show little to no interest in studying and fixing it.

قبل 5 أيامdiscuss

The difference mostly comes from the default detail levels and shape optimization. You can make it as fast in OsmAnd as both are GPU-accelerated, but OsmAnd renders the map more detailed by default.

قبل 5 أيامdiscuss

And at the cost of map fidelity. Complex curves look terrible at certain zoom levels in CoMaps/Organic Maps.