Pannoniae

2,565 karmajoined 3 yıl önce

contact: https://discord.gg/ZVySp5m9N8 or pannoniae

https://pannoniae.net

HackerSmacker profile: https://www.hackersmacker.org/user/Pannoniae?hs=1h0S7zRN1vKASKycIQ

comments

evvelsi gündiscuss

Well yeah, for example I'm working on a codebase written mostly in 2003.

The assumptions in the original code are at best tangentially relevant (stuff like "LUTs are cheaper than using ALU" or "CPU power is cheap, GPU power is very expensive").

Despite that, a grand rewrite would have failed massively (even with agents...), replicating millions of lines of code to be functional again just sounds like an absolute nightmare.

So I've been doing "targeted rewrites" instead - not sure, there's maybe a name for it? Basically do it concern by concern, module by module, so even if lots of code was touched, it only affects one logical feature at the same time.

27 gün öncediscuss

They're just clearly inferior in pretty much any situation.

The map stuff the other posters summed up well but even std::vector is dogshit with pretty much all implementations having inlined grow code in push_back, a not too great API and missed optimisations e.g. no trivial relocation when growing the vector / moving it and no useful APIs such as "grow but don't initialise"...

2 ay öncediscuss

Exactly, you write for your target, not some imaginary spec. The spec is only as useful as to predict what your target roughly does, it's not normative.

Compilers might have bugs where the spec is supposed to work but it doesn't, and many extensions without standard equivalents, or implementation-specific behaviour where undefined things in the standard do get assigned a meaningful outcome.

2 ay öncediscuss

Sure, if you're willing to write a large enough chunk that you can eat the cost of not inlining it. If you just write a small leaf function or two, it will probably be a wash or perform worse.

2 ay öncediscuss

sadly inline assembly is still at the ergonomics of "one compiler doesn't support it in x64 mode" and "you can choose between the readable syntax (which is a black box to the compiler) and the unreadable syntax (which can specify I/O/clobber regs)"

2 ay öncediscuss

Hi! I'm slightly under that so I feel like I'm qualified :)

I wholeheartedly agree, they're quite a regression.... although I don't think this is a popular opinion around here.

When people say "something used to be better" they usually don't mean literally, they mean that for the circumstances, it was better. Of course, more modern systems support more hardware, more features, etc., but if you made those same modern technical improvements on top of an older designs, you'd get much better results.

To me it looks like software design has been massively overtaken by "form over function", everyone just wants a unique "brand" but the actual UX is complete dogshit. Borderless buttons, zero indication what's clickable, no visual delimiters for different areas of programs, no good shortcut / altkey menu support, etc....

This has somehow infected even Linux to such a crazy extent...

2 ay öncediscuss

I've never said that, but using assembly in certain places can certainly be justified, especially for the performance-intensive parts.

2 ay öncediscuss

You also depend on {computers, the internet, electricity, groceries, etc.}, that's not the discriminating factor.... the question is, are you actually better at your job using it, and are you stunting your own growth?

I'm not claiming to have a definite answer to either, but I think the right question to ask is - are you going to benefit from using it in the long run? If yes, carry on, if no, re-evaluate what you're doing :)

2 ay öncediscuss

Look at the bright side, it's much more feasible now :)

Yes, it's not deterministic, and if you were using it commercially, the ROI would be terrible, and it's certainly not reliable but for a hobby project.... why not?

Encouraging people to understand the layer of abstractions they're building on is helpful, doesn't matter if they do it by hand or with clankers.

LLMs lower the barrier for execution - they make you faster. The unstated question is: faster at what - they can make you faster at something clever, or faster at the entirely wrong thing...

Your point is correct if we're looking at it through a scarcity lens - the effort to make it certainly decreased a lot - but that doesn't mean that anything is now worthless. We can just move onto doing bigger, better things now, until we hit the next limits...

2 ay öncediscuss

That's just not true... the flags to get preprocessed output and assembly are quite useful and used a fair bit, in fact. Multiple reasons - sanitychecking your code, finding bugs, or even finding compiler errors.

2 ay öncediscuss

You'd be surprised, again.... most compilers don't generate very good code, mostly because

1. the time for optimisation is limited

2. the constraints are overlapping and just completely intractable beyond a single function (do you want to inline this, saving on the call and increasing binary size, or not do it because it's cold?)

3. they don't have domain-specific knowledge about your code, and even with PGO, they might incorrectly decide what's hot and what's not - typical example are program settings. You didn't enable a setting during PGO instrumentation, compiler sees you didn't call that path, shoves it out of line. Now your PGO-optimised code is worse than -O2. And compilers have different levels of adherence to manual branch hinting - on MSVC you get a reorder at best, Clang and GCC try much harder at [[likely]] and [[unlikely]].

4. There's still quite a bit of low-hanging fruit left, mostly because progress is jagged ;) For example our calling conventions generally suck - this is actually why inlining is so helpful - and the inertia makes everyone emit the default calling convention and that's it.

For example, did you know that compilers have very inconsistent support for struct unpacking? It can be much faster to write

  int32 meow(int64 a, int64 b);

than

  struct mytype {
    int64 x;
    int64 y;
  };

  int32 meow(mytype a);

because the first one goes through registers on the MSVC ABI, the second one gets lowered to the caller passing a pointer to the stack. Before someone says "oh this just means MS sucks" - fair, but for std::unique_ptr the situation is the other way around... on the MSVC ABI the callee cleans it up so it's truly zero-cost, but on the Itanium ABI using it is worse than using T* as a raw pointer... see the GCC codegen :)

These examples might seem a bit cherrypicked but this is only scratching the surface, not to talk about the codegen in higher-level languages, which is even more dreadful. Manually optimising your code can usually get a magnitude worth of free performance, which is just tragic.

I wouldn't even rule out LLM codegen in the future - although they're quite unreliable today so you'd get miscompiles like crazy - but there's just so much low-hanging fruit left on the table that it wouldn't be too out of step...

2 ay öncediscuss

Just like LLMs, compilers are just another layer of abstraction and no they're not deterministic.

Just yesterday I've reported a codegen bug in MSVC. (Luckily they've fixed it very fast.) Can you realise that it's an optimiser bug without inspecting the assembly? Hardly.

All the arguments people claim against LLMs are similarly applicable to compilers, but compilers are old technology and LLMs are new.

If you're an expert, just about every compiled function contains obvious inefficiencies, and a skilled assembly programmer can speed it up by in the ballpark of 3x. If we're talking about your average webapp, you can usually get 1000x better resource usage in most ways, including CPU, RAM, storage and so on.

And the output isn't deterministic either - the bugs no withstanding, code generation is highly chaotic, optimisations have non-local impacts and you can't easily predict optimised codegen output from source.

LLMs aren't much worse. They have non-deterministic output, but you can steer it - similarly to a compiler. An expert can use it to gain great speed and efficiency, but in the hands of someone not as capable, you can make something awful just as fast. Both tools are force multipliers.

2 ay öncediscuss

Unless something radical changes (and that isn't unprecedented! I'm just writing this as of today), the trend is still "just" a bigger hammer. It's bigger, you can get way more done, but the blast radius is also larger.

Take the strawman: even if AI can one-shot basically any application below let's say, 1MLoC, if your prompt is 4 lines, it will generate something. It can't read your mind. If you make proper specs, then you'll get what you want - but many people don't know what they want. And even if they do, they might have contradictions in their requirements, might be asking for something impossible, etc.

2 ay öncediscuss

All of it hell no :D But just with any things, you break things down into subtasks. Then you break it down even more. You as a human don't hold all that stuff in your head either, so why would an LLM?

My current codebase is ~3 million LoC all in all (not greenfield, really old code), working on it by myself, the complexity is definitely manageable between Claude and me :)

2 ay öncediscuss

Calling win32 from other languages is supported, calling it from assembly is supported (as long as you use the calling convention properly, obviously), using ntdll to bypass the win32 API is not supported.

Basically on Linux the syscalls are the equivalent of Win32 except much narrower in scope.

2 ay öncediscuss

Sure but even wiring that utility function in is work :D If you have even just a 2-3-million LoC codebase, not even something truly enormous - making global changes does require typing, and a whole lot of it...

2 ay öncediscuss

I don't follow, sorry.

1. In order to publish a game to a console, you do need an agreement with the console manufacturer. (Unless you're doing homebrew but then the point is moot.)

2. In that case, you do get access to the official documentation, everything is solid.

3. The word "legally" has a lot of bearing there. Sure, they have to. And if they don't, what? Will the other contributors sue them? If they aren't willing to do that, then they can get away with it.

4. The context is about someone trying to hack the devices. ("Who knows what some enterprising hacker might be able to glean about the workings of the console by looking at the APIs.") They certainly won't be bothered by NDAs.

2 ay öncediscuss

Yeah, you're definitely right about the shifting goalposts ("it's a stochastic parrot" -> "it hallucinates all the time, it can't even get APIs right" -> "it can generate functions but can't reason about the codebase" -> "the bottleneck was never shipping code")

At the same time, humans can move up the abstraction ladder faster than the LLMs can. At least, some humans. Agents can produce lots of code. They can also do the entirely wrong thing. The impact of wrong decisions have been massively write-amplified with more and more intelligent LLMs. With earlier ones, it got a sentence or a function wrong, you reprompted, the cost of a mistake was 10 seconds. Now, you can burn hours or even days of work on the entirely wrong thing without a competent human operator stepping in and course-correcting.

The trajectory of agents have been bigger and bigger context windows, bigger autonomy, but at the same time, a bigger blast radius. In this context, I don't think the human experts will be out of their jobs any time soon.

2 ay öncediscuss

Ideally. But that requires the correct abstraction, requires keeping it up to date.... that's basically an unachievable ideal. You either have overabstraction/overengineering (most codebases) or you have repetition. Repetition is actually more preferable in the LLM-world because you have to keep less stuff in your head. And the LLM's head too.

Even if something does look copypasted, it might actually be semantically distinct enough that if you couple them, you'll create a brittle mess.

Additionally, there's always going to be global changes (update the code style, document things, refactor into a new pattern, add new functionality to callers, etc.). The question isn't whether you use your lanuage's tools or you do it by hand, the question is whether you use an LLM or do it by hand :P

2 ay öncediscuss

All of them you can already find on the internet with only a bit of searching btw. With the docs and headers and everything.

The only people who this stops are the people who want to do things "by the book". :P