MaskRay

20 days agodiscuss

While we could utilize zigzag encoding (i>>31) ^ (i<<1) to convert SLEB128-encoded type/addend to use ULEB128 instead, the generate code is inferior to or on par with SLEB128 for one-byte encodings on x86, AArch64, and RISC-V. Haven't tried wider values - but zigzag encoding is likely slower as well

// One-byte case for SLEB128 int64_t from_signext(uint64_t v) { return v < 64 ? v - 128 : v; }

// One-byte case for ULEB128 with zig-zag encoding int64_t from_zigzag(uint64_t z) { return (z >> 1) ^ -(z & 1); }

MaskRay

22 days agodiscuss

GetProgramPath consulting PATH is a pretty standard behavior inspired by GCC. Clang driver may find an external program from its program paths (e.g. -ccc-install-dir, the GCC installation's bin directory, -B, ) and then PATH.

> Clang relies on address layout for ordering things

Every such instance is a bug. I have fixed many issues in 2023. There is even an upstream build bot https://discourse.llvm.org/t/reverse-iteration-bots/72224

MaskRay

5 months agodiscuss

git switch v6.9

The riscv build succeeded. For the x86-64 build I ran into

    % make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 HOSTCC=/tmp/p/claudes-c-compiler/target/release/ccc-x86 LDFLAGS=-fuse-ld=bfd LD=ld.bfd -j30 vmlinux -k
    make[1]: Entering directory '/tmp/linux/x86'
    ...
      CC      arch/x86/platform/intel/iosf_mbi.o
    ccc: error: lgdtl requires memory operand
      AR      arch/x86/platform/intel-mid/built-in.a
    make[6]: *** [/home/ray/Dev/linux/scripts/Makefile.build:362: arch/x86/realmode/rm/wakeup_asm.o] Error 1
    ld.bfd: arch/x86/entry/vdso/vdso32/sigreturn.o: warning: relocation in read-only section `.eh_frame'
    ld.bfd: error in arch/x86/entry/vdso/vdso32/sigreturn.o(.eh_frame); no .eh_frame_hdr table will be created
    ld.bfd: warning: creating DT_TEXTREL in a shared object
    ccc: error: unsupported pushw operand

There are many other errors.

tinyconfig and allnoconfig have fewer errors.

    RELOCS  arch/x86/realmode/rm/realmode.relocs
    Invalid absolute R_386_32 relocation: real_mode_seg

Still very impressive.

MaskRay

5 months agodiscuss

I want to verify the claim that it builds the Linux kernel. It quickly runs into errors, but yeah, still pretty cool!

make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc -j30 defconfig all

``` /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:44:184: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) ~0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (~0x80000000); (void)pto_tmp__; } asm ("and" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:49:183: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) 0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (0x80000000); (void)pto_tmp__; } asm ("or" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:61:212: error: expected ';' after expression before 'pao_tmp__' ```

MaskRay

5 months agodiscuss

I accidentally published handling-long-branches.html . I actually removed it quickly, but Google Search already got it....

https://news.ycombinator.com/item?id=46759921 (Long branches in compilers, assemblers, and linkers), 2 days ago

MaskRay

6 months agodiscuss

Amazing! I read you article in 2012 when the link was https://homepages.cwi.nl/~tromp/maze.html I was learning Haskell and Ocaml and wrote my own article in Chinese then https://maskray.me/blog/2012-11-02-perfect-maze-generation

Now I should fix the link.

MaskRay

6 months agodiscuss

I recall that google3 switched to -funsigned-char for x86-64 a long time ago.

MaskRay

6 months agodiscuss

Open source project maintenance follows a similar model, but with a different set of stakes.

The "price tag" of voicing concerns is lower, yet raise them too often and you still earn a reputation as obstructionist. Meanwhile, the cost of accepting problematic changes can be higher—you may end up maintaining that code long after changing jobs. And unlike corporate politics, the "influence bank account" is public: communications are archived indefinitely.

There is a fascinating shift in how "withdrawals" are calculated: In a corporate hierarchy, the cost of dissent feels exponential: something like `cost = exp(their_level - your_level)`. Say, as a Google L3/L4/L5 engineer, opposing L6-L8 feels like trying to make a massive withdrawal with a tiny balance. In contrast, in OSS the cost almost stays constant despite the corporate level difference.

This created a paradox for me: leaving Google means less time for LLVM maintenance, but it also lets me voice objections more freely, without the shadow of internal performance ratings or hierarchical friction.

That said, I know I've been "withdrawing" heavily, including from a lot of previous colleagues. In a recent LLVM Project Council meeting:

> There is a pattern of behavior here of blocking contributions due to concerns about maintenance cost and design simplicity.

(I appreciate the transparency of making these meetings public, by the way.)

I had to respond at https://discourse.llvm.org/t/llvm-project-council-meeting-no...

MaskRay

6 months agodiscuss

We are missing tooling to partition a huge binary into a few larger shared objects.

As my https://maskray.me/blog/2023-05-14-relocation-overflow-and-c... (linked by author, thanks! But I maintain lld/ELF instead of "wrote" it - it's engineer work of many folks)

Quoting the relevant paragraphs below:

## Static linking

In this section, we will deviate slightly from the main topic to discuss static linking. By including all dependencies within the executable itself, it can run without relying on external shared objects. This eliminates the potential risks associated with updating dependencies separately.

Certain users prefer static linking or mostly static linking for the sake of deployment convenience and performance aspects:

* Link-time optimization is more effective when all dependencies are known. Providing shared object information during executable optimization is possible, but it may not be a worthwhile engineering effort.

* Profiling techniques are more efficient dealing with one single executable.

* The traditional ELF dynamic linking approach incurs overhead to support [symbol interposition](https://maskray.me/blog/2021-05-16-elf-interposition-and-bsy...).

* Dynamic linking involves PLT and GOT, which can introduce additional overhead. Static linking eliminates the overhead.

* Loading libraries in the dynamic loader has a time complexity `O(|libs|^2*|libname|)`. The existing implementations are designed to handle tens of shared objects, rather than a thousand or more.

Furthermore, the current lack of techniques to partition an executable into a few larger shared objects, as opposed to numerous smaller shared objects, exacerbates the overhead issue.

In scenarios where the distributed program contains a significant amount of code (related: software bloat), employing full or mostly static linking can result in very large executable files. Consequently, certain relocations may be close to the distance limit, and even a minor disruption (e.g. add a function or introduce a dependency) can trigger relocation overflow linker errors.

MaskRay

6 months agodiscuss

Note, sections without the SHF_ALLOC flag, such as `.debug_*` sections, do not contribute to the relocation distance pressure. Many 10+GiB binaries (likely due to not using split DWARF) might have much smaller code+data and not even close to the limit.

However, Google, Meta, and ByteDance have encountered x86-64 relocation distance issue with their huge C++ server binaries. To my knowledge industry users in other domains haven't run into this problem.

To address this, Google adopted the medium code model approximately two years ago for its sanitizer and PGO instrumentation builds. CUDA fat binaries also caused problems. I suggest that linker script `INSERT BEFORE/AFTER` for orphan sections (https://reviews.llvm.org/D74375 ) served as a key mitigation.

I hope that a range extension thunk ABI, similar to AArch64/Power, is defined for the x86-64 psABI. It is better than the current long branch pessimization we have with -mcmodel=large.

---

It seems that nobody has run into this .eh_frame_hdr implementation limitation yet

* `.eh_frame_hdr -> .text`: GNU ld and ld.lld only support 32-bit offsets (`table_enc = DW_EH_PE_datarel | DW_EH_PE_sdata4;`) as of Dec 2025.

MaskRay

10 months agodiscuss

Rex

Submissions

Long branches in compilers, assemblers, and linkers

Remarks on SFrame

LLVM's AI policy vs. code of confuct vs. reality

comments