dougall

no profile record

Submissions

How I made Tracy 30× faster

3 pointsby dougall3 anni fa0 comments

comments

6 mesi fadiscuss

Hi, author here. My version definitely shouldn't be faster unless something very weird is going on with the runtime (though I think with the benefit of hindsight some further optimisation of it is possible). I have never seen a good use for this, aside from as a proof that it is possible, but I can imagine it coming up if, say, you wanted to write an exploit for an esoteric programming language runtime.

If you still maintain this code and want to optimise it, I don't think you should need a full powers-of-two table, just having log(n) powers of two should do in a pattern like:

  if (v > 2**1024) { v *= 2**-1024; e += 1024; }
  if (v > 2**512) { v *= 2**-512; e += 512; }
  ...

That's a straightforward memory saving and also leaves v normalised, so gives you your fraction bits with a single multiplication or division. This is a little less simple than I'm making it look, because in reality you end up moving v to near the subnormal range, or having to use a different code path if v < 1 vs if v >= 2 or something. But otherwise, yeah, the code looks good.

2 anni fadiscuss

It's fun :)

2 anni fadiscuss

Yeah, exactly this.

To try to make it more "concrete", compilers tend to end up processing an absurd mess of code, that does things that superficially look silly, due to extensive inlining.

The code above is entirely plausible for C code checking the alignment of a total size, where "a" and "b" are the sizes of two parts, both calculated by a multiplication that ensures their evenness.

Another realistic examples are flags-fields, where each bit represents a boolean. For example, the compiler should be able to optimise:

    flags |= 0x100;
    if (!(flags & 0x100)) { ... }

Addition might factor in either as a substitution for bitwise-OR or bitwise-XOR where the author has prior knowledge that the two are equivalent (and assumes it will lead to better code generation), as part of SWAR (simd-within-a-word) code, or in cases where integer fields are packed alongside flags (e.g. a refcount and a couple of flags, or a string-length and some flags).

If this seems rare and unusual, that's cool – in general it is. But these strategies are very heavily used in performance-critical code, where using multiple variables would cause a measurable slowdown, so it makes sense that compilers care about optimising this kind of thing.

2 anni fadiscuss

Author here - I believe this 2020 post resurfaced as known-bits optimisation was recently added to PyPy (Python implementation using a JIT compiler), which this thread discusses:

https://mastodon.social/@cfbolz/112557672421317765

It includes discussion of where this kind of optimisation helps (not most "normal" Python code, but some code like emulators with lots of bitwise operations is ~30% faster), links to LLVM and Linux kernel implementations, and some related blog posts and papers. (I believe the Linux eBPF people came up with all of the ideas here before I did, though I wasn't aware of it at the time - it's easy to forget the Linux kernel has an optimising compiler in it.)

I'm not sure if that's what you mean by context - happy to answer questions.

3 anni fadiscuss

Awesome! I'm always happy to hear people are interested in SVE :)

(And yeah, creating technical documentation that a large company should have created themselves has somehow become a hobby of mine.)

3 anni fadiscuss

Heh, I'd have called it the Arm SIMD Instruction List, but Arm have been aggressively enforcing the Arm trademark [1], so I settled for A64 (the official name for the instruction set [2]).

[1] https://www.theregister.com/2023/08/31/a_star_star_domains/

[2] https://developer.arm.com/Architectures/A64%20Instruction%20...

3 anni fadiscuss

I hope it's useful, though I think most people care about the "Advanced SIMD" (Neon) instructions, which I'd also like to do. I started with SVE because I wasn't already familiar with it, so it was a more interesting project.

(For anyone unfamiliar, SVE is supported only on extremely recent ARM CPUs, and Apple CPUs do not yet support it, whereas AdvSIMD is available on all ARMv8-A CPUs.)

3 anni fadiscuss

It is a trade off, but a lot of processors have free offsets from loads, so pointer chasing is almost always free.

On ARM, "ldr x0, [x1]" just becomes "ldur x0, [x1, #-1]" - same size, same performance (at least on the Apple M1). If you have an array in a structure "add x0, x1, #16 ; ldr x0, [x0, x2, lsl #3]" becomes "add x0, x1, #15 ; ldr x0, [x0, x2, lsl #3]".

The only place I can think of penalties are pointers directly to arrays: "ldr x0, [x0, x1, lsl #3]" becomes "sub x0, x0, #1 ; ldr x0, [x0, x1, lsl #3]". Or loads from large offsets - ldur can only reach 256 bytes, whereas the aligned ldr can reach 32760 bytes. In either case the penalty is only one SUB operation.

3 anni fadiscuss

The tag bit can be inverted.

V8 uses 1 for pointers, 0 for integers - you're usually loading from constant offsets from pointers anyway, so that mostly folds away nicely. Then:

x + y is translated to CPU instructions x + y

x * y is translated to CPU instructions (x >> 1) * y

x / y is translated to CPU instructions (((x >> 1) / (y >> 1)) << 1)

x lsl y is translated to CPU instructions (x << (y >> 1))

3 anni fadiscuss

Yeah... I don't know if you saw Rodrigo Branco's damning "The Microarchitectures That I Saw And The Ones That I Hope To One Day See":

https://www.youtube.com/watch?v=WlcQrx7VK00 https://hardwear.io/usa-2023/presentation/the-microarchitect...

But it definitely seems to be a culture/disclosure problem.

(Also, hi - hope things are going well! We miss you on Mastodon)

3 anni fadiscuss

Browsers use multiple processes for security and reliability, not as an alternative to multi-threading. They extensively use multi-threading for performance (as does a lot of other modern software).

3 anni fadiscuss

Yeah – ARM specifically added EOR3 and BCAX instructions to accelerate SHA-3 hashes, both of which can be handled by VPTERNLOGD.

3 anni fadiscuss

Is there a reason you call out Discord and not GitHub?

3 anni fadiscuss

Good point, it's likely that a high-power in-order chip could go faster. Do you know of any examples of high-power in-order cores I could compare to?

3 anni fadiscuss

Alas, security generally isn't so important.

How many times have you been hacked by a side-channel exploit? (Or people you know? Or any publicly documented case?) Are you going to use a computer that runs at 1/10th the speed to mitigate that risk going forwards?

Keep in mind that a ton of non-side-channel exploits are caught in the wild every year, so your slow new computer isn't really secure, it's just not vulnerable to these specific attacks.

(For 1/10th: the Cortex-A55 in the following chart is the only "in-order" CPU: https://images.anandtech.com/doci/17102/SPECint2017_575px.pn... - though arguably even it isn't completely non-speculative, and it definitely has branch-prediction, but it's at least a reasonable ballpark.)

3 anni fadiscuss

Nice!

Sorry if this comment is overly pedantic, I just enjoy having an excuse to talk about assembly.

It's worth noting that 0x80000000 would pass this "is zero" check. (I think this is probably a legal compiler optimisation because signed integer overflow is undefined, but I'm not 100% sure either way.)

Using a jump is also a bit risky - slightly better if it's predictable, much worse if it's unpredictable.

As far as size, this is 5 bytes on 32-bit x86 (as stated), 6 bytes on 64-bit x86, but can be 8 bytes if different registers are used:

    4501C0            add r8d,r8d
    7503              jnz 0x8
    41FFC0            inc r8d

(And, unlike the ARM code, you'd need an additional mov instruction if you wanted to preserve the input value.)

It feels like an ADC-based variant might be possible on x86 too - CMP and ADC are also x86 instructions. The problem is that ARM and x86 invert the value of the carry flag on subtraction (and comparison), so it doesn't translate directly, and I can't immediately see how to fix it up without using more instructions.

3 anni fadiscuss

Though it'd be preferable to do:

    cmp wzr, w19      // set the carry flag if w19 is zero
    adc w8, w19, w19  // w8 = w19 + w19 + carry

3 anni fadiscuss

Some: https://news.ycombinator.com/item?id=35301630

3 anni fadiscuss

Really? Is there case law to this effect?

These things happen, and have value, but I wouldn't have considered them as "given as a quid pro quo" or "mutual promises".

3 anni fadiscuss

It's a huge issue, but multi-threading only helps with latency, not energy.