> For example, they invented the .eh_frame walking technique to get stack traces from binaries without frame pointers enabled.
This is not an accurate summary of what they developed.
Using .eh_frame to unwind stacks without frame pointers is not novel - it is exactly what it is for and perf has had an implementation doing it since ~2010. The problem is the kernel support for this was repeatedly rejected so the kernel samples kilobytes of stack and then userspace does the unwind
What they developed is an implementation of unwinding from an eBPF program running in the kernel using data from eh_frame.
No, wrong decade and wrong split - the test & measurement equipment and scientific equipment was long gone from HP at the time of the HP -> HP inc + HPE split. It ended up in Agilent (1999) and from there Keysight.
HP semiconductors went HP -> Agilent -> Avago, now broadcom.
How do get the weights for the right set of experts for a given batch of tokens into fast memory at the right time?
The activated experts is only available after routing, at which point you need the weights immediately and will have very poor performance if they are across PCIe
Why is there a new kernel driver here at all? It appears that all it does it allocate system RAM (“DDR4”) and export it as a dmabuf for import to cuda as mapped external memory. Then a userspace shim hijacks APIs to use that if gpu memory is full. cuda already supports allocating mapped system memory, so AFAICT this could be implemented in the userspace shim with no new kernel driver.
Also as other commenters have mentioned, redirecting allocations to managed memory would also enable similar oversubscription
And the hijack approach only makes sense for making apps have this behavior with no changes, and could be done with minor app changes (e.g. PyTorch has a pluggable allocator interface). App changes also enable intentionally placing specific allocations.
My impression is that this is vibe from beginning to end, starting from a design that only makes sense if you are hallucinating
This is not about reclaiming memory by swapping the contents out to disk. It is about killing processes due to having overcommitted beyond the available memory plus swap space. The processes thrown out of the plane (targeted by the OOM killer) cannot be resurrected
It’s not a cargo cult if the actions directly cause cargo to arrive based on well understood mechanics.
Regardless of whether it would be better in some situations to align to 128 bytes, 64 bytes really is the cache line size on all common x86 cpus and it is a good idea to avoid threads modifying the same cacheline.
> even on x86 on recent server CPUs, cache-coherency protocols may be operating at a different granularity than the cache line size. A typical case with new Intel server CPUs is operating at the granularity of 2 consecutive cache lines
I don’t think it is accurate that Intel CPUs use 2 cache lines / 128 bytes as the coherency protocol granule.
Yes, there can be additional destructive interference effects at that granularity, but that’s due to prefetching (of two cachelines with coherency managed independently) rather than having coherency operating on one 128 byte granule
AFAIK 64 bytes is still the correct granule for avoiding false sharing, with two cores modifying two consecutive cachelines having way less destructive interference than two cores modifying one cacheline.
This is not an accurate summary of what they developed.
Using .eh_frame to unwind stacks without frame pointers is not novel - it is exactly what it is for and perf has had an implementation doing it since ~2010. The problem is the kernel support for this was repeatedly rejected so the kernel samples kilobytes of stack and then userspace does the unwind
What they developed is an implementation of unwinding from an eBPF program running in the kernel using data from eh_frame.