usually the LLM will match against the style in the context, even if the words ask it to do otherwise. since your skill sounds like an LLM wrote it, I would be surprised if it didn't make things worse.
sorta like asking an LLM to get more creative with a visual design, that also never works.
interesting. that might be a fun intro project to MoltenVK. I hadn't dug into what was missing for Doom. I thought maybe the issue was that the intro/menu always ran in opengl mode or something. If it's just one missing op, that's way easier.
it is a separate stack, but that probably doesn't matter much. a user process (in my case, qemu) can communicate with a driverkit driver. the user process can also map memory through the driver, which is how this pci passthrough system works.
i don't think the issues with the project really are specific to driverkit.
i don't know for sure, but i suspect what makes the tinygrad stuff slow isn't the macos host driver itself. i think they're doing something very similar to what i'm doing, which is just mapping the PCI BARs to userspace, then they have a bunch of python code that drives the GPU.
this is only speculation, but i think the big thing that makes tinygrad slow is that the tinygrad inference engine has not really been optimized much for all these open LLM models. probably most of the work has gone towards optimizing the stack for george's self-driving hardware company. since you can't just run the existing CUDA kernels on their engine, that makes things a lot tougher, engineering-wise.
i am actually curious if my project could share a macos host driver with them. i think it would need some changes, but it seems like there's a lot of overlap
in fairness to the LLM critics, every time i ran into a minor speed bump in this project, it told me it probably just wasn't possible to get it to work well. the LLM did pretty actively discourage me from trying to get the whole thing working.
that said, since i was willing to ignore that aspect of it, it did accelerate getting the work done by a lot. it seems like it understands system programming really well, and did a good job navigating the qemu codebase. i have ~20 years of systems programming experience so i already knew what had to be done here. it didn't really guide the project much, but it did write a lot of the code.
1. Virtualization.framework seems to support some form of GPU passthrough from the host (granted, not eGPU - it's for the integrated GPU). I think the primary use case is having macOS guests get acceleration, while still sharing GPU time with the host. There is also a patch that recently hit QEMU mainline that supports using the "venus server" with virtio-gpu to support a similar functionality for Linux guests under Hypervisor.framework.
2. Apple internally has some kind of PCI Passthrough support available in Virtualization.framework. It seems like the code is shipped to customers in the framework, but it relies on some kind of kext or kernel component that isn't shipped in retail macOS. I can't say if that's intended to ever be released to customers, but clearly someone at Apple has thought about this the feature.
I very recently ran the numbers on these GPUs for an upcoming blog post. The token generation performance is bad, but the prefill performance is _really_ bad.
For a Qwen 3.6 35B / 3B MoE, 4-bit quant:
- parsing a 4k prompt on a M4 Macbook Air takes 17 seconds before generating a single token.
- on an M4 Max Mac Studio it's faster at 2.3 seconds
- on an RTX 5090, it's 142ms.
RTX 5090 uses more power than an M4 Max Mac Studio but it's not 16x more power.
i thought my post was already too long to include this, but to your point, you can run AI inference in this setup and the performance can be pretty good.
this is mostly right, but it's not true that you can just sign up at any time. there's an open enrollment period for the aca marketplace and if you miss it, you won't have the opportunity to buy health insurance until next year.
i have no doubt that other countries have some problems in their healthcare systems too, but i think you are downplaying a few key points:
1) united healthcare made 90 billion dollars gross profit in the last 12mo, and that's only one health insurance company. claiming that it's not a great business at a 2-3% profit margin ignores the scale of money involved, and ignores that the customer for health insurance is truly captive.
2) you're right that america has very high prices in healthcare. doesn't it seem bad that private insurance companies are incentivized to make things cost as much as possible so they can skim that 2-3% off the top? insurance companies negotiate and set prices for services and pharmaceuticals. they now own the pharmacy benefit management companies that would normally be incentivized to negotiate for lower prices.
i would expect in a public health care system that rejects procedures, they would follow consistent guidelines and rules. american health insurance companies will arbitrarily reject a percentage of procedures that they know they should be accepting in order to keep their profit margin in the right range.
i think it's hard for me to see the argument that health insurance companies are a net-positive or even net-neutral party in the united states. i don't think it's a coincidence that we have some of the highest prices and some of the worst outcomes.
sorta like asking an LLM to get more creative with a visual design, that also never works.