Not sure, there was a lot of complaints when it rolled out because it was actually peer to peer syncing to other laptops which was causing bandwidth issues for people working on vpn, from planes etc.
I’m not sure if it was actually a well thought out idea, since it’s not like fb has a lack of servers or bandwidth, but I didn’t work on it, so I don’t want to throw too much shade at it.
Hister sounds like something I wanted for a while, but never got around to building. Searching stuff I’ve seen before is most of what I do with a search engine, so having it local and fast would be amazing. Eager to give it a try.
At meta, there was a project for delivering binaries of internally built libraries / binaries to dev laptops using a private ipfs network. This was live for at least some period of time.
Yeah, I’m doing TP with two cards. The topology is configured based on yaml files, and if you are not using a predefined config you can just create a new config with your topology.
I’m not even using a 800G cable since they are expensive and I don’t think I need the bandwidth, opting for 400G instead. This just needs a config change for the number of Ethernet links it uses internally. (Apparently these cables are just many 200G links put together.)
I think it’s not really worth it compared to just buying tokens or a coding plan.
My setup has 4090 handling attention while TT accelerators handles MLP. With just a 4090 you can have CPU handle the MLP layers and use a MoE model, assuming sufficiently powerful cpu. I tried that setup with minimax 2.5 before, and was able to eke out around 10 to 15 tps (albeit with a 7965wx cpu)
The software stack is pretty immature, definitely very DIY. Their officially supported models are pretty old at this point, though there’s community support for gemma4, and models with GDN like qwen3.6 is supposedly very close.
The entire stack (minus some binary blobs in firmware) is open source, so if you have the time and persistence you can get whatever you want done.
A few community members have been working on support with llamacpp, where we can have supported operations offloaded to the TT cards, while having unsupported ops running on GPU or CPU. Llamacpp is pretty good at that. The existing kernels could definitely be better, and I’ll try my hand at writing some kernels some time.
80tp/s with 5080 3090 combo is wild. I’ve been working with a 4090 and two Tenstorrent p150 cards, and manage only about 30 tps utilizing all three for qwen3.6 27b q8. Guess I got more optimization to do.
Would like to see the perf of their setup with and without mtp and ngram speculative decoding though, as well as parallel decode performance (once llamacpp mtp plays well with multiple slots).
Being in California electricity alone puts this non-competitive with just paying a cloud though.
The thing I always wondered regarding obsidian plugins is how they are able to offer them on iOS, given that iOS has rules against downloading code that alters functionality of the software.
Not necessarily. Servers serving the model likely has enough traffic that they are batching decodes already. MTP reduces latency and increase efficiency only when the server can’t batch enough concurrent streams to be compute bound rather than memory bound.
Funny, when I got tired of trying to find a nice desktop background I just started using a solid color of muted blue or green. I never read about this specific usage of colors before but I bet I saw something somewhere that clued me in on this color.
I tried fedora silverblue for a while, but the way it works is that it builds a new root fs image whenever you change the installed packages, this makes system package changes take comparatively long vs a traditional os. They suggest installing most apps via flatpak, which is okay as long as you can deal with flatpak idiosyncrasies.
I also tried fedora coreos for a vm + container host, but found the recommended method to configure the system with ignition files and one shot systemd units to be too involved for making a one off system, and it’s probably better for a cloud deployment with many identical nodes.
I think it’s not so much that the asyncio primitives got wrong about shared state, as much as is what the authors got wrong about the usage of those primitives. They are classic concurrency primitives that’s been around for almost half a century. They work as designed, but require some care to use correctly.
I noticed requests that were exploiting the vulnerability were turning into timeouts pretty much immediately after rolling out the patch. I’m surprised it took so long for it to be announced.