Eh, I've got 200mb/s fiber for cheap. It's pretty good and definitely bottlenecked by crowded wifi and upstream sources moreso than the ISP. Ethernet helps somewhat.
At the same time, I do kind of want more bandwidth just so I can download massive files like model weights quickly, host a web service out of my own house, seed torrents, etc. What might cryptocurrency look like if typical residential internet speeds were measured in gb/s? Perhaps bitcoin might be capable of more than 7 tps!
Probably not that many lives, maybe like a handful of hikers every year I would guess? I think what attracts hikers in the first place is the danger, and the idea that they're exploring an area that is "outside of civilization"
I feel like I already have internet access pretty much everywhere with cell towers, and even then if I went to the middle of alaska or montana I could already get sattelite internet before starlink with hugesnet which is fine as long as you're not gaming or something.
But at the same time I think the low-earth-orbit is pretty nice in terms of latentcy, it's a pretty innovative approach.
I just don't get the idea behind AI datacenter sattelites and moving all this non-comms equipment up in space.
Do ISPs have to comply with the 1st ammendment? My understanding was that they have some sort of common carrier law but net neutrality did not hold up.
I have epyc 9654 ES and a 7900 XTX. I was running the numbers, and even if I maxxed out the ram to like 12x32 gig sticks, it would cost me thousands more and I could only run GLM-5.2 at a couple tokens per second at q3. So this project is very promising because it suggests I could get pretty high speed and this CPU/motherboard combination suggests I have a lot of pci bandwidth that is unused.
I think another route might be looking at holding an even larger chunk of model weights in ram, and taking advantage of RAM<->GPU bandwidth, perhaps using a PCIe 5 GPU. This was my first thought since I have dedicated GPU.
If you are using Laptop, you're looking at shared memory between the iGPU and CPU. I've also tried that route, but I have always been skeptical of killing flash with too many reads, it essentially uses SSD like it's a consumable item.
I'm going to benchmark this right now with what I have and I'll get back to you on github.
I am curious if it's possible to adjust this to use more RAM, as i've got a machine with 64GB RAM and 24GB VRAM. Or perhaps I could run Gemma/Qwen on the GPU and have GLM-5.2 delegate smaller tasks to it. It might take some retraining of GLM-5.2
I'm also curious if you can speed this up by using many disks in parallel to increase bandwidth.
>SSD Wear Warning
> Cold starts are heavy on random reads (~11 GB/token). Reads themselves are safe, but the OS page cache can generate writes. Heavy use may accelerate wear on cheaper SSDs. Use with caution and monitor your drive health.
Hmm, maybe a safe way to do this would be to make a separate partition for the model weights, and set them to read-only?
Not sure how the page cache works, if it's like per partition or per disk. If it's per disk, maybe you could have a read-only data.iso formatted as a partition and mount it as a disk?
I would say that they are snake oil because of that. Data breaches occur more often than rootkits because most developers see that this path adding easily-removable encryption does nothing in the long run.
>Why would China care about deflating the US AI bubble?
To weaken the stature of the USA on the global stage relative to themselves. Perhaps decrease US investment in AI and slow creation of some general AI superweapon I suppose.
Because the goal is to show that cheap chinese AI can compete with expensive USA AI, it's nessisarially a low-cost attack relative to the "damage" it could create.
>Why do we think there is a bubble for sure in 2025/2026?
Well that's the position that these chinese firms are trying to convince us of, and they can convince us by undercutting proprietary models in price/performance/openness.
In other words, we can be sure there is a bubble to the extent that open-weight models can successfully demonstrate that there is no moat.
>Why doesn't China also worry about their own AI bubble inside the country?
Because they haven't bet the farm on AI like the USA has.
At the same time, I do kind of want more bandwidth just so I can download massive files like model weights quickly, host a web service out of my own house, seed torrents, etc. What might cryptocurrency look like if typical residential internet speeds were measured in gb/s? Perhaps bitcoin might be capable of more than 7 tps!
But to be fair, I am a nobody.