HackerTrans
TopNewTrendsCommentsPastAskShowJobs

phirephly

no profile record

Submissions

Serving 90TB/Day of Linux Updates from Thin Clients

blog.thelifeofkenneth.com
106 points·by phirephly·3 lata temu·6 comments

comments

phirephly
·2 lata temu·discuss
If every torrent includes a webseed, then you're still left with the problem of needing to build a full HTTP CDN, and now have to also maintain the largest tracker infrastructure ever deployed for Bittorrent.

Under normal conditions with well behaved clients, raw bandwidth for the large packages is essentially never the issue. Misbehaving clients, cache thrashing, IOPS are the sort of issues that cause pain for mirrors.
phirephly
·2 lata temu·discuss
I've made a point of calling out Digital Ocean in Linux mirroring talks as the gold standard for being a good citizen; run their own internal mirrors, which are FAST, making it a value add feature for them as well.
phirephly
·2 lata temu·discuss
EPEL is a separate module from fedora-enchilada, but uses the same backend CDN infrastructure and most mirrors tend to carry both /fedora/ and /epel/, so they're not technically the same mirrors, but most mirrors tend to carry both.
phirephly
·3 lata temu·discuss
The map is a latency heat map. I should probably replace it with a POP map
phirephly
·3 lata temu·discuss
Sure. And switch ASICs hang off a CPU as a PCIe device, so the line between smartNICs and low end ASICs start to blur. The key is hardware acceleration of some sort.
phirephly
·3 lata temu·discuss
256 next hops isn't enough. Typical ASICs support 20,000 to 160,000 next hops FECs.

Cisco tried caching routing decisions from non-line rate routing engines in the 90s, and the industry learned the lesson that it's a bad idea. Caching works until you overflow the cache for some reason, and then the box completely falls over as it thrashes.
phirephly
·3 lata temu·discuss
The base power with any colo space is going to be minimal. You typically come back and spec out what additional power you want with the rack a la carte.
phirephly
·3 lata temu·discuss
When you want line rate forwarding across several Tbps of front panel ports, you need the packet pipeline to be able to make all the routing decisions without involvement from the OS. 8Bpps just doesn't give you time to be able to walk any kind of data structure in memory.

Running full internet tables on a x86 server where you can only get a few Gbps up to maybe a few dozen Gbps is much easier.
phirephly
·3 lata temu·discuss
I started an Internet Exchange Point adjacent to it, use it to host mirror.fcix.net, and got a second ASN to build the anycast ns-global.zone service
phirephly
·3 lata temu·discuss
I only got a few years out of it. I'm running an Arista 7280SR-48C8 now.

The problem is that the million tcam entries are split between IPv4 and IPv6, so I really ran out of space.
phirephly
·3 lata temu·discuss
This article is also essentially available as a podcast. https://oxide.computer/podcasts/on-the-metal/kenneth-finnega...
phirephly
·4 lata temu·discuss
A good old fashioned slash dotting on a self hosted server. Chill dude.