coldstartops

позавчераdiscuss

I think OPs mantra is "Scrape. Automate. Scale." It feels like people who consent to this "passive" income, do not understand what they consent to. The business model seems grey towards black. "Signup to be part of 3rd party bot farms exit node, you get paid, all will be good. Nothing bad happens, you consent"

coldstartops

3 дня назадdiscuss

I am curious if I got it right, but the business of the company is providing exit nodes for AI bots/ strangers via peoples residential IP's? This is my impression from accessing the website. I was interested at first, but now I got a bit of ethical doubts about it.

coldstartops

5 дней назадdiscuss

Got a Bachelor + Master, best investment in my life (in terms of time, as I did not pay for education, EU and such).

Job stuff not much related with stuff learned in school, sometimes related to some degree.

Skill stuff very closely tied to constantly applying stuff learned in school, but mostly on how to attack problems.

Me learn stuff because me like know stuff and level up. (edit: do not get me wrong, learning is still painful, and thinking even more painful)

Me job stuff because me like food and amneties, until the learn stuff transfers into sustainable food and amneties.

coldstartops

7 дней назадdiscuss

I did not measure agents, but lets take the following example:

Assume you have a 10 GB blu ray movie that you want to watch without waiting for upload/ donwload time.

Maybe you host it on a server on another continent, where you got a 200ms RTT.

And your wire at home supports around 500Mbps - approx 62 MB/s.

Thus to Download it fully, would take around: 161 seconds if you use the full wire and on the happy case.

But to actually watch it and skim through it, you do not need to use the full wire, but around ~11 Mbps for a 2-hour 10 GB film, which is like 2% of the wire. The only hard part is random jumps, as each seek is a full RTT.

If you prefetch too aggresivelly you hide this RTT, but if the wire is saturated, and if you have a cache miss, then you will hang and wait until the request goes through. Might be upt to 1-2 seconds. The whole game is to pace yourself such that you stay ahead of the consumption rate, without saturating the wire.

coldstartops

7 дней назадdiscuss

I just had a talk (power-rushed a bit) yesterday at Pass The SALT 2026 talking about (my version of) latency hiding filesystems in userspace over the network. My benchmarks, setups, and how to make the magic wawaweewa work.

Basically it went the "sshfs over iroh" approach, and instead of OP's post, it has eager metadata file sharing, and on demand file access. It is still a work on progress, but you can use git juse fine over the fuse mount points without introducing a new tool (edit: but do note that I did not encounter problems with large worktrees, as I do not use them in my flow, and still has some quirks). And still have similar "cold path" access times.

I touch on some insights of hiding the latency, the direct Alice to Bob connection, and has some recorded demos. Goes the generic approach.

You can watch it here if you are interested, but its around 25 minutes.

The first part is about the Post Quantum Crypto, and making direct IPv6 connections work.

The second part is about the filesystems in userspace that hide the latency. (The second part is around minute 12:00, and I think it is the relevant part to this discussion/ thread).

https://passthesalt.ubicast.tv/videos/2026-keibidrop-post-qu...

coldstartops

16 дней назадdiscuss

I see nothing wrong with something probabilistic. I think it is all about offsetting the risk and reducing the odds of bad outcomes. There is this concept of Defence in Depth, thus I assume some sort of binomial formula also applies here.

coldstartops

18 дней назадdiscuss

Couldn't run your benchmarks, as I did not create an account, and a bit of a different beast that I am comparing against (P2P distributed filesystem), but these are my numbers and setup, and the on-demand part lines up with what I have observed:

Setup: a Linux box on the other side of Romania (compared to where I am living) reading from a Windows box in Singapore (~200 ms RTT)

- reading 1 MiB of a 1 GB remote file pulls only 16 MiB (~98% avoided) - this is because of my fine tuning optimization choice - first byte approx: 2.3s - git-LFS repos also clone cold over the mount byte-perfect (separate Mac - Linux run on a ~20 ms RTT)

The thing that I do differently is that my metadata is eagerly pushed, as I optimized for content streaming.

And 100k-file tree mounts I did not test yet.

But my goal was to have instant file access for generic files between apps, and peer to peer, supports also Windows :D

here is the tool: https://github.com/KeibiSoft/KeibiDrop

coldstartops

18 дней назадdiscuss

I am curious, how do you handle latency issues for on demand access? I saw you use FUSE (and FSKit), and from my experience it is pain to make filesystems in userspace work on-demand over WAN because a) latency, and variable RTT; and b) you can't saturate the wire and aggressively read ahead things, otherwise native apps will freeze, lag, or just make the UX unpleasant, especially if there are too many placeholder files, or large files with random jumps in them.

coldstartops

20 дней назадdiscuss

Google hits 50% IPv6, very good for accessing websites.

But my TP-Link router blocks by default inbound IPv6 connections, without any option to configure it, still bad for pure IPv6 bidirectional streaming, gaming or services on home networks.

coldstartops

26 дней назадdiscuss

KEIBIDROP: Peer to Peer Instant file sharing

https://keibidrop.com

https://github.com/KeibiSoft/KeibiDrop/

It leverages virtual mount points to make file metadata available between Alice and Bob, and make the files appear as if locally.

So for example instead of waiting to upload then download a Blu Ray movie, or a large game, you can just add it into the virtual mountpoint and your recipient can instantly play it, or install it, and the download happens in the background with random offset jumps, and caches it locally.

coldstartops

в прошлом месяцеdiscuss

Highly recommend. The exampels are in c/c++ and the same concepts can be ported to other languages like golang.

My favorite part of it is Chapter 2 the bit manipulation tricks.

coldstartops

2 месяца назадdiscuss

Nice! Now the question is: How many classic Reynolds boids can you run on 1 CPU at 60FPS, without using any go routines?

I managed to get around 8192 using Serge Skoredin's approach from the blog post last year: https://skoredin.pro/blog/golang/cpu-cache-friendly-go

Also tried some of the techniques in this blog post, and managed to squeeze a bit more with the insights from your post.

coldstartops

2 месяца назадdiscuss

Let me know how it goes, or frustrations. I admit that currently it is not the most friendly UX. And found out a bug that if no direct P2P possible, it fails to fallback to the STUN-ish data relay. However LAN mode and WAN via direct P2P works on latest releases.

As soon as I merge this one, will create another release that fixes the fallback; and hopefully also get the apps in ios and gplay stores. https://github.com/KeibiSoft/KeibiDrop/pull/143

coldstartops

2 месяца назадdiscuss

Stream on-demand files between your devices. WAN, LAN, Direct IPV6, cross platform, pqc.

https://keibidrop.com/ repo: https://github.com/KeibiSoft/KeibiDrop

coldstartops

2 месяца назадdiscuss

It is possible. I tell to use cli app, and for the agent to ad timer and check the status once in a while. Especially if there is something with a long wait. Also if it can run some validators/ same tools locally, would be much faster.

Might tend to deviate and waste time, needs guiding once in a while, and to check what is it spewing out, point it in the correct direction.

coldstartops

2 месяца назадdiscuss

I treat the low level tasks as building blocks. You need a grasp and understanding of what is possible with them, but you do not need to remember the exact byte order and syntax. I think the idea is you should structure your workflow in a deterministic way, and just use Claude/ LLM as the interface. It is much easier and enjoyable to use high level language, where you give pointers to building blocks/ directions/ say hard no when you understand things deviate.

If I had to output the code myself, would take around 8 hours of constant writing to get around 1k LoC of code. For FUSE level tricky stuff, I might need to spend 3 weeks for 10 LoC. Very easy to burnout and build pain.

coldstartops

2 месяца назадdiscuss

Here is a public project.

https://github.com/KeibiSoft/KeibiDrop

It took me 2 years ago around 2k hours to build a cross platform FUSE vault, without using AI assisted tools.

The pain was debugging through logs and system traces. And understanding how things work.

Now managed to ship this one much faster, as an after hours project. Started it in may 2025, and around end of November 2025 started using claude on it.

Just by dumping logs into claude, and explaining the attack vector for the problems, saved me the FML moments of grindings walls of syscalls on 3 platforms.

I would say much easier to progress, and ship with the same rigour, minimize my time, focus and brain power involvement such that I can put the energy somewhere else.

coldstartops

2 месяца назадdiscuss

I've been through Introduction to Modern Cryptography by Katz and Lindell. Can recommend, as it starts with Caesars cipher, one time pads, and builds towards modern cryptography.

coldstartops

2 месяца назадdiscuss

I will add this on the roadmap. At the moment I am trying to add persistent identity, as by default it is ephemeral.

But I warn you, there are still some kinks. Only 75% pjdfstest (filesystem test suite) passes in terms of POSIX compliance. I do not support hardlinks/symlinks by design.

I was trying last to use postgresql on the virtual filesystem, then once the server is stoped, to remove the lock file, and open it on the peer. and some files got corrupted while running this test.

coldstartops

2 месяца назадdiscuss

I tried to explain here the differences, and where it fits in the gap of file transfer tools, also on the blog got some benchmarks and how they where setup.

https://keibisoft.com/tools/keibidrop-vs-alternatives.html

its more towards vibey marketing talk and at the same time tried to keep it as honest as possible.

comments