HackerLangs
トップ新着トレンドコメント過去質問紹介求人

Eikon

no profile record

投稿

Show HN: ZeroFS – A log-structured filesystem for S3

zerofs.net
128 ポイント·投稿者 Eikon·9 日前·52 コメント

Making POSIX filesystems replicated and highly available

zerofs.net
3 ポイント·投稿者 Eikon·12 日前·0 コメント

Show HN: ZeroFS – Make S3 your primary storage

zerofs.net
11 ポイント·投稿者 Eikon·29 日前·7 コメント

Show HN: PostgreSQL running in the browser, persisting to S3

zerofs.net
1 ポイント·投稿者 Eikon·3 か月前·1 コメント

Show HN: PostgreSQL running in a browser tab, persisting to S3

zerofs.net
2 ポイント·投稿者 Eikon·3 か月前·0 コメント

Static Certificate Transparency

merklemap.com
16 ポイント·投稿者 Eikon·7 か月前·1 コメント

Show HN: ZeroFS, the Filesystem That Makes S3 Your Primary Storage

github.com
64 ポイント·投稿者 Eikon·10 か月前·40 コメント

コメント

Eikon
·8 日前·議論
Kind-of should work :)
Eikon
·9 日前·議論
NFSv4 is a hard beast to implement correctly, with a lot of protocol surface (state, compound ops, delegations) for benefits ZeroFS mostly gets through 9P with extensions, over a much simpler protocol: https://www.zerofs.net/docs/9p-extensions. NFSv3 stayed in ZeroFS mostly for client compatibility.
Eikon
·9 日前·議論
ZeroFS aims to be a POSIX filesystem, the semantics here are the standard ones (ext4, xfs behave the same): write() is buffered (that's the batching) and "committed" maps to fsync(), which returns only once data is durable.
Eikon
·9 日前·議論
Thank you, appreciated!
Eikon
·9 日前·議論
> I believe the first version of this required the metadata to be stored on the ZeroFS server, making HA kinda hard.

Metadata has always been in the bucket itself.

For HA, there's now a "replicated mode" if you want automatic failover:

https://www.zerofs.net/docs/high-availability
Eikon
·9 日前·議論
That's fair advice, thanks.
Eikon
·9 日前·議論
[dead]
Eikon
·9 日前·議論
[dead]
Eikon
·9 日前·議論
Is it? :)

Ask me anything!
Eikon
·9 日前·議論
Thank you for the feedback, the idea behind this was to say "We make claims that are backed by workflows you can verify". I'll improve the phrasing.
Eikon
·9 日前·議論
Just a self hosted plausible instance :)
Eikon
·16 日前·議論
It's there! https://www.zerofs.net/docs/kubernetes-csi
Eikon
·16 日前·議論
Most of the pain here is the typical set of issues people run into trying to make S3 a filesystem as-is, common with S3FS-family approaches.

ZeroFS (https://github.com/Barre/zerofs) is 9P/NFS/NBD over S3 on an LSM. Point stock go-git, or just /usr/bin/git, at a mount and skip the gymnastics. Rename is a metadata op in the keyspace, so you get it atomic on any S3, no Tigris-specific X-Tigris-Rename needed.

Different point on the spectrum, but less square-peg, also most probably much, much faster (it works great on linux-sized repos) :)
Eikon
·27 日前·議論
> Is this only was S3 or any S3 compliant service?

Any S3 compliant service.

> Does it support any sort of storage tiering ?

Yes, it has a local cache that's populated on reads, but no writeback cache yet.
Eikon
·28 日前·議論
> Millions of files, Interrupted transfers, Backup Server Reboots, Long-term operation?

Yes, that should work fine!
Eikon
·28 日前·議論
> I assessed the tool on a CentOS 7 system today. It can't run on older CentOS 6. It will generate core dump.

Perhaps your cpu is too old? The prebuilt binaries require newer CPUs but you can build your own.

> Will ZeroFS support large Filesystem size about 800GB?

Absolutely, and much, much more than that.
Eikon
·先月·議論
Even with journaling, you might need one. ZeroFS [0] almost had a lost+found directory (even with the WAL enabled), because you might have consistency issues between your in-memory state and what was flushed, and especially in what order.

ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).

[0] https://github.com/Barre/ZeroFS

[1] https://github.com/Barre/ZeroFS/blob/main/zerofs/src/fs/writ...
Eikon
·2 か月前·議論
https://github.com/Barre/ZeroFS Should do a great job at this.
Eikon
·3 か月前·議論
It's definitely shared, and can be redundant.
Eikon
·3 か月前·議論
Or ZeroFS which doesn’t require a 3rd party database, just a s3 bucket!

https://github.com/Barre/ZeroFS