HackerTrans
TopNewTrendsCommentsPastAskShowJobs

suprasam

no profile record

Submissions

ZFS, not FUSE: 13 GB/s from $0.02/GB GCS object storage on a single VM

zettalane.com
4 points·by suprasam·vor 4 Monaten·0 comments

Native ZFS VDEV for Object Storage (OpenZFS Summit)

zettalane.com
127 points·by suprasam·vor 6 Monaten·31 comments

comments

suprasam
·vor 6 Monaten·discuss
ZeroFS doesn't exploit ZFS strengths with no native ZFS support, just an afterthought with NBD + SlateDB LSM Good for small burst workloads where everything kept it in memory for LSM batch writes. Once compaction hits all bets off with performance and not sure about crash consistency since it is playing with fire. ZFS special vdev + ZIL on ssd is much safer. No need for LSM. MayaNAS ZFS metadata at SSD speed and large blocks get throughput from high latency S3 at network speed.
suprasam
·vor 6 Monaten·discuss
Yes that is the value prop. Cheap S3 instead of expensive EBS.

  EBS limitations:
  - Per-instance throughput caps
  - Pay for full provisioned capacity whether filled or not

 S3:
  - Pay only for what you store
  - No per-instance bandwidth limits as long as you have network optimized instance
suprasam
·vor 6 Monaten·discuss
For RDBMS pages on object storage - you might be thinking of Neon.tech. They built a custom page server for PostgreSQL that stores pages directly on S3.
suprasam
·vor 6 Monaten·discuss
I hope you are not having that massive storage storage on public-cloud then you would need MayaNAS to reduce storage costs. For S3 as frontend use MinIO gateway - serves S3 API from your ZFS filesystem
suprasam
·vor 6 Monaten·discuss
It is all part of ZFS architecture with two tiers: - Special vdev (SSD): All metadata + small blocks (configurable threshold, typically <128KB) - Object storage: Bulk data only If the workload is randomized 4K small data blocks - that's SSD latency, not S3 latency.
suprasam
·vor 6 Monaten·discuss
Yes, this is a core use case ZFS fits nicely. See slide 31 "Multi-Cloud Data Orchestration" in the talk.

Not only backup but also DR site recovery.

  The workflow:

  1. Server A (production): zpool on local NVMe/SSD/HD
  2. Server B (same data center): another zpool backed by objbacker.io → remote object storage (Wasabi, S3, GCS)
  3. zfs send from A to B - data lands in object storage

  Key advantage: no continuously running cloud VM. You're just paying for object storage (cheap) not compute (expensive). Server B is in your own data center - it can be a VM too.
For DR, when you need the data in cloud:

  - Spin up a MayaNAS VM only when needed
  - Import the objbacker-backed pool - data is already there
  - Use it, then shut down the VM