HS5 ([website](https://hs5.eu) | [GitHub](https://github.com/uroni/hs5)) is a high-performance scale-up self-hosted S3 compatible object storage server. It is ideal for use cases where single-node scaling, performance, and data-loss risk are acceptable, or as a building block for more robust systems. For example use cases might be local testing environments or temporary storage for data processing workflows.
HS5 features a web interface for managing buckets and users.
License: LGPLv3+
I was previously using MinIO for my single node object storage needs, but it did not scale well with many objects since it creates multiple files for each object. Now it does not aim to solve the single-node use case anymore anyway and is unmaintained as well.
Some technical details: It is written in C++ since I already had an implementation of the main object storage from another project ( SingleFileStorage.cpp ). For the main object to disk location mapping it uses LMDB (a copy-on-write memory mapped database). Once the database memory is mapped, the mapping operations run at in-memory database speeds. The copy-on-write makes writing (small) changes slower. To mitigate this there is an optional WAL mode, where changes are committed to a WAL, then asynchronously committed to the LMDB. Alternatively there is a manual commit mode where commit is done via writing to a special object. Using this might be interesting for bulk-loading or for testing use cases.
0 コメント
—
Show HN: HS5 – Open Source fast single-node S3 compatible object storage · HackerTrans
HS5 features a web interface for managing buckets and users.
License: LGPLv3+
I was previously using MinIO for my single node object storage needs, but it did not scale well with many objects since it creates multiple files for each object. Now it does not aim to solve the single-node use case anymore anyway and is unmaintained as well.
Some technical details: It is written in C++ since I already had an implementation of the main object storage from another project ( SingleFileStorage.cpp ). For the main object to disk location mapping it uses LMDB (a copy-on-write memory mapped database). Once the database memory is mapped, the mapping operations run at in-memory database speeds. The copy-on-write makes writing (small) changes slower. To mitigate this there is an optional WAL mode, where changes are committed to a WAL, then asynchronously committed to the LMDB. Alternatively there is a manual commit mode where commit is done via writing to a special object. Using this might be interesting for bulk-loading or for testing use cases.