Show HN: IceGate – Observability data lake engine(github.com)
github.com
Show HN: IceGate – Observability data lake engine
https://github.com/icegatetech/icegate
6 comments
Why it matters:
→ 3x faster than nearest competitors. Rust + Apache DataFusion + Arrow do the heavy lifting.
→ Exactly-once delivery with WAL. No data loss, no duplicates - guaranteed.
→ Your data stays in open formats you control. No vendor lock-in. Ever.
→ Query logs through Loki-compatible API - works with your existing Grafana setup.
→ Ingest via standard OpenTelemetry (gRPC/HTTP). No proprietary agents.
→ Your data is in Iceberg tables - query it with Trino, Spark, Flink, DuckDB, or any tool in the Iceberg ecosystem.
→ Your observability data becomes a first-class asset of your data lake alongside business analytics.
→ Scale ingest, query, and storage independently. Full compute-storage separation provides stateless deployment.
→ 3x faster than nearest competitors. Rust + Apache DataFusion + Arrow do the heavy lifting.
→ Exactly-once delivery with WAL. No data loss, no duplicates - guaranteed.
→ Your data stays in open formats you control. No vendor lock-in. Ever.
→ Query logs through Loki-compatible API - works with your existing Grafana setup.
→ Ingest via standard OpenTelemetry (gRPC/HTTP). No proprietary agents.
→ Your data is in Iceberg tables - query it with Trino, Spark, Flink, DuckDB, or any tool in the Iceberg ecosystem.
→ Your observability data becomes a first-class asset of your data lake alongside business analytics.
→ Scale ingest, query, and storage independently. Full compute-storage separation provides stateless deployment.
> 3x faster than nearest competitors
Wow, that's awesome! Do you have any public benchmarks?
Wow, that's awesome! Do you have any public benchmarks?
There will be a series of articles on Linkedin https://www.linkedin.com/company/icegatetech in the near future, where we will tell you more about the technical side and show benchmarks.
How do you achieve fault tolerant?
Since the state is not stored in compute, fault tolerance is more likely to be transferred to S3. For compute fault tolerance (ingest + query), it is better to place 3 nodes in different AZ or different regions.
Observability costs are out of control. Teams pay per GB ingested, get locked into proprietary formats, and can't use their own data outside the vendor's UI. We built the first observability data lake engine to fix this.
This is our first release under Apache 2.0. We're building in the open and looking for contributors and early adopters who want observability without the markup.
Star us on GitHub: https://github.com/icegatetech/icegate
More info on https://icegate.tech
Documentation: https://docs.icegate.tech
LinkedIn: https://www.linkedin.com/company/icegatetech/