Goal: make durability demos less “trust me”. Click “Run double proof” → it produces N, docker kill -s KILL, restarts, consumes/commits, then proves “no loss + no dupes” while exposing the entire trail (proof JSON + docker inspect/events/log tail).
To clarify: I’m looking for 2–3 teams who already run Kafka/Redpanda/NATS
and are open to testing an alternative focused on simpler ops and lower tail latency.
Not selling anything, just early feedback and real workloads.
Thanks, this is genuinely helpful framing.
We made the mistake of thinking in “roles” (GTM cofounder) instead of pulling people into the problem and watching for ownership.
We’re now doing short problem interviews with early users / people who engaged deeply with our Show HN, and tracking who (1) reframes the problem, (2) proposes concrete next steps, and (3) follows up unprompted. Those are strong signals.
Also +1 on the “technical storyteller” point our ideal partner might be technical but customer-obsessed rather than a traditional sales profile.
One question: when you’ve seen this work best, what’s a good lightweight way to test alignment/commitment before talking equity (e.g., a 2-4 week project sprint, shared doc, pre-defined milestones)?
That makes a lot of sense the hard part isn’t “HTTP paging”, it’s defining a safe cursor (in SQL that becomes “which column is actually stable/monotonic”), and without an external spec/libs it turns into bikeshedding. In Ayder the cursor is an explicit per-partition log offset, so resumability/paging is inherent, which is why Feed API’s mental model resonates a lot. I’d love to see a minimal “event log profile” of that spec someday.
Classic HTTP Range is byte-oriented, but custom range units (e.g. `Range: offsets=…`) or using `Link` headers for pagination both fit log semantics well.
I kept the initial API explicit (`offset` / `limit`) to stay obvious for curl users, but offset-range via headers is something I want to experiment with, especially if it helps generic tooling.
I’d love a world where “consume an event log” is a standard protocol and client-side tooling doesn’t care which broker is behind it.
Feed API is very close to the mental model I’d want: stable offsets, paging, resumability, and explicit semantics over HTTP. Ayder’s current wedge is keeping the surface area minimal and obvious (curl-first), but long-term I’d much rather converge toward a shared model than invent yet another bespoke API.
If you’re open to it, I’d be very curious what parts of Feed API were hardest to standardize in practice and where you felt the tradeoffs landed in real systems.
Totally fair, if this were “single-node HTTP handler on localhost”, then yeah, you can hit big numbers quickly in many stacks.
The point of these numbers is the envelope: 3-node consensus (Raft), real network (not loopback), and sync-majority writes (ACK after 2/3 replicas) plus the crash/recovery semantics (SIGKILL → restart → offsets/data still there).
If you have a quick Python setup that does majority-acked replication + fast crash recovery with similar measurements, I’d honestly love to compare apples-to-apples happy to share exact scripts/config and run the same test conditions.
That’s a great comparison nsq is a project I have a lot of respect for.
I think there’s a similar philosophy around simplicity and operator experience.
Where Ayder diverges is in durability and recovery semantics nsq intentionally
trades some of that off to stay lightweight.
The goal here is to keep the “easy to run” feeling, but with stronger guarantees
around crash recovery and replication.
Yes I’ve read through TigerBeetle’s VSR design and their rationale for not using Raft.
VSR makes a lot of sense for their problem space: fixed schema, deterministic state machine,
and a very tight control over replication + execution order.
Ayder has a different set of constraints:
- append-only logs with streaming semantics
- dynamic topics / partitions
- external clients producing arbitrary payloads over HTTP
Raft here is a pragmatic choice: it’s well understood, easier to reason about for operators,
and fits the “easy to try, easy to operate” goal of the system.
That said, I think VSR is a great example of what’s possible when you fully own the problem
and can specialize aggressively. Definitely a project I’ve learned from.
The demo intentionally starts with SIGKILL to show crash recovery first.
For benchmarks: I used real network (not loopback) and sync-majority writes in a 3-node Raft cluster.
Happy to answer questions about tradeoffs vs Kafka / Redis Streams and what’s still missing.
This is gold, thank you. The “easy to reason about” part is exactly what I’m going for.
A couple quick questions if you don’t mind:
Roughly what volume are you pushing per device (MB/day or events/sec), and what’s your typical offline window?
What’s your biggest failure mode today: disk-full/rotate policy, encryption key handling, replay storms on reconnect, or Lambda fanout/cost?
I’m thinking Ayder could replace the “rotate → ship” backend with a durable local log + priority queues + replay, but you’re right that the hardest part is the policy (what to drop first, how to bound disk, and how to preserve critical streams). If you’re open, I’d love to learn what heuristics you ended up with.
Disclosure: I built an OSS single-binary, HTTP-native durable event log aimed at this edge “store-and-forward + replay” problem. Repo: github.com/A1darbek/ayder
If anyone is open to a tiny design-partner pilot (30–60 min): run docker compose → ingest some telemetry → simulate outage (kill -9 / disconnect) → restart → verify replay + zero loss. I’ll do white-glove onboarding and turn the learnings into a short case study (can be anonymous).
Author here. Happy to answer questions about the implementation.
A few details that might interest HN:
- Raft replication handles 50K msg/s with sync-majority writes
- HTTP parsing is 0.4ms at P99.999 (picohttpparser, zero-copy)
- Tested durability with SIGKILL — recovery replays sealed AOF
What would you want from an HTTP-native event log?