Show HN: Zedis – A Redis clone I'm writing in Zig(github.com)
github.com
Show HN: Zedis – A Redis clone I'm writing in Zig
https://github.com/barddoo/zedis
Writing Redis from scratch in Zig.
115 comments
Sorry if somewhat off-topic, but might help with getting more users. Redis is actually good enough and easy to install for most projects, but it has downside that it's "memory-only" (needs to fit in ram), and if you're in a tight container/vm – you'd better not rely on it solely.
So, there's another project – called kvrocks https://github.com/apache/kvrocks , which is for people which don't need in-memory perf of redis but would like to use its protocol and be still very performant. However, its devs never packaged a release in deb and other formats.
So, if you were to implement something similar ^ which will implement the protocol, have ability to grow on disk, and be available in distro repos (ideally) – would definitely hold a niche of many little projects which benefit from redis-like thing I've done.
So, there's another project – called kvrocks https://github.com/apache/kvrocks , which is for people which don't need in-memory perf of redis but would like to use its protocol and be still very performant. However, its devs never packaged a release in deb and other formats.
So, if you were to implement something similar ^ which will implement the protocol, have ability to grow on disk, and be available in distro repos (ideally) – would definitely hold a niche of many little projects which benefit from redis-like thing I've done.
Redis is for high frequency cache, disk latencies would slow it too much. Maybe putting something that would normally be evicted to disk could be valid compromise.
another major drawbacks of Redis is that It is single threaded and sync. you need expensive high frequency CPU if you need more performance. something that could benefit from multi core system and if you would want to use disk cache async is a must. single threading also affects scripts running on Redis side, when they execute whole Redis is waiting and is unable to fulfil other requests.
another place for improvement would be proper easier to use HA, and maybe also auto scaling.
another major drawbacks of Redis is that It is single threaded and sync. you need expensive high frequency CPU if you need more performance. something that could benefit from multi core system and if you would want to use disk cache async is a must. single threading also affects scripts running on Redis side, when they execute whole Redis is waiting and is unable to fulfil other requests.
another place for improvement would be proper easier to use HA, and maybe also auto scaling.
Most projects need to start with something and know that they will "be good". Many will hit RAM limitations earlier than kvrocks latencies or single-threaded CPU limitations (if they will – usually author doesn't need advice). Most would benefit from "moving fast" (in terms of development).
Redis-compatible protocol guarantees that you can think about the things you've mentioned later. If project is successful – you'll find money to pay for RAM and move to pure Redis. If it's CPU-bound – you'll find multi-threaded alternatives.
So this is a pretty nice niche IMO actually.
Redis-compatible protocol guarantees that you can think about the things you've mentioned later. If project is successful – you'll find money to pay for RAM and move to pure Redis. If it's CPU-bound – you'll find multi-threaded alternatives.
So this is a pretty nice niche IMO actually.
Why not use a managed Redis or a large memory instance and run Redis yourself then? They aren't that much more expensive if you need to move fast.
Yes, but why pay for it if `sudo apt install zedis` works? Many use-cases for redis growing out of memory would actually be ok if it stores old values on disk for you and keeps hot items in cache (and does compression).
It’s not feasible to do that in a single threaded system.
The main drawback of disk persistence is that you need a log of memory to keep the file buffer open and pointers.
If the key you’re looking for is not on disk, you’ll get awful results, the rdb format will give you worst then Postgres performance. You have to use another file format to optimize for a larger cache
The Redis HA story is horrendous. I wonder if Sentinel is intentionally just crap enough to get your application locked in but then force you to pay them for hosted option.
It regularly gets itself into messed up state and even with graceful shutdown for node updates the official client libraries throw errors for up to a minute.
It regularly gets itself into messed up state and even with graceful shutdown for node updates the official client libraries throw errors for up to a minute.
fwiw redis has persistence https://redis.io/docs/latest/operate/oss_and_stack/managemen...
If you're in a tight VM and are running Redis inside it, as opposed to a separate VM / cluster, you need an embedded DB, not Redis.
You can use the ancient BerkeleyDB which is supported everywhere for last like 30 years. Or a more recent LevelDB. Or even SQLite which is plenty fast on simple queries.
You'll have to tolerate the high latency of safely writing tings to a block device, compared to RAM. If your load is mostly reads, it should be OK.
You can use the ancient BerkeleyDB which is supported everywhere for last like 30 years. Or a more recent LevelDB. Or even SQLite which is plenty fast on simple queries.
You'll have to tolerate the high latency of safely writing tings to a block device, compared to RAM. If your load is mostly reads, it should be OK.
Redka may be interesting to you: https://github.com/nalgeon/redka
I didn’t know that, that’s pretty cool
Redis compatible projects with larger-than-memory support do exists, e.g. https://microsoft.github.io/garnet/docs/welcome/features#tie...
> Written in Zig for optimal performance and memory safety
Ironic to see this, as even a cursory glance over the code shows a lot of memory bugs, such as dangling pointers, data races, some potential bugs like double free, etc.
It is good that the OP is learning things, but I would caution against relying on LLMs and taking on a bigger project like this, before the basics are well understood.
Personally, this is one of the reasons I dislike the LLM hype, people are enabled to produce much more code, code that they aren't qualified to support or even understand.
While the project linked is clearly designated as strictly for "learning purposes", the applications we will get in the large will be of no better quality.
The difference being, before LLMs, those, who didn't have qualification, wouldn't even approach problems like this, now they can vibecode something that works on a lucky run, but is otherwise completely inadequate.
Ironic to see this, as even a cursory glance over the code shows a lot of memory bugs, such as dangling pointers, data races, some potential bugs like double free, etc.
It is good that the OP is learning things, but I would caution against relying on LLMs and taking on a bigger project like this, before the basics are well understood.
Personally, this is one of the reasons I dislike the LLM hype, people are enabled to produce much more code, code that they aren't qualified to support or even understand.
While the project linked is clearly designated as strictly for "learning purposes", the applications we will get in the large will be of no better quality.
The difference being, before LLMs, those, who didn't have qualification, wouldn't even approach problems like this, now they can vibecode something that works on a lucky run, but is otherwise completely inadequate.
I think this is the most apt summary of LLMs I’ve seen to date
I'm not really sure what it costs these days - i know certain projects not entirely free like they were a few years ago, but there's a pretty good "build your own redis" among other things.
It has little step-by-step tasks with automated tests. There are some other good ones like git and docker. It's pretty cool.
https://app.codecrafters.io/courses/redis/overview
It has little step-by-step tasks with automated tests. There are some other good ones like git and docker. It's pretty cool.
https://app.codecrafters.io/courses/redis/overview
It's easy to write a KV store, even a KV store with key expiration. But Redis is quite a bit more than that. Consider sharding and replication at least. Then lists, sets, zsets, bitfields, streams, geospatial indexes, hyperloglog / bloom filters, time series.
Something like LevelDB is relatively easy to write. Then you can build the rest of Redis on top of it.
Something like LevelDB is relatively easy to write. Then you can build the rest of Redis on top of it.
> Then lists, sets, zsets, bitfields, streams, geospatial indexes, hyperloglog / bloom filters, time series.
I mean, I have my own C libraries for most of these!
I mean, I have my own C libraries for most of these!
In Redis those operations are performed server-side, on a shared dataset, and concurrently by many clients.
Sounds pretty interesting.
Seems like LLMs are getting good at Zig (with some help, I presume).
Is there anything about this project that seems LLM-generated?
I've found that LLMs are particularly bad at writing Zig because the language evolves quickly, so LLMs that are trained on Zig code from two years ago will write code that no longer compiles on modern Zig.
I've found that LLMs are particularly bad at writing Zig because the language evolves quickly, so LLMs that are trained on Zig code from two years ago will write code that no longer compiles on modern Zig.
I skimmed, for me it was this: https://github.com/barddoo/zedis/blob/87321b04224b2e2e857b67...
There seems to be a fair amount of stigma around using llms. And many people that use them are uncomfortable talking about it.
It's a weird world. Depending on who is at the wheel, whether an llm is used _can_ make no difference.
But the problem is, you can have no idea what you're doing and make something that feels like it was carefully hand-crafted by someone - a really great project - but there are hidden things or outright lies about functionality, often to the surprise of the author. Like, they weren't trying to mislead, just didn't take them time to see if it did all of what the LLM said it did.
There seems to be a fair amount of stigma around using llms. And many people that use them are uncomfortable talking about it.
It's a weird world. Depending on who is at the wheel, whether an llm is used _can_ make no difference.
But the problem is, you can have no idea what you're doing and make something that feels like it was carefully hand-crafted by someone - a really great project - but there are hidden things or outright lies about functionality, often to the surprise of the author. Like, they weren't trying to mislead, just didn't take them time to see if it did all of what the LLM said it did.
Agree. I used it mostly for getting ideas, the memory management for example, Gemini listed so many different ways of managing memory I didn’t even know existed.
I know I wanted to pre allocate memory like tigerbeetle does, so the hybrid approach was perfect. Essentially it has 3 different allocators, a huge one for the cache, a arena allocator for context, intermediate state like pub/sub and temp one, for requests.
It was 100% Gemini’s idea.
3 months ago I was vibe coding an idea and for some reason (and luck) I went to check a less important part of the code and saw that the LLM changed the env variable of an API key and hard coded the key explictly in the code. That was scary. I'm glad I saw it before PR and shit like that.
I generally do not think it is a bad thing. I use LLMs too and I know what I am doing, so I do not know if it could be qualified as vibe coding.
I think it is not inherently a bad thing to use LLMs, only if you have absolutely no clue about what you are doing, but even then, if the project is usable and as-is advertised, why not? shrugs
As for the link, that is exactly the same code that caught my eye, besides the README.md itself. The LRU eviction thing is what GPT (and possibly other LLMs) always comes up with according to my experiences, and he could have just had it properly implemented then. :D
Edit: I am glad author confirmed the use of an LLM. :P
I think it is not inherently a bad thing to use LLMs, only if you have absolutely no clue about what you are doing, but even then, if the project is usable and as-is advertised, why not? shrugs
As for the link, that is exactly the same code that caught my eye, besides the README.md itself. The LRU eviction thing is what GPT (and possibly other LLMs) always comes up with according to my experiences, and he could have just had it properly implemented then. :D
Edit: I am glad author confirmed the use of an LLM. :P
As an avid Claude Code user, I can tell you with 99% probability, that README is LLM-generated. This exactly the same structure and wording used by Claude (of course, it has some human modification because otherwise I’d filled with emojis)
In my experience, when you work with something like agentic development tools, you describe your goals and give it some constraints like “use modern zig”, “always run tests”… and when you ask it to write a README, it will usually reproduce those constraints more or less verbatim.
The same thing happens with the features section, it reads like instructions for an LLM.
I might be wrong but I spend way too much time using Claude, Gemini, Codex… and IMHO it’s pretty obvious.
But hey, I don’t think it’s a problem! I write a lot of code using LLMs, mostly for learning (and… ejem, some of it might end up in production) and I’ve always found them great tools for learning (supposing you use the appropriate context engineering and make sure the agent has access to updated docs and all of that). For example, I wanted to learn Rust so I half-vibed a GPUI-based chat client for LLMs that works just fine and surprisingly enough, I actually learned and even had some fun.
In my experience, when you work with something like agentic development tools, you describe your goals and give it some constraints like “use modern zig”, “always run tests”… and when you ask it to write a README, it will usually reproduce those constraints more or less verbatim.
The same thing happens with the features section, it reads like instructions for an LLM.
I might be wrong but I spend way too much time using Claude, Gemini, Codex… and IMHO it’s pretty obvious.
But hey, I don’t think it’s a problem! I write a lot of code using LLMs, mostly for learning (and… ejem, some of it might end up in production) and I’ve always found them great tools for learning (supposing you use the appropriate context engineering and make sure the agent has access to updated docs and all of that). For example, I wanted to learn Rust so I half-vibed a GPUI-based chat client for LLMs that works just fine and surprisingly enough, I actually learned and even had some fun.
> But hey, I don’t think it’s a problem! I write a lot of code using LLMs, mostly for learning (and… ejem, some of it might end up in production) and I’ve always found them great tools for learning (supposing you use the appropriate context engineering and make sure the agent has access to updated docs and all of that). For example, I wanted to learn Rust so I half-vibed a GPUI-based chat client for LLMs that works just fine and surprisingly enough, I actually learned and even had some fun.
As I wrote in another comment, it is not inherently a bad thing. I use LLMs too (while knowing what I am doing), and if the project works, why not?
As I wrote in another comment, it is not inherently a bad thing. I use LLMs too (while knowing what I am doing), and if the project works, why not?
I don't know why you're being downvoted. This follows the LLM-generated-README template perfectly. And yeah, it usually ends up being a dumping ground for the constraints you gave it, almost verbatim.
Fair enough, but I have made a couple of projects in Odin using Claude, and Odin is evolving too, and it is much more obscure than Zig. I made these projects successfully by feeding the LLM the documentation of Odin and gave it example projects... thus, if I could create Odin projects (with some reiterations and hand-holding, true), then Zig should be even better.
https://github.com/barddoo/zedis/blob/87321b04224b2e2e857b67...
these seem to occur only in college assignment projects, and in the output of text generators trained on those.
these seem to occur only in college assignment projects, and in the output of text generators trained on those.
Has been doing this for years, even before LLMs were a thing. No, not in college assignments; by the time emoji appeared, I had long since walked out of my PhD program and went to the industry.
I put such emojis at the beginning of big headings, because my eyes detect compact shapes and colors faster than entire words and sentences. This helps me (and hopefully others) locate the right section easier.
In Slack, I put large emojis at the beginning of messages that need to stand out. These are few, and emojis work well in this capacity.
(Disclaimer: I may contain a large language model of some kind, but very definitely I cannot be reduced to it in any area of my activity.)
I put such emojis at the beginning of big headings, because my eyes detect compact shapes and colors faster than entire words and sentences. This helps me (and hopefully others) locate the right section easier.
In Slack, I put large emojis at the beginning of messages that need to stand out. These are few, and emojis work well in this capacity.
(Disclaimer: I may contain a large language model of some kind, but very definitely I cannot be reduced to it in any area of my activity.)
FWIW it is really confusing to me and others. What is this emoji supposed to mean? Heck if I know.
But the telltale signs are far more than just that. The whole document is exactly the kind of README produced by Claude.
But the telltale signs are far more than just that. The whole document is exactly the kind of README produced by Claude.
I had assumed they were referring to stuff like "Type-safe operations with compile-time guarantees". What a weird detail to add to a readme. And the whole section is like that. I wonder if that's part of a prompt leaking through.
I will never place emojis in any of my readmes ever again.
People were doing this before llms, otherwise how did they learn it?
Sure, but llms absolutely love to do it for some reason.
I just took a look at a Readme I had cursor write a couple months ago and there's no emojis
You can always have it in your custom prompt: "Do not use emojis, ever".
I had a manager who did this in his slack messages
That's why he said "never again"
spell out 'development' with hammer emojis. bring ascii art back as emoji art.
(i actually do this in slack messages and folks find it funny and annoying, but more funny)
(i actually do this in slack messages and folks find it funny and annoying, but more funny)
Oh that’s nice of you. I take that as a compliment.
It was not intended to be a criticism. I care about the end result, not the process. I use LLMs myself, too.
From the README:
> Memory safety with RAII patterns
I'm curious to see how they achieve RAII in Zig, which doesn't have destructors. If they mean defer + deinit methods, that's not the same thing.
> Memory safety with RAII patterns
I'm curious to see how they achieve RAII in Zig, which doesn't have destructors. If they mean defer + deinit methods, that's not the same thing.
> If they mean defer + deinit methods, that's not the same thing.
No? It's the first thing that came to my mind when I've read RAII.
No? It's the first thing that came to my mind when I've read RAII.
You have to call defer after initialization so you can’t technically call it RAII. As long as you remember to call it then it is effectively the same. Personally defer makes it so very similar to be a difference without a distinction but the “is initialization” part can’t be true if you need to do a second step.
Zig is all about making things explicit. Destructors are hidden code.
I presume there will be an application level tracking of lifetime.
Yes, which is why Zig can't have RAII. Zig initialisation and destruction are fundamentally different to how RAII works in other languages. That's not a negative point; it's just how Zig is. But it's not RAII.
You are right. It seems the developer agrees and has changed the README
It's explicit opt in RAII, you opt in by using a defer.
RAII, which stands for Resource Acquisition Is Initialization, is a programming technique in which a resource's lifetime is tied to an object's lifespan. An explicit defer does not fit this paradigm.
free is an explicit opt-in garbage collector.
[deleted]
Consider adding support for vector search and sketching (https://www.sketchingbigdata.org/).
I'll take a look. Thanks for the suggestion!
This is an interesting project but a poorly chosen name because it can easily be considered trademark infringement for being confusingly-similar to "Redis", especially since it's a self-admitted clone. I would change it now before you are forced to change your name and you lose time confusing people and wasting time renaming everything.
I don't think it's particularly confusing. I wouldn't expect something official to have a name that changes the first letter. Would you honestly be confused about whether a search engine called Boogle, a laptop called a Yacbook, or a browser called Tirefox was an official product by Google, Apple, or Mozilla?
I’d see it in swift and then being called "Swedish" for giggles.
Naming is famously difficult in this world!
Naming is famously difficult in this world!
i think “$proj - implements the redis protocol” are going to be as ubiquitous as the things which claim to be “S3 compatible”
idk if redis labs (the organization aka big bad) are going to patent the “protocol” to disallow clones (or they’d be going after valkey first)
idk if redis labs (the organization aka big bad) are going to patent the “protocol” to disallow clones (or they’d be going after valkey first)
That’s a good point, I should had renamed it. You can’t trust these big corp
Is zig lang stable enough now to start basing real projects on it?
Depends on your definition of stable. I would suspect most people would consider it unstable as they are still breaking the API for async. Wait for 1.0 if you really need true stability. But zig is just so good not to use now
The standard library is changing too much to be honest. During the development of the library Zig 0.15 launched and changed the whole Io library.
"Is zig lang stable enough now to start basing real projects on it?"
only if you want to refactor/rewrite a lot
only if you want to refactor/rewrite a lot
Check out Tigerbeetle and Ghostty
Also bun
I mean no offense but a billionaires vanity terminal and a database with an anime bug mascot are a bit different than a redis alternative
Good point, the database written in Zig with a bug mascot tells us nothing about writing a database in Zig without a bug mascot
im not sure i trust any hot takes from a person that doesn't know anything about anime
If all you know about Tigerbeetle is the mascot you should probably take another look before judging. Some very interesting db tech.
Is software not real software if it's written by a billionaire? What is it about the bug mascot that detracts from the database's legitimacy?
What's the problem with the mascot? PostgreSQL has an elephant and MySQL a dolphin. Is the bug too detailed for database software?
Why would we want a literal BUG as a mascot?? /s
[deleted]
bun is real and spectacular. super loving using it
[deleted]
it hasn't reached version 1.0
Did you try to run redis-benchmark? (Compared to Redis)
What about calling it Zigdisk and make it save to disk?
cool stuff, keep at it!
Thanks! There are so many things to do still, authentication, glob-style keys search, json parsing.
So riiR is now riiZ? ;)
"Rewrite it away from C (or C++)". Whether you choose Rust, Zig, Ada, D, Nim, even Pascal, it's likely going to become more secure. It will be supported on fewer platforms though, but still should run fine under Linux, macOS, Windows, *BSD, on x64, Arm64, and likely RISC-V, too.
Zig isn't memory safe, though. It's safer than C or C++, but not much.
This is what I have struggled to understand about Zig. It seems pretty much like C in a mental model aspect - you are responsible for everything. It's slightly better than C, but C already runs on everything on the planet and can be made secure even if painfully so. So what niche is Zig aiming to fill?
no, null pointers are enforced safe at the type level in zig, as are array bounds, this eliminates huge classes of errors, so you are not "responsible for everything". unlike c, you often (unless highly tuned performance is needed) do not have to resort to opaque void pointers, and the compiler gives you typesafety on that, another major footgun in c.
also operators and integer types are unambiguous, and there is no UB in safe compilation modes.
It's arguably much better than C, not "slightly better than C"
also operators and integer types are unambiguous, and there is no UB in safe compilation modes.
It's arguably much better than C, not "slightly better than C"
If you write a modern style of C, you can have bounds checked code and do not need to use void pointers. I usually find that people overestimate the advantages of newer languages compared to using C by comparing to old and badly written C.
GP: "you are responsible for everything"
AKA You are responsible for opting into "modern C". In order to be unsafe in zig (in the dimensions I mentioned) you must opt out.
AKA You are responsible for opting into "modern C". In order to be unsafe in zig (in the dimensions I mentioned) you must opt out.
You would need to opt-in into zig first, which is more effort than opting into modern C when you come from C.
i suggest you show this thread to a neutral third party and ask them if your line of argument makes sense.
I used Zig for my projects for a while, but moved back to C for similar reasons. C23, with gcc extensions, and using MISRA-like coding style where it makes sense, provides a similar experience to Zig but with seamless C interop. You dont have comptime, but my biggest lesson from my time in Zig is that I usually want to pass a vtable instead of monomophizing over input types.
Considering that basically nobody uses Zig, I think most neutral third-parties seem to agree.
[deleted]
Zig detects memory leaks pretty well when you build it using -Doptimize=Debug.
checked array bounds is memory safety.
Runtime bounds checking, but only in Debug/ReleaseSafe builds, besides there are many other types of memory safety issues left unaddressed.
> Runtime bounds checking
Compile time slice bounds checking is generally not a thing, since the length is not known until runtime. Its still safe.
> but only in Debug/ReleaseSafe builds
So, the default build modes?
Istg zig should just eliminate releasefast and releasesmall and force you to opt in on a function by function basis.
Compile time slice bounds checking is generally not a thing, since the length is not known until runtime. Its still safe.
> but only in Debug/ReleaseSafe builds
So, the default build modes?
Istg zig should just eliminate releasefast and releasesmall and force you to opt in on a function by function basis.