Build an Elixir Redis Server that’s faster than HTTP(docs.statetrace.com)
docs.statetrace.com
Build an Elixir Redis Server that’s faster than HTTP
https://docs.statetrace.com/blog/redis-server/
73 コメント
You could implement backpressure by either adding a buffer (which you already have here) or by rejecting requests optionally with more data on how hard and long to back off.
If you can tolerate (debounced) data loss in the buffer, a ring buffer works really well with predictable memory and performance.
Since you relate this example to Go, would you mind sharing thoughts about how heavy I/O from network sockets compares between the two or gotchas that might not be apparent to Erlang/Go developers about the other?
I think they're both relatively straightfoward, with the biggest difference being active mode. (I'll use "Elixir", but it all applies to Erlang as well).
Elixir sockets can be safely shared between processes, which might not be obvious to an Elixir programmer.
To the best of my knowledge, Elixir will use writev where possible, so it's iolist friendly and can be extremely efficient.
Binary pattern is a productivity boost when it comes to networking work.
Every Elixir socket is associated with a "controlling_process". This creates a link between the socket and the process. The linked article uses it. You generally don't want this to be the acceptor loop, since if the acceptor loop crashed, it would close every socket. Fun fact, I believe earlier versions had bugs / race conditions with respect to changing the controlling_process while data was incoming. This has since been fixed, and things "just work" like you expect, but I can only imagine that it involved some coding gymnastics to fix.
Since the Elixir sockets are more abstracted, there's more knobs you can turn and tweak, e.g., you can tweak the buffers that Erlang is using to read/write. Or it has built-in parsers for common formats, including simple things like automatically being able to send/receive 1, 2, or 4 byte length-prefixed messages.
Elixir has two socket APIs. The traditional gen_tcp, and a new one, socket, which is meant to be "as close as possible to the OS level socket interface." I haven't tried the new one yet.
Elixir sockets can be safely shared between processes, which might not be obvious to an Elixir programmer.
To the best of my knowledge, Elixir will use writev where possible, so it's iolist friendly and can be extremely efficient.
Binary pattern is a productivity boost when it comes to networking work.
Every Elixir socket is associated with a "controlling_process". This creates a link between the socket and the process. The linked article uses it. You generally don't want this to be the acceptor loop, since if the acceptor loop crashed, it would close every socket. Fun fact, I believe earlier versions had bugs / race conditions with respect to changing the controlling_process while data was incoming. This has since been fixed, and things "just work" like you expect, but I can only imagine that it involved some coding gymnastics to fix.
Since the Elixir sockets are more abstracted, there's more knobs you can turn and tweak, e.g., you can tweak the buffers that Erlang is using to read/write. Or it has built-in parsers for common formats, including simple things like automatically being able to send/receive 1, 2, or 4 byte length-prefixed messages.
Elixir has two socket APIs. The traditional gen_tcp, and a new one, socket, which is meant to be "as close as possible to the OS level socket interface." I haven't tried the new one yet.
is there somewhere where there is a comparison of the performance between active and passive mode? I don't imagine it to be significant for most use cases, so the extra safety seemed worth it to me in all the libraries I wrote, though I suppose it wouldn't be hard to rewrite those with {active, 1}.
https://stressgrid.com/blog/cowboy_performance_part_2/
Might be interesting. Amongst other things, they experimented with a patch to Cowboy to use {active, 100} instead of the default {active, 1} and got some nice wins.
Might be interesting. Amongst other things, they experimented with a patch to Cowboy to use {active, 100} instead of the default {active, 1} and got some nice wins.
Ranch is a pretty well optimized and battle hardened tcp acceptor. It powers the Cowboy/Phoenix server which scales to extreme level of concurrency and low latency. Cowboy uses ranch to pool and accept connections and I believe it uses {active,once}.
https://github.com/ninenines/cowboy
https://github.com/ninenines/ranch
https://github.com/ninenines/cowboy
https://github.com/ninenines/ranch
Using active mode is a nice because you can receive other messages as well, instead of blocking to receive just a tcp message. I like active once myself.
you can time out your recv, even with the value 0!
At first I was wondering, wait, then how do you talk to the server from the browser? ...Wait, if you control both the client and server, why even use HTTP?
I feel like there's this distressing trend where web programmers export their practices everywhere, and over time, those become the new accepted "primitives" instead of the actual primitives. What used to be a socket is now apparently a whole black-box protocol, where swapping one protocol with another to get speed gains is now noteworthy, and what used to be just FFI or a shared library is now a whole client/server architecture. This is basically the intellectual lineage of stuff like "language servers" where every keystroke your editor heap allocates a JSON tree, serializes it, initiates an HTTP connection, sends it over, reads the response, deserializes it into more heap allocated nodes, etc.... when the actual stated goal (making code intelligence consumable by any client) could be achieved using much less.
Standby as I follow up on this with my own post, "Build a TCP server that's faster than the Redis protocol."
I feel like there's this distressing trend where web programmers export their practices everywhere, and over time, those become the new accepted "primitives" instead of the actual primitives. What used to be a socket is now apparently a whole black-box protocol, where swapping one protocol with another to get speed gains is now noteworthy, and what used to be just FFI or a shared library is now a whole client/server architecture. This is basically the intellectual lineage of stuff like "language servers" where every keystroke your editor heap allocates a JSON tree, serializes it, initiates an HTTP connection, sends it over, reads the response, deserializes it into more heap allocated nodes, etc.... when the actual stated goal (making code intelligence consumable by any client) could be achieved using much less.
Standby as I follow up on this with my own post, "Build a TCP server that's faster than the Redis protocol."
As one of those web programmers, I'm also rediscovering that you don't have to use HTTP all the time. To be fair, there's already so much to learn just on the web side that it's hard to know everything.
Instead of stereotyping and blaming someone, better to find cause in ecosystem. Except for last few years most non HTTP communication systems were complex, platform specific and with enterprisey bloat.
May not be for this instance, but if people are gravitating towards a bad tooling alternative must be worse by some important parameter.
May not be for this instance, but if people are gravitating towards a bad tooling alternative must be worse by some important parameter.
Thank god I have a machine manufactured in the last decade, or else that would be a problem.
Okay, but then why optimize at all? Computers are fast, why not just run HTTP everywhere? What’s the point of the article?
My Mac mini runs at three whole billion clock cycles per second. If we’re going to depend on hardware to make our argument, we’re all basically running supercomputers. And yet everything is slow, and my tools lag for no reason, the web in general is slow. Which is all fine, but I thought computers were supposed to be fast? Somehow we’ve arrived at a point where on the one hand, computers are supposed to be fast enough to support these huge skyscrapers of abstraction, but on the other hand… the result is mind blowingly slow.
My Mac mini runs at three whole billion clock cycles per second. If we’re going to depend on hardware to make our argument, we’re all basically running supercomputers. And yet everything is slow, and my tools lag for no reason, the web in general is slow. Which is all fine, but I thought computers were supposed to be fast? Somehow we’ve arrived at a point where on the one hand, computers are supposed to be fast enough to support these huge skyscrapers of abstraction, but on the other hand… the result is mind blowingly slow.
Using the Redis protocol doesn't seem like the most efficient way to implement a fast client/server architecture. The Redis protocol[1] is text-based, where each command is an array with a header describing the number of elements in the command, and each element is prefixed by `$` and its length, with abundant \r\n to separate all those (this is a simplification, but almost all commands sent to Redis follow this array format).
For example, to send "SET my-key hello" to Redis, this is a 3-element array (the initial *3) with elements of length 3, 6, and 5. So you'd send:
Decoding lengths from their textual representation is inefficient in both space and time, and the various \r\n add to the request and response sizes without adding anything useful to a program that uses it (it is of course useful for humans typing those commands, but who does that?).
Even using something as common as protobuf would likely be more efficient, with smaller requests and responses – although object creation can lead to memory pressure for complex messages.
But really, since Elixir leverages Erlang, why not have a basic common protocol with pattern matching on binaries? This would make a lot more sense than using a text-based protocol, in my opinion.
[1] https://redis.io/topics/protocol
For example, to send "SET my-key hello" to Redis, this is a 3-element array (the initial *3) with elements of length 3, 6, and 5. So you'd send:
*3\r\n
$3\r\n
SET\r\n
$6\r\n
my-key\r\n
$5\r\n
hello\r\n
Decoding lengths from their textual representation is inefficient in both space and time, and the various \r\n add to the request and response sizes without adding anything useful to a program that uses it (it is of course useful for humans typing those commands, but who does that?).
Even using something as common as protobuf would likely be more efficient, with smaller requests and responses – although object creation can lead to memory pressure for complex messages.
But really, since Elixir leverages Erlang, why not have a basic common protocol with pattern matching on binaries? This would make a lot more sense than using a text-based protocol, in my opinion.
[1] https://redis.io/topics/protocol
One of the things, among many then unique features, that charmed me about Redis when I first encountered it when it first surfaced (2008?) was the ability to open a telnet session to Redis and type commands see the responses. It also hugely helped in writing drivers for it. As to your intuition (“seems”), the fact is that Redis is a very high performance server. I think the key is considering what fraction of time of servicing a request is spent on parsing the requests — given the network hop and other non-protocol computations, it is likely to be negligible.
I think Redis would have been successful even without a text-heavy protocol.
Redis took a middle ground and [ascii]-length prefixed responses (though I don't think it was initially like this). That's a reasonable and likely necessary compromise. A bit similar to how HTTP can include a content-length. You'd hate to be scanning large payloads for that trailing \r\n.
So Redis terminates with \r\n (which isn't strictly necessary for all message types) and ascii encodes the types and length prefix. Everything else is just bytes.
Here's (1) a paper that looked at text vs binary protocols. They looked at it in the context of MonetDB though, which has, by far, the worse text-based protocol I've ever seen (so much so that even their official drivers have (had?) plenty of protocol-related bugs)
(1) - https://15721.courses.cs.cmu.edu/spring2018/papers/14-networ...
Redis took a middle ground and [ascii]-length prefixed responses (though I don't think it was initially like this). That's a reasonable and likely necessary compromise. A bit similar to how HTTP can include a content-length. You'd hate to be scanning large payloads for that trailing \r\n.
So Redis terminates with \r\n (which isn't strictly necessary for all message types) and ascii encodes the types and length prefix. Everything else is just bytes.
Here's (1) a paper that looked at text vs binary protocols. They looked at it in the context of MonetDB though, which has, by far, the worse text-based protocol I've ever seen (so much so that even their official drivers have (had?) plenty of protocol-related bugs)
(1) - https://15721.courses.cs.cmu.edu/spring2018/papers/14-networ...
Ah yes I remember trying out Redis with telnet, it was helpful at the very beginning. Thankfully command-line clients came around soon enough. Also, the protocol as it is today was only introduced in Redis 1.2, and wasn't around at the very beginning.
It's true that the parsing time is small compared to the network round-trip, but Redis being single-threaded means that while it's parsing a large nested command and allocating memory as it goes through it… well nothing else is being processed.
I'm pretty familiar with the costs of parsing this protocol, being the original author of the phpredis extension. Written in C, it's a high-performance client whereas pure-PHP clients can't use the same zero-copy tricks or careful memory management in general to avoid repeated calls to strlen/malloc/memcpy/free (in that order). In Webdis – another high-performance Redis tool I wrote – protocol parsing with Hiredis is similar to HTTP parsing in the sense that it's where most CPU cycles are spent.
I'm not really sure why Salvatore chose to keep the text-based protocol when his focus was so often on performance. I'm not aware of any attempts made by him or other people to change the protocol, but it would be interesting to see how much of a difference this could make.
It's true that the parsing time is small compared to the network round-trip, but Redis being single-threaded means that while it's parsing a large nested command and allocating memory as it goes through it… well nothing else is being processed.
I'm pretty familiar with the costs of parsing this protocol, being the original author of the phpredis extension. Written in C, it's a high-performance client whereas pure-PHP clients can't use the same zero-copy tricks or careful memory management in general to avoid repeated calls to strlen/malloc/memcpy/free (in that order). In Webdis – another high-performance Redis tool I wrote – protocol parsing with Hiredis is similar to HTTP parsing in the sense that it's where most CPU cycles are spent.
I'm not really sure why Salvatore chose to keep the text-based protocol when his focus was so often on performance. I'm not aware of any attempts made by him or other people to change the protocol, but it would be interesting to see how much of a difference this could make.
IIRC he did discuss his decision in the redis google group - don’t recall the details.
14ms for an echo server is pretty slow. 1.8s for an HTTP echo server of the same data... I don't even know how this can possibly be valid. Something's wrong, or I misunderstood the table.
Something seems wrong here. Over 1second on average to handle 1k http requests? I'm not sure the conclusion is valid. The test seems very wrong. Http being slow seems like a wild thing to say. Apache can serve that much
There is no way to get around the fact that the HTTP protocol adds overhead (serialization and parsing of additional data) at the application layer. Binary RPC protocols are faster, and that is why Redis uses one of those instead of HTTP. There is no debate about that, it is simply a set of trade-offs that balances out differently for different use cases.
Your confused too, resp is a text protocol. So is http. This isn't binary vs text protocol expermint. None of the http parsing should take seconds like this suggests.a simple protocol might be faster to handle but another post down below makes a similar point where something implented well can respond is micro seconds. Really all this shows is that handling text off a socket is faster then this http framework in eleixr
RESP can transfer raw binaries, the only overhead is a message length. In the TFA they are talking about using MessagePack for the payloads, so it is a binary protocol.
http transfers binary, what does that even mean? the only thing the message pack thing is used for is encoding the value in the requests. but all of that is irrelevant because thats not what i was talking about in the first place. the test is poorly done.
An average latency of 17ms to handle a simple request with the resp protocol is a red flag it self that something is wrong. Seconds to have an http response returned is a problem. All this article is showing is elixir and the libraries used are incredibly slow. Calling HTTP slow, based off the tests done in this article is just wrong, because the whole test is slow and not testing the right thing.
Micro bench marking of the parsing functions would be the right thing to do here.
really someone should profile the server and see what is taking seconds to respond. it wont be the parsing related functions
parsing http really isnt that big of a deal. its a simple protocol, the action on line one, headers split by : and the body. resp has a simmilar amount of work too, uses multiple lines
An average latency of 17ms to handle a simple request with the resp protocol is a red flag it self that something is wrong. Seconds to have an http response returned is a problem. All this article is showing is elixir and the libraries used are incredibly slow. Calling HTTP slow, based off the tests done in this article is just wrong, because the whole test is slow and not testing the right thing.
Micro bench marking of the parsing functions would be the right thing to do here.
really someone should profile the server and see what is taking seconds to respond. it wont be the parsing related functions
parsing http really isnt that big of a deal. its a simple protocol, the action on line one, headers split by : and the body. resp has a simmilar amount of work too, uses multiple lines
https://stressgrid.com/blog/webserver_benchmark/
These benchmarks, while dated and haven't been tested with the newly released updates to address these dynamics - might interest some folks.
These benchmarks, while dated and haven't been tested with the newly released updates to address these dynamics - might interest some folks.
How come HTTP be this slow? If it's slow, then the problem is on the library that we use for parsing HTTP and not HTTP protocol itself.
In this case, both HTTP and Redis use TCP. If HTTP header and data fit in 1 MTU, there should be no difference. No way searching for 2 CRLF that too in 1 MTU data size be 100x slower
In this case, both HTTP and Redis use TCP. If HTTP header and data fit in 1 MTU, there should be no difference. No way searching for 2 CRLF that too in 1 MTU data size be 100x slower
I’m sorry but this benchmark is not testing the underlining protocol but the clients themselves (pooling mechanism, pipelining, keep-alive, etc). The Redis protocol is probably faster to parse than HTTP but not a 100x factor.
It looks like websocket+msgpack would have similar overhead once the initial initial overheard is passed.
But you won't miss all the http reverse proxy ecosystem.
But you won't miss all the http reverse proxy ecosystem.
[deleted]
Seems like the author should have stopped when they concluded that it takes 2 seconds to serve an HTTP request before publishing this and embarrassing themselves. gRPC C++ serves HTTP/2 in less than 120 microseconds, which includes server-to-server network flight time.
https://performance-dot-grpc-testing.appspot.com/explore?das...
https://performance-dot-grpc-testing.appspot.com/explore?das...
It's 1000 iterations in 2 seconds. That may be including connection setup time as well, I didn't dive into whether there's keepalive or not.
If the idea is to leverage the Redis protocol for an “API”, what are the benefits to this approach over using the built-in `pub-sub` and the regular old C implementation of Redis Server?
I could maybe see some really specific use-cases for this but for probably 90% of cases implementing a distributed process API should be able to go over pubsub fine shouldn’t it? Or does pubsub have some sort of massive overhead?
EDIT: Just because it’s tangential to the topic at hand, and since I’m up here at the top, I also wanna throw out a nod to libraries like `nng` and `nanomsg` which are spiritual successors to ZeroMQ and they have functionality like brokerless RPC/pubsub built in as messaging models. I don’t see tools like this talked about a lot in this space cuz systems software isn’t the sexiest but if you need to embed a small and lightweight messaging endpoint in your backend stuff then look at those as well. No horse in the race, just like sharing useful tools with people.
https://nng.nanomsg.org
I could maybe see some really specific use-cases for this but for probably 90% of cases implementing a distributed process API should be able to go over pubsub fine shouldn’t it? Or does pubsub have some sort of massive overhead?
EDIT: Just because it’s tangential to the topic at hand, and since I’m up here at the top, I also wanna throw out a nod to libraries like `nng` and `nanomsg` which are spiritual successors to ZeroMQ and they have functionality like brokerless RPC/pubsub built in as messaging models. I don’t see tools like this talked about a lot in this space cuz systems software isn’t the sexiest but if you need to embed a small and lightweight messaging endpoint in your backend stuff then look at those as well. No horse in the race, just like sharing useful tools with people.
https://nng.nanomsg.org
> and the regular old C implementation of Redis Server?
To be clear, this article is talking about implementing your own synchronous-RPC-request server, i.e. a network service that other services talk to through an API over some network wire protocol, to make requests over a socket and then wait for responses to those requests over that same socket. This article assumes that you already know that that's what you need. This article then offers an additional alternative to the traditional wire protocols one might expose to clients in a synchronous-RPC-request server (RESTful HTTP, gRPC, JSON-RPC over HTTP, JSON-RPC over TCP, etc.); namely, mimicking the wire protocol Redis uses, but exposing your own custom Redis commands. This choice allows you to use existing Redis client libraries as your RPC clients, just as writing a RESTful HTTP server allows you to use existing HTTP client libraries as your RPC clients.
The alternative to doing so, if you want to call it that, would be to write these custom commands as a Redis module in C. But then you have to structure your code to live inside a Redis server, when that might not be at-all what you want, especially if your code already lives inside some other kind of framework, or is written in a managed-runtime language unsuited to plugging into a C server.
Or think of it like this: this article is about taking an existing daemon, written in some arbitrary language (in this case Elixir), where that daemon already speaks some other, slower RPC protocol (e.g. REST over HTTP); and adding an additional Redis-protocol RPC listener to that daemon, so that you can use a Redis client as a drop-in replacement for an HTTP client for doing RPC against the daemon, thus (presumably) lowering per-request protocol overhead for clients that need to pump through a lot of RPC requests.
I do realize that you're suggesting that you could use Redis as an event-bus between two processes that each connect to it via the Redis protocol; and then use a "fire an async request as an event over the bus, and then await a response event containing the request's ref to show up on the bus" RPC strategy, ala Erlang's own gen_server:call messaging strategy. All I can say is that, due to there being three processes and two separate RPC sessions involved, with their own wire-protocol encoding/decoding phases, that's likely higher-overhead than even a direct RESTful-HTTP RPC session between the client and the relevant daemon; let alone a direct Redis RPC session between the client and the daemon.
To be clear, this article is talking about implementing your own synchronous-RPC-request server, i.e. a network service that other services talk to through an API over some network wire protocol, to make requests over a socket and then wait for responses to those requests over that same socket. This article assumes that you already know that that's what you need. This article then offers an additional alternative to the traditional wire protocols one might expose to clients in a synchronous-RPC-request server (RESTful HTTP, gRPC, JSON-RPC over HTTP, JSON-RPC over TCP, etc.); namely, mimicking the wire protocol Redis uses, but exposing your own custom Redis commands. This choice allows you to use existing Redis client libraries as your RPC clients, just as writing a RESTful HTTP server allows you to use existing HTTP client libraries as your RPC clients.
The alternative to doing so, if you want to call it that, would be to write these custom commands as a Redis module in C. But then you have to structure your code to live inside a Redis server, when that might not be at-all what you want, especially if your code already lives inside some other kind of framework, or is written in a managed-runtime language unsuited to plugging into a C server.
Or think of it like this: this article is about taking an existing daemon, written in some arbitrary language (in this case Elixir), where that daemon already speaks some other, slower RPC protocol (e.g. REST over HTTP); and adding an additional Redis-protocol RPC listener to that daemon, so that you can use a Redis client as a drop-in replacement for an HTTP client for doing RPC against the daemon, thus (presumably) lowering per-request protocol overhead for clients that need to pump through a lot of RPC requests.
I do realize that you're suggesting that you could use Redis as an event-bus between two processes that each connect to it via the Redis protocol; and then use a "fire an async request as an event over the bus, and then await a response event containing the request's ref to show up on the bus" RPC strategy, ala Erlang's own gen_server:call messaging strategy. All I can say is that, due to there being three processes and two separate RPC sessions involved, with their own wire-protocol encoding/decoding phases, that's likely higher-overhead than even a direct RESTful-HTTP RPC session between the client and the relevant daemon; let alone a direct Redis RPC session between the client and the daemon.
> This article assumes that you already know that that's what you need
This is how I read the article. It was about how to implement network protocols in elixir and here are two of them: redis and msgpack.
Having an elixir based redis server is not the same piece of the puzzle as having elixir simply talk to a redis server. For one, the elixir based redis server can have arbitrary rules around keys and values that are not supported by redis. (Same said for a redis server written in C or python or rust or...)
This approach lets you store all keys and values in a dict, b-tree, sqlite or postgres etc. Want to store each value in a flat file? Sure, now you can. Only you know if this is actually useful.
At least, this is how I made sense of this.
This is how I read the article. It was about how to implement network protocols in elixir and here are two of them: redis and msgpack.
Having an elixir based redis server is not the same piece of the puzzle as having elixir simply talk to a redis server. For one, the elixir based redis server can have arbitrary rules around keys and values that are not supported by redis. (Same said for a redis server written in C or python or rust or...)
This approach lets you store all keys and values in a dict, b-tree, sqlite or postgres etc. Want to store each value in a flat file? Sure, now you can. Only you know if this is actually useful.
At least, this is how I made sense of this.
I don't think the article was talking writing a "Redis Server" in the sense of writing something that tries to do what redis-server does — i.e. to be a "data-structure server" that has a root-level keyspace of variously-typed values, and commands that atomically build up and query those data structures. Maybe they used that as an example of what you can do, but it's not the most useful example. (If that's what you wanted, why not just use a regular redis-server deployment?)
I think the article was instead just using the term "Redis server" to mean "any server that speaks the server side of the client-server protocol that redis-server speaks" (without implication that it stores data ala redis-server) — in the same way that an "HTTP server" is "any server that speaks the server side of the client-server protocol that HTTP servers speak" (without implication that it serves HTML files, generates directory indices, and supports per-user multitenant shares, ala default-configuration Apache.)
Note that the command set of Redis isn't part of the Redis protocol. A "Redis protocol" server could have an entirely novel set of commands, none of which have anything to do with keys or data-structures. It's just another way of exposing an API to clients. Your application-layer protocol over the Redis wire protocol could be "an API for triggering webhooks", or "a group-chat software protocol ala IRC/XMPP", etc.; and in none of those cases do you need to implement GET/SET/DEL/etc., or to describe your own use-case in terms of GET/SET/DEL/etc.†
The only thing the Redis wire-protocol necessitates, IIRC, is that each command start with a verb; that verbs consist of ASCII characters, with a certain maximum length; and that each verb have a fixed "schema" for the members of its parameter list, that pre-determines the encoding a client should use to send an instance of that command over the text or binary wire-protocols, without any connection-time schema discovery.
And yes, most Redis clients do have some way of sending custom commands to the server, with the schema for those commands specified at runtime (at least over the Redis text protocol), even if they don't have syntax sugar for doing it the way they do for the redis-server built-in commands. Even if a Redis client's aim is only to talk to redis-server deployments, they still have to support custom commands, because individual redis-server deployments are extensible with https://redis.io/modules that expose arbitrary commands, and client libraries can't possibly know about those modules at compile-time. So they have to support potentially any command at runtime, somehow or another.
-----
† Mind you, just like HTTP has "REST" (which basically means "using the default HTTP verbs for analogous purposes in your own API, instead of totally abusing theirs semantics or inventing your own verbs"), there could be a similar convention on top of the Redis wire protocol, where you implement your API in terms of the built-in redis-server verbs GET/SET/DEL/etc. Then you could use the full syntax-sugared default commands built into Redis-protocol clients, to talk to your server, instead of needing to rely on the runtime custom-command support. However, unlike with REST, I don't think this use-case is very useful — the schema of redis-server's built-in command verbs has pretty tight tolerances, and doesn't allow for too many use-cases that aren't just "building a data-structure server."
I think the article was instead just using the term "Redis server" to mean "any server that speaks the server side of the client-server protocol that redis-server speaks" (without implication that it stores data ala redis-server) — in the same way that an "HTTP server" is "any server that speaks the server side of the client-server protocol that HTTP servers speak" (without implication that it serves HTML files, generates directory indices, and supports per-user multitenant shares, ala default-configuration Apache.)
Note that the command set of Redis isn't part of the Redis protocol. A "Redis protocol" server could have an entirely novel set of commands, none of which have anything to do with keys or data-structures. It's just another way of exposing an API to clients. Your application-layer protocol over the Redis wire protocol could be "an API for triggering webhooks", or "a group-chat software protocol ala IRC/XMPP", etc.; and in none of those cases do you need to implement GET/SET/DEL/etc., or to describe your own use-case in terms of GET/SET/DEL/etc.†
The only thing the Redis wire-protocol necessitates, IIRC, is that each command start with a verb; that verbs consist of ASCII characters, with a certain maximum length; and that each verb have a fixed "schema" for the members of its parameter list, that pre-determines the encoding a client should use to send an instance of that command over the text or binary wire-protocols, without any connection-time schema discovery.
And yes, most Redis clients do have some way of sending custom commands to the server, with the schema for those commands specified at runtime (at least over the Redis text protocol), even if they don't have syntax sugar for doing it the way they do for the redis-server built-in commands. Even if a Redis client's aim is only to talk to redis-server deployments, they still have to support custom commands, because individual redis-server deployments are extensible with https://redis.io/modules that expose arbitrary commands, and client libraries can't possibly know about those modules at compile-time. So they have to support potentially any command at runtime, somehow or another.
-----
† Mind you, just like HTTP has "REST" (which basically means "using the default HTTP verbs for analogous purposes in your own API, instead of totally abusing theirs semantics or inventing your own verbs"), there could be a similar convention on top of the Redis wire protocol, where you implement your API in terms of the built-in redis-server verbs GET/SET/DEL/etc. Then you could use the full syntax-sugared default commands built into Redis-protocol clients, to talk to your server, instead of needing to rely on the runtime custom-command support. However, unlike with REST, I don't think this use-case is very useful — the schema of redis-server's built-in command verbs has pretty tight tolerances, and doesn't allow for too many use-cases that aren't just "building a data-structure server."
> This choice allows you to use existing Redis client libraries as your RPC clients, just as writing a RESTful HTTP server allows you to use existing HTTP client libraries as your RPC clients.
Thanks for this beautiful analogy. I'm familiar with redis and elixir and found the article interesting but I didn't quite understand the "why" and now this makes sense. Highlighting it here for others as well.
Thanks for this beautiful analogy. I'm familiar with redis and elixir and found the article interesting but I didn't quite understand the "why" and now this makes sense. Highlighting it here for others as well.
That’s fair. Although I think the characterization of doing RPC over Redis’s built-in pubsub is a little uncharitable. You’d just fire off a PUBLISH command w/ the channel and payload, and you’d receive the response to the RPC request on a subscriber that you can immediately drop (a pseudo-one-shot channel). It doesn’t have to be an event bus (even though that’s close to how Redis does pubsub internally).
When I say "event bus", I mean "an async RPC architecture using a reliable at-least-once message-queuing model, where clients connect to a message broker [e.g. a Redis stream], and publish RPC workloads there; backends connect to the same message broker, and subscribe as consumers of a shared consumer-group for RPC workloads, greedily take messages from the queue, and do work on them; backends that complete RPC workloads publish the workload-results back to the broker on channels specific to the original clients, while ACKing the original workloads on the workloads channel; and clients subscribe to their own RPC workload-results channel, ACKing messages as they receive them."
Event bus is the name for this network architecture. And if you're trying to replicate what synchronous client-server RPC does in a distributed M:N system, it's what you'd have to use. You can't use at-most-once/unreliable PUBSUB to replicate how synchronous client-server RPC works, as a client might sit around forever waiting for a response that got silently dropped due to the broker or a backend crashing, without knowing it. All the queues and ACKs are there to replicate what clients get for free from having a direct TCP connection to the server.
(Yes, Erlang uses timeouts on gen_server:call to build up distributed-systems abstractions on top of an unreliable message carrier. But everything else in an Erlang system has to be explicitly engineered around having timeouts on one end and idempotent handling of potentially-spurious "leftover" requests on the other. Clients that were originally doing synchronous RPC, where you don't know exactly how they were relying on that synchronous RPC, can switch to a Redis-streams event-bus based messaging protocol as a drop-in replacement for their synchronous client-server RPC, because reliable at-least-once async delivery can embed the semantics of synchronous RPC; but they can't switch to unreliable async pubsub as a drop-in replacement for their synchronous client-server RPC. Doing the latter would require investigation and potentially re-engineering, on both sides. If you don't control one end — e.g. if the clients are third-party mobile apps — then that re-engineering might even be impossible.)
Event bus is the name for this network architecture. And if you're trying to replicate what synchronous client-server RPC does in a distributed M:N system, it's what you'd have to use. You can't use at-most-once/unreliable PUBSUB to replicate how synchronous client-server RPC works, as a client might sit around forever waiting for a response that got silently dropped due to the broker or a backend crashing, without knowing it. All the queues and ACKs are there to replicate what clients get for free from having a direct TCP connection to the server.
(Yes, Erlang uses timeouts on gen_server:call to build up distributed-systems abstractions on top of an unreliable message carrier. But everything else in an Erlang system has to be explicitly engineered around having timeouts on one end and idempotent handling of potentially-spurious "leftover" requests on the other. Clients that were originally doing synchronous RPC, where you don't know exactly how they were relying on that synchronous RPC, can switch to a Redis-streams event-bus based messaging protocol as a drop-in replacement for their synchronous client-server RPC, because reliable at-least-once async delivery can embed the semantics of synchronous RPC; but they can't switch to unreliable async pubsub as a drop-in replacement for their synchronous client-server RPC. Doing the latter would require investigation and potentially re-engineering, on both sides. If you don't control one end — e.g. if the clients are third-party mobile apps — then that re-engineering might even be impossible.)
I don’t know if anyone ever told you before but you have a real talent for explaining thing clearly and simply!!
You should write technical book or something!!
You should write technical book or something!!
This article is about implementing the Redis Protocol on a socket yourself. Not about the Redis Database. Sorry for the confusion.
I actually did understand that and I think it’s super cool from a technical perspective. I was just curious as to the use case where this was a strong contender to beat out the built-in stuff you can get from Redis server. It’s a “small” implementation but you still have to test it and own it throughtout the lifecycle of the product that’s using it.
It's pretty common that you won't have a specific use case in mind when learning something new, but I've often found that later it ends up fitting in somehow.
If it's straight RPC there's no need for the extra moving parts, and the simpler your topology the fewer ways for it to go wrong.
I often build TPC based "protocols" that are just newline delimited json in each direction, it's a nice middle ground between using an HTTP POST and something like gRPC.
I often build TPC based "protocols" that are just newline delimited json in each direction, it's a nice middle ground between using an HTTP POST and something like gRPC.
I’m not 100% sure I’m understanding but if the idea is to implement a “traditional”/synchronous RPC mechanism you can do that using the PubSub functionality trivially and I’m not sure which additional moving parts it adds unless you’re in a situation where you have to cluster Redis (which has a reputation for being a PITA but isn’t that hard honestly).
The only use case that really jumps out is when you don’t want a broker because you don’t want a single point of failure but now you’re embedding a Redis server implementation in every one of the services in your mesh and I’m not convinced that’s much better but I can see where it might be helpful.
The only use case that really jumps out is when you don’t want a broker because you don’t want a single point of failure but now you’re embedding a Redis server implementation in every one of the services in your mesh and I’m not convinced that’s much better but I can see where it might be helpful.
> I’m not sure which additional moving parts it adds
> The only use case that really jumps out is when you don’t want a broker
The broker is exactly the additional moving part I was referring to.
Pubsub over a broker is more complicated than RPC operationally, whether you're already using the broker software elsewhere or not.
Especially when you're looking at the RPC server living inside an erlang VM that's already really good at handling things like load shedding of direct connections.
> The only use case that really jumps out is when you don’t want a broker
The broker is exactly the additional moving part I was referring to.
Pubsub over a broker is more complicated than RPC operationally, whether you're already using the broker software elsewhere or not.
Especially when you're looking at the RPC server living inside an erlang VM that's already really good at handling things like load shedding of direct connections.
I don't think this is anything to do with Redis server, you are simply using the Redis wire protocol/parser? I could be wrong...
That’s correct but my point was that Redis has an out-of-the-box solution for “message passing arbitrary data using the Redis protocol over TCP to named endpoints”, and Redis Server is a very lightweight piece of software. Even if you aren’t using it as an in-memory key-value DB it’s not a big problem to pull it in to your stack even if you’re only gonna use it for PubSub or RPC/IPC.
This is a super cool write-up and I’m not saying anything negative about what the author did. I like it a lot. I’m just asking from a technical and curiosity perspective what the advantages are to this over using the stuff Redis Server already provides which can do the same thing.
This is a super cool write-up and I’m not saying anything negative about what the author did. I like it a lot. I’m just asking from a technical and curiosity perspective what the advantages are to this over using the stuff Redis Server already provides which can do the same thing.
With Elixir specifically, you have the option of clustering these things. We (fly.io) send messages between servers using NATS. This works well with geographically distributed infrastructure, messages are somewhat peer to peer. If we were using Redis, we'd need a round trip to a centralized server. And we'd need the internet to always work well.
You can do a similar thing with Elixir and your own protocol (or the Redis protocol).
You can do a similar thing with Elixir and your own protocol (or the Redis protocol).
Problem with redis as a Pub-sub is it’s not transactional.
So a consumer can’t take temporary lease on a message try to process it and then delete the message if it was processed successfully.
This pattern is very common in banking but require something like RabbitMQ or Azure service bus …
This pattern is very common in banking but require something like RabbitMQ or Azure service bus …
I suppose the main advantages would be less dependencies (no Redis server). Less overhead (you aren’t routing though anything)
I haven’t looked at the exact performance characteristics but it would be fun! You would have built in load balancing!
I haven’t looked at the exact performance characteristics but it would be fun! You would have built in load balancing!
Correct this post is about the Redis Protocol.
It seems like kind of a fun hack but I'm not sure what this is getting you. Is this some kind of an RPC-style thing built on Redis? And I'm not sure where the comparison to HTTP is coming from. I have a lot of questions.
It's making the correct argument that handling HTTP is costly, and you don't always need it.
People use it because it's really easy to get something up and running with HTTP short term, but long term there are better ways of doing server-server communication.
> People use it because it's really easy to get something up and running with HTTP short term
And because there are decades of best-practices and standard ways to do very common things like authentication, caching, signaling errors, load-balancing, logging. Maybe a homemade RPC doesn't need any of that, but I would bet that at some point it will need at least some of it. And now you have to implement it all yourself.
This is cool though and kinda fun to think about!
And because there are decades of best-practices and standard ways to do very common things like authentication, caching, signaling errors, load-balancing, logging. Maybe a homemade RPC doesn't need any of that, but I would bet that at some point it will need at least some of it. And now you have to implement it all yourself.
This is cool though and kinda fun to think about!
> decades of best-practices and standard ways to do (...) caching
Essentially either you want RPC, or you might want caching. If you want to put a varnish cache in there, probably stick with http. If you know you're doing (very high thro / low latency) RPC - this might make a lot of sense.
Essentially either you want RPC, or you might want caching. If you want to put a varnish cache in there, probably stick with http. If you know you're doing (very high thro / low latency) RPC - this might make a lot of sense.
What if I want to memoize my RPC function though? I would have to implement that myself I think
If your messaging lends itself to caching, it might very well fit with REST - which means http might be a good option.
So it's not so much if you want to memorize (cache), but if you can* (and/or should).
I'm afraid I'm repeating myself - but I really whish people would read all of Fielding's thesis - it's got all kinds of reasonable architectures in it - even if it argues hypermedia/hypertext applications are well served by REST.
https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
So it's not so much if you want to memorize (cache), but if you can* (and/or should).
I'm afraid I'm repeating myself - but I really whish people would read all of Fielding's thesis - it's got all kinds of reasonable architectures in it - even if it argues hypermedia/hypertext applications are well served by REST.
https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
Author here. EXACTLY.
I’m showing that it is just as easy to use a different protocol than HTTP.
Instead of reaching for an HTTP framework to do RPC, I’m showing that Redis can be just as easy.
I’m showing that it is just as easy to use a different protocol than HTTP.
Instead of reaching for an HTTP framework to do RPC, I’m showing that Redis can be just as easy.
You're doing God's work. Anything reminding people that there's more to the Internet than the Web is a step in the right direction.
Isn't the other reason that HTTP APIs are the defacto, is because you often don't get to control the other side of the equation, so we default to the "common lowest denominator" case?
Tangentially related: I wish more protocols used netstrings [0] as transport. This would trivially eliminate a whole class of bugs that are all too easy to commit, especially in low-level languages.
[0]: https://cr.yp.to/proto/netstrings.txt
[0]: https://cr.yp.to/proto/netstrings.txt
As I understand it its just a way of building a generic service using the Redis protocols instead of HTTP. For example if you have a microservice architecture where services currently communicate over HTTP, you could potentially replace it with the Redis protocol. What I don't understand is how it compares to other rpc protocol formats (e.g. protocolbuffers).
Author here, combining Redis protocol and MsgPack is a much more platform independent way of building a protocol.
The redis protocol and Msppack are both only a a hundred lines or so for a parser. Meaning you can build your own from scratch in a new language if one isn’t supported.
Its also stupid fast.
Compared to protocol buffers which can be extremely complicated to grok on the binary level.
I built a more robust API RPC for Python here based on Redis and MsgPack: https://github.com/hansonkd/tino
The redis protocol and Msppack are both only a a hundred lines or so for a parser. Meaning you can build your own from scratch in a new language if one isn’t supported.
Its also stupid fast.
Compared to protocol buffers which can be extremely complicated to grok on the binary level.
I built a more robust API RPC for Python here based on Redis and MsgPack: https://github.com/hansonkd/tino
NATS has a similar protocol to Redis and optimized for this use cause of doing pub/sub based low latency Request/Response instead of HTTP. The payload is opaque so you can use msgpack if needed, and the protocol also supports sending headers now: https://docs.nats.io/nats-protocol/nats-protocol#protocol-me...
(disclaimer: I'm one of the NATS maintainers :) )
(disclaimer: I'm one of the NATS maintainers :) )
> extremely complicated to grok on the binary level.
That seems like a weird claim. Protobuf on the wire only has four types: short fixed-length values, long fixed-length values, variable-length values with continuation-bit encoding, and length-prefixed values, where the length is continuation-bit encoded.
Msgpack has 37 different wire types!
That seems like a weird claim. Protobuf on the wire only has four types: short fixed-length values, long fixed-length values, variable-length values with continuation-bit encoding, and length-prefixed values, where the length is continuation-bit encoded.
Msgpack has 37 different wire types!
No use in changing things if it's working for you, but you might be interested in trying out msgspec (https://jcristharif.com/msgspec/) in tino instead of using msgpack-python. The msgpack encoder/decoder is faster, and it comes with structured type validation (like pydantic) for no extra overhead (https://jcristharif.com/msgspec/benchmarks.html).
Did you consider ASN.1 for the protocol?
Lol. No offense, but I did a report in uni comparing a few alternatives (asn.1, json/xml over http, protobuf v1 (it's a while ago).
And on one hand, sure asn.1 exists, on the other there's been some issues in for example nfs. I don't think you want asn.1 today - but maybe cap'n'proto.
But i think redis is probably an interesting approach.
Not sure how many of these beyond whatever/http2 have a sane pipelining+authenticated encryption story? I suppose you could "dictate" security at the ip level via vpn/wireguard or something. Or use Unix sockets.
And on one hand, sure asn.1 exists, on the other there's been some issues in for example nfs. I don't think you want asn.1 today - but maybe cap'n'proto.
But i think redis is probably an interesting approach.
Not sure how many of these beyond whatever/http2 have a sane pipelining+authenticated encryption story? I suppose you could "dictate" security at the ip level via vpn/wireguard or something. Or use Unix sockets.
What are the problems with ASN.1? It seems to work in 5G and LTE just fine? ASN.1 is simply a way to have portable data structures — are your problems with ASN.1 that it cannot effectively implement portable data structures or is your problem with the systems that have been built upon ASN.1?
I seem to recall the issue was with the tooling around generating clients and servers and preserving type information - but it's been almost 20 years and I don't have my notes on hand.
I also think there were some nfs security issues relating to asn.1
I also think there were some nfs security issues relating to asn.1
Might have been something along the lines of this:
https://www.infosecmatter.com/metasploit-module-library/?mm=...
I guess that (and related errors in openssl and bouncy castle) might not be due to problems with asn.1 per se, but rather difficulty in writing safe libraries for parsing in C.
https://www.infosecmatter.com/metasploit-module-library/?mm=...
I guess that (and related errors in openssl and bouncy castle) might not be due to problems with asn.1 per se, but rather difficulty in writing safe libraries for parsing in C.
But if you want better performance, you use active mode. In active mode, the runtime is receiving data on your behalf, as fast as possible, and sending it to the socket's owning process (think a goroutine) just as fast. Data is often waiting there for you, not just already in user space, but in your Elixir process's mailbox. (Also, this doesn't block your process the way recv/2 does, so you could handle other messages to this process.)
You could imagine doing something similar with 2 goroutines and a channel. Where 1 goroutine is constantly recv from the socket and writing to a buffered channel, and the other is processing the data.
One problem with active mode, and to some degree how messages work in Elixir in general, is that there's no back pressure. Messages can accumulate much faster than you're able to process them. So, instead of active mode, you can use "once" mode or {active, N} mode (once being like {active, 1}. In these modes you get N messages sent to you, at which point it turns into passive mode so you can recv manually. You can put the socket into active/once mode at any point in the future.