SDS: Simple Dynamic Strings library for C(github.com)
github.com
SDS: Simple Dynamic Strings library for C
https://github.com/antirez/sds
47 comments
I'm surprised this is aliased to char*, not const char*. The benefit of the aliasing is convenience, but the main risk is absent-mindedly passing it to a libc function that modifies the string without updating the SDS metadata. Const would result in a compiler warning while letting the intended use cases (e.g., the printf example) work fine.
The only thing the SDS metadata holds is the string's length. Just like how you'd have to realloc() a regular string before using strcat(), you have to sdsgrowzero() an sds string before using strcat().
Basically, standard libc functions that tamper with the string have the same constraints as malloc()ed strings in terms of safety, only you might want to call sdsupdatelen() after truncating a string.
Hi! The Redis tree contains more advanced versions of this library. Most of the development continued there, eventually.
It might be worth extracting it back out, it seems pretty useful.
Indeed, but in some way the Redis version is a bit too Redis-ish, that is, memory saving concerns are taken to the extreme instead of having a more balanced approach about simplicity. In my YouTube channel C course, I'm showing something similar to SDS in the latest lessons, and I may use SDS again in later course in order to show how to integrate back the useful features that diverged. Maybe an SDS3 maybe a middle ground among the Redis version, some API error that should be corrected (but not in Redis: not worth it), and other improvements.
This sounds intriguing. Do you have a link to your C course?
It's in Italian, you have to use subtitles:https://youtube.com/playlist?list=PLrEMgOSrS_3cFJpM2gdw8EGFy...
Hi! Are there no performance penalties that come from alignment issues? Or are your prefix structs aligned to cache line sizes?
Thank you for creating sds, btw. Very useful to have it on the toolbelt.
Oh, and redis. That too. :)
Oh, and redis. That too. :)
.. and LOAD81! So much fun!
Related:
Simple Dynamic Strings library for C, compatible with null-terminated strings - https://news.ycombinator.com/item?id=21692400 - Dec 2019 (83 comments)
Simple Dynamic Strings library for C - https://news.ycombinator.com/item?id=7190664 - Feb 2014 (127 comments)
Simple Dynamic Strings library for C, compatible with null-terminated strings - https://news.ycombinator.com/item?id=21692400 - Dec 2019 (83 comments)
Simple Dynamic Strings library for C - https://news.ycombinator.com/item?id=7190664 - Feb 2014 (127 comments)
Was:
s = sdscat(s,"Some more data");
stscat(&s, "Some more data");
enums > defines
Why not just use C++ strings and string_views? So weird to see this masochistic obsession some people have with doing everything in plain C.
It's 2026, there are better, more memory safe, more efficient solutions out there.
It's 2026, there are better, more memory safe, more efficient solutions out there.
To actually answer your question (beyond the snark/appeal to authority replies you’ve already gotten), there are a couple good reasons:
— You're working in embedded development (but somehow need a full-fledged dynamic string library).
— While it's true that C++ is (almost) a strict superset of C, and “you don’t pay for what you don’t use” is a good rule of thumb, it can be very hard to restrict a team of developers to eschew all that complexity you dearly pay for and treat C++ as “C with classes and the STL.” Without very strict coding standards (and a means of enforcing them), letting a team of developers use C++ is often opening a Pandora's Box of complex, obscure language features. Restricting a project to plain old C heads that off at the pass.
— You're working in embedded development (but somehow need a full-fledged dynamic string library).
— While it's true that C++ is (almost) a strict superset of C, and “you don’t pay for what you don’t use” is a good rule of thumb, it can be very hard to restrict a team of developers to eschew all that complexity you dearly pay for and treat C++ as “C with classes and the STL.” Without very strict coding standards (and a means of enforcing them), letting a team of developers use C++ is often opening a Pandora's Box of complex, obscure language features. Restricting a project to plain old C heads that off at the pass.
> You're working in embedded development (but somehow need a full-fledged dynamic string library).
The situation isn't all that implausible: e.g., many ESP32-based devices want to work with strings to interface with HTTP servers, and they do have C++ support, but the size limit is small enough that you can easily bump your head into it if you aren't careful.
The situation isn't all that implausible: e.g., many ESP32-based devices want to work with strings to interface with HTTP servers, and they do have C++ support, but the size limit is small enough that you can easily bump your head into it if you aren't careful.
Or anything processing JSON— it's nice to be able to get string views directly into the original payload without having to copy them into fixed size buffers elsewhere.
> and the STL
Even this has a lot of "payment" for what you don't use. Even some C++ libraries forbid it just because of the size of debug symbols.
Even this has a lot of "payment" for what you don't use. Even some C++ libraries forbid it just because of the size of debug symbols.
As soon as you move to "C with classes and the STL" you've now also bought into exceptions, as the STL is not even remotely ergonomic with exceptions disabled.
> SDS was a C string I developed in the past for my everyday C programming needs, later it was moved into Redis where it is used extensively and where it was modified in order to be suitable for high performance operations. Now it was extracted from Redis and forked as a stand alone project.
Seems like everyone wants to believe they’re as skilled and hardcore as the kernel devs. In reality, I agree - C++ is basically a superset of C and the whole point of “you don’t pay for what you don’t use” is to be able to avoid ridiculous situations like these.
> "you don’t pay for what you don’t use"
In my experience (mostly embedded development) including C++ in a C project adds a lot of build complexity and build time, whereas C99 or C89 is trivial to install in pretty much all situations
In my experience (mostly embedded development) including C++ in a C project adds a lot of build complexity and build time, whereas C99 or C89 is trivial to install in pretty much all situations
This is the guy that created Redis. I would look at his repos in a different way.
If your code is plain C, then anyone can extend it with, or embed it inside, code of literally any other language; and in so doing, they will have full access/exposure to everything in your codebase — all the same stuff that they would if they were writing their host/extension code in C.
This is not true of C++ (or most other languages):
• C++ has a runtime (however minimal); and so, by including any C++ code in a codebase, you're making it much more difficult to link/embed the resulting code — you now have to also dynamically link the C++ runtime, and ensure that your host code spins it up "early", before any of the linked C++ code gets to run. (This may even be impossible in some host languages!)
• Also, even if there was no associated runtime to deal with, C++ isn't wholly C-FFI-clean. All the stuff that people like about C++ — all the reasons you'd want to use C++ — result in codebases that aren't cleanly C-FFI exposable, due to name mangling, functions taking parameters with non-C-exportable types, methods + closures not being C-FFI thunkable [and functions returning those], etc.
• And even if you bite that bullet, and write your library in C++ but carefully wrap its API to give it C-FFI-clean linkage (usually via a hybrid C / C++ project), this still introduces a layer of FFI runtime overhead. When another non-C language consumes your code, it's then getting double FFI overhead — a call from its code to yours has to convert from its abstractions, to C's abstractions, to C++'s abstractions, and back. (This is why you don't tend to see e.g. non-C++ projects embedding LLVM, or LLVM being extended with non-C++ passes, despite LLVM being designed in this "C wrapper around a C++ core" style.)
C is one of the only languages with a zero-impedance-mismatch, zero-overhead default or forced binding of external symbols to the C FFI (i.e. the C set of platform ABIs + C symbol naming standard.)
The others that do this are: C3 (https://c3-lang.org/); Zig, unless you do weird things on purpose, and... that's really it. Everything else has the same two problems as C++ outlined above.
Even Rust, even Odin, etc. only provide C-FFI linkage as an opt-in feature; and they do nothing to incentivize use of it; and so, of course, due to their useful non-C-FFI-clean features, developers are disincentivized from ever enabling it before they "need" it. So in practice, most libraries in those languages are not consumable from C [or other C-FFI-compatible languages] — and most software in those languages are not extendable in C [or another C-FFI-compatible language] — without extra effort on the upstream's part to add explicit support for doing that. And most upstreams don't bother.
Writing software in C itself, is essentially a way for a project to "tie itself to the mast" and commit to its ABI always being C-FFI clean; such that it can be consumed not only from C, but also from any other language a project might use that supports importing C-FFI libraries. (Which is most languages.)
This is not true of C++ (or most other languages):
• C++ has a runtime (however minimal); and so, by including any C++ code in a codebase, you're making it much more difficult to link/embed the resulting code — you now have to also dynamically link the C++ runtime, and ensure that your host code spins it up "early", before any of the linked C++ code gets to run. (This may even be impossible in some host languages!)
• Also, even if there was no associated runtime to deal with, C++ isn't wholly C-FFI-clean. All the stuff that people like about C++ — all the reasons you'd want to use C++ — result in codebases that aren't cleanly C-FFI exposable, due to name mangling, functions taking parameters with non-C-exportable types, methods + closures not being C-FFI thunkable [and functions returning those], etc.
• And even if you bite that bullet, and write your library in C++ but carefully wrap its API to give it C-FFI-clean linkage (usually via a hybrid C / C++ project), this still introduces a layer of FFI runtime overhead. When another non-C language consumes your code, it's then getting double FFI overhead — a call from its code to yours has to convert from its abstractions, to C's abstractions, to C++'s abstractions, and back. (This is why you don't tend to see e.g. non-C++ projects embedding LLVM, or LLVM being extended with non-C++ passes, despite LLVM being designed in this "C wrapper around a C++ core" style.)
C is one of the only languages with a zero-impedance-mismatch, zero-overhead default or forced binding of external symbols to the C FFI (i.e. the C set of platform ABIs + C symbol naming standard.)
The others that do this are: C3 (https://c3-lang.org/); Zig, unless you do weird things on purpose, and... that's really it. Everything else has the same two problems as C++ outlined above.
Even Rust, even Odin, etc. only provide C-FFI linkage as an opt-in feature; and they do nothing to incentivize use of it; and so, of course, due to their useful non-C-FFI-clean features, developers are disincentivized from ever enabling it before they "need" it. So in practice, most libraries in those languages are not consumable from C [or other C-FFI-compatible languages] — and most software in those languages are not extendable in C [or another C-FFI-compatible language] — without extra effort on the upstream's part to add explicit support for doing that. And most upstreams don't bother.
Writing software in C itself, is essentially a way for a project to "tie itself to the mast" and commit to its ABI always being C-FFI clean; such that it can be consumed not only from C, but also from any other language a project might use that supports importing C-FFI libraries. (Which is most languages.)
C also has a (granted very small on Unix) runtime[1]. On windows the C runtime is a bit larger, since windows processes get a single string for their arguments, which must be parsed into argc/argv.
As far as "you have to also dynamically link the C++ runtime", try calling malloc from two different libc implementations in the same process and see "interesting" things happen. Even more interesting is calling free() on a pointer that was malloc'd from a different C library.
1: E.g. for musl https://git.musl-libc.org/cgit/musl/tree/crt
As far as "you have to also dynamically link the C++ runtime", try calling malloc from two different libc implementations in the same process and see "interesting" things happen. Even more interesting is calling free() on a pointer that was malloc'd from a different C library.
1: E.g. for musl https://git.musl-libc.org/cgit/musl/tree/crt
C++ has a runtime (however minimal);
No it doesn't.
Also, even if there was no associated runtime to deal with, C++ isn't wholly C-FFI-clean
Yes it is, you just extern "C" whatever you want.
All the stuff that people like about C++ — all the reasons you'd want to use C++ — result in codebases that aren't cleanly C-FFI exposable
Not true at all, the biggest two things, destructors and move semantics you still have everywhere except for the boundaries with C.
And even if you bite that bullet, and write your library in C++ but carefully wrap its API to give it C-FFI-clean linkage (usually via a hybrid C / C++ project), this still introduces a layer of FFI runtime overhead
There is no overhead here, it is not different from C.
I don't know where all this comes from, but I doubt it comes from heavy experience with modern C++.
No it doesn't.
Also, even if there was no associated runtime to deal with, C++ isn't wholly C-FFI-clean
Yes it is, you just extern "C" whatever you want.
All the stuff that people like about C++ — all the reasons you'd want to use C++ — result in codebases that aren't cleanly C-FFI exposable
Not true at all, the biggest two things, destructors and move semantics you still have everywhere except for the boundaries with C.
And even if you bite that bullet, and write your library in C++ but carefully wrap its API to give it C-FFI-clean linkage (usually via a hybrid C / C++ project), this still introduces a layer of FFI runtime overhead
There is no overhead here, it is not different from C.
I don't know where all this comes from, but I doubt it comes from heavy experience with modern C++.
No, it does. "The only two features in the language that do not follow the zero-overhead principle are runtime type identification and exceptions, and are why most compilers include a switch to turn them off." - https://en.cppreference.com/w/cpp/language/Zero-overhead_pri...
So saying 'it has a runtime' doesn't really make sense, it has a runtime if you want for two features that aren't necessary.
When most people refer to Java, they're referring to the "Java SE Platform" (the one that ships with all the stdlib stuff you'd expect, requiring a ton of process-boot-time logic and runtime support threads); not the "Java SE Embedded Platform" (the one you'd use to write e.g. Java Card software for smartcards, that doesn't have all of that stuff.) "Java for the Java SE Platform" and "Java for the Java SE Embedded Platform" are essentially two different languages, with the latter being a strict subset of the former.
When most people refer to C++, they're referring to the platform you get by linking the C++ STL. There's also what you might informally call "RTOS kernel C++" — C++ with all its syntax features (exceptions and destructors and RTTI) but no STL — that you get by linking only to a core C++ runtime support libraries (libsupc++/libc++abi/etc); and there's also "ultra-low-power embedded C++", where you don't even link to these, and don't get to use those features, so you instead have to use placement `new` and manual destructor calls, do your own error-handling with sentinel values or tagged unions or what-have-you, etc. These platforms are, again, essentially their own languages, each a strict subset of the one before.
Just like you can't hire a random Java programmer and expect them to be immediately productive on a Java Card codebase, C++ programmers are not "ultra-low-power embedded C++" programmers, any more than C# or Objective-C programmers are C programmers, or any more than C programmers are assembly-language programmers.
---
You might rebut with "C++ has a lot of subsets; it's a language with a lot of features that people can take or leave; people still call all of these subsets C++."
Yes, people do get used to thinking of C++ as "a language that has a bunch of different features that you can choose to use or not", and that mental-schema inertia carries over into this case — but it doesn't / shouldn't actually be applied here, and people are wrong to do so.
In the case of other C++ features, if you're not using a feature, it's because you just don't need what it does. When you choose to constrain your use of most C++ features, this results in the codebase being easier to understand. For most C++ features, avoiding use of the feature reduces the experience barrier required to begin contributing to the codebase. Which is why the subsets of C++ that don't use these features, are still just "C++": anyone who is a "C++ programmer" can instantly and intuitively maintain a codebase that is written in one of these subsets of C++.
But in the case of the syntax features requiring C++ runtime support, if you're strictly adhering to not using those... then you're having to do far more onerous and esoteric stuff (placement new + manual destructing; C-like error handling; BYO type metadata with intermediate void-ptr casting) instead. And that's stuff they don't even teach you how to do in a C++ course, or even a very thick C++ textbook.
As a "C++ programmer" cannot be expected to code for this "ultra-low-power embedded C++", you may as well consider it a separate language.
---
And if you agree with that, then you should agree that it makes sense to claim that C++ has a runtime. It is only the separate language, "ultra-low-power embedded C++", that doesn't.
When most people refer to C++, they're referring to the platform you get by linking the C++ STL. There's also what you might informally call "RTOS kernel C++" — C++ with all its syntax features (exceptions and destructors and RTTI) but no STL — that you get by linking only to a core C++ runtime support libraries (libsupc++/libc++abi/etc); and there's also "ultra-low-power embedded C++", where you don't even link to these, and don't get to use those features, so you instead have to use placement `new` and manual destructor calls, do your own error-handling with sentinel values or tagged unions or what-have-you, etc. These platforms are, again, essentially their own languages, each a strict subset of the one before.
Just like you can't hire a random Java programmer and expect them to be immediately productive on a Java Card codebase, C++ programmers are not "ultra-low-power embedded C++" programmers, any more than C# or Objective-C programmers are C programmers, or any more than C programmers are assembly-language programmers.
---
You might rebut with "C++ has a lot of subsets; it's a language with a lot of features that people can take or leave; people still call all of these subsets C++."
Yes, people do get used to thinking of C++ as "a language that has a bunch of different features that you can choose to use or not", and that mental-schema inertia carries over into this case — but it doesn't / shouldn't actually be applied here, and people are wrong to do so.
In the case of other C++ features, if you're not using a feature, it's because you just don't need what it does. When you choose to constrain your use of most C++ features, this results in the codebase being easier to understand. For most C++ features, avoiding use of the feature reduces the experience barrier required to begin contributing to the codebase. Which is why the subsets of C++ that don't use these features, are still just "C++": anyone who is a "C++ programmer" can instantly and intuitively maintain a codebase that is written in one of these subsets of C++.
But in the case of the syntax features requiring C++ runtime support, if you're strictly adhering to not using those... then you're having to do far more onerous and esoteric stuff (placement new + manual destructing; C-like error handling; BYO type metadata with intermediate void-ptr casting) instead. And that's stuff they don't even teach you how to do in a C++ course, or even a very thick C++ textbook.
As a "C++ programmer" cannot be expected to code for this "ultra-low-power embedded C++", you may as well consider it a separate language.
---
And if you agree with that, then you should agree that it makes sense to claim that C++ has a runtime. It is only the separate language, "ultra-low-power embedded C++", that doesn't.
It's easy to claim generalities and from there claim things that must be true, but when it comes down to it, C++ enables things that just can't be done without changing. Turning off a feature is trivial, adding destructors, move semantics and being able to wrap generic data structures up so they have value semantics is just not something you can do with C.
Saying "people generally mean this" is not only not true it isn't any sort of technical argument.
Saying "people generally mean this" is not only not true it isn't any sort of technical argument.
> C++ has a runtime (however minimal)
I'm not familiar with this, are you able to explain it? Do you mean something analogous to _start?
I'm not familiar with this, are you able to explain it? Do you mean something analogous to _start?
It's the stuff you find in LLVM's libc++abi (https://libcxxabi.llvm.org/spec.html) or GCC's libsupc++ (https://wiki.osdev.org/Libsupcxx).
• These libraries provide the set of functions that compiled C++ code expects to call into to implement syntax-level features, like exception unwinding, destructors, and runtime type inference (RTTI).
• These libraries also define the C++ equivalents of the C runtime's _init and _fini functions. The C++ versions do a lot more: they set up the exception unwinder (copying and fixing-up .data-section DWARF tables into exception-unwind lookup tables for the active .text-section address-space mappings); they register global destructors; they register signal handlers (and per-thread-create handlers to re-register those signal handlers), to make the particular signal received by https://man7.org/linux/man-pages/man3/pthread_cancel.3.html trigger exception-unwind and destructors; ...etc.
Usually, a C++ runtime support library gets static-linked into a C++ stdlib; and then a program implicitly receives linkage to the C++ runtime support library as part of statically or dynamically linking to the C++ stdlib.
However, if you want a "minimal" C++ binary — and you never use anything in `std` (maybe because you're just using C++ to be able to write "C with templates") — then you can just link only the C++ runtime support library itself.
(And yes, all of these C++ syntax features that require runtime support are technically optional. You can just `-fno-exceptions -fno-rtti -ffreestanding -fno-asynchronous-unwind-tables -fno-unwind-tables`, and then never link a C++ runtime support library at all — if you're willing to only use placement `new` and never use destructors or exceptions or RTTI. But at that point you're not exactly writing C++; you're writing an "embedded profile" of C++, akin to targeting the Java SE Embedded platform. Which makes sense if you're coding e.g. an RTOS kernel; but goes against the much of the point if you're just writing e.g. the "native loadable module" for an HLL ecosystem package. Now your code can't depend on arbitrary third-party C++ code [because there isn't an "embedded C++" ecosystem, only a regular C++ ecosystem]; you can't find any FOSS contributors willing to help you maintain it [because 99% of people only know C++, not "embedded C++"]; and so on.)
• These libraries provide the set of functions that compiled C++ code expects to call into to implement syntax-level features, like exception unwinding, destructors, and runtime type inference (RTTI).
• These libraries also define the C++ equivalents of the C runtime's _init and _fini functions. The C++ versions do a lot more: they set up the exception unwinder (copying and fixing-up .data-section DWARF tables into exception-unwind lookup tables for the active .text-section address-space mappings); they register global destructors; they register signal handlers (and per-thread-create handlers to re-register those signal handlers), to make the particular signal received by https://man7.org/linux/man-pages/man3/pthread_cancel.3.html trigger exception-unwind and destructors; ...etc.
Usually, a C++ runtime support library gets static-linked into a C++ stdlib; and then a program implicitly receives linkage to the C++ runtime support library as part of statically or dynamically linking to the C++ stdlib.
However, if you want a "minimal" C++ binary — and you never use anything in `std` (maybe because you're just using C++ to be able to write "C with templates") — then you can just link only the C++ runtime support library itself.
(And yes, all of these C++ syntax features that require runtime support are technically optional. You can just `-fno-exceptions -fno-rtti -ffreestanding -fno-asynchronous-unwind-tables -fno-unwind-tables`, and then never link a C++ runtime support library at all — if you're willing to only use placement `new` and never use destructors or exceptions or RTTI. But at that point you're not exactly writing C++; you're writing an "embedded profile" of C++, akin to targeting the Java SE Embedded platform. Which makes sense if you're coding e.g. an RTOS kernel; but goes against the much of the point if you're just writing e.g. the "native loadable module" for an HLL ecosystem package. Now your code can't depend on arbitrary third-party C++ code [because there isn't an "embedded C++" ecosystem, only a regular C++ ecosystem]; you can't find any FOSS contributors willing to help you maintain it [because 99% of people only know C++, not "embedded C++"]; and so on.)
Redis started in 2009, and this library was started there. string_view didn't appear until C++17.
I switched to C because I could not stand the pain of using C++ anymore. I find C refreshingly simple.
(Also, as a comment to other responses: C++ is not a superset of C, it is a fork from 95 with divergent language evolution since then).
(Also, as a comment to other responses: C++ is not a superset of C, it is a fork from 95 with divergent language evolution since then).
It is not a fork from 95. Cfront, the original C++ compiler was a new front-end for the C compiler from 1983 that output C which was then compiled regularly.
https://en.wikipedia.org/wiki/Cfront
https://en.wikipedia.org/wiki/Cfront
Maybe, but it is 1995 when it diverged: https://isocpp.org/wiki/faq/c Before it was an extension since then it is a fork.
There are real downsides to even #including C++ headers. And there are certainly downsides to introducing a templated string type. It's not hard to imagine why people would want another solution.
How am I supposed to use C++ strings and string_views in C?
I guess those stuck on MSVC might have this perception, but newer C standards have added plenty of niceties. Unsure if the claim that Cpp is safer is correct.
Someone had to create C++ strings so C++ developers could use them. What's wrong with someone doing the same for C so C developers can use it?
But if your in C...