Using io_uring to make a high-performance finger server(drewdevault.com)
drewdevault.com
Using io_uring to make a high-performance finger server
https://drewdevault.com/2021/05/24/io_uring-finger-server.html
16 comments
> I wonder what author has against Zig (probably at least current dependence on LLVM)
Here are some opinions shared by ddevault: https://news.ycombinator.com/item?id=15494222
Here are some opinions shared by ddevault: https://news.ycombinator.com/item?id=15494222
Those opinions might've changed:
> I’m interested to see how it measures up under more typical workloads. People keep asking me what I think about Zig in general, and I think it has potential, but I also have a lot of complaints. It’s not likely to replace C for me, but it might have a place somewhere in my stack.
https://drewdevault.com/2020/01/08/Re-Slow.html
> I’m interested to see how it measures up under more typical workloads. People keep asking me what I think about Zig in general, and I think it has potential, but I also have a lot of complaints. It’s not likely to replace C for me, but it might have a place somewhere in my stack.
https://drewdevault.com/2020/01/08/Re-Slow.html
> Please do not share this website with others until we believe it's ready for broader distribution.
Ooops. I sometimes totally ignore all those notices, because of all the cookie and consent garbage. But I guess if one does a blog post using the language it probably is somewhat ready for distribution.
Drew has been mentioning stuff about it increasingly much, but has still never posted the name of the project or any links to a website for it or its code publicly; only to people that are getting involved in it. I didn’t know its name or where it was before today.
Sorry! I can't edit the comment now, but at least it is not on the front page anymore.
I know that it is miles different then posting direct link, because people are averse to any kind of work, but I found the site in the first search. Searching for "drew devault programming language" gives me the language homepage as the 5th result.
EDIT: Also looking on the blog page which is the result from the search mentioned above there is no warning there, that's also why I didn't payed much attention to the warning, as I just stripped the path leaving only the hostname part. Sorry again.
I know that it is miles different then posting direct link, because people are averse to any kind of work, but I found the site in the first search. Searching for "drew devault programming language" gives me the language homepage as the 5th result.
EDIT: Also looking on the blog page which is the result from the search mentioned above there is no warning there, that's also why I didn't payed much attention to the warning, as I just stripped the path leaving only the hostname part. Sorry again.
The issue with most Better C languages is that they focus on ABI rather than API compatibility and throw away C header parsing because no one likes libc.
With cross-platform standards like OpenGL, POSIX, Vulkan the API is declared in terms of high-level ANSI C or C99 language constructs and binary constants may be unspecified and left up to vendor.
Ideally new language designers which don't want to support H files would propose an even simpler header file format for listing procedures, enumerations, structs, constants that existing C compilers could also support. Library vendors don't have time to generate header bindings for every new language using language specific constructs, and application developers don't have time to manually write bindings for large libraries when testing new languages.
Zig supports C header parsing, but as you mention is based on LLVM. It also allows defining new functions inside structs and return values, which can result in heavily nested code that some programmers may not be a fan of.
So there is theoretically some room for another language which maintains compatibility with C headers (or defines an even simpler header format which can be supported by C compilers), which does not use LLVM, and which maintains stronger conceptual separation between namespaces and structs.
With cross-platform standards like OpenGL, POSIX, Vulkan the API is declared in terms of high-level ANSI C or C99 language constructs and binary constants may be unspecified and left up to vendor.
Ideally new language designers which don't want to support H files would propose an even simpler header file format for listing procedures, enumerations, structs, constants that existing C compilers could also support. Library vendors don't have time to generate header bindings for every new language using language specific constructs, and application developers don't have time to manually write bindings for large libraries when testing new languages.
Zig supports C header parsing, but as you mention is based on LLVM. It also allows defining new functions inside structs and return values, which can result in heavily nested code that some programmers may not be a fan of.
So there is theoretically some room for another language which maintains compatibility with C headers (or defines an even simpler header format which can be supported by C compilers), which does not use LLVM, and which maintains stronger conceptual separation between namespaces and structs.
LLVM is about to become an optional component of Zig once the self-hosted compiler is done.
https://kristoff.it/blog/zig-new-relationship-llvm/
https://kristoff.it/blog/zig-new-relationship-llvm/
What is the point? We don't need another systems programming language. We've got D, Rust, Zig, and C, and to be honest it just looks like a worse C.
const grent = match (passwd::getgroup(group)) {
void => fmt::fatal("No '{}' group available", group),
gr: passwd::grent => gr,
};
I'm missing some "high level" context about io_uring, I might as well ask here:
- If I want to write a multithreaded program, is it best to have one io_uring per thread?
- Per CPU core?
- Per device or PCIe lane to the device? (It might make sense ...)
- Or one for the whole program? (Will Linux distribute requests across cores and run them in parallel for me?)
- If I'm writing a library that uses io_uring, should I create my own io_uring or offer an interface to add requests to one which the main program creates? (Or perhaps both?)
- If I want to write a multithreaded program, is it best to have one io_uring per thread?
- Per CPU core?
- Per device or PCIe lane to the device? (It might make sense ...)
- Or one for the whole program? (Will Linux distribute requests across cores and run them in parallel for me?)
- If I'm writing a library that uses io_uring, should I create my own io_uring or offer an interface to add requests to one which the main program creates? (Or perhaps both?)
> - If I want to write a multithreaded program, is it best to have one io_uring per thread?
Yes! But it's not quite that simple: Sometimes you might need some of the ordering operations in io_uring, which then will force you to use a separate (possibly dedicated) ring for that. Typical cases might be a database's journal.
If you use multiple threads (or processes) to access the same ring you'll need synchronization around submission / completion (potentially separately). Avoiding that when not necessary is obviously good.
> - Per CPU core?
That'll depend heavily on your program. If you have a program which has tasks running on specific CPUs, yes, that can make sense - but at that point it's basically a subset of the muti-threaded program question.
> - Per device or PCIe lane to the device? (It might make sense ...)
Hm. I think that will effectively also boil down to the per-task thing above. Particularly if you use polling it's good to be on the actual NUMA node the PCIe device is attached too.
> - Or one for the whole program? (Will Linux distribute requests across cores and run them in parallel for me?)
I think that'll depend on the type of device. For e.g. NVMe devices, there'll be multiple hardware queues. The CPU the submitter is on will determine which hardware queue is used (and that in turn will influence the interrupt location, at least by default).
> - If I'm writing a library that uses io_uring, should I create my own io_uring or offer an interface to add requests to one which the main program creates? (Or perhaps both?)
I don't think there's a generic answer to this one. It'll heavily depend on the type of library.
Caveat: I've used io_uring a bunch, read some of the code, but I'm not an authority on it.
Yes! But it's not quite that simple: Sometimes you might need some of the ordering operations in io_uring, which then will force you to use a separate (possibly dedicated) ring for that. Typical cases might be a database's journal.
If you use multiple threads (or processes) to access the same ring you'll need synchronization around submission / completion (potentially separately). Avoiding that when not necessary is obviously good.
> - Per CPU core?
That'll depend heavily on your program. If you have a program which has tasks running on specific CPUs, yes, that can make sense - but at that point it's basically a subset of the muti-threaded program question.
> - Per device or PCIe lane to the device? (It might make sense ...)
Hm. I think that will effectively also boil down to the per-task thing above. Particularly if you use polling it's good to be on the actual NUMA node the PCIe device is attached too.
> - Or one for the whole program? (Will Linux distribute requests across cores and run them in parallel for me?)
I think that'll depend on the type of device. For e.g. NVMe devices, there'll be multiple hardware queues. The CPU the submitter is on will determine which hardware queue is used (and that in turn will influence the interrupt location, at least by default).
> - If I'm writing a library that uses io_uring, should I create my own io_uring or offer an interface to add requests to one which the main program creates? (Or perhaps both?)
I don't think there's a generic answer to this one. It'll heavily depend on the type of library.
Caveat: I've used io_uring a bunch, read some of the code, but I'm not an authority on it.
I tried to use finger in Ubuntu, and it doesn't display UTF-8 correctly. It seems that a patch would fix this but it's been untouched for 6 years https://bugs.launchpad.net/ubuntu/+source/bsd-finger/+bug/46...
Is this because the finger protocol is defined such that it must use ASCII, and not UTF-8? (Honest question, have not used finger in a looooong time)
https://datatracker.ietf.org/doc/html/rfc1288#section-2.2
https://datatracker.ietf.org/doc/html/rfc1288#section-2.2
I think that's more because they used bsd-finger and never patched it, while it was patched on other distributions. bsd-finger probably follows the RFC, but the RFC may not follow usage (especially since it's almost 30 years old now). According to wikipedia the first implementation of UTF-8 was made for Plan 9 in 1992.
[1] https://harelang.org/
[2] https://c9x.me/compile/
[3] https://myrlang.org/
[4] https://suckless.org/