Error payloads in Zig(srcreigh.ca)
srcreigh.ca
Error payloads in Zig
https://srcreigh.ca/posts/error-payloads-in-zig/
40 comments
Thanks Andy.
Would you endorse the pattern I described in my article if the calling code could have other uses for the diagnostic info than just logging?
In my project I decided that the CLI main should be the place with a reporting mechanism. All the internal code is agnostic to how errors are handled. Many errors could be retried, even if they are currently not, and in some cases you also need extra diagnostic data in order to retry. For example if 1/4 plugins fails to load, I would need to know which one failed to ask the user interactively where to find it or whether to skip it before retrying. There's a few different errors like this, and layers between the handling code and the error source code, which is how I ended up with the union(enum) diagnostic keyed by error code pattern.
It seems that "error codes aren't a reporting mechanism", practically speaking, means that internal application code should just log its own error and return a generic error code that bubbles up unhandled out of main. You decided that this wasn't ok for allocation failures and encourage retrying those, which I find to be inspiring, but apparently most types of application errors are fine to essentially log and ignore. So that is confusing.
I love Zig, and look up to you a lot, so thanks for all your efforts over the years!
Would you endorse the pattern I described in my article if the calling code could have other uses for the diagnostic info than just logging?
In my project I decided that the CLI main should be the place with a reporting mechanism. All the internal code is agnostic to how errors are handled. Many errors could be retried, even if they are currently not, and in some cases you also need extra diagnostic data in order to retry. For example if 1/4 plugins fails to load, I would need to know which one failed to ask the user interactively where to find it or whether to skip it before retrying. There's a few different errors like this, and layers between the handling code and the error source code, which is how I ended up with the union(enum) diagnostic keyed by error code pattern.
It seems that "error codes aren't a reporting mechanism", practically speaking, means that internal application code should just log its own error and return a generic error code that bubbles up unhandled out of main. You decided that this wasn't ok for allocation failures and encourage retrying those, which I find to be inspiring, but apparently most types of application errors are fine to essentially log and ignore. So that is confusing.
I love Zig, and look up to you a lot, so thanks for all your efforts over the years!
Do you think it would fundamentally be a mistake to thread data along with error type through the error return path, or is it just that it costs too much say from a calling convention perspective or some other technical reason? Is it related to ownership?
Does the Zig language not have useful exceptions/stacktraces that can be propagated out?
At a high level, all non-trivial programming is composition. And (see principle of encapsulation) the vast majority of errors shouldn't be recovered and just need to be propagated out.
Then, to be useful, errors need enough information to be diagnosed or investigated. It seems like this should have been a straight-forward requirement in the language design.
At a high level, all non-trivial programming is composition. And (see principle of encapsulation) the vast majority of errors shouldn't be recovered and just need to be propagated out.
Then, to be useful, errors need enough information to be diagnosed or investigated. It seems like this should have been a straight-forward requirement in the language design.
Zig has error unions on return types which are basically a special cased (rust enum aka sum type) of the return type with a special cased (c enum aka enumerated type), and stacktraces are only available in certain build modes.
A "diagnostics" pattern has emerged in the community to optionally request extra information about a failure. You can pass a pointer to the diagnostic (it can be on the stack) and get the extra info back. It's just a more explicit version of what would otherwise happen in a language with error payloads.
A "diagnostics" pattern has emerged in the community to optionally request extra information about a failure. You can pass a pointer to the diagnostic (it can be on the stack) and get the extra info back. It's just a more explicit version of what would otherwise happen in a language with error payloads.
> stacktraces are only available in certain build modes.
Minor correction: stack traces are available on all build modes, but different build modes have different defaults. See: std.Options.allow_stack_tracing
Minor correction: stack traces are available on all build modes, but different build modes have different defaults. See: std.Options.allow_stack_tracing
Note that this "diagnostics" pattern is only meant for handling a error locally with potential extra information, or showing a more useful error to a end user of the software. For software bugs, crashes, or developer facing errors, you often don't have to do anything as zig has pretty good error traces by default.
> stacktraces are only available in certain build modes
> zig has pretty good error traces by default
These seem rather conditional. If I need to run release-fast in prod, say, do we loose these error traces (for bugs)?
> zig has pretty good error traces by default
These seem rather conditional. If I need to run release-fast in prod, say, do we loose these error traces (for bugs)?
You do, to a significant extent. Though there is always the option of running ReleaseSafe.
> Does the Zig language not have useful exceptions/stacktraces that can be propagated out?
Yes, it has error return traces.
Yes, it has error return traces.
That is indeed more or less what you commonly do when you need something more than just error codes.
As an example: https://github.com/kristoff-it/ziggy/blob/852053b09a5f8f5b79...
The part about mapping every error kind to different error code in Zig is debatable. It might be useful in some cases maybe (I don't have the confidence to fully exclude it), but at the very least in my experience I never ever needed that.
As an example: https://github.com/kristoff-it/ziggy/blob/852053b09a5f8f5b79...
The part about mapping every error kind to different error code in Zig is debatable. It might be useful in some cases maybe (I don't have the confidence to fully exclude it), but at the very least in my experience I never ever needed that.
In general if you have the (IMO sensible) approach of taking as few dependencies as possible and not treating them like a black box, then for any error you can simply look at the call stack and figure out the problem from reading the code during development.
Outside of that, error codes are useful for debugging code that is running on other people's machines (i.e. in production) for and for reporting reasons.
Outside of that, error codes are useful for debugging code that is running on other people's machines (i.e. in production) for and for reporting reasons.
It wasn't clear from the examples, and the gist doesn't have a `deinit` method, so what happens if an error needs to own data?
> Here, sqlite.ErrorPayload.init saves 500 bytes of error message from sqlite
Who owns those 500 bytes and where are they being freed?
> Here, sqlite.ErrorPayload.init saves 500 bytes of error message from sqlite
Who owns those 500 bytes and where are they being freed?
It's just stored as a [256]u8 in the struct.
// sqlite.zig
pub const ErrorPayload = struct {
message: [256]u8,
pub fn init(db: *c.sqlite3) @This() {
var self = std.mem.zeroes(@This());
var fw = std.Io.Writer.fixed(self.message[0..]);
_ = fw.writeAll(std.mem.span(c.sqlite3_errmsg(db))) catch |err| switch (err) {
error.WriteFailed => return self, // full
};
return self;
}
};And what pattern would you recommend if you needed to allocate?
It's worth every effort to avoid situations where a function creates extra clean up responsibilities to the caller only on error conditions.
If I really needed a large error payload, too big to fit on the stack, I'd probably want to do something like this:
If I really needed a large error payload, too big to fit on the stack, I'd probably want to do something like this:
const errbuf = try alloc.alloc(u8, 1024*1024*1024);
module.setErrorBuf(&errbuf)
defer {
module.setErrorBuf(null);
alloc.free(errbuf);
}
var func_diag = diagnostics.OfFunction(module.func){};
module.func(foo, bar, &func_diag) catch |err| switch (err) {
error.BigPayload => {
const payload = func_diag.get(error.BigPayload);
// The payload can reference the 1MiB of data safely here,
// and it's cleaned up automatically.
}
}The diagnostic struct could contain a caller-provided allocator field which the callee can use, and a deinit() function on the diagnostic struct which frees everything.
Continues to be a point of annoyance that Zig doesn't properly support payloads in errors.
What more do you want than what's covered in the post?
To me, this post is proof zig doesn't need "proper support". You can already have extremely ergonomic error payloads with existing language features.
Earlier version of this post had some language feature ideas, but then I realized Zig already had all the capabilities, so I just removed that section.
For example, I used to think it'd be nice for functions to be a namespace, so I didn't have `myFunc` and `MyFuncDiagnostics`. But then I realized that the Diagnostics type doesn't need its own name; you can just put the type in the function signature, and use a function like `diagnostics.OfFunction(fn)` to extract the type from the function definition, so you can just write this:
Do you want automatic inference of the error payload type? So zig creates a special tagged union error payloads type for you automatically? It seems complicated and maybe not a good idea. Do you really want your function to return an invisible 20 elements union on error? Do you want to call a someone else's function which returns an invisible 20 elements union on error? You know, maybe it's not a good idea.
To me, this post is proof zig doesn't need "proper support". You can already have extremely ergonomic error payloads with existing language features.
Earlier version of this post had some language feature ideas, but then I realized Zig already had all the capabilities, so I just removed that section.
For example, I used to think it'd be nice for functions to be a namespace, so I didn't have `myFunc` and `MyFuncDiagnostics`. But then I realized that the Diagnostics type doesn't need its own name; you can just put the type in the function signature, and use a function like `diagnostics.OfFunction(fn)` to extract the type from the function definition, so you can just write this:
var diag: diagnostics.OfFunction(myFunc) = .{};
const res = myFunc(foo, bar, &diag) catch |err| ...;
Do you want automatic inference of the error payload type? So zig creates a special tagged union error payloads type for you automatically? It seems complicated and maybe not a good idea. Do you really want your function to return an invisible 20 elements union on error? Do you want to call a someone else's function which returns an invisible 20 elements union on error? You know, maybe it's not a good idea.
More than anything I want "Here's the current officially sanctioned best practice of how to report errors with payloads". For a language that's highly opinionated about everything it's strangely unopinionated here and worse off for it because many libraries just swallow useful diagnostic information, some of my own projects included.
There's a barrier to setting up the diagnostic pattern. When you're in the greenfield phase it's easy to search for information about error reporting, discover various different approaches and long threads of people arguing about what better and just say "ah screw it, I have more important things to do right now" and postpone the decision.
Your approach is fine, I don't love how verbose it is but it could probably be tweaked. If this is the way forward then it should be included in stdlib, documented, and promoted as the recommended way of reporting errors.
There's a barrier to setting up the diagnostic pattern. When you're in the greenfield phase it's easy to search for information about error reporting, discover various different approaches and long threads of people arguing about what better and just say "ah screw it, I have more important things to do right now" and postpone the decision.
Your approach is fine, I don't love how verbose it is but it could probably be tweaked. If this is the way forward then it should be included in stdlib, documented, and promoted as the recommended way of reporting errors.
I agree. You understand why I wrote this post. It's what I wanted to read 3 weeks ago. We're told "Use Diagnostics like the json stdlib module!" but then you realize the json module is way too simplistic for a complicated application.
But also, I'm sure this method has flaws and can be greatly improved, so hopefully we can come to the right solution.
But also, I'm sure this method has flaws and can be greatly improved, so hopefully we can come to the right solution.
> You can already have extremely ergonomic error payloads with existing language features.
I think you meant extremely unergonomic. If you take a dev poll, 8/10 developers would not find the solution ergonomic at all.
I think you meant extremely unergonomic. If you take a dev poll, 8/10 developers would not find the solution ergonomic at all.
Are you sure? Just because you feel that way doesn't mean you can just make up a statistic that supports your viewpoint.
In my experience, ergonomics in Zig means something other than ergonomics in many other languages. In plenty of languages, ergonomic code is basically just writing as few characters as possible and being able to achieve complex logic with little boilerplate. In Zig, it feels good to be able to compose the conceptually simple language features into an optimal solution to your problem, even if it results in code that is maybe not aesthetically pleasing on first glance.
I'm not going to declare that this solution is ergonomic or not, you can't always tell just by looking at Zig code whether using it is ergonomic, but by that same logic we shouldn't dismiss it off hand.
In my experience, ergonomics in Zig means something other than ergonomics in many other languages. In plenty of languages, ergonomic code is basically just writing as few characters as possible and being able to achieve complex logic with little boilerplate. In Zig, it feels good to be able to compose the conceptually simple language features into an optimal solution to your problem, even if it results in code that is maybe not aesthetically pleasing on first glance.
I'm not going to declare that this solution is ergonomic or not, you can't always tell just by looking at Zig code whether using it is ergonomic, but by that same logic we shouldn't dismiss it off hand.
> In my experience, ergonomics in Zig means something other than ergonomics in many other languages.
So an opinion...
So an opinion...
Obviously
I thought so too at first, coming from a language (Hare) where they are very easy and common, but the Diagnostics pattern isn't that bad once you expect it. Various examples: https://ziggit.dev/search?q=Diagnostics
[deleted]
You can't really easily have useful user defined payloads in errors without implicit allocations.
Best way to do it is to pass a payload pointer into the function call. You can put it into an options struct with a default null pointer that noops if you prefer the ergonomics of a kwarg.
Best way to do it is to pass a payload pointer into the function call. You can put it into an options struct with a default null pointer that noops if you prefer the ergonomics of a kwarg.
Right. It's one thing to build the equivalent of Result into the language -- great. It's another to make it only support simple enum variants and not be extensible.
[deleted]
Why need error payloads when:
You can do validation at user interface and report actual nice errors.
And if something happens after that, you can save the stack trace into somewhere so the developer can see it. And you report unexpected error to user as that is exactly that
You can do validation at user interface and report actual nice errors.
And if something happens after that, you can save the stack trace into somewhere so the developer can see it. And you report unexpected error to user as that is exactly that
What zig really needs is a stack trace that works properly on release mode. And a way to collect the stack trace into a string at runtime so it can be saved as a log or into a db
Alternative title; instead of learning a new language, I adapted it into something I'm more familiar with.
Depending on your language preference; Zig has issues[citation needed], but 1) it's still version 0 and 2) and this is the important part: who cares?
I get you have a pattern and shape you like a lot. But there's less value in that existing shape, than there is in being mildly uncomfortable, and expanding your world view and trying things in ways you wouldn't normally do.
If you tried it and didn't like it, cool, go back to the language you're already used to. But for everyone else. I'd encourage you to try doing things "wrong" for a while and seeing if you can't learn something from forcing yourself to do so.
Something especially true for a language that HN likes to pretend is just a toy and can't compete with [ language you already decided won ]
Depending on your language preference; Zig has issues[citation needed], but 1) it's still version 0 and 2) and this is the important part: who cares?
I get you have a pattern and shape you like a lot. But there's less value in that existing shape, than there is in being mildly uncomfortable, and expanding your world view and trying things in ways you wouldn't normally do.
If you tried it and didn't like it, cool, go back to the language you're already used to. But for everyone else. I'd encourage you to try doing things "wrong" for a while and seeing if you can't learn something from forcing yourself to do so.
Something especially true for a language that HN likes to pretend is just a toy and can't compete with [ language you already decided won ]
This is just a "complex real world app code" extension of the stdlib Diagnostics pattern.
% rg Diagnostics zig/lib/std | wc -l
165
page is dead
The pattern here is to return something like error.Diagnostics for all errors that have been reported via diagnostics.
The only reason you'd have a different error code would be if the control flow should be different. For example error.OutOfMemory makes sense to be separate because it's a retryable error, which means it should be handled by different control flow.