Quartz: A Deterministic Time Testing Library for Go(coder.com)
coder.com
Quartz: A Deterministic Time Testing Library for Go
https://coder.com/blog/introducing-quartz
51 comments
Of possible interest to anyone testing concurrent Go code in general and time in particular, a proposed standard library package: https://go.dev/issue/67434
I still don't like that it assumes global monotonic time. It is completely possible for an event to be scheduled in 100ms on one thread and in 200ms in another, and for the latter to run and/or finish first. A time testing system ought to let you test that case, because you're going to hit it in production.
For that you really have no choice but to abandon the time package's API anyhow because you're going to need some way to trigger those things out-of-order.
I've been using this for a while now: https://github.com/thejerf/abtime (my package) and it really lets me get down and dirty with order of operations, when that matters.
For that you really have no choice but to abandon the time package's API anyhow because you're going to need some way to trigger those things out-of-order.
I've been using this for a while now: https://github.com/thejerf/abtime (my package) and it really lets me get down and dirty with order of operations, when that matters.
I think the purpose of this package is simple - to allow tests to use time.Sleep without having them actually sleep. A lot of things will not be tested with this model - e.g. you can't really test correctness in all traces of events in your concurrent program - but that goal itself is much harder to achieve.
Related: the behavior of time in this proposal was implemented in Go playground more than a decade ago: https://go.dev/blog/playground#faking-time. Unfortunately it was never exposed.
Thanks for sharing this. This is one of the best testing features I've ever seen - I've had so many troubles with timing tests and refactoring around them. I really hope this gets merged.
That looks amazing. Advancing time while things are idle sounds practically perfect, and that's essentially impossible to build as a library. As much as I like the sound of Quartz (the issues they call out are real and very painful and I was getting close to building my own version to solve some), that issue might remove my need for it entirely.
I'm curious if the author of Quartz has seen this proposal and could provide a comparison?
I'm Quartz's primary author, and I hadn't seen this proposal until today.
They are similar in the sense of using a faked clock to make things deterministic. Other than not having to inject the Clock interface, tests using Quartz will probably look fairly similar to this proposal in many cases.
The proposal has a simpler API than Quartz, being able to just run some goroutines and wait until everything is blocked.
However, this brings some limitations on what can be tested, such as time elapsing while computation is occurring (not just elapsing while things are waiting for something else). IMO this is an important limitation, as many interesting time-of-check, time-of-use bugs and edge cases occur because time does elapse during computation in real systems.
An example of this from coder/coder that motivated the design of Quartz's API: https://github.com/coder/coder/blob/a5e4bf38fec66c5e7ecc96b0...
They are similar in the sense of using a faked clock to make things deterministic. Other than not having to inject the Clock interface, tests using Quartz will probably look fairly similar to this proposal in many cases.
The proposal has a simpler API than Quartz, being able to just run some goroutines and wait until everything is blocked.
However, this brings some limitations on what can be tested, such as time elapsing while computation is occurring (not just elapsing while things are waiting for something else). IMO this is an important limitation, as many interesting time-of-check, time-of-use bugs and edge cases occur because time does elapse during computation in real systems.
An example of this from coder/coder that motivated the design of Quartz's API: https://github.com/coder/coder/blob/a5e4bf38fec66c5e7ecc96b0...
This looks infinitely preferable.
What do you like better about it?
--- (btw: just read your blog on R -- love it)
--- (btw: just read your blog on R -- love it)
It feels like every second, sorry every other time-related library is called Quartz.
not a great name considering Quartz schedular for java...
In particular, there is also another library for task scheduling named go-quartz: https://github.com/reugn/go-quartz.
Naming is indeed hard.
Naming is indeed hard.
Or Quartz Composer, the rendering thingy for macOS
Or for .NET
only on HN would a .NET comment get downvoted
Why would you say that? It’s an archaic framework which sticks around because it’s “good enough” for anything with a complexity low enough to be handled by its “magic”. As soon as you need something to scale or handle real complexity where you have to step outside all the comfort zone .net developers tend to operate in, it’s one of the worst frameworks that will constantly get in your way.
It’s also an ecosystem riddled with traps that is still so stuck in its OOP past that very few programming languages operating on it are even capable of having functions without a class object.
Don’t get me wrong. It’s a good language when you’re building relatively simple backend APIs that are small enough that they can actually use things like entity framework. Even more so if you’re putting them into Azure and letting your security be handled by EntraID.
But how many people on HN do you expect that is?
It’s also an ecosystem riddled with traps that is still so stuck in its OOP past that very few programming languages operating on it are even capable of having functions without a class object.
Don’t get me wrong. It’s a good language when you’re building relatively simple backend APIs that are small enough that they can actually use things like entity framework. Even more so if you’re putting them into Azure and letting your security be handled by EntraID.
But how many people on HN do you expect that is?
This may be your personal experience but it is used successfully and happily by many in a variety of domains way different to "boring OOP back-ends". No popular general purpose language is confined to a single scenario the way you describe (even Swift and Kotlin are expanding). C# lets you do things like writing a competitive Redis implementation[0], advanced physics engine that beats Jolt[1] or a game engine that does not rely on C++ or C[2], cross-platform applications[3] and so on and so forth. It is possible to successfully do both high level FP style approach for business domain modeling and very low-level data crunching that is competitive with systems programming languages.
[0]: https://microsoft.github.io/garnet/docs/benchmarking/results...
[1]: https://github.com/bepu/bepuphysics2
[2]: https://www.stride3d.net
[3]: https://avaloniaui.net
[0]: https://microsoft.github.io/garnet/docs/benchmarking/results...
[1]: https://github.com/bepu/bepuphysics2
[2]: https://www.stride3d.net
[3]: https://avaloniaui.net
My personal opinion is a little more blunt than that. In my world C# developers are typically people who are attaching wall anchors with the hammer function on their power drill because they can’t be bothered to pick up an actual hammer. It works, but it’s not a good idea. Like I said. I think C# is a good language for a lot of things. It’s just that it’s often also the wrong language for specific things. If I were to use a “power drill” for everything I’d personally pick Typescript but it’s not like that would really be a better idea.
This being HN, however, most people here are more prone to actually go pick up the hammer.
This being HN, however, most people here are more prone to actually go pick up the hammer.
Feel free to be stuck in this limited sort of perception. I do not see any practical arguments, only vague analogies without concrete points.
At the risk of appearing low-tech, a much more simple, Goroutine-safe solution for dealing with "now-dependent" code:
type NowFunc func() time.Time
func getGreeting(nowFunc NowFunc) string {
And just pass in `time.Now` in for live code, and your own inline function for simulating a time in tests.
type NowFunc func() time.Time
func getGreeting(nowFunc NowFunc) string {
now := nowFunc()
if now.Hour() < 12 {
return "Good Morning"
}
return "Good day"
And just pass in `time.Now` in for live code, and your own inline function for simulating a time in tests.
For just calls to time.Now, sure.
Once you've done that same "make a low-tech simple replacement" for all of sleep, context deadlines, tickers, AfterFuncs, etc all of which are quite commonly used... you've basically done everything these libraries do. At about the same level of complexity.
Once you've done that same "make a low-tech simple replacement" for all of sleep, context deadlines, tickers, AfterFuncs, etc all of which are quite commonly used... you've basically done everything these libraries do. At about the same level of complexity.
I think https://pkg.go.dev/k8s.io/utils/clock and https://pkg.go.dev/k8s.io/utils/clock/testing is already doing these?
At a glance, that looks almost identical to (an older version of) https://github.com/jonboulle/clockwork (there are several that are structurally very similar)
In which case no, it falls into the same kind of issues that the post is describing as being problematic.
In which case no, it falls into the same kind of issues that the post is describing as being problematic.
I have a fair bit of experience writing tests for concurrent code that uses timers on Go. We started with an open source test library ( https://pkg.go.dev/github.com/benbjohnson/clock ). It had a lot of problems. This was many years ago, looks like it's seen some maintenance since so maybe it's better? Then we tried to fix it. Fixed all the obvious bugs but still had a lot of problems using it in practice. It's not enough to just handle the calls without context of who is calling them in concurrent code. Then we switched to using gomock which ended up also being hard to use.
It's quite tricky is sort of the bottom line. It's not enough to just create fake time there's a lot more to it.
It's quite tricky is sort of the bottom line. It's not enough to just create fake time there's a lot more to it.
It's been abandoned/archived. I've since been using github.com/jonboulle/clockwork which is much less problematic. For my use it is perfectly fine.
Clockwork has been mostly very good for me too. There are some sharp corners that I've run into periodically (e.g. it can't observe tickers, they're just an always-present Waiter) that I've been trying to figure out how to resolve in a new library, but overall I've been happier with clockwork than with any other I've seen so far.
Quartz looks quite interesting though. Gonna have to explore it in detail, because I agree with most of their claims in the article. I'm not 100% sold on their solutions being actual solutions and not just "often an improvement", but that'll probably be clear after using it for a bit / reading it in detail.
Quartz looks quite interesting though. Gonna have to explore it in detail, because I agree with most of their claims in the article. I'm not 100% sold on their solutions being actual solutions and not just "often an improvement", but that'll probably be clear after using it for a bit / reading it in detail.
For linux, there is faketime, which will set a specific date and time for all child processes. This enables you to test software you don't even compile for time-based problems, e.g. logic around Feb 29th or daylight saving time.
LD_PRELOAD is bit yucky though in my opinion. I suppose ptrace is the only other option though :/
It's not the only other option! It's not less yucky, but you can creatively stomp on the vDSO functions which back the time-related functions (gettimeofday, clock_gettime, time); I've written about it at [0] and have a "library" (more like a proof of concept) at [1].
[0]: https://blog.davidv.dev/posts/cursing-a-process-vdso-for-tim...
[1]: https://github.com/DavidVentura/tpom
[0]: https://blog.davidv.dev/posts/cursing-a-process-vdso-for-tim...
[1]: https://github.com/DavidVentura/tpom
[deleted]
Nice, Golang is Awesome!
Why is Golang awesome? Because in 2024 there is now a decent mock clock library for testing? I thought that one of go's selling points was the stdlib and tooling being great out of the box?