In praise of --dry-run(gresearch.co.uk)
gresearch.co.uk
In praise of --dry-run
https://www.gresearch.co.uk/article/in-praise-of-dry-run/
160 comments
> I've found that you really need to design for it
It's an architectural step a lot of people skip, and by doing so you often end up in a situation where the only way to make things faster is via caching, and chasing caching bugs for the rest of your tenure.
One of the less appreciated aspects of model-view-controller is that usually it demands that you plan out your action before you do it - especially when that action is read-only. Like a cooking recipe, you gather up all of the ingredients at the beginning and then use them afterward.
By fetching all of the data early, you reduce the length of the read transactions to the database, allowing MVCC to work more efficiently. You also paint a picture of the dependency tree in your application, up high where people can spot performance regressions as they show up in the architecture - where there's a better opportunity to intercede, and to create teachable moments.
To make dry-run work you are best served by book-ending your reads and your writes, because if the writes are spread across your codebase how do you disable all of the writes? And if any reads are after writes, how do you simulate that in your dry-run code? It won't be easy, that's for sure.
The problem is that we tend to write code stream-of-consciousness, grabbing things just as we need them instead of planning things out. This results in a web of data dependencies that is scattered throughout the code and difficult or impossible to reason about. This is where your caching insanity really kicks into high gear.
To my eye, Dependency Injection was sort of a compromise. You still get a good deal of the ability to statically analyze the code but you can work more stream-of-consciousness on a new feature. But it does rob you of some more powerful caching mechanisms (memoization, hand tuning of concurrency control, etc)
It's an architectural step a lot of people skip, and by doing so you often end up in a situation where the only way to make things faster is via caching, and chasing caching bugs for the rest of your tenure.
One of the less appreciated aspects of model-view-controller is that usually it demands that you plan out your action before you do it - especially when that action is read-only. Like a cooking recipe, you gather up all of the ingredients at the beginning and then use them afterward.
By fetching all of the data early, you reduce the length of the read transactions to the database, allowing MVCC to work more efficiently. You also paint a picture of the dependency tree in your application, up high where people can spot performance regressions as they show up in the architecture - where there's a better opportunity to intercede, and to create teachable moments.
To make dry-run work you are best served by book-ending your reads and your writes, because if the writes are spread across your codebase how do you disable all of the writes? And if any reads are after writes, how do you simulate that in your dry-run code? It won't be easy, that's for sure.
The problem is that we tend to write code stream-of-consciousness, grabbing things just as we need them instead of planning things out. This results in a web of data dependencies that is scattered throughout the code and difficult or impossible to reason about. This is where your caching insanity really kicks into high gear.
To my eye, Dependency Injection was sort of a compromise. You still get a good deal of the ability to statically analyze the code but you can work more stream-of-consciousness on a new feature. But it does rob you of some more powerful caching mechanisms (memoization, hand tuning of concurrency control, etc)
These observations speaks volumes as to how testing is often done badly. It's a bit of a generalisation, but if you move your reads/writes to the periphery, the core tends to be pure functions that are easy to unit test without side-effects. If you don't move your reads/writes to the periphery, everything becomes an integration test, with the accompanying brittleness, lack of performance, and resistance to change. TL; DR: Stop designing/implementing things such that you have to mock everything; DI is just a tool, and as such it is used for both good and evil.
I think this is a case where the Free Monad could help a lot. I've only ever built toy-examples but I really liked the way it allows you to structure the program's plan separately from interpreting it.
A dry-run mode could be implemented as a side-effect free interpreter .. without ever having to touch the plan or the program's grammar.
This is the first comment I've used HN's "favorite" feature on. How wonderfully insightful.
Dry run is among the things you really need to design for, others are:
- cancel
- undo/redo
- progress bars
Progress bars with accurate timing are notoriously difficult, or even impossible to get right. But even regardless of timing, having a progress bars that really shows progress and doesn't freeze is hard. Every slow operation has to have some sort of callback mechanism to update progress, and you have to know the number of steps in advance.
Undo requires, for each operation, to know how to roll back. You also need to have every operation go through the undo stack. Another option is to have an efficient snapshot system, which may be just as hard or even harder.
Cancel is actually the hardest to do right because it combines the difficulties of both the progress bar and undo. You have to have a way to interrupt a long operation at any time and get back to before it started. And because "cancel" is most often used when things go wrong (ex: disk full, bad connection, ...) you have to be very careful with error handling.
- cancel
- undo/redo
- progress bars
Progress bars with accurate timing are notoriously difficult, or even impossible to get right. But even regardless of timing, having a progress bars that really shows progress and doesn't freeze is hard. Every slow operation has to have some sort of callback mechanism to update progress, and you have to know the number of steps in advance.
Undo requires, for each operation, to know how to roll back. You also need to have every operation go through the undo stack. Another option is to have an efficient snapshot system, which may be just as hard or even harder.
Cancel is actually the hardest to do right because it combines the difficulties of both the progress bar and undo. You have to have a way to interrupt a long operation at any time and get back to before it started. And because "cancel" is most often used when things go wrong (ex: disk full, bad connection, ...) you have to be very careful with error handling.
The thing I've gather from this whole thread is that I made the right decision by choosing a functional, immutable language.
What I like is a system where you first create a plan for what you will do, then there is an executor that will execute the plan. So --dry-run just doesn't run the executor. Of course, depending on what you are working with that might not be possible, but if you can design it like so, do it. It also makes everything nicely decoupled.
Yeah, not sure how this would work with any workflow that caLLs out to another service and uses the result of that to determine the next step, if that intermediate call changes state somewhere.
Are you worried about that intermediate call changing between the dry run and the actual run? That can be avoided by saving the execution plan and allowing the actual run to load the execution plan from the dry run.
I think a large part of the next 10 years is us figuring out that frameworks are an anti-pattern and that everything is going to have to move to libraries that abide by CQRS.
Did not know what CQRS is, did some light googling[1]:
CQRS (Command and Query Responsibility Segregation) is an alternative to a simple CRUD-based database interface. CQRS separates reads and writes into different models, using commands to update data, and queries to read data.
[1]
CQRS (Command and Query Responsibility Segregation) is an alternative to a simple CRUD-based database interface. CQRS separates reads and writes into different models, using commands to update data, and queries to read data.
Commands should be task-based, rather than data centric. ("Book hotel room", not "set ReservationStatus to Reserved").
Commands may be placed on a queue for asynchronous processing, rather than being processed synchronously.
Queries never modify the database. A query returns a DTO that does not encapsulate any domain knowledge.
[1]
I really like this book on CQRS & ES: https://www.amazon.com/Practical-Microservices-Event-Driven-...
It’s a really practical treatment of CQRS & ES with JS examples. I had heard about CQRS & ES on HN for years but I had never seen any system that actually used it or any practical examples so this book is amazing cause it takes you through how to build a system using them.
It’s a really practical treatment of CQRS & ES with JS examples. I had heard about CQRS & ES on HN for years but I had never seen any system that actually used it or any practical examples so this book is amazing cause it takes you through how to build a system using them.
Then you can only really dry run that first phase anyway.
Indeed, the other service then needs to support dry runs.
Yes, I like this a lot. You often end up with a main code path that looks like this:
do a bunch of stuff
make a plan
if dryRun {
plan.Print()
exit()
}
plan.Execute()Lines 1 & 3 had side effects, sad panda
Reading from disk, spending CPU cycles, allocating memory, and making queries on the network are all technically side effects--but they're not really cause for "sad panda".
Everything on a computer has some side effects, and the purpose of --dry-run is to stop execution without certain effects that we care about. It's impossible to eliminate side effects entirely, so this is not a goal.
Everything on a computer has some side effects, and the purpose of --dry-run is to stop execution without certain effects that we care about. It's impossible to eliminate side effects entirely, so this is not a goal.
I had http:// 192.168.100.123 :7654/lights_on.sh that idempotentially turns on lights in my room. Trying to query or even pre-fetching does it. I was aware it’s not wisest, wasn’t as stupid as to expose it, also it broke so it is no more, but there’s always the guy who does that.
The "n" key on your keyboard is connected to some dynamite attached under the desk, if you type "--dry-run" or even just "-n", it plays the song "Those Endearing Young Charms" and then explodes, killing everyone in the room.
https://www.youtube.com/watch?v=ZLNduq2pnN0
At some point it doesn't matter if there's "the guy who does that", because the right choice is to let "the guy who does that" deal with the consequences of their own crazy setup.
https://www.youtube.com/watch?v=ZLNduq2pnN0
At some point it doesn't matter if there's "the guy who does that", because the right choice is to let "the guy who does that" deal with the consequences of their own crazy setup.
HTTP PUT is for idempotent operations, and would be appropriate here.
[deleted]
HTTP GET is also idempotent in REST, and slightly easier than PUT (since you can just append any parameters onto the end of the URL with GET instead of inserting them into the HTTP headers with PUT)
I believe HTTP GET is "nullipotent", meaning that accessing a resource 0 times is supposed to be the same as accessing it once or more. So I don't think it's appropriate for actions like "lights on".
I guess you really do have to preface your jokes with joke disclaimers these days.
It's impossible to tell if someone is being sarcastic on the internet, because no matter how stupid your comment is at face value, there's always a percentage of people on the forum stupid enough to say it in earnest.
https://en.wikipedia.org/wiki/Poe%27s_law
Use a winking smiley face emoticon ;-) or /s
https://en.wikipedia.org/wiki/Poe%27s_law
Use a winking smiley face emoticon ;-) or /s
But I was smiling when I wrote my comment, isn't that enough?
With the number of fanatics of Haskell and functional programming on HN, it's completely reasonable to think you were serious with your comment.
I'm sure there have been older tools with this, but Terraform is the first I encountered with it and I really like this model.
rsync is where I first encountered it. I've made it a habit to always do a dry run first as a sanity check and it has saved me some headaches.
rsync has a plan mode? I had no idea.
-n or --dry-run. Rsync has a TON of features, it is worth the time to thoroughly read the man page at least once
Right, but `--dry-run` is just a dry run mode, right? You have to sort of post-process it to create a plan.
I feel like the context for this comment thread was lost. Just to repeat it to forestall any further discussion in this subthread on non-plan dry-run, this is how I saw it:
* Igor: a good way to do dry run is to make plan
* * Me: first time i have seen 'plan' as explicit step for dry-run is with Terraform
* * * fickle: rsync is where i first encounter it [it here is not plan, it is dry-run]
* * * * me: really? rsync has 'plan'?
* * * * * john: rsync has dry-run. read manpage
i.e. Igor and I are talking about explicit plan (which is interesting model) and you guys are talking about dry-run.
I feel like the context for this comment thread was lost. Just to repeat it to forestall any further discussion in this subthread on non-plan dry-run, this is how I saw it:
* Igor: a good way to do dry run is to make plan
* * Me: first time i have seen 'plan' as explicit step for dry-run is with Terraform
* * * fickle: rsync is where i first encounter it [it here is not plan, it is dry-run]
* * * * me: really? rsync has 'plan'?
* * * * * john: rsync has dry-run. read manpage
i.e. Igor and I are talking about explicit plan (which is interesting model) and you guys are talking about dry-run.
Terraform does frequently suffer from the lack of good separation here though, depending on the provider in question - a plan can often succeed where a subseuqent apply will fail. Too much external state / not enough validation, generally.
Usually I just use dry run directives to make sure stuff is working right when I first develop it, so all that is worked out from the very beginning. I also like to develop with a lot of debug output gated behind debug as well, and just leave that there for the inevitable but at some point that would make me add it if not already present.
What I don't like is passing those as params as args. Currying this stuff around is very error prone and cumbersome. I prefer to set environment variables for DRY_RUN and DEBUG (with debug accepting higher integer levels for more debugging).
This works wonderfully for me since I never have to worry I didn't pass something along correctly, I'm always asking the global set at runtime.
What I don't like is passing those as params as args. Currying this stuff around is very error prone and cumbersome. I prefer to set environment variables for DRY_RUN and DEBUG (with debug accepting higher integer levels for more debugging).
This works wonderfully for me since I never have to worry I didn't pass something along correctly, I'm always asking the global set at runtime.
So, for a top level bit of code to pass arguments to lower level bits of code you set environment variables? Seems a bit odd to me
Using environment variables for the user to configure their environment or pass information into a program seems normal, but code passing arguments to other bits of code in the same language with environment variables seems odd.
Using environment variables for the user to configure their environment or pass information into a program seems normal, but code passing arguments to other bits of code in the same language with environment variables seems odd.
In many programming languages, environment variables are a mutatable dictionary/map, which makes it the ultimate place to store global variables.
It's definitely my guilty go-to-hack when I'm not up for refactoring everything to be more functional and/or take a dry_run function parameter everywhere.
It's definitely my guilty go-to-hack when I'm not up for refactoring everything to be more functional and/or take a dry_run function parameter everywhere.
It's a global mutable map that multiple uncoordinated processes can change. If you want a global map, just make a global map. Or just make global variables, since the variable namespace itself is a map.
All that assumes you have some stuff to set up that global map, and that means that setup code is a requirement.
I end up writing lots of library code. Sometimes that library code is called from within a command line utility I created, sometimes it's called from a web service, sometimes it's just a small driver script because I'm not developing or testing the utility, but the library code itself.
I can make that include to set up the shared global and try to make sure it's included in all instances and all ways I want to use the code, or make all the call sites resilient to it not existing, or I can just use the included OS mechanism for doing this and since that's always available, I get it for free.
Also, dry run mode isn't necessarily something you want set in a config. It's generally something you run once or twice prior to running for real (the normal case) or while in development/debugging. It's not something you would want to set in code and accidentally forget and push live, and generally a good dry run mode will look like it succeeded without actually succeeding, mocking responses that would fail along the way, because you aren't testing one small thing you're testing a workflow of some sort generally which has a few steps.
That said, I fully admit the trade-off might go a different way for different languages. Using a compiled strongly typed language may mean there's enough bits to check that you need to write a debug/dry run helper function to make it convenient, so there's not a lot lost by requiring setup in that as well. But for something like Perl (and I assume Python and Ruby and JS, to almost the same degree) where I can do:
I end up writing lots of library code. Sometimes that library code is called from within a command line utility I created, sometimes it's called from a web service, sometimes it's just a small driver script because I'm not developing or testing the utility, but the library code itself.
I can make that include to set up the shared global and try to make sure it's included in all instances and all ways I want to use the code, or make all the call sites resilient to it not existing, or I can just use the included OS mechanism for doing this and since that's always available, I get it for free.
Also, dry run mode isn't necessarily something you want set in a config. It's generally something you run once or twice prior to running for real (the normal case) or while in development/debugging. It's not something you would want to set in code and accidentally forget and push live, and generally a good dry run mode will look like it succeeded without actually succeeding, mocking responses that would fail along the way, because you aren't testing one small thing you're testing a workflow of some sort generally which has a few steps.
That said, I fully admit the trade-off might go a different way for different languages. Using a compiled strongly typed language may mean there's enough bits to check that you need to write a debug/dry run helper function to make it convenient, so there's not a lot lost by requiring setup in that as well. But for something like Perl (and I assume Python and Ruby and JS, to almost the same degree) where I can do:
warn "Calling out to foo() with args: " . Dumper($args) if $ENV{DEBUG} and $ENV{DEBUG} >= 2;
foo($args) if not $ENV{DRY_RUN};
Is this true on other OSs? On Windows at least env vars are inherited and will not change in a long running process if changed elsewhere.
So you can do something like nuke $ENV{PATH} in your perl script and it'll apply to any subsequent child system() calls.
So you can do something like nuke $ENV{PATH} in your perl script and it'll apply to any subsequent child system() calls.
on POSIX, environment variables is just a pointer to an array of pointers to NAME=VALUE strings, terminated with NULL.
The contents are private to a process, and the only method to modify them from outside the process is to have write access to process memory and change the pointer to the memory block.
When you start a new process, ultimately your calls turn into execve family with full set of arguments, i.e. actual binary to replace your process with, arguments, and contents of the environment it's going to use. The wrapper functions that don't ask for new environ value simply copy the current one.
The contents are private to a process, and the only method to modify them from outside the process is to have write access to process memory and change the pointer to the memory block.
When you start a new process, ultimately your calls turn into execve family with full set of arguments, i.e. actual binary to replace your process with, arguments, and contents of the environment it's going to use. The wrapper functions that don't ask for new environ value simply copy the current one.
The only two things I use this for are for setting things that I consider runtime flags that I want honored throughout the instance of that applicaton run.
In the instance of dry run, I think it's far more important that it's correctly seen and honored than that globals should be avoided. I view dry run mode as a contract with the person running, and if I can avoid having to pass arguments along every time, and avoid having to got change the arguments of all the utility functions I call and then all their call sites, then that's a win because if that needs to be done to correctly honor that sometimes it won't be done and some worse solution will happen, if anything.
> but code passing arguments to other bits of code in the same language with environment variables seems odd.
I don't pass the environment variables around, they exist as part of the program state. if foo() calls bar(), I don't pass the environment variable to bar, it exists as a global flag, I just check for it in bar(). That's the benefit, it's a global and set at runtime.
The other benefit is that as an environment variable, it's inherited by child processes. Even if I system out to another utility script, I don't have to pass a debug flag their either, it's inherited as part of the environment the child runs in.
In the instance of dry run, I think it's far more important that it's correctly seen and honored than that globals should be avoided. I view dry run mode as a contract with the person running, and if I can avoid having to pass arguments along every time, and avoid having to got change the arguments of all the utility functions I call and then all their call sites, then that's a win because if that needs to be done to correctly honor that sometimes it won't be done and some worse solution will happen, if anything.
> but code passing arguments to other bits of code in the same language with environment variables seems odd.
I don't pass the environment variables around, they exist as part of the program state. if foo() calls bar(), I don't pass the environment variable to bar, it exists as a global flag, I just check for it in bar(). That's the benefit, it's a global and set at runtime.
The other benefit is that as an environment variable, it's inherited by child processes. Even if I system out to another utility script, I don't have to pass a debug flag their either, it's inherited as part of the environment the child runs in.
[deleted]
There are other approaches to this as well:
1. Passing around a configuration or context (similar to your gripe around params)
2. Referencing some kind of global configuration (a la env vars)
3. Referencing a local or scoped configuration (for example, a method of an object can check instance variables that dictate behavior)
I personally prefer either a passed context or a local configuration; I find both easier to test in isolation. Global contexts have their uses, but tend to become problematic when they clash with other libraries or tools that may also be present in the execution environment.
1. Passing around a configuration or context (similar to your gripe around params)
2. Referencing some kind of global configuration (a la env vars)
3. Referencing a local or scoped configuration (for example, a method of an object can check instance variables that dictate behavior)
I personally prefer either a passed context or a local configuration; I find both easier to test in isolation. Global contexts have their uses, but tend to become problematic when they clash with other libraries or tools that may also be present in the execution environment.
Those are good approaches, and I wouldn't try to use manually set env vars for most things. I do tend to think they work very well for debug an dry run options though, because those are generally ephemeral, and things you might want to set ad-hoc in different environments easily without changing the config for everything in that environment, or passing around a special config which may not be updated when the real one is.
That said, I'm not married to it, if I saw something that seemed obviously better, I would switch. I also suspect that different languages may make one approach easier/better than others based on their capabilities, idioms, etc. In many scripting languages, accessing an environment variable is extremely easy. In some compiled or more strictly typed languages, the access and conversion to the expected type might be cumbersome enough to do on site that it's worse, and if you are standardizing in come parsing routine, that might tip the benefits in favor of some global context that is used instead.
That said, I'm not married to it, if I saw something that seemed obviously better, I would switch. I also suspect that different languages may make one approach easier/better than others based on their capabilities, idioms, etc. In many scripting languages, accessing an environment variable is extremely easy. In some compiled or more strictly typed languages, the access and conversion to the expected type might be cumbersome enough to do on site that it's worse, and if you are standardizing in come parsing routine, that might tip the benefits in favor of some global context that is used instead.
I have some code that lacks permissions to run to completion except on Bamboo or other servers, so I practically have to have a dry-run mode anyway.
> 2. You still need safety nets for when someone is doing dangerous operations without dry-run.
Another important property is idempotency. Especially if the script involves network requests or moving files around, you'll want to reach the goal by re-running in case something breaks half-way.
Another important property is idempotency. Especially if the script involves network requests or moving files around, you'll want to reach the goal by re-running in case something breaks half-way.
Is idempotence the right word for this? Honest question. This is more like eventual consistency after a small number of operations, where each individual operations is idempotent.
> Regardless of where you push your run/dry-run dispatch to, the underlying "do it" logic really needs to be factored properly for individual side effects (think functional). Otherwise, you inevitably end up with loads of "if dry-run / else" soup or worse, bugs _caused_ by your dry-run support.
--dry-run is one of the best motivations for using granular, extensible effects
when you constrain literally every bit of IO your program is allowed to do, you can now 100% know you've stubbed them all out when doing a dry run
--dry-run is one of the best motivations for using granular, extensible effects
when you constrain literally every bit of IO your program is allowed to do, you can now 100% know you've stubbed them all out when doing a dry run
One thing that I've used in the past for safety that has been extremely helpful is to always log results to a detail level that allows reversal of whatever you did. For example, for a big DB update, create a CSV (or just write to STDOUT) all affected records along with their old data. If you get into this pattern enough, and your operations are similar enough, you can even standardize on the format so you can have a common tool that can undo an operation for you given the log.
Certainly it's not something that can be applied for every use case, but when it can, it has so many benefits - an audit trail of what actually happened, ability to undo, and a common structure and approach for what dry runs should do (just output the log)
Certainly it's not something that can be applied for every use case, but when it can, it has so many benefits - an audit trail of what actually happened, ability to undo, and a common structure and approach for what dry runs should do (just output the log)
I have yet to see anywhere implemented like this, but IMO mutation/deletion should be on dry run mode by default.
At my work I've gone through and setup a variety of "replacement" commands for commands that may have dire impact on production systems. All of these replacement commands dry run by default (or fake dry run/try to create a dry run the best they can) and require a "--do-it" (think emperor palpatine voice) flag to do the intended operation. I've had multiple coworkers thank me for this setup as it's saved people from silly typos and mistakes, one such being the different handling of programs in the use of and ending "/" in folders. ex. "/etc" is different than "/etc/" sometimes for certain operations.
I'd like to one day see a global flag on operating systems to dry run nearly all commands that changed anything.
I'd like to one day see a global flag on operating systems to dry run nearly all commands that changed anything.
> All of these replacement commands dry run by default
Same here. In addition if there's never a use case for running in production (e.g. at one point we had a command that reset a dev environment to a clean slate) then the command will actually specifically check for the production environment and refuse to run as an extra failsafe.
Same here. In addition if there's never a use case for running in production (e.g. at one point we had a command that reset a dev environment to a clean slate) then the command will actually specifically check for the production environment and refuse to run as an extra failsafe.
I've largely switched to writing all my tools dry-run by default. We've had a few undesirable events from tools that had --dry-run modes and accidentally got run without. Making --run or similar necessary almost guarantees it's never run without intention.
It's official best practice for powershell to enable - whatif in any potentially dangerous scripts.
Seems to be followed most of the time.
Seems to be followed most of the time.
Ditto :). I use --really though.
--void-warranty
--deliberately-do-insane-thing
React.__SECRET_DOM_DO_NOT_USE_OR_YOU_WILL_BE_FIRED
How about Handler.runWithScissors()?
https://android.googlesource.com/platform/frameworks/base/+/...
https://android.googlesource.com/platform/frameworks/base/+/...
--launch-the-missles
It should be a parameter that takes as an argument the current user and logs the invocation, "--if-it-breaks-i-take-personal-responsibility $USER" to really make sure folks understand what they're doing.
--make-it-so is even better.
I sometimes create a makefile with "it so: build" so that I can make it so :)
Good idea. I'm writing some db scripts, and I think I'll do this.
Exactly how I do it. It guards against people who don't read the docs or bump enter halfway through accidentally (damn laptop keyboards).
Their example of pushing it into the type system is nice but requires a language with a fairly powerful type system. Once simple way to do this in basically any statically typed language is just requiring a "Run Token" for any bit of code that does something "for real". This method works well if you want the output to be similar or identical for dry run and actual run.
You can make it a bit more tedious to generate that RunToken so that a `do_the_think(RunToken, ...)` looks more natural. But the idea is simply that if the only place you create a RunToken is when parsing the --no-dry-run flag you can't accidentally do_the_thing when you move the code out of the `if is_dry_run` block without noticing.
Of course this is a simple way to get fairly reliable dry run. I do agree that having seperate plan and apply steps is approach is the best when you can do it.
struct RunToken;
fn do_the_thing(_: RunToken, ...);
You can make it a bit more tedious to generate that RunToken so that a `do_the_think(RunToken, ...)` looks more natural. But the idea is simply that if the only place you create a RunToken is when parsing the --no-dry-run flag you can't accidentally do_the_thing when you move the code out of the `if is_dry_run` block without noticing.
Of course this is a simple way to get fairly reliable dry run. I do agree that having seperate plan and apply steps is approach is the best when you can do it.
This is almost like one step toward an object-capability system. In an object-capability language, the main function would receive capability objects, like a reference to a module for accessing the filesystem, and functions could only use those capabilities if they were passed a reference to them. You could write your main function so it takes the filesystem module, creates a wrapper around it that only exposes read-only functionality, passes that read-only wrapper around to most functions that build up some kind of plan, and then the main function uses the raw unrestricted filesystem module to execute the plan at the end depending on the dry-run flag. You can then be easily sure that code doesn't do any writes to the filesystem if the unrestricted filesystem module isn't passed to it.
For particularly dangerous operations, even better is to make dry run the default, and force users to explicitly opt-in to actually running it.
I've done that and called the counter option --for-real, but I'd like both args so runbooks can spell out:
Internally, I'd like an enum flag that's clearly dry_run or real_run, so the guards are using positive logic:
1. do-step-one --dry-run
2. (do validation)
3. do-step-one --real-run
4. (do validation)
Internally, I'd like an enum flag that's clearly dry_run or real_run, so the guards are using positive logic:
switch(run) {
case dry_run:
print("Would do this...");
case real_run:
do_real_thing();
}coughs gently I use `DryRunMode.Dry` and `DryRunMode.Wet` in the OP.
I think the point here, though, is that the user needs to be explicit on the command line: they have to specify one of `--wet` or `--dry`; specifying neither is an error. It's not clear from your code if you do that, or if you interpret `--dry-run` as `DryRunMode.Dry` and the absence of an option as `DryRunMode.Wet`.
While I kinda like this idea in principle, I haven't really seen any CLI apps that require an explicit option for both modes, so it might be a bit of unexpected UX for people.
While I kinda like this idea in principle, I haven't really seen any CLI apps that require an explicit option for both modes, so it might be a bit of unexpected UX for people.
I noticed that after I posted and had read a bit more; but you're also using F# which has proper ADTs. I'm typically writing support scripts in Python or the like, and even very good libraries like click[1] nevertheless reflect the idioms popular in the language.
I really came back to this post because I was struggling with Gradle and if there is a more perfect illustration of how a dry run mode could help than that, I don't know what it could be. Gradle builds are a fucking mystery, every time, and there's no excuse for it.
[1]: https://palletsprojects.com/p/click/
I really came back to this post because I was struggling with Gradle and if there is a more perfect illustration of how a dry run mode could help than that, I don't know what it could be. Gradle builds are a fucking mystery, every time, and there's no excuse for it.
[1]: https://palletsprojects.com/p/click/
I've written a couple tools that default to dry run and require `--yes-actually-delete` and similar, but I can't think of any publicly available tools that follow this pattern - anybody have suggestions?
`hdparm --yes-i-know-what-i-am-doing --please-destroy-my-drive --fwdownload`
https://github.com/Distrotech/hdparm/blob/4517550db29a91420f...
https://github.com/Distrotech/hdparm/blob/4517550db29a91420f...
Not quite the same, but apt commands like apt-get require user interaction to confirm what you are about to do, unless you explicitly add a "-y" / "--yes" to the command.
apt will sometimes prompt you to confirm really dangerous options too:
sudo apt-get purge login
..
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
login
0 upgraded, 0 newly installed, 1 to remove and 303 not upgraded.
After this operation, 1,212 kB disk space will be freed.
You are about to do something potentially harmful.
To continue type in the phrase 'Yes, do as I say!'
?]Not exactly the same, but along the same lines: mysql has an --i-am-a-dummy flag so that DELETE queries won't run without a WHERE.
React has a deliberately inconvenient API for injecting unsanitized HTML markup straight into the DOM:
<div dangerouslySetInnerHTML={{ __html: "Some raw HTML" }}></div>
Also not quite the same, but `git clean` bails out extremely early unless you supply one of -i, -f, or -n (the latter of which is `--dry-run`).
Perforce’s command line app uses this pattern for one of its most dangerous commands: p4 obliterate //depot/path/... which prints out exactly which files will be obliterated from the server (which not only removes their entire history, but can corrupt other files too if you don’t p4 snap them first).
To actually carry out the obliterate action, you have to add the -y option.
To actually carry out the obliterate action, you have to add the -y option.
https://github.com/dmerejkowsky/ruplacer
Uses `--go` to confirm.
Uses `--go` to confirm.
rm has "--no-preserve-root", which isn't quite "--no-seriously-delete-my-root-directory" but is at least heading in that direction.
Obligatory Cantrill reference https://youtu.be/wTVfAMRj-7E?t=5046. The whole series of these are well worth a listen.
if you squint, you can kind of see file permissions as implementing this pattern? rm /usr fails unless you sudo rm /usr
sed does not operate on files directly unless turned (-i)nteractive.
-i is short for --in-place
I only use sed on version controlled files and immediately run git diff afterwards to check the results.
Minor quibble the -i option mnemonic is "in-place" rather than "interactive".
oh good one, prettier does the same thing - output to stdout unless --write is set
It’s a good opportunity to use long options like:
—i-acknowledge-this-is-dangerous-and-i-gave-this-more-than-cursory-thought
—i-acknowledge-this-is-dangerous-and-i-gave-this-more-than-cursory-thought
I don't like this because it trains users to assume that commands can never do dangerous stuff by default.
Definitely true. In this case, it's important to vet guides/examples to make sure you minimize copypastable examples that have the --no_dry_run flag
Yep, this is what I do now. The dry run is the default, you have to flag `--really` to do anything permanent.
(Author here.)
A whistlestop tour of the technique "insert a lightweight API boundary down the middle of a tool, for fun and profit". It leads you towards a structure that has lots of benefits, such as meaningful `--dry-run` output and more ready librarification.
A whistlestop tour of the technique "insert a lightweight API boundary down the middle of a tool, for fun and profit". It leads you towards a structure that has lots of benefits, such as meaningful `--dry-run` output and more ready librarification.
This was a great read!
It's not material to the point, but I think there's a small typo on line 12 of the second "Finishing the example" snippet: it looks like
It's not material to the point, but I think there's a small typo on line 12 of the second "Finishing the example" snippet: it looks like
let instructions = gather args
let instructions = gather inputGlobsThanks!
You're quite right; thanks for letting me know. I'll get that fixed.
You're quite right; thanks for letting me know. I'll get that fixed.
Nice to see F# in posts from time to time. I might try the technic in the future.
Dry run is OK, but if we want to look at it closer, it occurs in tools with mixed query + command responsibilities.
For example
For example
rm PATTERN
find -name PATTERN | rm
This is a good point, but notice that what makes the segregation easy in your example is that the query and the command have a shared way to refer to what the command will operate on: the filename.
Consider a conceptually very similar case: instead of finding and deleting files in a filesystem, think about finding and deleting lines matching a pattern within a file. Then the query is something like grep...but what do you put for the command on the other side of the pipe?
Of course, you can tell grep to output line numbers and use a command that operates on line numbers, or similar. The point is, in order to achieve this kind of segregation, you need some common way of naming operands on both sides of the divide. And naming things is hard, so segregating things this way is hard, and thus there's a lot of tools with mixed query/command responsibilities.
Consider a conceptually very similar case: instead of finding and deleting files in a filesystem, think about finding and deleting lines matching a pattern within a file. Then the query is something like grep...but what do you put for the command on the other side of the pipe?
Of course, you can tell grep to output line numbers and use a command that operates on line numbers, or similar. The point is, in order to achieve this kind of segregation, you need some common way of naming operands on both sides of the divide. And naming things is hard, so segregating things this way is hard, and thus there's a lot of tools with mixed query/command responsibilities.
PowerShell solved this problem by outputting easy to map objects instead of just plain text. Unfortunately they made it a bit more verbose than it could've been, and the industry didn’t recognize the potential, so it remained a niche product.
I still love PowerShell.
I still love PowerShell.
Finding is a query. Deleting is destructive. Make sure that you are not nesting your destructions within a query.
You would first build a pure function that read the files and returned matching lines. You could then display the count of the matched lines, and if the user wants they can see those lines and what line number it is. This looks a lot like a dry-run.
You can then pass these results into an executor. You then build a system that combines all deletes of a file and passes that to your DeleteLinesInFile fuction that does so in a single transaction. Cycle through each file and voila.
You would first build a pure function that read the files and returned matching lines. You could then display the count of the matched lines, and if the user wants they can see those lines and what line number it is. This looks a lot like a dry-run.
You can then pass these results into an executor. You then build a system that combines all deletes of a file and passes that to your DeleteLinesInFile fuction that does so in a single transaction. Cycle through each file and voila.
My point is not that this is impossible; it's that separating the tasks of building a list of query results and then executing an operation on those results requires some kind of convention about how to refer to those results across the two phases. Developing such a convention is a harder problem than it looks like.
The example I was responding to was from the (Unix) shell, which as far as I am aware does not offer any such convention other than filenames. Real programming languages of course offer more options, but even there the problem is not solved: as you say, you must then build the system which understands how to take the results of the query and execute something on them. That means designing such a convention inside your program. The article is about using a type system to help you do that cleanly.
The example I was responding to was from the (Unix) shell, which as far as I am aware does not offer any such convention other than filenames. Real programming languages of course offer more options, but even there the problem is not solved: as you say, you must then build the system which understands how to take the results of the query and execute something on them. That means designing such a convention inside your program. The article is about using a type system to help you do that cleanly.
rm doesn't do pattern matching though - that part is in your shell so the responsibilities are already separated in this case.
For all you Bash heads out there:
DRYRUN=1
VERBOSE=1
cmd() {
[ $DRYRUN -eq 1 -o $VERBOSE -eq 1 ] \
&& echo -e "\033[0;33;40m# $(printf "'%s' " "$@")\033[0;0m" >&2
if [ $DRYRUN -eq 0 ]; then
"$@"
fi
}
cmd echo "don't worry about quoting"
cmd do_something_dangerous "$scary_arg" "$scary_arg2"Why not simply
DRYRUN=echo
cmd() {
$DRYRUN "$@"
}
?The printf version will give you automatic quoting making copy'n'paste really easy to test. It' also mixes well w/ a verbose option for "--verbose" or such.
For those using zsh, you can get better quoting (and only when necessary, and handles non-printable characters and newlines better) than using printf.
run() { if dry; then echo "${(q-)@}" else "$@" fi }
run() { if dry; then echo "${(q-)@}" else "$@" fi }
I use this all the time (I call it $EXEC instead), and have gotten into the habit of starting all my scripts with EXEC=echo and put it on every command that does any file changes. It has saved my bacon many times.
It doesn't work all the time as the previous poster noted, but it is very low friction which is especially important when writing quick throw-away scripts.
It doesn't work all the time as the previous poster noted, but it is very low friction which is especially important when writing quick throw-away scripts.
Fantastic article, and I promote a stricter approach: `--dry-run` is the default, and you need to disable with `--no-dry`.
For irreversible (such as deletions) actions I even go one step further with a `--no-dry=<some-non-trivial-value>` in order to force some thinking.
Edit: apparently this is way more common than I thought, by reading comments here!
For irreversible (such as deletions) actions I even go one step further with a `--no-dry=<some-non-trivial-value>` in order to force some thinking.
Edit: apparently this is way more common than I thought, by reading comments here!
How do you all write dry-run logic for utilities that perform file operations (edits, deletions, creations) and have some kind of sequential dependency in operations such that later steps depend on the previous steps having been executed?
I have found it a frustrating nut to crack because it seems like it needs to involve either writing a simulation layer that tracks the state of all the file calls virtually, or else copy all the files to a temp folder (so the dry run isn't dry, it's just on a separate version of the data). Both of these seem like bad solutions.
I have found it a frustrating nut to crack because it seems like it needs to involve either writing a simulation layer that tracks the state of all the file calls virtually, or else copy all the files to a temp folder (so the dry run isn't dry, it's just on a separate version of the data). Both of these seem like bad solutions.
Copying to a temp folder sounds pretty reasonable to me.
Yes, if you call a black box outside of your code base which does something then subsequent operations can't easily be dry-run'd.
So if you need to install a package which sets up a systemd service, the subsequent dry-run of managing that systemd service would fail because the service doesn't exist yet. Or you need to make assumptions in the service dry run that it should have been set up before.
You can just assume that the service would have been setup and report that you were, say, enabling and starting the service. Or you could require some kind of hint to be setup from the package installation to the specific service configuration (for well known things this could be done by default, but for arbitrary user-supplied packages this cannot be done). Or you could list the entire package contents and try to determine if the manifest looks like it is configuring a service.
And it still may fail because you don't know the contents of that file without extracting it and the service may not parse at all.
And that's just a simple package-service interaction example. You could spend a week noodling on how to do that fairly precisely, then there's a hundred or a thousand other interactions to get correct.
You're being told not to do the thing, so there's fundamentally a black-box there, and you need to figure out how much you're going to cheat, how much you're going to try to crack open the black box into some kind of sandbox so you can figure out its internals, and how much you're going to offload onto the user. Not actually an easy problem at all.
So if you need to install a package which sets up a systemd service, the subsequent dry-run of managing that systemd service would fail because the service doesn't exist yet. Or you need to make assumptions in the service dry run that it should have been set up before.
You can just assume that the service would have been setup and report that you were, say, enabling and starting the service. Or you could require some kind of hint to be setup from the package installation to the specific service configuration (for well known things this could be done by default, but for arbitrary user-supplied packages this cannot be done). Or you could list the entire package contents and try to determine if the manifest looks like it is configuring a service.
And it still may fail because you don't know the contents of that file without extracting it and the service may not parse at all.
And that's just a simple package-service interaction example. You could spend a week noodling on how to do that fairly precisely, then there's a hundred or a thousand other interactions to get correct.
You're being told not to do the thing, so there's fundamentally a black-box there, and you need to figure out how much you're going to cheat, how much you're going to try to crack open the black box into some kind of sandbox so you can figure out its internals, and how much you're going to offload onto the user. Not actually an easy problem at all.
Yeah, the ability to indicate a shell command is pure (or that it must be pure) is something that's really missing in POSIX-like APIs. It's something I've certainly missed. Something like fork that disables write outside a handful of file descriptors (like one the parent starts with popen) would be pretty awesome. Maybe BSD jails do that.
In general an composable dry run API with the ability to make promises would be good. Then its on the lower level black box to have been tested correctly and make accurate promises.
In practice though what you'll find is that its easier to treat whole systems as black boxes, and test changes in throwaway virt systems, then test them in development environments, then roll them out to prod (or that whole immutable infrastructure thing and throw it all away and replace so you don't get bitten by dev-prod discrepancies in theory).
In practice though what you'll find is that its easier to treat whole systems as black boxes, and test changes in throwaway virt systems, then test them in development environments, then roll them out to prod (or that whole immutable infrastructure thing and throw it all away and replace so you don't get bitten by dev-prod discrepancies in theory).
Besides having dry-run support, I think this gets to the question of what the level of abstraction should be, and I think that is clearly application-specific.
Maybe in your case the level you are trying to report dry-run information at is too granular.
Or maybe you have too tight coupling between your code that determines what needs to be changed and that which actually does the change and you might want to refactor the code that determines changes to the code that makes the changes.
Maybe in your case the level you are trying to report dry-run information at is too granular.
Or maybe you have too tight coupling between your code that determines what needs to be changed and that which actually does the change and you might want to refactor the code that determines changes to the code that makes the changes.
I don't see anything wrong with the "copy to a temporary directory" approach if your function actually does operate on files within a directory. In that scenario, copying to a different directory is actually probably exactly what you want for a dry-run, since then the code that you're executing is the same exact code that would run were you to execute in non-dry-run mode (as opposed to virtual operations which are prone to bugs if that virtualization layer and your real operations ever fall out of sync).
I don't know of any system that works like this currently, but it seems like you are asking if you can switch branches, do the work, then your "dry-run" is a pull request complete with all files changes which you can approve manually.
My kdesrc-build has a dry-run style of logic, not because of destructiveness but just because it was annoying to only realize after 4 hours of compiling software that a module you needed was somehow not in the build, or that your configuration change caused you to re-download all the repositories instead of pulling small updates.
This is problematic in multiple ways:
1) The script uses a single repository of project metadata to derive the rest of the build. This repo needs to be present and updated to figure out what to do.
2) There are sequential dependencies on a great deal of steps that can fail without much predictability (including network traffic, git commands, and the module build itself). But in a mode where you're compiling every day, even a transient module build failure does not necessarily mean you shouldn't press on to also try and build dependent modules.
I've ended up using an amalgamation of logical steps:
* If a pass/fail check can be performed non-destructively, do it (e.g. a dry run build will fail if a needed host system install is missing, or the script will decide on git-pull vs. git-clone correctly based on whether the source dir is present) * If pass/fail can't be done, just assume success during dry run (e.g. for git update or a build step) * For the metadata in particular, download it to a temp dir if we're in dry run mode and there's no older metadata version available. * Have the user pass in whether they want a build failure to stop the whole build or not.
For separate reasons, I've also needed to maintain logic to help modules with build systems that didn't support a separate build directory to act as if the build was being executed in the source directory (this was done by copying symlinks to the source directory contents into the proper build directory, file by file).
Having this separate, disposable directory made it safer to not have to be perfect on tracking files changes or alterations perfectly. And of course it helped greatly being able to offload the version control heavy lifting to CVS (back in the day, then Subversion and now Git).
This is problematic in multiple ways:
1) The script uses a single repository of project metadata to derive the rest of the build. This repo needs to be present and updated to figure out what to do.
2) There are sequential dependencies on a great deal of steps that can fail without much predictability (including network traffic, git commands, and the module build itself). But in a mode where you're compiling every day, even a transient module build failure does not necessarily mean you shouldn't press on to also try and build dependent modules.
I've ended up using an amalgamation of logical steps:
* If a pass/fail check can be performed non-destructively, do it (e.g. a dry run build will fail if a needed host system install is missing, or the script will decide on git-pull vs. git-clone correctly based on whether the source dir is present) * If pass/fail can't be done, just assume success during dry run (e.g. for git update or a build step) * For the metadata in particular, download it to a temp dir if we're in dry run mode and there's no older metadata version available. * Have the user pass in whether they want a build failure to stop the whole build or not.
For separate reasons, I've also needed to maintain logic to help modules with build systems that didn't support a separate build directory to act as if the build was being executed in the source directory (this was done by copying symlinks to the source directory contents into the proper build directory, file by file).
Having this separate, disposable directory made it safer to not have to be perfect on tracking files changes or alterations perfectly. And of course it helped greatly being able to offload the version control heavy lifting to CVS (back in the day, then Subversion and now Git).
I've found that while having a dry-run code path is always super useful, having a --dry-run flag is an anti-pattern. I prefer to make dry run the default, with all state-changing actions being in the form:
if (clear_db_flag) {
I.e. instead of a --dry-run flag to make the script harmless, I have an --execute flag that weaponizes it. This means that while developing the tool I'm more likely to run in dry-run mode (i.e. without the --execute flag) and so test the dry-run path more than I perhaps would have.
It also means I'm not very likely to delete the production database by accident, through running my script in what I intended to be dry-run mode but actually ending up deleting something I really didn't want to delete.
I think it's a nice methodology, to always have a print statement first, then perform an action that the print statement informed the user about. Easy to remember to add the print statements and more likely that you keep the dry-run code path up to date.
if (clear_db_flag) {
if (verbosity_level > 0) print("DROP DATABASE customers;")
if (execute_flag) exec_sql("DROP DATABASE customers;")
I.e. instead of a --dry-run flag to make the script harmless, I have an --execute flag that weaponizes it. This means that while developing the tool I'm more likely to run in dry-run mode (i.e. without the --execute flag) and so test the dry-run path more than I perhaps would have.
It also means I'm not very likely to delete the production database by accident, through running my script in what I intended to be dry-run mode but actually ending up deleting something I really didn't want to delete.
I think it's a nice methodology, to always have a print statement first, then perform an action that the print statement informed the user about. Easy to remember to add the print statements and more likely that you keep the dry-run code path up to date.
If you're doing a lot of modifications to a filesystem or a database, one pattern I've used and liked in the past is to have your code simply dump out the commands to run. Usually to stdout and you save it as a script.
That way you don't need to worry about issues with bugs in the dry run logic and you have a record (a script) of what you actually ran.
Even better if you can extend this output script so that it's idempotent and fails on the first failed command.
That way you don't need to worry about issues with bugs in the dry run logic and you have a record (a script) of what you actually ran.
Even better if you can extend this output script so that it's idempotent and fails on the first failed command.
I've made a further extension of this principle for a video game release script, by adding a --unsafe flag. The script was responsible for building the game on a bunch of platforms, uploading it to various storefronts and announcing the release on twitter, reddit etc. There was a --dry-run flag too, that just prints what it would do, but sometimes you want to test part of the script without doing anything public. There were various flags like --no-upload and --no-twitter etc, but it was easy to forget them, so I added in a top level --unsafe flag. At every point where we do something public, at the deepest level of the call stack just before doing it, I check for unsafe, and if its not present, throw an exception. It worked pretty well, and definitely saved my ass a few times, especially during refactoring.
First I found myself nodding along, but then a troublesome question popped into my mind. What about Toc/tou race conditions?
Indeed, that's the thing I rather swept under the rug. I'll definitely take precautions if it's at all easy to avoid the assumption that just because my `--dry-run` stage said something was safe, it will definitely be safe by the time execution passes to the "do-it" stage. However, in general you are indeed doomed the instant you decided to split the tool up this way.
I have never actually encountered such a race condition while using any tool I've written this way. But I do remember one place where I specifically didn't add a feature to the tool because it was so vulnerable to a TOC/TOU race condition. Instead, I turned that feature into a "did I just manage to do the thing successfully, and if not, why not" check at the end of the "do-it" stage.
So yes, that concern is extremely valid, and if you're doing an operation whose validity is liable to change after you've done the check, then you do just need to follow a different approach for that operation. (But then there's probably no possibility even in principle of a `--dry-run` for such an operation anyway!)
I have never actually encountered such a race condition while using any tool I've written this way. But I do remember one place where I specifically didn't add a feature to the tool because it was so vulnerable to a TOC/TOU race condition. Instead, I turned that feature into a "did I just manage to do the thing successfully, and if not, why not" check at the end of the "do-it" stage.
So yes, that concern is extremely valid, and if you're doing an operation whose validity is liable to change after you've done the check, then you do just need to follow a different approach for that operation. (But then there's probably no possibility even in principle of a `--dry-run` for such an operation anyway!)
Something I've found can work is "check, output plan, prompt for confirmation, if 'yes' execute plan in a way that maximises the odds of bailing out if anything changed while you were displaying the prompt" - for bonus points, recalculate after each action that had to be simulated and do something policy-defined if anything differs from your prior expectations.
Obviously the effort of doing this becomes a trade-off but it's worth considering.
Obviously the effort of doing this becomes a trade-off but it's worth considering.
I think this can be mitagated with more abstract descriptions, like:
"Would have gotten all servers tagged foo (currently 1,341) and deleted them"
"Would have gotten all servers tagged foo (currently 1,341) and deleted them"
That's all very well for just `--dry-run`, but it's still a problem if you use the method in OP to architect the entire tool (and in particular the non-dry-run execution) around the existence of a `--dry-run` phase.
I don't think so? The planning stage generates something like DeleteOp(ServerSpec { tag: "foo" }).
I think the problem is that describing it as "would have deleted 1378 servers", and then going off and deleting those 1378 servers, doesn't solve the race condition problem that leaves you with some servers left undeleted because they popped into existence after the check.
With server deletion, of course, the problem is less visible because it's naturally idempotent (unless you're referring to servers by some non-unique key). In that case, you can safely just pretend the leftover servers came into existence after the entire tool came into being.
Of course, if you're referring to servers by name or IP or something, then you hit exactly the problem. Concretely: I run a command that deletes all servers older than one day. The 'dry-run' section determines what servers need to be deleted and it turns out that server "foo" needs to be deleted. Elsewhere, someone spots that "foo" exists, deletes it, and creates a new server with the name "foo". Then the 'execute' flow deletes the new "foo", which is entirely not the server we wanted to delete.
With server deletion, of course, the problem is less visible because it's naturally idempotent (unless you're referring to servers by some non-unique key). In that case, you can safely just pretend the leftover servers came into existence after the entire tool came into being.
Of course, if you're referring to servers by name or IP or something, then you hit exactly the problem. Concretely: I run a command that deletes all servers older than one day. The 'dry-run' section determines what servers need to be deleted and it turns out that server "foo" needs to be deleted. Elsewhere, someone spots that "foo" exists, deletes it, and creates a new server with the name "foo". Then the 'execute' flow deletes the new "foo", which is entirely not the server we wanted to delete.
Your example inaccurately describes what would happen with GP's approach.
Planning stage would generate `Delete(Server{CreatedBefore: 1621887357}))`.
It would output something like "Deleting servers created before $time$, currently $servers$."
The point is the planning stage would encode the operation being done, not a list of servers to delete.
Planning stage would generate `Delete(Server{CreatedBefore: 1621887357}))`.
It would output something like "Deleting servers created before $time$, currently $servers$."
The point is the planning stage would encode the operation being done, not a list of servers to delete.
Ohh, sorry - I got the wrong end of the stick entirely. Yes, in that case I'm satisfied!
Even better than a dry-run flag is to have the noop mode be the default behavior and have e.g. "--execute" enable non-read-only behavior (or, for really scary commands like programmatic data destroyers, "--nuke-my-data-i-solemnly-swear-i-know-what-i-am-doing"). Prevents plenty of accidental mistakes by sleepy operators.
That's a quibble though, the article is well-put and quite correct about the value of a dry-run mode.
That's a quibble though, the article is well-put and quite correct about the value of a dry-run mode.
Oops, I read the article but not the comments. I see I'm far from the first person to suggest this, apologies for the duplication.
Abstracting a bit, this is an application of the principle "represent important intermediate results as explicit data structures". It shows up in many places: testing, extensibility, logging, and parallelization are all easier to implement when the result is represented explicitly instead of implicitly in the program's call stack.
Has anyone written about this idea? It seems to have some interesting interaction with laziness.
Has anyone written about this idea? It seems to have some interesting interaction with laziness.
Dry-run can be implemented succinctly in Python with a function decorator wrapping those functions that should run conditionally.
Nice aspects of this technique:
Inside the decorator you have access to both the function’s name and its arguments so you can print descriptive messages to the user e.g. “In dry run, would have run function ‘delete’ on ‘/foo/bar’”
You can use the name of the decorator as de facto documentation, e.g. “@destructive”
Nice aspects of this technique:
Inside the decorator you have access to both the function’s name and its arguments so you can print descriptive messages to the user e.g. “In dry run, would have run function ‘delete’ on ‘/foo/bar’”
You can use the name of the decorator as de facto documentation, e.g. “@destructive”
In the PowerShell world: "-WhatIf" is your friend.
To implement this in your own scripts is not difficult at all. The template snippet below automatically wires up "-Confirm" and "-WhatIf" support:
It also passes through that preference to commands invoked witin the script, so things like "Format-Volume" will do nothing if you invoke the outer script with "-WhatIf".
To implement this in your own scripts is not difficult at all. The template snippet below automatically wires up "-Confirm" and "-WhatIf" support:
[CmdletBinding(ConfirmImpact='High',SupportsShouldProcess=$true)]
PARAM(
[parameter(ValueFromPipeline=$true,Mandatory=$true)]
$TheThingToProcess
)
PROCESS {
if ( $PSCmdlet.ShouldProcess( $TheThingToProcess, 'Do scary thing' )) {
# Scary step
}
}
It also passes through that preference to commands invoked witin the script, so things like "Format-Volume" will do nothing if you invoke the outer script with "-WhatIf".
I have been incorporating dry run flags into the tools I have been working on (typically etl like tools). Typically this flag only stops the final call to external services that would change the internal state of those services.
Its been really helpful when hacking on things, adjusting logic, or validating that input data does what I think it will.
Its been really helpful when hacking on things, adjusting logic, or validating that input data does what I think it will.
This resonates with me. I recently published a cross-platform tool [1] for bulk renaming files with dry run as the default. In this mode, it shows the changes about to be made and any possible problems that may occur (such as overwriting existing files). To actually carry out the changes a flag must be used.
I felt it was a good design decision since the effects of bulk renaming can be substantial. Most other similar tools have it backwards with their dry-run mode relegated to a secondary action.
[1]: https://github.com/ayoisaiah/f2
I felt it was a good design decision since the effects of bulk renaming can be substantial. Most other similar tools have it backwards with their dry-run mode relegated to a secondary action.
[1]: https://github.com/ayoisaiah/f2
This is obvious but a good way to implement dry run in some settings is to obtain a read-only connection to the relevant datastore / roll back a transaction without committing, etc.
all CLIs that do substantial work should have dry runs.
more examples I've collected:
- Angular CLI https://malcoded.com/posts/angular-fundamentals-cli/#--dry-r...
- AWS CLI https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-h...
- Rspec https://relishapp.com/rspec/rspec-core/v/3-8/docs/command-li...
- Serverless framework https://forum.serverless.com/t/dry-run-with-serverless-frame...
at netlify we didnt do a dry run but Netlify Dev is close: https://news.ycombinator.com/item?id=19615546
I'm requesting one at Gatsby: https://github.com/gatsbyjs/gatsby/discussions/16384
more examples I've collected:
- Angular CLI https://malcoded.com/posts/angular-fundamentals-cli/#--dry-r...
- AWS CLI https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-h...
- Rspec https://relishapp.com/rspec/rspec-core/v/3-8/docs/command-li...
- Serverless framework https://forum.serverless.com/t/dry-run-with-serverless-frame...
at netlify we didnt do a dry run but Netlify Dev is close: https://news.ycombinator.com/item?id=19615546
I'm requesting one at Gatsby: https://github.com/gatsbyjs/gatsby/discussions/16384
Simulation is a great thing. I use this wrapper, even in my most trivial shell scripts:
pretend=${scriptname_pretend:-false}
run() { echo "$@"; $pretend || "$@"; }
I don’t even trust dry run for some things.
Yesterday I wrote a cleanup tool. In fact it’s two tools: one that outputs a parseable list of things to clean up...
Yesterday I wrote a cleanup tool. In fact it’s two tools: one that outputs a parseable list of things to clean up...
# snapshotX # keep
snapshotY # delete
One lesson from making tools over the years having dry-run options is to always very explicitly state what you mean by dry-run.
The most common use is as a write/modify gate, but many tools have uses where it may need to avoid certain classes of reads as well (network, db, etc).
The most common use is as a write/modify gate, but many tools have uses where it may need to avoid certain classes of reads as well (network, db, etc).
What do you guys think of "dry run by default" where you have to turn dry run OFF via an argument? I've done this on some of my utilities just because screwing up the arguments would result in much grief (lost data).
I wonder how it would feel to have every program dry-runnable but composable.. basically FP/virtual systems. You can (dry-run cmd-0) | (dry-run ...) | (dry-run cmd-n) and it will yield a potential system state.
I use --dry-run with rsync almost every time I use rsync. I don't really use rsync enough to remember if I should use a trailing slash or not...
Duplicity has this option. I love it. Especially when something bad can happen, like blowing up my disk because an ISO image is in the backup area.
Even better is to invert the flag, that default is to dump the plan and have to pass an `--actually-do-it` to have it make the changes.
Gresearch puts out some interesting articles, and they work in F#. I wish I were in the UK, I'd love to try and get a job there.
I believe (though can't swear to it) that we do provide immigration/relocation assistance and money, at least for quant researchers and possibly for all employees. If that would be enough to get you interested, feel free to drop me a line at [email protected] and I can find out actual details rather than half-remembered possibly-false snippets from years ago.
Sadly money is not the issue proximity to family is. I'll just hope one day they open an NZ or south east Asia branch.
They’re an interesting organization.
See this past submission: https://news.ycombinator.com/item?id=18499712
See this past submission: https://news.ycombinator.com/item?id=18499712
Here are two other things I've found:
1. Regardless of where you push your run/dry-run dispatch to, the underlying "do it" logic really needs to be factored properly for individual side effects (think functional). Otherwise, you inevitably end up with loads of "if dry-run / else" soup or worse, bugs _caused_ by your dry-run support.
2. You still need safety nets for when someone is doing dangerous operations without dry-run. Pointing and calling [0] is a great trick for this. For example, say your tool is about to delete X, Y, and Z. Instead of a simple "Yes" confirmation which the user would quickly end up doing on auto-pilot, you could have a more involved confirmation like "Enter the number of resources listed above to proceed" and then only proceed to delete if the user enters "3".
Very curious to hear about design idioms folks have come up with!
[0] https://en.wikipedia.org/wiki/Pointing_and_calling