Checklists are hard, but still a good thing(utcc.utoronto.ca)
utcc.utoronto.ca
Checklists are hard, but still a good thing
https://utcc.utoronto.ca/~cks/space/blog/sysadmin/ChecklistsAreHardButGood
85 comments
When business approach me asking for automating a business process, I usually ask them to do the process in manual steps and create check lists for it. The check lists force them to think it through and are the agreed upon steps to perform the process. After they do it by hand for a while and find out where the pain points are, we can then automate. Since the check lists are already in place, automation is really easy.
One thing that might not be obvious about checklists is how they're used.
I used to think checklists were used by reading the item, then doing the thing. I literally thought of them as a recipe that you would follow. Complete a step, check the box, repeat... This is typically referred to as a "read, do" checklist. In aviation this style of checklist is typically reserved for non-normal operations—procedures that you wouldn't use often enough to commit to memory.
The other style of checklist is "do, confirm". In this style you complete a procedure by memory, and read through a checklist to ensure you didn't miss anything (no box ticking, you just read the items and confirm to yourself that they're complete). In aviation this is the style used for normal operations, and for the initial action-items in an emergency (which although not commonly used, must be committed to memory because they are time-critical).
Because you're expecting that the procedure is completed by memory, a "do, confirm" checklist can be extremely brief. You do not need to write in detail what each step involves, you just need a word or two to name the step. Additionally, they're an extremely low operational burden; it takes a couple of seconds to read through a "do, confirm" checklist but the upside of catching common errors is significant.
I used to think checklists were used by reading the item, then doing the thing. I literally thought of them as a recipe that you would follow. Complete a step, check the box, repeat... This is typically referred to as a "read, do" checklist. In aviation this style of checklist is typically reserved for non-normal operations—procedures that you wouldn't use often enough to commit to memory.
The other style of checklist is "do, confirm". In this style you complete a procedure by memory, and read through a checklist to ensure you didn't miss anything (no box ticking, you just read the items and confirm to yourself that they're complete). In aviation this is the style used for normal operations, and for the initial action-items in an emergency (which although not commonly used, must be committed to memory because they are time-critical).
Because you're expecting that the procedure is completed by memory, a "do, confirm" checklist can be extremely brief. You do not need to write in detail what each step involves, you just need a word or two to name the step. Additionally, they're an extremely low operational burden; it takes a couple of seconds to read through a "do, confirm" checklist but the upside of catching common errors is significant.
Is the checklist complete? If not, it might lull you into a false sense of security. How do you know if it's complete or not?
Would't any additional item increase safety?
No, not if you overdo it. You start getting into https://en.wikipedia.org/wiki/Alarm_fatigue territory.
If your checklist is PITA to got through, then completing it will more like to lure you into that false sense of security that you might even miss something obvious.
IMO the best way is to start small, and every time checklist didn't catch an issue either modify existing item(s) or add new item(s). Organic complexity is the best complexity.
IMO the best way is to start small, and every time checklist didn't catch an issue either modify existing item(s) or add new item(s). Organic complexity is the best complexity.
Clearly you need a checklist for making checklists.
The checklist doesn't have to be perfect. Just continually improve it.
I keep several checklists - some I use several times a week, others every few months or so. If I notice something needs to be added to the checklist or removed, I do so.
It's always better to start with an imperfect checklist vs not having any checklist at all. With no checklist, you start from scratch every single time. Not starting from scratch allows you to focus on marginal improvements with each use.
I keep several checklists - some I use several times a week, others every few months or so. If I notice something needs to be added to the checklist or removed, I do so.
It's always better to start with an imperfect checklist vs not having any checklist at all. With no checklist, you start from scratch every single time. Not starting from scratch allows you to focus on marginal improvements with each use.
A checklist is never truly "complete" - treat it as a living document with a built-in review step that asks "what did we miss?" after each use.
I did this at a previous job for running a complicated regression test (that couldn't be fully automated for reasons). I initially did this for myself as the regression test wasn't run that often. I also made sure that anyone new to the department would have to run the regression test and report/update with any missing steps.
"Update checklist if needed" should be the last thing on every checklist!
That's just like saying that if no tests are failing then we don't have any bugs.
Checklists are criminally underused in most industries. It's a testament to how great they work when looking at highly-antifragile contexts: aviation, military, etc., where a mistake can (and often does) cost lives.
The problem is that to use them broadly, you also need to implement operational-level changes. So if I made a startup that helped build checklists, the problem would not just be selling the software (which everyone has to do), but also convincing executives that checklists (even moreso than to-do lists) are worth investing in. The friction a checklist brings, at least in non-ultra-risk-averse verticals, might not be worth it.
The problem is that to use them broadly, you also need to implement operational-level changes. So if I made a startup that helped build checklists, the problem would not just be selling the software (which everyone has to do), but also convincing executives that checklists (even moreso than to-do lists) are worth investing in. The friction a checklist brings, at least in non-ultra-risk-averse verticals, might not be worth it.
Heck, they're underused in general. Checklists to pack for vacation. Checklists for "what did I mean to do this weekend". Checklists for "here's what we need to run the volunteer dinner this organization I belong to does weekly". Checklists for "things to check before my presentation".
I'm nowhere near as obsessed with them as that may make me sound; by temperament I'm definitely a go-with-the-flow sort of guy... which is perhaps exactly why I make sure to use them in certain important places. Go-with-the-flow has its strengths but also its weaknesses. Selective augmentation with checklists means I can often end up with the best of both worlds between that and being "detail oriented" for important tasks.
I have been called "detail oriented" a few times in my professional career, and I sort of laugh to myself because it is absolutely and utterly untrue. I'm not detail oriented in the slightest. It's all offloaded to the computer, with my unit tests, checklists, and such. I just know I need that augmentation and am quite aggressive about getting it precisely because I know I need it.
I'm nowhere near as obsessed with them as that may make me sound; by temperament I'm definitely a go-with-the-flow sort of guy... which is perhaps exactly why I make sure to use them in certain important places. Go-with-the-flow has its strengths but also its weaknesses. Selective augmentation with checklists means I can often end up with the best of both worlds between that and being "detail oriented" for important tasks.
I have been called "detail oriented" a few times in my professional career, and I sort of laugh to myself because it is absolutely and utterly untrue. I'm not detail oriented in the slightest. It's all offloaded to the computer, with my unit tests, checklists, and such. I just know I need that augmentation and am quite aggressive about getting it precisely because I know I need it.
checklists do wonders for my anxiety.
too often i'll have a sudden thought about "oh, i need to make sure to fix X" or "I cant forget to pack Y before the trip". My options to deal with those sudden thoughts are:
1. Do it immediately, which isn't always possible
2. Do it later, which means that I'll obsessively stress out about it because "what if I forget to do it? I need to make sure this stays at the forefront of my mind at all times."
3. Put it on a checklist and give myself permission to stop thinking about it until its time to do it
too often i'll have a sudden thought about "oh, i need to make sure to fix X" or "I cant forget to pack Y before the trip". My options to deal with those sudden thoughts are:
1. Do it immediately, which isn't always possible
2. Do it later, which means that I'll obsessively stress out about it because "what if I forget to do it? I need to make sure this stays at the forefront of my mind at all times."
3. Put it on a checklist and give myself permission to stop thinking about it until its time to do it
"The mind is for having ideas, not holding them."
-- David Allen, author of Getting Things Done.
This is one of the key concepts of "Getting Things Done". Getting tasks and information out of your head so you can actually focus on what you want to accomplish or need to do. GTD has a lot more to it, but this is around 80-90% of it with the rest being implementation details. When you get the idea, as you have, you can start reflecting and trialing different systems to see what specifically works for you and in what contexts.
This is a very-HN way of describing it, but one way I conceptualize my list-making is memoization/function caching.
My brain already went through the hard work of figuring out what to do, so I should cache that result in the todo list. later I can do a quick O(1) lookup of what do and use all of my focus on doing that.
My brain already went through the hard work of figuring out what to do, so I should cache that result in the todo list. later I can do a quick O(1) lookup of what do and use all of my focus on doing that.
> Checklists to pack for vacation
I often have to travel for work. Sometimes I have a fair bit of notice in advance and sometimes I don't. In either case, checklists are super handy for me. I generally use either Apple Notes or Obsidian for these (previously Emacs org-mode but it's less convenient for the "add an item from your phone" case than the other options).
For the "notice in advance" case, I'll start a packing checklist pretty much the moment I know I have to travel and I'll immediately write down just a sentence or two about the purpose/objective of the trip and add any special things I might need (e.g. a Saleae logic analyzer for debugging or a GoPro for documenting). As things get discussed and the trip gets closer I'll keep adding things to the list.
For the "not much advanced notice" case, I'll look back at previous checklists to find one that has a similar objective and basically just copy it wholesale and tweak. Super super useful keeping those old checklists around, even if they're only 80% correct for the next trip.
I often have to travel for work. Sometimes I have a fair bit of notice in advance and sometimes I don't. In either case, checklists are super handy for me. I generally use either Apple Notes or Obsidian for these (previously Emacs org-mode but it's less convenient for the "add an item from your phone" case than the other options).
For the "notice in advance" case, I'll start a packing checklist pretty much the moment I know I have to travel and I'll immediately write down just a sentence or two about the purpose/objective of the trip and add any special things I might need (e.g. a Saleae logic analyzer for debugging or a GoPro for documenting). As things get discussed and the trip gets closer I'll keep adding things to the list.
For the "not much advanced notice" case, I'll look back at previous checklists to find one that has a similar objective and basically just copy it wholesale and tweak. Super super useful keeping those old checklists around, even if they're only 80% correct for the next trip.
I often use a checklist version for travel packing, where every item simply goes after each other with a checkbox, instead of usual checklist with every new item going on new line.
checklists for AI to track its state has been killer for getting the LLM to work through a process from end to end.
ive never considered using emojis as the checkboxes to compare between not done, in process, done, or failed though.
ive never considered using emojis as the checkboxes to compare between not done, in process, done, or failed though.
> Checklists to pack for vacation.
A thousand times yes. Here's an older version of mine: https://honeypot.net/2023/01/13/my-travel-project.html
While I'm on vacation, I make notes about things I've forgotten. Near the bottom of that list is a reminder to add those notes to the template so I'll do them the next time I travel.
My wife thinks I'm nuts. OTOH, I'm so much more relaxed before we depart because I know that I have everything I'll need for a fun, comfortable trip. And if I don't, it's going to be some relatively minor long-tail thing that's never come up before, not something basic like "pack underwear".
A thousand times yes. Here's an older version of mine: https://honeypot.net/2023/01/13/my-travel-project.html
While I'm on vacation, I make notes about things I've forgotten. Near the bottom of that list is a reminder to add those notes to the template so I'll do them the next time I travel.
My wife thinks I'm nuts. OTOH, I'm so much more relaxed before we depart because I know that I have everything I'll need for a fun, comfortable trip. And if I don't, it's going to be some relatively minor long-tail thing that's never come up before, not something basic like "pack underwear".
For your 'Dopp kit' - do you have duplicates for things that you use on a daily (or more frequent) basis? e.g. Toothbruth, hair gel
While I follow this approach generally pre-travel, I find that on occasion I can end up taking the 'regulars' out of my travel kit for use and absentmindedly place them back in their 'normal' locations when using them as part of my regular routine.
While I follow this approach generally pre-travel, I find that on occasion I can end up taking the 'regulars' out of my travel kit for use and absentmindedly place them back in their 'normal' locations when using them as part of my regular routine.
Not completely. I buy little travel toothpastes that inevitably aren’t the one I like using. I use a hot brush and soap for shaving with a safety razor at home, but pack a little travel shave cream and disposable razor for the road.
When it’s packing time, I treat that travel kit part as its own separate list and validate that each item is still in there, or temporarily borrow it from my normal setup. It’s also one of the first things I unpack when I get home so that I can brush my teeth that night.
When it’s packing time, I treat that travel kit part as its own separate list and validate that each item is still in there, or temporarily borrow it from my normal setup. It’s also one of the first things I unpack when I get home so that I can brush my teeth that night.
Cheat code: it's not illegal to rinse out the little toothpaste tubes and fill it with your favorite brand! :-P
I have found that a tech organizer bag/pouch works great for dopp kit. It is stocked and ready to go basically all the time, with duplicates or near-discards (or specialty travel sized things).
I go even a step farther and some unique items I store in the travel kit. Not sexy, but my manual (rotating) nose-hair trimmer is small and lives in that travel kit b/c at least it has a home for when I need it, and doubles that I'd have it "just in case" when traveling.
Small tweezers, a flat nail file, small nail clippers, little comb, tide pen, folding travel toothbrush, extra tray of razor blades (in a zip-loc bag), sliver of bar shampoo (also in that ziploc bag), some hair-care products that I've transferred into tiny plastic screwtop containers from Daiso (search for "5 gram cosmetic containers"), a refillable travel perfume atomizer, etc. etc. etc.
Cork Puller (two thin strips of metal that slide down the edges of a cork) since that usually doesn't draw the ire of TSA, little round hair brush that folds up and has a sewing/button repair kit built into it. Also: some minimal medicines! Tylenol, Benadryl (runny nose), couple blister packs of DayQuil/NyQuil, anything that'll hold me over until I can walk/taxi to a convenience store.
All that stuff is in a little pouch that I keep next to my socks (https://www.amazon.com/dp/B017SKRWL4 - bagsmart electronics travel organizer case). Grab 5 socks, 5 underwear, that pouch, into a packing cube and I'm 99% ready to go. Worst case: buy shirts and pants at your destination, but brush, shave, groom, bathe are all taken care of.
I go even a step farther and some unique items I store in the travel kit. Not sexy, but my manual (rotating) nose-hair trimmer is small and lives in that travel kit b/c at least it has a home for when I need it, and doubles that I'd have it "just in case" when traveling.
Small tweezers, a flat nail file, small nail clippers, little comb, tide pen, folding travel toothbrush, extra tray of razor blades (in a zip-loc bag), sliver of bar shampoo (also in that ziploc bag), some hair-care products that I've transferred into tiny plastic screwtop containers from Daiso (search for "5 gram cosmetic containers"), a refillable travel perfume atomizer, etc. etc. etc.
Cork Puller (two thin strips of metal that slide down the edges of a cork) since that usually doesn't draw the ire of TSA, little round hair brush that folds up and has a sewing/button repair kit built into it. Also: some minimal medicines! Tylenol, Benadryl (runny nose), couple blister packs of DayQuil/NyQuil, anything that'll hold me over until I can walk/taxi to a convenience store.
All that stuff is in a little pouch that I keep next to my socks (https://www.amazon.com/dp/B017SKRWL4 - bagsmart electronics travel organizer case). Grab 5 socks, 5 underwear, that pouch, into a packing cube and I'm 99% ready to go. Worst case: buy shirts and pants at your destination, but brush, shave, groom, bathe are all taken care of.
Wow. I independently came up with something almost identical for identical reasons, including the checklists for months out, weeks out, day before, etc. It was mostly a coping mechanism for ADHD - before I did this I would reliably forget to do at least some of the things on my list or do them too late and become massively stressed as a result.
I also have job interview checklists, beginning / ending job checklists, repetitive admin checklists (beginning/end of month, beginning end of year, beginning/end of financial year etc...).
For all of these the checklist template is something I edit after one of these events after thinking "man, I wish I'd thought to do X" or "I wish Id thought of doing X a bit earlier than I did".
I also have job interview checklists, beginning / ending job checklists, repetitive admin checklists (beginning/end of month, beginning end of year, beginning/end of financial year etc...).
For all of these the checklist template is something I edit after one of these events after thinking "man, I wish I'd thought to do X" or "I wish Id thought of doing X a bit earlier than I did".
That’s exactly how I ended up doing this. Otherwise I’d be all stressed that my excitement-addled brain was forgetting something, which it inevitably was. Now I have more concrete work to do before a trip, but it’s easier and far less stressful.
OMG. I showed my friends the wiki I made for a recent job hunt with details about every job, every person I talked to, a status page for “initial contact”/“recruiter screen”/“tech screw”/… etc companies, and a timeline of events. Some responded in horror. Others started taking notes. I’ve never been a PM, but I PMed the “get a job” to the extreme.
OMG. I showed my friends the wiki I made for a recent job hunt with details about every job, every person I talked to, a status page for “initial contact”/“recruiter screen”/“tech screw”/… etc companies, and a timeline of events. Some responded in horror. Others started taking notes. I’ve never been a PM, but I PMed the “get a job” to the extreme.
My wife uses tabs on a google sheet for packing lists. Different destinations and trips have different needs. But starting with history, and editing is much more reliable.
Then a post-review for things not used, things missed, etc.
Then a post-review for things not used, things missed, etc.
A massive plus one to that. Being "detail oriented" to me is a of knowing "what needs to be done" and "how can I sort it out". Checklists excel at it.
Also kinda related, but I'm a huge fan of the "getting things done"[0] philosophy. Putting things down on a list, helps ease so much of the anxiety about not knowing, planning the next item, and most of all, relieving the cognitive burden of having to remember, which most humans aren't built for. There's a (personal and individual) limit to how many tokens our wet neural networks can hold, and I'd rather use them to focus on the task, rather than having to remember the task from memory.
Get that on repeat mode, and boom, you have checklists!
My wife and friends are at times annoyed by how anal I am about pulling out my phone to note things down whenever a new thought or task appears for me, but I wouldn't change anything about that. So much so, my task manager app is the only one app on my phone and computer that I have a subscription for (other than news and entertainment stuff).
[0]: https://en.wikipedia.org/wiki/Getting_Things_Done
Also kinda related, but I'm a huge fan of the "getting things done"[0] philosophy. Putting things down on a list, helps ease so much of the anxiety about not knowing, planning the next item, and most of all, relieving the cognitive burden of having to remember, which most humans aren't built for. There's a (personal and individual) limit to how many tokens our wet neural networks can hold, and I'd rather use them to focus on the task, rather than having to remember the task from memory.
Get that on repeat mode, and boom, you have checklists!
My wife and friends are at times annoyed by how anal I am about pulling out my phone to note things down whenever a new thought or task appears for me, but I wouldn't change anything about that. So much so, my task manager app is the only one app on my phone and computer that I have a subscription for (other than news and entertainment stuff).
[0]: https://en.wikipedia.org/wiki/Getting_Things_Done
[deleted]
Checklists are the first step to automation, really.
Run the list manually, then automated each check point by point.
Voilà!
Run the list manually, then automated each check point by point.
Voilà!
I've proposed using checklists at multiple jobs and people never want them. They always claim they're not necessary and add a ton of work. Then we all keep making the same mistakes every time.
I've quit proposing them for this exact reason. Sometimes I just make my own.
I've quit proposing them for this exact reason. Sometimes I just make my own.
We use them at our business for travel deployments/installs. They must be printed, each line item must be signed off + dated + timestamped in the presence of another employee on the same travel team (who attests the signature and the work), and they can never be electronic in nature except for the creation + printing of them.
Typically we have a non-traveling employee or manager do the final once-over and attestation, though they don't review the work necessarily. (Usually they just spot check the gear being shipped/sent to the airport, Pelican cases, Airtag trackers, etc.) This has been helpful for organizational purposes.
We don't even computerize them in any way outside of taking a cell phone picture of the final checklist and uploading it to a shared Slack channel for archival purposes.
It has prevented so many problems. It also fails to prevent some... which get added to the next checklist.
Typically we have a non-traveling employee or manager do the final once-over and attestation, though they don't review the work necessarily. (Usually they just spot check the gear being shipped/sent to the airport, Pelican cases, Airtag trackers, etc.) This has been helpful for organizational purposes.
We don't even computerize them in any way outside of taking a cell phone picture of the final checklist and uploading it to a shared Slack channel for archival purposes.
It has prevented so many problems. It also fails to prevent some... which get added to the next checklist.
None of those contexts are anti-fragile.
You probably don't want a lot of process reliability in anti-fragile contexts, like the startups you mentioned. You insist on reliability in fragile contexts.
You probably don't want a lot of process reliability in anti-fragile contexts, like the startups you mentioned. You insist on reliability in fragile contexts.
> None of those contexts are anti-fragile.
Nassim Taleb (who coined the term), literally uses the aviation industry as a prime example of antifragility in his book[1], so I think our wires are getting crossed somewhere.
[1] https://www.amazon.com/Antifragile-Things-That-Disorder-Ince...
Nassim Taleb (who coined the term), literally uses the aviation industry as a prime example of antifragility in his book[1], so I think our wires are getting crossed somewhere.
[1] https://www.amazon.com/Antifragile-Things-That-Disorder-Ince...
Yeah, I love the book and remember that being brought up. I might have to go revisit that part of the book as a refresher. Aviation, in general and in my current mindset, doesn't seem to benefit from disorder the same way a lot of the other examples he uses in that book.
Aviation is absolutely robust though.
Do you happen to remember how Taleb framed aviation as anti-fragile instead of just robust/not-fragile? As an industry I suppose we're quite good at taking lessons-learned from failures and incorporating those lessons back into making aviation safer as a whole.
Aviation is absolutely robust though.
Do you happen to remember how Taleb framed aviation as anti-fragile instead of just robust/not-fragile? As an industry I suppose we're quite good at taking lessons-learned from failures and incorporating those lessons back into making aviation safer as a whole.
Antifragile means that whenever something bad happens to the system (i.e. an airplane crash), the system gets stronger by design. A robust system is just a system that is resistant to bad stuff happening (but does not necessarily improve once it happens, or at least it's not built that way).
I'd argue that most Western government institutions (say, governments that have been around for 200+ years) are what you'd call "robust"; likely churches (the Catholic Church, since it's quite centralized), as well.
"Fragile" systems are systems like the Prisoner's Dilemma, the Tragedy of the Commons, tenuous alliances, societies/industries that are constantly in turmoil, etc.
I'd argue that most Western government institutions (say, governments that have been around for 200+ years) are what you'd call "robust"; likely churches (the Catholic Church, since it's quite centralized), as well.
"Fragile" systems are systems like the Prisoner's Dilemma, the Tragedy of the Commons, tenuous alliances, societies/industries that are constantly in turmoil, etc.
Awesome, thanks for the reminder!
Aviation safety is designed to be anti-fragile.
Each flight and aviation as a market are very fragile, what is the entire reason its safety was designed to be anti-fragile. An anti-fragile safety process makes aviation robust, despite its inherent fragility.
Each flight and aviation as a market are very fragile, what is the entire reason its safety was designed to be anti-fragile. An anti-fragile safety process makes aviation robust, despite its inherent fragility.
What's the difference between a checklist and a todo list? Is it that a todo list is the things you need to do and a checklist is how todo one of those things?
The semantics of list types is an underrated problem that probably holds back the market of software tutorials.
A todo list is typically a set of unrelated things to be performed once and annotated as "done." Something may be performed periodically, like iOS reminders adds take out the garbage on a weekly basis, but each performance is really its own item. Sometimes a todo list will have parent-child tasks.
A checklist is a two lists: one a template and the other the record of performing the checklist. The template and performance instance records are a set of related things which may or may not be ordered or conditional. A special type of checklist is an inspection checklist which may include results from performing a particular checklist item as well as the fact that it was performed.
A todo list is typically a set of unrelated things to be performed once and annotated as "done." Something may be performed periodically, like iOS reminders adds take out the garbage on a weekly basis, but each performance is really its own item. Sometimes a todo list will have parent-child tasks.
A checklist is a two lists: one a template and the other the record of performing the checklist. The template and performance instance records are a set of related things which may or may not be ordered or conditional. A special type of checklist is an inspection checklist which may include results from performing a particular checklist item as well as the fact that it was performed.
They're similar, lists of tasks/items to check off. However, when most people talk about checklists they're often talking about it in particular contexts and a more deliberate construct than todo lists. Pilots, for instance, have pre-flight checklists. This is not an ad hoc todo list (the more common sense of "todo list") with arbitrary and possibly unrelated tasks (go to grocery store, pick up dry cleaning, outline paper).
Checklists, especially in system operations and safety contexts, also tend to have a bit of a procedural characteristic, possibly including conditional and even looping constructs or sub-procedures.
Checklists, especially in system operations and safety contexts, also tend to have a bit of a procedural characteristic, possibly including conditional and even looping constructs or sub-procedures.
- If HPA is energized
- [ ] De-energize HPA
- [ ] Enter radome
I think there are two differences: urgency and immediacy. A checklist is urgent in a way a to-do list isn't. The sky will fall if a checklist is not followed when X happens, whereas a to-do list is kind of a nebulous list of nondescript tasks. A checklist's immediacy also implies a strict ordering, whereas a to-do list does not. For example, task A must be completed before task B or the sky will fall.
It's just a lot stricter operationally, which is why I think you'd get a lot of pushback trying to implement one for non-critical business tasks.
It's just a lot stricter operationally, which is why I think you'd get a lot of pushback trying to implement one for non-critical business tasks.
A todo list is a unique thing, it’s about remembering to do all the things you said you’d do. There’s no implied consistency from one todo list to another.
A checklist is used repeatedly, and it’s about completing a task (or set of tasks) to a certain standard. If you’re following a checklist, you’re aiming for consistency.
A checklist is used repeatedly, and it’s about completing a task (or set of tasks) to a certain standard. If you’re following a checklist, you’re aiming for consistency.
Checklists are usually for repeatable activities (or at least similar-enough ones).
A checklist usually have dependencies; some items should be done in specific order
Well „checklists” on its own are just an idea for which selling won’t work.
Having dozen of checklists that work for specific scenarios would be worth it.
Having dozen of checklists that work for specific scenarios would be worth it.
The travel checklist goes with me on the trip and serves to answer three more questions: (1) Am I bringing back everything I took, (2) what did I not use on the trip, and (3) what should I have brought (i.e. add it to the checklist after the trip has started)?
I take the list for a completed trip and use it to plan the next trip of a like nature. It has been very helpful.
I take the list for a completed trip and use it to plan the next trip of a like nature. It has been very helpful.
I also have a second list for "things to do to the house before travelling" - did I shut the windows, turn off the lights, empty the trash, etc - yup, it's all on the list I followed before I left.
I do the same thing. A fun one I added a few years ago: Make sure to visually verify garage door is totally closed.
Two days into a trip our neighbor contacted us about our garage being open. We asked him to close it.
When we got back I found the automatic opener disconnected from the then-closed door. I asked the neighbor about it. He said the door would almost close using the automatic opener, but would stop and reverse. He disconnected the door from the opener and pulled it closed. (Certainly not the most secure thing, since anybody could walk up and push the door open, but arguably better than sitting open.)
I found the track was slightly bent on one side, as a result of banging into it with a heavy object while getting our camping gear out. It was causing the door to bind enough for the automatic opener to be triggered to reverse (presumably the "don't crush a human" safety feature).
When we left I pushed the button on the remote, saw the door start to go down, and left. By the time we were at the end of the street, with the door out of view, it had no doubt reversed and opened.
When I leave now I make sure I see the door fully close.
Two days into a trip our neighbor contacted us about our garage being open. We asked him to close it.
When we got back I found the automatic opener disconnected from the then-closed door. I asked the neighbor about it. He said the door would almost close using the automatic opener, but would stop and reverse. He disconnected the door from the opener and pulled it closed. (Certainly not the most secure thing, since anybody could walk up and push the door open, but arguably better than sitting open.)
I found the track was slightly bent on one side, as a result of banging into it with a heavy object while getting our camping gear out. It was causing the door to bind enough for the automatic opener to be triggered to reverse (presumably the "don't crush a human" safety feature).
When we left I pushed the button on the remote, saw the door start to go down, and left. By the time we were at the end of the street, with the door out of view, it had no doubt reversed and opened.
When I leave now I make sure I see the door fully close.
The Checklist Manifesto by Atul Gawande is well worth a read.
Gawande is a crazy smart surgeon and writer. The book is pleasantly short and makes its points really well.
https://en.wikipedia.org/wiki/The_Checklist_Manifesto
Gawande is a crazy smart surgeon and writer. The book is pleasantly short and makes its points really well.
https://en.wikipedia.org/wiki/The_Checklist_Manifesto
Which is why programming, reality, and even recipes are so hard.
Ever go try to cook something new and you read "Pre-heat the pan on low-medium...", and your programmer brain just can't take it? What kind of pan, what's low-medium on this burner, how much pre-heating are you talking about? These can't be all the instructions.
And perhaps like programming, it takes a few recipes and a few burnt steaks for you "not to worry" about that, you know what's good enough eventually. These lists (and algorithms) are never completely thorough.
Ever go try to cook something new and you read "Pre-heat the pan on low-medium...", and your programmer brain just can't take it? What kind of pan, what's low-medium on this burner, how much pre-heating are you talking about? These can't be all the instructions.
And perhaps like programming, it takes a few recipes and a few burnt steaks for you "not to worry" about that, you know what's good enough eventually. These lists (and algorithms) are never completely thorough.
Good recipes do explain this though. They say heat the oil until it shimmers, or until it smokes, or until beads of water in the pan sizzle. Or they give an exact temperature, which you can read (imperfectly) with an infrared thermometer.
None of these descriptions is perfect, but each is less likely to result in a burnt steak.
None of these descriptions is perfect, but each is less likely to result in a burnt steak.
By that definition, good recipes are vanishingly rare.
The difficulty is that the more precise you make your recipe, the more you need to account for the specifics of the situation; which you cannot possibly know.
There is one time in my life that I recall legit burning a steak. I did what I had countless times before. Heated the pan until the oil started smoking, put on steak, and reduced stove temperature. Just like how I would have written the recipe before without a second thought. This time, however, the outside was thoroughly burnt before the inside even started to cook. The difference was I was using a cast iron pan for the first time, which has a lot more thermal mass than what I was used to. My old process relief on the steak cooling down the pan.
For recipes I'm reading, I've almost always found the temperature and time details to be nearly useless. If the recipe says to make at 400 F for 30 minutes, I bake on "high" (450F) until done. If I'm in someone else's kitchen, my cooking turns out a bit worse than when I'm at home.
This is a problem you always run into when writing down a process. You need to rely on the knowledge of the person following the process to apply it correctly to their specific situation. Trying to prescribe every detail does not work well.
There is one time in my life that I recall legit burning a steak. I did what I had countless times before. Heated the pan until the oil started smoking, put on steak, and reduced stove temperature. Just like how I would have written the recipe before without a second thought. This time, however, the outside was thoroughly burnt before the inside even started to cook. The difference was I was using a cast iron pan for the first time, which has a lot more thermal mass than what I was used to. My old process relief on the steak cooling down the pan.
For recipes I'm reading, I've almost always found the temperature and time details to be nearly useless. If the recipe says to make at 400 F for 30 minutes, I bake on "high" (450F) until done. If I'm in someone else's kitchen, my cooking turns out a bit worse than when I'm at home.
This is a problem you always run into when writing down a process. You need to rely on the knowledge of the person following the process to apply it correctly to their specific situation. Trying to prescribe every detail does not work well.
Since you seem to care about your steaks, you may find this post interesting.
https://news.ycombinator.com/item?id=44662757
https://news.ycombinator.com/item?id=44662757
"Pre-heat" is done until your apparatus reaches the stable cooking temperature. The recipe writer doesn't know your pan size, room temperature, stove power, or anything like that, so they can't tell you the details.
"Low-medium" is just bad instructions. The recipe should be more detailed.
Anyway, what you are complaining about on your example is just jargon ignorance. You need to learn some stuff before you understand recipes. That's not really what makes programming hard. But it does make learning new things hard.
"Low-medium" is just bad instructions. The recipe should be more detailed.
Anyway, what you are complaining about on your example is just jargon ignorance. You need to learn some stuff before you understand recipes. That's not really what makes programming hard. But it does make learning new things hard.
Not everything needs to be rationalised back to programming.
It's not programming experience that makes you ask those questions but because they are basic cooking questions you would ask.
Checklists are either checklists or a set of instructions to follow.
Good instructions would specify some of those attributes, other times experience helps to understand what general pan you might use or what medium-low heat looks like. Exactly to your point, it takes a few burnt steaks to figure it out.
It's not programming experience that makes you ask those questions but because they are basic cooking questions you would ask.
Checklists are either checklists or a set of instructions to follow.
Good instructions would specify some of those attributes, other times experience helps to understand what general pan you might use or what medium-low heat looks like. Exactly to your point, it takes a few burnt steaks to figure it out.
Checklist is for people who have a fair amount of experience doing the task. It's just a reminder to perform the steps. An experienced chef reading "pre-heat the pan on low-medium.." would know what those terms mean.
That's one of the reasons I loved Ansible from the moment I saw it. As the OP points out, traditionally machines accumulated ad-hoc changes over a long period of time. Describing the "known good" state and running this "checklist" to make sure it is in that state both documents the checklist and evaluates it.
Same reason we haven't typed "cc" on the command line to call the C compiler on individual files for about 30 years or more.
Same reason we haven't typed "cc" on the command line to call the C compiler on individual files for about 30 years or more.
The last time I typed (well, pasted) "cc" on the command line to call the C compiler on an individual file was 26 hours ago. I wanted to recompile a single-file program I'd just written with debugging information (-g) and it seemed easier to copy, paste, and edit the command line rather than to manually delete the file and reinvoke make with different CFLAGS.
I mean, I've surely compiled orders of magnitude more C files without typing "cc" on the command line over the last week. But it's actually pretty common for me to tweak options like -mcpu, -m32, -pg, -Os, or -std=c11 -pedantic (not to mention diet cc) by running a "cc" command line directly.
Similarly, I often run Python or JS code in the REPL or in Jupyter rather than putting it in a file. The rapid feedback sometimes helps me learn things faster. (Other times it's an attractive nuisance.)
But I may be a bit of an odd duck. I've designed my own CPU, on paper. I write assembly code for fun. I've implemented several different programming languages for fun. I like to know what's underneath, behind the surface appearances of things. And that requires experimenting with it.
I mean, I've surely compiled orders of magnitude more C files without typing "cc" on the command line over the last week. But it's actually pretty common for me to tweak options like -mcpu, -m32, -pg, -Os, or -std=c11 -pedantic (not to mention diet cc) by running a "cc" command line directly.
Similarly, I often run Python or JS code in the REPL or in Jupyter rather than putting it in a file. The rapid feedback sometimes helps me learn things faster. (Other times it's an attractive nuisance.)
But I may be a bit of an odd duck. I've designed my own CPU, on paper. I write assembly code for fun. I've implemented several different programming languages for fun. I like to know what's underneath, behind the surface appearances of things. And that requires experimenting with it.
Of course I cc one file quickie programs all the time. What I am talking about is a whole directory of source files, and just "knowing" which ones are out of date and building the object files manually.
I still remember years ago trying to convince one dev to use make on a package with 20-30 source files.
I still remember years ago trying to convince one dev to use make on a package with 20-30 source files.
Running just cc instead of make is actually a much more reasonable thing to do nowadays than it was 10, 20, or 30 years ago.
https://gitlab.com/kragen/bubbleos/-/blob/master/yeso/admu-s... is the entry point to a terminal emulator I wrote, for example. `make -j 8` can build it with GCC from a `make clean` state in 380ms, but if I, for example, `touch admu-shell.c` after a build and run `make -j 8` to run an incremental build, it recompiles and relinks just that one file, which takes 200–250ms. So the incrementality of the build is saving me 230ms–280ms in that case.
Without -j, a nonincremental `make admu-shell` takes about 1100ms.
But if I instead run
That's something, to be sure; it's a saving† that makes the compilation feel immediate. But it's really pretty minor. 900ms is small enough that it only affects my development experience slightly. If I were to run the build in the background as I was editing, I wouldn't be able to tell if it were incremental or from-scratch.
Unless it screwed up, that is, for example because I didn't bother to set up makedepends, so if I edit a header file or upgrade a system library I might have to do a scratch build anyway. The `make` incremental-build savings doesn't come without a cost, so we have to question whether that cost is worth the benefit. (In this case I think it's worthwhile to use separate source files and `make` for other reasons: most of that source code is used in multiple Yeso programs, and `make -j` also makes a full build from scratch four or five times faster.)
If we extrapolate that 700ms saving backward to 25 years ago when our computers ran 500 million instructions per second instead of 30 billion, it's something like 45 seconds, which is enough of a wait to be distracting and maybe make me lose my train of thought. And 5 years further back, it would have taken several minutes. So `make` was an obvious win even for small projects like this at the time, and an absolute necessity for larger ones.
At the time, I was the build engineer on a largish C++ project which in practice took me a week to build, because the build system was kind of broken, and I had to poke at it to fix the problems whenever something got miscompiled. The compiler and linker were writing their output files to an NFS server over shared 10-megabit Ethernet.
As another data point, I just rebuilt the tcl8.6-8.6.13+dfsg Debian package. It took 1m24.514s. Recompiling just generic/tclIO.c (5314 SLOC) takes 1.7 seconds. So not doing a full rebuild of the Tcl library can save you a minute and a half, but 25 years ago (when Tcl 8 already existed) that would have been an hour and a half. If it's the late afternoon, you might as well go home for the day, or swordfight somebody in the hallway or something.
So incremental builds at the time were totally essential. Now they're a dispensable optimization that isn't always worth it.
______
† 1200 lines of C per second is pretty slow, so probably almost all of that is repeatedly lexing the system header files. I'm guessing that if I took the time to do a "unity build" by concatenating all the C files and consolidating the #includes, I could get that time down to basically the same as the incremental build.
https://gitlab.com/kragen/bubbleos/-/blob/master/yeso/admu-s... is the entry point to a terminal emulator I wrote, for example. `make -j 8` can build it with GCC from a `make clean` state in 380ms, but if I, for example, `touch admu-shell.c` after a build and run `make -j 8` to run an incremental build, it recompiles and relinks just that one file, which takes 200–250ms. So the incrementality of the build is saving me 230ms–280ms in that case.
Without -j, a nonincremental `make admu-shell` takes about 1100ms.
But if I instead run
time cc -Wall -Wno-cpp -g -Os -I. -std=gnu99 \
admu-shell.c admu.c yeso-xlib.c yeso-pic.c \
png.c jpeg.c ppmp6-read.c readfont.c ypathsea.c \
-lX11 -lXext -lpng -ljpeg -lm -lbsd -lz \
-o admu-shell
That's something, to be sure; it's a saving† that makes the compilation feel immediate. But it's really pretty minor. 900ms is small enough that it only affects my development experience slightly. If I were to run the build in the background as I was editing, I wouldn't be able to tell if it were incremental or from-scratch.
Unless it screwed up, that is, for example because I didn't bother to set up makedepends, so if I edit a header file or upgrade a system library I might have to do a scratch build anyway. The `make` incremental-build savings doesn't come without a cost, so we have to question whether that cost is worth the benefit. (In this case I think it's worthwhile to use separate source files and `make` for other reasons: most of that source code is used in multiple Yeso programs, and `make -j` also makes a full build from scratch four or five times faster.)
If we extrapolate that 700ms saving backward to 25 years ago when our computers ran 500 million instructions per second instead of 30 billion, it's something like 45 seconds, which is enough of a wait to be distracting and maybe make me lose my train of thought. And 5 years further back, it would have taken several minutes. So `make` was an obvious win even for small projects like this at the time, and an absolute necessity for larger ones.
At the time, I was the build engineer on a largish C++ project which in practice took me a week to build, because the build system was kind of broken, and I had to poke at it to fix the problems whenever something got miscompiled. The compiler and linker were writing their output files to an NFS server over shared 10-megabit Ethernet.
As another data point, I just rebuilt the tcl8.6-8.6.13+dfsg Debian package. It took 1m24.514s. Recompiling just generic/tclIO.c (5314 SLOC) takes 1.7 seconds. So not doing a full rebuild of the Tcl library can save you a minute and a half, but 25 years ago (when Tcl 8 already existed) that would have been an hour and a half. If it's the late afternoon, you might as well go home for the day, or swordfight somebody in the hallway or something.
So incremental builds at the time were totally essential. Now they're a dispensable optimization that isn't always worth it.
______
† 1200 lines of C per second is pretty slow, so probably almost all of that is repeatedly lexing the system header files. I'm guessing that if I took the time to do a "unity build" by concatenating all the C files and consolidating the #includes, I could get that time down to basically the same as the incremental build.
I find checklists invaluable for running my software product business.
But any checklist needs to be a living document that has is easily updated and under version control.
Personally, I used MyLifeOrganized documents stored in Subversion.
But any checklist needs to be a living document that has is easily updated and under version control.
Personally, I used MyLifeOrganized documents stored in Subversion.
Same here. I use checklists heavily for my projects.
I write them in (emacs) org mode in a way that I can “execute” them repeatedly. Have written about it: https://www.naiquev.in/recurring-checklists-using-org-mode-i...
I write them in (emacs) org mode in a way that I can “execute” them repeatedly. Have written about it: https://www.naiquev.in/recurring-checklists-using-org-mode-i...
I made a checklist to migrate storage from one host to another recently. Like TFA mentions, it ended up being incomplete with some key details missing. But I filled in the gaps as I went and saved the checklist for next time.
Tim Harford mentions in his podcast "Cautionary Tales – A Fascination with Failure / Death on the Dance Floor" that checklists might have prevented the 1981 skywalk collapse in Kansas City Hyatt Regency Hotel (killing 114 people).
It's really a good podcast (pretty much all of his podcasts are)
https://timharford.com/2023/07/cautionary-tales-a-fascinatio...
It's really a good podcast (pretty much all of his podcasts are)
https://timharford.com/2023/07/cautionary-tales-a-fascinatio...
A checklist can be gradually automated, step-by-step.
Start by automating the low hanging fruit.
After a few iterations you have a somewhat automated process and some steps that are harder to automate. Those hard-to-automate steps can be annotated with detailed instructions that expose opportunities to partly automate them. You can break down each step more over time. As you run the checklist, you’ll learn & iterate. Then with the newly broken down steps, you can automate what’s become automatable. Repeat forever!
Start by automating the low hanging fruit.
After a few iterations you have a somewhat automated process and some steps that are harder to automate. Those hard-to-automate steps can be annotated with detailed instructions that expose opportunities to partly automate them. You can break down each step more over time. As you run the checklist, you’ll learn & iterate. Then with the newly broken down steps, you can automate what’s become automatable. Repeat forever!
Hurrah for do-nothing scripting[0]!
[0] https://blog.danslimmon.com/2019/07/15/do-nothing-scripting-...
[0] https://blog.danslimmon.com/2019/07/15/do-nothing-scripting-...
The Checklist Manifesto by Atul Gawande is a must read
Indeed. It’s actually one of the books that influenced me the most. It’s squarely in the middle of a Venn diagram of psychology, systems thinking, project management, neuroscience, healthcare, technology, productivity, and problem solving.
One of the great thing about that book was how he described checklist failure, when best practices checklists from one institution didn't generate the same results because the people in the receiving institution weren't bought into all the items within the checklist and it was just something to click through and get done.
Oh man, this was the entire premise of a business I worked on years ago: http://manifest.ly/
I've been fascinated for checklists for a long time, and I've been thinking about creating this "checklists on steroids" app, where checklists can be shared, created from templates, executed (and information about each executing collected), etc.
I finally had some time to spend on it, and here's the result: https://checkoff.ai/
Would be grateful for any feedback!
I finally had some time to spend on it, and here's the result: https://checkoff.ai/
Would be grateful for any feedback!
Checklists are extremely hard. Writing them requires the skill of the programmer and direct response copywriter, and few people have both. Ensuring that people actually use the checklists is brutally difficult. And creating systems to improve the checklists as they are used is even harder. I suspect checklists being so hard is the reason they're so rarely used.
I have be wondering about do-not-check lists to go with checklists.
I am mostly thinking about packing list, as it is common to overpack when travelling, so it can be good to have a list of things you think will be useful but are not. For example, if you are travelling in hotels (as opposed to hitchhiking ;)), you probably don't need a towel, because they are usually provided, so, to the do-not-pack list.
The problem with checklist, especially the ones that change over time is that they sometimes go out of control with useless items, a do-not-check list means "I removed this from the checklist, there is a good reason, don't put it back"
I am mostly thinking about packing list, as it is common to overpack when travelling, so it can be good to have a list of things you think will be useful but are not. For example, if you are travelling in hotels (as opposed to hitchhiking ;)), you probably don't need a towel, because they are usually provided, so, to the do-not-pack list.
The problem with checklist, especially the ones that change over time is that they sometimes go out of control with useless items, a do-not-check list means "I removed this from the checklist, there is a good reason, don't put it back"
I learned about checklists (versus to do lists) from Dr. Atul Gawande. He has done a great deal of work on checklists in medicine because of the complexity in the field and as a way to force/facilitate communication.
He wrote The Checklist Manifesto - How to Get Things Right, has been interviewed a gazillion times about the need for the surgical checklist, and talks extensively about the challenges associated with getting hospitals and medical practices to adopt the surgical checklist.
NPR's Atul Gawande's Checklist For Surgery Success [1] is a short version of the problem, challenges, inspiration, and solution.
And more recently, look forward to Captain Steeeve [2] talking about checklists pilots use.
[1] https://www.npr.org/2010/01/05/122226184/atul-gawandes-check... [2] https://www.youtube.com/@CaptainSteeeve
He wrote The Checklist Manifesto - How to Get Things Right, has been interviewed a gazillion times about the need for the surgical checklist, and talks extensively about the challenges associated with getting hospitals and medical practices to adopt the surgical checklist.
NPR's Atul Gawande's Checklist For Surgery Success [1] is a short version of the problem, challenges, inspiration, and solution.
And more recently, look forward to Captain Steeeve [2] talking about checklists pilots use.
[1] https://www.npr.org/2010/01/05/122226184/atul-gawandes-check... [2] https://www.youtube.com/@CaptainSteeeve
I love that Eisenhower quote which I feel is quite relevant here.
"plans are useless, planning is essential."
"plans are useless, planning is essential."
Many people in this discussion seem to confuse checklists with to do lists.
A checklist is an established, recurring procedure that is ideally memorised, but despite that one goes through each item on the checklist to make sure nothing is forgotten or performed out of sequence or twice.
A to do list is an ad hoc collection of tasks that need to be performed. The list is built for each occasion rather than covering a standard procedure.
A checklist is an established, recurring procedure that is ideally memorised, but despite that one goes through each item on the checklist to make sure nothing is forgotten or performed out of sequence or twice.
A to do list is an ad hoc collection of tasks that need to be performed. The list is built for each occasion rather than covering a standard procedure.
The company that I used to work for, used huge checklists (sometimes, thousands of items).
Of course, these checklists manifested as Excel spreadsheets.
Of course, these checklists manifested as Excel spreadsheets.
Wow, doing what?
Japanese manufacturing company. Checklists are very popular in Japan, as is Excel.
I use checklist as a pilot, but trying to introduce the same habit to software engineers are hard.
The cost of reversing a decision is cheaper in software, than to say flying or doing hardware. Plus the spec requirements for flying and hardware design rarely changes as often as software.
Still swear by checklists and SOPs but have to be at peace that others don't see it the same way.
The cost of reversing a decision is cheaper in software, than to say flying or doing hardware. Plus the spec requirements for flying and hardware design rarely changes as often as software.
Still swear by checklists and SOPs but have to be at peace that others don't see it the same way.
Checklists are one of the most powerful universal tool I know for avoiding/minimizing disasters/failures. In the context of software, I try to create user interfaces as masked checklists, because it forces making critical steps/decisions visible and clear (common domain specific understanding for both developer and end user), and guides user to goal. It's almost like old-school "wizard" UIs, but professionals in mind instead of beginners.
Also, critical business processes are great expressed as checklists.
Also, critical business processes are great expressed as checklists.