Using the 5S Principle in Coding(santhoshsundar.medium.com)
santhoshsundar.medium.com
Using the 5S Principle in Coding
https://santhoshsundar.medium.com/using-the-5s-principle-in-coding-6237a1614a02
54 comments
The folder thing is something I’ve noticed too. The problem with folders is that in most interfaces you can’t see their content until they’re opened. In some really bad interfaces you can only see one open folder at a time.
Flat lists let you see the entire thing at once. As long as there is some sort of natural sort order, it’s still a “neat” way to organise things.
E.g.:
If you had folders you’d just see this:
Similar concepts apply to code at every level. I avoid overly abstract functions that do their job through seven levels of trivial abstractions for the same reason. A one-page “flat” function can be more readable and maintainable.
Flat lists let you see the entire thing at once. As long as there is some sort of natural sort order, it’s still a “neat” way to organise things.
E.g.:
SomeApp-PRD-Web
SomeApp-PRD-DB
OtherApp-PRD-Web
OtherApp-PRD-DB
OtherApp-PRD-API
If you had folders you’d just see this:
SomeApp
OtherApp
Similar concepts apply to code at every level. I avoid overly abstract functions that do their job through seven levels of trivial abstractions for the same reason. A one-page “flat” function can be more readable and maintainable.
The problem with relying on a tree is most brains don't organize information like that. This is why code search is much more valuable. Naturally, having one canonical tree representation is obviously a good idea but the value is really in introducing a constraint that acts as a forcing function (if done right) to simplify the code base (it can also be theater - spaghetti code masquerading as as well structured code base).
Personally I found when onboarding to a repo, whether memory firmware or hardware bringup toolkit, that flatter directories lead to a much slower grasp of the architecture. I prefer leaning toward descriptive directories.
The problem is when you guess ahead of time what directory structure you need and get it slightly but not obviously wrong. Start flat, and when that becomes a problem, you'll know why it's a problem and how to restructure it.
In general, problems are much easier to solve when you have them, than when you try to guess what they will be.
I've been considering the topic of solving problems too early for quite some time now, and you very effectively communicated this point; thank you for that clear insight.
Relatedly and at the same time, I sometimes have a hard time figuring out when the right time to 'solve' the problem is. Speaking generally, if left unchecked for too long, it seems like more effort to go through and find instances of the problem and create a valid solution and then apply the solution, then if I am able to spot the reoccurring problem after only a few instances. This is particularly worse when I go back and solve the problem in a few areas, but there are more areas that I've missed (and now I have some spots with the solution, some without, and everything is messy). I suppose I need to spend more time after creating a solution to see if it's applicable apply anywhere else (but then that's more time refactoring than actually working on the problem at hand).
Relatedly and at the same time, I sometimes have a hard time figuring out when the right time to 'solve' the problem is. Speaking generally, if left unchecked for too long, it seems like more effort to go through and find instances of the problem and create a valid solution and then apply the solution, then if I am able to spot the reoccurring problem after only a few instances. This is particularly worse when I go back and solve the problem in a few areas, but there are more areas that I've missed (and now I have some spots with the solution, some without, and everything is messy). I suppose I need to spend more time after creating a solution to see if it's applicable apply anywhere else (but then that's more time refactoring than actually working on the problem at hand).
Is one effective method perhaps to carve out a consistent, say, 5% of the day to focus on this cross-polination of implemented solutions?
This could work (perhaps on a more weekly level for me). I would have to note the solutions and problems as I come across them and put them into a backlog, otherwise I'll definitely forget about them (until I run into them again, haha).
This does go back to topic focus though. Regarding programming specifically, I generally will have a topic (such as a feature enhancement). If I implement a new solution to a problem, it's not clear-cut to me if I should go and implement that solution everywhere relevant during the current topic, or if I should wait until a more tech-debt rework to clean things up a bit. My inclination is to focus on rework only during those tech-debt reducing stints. I really should figure out a clear and well-defined process for this.
This does go back to topic focus though. Regarding programming specifically, I generally will have a topic (such as a feature enhancement). If I implement a new solution to a problem, it's not clear-cut to me if I should go and implement that solution everywhere relevant during the current topic, or if I should wait until a more tech-debt rework to clean things up a bit. My inclination is to focus on rework only during those tech-debt reducing stints. I really should figure out a clear and well-defined process for this.
Those are genuinely hard problems you describe, so don't feel bad about struggling with them.
Personally I am very focused on refactoring. When in doubt, if I can improve the code, I will! I don't know that that's "right", but it's how I live.
Personally I am very focused on refactoring. When in doubt, if I can improve the code, I will! I don't know that that's "right", but it's how I live.
This is a good generalization but I worry that it's too broad. I.e. if your lifeboat is sinking, it's too late to find a better lifeboat.
But at that point your problem isn't needing a better lifeboat, the problem is why it's sinking. At this point solving the problem of why it's sinking is the problem you've got, solving it before you had that problem seems silly.
Brilliant
Very aptly put.
Do you like programming in Java?
Different strokes for different folks. I think it's important to find a good balance. I've seen Go and C projects that don't use enough directories, so navigating the code can be a little untenable. In the example given in the article, I would scrap all the subdirectories in `/features/UserManagement`. I think the file names are sufficient to indicate what purpose they serve and putting them in subdirectories by type doesn't really offer any value.
I like the angular style guides take on this:
> Flat
> Style 04-04
> Do keep a flat folder structure as long as possible. >
> Consider creating sub-folders when a folder reaches seven or more files.
It goes more into the why on the actual guide https://angular.io/guide/styleguide
> Flat
> Style 04-04
> Do keep a flat folder structure as long as possible. >
> Consider creating sub-folders when a folder reaches seven or more files.
It goes more into the why on the actual guide https://angular.io/guide/styleguide
> "Shove Stupid Shit Someplace Secret"
This sums up the broad concept of Compliance and security in general. The people who pretend to care about compliance (i.e. management) don't have the technical know-how to grok the implementations by those who pretend to implement said compliant systems. There isn't remotely enough QA/red-teaming (expensive to do in a bias-free manner), most of the time all you need to do is scribble in the box and everyone is going to squint and pretend it's checked.
This sums up the broad concept of Compliance and security in general. The people who pretend to care about compliance (i.e. management) don't have the technical know-how to grok the implementations by those who pretend to implement said compliant systems. There isn't remotely enough QA/red-teaming (expensive to do in a bias-free manner), most of the time all you need to do is scribble in the box and everyone is going to squint and pretend it's checked.
I work on software for the healthcare industry, and recently found out about PHI (private health information) insurance. Basically they pay out if you suffer a breach,
The part I found interesting is that the premiums vary based on their assessment of your practices. This strongly aligns their incentives with actual best practices when evaluating your systems, as they are on the hook financially for any breach.
The part I found interesting is that the premiums vary based on their assessment of your practices. This strongly aligns their incentives with actual best practices when evaluating your systems, as they are on the hook financially for any breach.
Every time I get involved in a project it seems to start with this neatly organised tree where everything is squirreled away.
By contrast my own systems are often just one giant folder. Conceptually I'm just not bothered by large directories with 20000 files. And I'm happy to mix files of different types in one folder (OMG .c files and .ico files in the same folder like a barbarian!!!)
I find the system is simple, and everything I need is in one place. But it freaks people out.
I guess we're all just wired differently.
(Obviously there is organisation, I have different projects in different folders etc, but there's no "extra" organisation. I don't make more folders just because this one is "full".)
By contrast my own systems are often just one giant folder. Conceptually I'm just not bothered by large directories with 20000 files. And I'm happy to mix files of different types in one folder (OMG .c files and .ico files in the same folder like a barbarian!!!)
I find the system is simple, and everything I need is in one place. But it freaks people out.
I guess we're all just wired differently.
(Obviously there is organisation, I have different projects in different folders etc, but there's no "extra" organisation. I don't make more folders just because this one is "full".)
It maybe that this works for you individually, but does it work well as part of a team? I think it's one of those cases where one needs to do what is convenient for most of the team rather than one member?
Are there people for whom it works better to have nearly as many folders as files?
It's worth keeping in mind that lean manufacturing and all the interesting things that Toyota did that get written up are about building cars. In software the equivalent process is compiling and deploying code. Writing software is equivalent to designing, prototyping and testing a car. So while there are many interesting lessons to learn they are about the deployment and running of code, not about writing it. In the mass manufactured automobile industry you spend more time and effort building the product than designing it. In software you spend much more time designing (specs, coding, testing, all that stuff) the product than deploying it. The lessons of lean manufacturing may not apply to design.
Lean manufacturing and the Toyota Production System are absolutely about manufacturing. It's rooted in industrial engineering and operations research. It is about addressing root causes and truly solving issues. What we see in the lean books is a bunch of solutions to problems Toyota was facing at the time and how they solved them.
Even the person who coined "lean manufacturing" says, "Don't try to bring lean manufacturing upstream to product development. The application of Lean in product development and manufacturing are different. Some aspects may look similar, but they are not! Be weary of an expert with experience in lean manufacturing that claims to know product development."[1]
There are a couple of books that have tried to capture the design process from Toyota. [1] is the Wikipedia page for [2]. [3] is an alternative take. Unfortunately I haven't read these books, so can't provide anything beyond the table of contents.
[1] https://en.wikipedia.org/wiki/Lean_product_development
[2] the table of contents needs to be downloaded from https://www.lean.org/store/book/lean-product-and-process-dev...
[2] https://www.routledge.com/The-Toyota-Product-Development-Sys...
Even the person who coined "lean manufacturing" says, "Don't try to bring lean manufacturing upstream to product development. The application of Lean in product development and manufacturing are different. Some aspects may look similar, but they are not! Be weary of an expert with experience in lean manufacturing that claims to know product development."[1]
There are a couple of books that have tried to capture the design process from Toyota. [1] is the Wikipedia page for [2]. [3] is an alternative take. Unfortunately I haven't read these books, so can't provide anything beyond the table of contents.
[1] https://en.wikipedia.org/wiki/Lean_product_development
[2] the table of contents needs to be downloaded from https://www.lean.org/store/book/lean-product-and-process-dev...
[2] https://www.routledge.com/The-Toyota-Product-Development-Sys...
In my experience, whenever a site installs lean processes, somebody gets the bright idea that if it works in manufacturing, then surely it can work in R&D. Every senior (hardware) engineer has a story of this happening at a past workplace.
The compromise is to tidy the workplace as well as possible, call it "5S", fill the dumpster with stuff that's really junk, and let each engineer stash their Undecidable Things under their desk.
As for software, maybe the best thing is to just not let hardware concepts such as "designing" and "manufacturing" creep in. Especially when those things imply a social hierarchy.
The compromise is to tidy the workplace as well as possible, call it "5S", fill the dumpster with stuff that's really junk, and let each engineer stash their Undecidable Things under their desk.
As for software, maybe the best thing is to just not let hardware concepts such as "designing" and "manufacturing" creep in. Especially when those things imply a social hierarchy.
Yes! I still meet too many people who think what we are manufacturing software, while what we are doing is designing it. The manufacturing, that is, the building and deployment is already highly automated and very cheap.
A significant percentage of the time you’re researching software — determining if something is even possible.
This is the even more crucial difference and the reason for all the time estimation arguments.
If a manager can’t estimate a car production run schedule they’re a bad manager.
No manager can schedule — to the day — when fusion powered cars will be ready for shipping.
Yet, this is expected from people trying something entirely new in software.
“Integrate these two things that no human has put together before. Now that you’ve heard this single sentence, tell me: will it be ready Tuesday or Wednesday… a year from now?”
This is the even more crucial difference and the reason for all the time estimation arguments.
If a manager can’t estimate a car production run schedule they’re a bad manager.
No manager can schedule — to the day — when fusion powered cars will be ready for shipping.
Yet, this is expected from people trying something entirely new in software.
“Integrate these two things that no human has put together before. Now that you’ve heard this single sentence, tell me: will it be ready Tuesday or Wednesday… a year from now?”
This is why Agile/Standups make people go crazy! The daily standup makes sense when all the topics discussed will close out within hours, blockers sometimes stand literally upstream on the assembly line, and no one can really talk to each other outside of the standup because they’re too busy running manufacturing work centers that need 100% of their constant attention.
Imagine if compiling/deploying involved teams on the daily hand writing assembly/machine code from code base?
Imagine if compiling/deploying involved teams on the daily hand writing assembly/machine code from code base?
The lessons of lean manufacturing may not apply to design processes themselves, but a lean transformation in manufacturing often relies a lot on certain design choices (poka-yoke in assembling, standardization to cut setup times, etc). Design is often one of the most important factors in successful lean companies. I totally get the point, though.
Toyota practice had such impact because it empowered people to speak up.
It's not clear this article is empowering in that sense.
Instead of tracking/pleasing your boss, quality principles were stated and your job was to implement those principles, halting work as needed. This leads to problems being dealt with in time and in context, instead of being ignored or concealed. (Remind you of PG's distinction between being persistent vs. obstinate?) The difference lay not really in the major premise (the principles) but in how and when the minor premise of fault-correction was applied.
To me in software that's all about scheduling: scheduling infrastructure work and cross-training early, doing post-mortem's immediately, building design consensus iteratively with discussion and prototyping, etc. Scheduling and objectivity: not who's saying it, but what's being said. Both should empower IC's by giving them actual time and actual say.
In particular, clean code is often not the best, but can empower those otherwise bereft of natural authority. Sometimes duplication is better than dependencies, a little pile is better than a lot of structure, a complex PR is easier to review all at once, etc.
Empowering people makes selecting and orienting them quite important - but that's a separate issue.
It's not clear this article is empowering in that sense.
Instead of tracking/pleasing your boss, quality principles were stated and your job was to implement those principles, halting work as needed. This leads to problems being dealt with in time and in context, instead of being ignored or concealed. (Remind you of PG's distinction between being persistent vs. obstinate?) The difference lay not really in the major premise (the principles) but in how and when the minor premise of fault-correction was applied.
To me in software that's all about scheduling: scheduling infrastructure work and cross-training early, doing post-mortem's immediately, building design consensus iteratively with discussion and prototyping, etc. Scheduling and objectivity: not who's saying it, but what's being said. Both should empower IC's by giving them actual time and actual say.
In particular, clean code is often not the best, but can empower those otherwise bereft of natural authority. Sometimes duplication is better than dependencies, a little pile is better than a lot of structure, a complex PR is easier to review all at once, etc.
Empowering people makes selecting and orienting them quite important - but that's a separate issue.
Seiketsu (清潔) doesn't mean standardize; it means "clean, hygienic, sanitary". You hear it in the negative: fuseiketsu means filthy.
The first four S's are all in the same semantic neighborhood: they all have to do with neatness, cleanliness and order. Basically they collapse into one. The last S, shitsuke, is just discipline.
"Be clean/neat/tidy, and have discipline".
Yay ...
The first four S's are all in the same semantic neighborhood: they all have to do with neatness, cleanliness and order. Basically they collapse into one. The last S, shitsuke, is just discipline.
"Be clean/neat/tidy, and have discipline".
Yay ...
I worked for a manufacturing company that rolled out 5S. It seemed to make a lot of sense on the shop floor, but the executives had us doing the same thing the same way in engineering.
All the math, software, and mechanical folks had to "5S their work area". Our manager gave us rolls of tape and a label printer so we could mark off where everything on our desk was supposed to be put away. The instructions we were handed showed marking off staplers and tape dispensers for desk jobs. It really was absurd!
They furloughed everyone in the company a few weeks later and I decided it really wasn't the place for me.
All the math, software, and mechanical folks had to "5S their work area". Our manager gave us rolls of tape and a label printer so we could mark off where everything on our desk was supposed to be put away. The instructions we were handed showed marking off staplers and tape dispensers for desk jobs. It really was absurd!
They furloughed everyone in the company a few weeks later and I decided it really wasn't the place for me.
This _feels_ good to a single dev working on a project, where the rules are intuitive, no explanation is needed, and deadlines are fluid.
But I have to imagine this creates a lot of friction within a team, all trying to adhere to rules under the guise of efficiency, at great opportunity cost. And what's the ultimate impact to the customer? Do they care how elegant the codebase is?
It's worth noting what The Toyota Way is. It's a way to ensure high production quality. It is also slow. This is why Toyota effectively abandoned The Toyota Way in the mid 2000s, and why they were raked through the coals over recall issues in 2013. Growth.
If deadlines are involved, I struggle to see the argument for making view files beautifully atomic vs. say, making an improvement that adds customers / reduces churn.
But I have to imagine this creates a lot of friction within a team, all trying to adhere to rules under the guise of efficiency, at great opportunity cost. And what's the ultimate impact to the customer? Do they care how elegant the codebase is?
It's worth noting what The Toyota Way is. It's a way to ensure high production quality. It is also slow. This is why Toyota effectively abandoned The Toyota Way in the mid 2000s, and why they were raked through the coals over recall issues in 2013. Growth.
If deadlines are involved, I struggle to see the argument for making view files beautifully atomic vs. say, making an improvement that adds customers / reduces churn.
I think this has one of the cases where trying to minimalistic in examples actually defeats the purpose, e.g., in the "Sort/Seiri" React examples, the state variable in the component and all the imports that are retained are actually unused (a state variable in a component with no mutation code is just a constant with extra work, useState is only used for that variable, and the React import -- assuming this is the current major version of React for the last two years -- also isn't used [in older versions, it was needed for JSX to be used as a side effect even if the import wasn't otherwise used, and its a reflex legacy pattern that a lot of people probably haven't broken even with current react].)
This is the sort of thing that makes me love (or hate) a codebase. I just have to point out that this sort of thing is extremely easy to do with typed languages, and can be (but not inherently) a total nightmare with untyped languages. Types matter, not just for correctness, but also cleanliness, order, and ease of maintenance.
The documentation example is a clear example of absurdity. This is just a multiplication that might as well be inlined in all places where it is needed and then the 'standard' is applied and it turns into a 21 line hellscape that points out the obvious.
5S is more about organizing your workspace, but this article is about organizing your work product. And it doesn’t say anything that isn’t standard dev practice, it just organizes the items under a strained 5S umbrella. Imho
Toyota envy. Not the first or the last time someone will abuse 5S to fit some grand shower thought.
Different people need a different analogy to absorb the same information.
Because of this, the more people writing these practices down from a different viewpoint, the more likely another person is influenced and improves their craft.
Because of this, the more people writing these practices down from a different viewpoint, the more likely another person is influenced and improves their craft.
Consistant error handling should be number 1.
Followed by tests.
Optimization is in >80% of all Software unnecessary, it is a nice to have feature.
Optimization is in >80% of all Software unnecessary, it is a nice to have feature.
This is for small projects or intermediate level, what people do for larger systems ? how do you organize shared api constraints / models / tests, branching model, versioning scheme, data migrations in case of a large refactor, infrastructure change. Every aspects can lead to fragmentation, friction / grind and bitrot.
There is something about branded practice names that just instantly invokes a gag reflex. Too many times, the branding is for a reason: some programmer wants to elevate themselves into thought leadership, sell books (mostly filler of course), speaking engagements m, etc., while never really adding anything to the industry.
Great article! As someone who has inherited poorly documented and maintained codebases (and that has worked in the chemical engineering world), this article hits home on so many points. This should be normalized as best practice or at the very least regular time allotments should be given to make codebases less inscrutable.
> This should be normalized as best practice
I didn't see anything listed that isn't already considered a best practice (at least the headings, the specifics could be debated). Now, whether it's a normal practice or not is another matter, but when you have a huge number of novices joining every year and companies happily laying off senior people who could mentor them you get exactly what we have today.
I didn't see anything listed that isn't already considered a best practice (at least the headings, the specifics could be debated). Now, whether it's a normal practice or not is another matter, but when you have a huge number of novices joining every year and companies happily laying off senior people who could mentor them you get exactly what we have today.
"We heard there wasn't enough buzzwords in the software dev world, so we decided to import some buzzwords from other industries..."
Anyone runs a zero inbox policy of email organization?
`/organisms/`
What?
What?
Hm. Hard to add middle layers to that unless you have e.g. tissues, organs.
I propose going back to line numbers. Start with 1000, 2000, 3000. Sub-divide as needed.
I propose going back to line numbers. Start with 1000, 2000, 3000. Sub-divide as needed.
Yeah, I've encountered people who insisted on using this structure. I was not too fond of it.
Honestly it's just a nomenclature or convention to define hierarchies.
I've had certain projects where it was useful to get teams aligned on how to view a particular project. That being said, sometimes it's also heralded as something super life changing, which it isn't.
Joking aside, I think this article does a decent job of translating Lean manufacturing principles to coding, but the "Set in Order" section grinds my gears a little bit. I have worked on many projects of various sizes, and I think my fellow web devs tend to lean too hard into an overly nested directory structure. In manufacturing terms, imagine if you needed to open a cabinet that contains a toolbox with a plastic organizer in one of the drawers that contains a box that you had to open to get access to some rivets. It doesn't take much for that level of organization to kneecap efficiency. I would much rather have a single directory with 20 files than try to chase down a component file that's eight directories deep. Give me a big list of long descriptive file names, not short names that rely on the directory name to give context.