What is simplicity in programming and why does it matter?(blog.jakubholy.net)
blog.jakubholy.net
What is simplicity in programming and why does it matter?
https://blog.jakubholy.net/2021/simplicity/
62 comments
Sorry to be pedantic, but.. Like most articles on this topic, this one fails to distinguish between complex and complicated. Some problems are inherently complex, and attempting to model those in a “simpler” way can actually be a mistake. Complicated software, however, is almost always bad, as it implies that there is room for simplification. Identifying where it makes sense to simplify and where not to is mastery and there are no global rules for how to achieve this, only experience. Software should be as simple as possible, but no simpler.
Looking at the dictionary for both words:
https://www.merriam-webster.com/thesaurus/complicated
https://www.merriam-webster.com/thesaurus/complex
... And they literally have the same first definition! Your point is not very clear to me. Can you please help me understand better what you meant here?
In Clojure-land, we always like to distinguish between "accidental" and "essential" complexity as per https://github.com/papers-we-love/papers-we-love/blob/master...
"Accidental" complexity is "your fault" because "you" introduced it into the problem. For example, you need to use some mutable variables and that is making it harder to reason about some code. It would be "simpler" to use immutable values which are closer to pure mathematics and simpler, but you need maximum performance and most hardware is better optimized for mutable state in the extreme case.
"Essential" complexity is part of the problem and can't be reduced away as per current knowledge. For example, you need to know a fair bit of mathematics to be able to understand general relativity.
https://www.merriam-webster.com/thesaurus/complicated
https://www.merriam-webster.com/thesaurus/complex
... And they literally have the same first definition! Your point is not very clear to me. Can you please help me understand better what you meant here?
In Clojure-land, we always like to distinguish between "accidental" and "essential" complexity as per https://github.com/papers-we-love/papers-we-love/blob/master...
"Accidental" complexity is "your fault" because "you" introduced it into the problem. For example, you need to use some mutable variables and that is making it harder to reason about some code. It would be "simpler" to use immutable values which are closer to pure mathematics and simpler, but you need maximum performance and most hardware is better optimized for mutable state in the extreme case.
"Essential" complexity is part of the problem and can't be reduced away as per current knowledge. For example, you need to know a fair bit of mathematics to be able to understand general relativity.
Language is not what is written in some dictionary, it is what people use to communicate. I've seen complex vs. complicated distinction (meaning inherent complexity vs. accidental complexity) used in software-related discussions.
It's basically inherent complexity and accidental complexity. You have to live up to the first, while try to minimize the latter.
At least that's how I read OP.
At least that's how I read OP.
To not invent new ambiguities I would phrase this succinctly as the solution having intrinsic complexity vs extrinsic complexity.
I guess a generous reading of "complex" versus "complicated":
Complex - a plain adjective. It's just how the thing is.
Complicated - past tense of the verb "to complicate," used as an adjective. Somebody made this complicated.
But yeah, I still prefer thinking of inherent and accidental complexity. Expresses the actual idea directly.
Complex - a plain adjective. It's just how the thing is.
Complicated - past tense of the verb "to complicate," used as an adjective. Somebody made this complicated.
But yeah, I still prefer thinking of inherent and accidental complexity. Expresses the actual idea directly.
The webster definitions don't mean the same thing because the definitions are overly broad.
If you took another look at that TERRIBLE resource, convoluted and byzantine (and probably more) have the exact same definition. Complicated and convoluted do not mean the same thing.
If you took another look at that TERRIBLE resource, convoluted and byzantine (and probably more) have the exact same definition. Complicated and convoluted do not mean the same thing.
I agree it does not seem to be the best resource. Do you know of a dictionary that provides better definitions?
Checking the Oxford English Dictionary (https://www.oed.com), and taking the first non-obsolete definition for each adjective:
Complex: Consisting of or comprehending various parts united or connected together; formed by combination of different elements; composite, compound. Said of things, ideas, etc.
Complicated: Consisting of an intimate combination of parts or elements not easy to unravel or separate; involved, intricate, confused.
Complex: Consisting of or comprehending various parts united or connected together; formed by combination of different elements; composite, compound. Said of things, ideas, etc.
Complicated: Consisting of an intimate combination of parts or elements not easy to unravel or separate; involved, intricate, confused.
Thank you for that recommendation!
I was able to access it via my public library login. Very neat.
I was able to access it via my public library login. Very neat.
I think your example of accidental complexity is actually essential complexity. The requirement to use mutable variables and mess with memory is part of the complexity of problem space, not an accident.
Complex is used in the sense that you have to consider multiple things (such as constraints, logic, etc) in order to come up with a working solution. You cannot really escape from a complex problem because you have to satisfy all of its needs.
On the other hand, 'complicated' here is probably meant in the sense that you have a solution for your problem, but that might not be the simplest solution to the problem. In other words, you have satisfied all of the problem's needs but the solution itself is going to be hard to understand for anyone other than the person implementing it. You generally want to avoid such situations.
On the other hand, 'complicated' here is probably meant in the sense that you have a solution for your problem, but that might not be the simplest solution to the problem. In other words, you have satisfied all of the problem's needs but the solution itself is going to be hard to understand for anyone other than the person implementing it. You generally want to avoid such situations.
Yes, and where to place that simplicity
A DSL is simple to use, but complexity has moved to the compiler
An assembly is complex to use, but the compiler is simple
--
Part of the answer is in expressivity -- in breaking down problems into simpler, combinable parts. Like Unix pipes or lego
But as expressivity increases, the number of combinations increases and hence complexity itself increases
At one limit atoms are the simplest unit, and programmers can combine them to create any known physical object
At the other limit, we have a one key keyboard that can only do one job and be programmed no other way
Both extremes are extremely simple in their own way, and show us that neither language expressivity, nor even language simplicity is a true savior
A DSL is simple to use, but complexity has moved to the compiler
An assembly is complex to use, but the compiler is simple
--
Part of the answer is in expressivity -- in breaking down problems into simpler, combinable parts. Like Unix pipes or lego
But as expressivity increases, the number of combinations increases and hence complexity itself increases
At one limit atoms are the simplest unit, and programmers can combine them to create any known physical object
At the other limit, we have a one key keyboard that can only do one job and be programmed no other way
Both extremes are extremely simple in their own way, and show us that neither language expressivity, nor even language simplicity is a true savior
Part of the answer is in expressivity -- in breaking down problems into simpler, combinable parts. Like Unix pipes or lego
But as expressivity increases, the number of combinations increases and hence complexity itself increases.
This is first hard lesson learned for those that approach the challenge of achieving simplicity. The first idea one has is ‘I’ll break everything into the smallest little parts’, and suddenly you have a thousand little parts.
Sony products were considered extremely fragile because they were constructed out of tons of small parts. If you dropped it once, every little piece would fall off. Compare that to the Mac unibody.
But as expressivity increases, the number of combinations increases and hence complexity itself increases.
This is first hard lesson learned for those that approach the challenge of achieving simplicity. The first idea one has is ‘I’ll break everything into the smallest little parts’, and suddenly you have a thousand little parts.
Sony products were considered extremely fragile because they were constructed out of tons of small parts. If you dropped it once, every little piece would fall off. Compare that to the Mac unibody.
Nice.
I used to think that creating the smallest services, function, methods, etc would produce the easier software to deal with from the programmer perspective. But turns out you will end with many interfaces to keep when perhaps you just needed one
Is there any books that touch in this subject directly?
Is there any books that touch in this subject directly?
To expound:
An emergency shutdown system wants a language comprised of a single big red button
My lunch wants a language of nutrients and flavors for the VM of my tounge and stomach to trigger the happy(); syscall to my brain
-- The best language, and even simplicty itself, must be viewed from both your current position and end goal
An emergency shutdown system wants a language comprised of a single big red button
My lunch wants a language of nutrients and flavors for the VM of my tounge and stomach to trigger the happy(); syscall to my brain
-- The best language, and even simplicty itself, must be viewed from both your current position and end goal
With respect to hiring, I would say this is one of those things interviewers fail to analyze. When we look for technical prowess and attitude/behavior, the truth is those things are never expressed in the code base. Mental models, however, are exactly what is expressed in the code base. We are interviewing people and trying to see where on the normal distribution they land on the pure technical level, experience, and attitude, but rarely do those three things have an outsized impact on a generic codebase.
Instead, if you pull up someone’s GitHub and see wonderfully over abstracted code, boom, there’s your guy who is not in the middle of the normal distribution. In other words, they don’t write common code (which means they don’t write clearly, or with common sense, and certainly do not write simply).
Soon as you introduce someone like that into a code base, suddenly things that used to be simple are physically painful to touch. Jumping through files, following the wire to see where it leads, layers of abstraction, and so on. That is a codebase terrorist and one interviews for it. The fact that the entire team must now take on the mental model and enter the headspace of such a person is exhausting.
Hire commoners, and test for obtuse mental models (something that is easily missed in an isolated coding challenge and no way visible in the behavioral portion. But alas, this will fuck with your whole team more than anything else).
Instead, if you pull up someone’s GitHub and see wonderfully over abstracted code, boom, there’s your guy who is not in the middle of the normal distribution. In other words, they don’t write common code (which means they don’t write clearly, or with common sense, and certainly do not write simply).
Soon as you introduce someone like that into a code base, suddenly things that used to be simple are physically painful to touch. Jumping through files, following the wire to see where it leads, layers of abstraction, and so on. That is a codebase terrorist and one interviews for it. The fact that the entire team must now take on the mental model and enter the headspace of such a person is exhausting.
Hire commoners, and test for obtuse mental models (something that is easily missed in an isolated coding challenge and no way visible in the behavioral portion. But alas, this will fuck with your whole team more than anything else).
The problem with what you're saying is that it's meaningless without concrete examples?
What is "over abstracted code"? Perhaps we agree on this point, but I don't know because when people talk about these things no one has any idea of what they mean - we use "over-testing", "under-testing", "over-engineered", "sloppy", "too much abstraction", "simplicity", "complexity", and everyone has different takes on what these words mean but everyone agrees they're bad or good. It's a bit non-sensical.
What is the problem of jumping through files? Do you feel the same about jumping through functions? Is it better to have a soup that you can only test by setting up a new solar system? Is that why testing is considered "expensive"? And what is the solution when products need to be evolved, redesigned or pivoted? Re-write? Re-hire a team? Do the same crap again but with new tools? No tests? Or you simply don't like jumping?
Is it worth it to have bad api's, bad products, that offset millions of hours in work around to all developers (and non-developers as well) who have to use them because someone couldn't be bothered? Shouldn't the premium on salaries for developers conjure some real interest in doing things right? Is it normal that governmental institutions, public companies, banks, etc have awful interfaces, buggy behaviour, etc?
Isn't it normal that if you're commanding a very comfortable salary there should be an expectation of continuous improvement, research and learning for the work you choose to do?
Perhaps I agree with what you're saying, or perhaps I completely oppose it, but I can't tell.
What is "over abstracted code"? Perhaps we agree on this point, but I don't know because when people talk about these things no one has any idea of what they mean - we use "over-testing", "under-testing", "over-engineered", "sloppy", "too much abstraction", "simplicity", "complexity", and everyone has different takes on what these words mean but everyone agrees they're bad or good. It's a bit non-sensical.
What is the problem of jumping through files? Do you feel the same about jumping through functions? Is it better to have a soup that you can only test by setting up a new solar system? Is that why testing is considered "expensive"? And what is the solution when products need to be evolved, redesigned or pivoted? Re-write? Re-hire a team? Do the same crap again but with new tools? No tests? Or you simply don't like jumping?
Is it worth it to have bad api's, bad products, that offset millions of hours in work around to all developers (and non-developers as well) who have to use them because someone couldn't be bothered? Shouldn't the premium on salaries for developers conjure some real interest in doing things right? Is it normal that governmental institutions, public companies, banks, etc have awful interfaces, buggy behaviour, etc?
Isn't it normal that if you're commanding a very comfortable salary there should be an expectation of continuous improvement, research and learning for the work you choose to do?
Perhaps I agree with what you're saying, or perhaps I completely oppose it, but I can't tell.
This is why I don’t share my GitHub profile during the hiring process. Code is just too easy to criticize.
Write a 1 liner? Too terse and hard to understand.
Use multiple lines? Too verbose.
I don’t want to give the people who are going to be deciding my future a target to pick on. It only takes one guy to feel insecure about hiring me to say something like “he over abstracts his code”.
The only things I share are the finished products, it keeps the nitpickers quiet.
Write a 1 liner? Too terse and hard to understand.
Use multiple lines? Too verbose.
I don’t want to give the people who are going to be deciding my future a target to pick on. It only takes one guy to feel insecure about hiring me to say something like “he over abstracts his code”.
The only things I share are the finished products, it keeps the nitpickers quiet.
I will respond to the other comment in a second, but the things you are concerned with are not what I am advocating. The nit picky stuff doesn’t expose one’s mental model. It exposes some modicum of taste.
What I’m advocating is to probe how people structure the problem. There is a noticeable percentage of people that manage to have technical competence but map problems in a complicated way.
What I’m advocating is to probe how people structure the problem. There is a noticeable percentage of people that manage to have technical competence but map problems in a complicated way.
The part about END, JSON, EQL, GraphQL is a bit of a strawman. EDN and EQL are specific to Clojure, JSON and GraphQL can be used with pretty much anything. Of course if you're writing a data notation and a query language specifically for a language it will be better for that language, at least I hope so.
For the part about "having a few data structure", doesn't that means that you lose all type safety? If you have an Order type and a PaidOrder type, the compiler can enforce that you won't use an Order that hasn't been paid in places you shouldn't.
This part is not about this article specifically but about the way people talk about Clojure in general. Most of the time Clojure users compare Clojure to Java, which in a way is fair since Clojure leverages the JVM, but on the other hand Java isn't known for simplicity (part of it may be due to design, part of it is due to being older). Some tradeoffs are presented as strict wins (see above the part about type safety vs a few collections). Most people mention being skeptic at first with Clojure, and now being in love with it. Shouldn't that mean that they should keep looking for new things since they were trapped in some kind of local maximum before?
For the part about "having a few data structure", doesn't that means that you lose all type safety? If you have an Order type and a PaidOrder type, the compiler can enforce that you won't use an Order that hasn't been paid in places you shouldn't.
This part is not about this article specifically but about the way people talk about Clojure in general. Most of the time Clojure users compare Clojure to Java, which in a way is fair since Clojure leverages the JVM, but on the other hand Java isn't known for simplicity (part of it may be due to design, part of it is due to being older). Some tradeoffs are presented as strict wins (see above the part about type safety vs a few collections). Most people mention being skeptic at first with Clojure, and now being in love with it. Shouldn't that mean that they should keep looking for new things since they were trapped in some kind of local maximum before?
> If you have an Order type and a PaidOrder type, the compiler can enforce that you won't use an Order that hasn't been paid in places you shouldn't.
First programmers make Order and PaidOrder classes to enforce static type checks, then they tell you that the system absolutely cannot support credit card chargebacks because PaidOrder becoming unpaid would ruin everything in the architecture, and you should teach the clients to never charge back.
First programmers make Order and PaidOrder classes to enforce static type checks, then they tell you that the system absolutely cannot support credit card chargebacks because PaidOrder becoming unpaid would ruin everything in the architecture, and you should teach the clients to never charge back.
I could also make a strawman about Clojure not working at all because you can never differentiate between paid orders and orders but I didn't because it would add nothing to the discussion. Adding features and refactoring statically typed code is feasible and has been done by lots of people, your point is invalid.
A neat perspective I had not really ever thought of before:
> State is never simple because it complects - tangles - time and value.
> State is never simple because it complects - tangles - time and value.
Another point that evokes the wisdom reflex:
> It took me years to realize that in truth Clojure is a philosophy. The language embodies it, the ecosystem embraces it and grows from it, you the developer eventually soak it up.
The ardent way in which some programmers promote their languages to the detriment of others is indeed like other fanaticisms. I have always thought that software development is the art of choosing the right tool for the task, not making all tasks fit my favourite tool.
> It took me years to realize that in truth Clojure is a philosophy. The language embodies it, the ecosystem embraces it and grows from it, you the developer eventually soak it up.
The ardent way in which some programmers promote their languages to the detriment of others is indeed like other fanaticisms. I have always thought that software development is the art of choosing the right tool for the task, not making all tasks fit my favourite tool.
No matter how hard we try to define simplicity as an objective thing, in practice it is subjective and always depend on features of programming language, the programmer and problem being solved
Any quality can be subjective at some level, but simplicity does have an objective definition when related to complexity. It means "composed of fewer intertwined things as opposed to more intertwined things".
For example, a single atom of carbon is, objectively, simpler than a complex organic molecule which contains many more different atoms.
For example, a single atom of carbon is, objectively, simpler than a complex organic molecule which contains many more different atoms.
Agreed there is a definition somewhere, but if im trying to build a table, or pretty much anything. Kindly dont hand me single atoms, and tell me you're making my job simpler ;)
This is a probably very broken definition, but maybe: simplicity is the most direct route to our goal, from where we're standing
Maybe?
This is a probably very broken definition, but maybe: simplicity is the most direct route to our goal, from where we're standing
Maybe?
Agreed, there's a right level of abstraction.
I would say easier is the correct word here: Kindly dont hand me single atoms, and tell me you're making my job easier"
I would say easier is the correct word here: Kindly dont hand me single atoms, and tell me you're making my job easier"
For computer languages tho, definitely agreed there is an objective best when averaging over all the things we try to do with them. And it is very surprising how much commonality and uniformity there is across that space
I feel we can say imperative/functional languages are almost identical to each other when compared against the design space that assembly allows -- computed goto, jumpfrom, jumpwhen, so many options, and yet 99% of everything is if/then/switch
Does this mean programming languages are evolving into a meta description of some underlying 'thought-process' innate to God, or the-thing-that-causes-physics?
I feel we can say imperative/functional languages are almost identical to each other when compared against the design space that assembly allows -- computed goto, jumpfrom, jumpwhen, so many options, and yet 99% of everything is if/then/switch
Does this mean programming languages are evolving into a meta description of some underlying 'thought-process' innate to God, or the-thing-that-causes-physics?
[deleted]
Would be interesting to read a rebuttal of the presented definition.
I like the definition “Simple means not tangled.” But that definition isn’t objective, it depends on what you think is tangled.
The author claims “Simplicity is objective because - if we can distinguish them - we can count how many simpler elements are there, tangled together.” This admits that objectivity depends on distinguishing what the elements are, which is often subjective. So this claim can be seen as false in the sense that you can’t make something objective out of subjective parts, or true in the sense that it’s trying to punt by treating the whole as a separate thing from the elements that comprise it.
One good example of subjective elements is the discussion on state - “State is never simple because it complects - tangles - time and value.” I agree with this, but it’s important to note that state does not encode time explicitly. Time and state are byproducts of assignment. We can remove the notion of time (assignment) from some bit of code and make the code functional. But we can’t move the notion of time to an element of it’s own. And if we could, and then compose it with an element of value, we’d end up right back where we started with some state.
Generic discussions on state can be hard to parse too since there doesn’t exist a computer program without state. We move it around, organize it in various ways. We hide it with interpreters and compilers. But it’s always there. At a high level, there’s no way to detangle time and value when users and user input are involved. A feedback-response cycle often is state at a fundamental level (unless your program is idempotent, such as limited to database lookups). You can’t create anything in a stateless system.
The “What’s in your Toolkit” chart is pretty bad as a supporting argument for what “simple” means, IMO. Methods vs namespaces functions does not have a meaningful difference in this context; namespacing is an organizational tool, it doesn’t simplify things or protect you from complexity. Data vs Syntax is not a choice in any programming language that exists, all of them have both. Actors vs Queues doesn’t make sense to me, I don’t understand what they were trying to say, and suggesting queues are simpler than actors seems backwards from a game programming perspective. Consistency and Inconsistency are things you can have in any programming language at any level of complexity.
The author claims “Simplicity is objective because - if we can distinguish them - we can count how many simpler elements are there, tangled together.” This admits that objectivity depends on distinguishing what the elements are, which is often subjective. So this claim can be seen as false in the sense that you can’t make something objective out of subjective parts, or true in the sense that it’s trying to punt by treating the whole as a separate thing from the elements that comprise it.
One good example of subjective elements is the discussion on state - “State is never simple because it complects - tangles - time and value.” I agree with this, but it’s important to note that state does not encode time explicitly. Time and state are byproducts of assignment. We can remove the notion of time (assignment) from some bit of code and make the code functional. But we can’t move the notion of time to an element of it’s own. And if we could, and then compose it with an element of value, we’d end up right back where we started with some state.
Generic discussions on state can be hard to parse too since there doesn’t exist a computer program without state. We move it around, organize it in various ways. We hide it with interpreters and compilers. But it’s always there. At a high level, there’s no way to detangle time and value when users and user input are involved. A feedback-response cycle often is state at a fundamental level (unless your program is idempotent, such as limited to database lookups). You can’t create anything in a stateless system.
The “What’s in your Toolkit” chart is pretty bad as a supporting argument for what “simple” means, IMO. Methods vs namespaces functions does not have a meaningful difference in this context; namespacing is an organizational tool, it doesn’t simplify things or protect you from complexity. Data vs Syntax is not a choice in any programming language that exists, all of them have both. Actors vs Queues doesn’t make sense to me, I don’t understand what they were trying to say, and suggesting queues are simpler than actors seems backwards from a game programming perspective. Consistency and Inconsistency are things you can have in any programming language at any level of complexity.
Appreciate the answer!
> But we can’t move the notion of time to an element of it’s own. And if we could, and then compose it with an element of value, we’d end up right back where we started with some state.
I see this differently. If you compose a timestamp with a value, it is still a value. This is the essence of temporal data modelling and accounting.
> You can’t create anything in a stateless system.
Agreed. The idea is however to avoid state whenever possible, pushing it to the edges.
> Methods vs namespaces functions does not have a meaningful difference in this context
Depends on how you use them. Classes and methods can be used as quasi namespaces. The nice thing about namespaces is that they provide you with that one, "simple" thing, while classes and methods are overloaded with semantics and utility.
> Data vs Syntax is not a choice in any programming language that exists, all of them have both.
The question is the degree. There are languages with minimal syntax that express more via data and there are languages with differing degrees of homoiconicity.
I agree with the rest of your critique generally, as in that the context and the assumptions around a given model are typically subjective, we objectively deduce the parts from these assumptions.
Say for example from my point of view a given tool might be simple, because I only use it to do that one thing, but you have to use it for that other (maybe intended) purpose as well, so you perceive it as complected.
> But we can’t move the notion of time to an element of it’s own. And if we could, and then compose it with an element of value, we’d end up right back where we started with some state.
I see this differently. If you compose a timestamp with a value, it is still a value. This is the essence of temporal data modelling and accounting.
> You can’t create anything in a stateless system.
Agreed. The idea is however to avoid state whenever possible, pushing it to the edges.
> Methods vs namespaces functions does not have a meaningful difference in this context
Depends on how you use them. Classes and methods can be used as quasi namespaces. The nice thing about namespaces is that they provide you with that one, "simple" thing, while classes and methods are overloaded with semantics and utility.
> Data vs Syntax is not a choice in any programming language that exists, all of them have both.
The question is the degree. There are languages with minimal syntax that express more via data and there are languages with differing degrees of homoiconicity.
I agree with the rest of your critique generally, as in that the context and the assumptions around a given model are typically subjective, we objectively deduce the parts from these assumptions.
Say for example from my point of view a given tool might be simple, because I only use it to do that one thing, but you have to use it for that other (maybe intended) purpose as well, so you perceive it as complected.
> I see this differently. If you compose a timestamp with a value, it is still a value. This is the essence of temporal data modelling and accounting.
I agree completely that composing a timestamp with a value is still a value. I think the concept of assignment (values changing over time) is not the same thing as a timestamp at all; a timestamp doesn't represent state in the same way that mutability does.
> The idea is however to avoid state whenever possible, pushing it to the edges.
Yeah, this is where I think the problem lies. Saying "avoid state" isn't particularly helpful, especially if there is a certain amount of state that is essential and necessary. What should I do with the state that is required? What does it mean to push state to the edges, and in what cases does pushing state to the edges simplify my code? What if pushing the state causes complications by having to introduce more complex communication or more complex ordering between the elements of a system?
I feel like one of the implicit suggestions when someone says to avoid state is that there is unnecessary state hanging around. But in practice, while there is sometimes redundant state, most programmers I've worked with are pretty conscientious about duplicating data. Most of the imperative production code I've worked with doesn't have a lot of redundant or duplicated state, it is already designed to minimize the amount of state.
I love writing functional code when I can, it does feel simpler and easy to reason about. The two reasons I can't always write functional code are: performance and necessary state. Performance for functional constructs in various languages is improving, but it's still extremely common for functional designs to be very slow compared to equivalent imperative stateful designs. (And I do a lot of high performance computing as part of my current job.) And we need a better philosophy than just "avoid it" when it comes to all the necessary, unavoidable state.
I agree completely that composing a timestamp with a value is still a value. I think the concept of assignment (values changing over time) is not the same thing as a timestamp at all; a timestamp doesn't represent state in the same way that mutability does.
> The idea is however to avoid state whenever possible, pushing it to the edges.
Yeah, this is where I think the problem lies. Saying "avoid state" isn't particularly helpful, especially if there is a certain amount of state that is essential and necessary. What should I do with the state that is required? What does it mean to push state to the edges, and in what cases does pushing state to the edges simplify my code? What if pushing the state causes complications by having to introduce more complex communication or more complex ordering between the elements of a system?
I feel like one of the implicit suggestions when someone says to avoid state is that there is unnecessary state hanging around. But in practice, while there is sometimes redundant state, most programmers I've worked with are pretty conscientious about duplicating data. Most of the imperative production code I've worked with doesn't have a lot of redundant or duplicated state, it is already designed to minimize the amount of state.
I love writing functional code when I can, it does feel simpler and easy to reason about. The two reasons I can't always write functional code are: performance and necessary state. Performance for functional constructs in various languages is improving, but it's still extremely common for functional designs to be very slow compared to equivalent imperative stateful designs. (And I do a lot of high performance computing as part of my current job.) And we need a better philosophy than just "avoid it" when it comes to all the necessary, unavoidable state.
The chart comes from Rich Hickey’s Simple Made Easy presentation. Watching that will provide missing context that may answer some of your questions.
https://www.infoq.com/presentations/Simple-Made-Easy/
https://www.infoq.com/presentations/Simple-Made-Easy/
I agree with your hard-to-parse comment. I think without having a face-to-face and showing code samples this can't really be communicated(which is further complicated that the code is in a form that we both won't be able to grok), but I'll try anyway.
I think this is driving at the crux of the matter. We've been building our applications on top of our databases. We do stuff, we call APIs, those APIs access databases, they return. The database holds the state. Files hold state. Outside dependencies hold state. State, within a normally designed OOP-based program, is at the inner-most level. Within your logic you query the state, and you thus depend on this state and need to inject those dependencies. When you need to unit test, you need to mock out these dependencies to return fake values.
The Hexagonal Architecture still has state, but that state lies exclusively in the outside layer. When you change the state you are changing it at the outer level. State changes then trigger calls to your domain, they trigger changes to your services. But all of your domains, all of your services are stateless. Given this input I translate this data structure to this output data structure. That does not change state, it merely produces a result. That result makes its way back up the stack until it reaches the outer-most layer. The outer-most layer can then decide to perform the next state change.
None of your inner code has any dependency injection, because you refuse dependencies. The core of your application is now pure. They are now inherently unit testable. Does this input provide this output. You don't need to worry about any dependencies, since none of your logic relies on anything but itself. Mocks become obsolete.
I think this is driving at the crux of the matter. We've been building our applications on top of our databases. We do stuff, we call APIs, those APIs access databases, they return. The database holds the state. Files hold state. Outside dependencies hold state. State, within a normally designed OOP-based program, is at the inner-most level. Within your logic you query the state, and you thus depend on this state and need to inject those dependencies. When you need to unit test, you need to mock out these dependencies to return fake values.
The Hexagonal Architecture still has state, but that state lies exclusively in the outside layer. When you change the state you are changing it at the outer level. State changes then trigger calls to your domain, they trigger changes to your services. But all of your domains, all of your services are stateless. Given this input I translate this data structure to this output data structure. That does not change state, it merely produces a result. That result makes its way back up the stack until it reaches the outer-most layer. The outer-most layer can then decide to perform the next state change.
None of your inner code has any dependency injection, because you refuse dependencies. The core of your application is now pure. They are now inherently unit testable. Does this input provide this output. You don't need to worry about any dependencies, since none of your logic relies on anything but itself. Mocks become obsolete.
I've tried putting these ideas into practice in a couple of previous jobs doing web development. Some applications I've written & worked on use React along with Redux for state management. For a different application that didn't use React, I used a pub-sub model to communicate state changes.
Both of these work really well for certain kinds of data, and both of them also fell apart for me when things got complicated. I haven't thought hard enough about the problems I ran into to be able to categorize them in any concrete or helpful way. But, I do think that the breakdown is related to having to communicate state between components and across the system. Sometimes state changes have a necessary dependency on ordering things which can get tricky if one side is trying hard to avoid state. Sometimes wiring up state changes to the components that need it just adds communication complexity.
The other thing that seems to happen when you put state on the outside is that your data schema evolves toward your code structure. In some ways, you end up duplicating the hierarchy. This has caused me trouble when refactoring. Once you realize some sizeable portion of code needs to be rewritten, but you can't change the data schema to match the new code because the data is production live, all the cleanness of the design starts to break down.
Both of these work really well for certain kinds of data, and both of them also fell apart for me when things got complicated. I haven't thought hard enough about the problems I ran into to be able to categorize them in any concrete or helpful way. But, I do think that the breakdown is related to having to communicate state between components and across the system. Sometimes state changes have a necessary dependency on ordering things which can get tricky if one side is trying hard to avoid state. Sometimes wiring up state changes to the components that need it just adds communication complexity.
The other thing that seems to happen when you put state on the outside is that your data schema evolves toward your code structure. In some ways, you end up duplicating the hierarchy. This has caused me trouble when refactoring. Once you realize some sizeable portion of code needs to be rewritten, but you can't change the data schema to match the new code because the data is production live, all the cleanness of the design starts to break down.
Tangled is that which measurably interacts with something else for any practical purposes at the current level of analysis. If there's a function A that does not call function B (and vice versa), they are not tangled. They are separate (assuming the functions are pure, no global shared state, etc).
Now we can go down the physics rabbit hole and start arguing that everything is tangled at some level but I don't think that's useful for a discussion at this level of analysis.
I think we can stick to the definition that if function A does not call function B, they are completely separate for the purpose of this discussion.
Now we can go down the physics rabbit hole and start arguing that everything is tangled at some level but I don't think that's useful for a discussion at this level of analysis.
I think we can stick to the definition that if function A does not call function B, they are completely separate for the purpose of this discussion.
Is there a good full example of Closure (or languages like Closure) that deals with a complete and real-world data-intensive problem (eg. order processing system) that I can go through to see how languages like Closure embody this simplicity philosophy against the backdrop of the existing Java/C# paradigm?
Here is an example that I'm quite partial to:
Presentation - https://www.infoq.com/presentations/DDD-Clojure/
Code - https://github.com/zololabs/zolodeck
Its a Domain-Driven/Hexagonal Architecture codebase written in Clojure. Its not uncommon to see this sort of approach in Java/C# so it makes comparisons of the code relatively easy.
While the overall code layout is the same as what you'd see in Java/C#, a big distinction is that the "entities" that make up the core functionality of the domain in the Clojure version are simple data structures (rather than classes), and the logic that operates on them are pure functions (rather than methods on the classes themselves).
With this approach, a lot of the hoops you'd usually jump through with dependency injection to enable testability go away, since its easy to test pure functions that operate on simple data structures, no mocking required.
This sort of approach isn't limited to Clojure [0], but Clojure makes it idiomatic.
[0] - https://www.destroyallsoftware.com/talks/boundaries
Presentation - https://www.infoq.com/presentations/DDD-Clojure/
Code - https://github.com/zololabs/zolodeck
Its a Domain-Driven/Hexagonal Architecture codebase written in Clojure. Its not uncommon to see this sort of approach in Java/C# so it makes comparisons of the code relatively easy.
While the overall code layout is the same as what you'd see in Java/C#, a big distinction is that the "entities" that make up the core functionality of the domain in the Clojure version are simple data structures (rather than classes), and the logic that operates on them are pure functions (rather than methods on the classes themselves).
With this approach, a lot of the hoops you'd usually jump through with dependency injection to enable testability go away, since its easy to test pure functions that operate on simple data structures, no mocking required.
This sort of approach isn't limited to Clojure [0], but Clojure makes it idiomatic.
[0] - https://www.destroyallsoftware.com/talks/boundaries
Your question make it sound like most order processing system are built on a Java / C# paradigm.
To clarify, that my limited experience with those type of system. Same for financial transactions.
But I also often assumed that serious people would use more narrow tools for that task. Or at least something more functional.
Are Java / c# the norm in those space ?
To clarify, that my limited experience with those type of system. Same for financial transactions.
But I also often assumed that serious people would use more narrow tools for that task. Or at least something more functional.
Are Java / c# the norm in those space ?
Simplicity is PHP – everything is included and you can get things done in minutes.
PHP has some neat concepts and attributes. It’s stateless, „serverless“, flexible, fast and has batteries included. This leads to ease and some simplicity.
But it’s plagued by footguns on one side and overengineered „OO“ frameworks on the other. These lead to complexity and suffering. The language requires too much discipline and „special“ knowledge for my taste.
But it’s plagued by footguns on one side and overengineered „OO“ frameworks on the other. These lead to complexity and suffering. The language requires too much discipline and „special“ knowledge for my taste.
That's easy, not simple.
I am not saying PHP is not simple.
What’s involved in building a hello world web server in java and deploying it? One wonders why PHP is not the gold standard that all languages should be aspiring to achieve with regards to going from zero to running server (if we are to borrow the good and leave the bad).
Since idiomatic PHP is effectively pseudo-Java it has the same incidental complexity as Java.
Complexity is something we can visualize. Take a piece of paper and draw a graph of your code. You‘ll need different types of lines and shapes. It helps to see problems and opportunities to simplify.
I think Rich Hickey's simplicity talks are an important contribution to our industry. I think they are an eloquent message that can help get developers on a similar page. But ultimately, I feel, it's just poetics.
If I asked the best developers in the world to write down all of their wisdom with the goal that they would only be allowed to criticize code they review if a computer could show that one of their rules was violated. And the take the results to an alien world. I assert we would find the vast majority of the rules not helpful. Or if we went to the past and changed the course of history such that all modern programming languages are (at least a little) different. Or if we even took the rules to a different sub industry (videogames to aerospace or vice versa).
Our ideas of quality, simplicity, and everything else all have an unbelievable high dependency on context (at least when they aren't coming from ego, arrogance, ignorance, preference, or anecdote). Even something as small as adding a new library/framework, getting a new team member, or changing a feature or product goal can be enough to invalidate or radically change what guidance we give for success.
We have best practices but that phrase is used to justify mutually exclusive and sometimes downright opposite courses of action. When we see something we dont like we call it a code smell. Smell. Like the most primal and instinct driven sense. Not "hey we have some data on why this is bad" but "this code makes my tummy feel bad".
So while I'm glad that people are still meditating on how to make the code simpler. It seems to come down to a religious exercise or a spiritual awakening. The individual comes back enlightened, energized, and quoting poetry. But whether or not anyone listens to them is dependent on how good of a story they can tell.
And whether or not I can participate depends on whether or not they can actually tell me anything real or true. "But what does it really mean," I ask. They blink as if confused. "But I just told you, yellow is the most simple of the colors."
I want you to tell me why my code is bad. But you have to tell me why before I show it to you. A mathematician can tell me why my calculus might be bad before seeing it. A scientist can tell me why my experiment is bad before seeing it. An engineer can tell me why my circuit is bad before seeing it. Even a writer can make some pretty good guesses as to why my story is bad before seeing it. Nobody has ever been able to tell me why the code was bad ahead of time in any way that wasn't hopelessly invalid by the end of the week.
If I asked the best developers in the world to write down all of their wisdom with the goal that they would only be allowed to criticize code they review if a computer could show that one of their rules was violated. And the take the results to an alien world. I assert we would find the vast majority of the rules not helpful. Or if we went to the past and changed the course of history such that all modern programming languages are (at least a little) different. Or if we even took the rules to a different sub industry (videogames to aerospace or vice versa).
Our ideas of quality, simplicity, and everything else all have an unbelievable high dependency on context (at least when they aren't coming from ego, arrogance, ignorance, preference, or anecdote). Even something as small as adding a new library/framework, getting a new team member, or changing a feature or product goal can be enough to invalidate or radically change what guidance we give for success.
We have best practices but that phrase is used to justify mutually exclusive and sometimes downright opposite courses of action. When we see something we dont like we call it a code smell. Smell. Like the most primal and instinct driven sense. Not "hey we have some data on why this is bad" but "this code makes my tummy feel bad".
So while I'm glad that people are still meditating on how to make the code simpler. It seems to come down to a religious exercise or a spiritual awakening. The individual comes back enlightened, energized, and quoting poetry. But whether or not anyone listens to them is dependent on how good of a story they can tell.
And whether or not I can participate depends on whether or not they can actually tell me anything real or true. "But what does it really mean," I ask. They blink as if confused. "But I just told you, yellow is the most simple of the colors."
I want you to tell me why my code is bad. But you have to tell me why before I show it to you. A mathematician can tell me why my calculus might be bad before seeing it. A scientist can tell me why my experiment is bad before seeing it. An engineer can tell me why my circuit is bad before seeing it. Even a writer can make some pretty good guesses as to why my story is bad before seeing it. Nobody has ever been able to tell me why the code was bad ahead of time in any way that wasn't hopelessly invalid by the end of the week.
Somehow, I don't feel like it's particularly meaningful to say that the point of _____ high level language is simplicity.
Unidirectional data flow is what i see Simplicity, bidirectional is not.
returning a [data, error] is simpler than a Try...catch.
Simplicity means it's easier to test and debug it, it's not about easier to read or to write, or to compose.
returning a [data, error] is simpler than a Try...catch.
Simplicity means it's easier to test and debug it, it's not about easier to read or to write, or to compose.
It can be simpler, but it isn't always.
Returning a [data, error] could necessitate passing that error back an arbitrary distance in the call stack until it's handled. This necessarily complicates every procedure between where the error is handled and where it is generated. try/catch bypasses that complication of the code, but complicates the static tracing ability of the system to identify the actual control flow.
Neither approach is strictly simpler as both approaches have distinct complications they add to the system. Your opinion is that simplicity is around debugging and testing, others have different ideas about it because they value different aspects of programming differently than you do.
EDIT: To expand on my point, passing errors back directly integrates the error control flow into the "happy path" control flow. It's for a similar reason that I consider programs written to take advantage of concurrency to be "simpler" than straight linear programs, at least in many cases though not all. Even when the concurrency includes blocking statements that make the two versions equivalent in actual execution time (that is, no parallelism). The reason being that concurrency permits a separation of concerns rather than a tight integration. Real world programs often have a structure like this, where actions on different contexts are interleaved:
Having a single control flow that integrates both error handling and happy path data handling can produce a similar kind of obfuscated interleaving of code which makes the actual program intent (both the error handling and the happy path data handling) harder to understand or maintain in the long run. Again, not always, there are always trade offs. But neither [data, error] nor try/catch are obviously "simpler" than the other in any universal sense.
Returning a [data, error] could necessitate passing that error back an arbitrary distance in the call stack until it's handled. This necessarily complicates every procedure between where the error is handled and where it is generated. try/catch bypasses that complication of the code, but complicates the static tracing ability of the system to identify the actual control flow.
Neither approach is strictly simpler as both approaches have distinct complications they add to the system. Your opinion is that simplicity is around debugging and testing, others have different ideas about it because they value different aspects of programming differently than you do.
EDIT: To expand on my point, passing errors back directly integrates the error control flow into the "happy path" control flow. It's for a similar reason that I consider programs written to take advantage of concurrency to be "simpler" than straight linear programs, at least in many cases though not all. Even when the concurrency includes blocking statements that make the two versions equivalent in actual execution time (that is, no parallelism). The reason being that concurrency permits a separation of concerns rather than a tight integration. Real world programs often have a structure like this, where actions on different contexts are interleaved:
create graphics context
get some data
construct a renderable version of data
pass renderable version to graphics context
repeat with variations
graphics:
create context
loop:
receive data
render data
data (graphics context):
loop:
get data
construct renderable version of data
transmit data to graphics context
Having a single control flow that integrates both error handling and happy path data handling can produce a similar kind of obfuscated interleaving of code which makes the actual program intent (both the error handling and the happy path data handling) harder to understand or maintain in the long run. Again, not always, there are always trade offs. But neither [data, error] nor try/catch are obviously "simpler" than the other in any universal sense.
All Lisp-like languages incur significant complexity, but of these the simplest is probably some form of Scheme, definitely not Clojure. I think even Common Lisp may be simpler than Clojure.
Common Lisp has a condition system and optional OOP so more incidental complexity than Clojure.
Care to qualify this at all?
Lisps incurs significant complexity as opposed to which other simpler language?
C, Forth, you can throw Rust and C++ in there if we're talking runtime simplicity. Full-on Lisp requires a runtime that does GC, interns symbols, decodes machine words into pointers, bignums, etc. You can implement a constrained version of the language (GOAL comes to mind) that leaves some or all of these out, but you'd lose the convenience of the full Lisp and compatibility with programs written in it.
Brainfuck is even simpler than C. But a Brainfuck program is more complicated than a C program for any non-trivial computation.
While it’s fair to consider implementation complexity of the language and associated runtime or libraries, the programs produced in the language also have to be considered.
While it’s fair to consider implementation complexity of the language and associated runtime or libraries, the programs produced in the language also have to be considered.