Late Architecture with Functional Programming(infoq.com)
infoq.com
Late Architecture with Functional Programming
https://www.infoq.com/news/2023/04/late-arch-functional-programming/
28 comments
I think you need do-notation for true simplicity. This is because handling results, optionals etc without this is really hard to read, maybe more so than the imperative code equivalent. I hope TypeScript gets this but not holding my breath!
In a sufficiently expressive language, do-notation isn't that important. Consider the following pieces of (equivelent haskell)
f1 = do
fd <- openFile "/file/path"
contents <- readFile fd
let result = process contents
closeFile fd
return result
f2 = openFile "/file/path" >>= \fd ->
readFile_ fd >>= \contents ->
let result = process contents in
closeFile fd >>
return result
let process contents -> result indo-notation can be easily implemented using delimited continuations (ie. generators). Generators compose well and flatten tail calls so you don't need TCO or trampolines. The only notable issue is that one-shot delimited continuations like generators don't work with non-deterministic monads (ie. List). Multi-shot can be emulated by keeping a cache of past values and replaying the generator, but performance will suffer. See burrido [1] for a JavaScript do-notation implementation.
[1] https://github.com/pelotom/burrido
[1] https://github.com/pelotom/burrido
I believe it is possible to implement effective and fundamentally sound functional programming without types. Types just make it easier to avoid mistakes in functional programming. Like object oriented paradigms can be implemented in procedural languages like C, but they are easier in languages with built-in object oriented tools.
First class functions are definitely a must.
First class functions are definitely a must.
> I believe it is possible to implement effective and fundamentally sound functional programming without types.
I prefer working in statically typed languages, but I agree with you (though only if we use a colloquial definition of "sound").
And, what's more, I think the correspondent in the article would agree also. Michael Sperber is very familiar with Scheme, as evidenced by his publication history. I think your point actually connects with a line from the article:
> Components in functional programming are essentially just data types and functions, and these functions work without mutable state, [Sperber] said.
Even in a dynamically typed language, the data still "has a type" — but now we're talking about the "shape of the data" rather than a formal type system. Reasoning about data-shapes is perfectly valid, and still supports the style of code that Sperber talks about throughout the interview.
This also connects to the "Design Recipe" from the textbook How to Design Programs, which is an introductory CS text based on Racket, a derivative of Scheme. In the Design Recipe, students are told that they should write down the contract of their function before implementing the body of the function, and part of this is the specification of the "types" of the inputs and the output. Again, since the language is untyped, this has much more to do with conveying a sense of the "shape" of the data to the programmer, but it is still highly effective in guiding the design and implementation of a functioning software system.
I prefer working in statically typed languages, but I agree with you (though only if we use a colloquial definition of "sound").
And, what's more, I think the correspondent in the article would agree also. Michael Sperber is very familiar with Scheme, as evidenced by his publication history. I think your point actually connects with a line from the article:
> Components in functional programming are essentially just data types and functions, and these functions work without mutable state, [Sperber] said.
Even in a dynamically typed language, the data still "has a type" — but now we're talking about the "shape of the data" rather than a formal type system. Reasoning about data-shapes is perfectly valid, and still supports the style of code that Sperber talks about throughout the interview.
This also connects to the "Design Recipe" from the textbook How to Design Programs, which is an introductory CS text based on Racket, a derivative of Scheme. In the Design Recipe, students are told that they should write down the contract of their function before implementing the body of the function, and part of this is the specification of the "types" of the inputs and the output. Again, since the language is untyped, this has much more to do with conveying a sense of the "shape" of the data to the programmer, but it is still highly effective in guiding the design and implementation of a functioning software system.
> Types just make it easier to avoid mistakes
If you look at a language like Haskell, types are there to provide polymorphism and allow you to rearrange the language in convenient ways. It's impossible to say what is the single main use of types there, but this is certainly one of the most important ones. Another main use of types there is documenting interfaces.
People only get to say that types provide nothing else but correction if they decide that all the other uses are non-important and should be ignored. But then, that's a pretty trivial and useless information.
If you look at a language like Haskell, types are there to provide polymorphism and allow you to rearrange the language in convenient ways. It's impossible to say what is the single main use of types there, but this is certainly one of the most important ones. Another main use of types there is documenting interfaces.
People only get to say that types provide nothing else but correction if they decide that all the other uses are non-important and should be ignored. But then, that's a pretty trivial and useless information.
All programs have types because a sound functional program has well known inputs and outputs. A type system just enforces and encodes it to some degree, and if the compiler knows the types as well then further abstractions can be built on that.
[deleted]
I think there's a lot of truth in this. I worked for a startup that uses a functional language. Without the flexibility that gave us to adjust our architecture at various points along a sometimes torturous evolution, I don't think we would have survived to a successful exit. I firmly believe that had we been in a more "traditional" environment we'd be dead and that would have been that.
I talk to a lot of very smart functional programmers, and I'm very glad I've been exposed to it, it's greatly improved how I program things.
But I really wish FP advocates would learn more about paradigms like OO before criticising them. It's too often levelled against the strawman of industry, imperative, pseudo-oo. Take this:
Components in functional programming are essentially just data types and functions, and these functions work without mutable state, he said.
I could make the same arguments for doing "late architecture" with objects:
Components in object-oriented programming are essentially just objects and messages, and these messages work without mutable state, he said.
But I really wish FP advocates would learn more about paradigms like OO before criticising them. It's too often levelled against the strawman of industry, imperative, pseudo-oo. Take this:
Components in functional programming are essentially just data types and functions, and these functions work without mutable state, he said.
I could make the same arguments for doing "late architecture" with objects:
Components in object-oriented programming are essentially just objects and messages, and these messages work without mutable state, he said.
Most of the functional programmers I know have a deeper understanding of OOP than the OOP programmers I know. For example, most of the OOP programmers I know do not understand covariance and contravariance (whereas just about every functional programmer I know has mastered them), even though those concepts frequently come up in the context of OOP. People who study programming language theory tend to gravitate toward the functional paradigm, but it's not because they don't understand OOP.
If you aren't aware that contra and covariance are typically taught under the banner of the "liskov substitution principle" rather than with the name variance you don't know OOP better than OOP programmers. You just convinced yourself you did because you use different terminology. I think maybe the guy you're responding to is correct.
>most of the OOP programmers I know do not understand covariance and contravariance (whereas just about every functional programmer I know has mastered them)
I very much doubt this and I suspect it is more of a problem of how you framed it. I'm sure you would see more success if you framed it seeing if the following would compile.
I very much doubt this and I suspect it is more of a problem of how you framed it. I'm sure you would see more success if you framed it seeing if the following would compile.
List list = new ArrayList<String>;
Consumer printHashCode = o -> System.out.println(o.hashCode());
Consumer<String> printStringHashCode = printHashCode;Where is the variance in your example?
If you covariance, it's the first example because String is a subtype of Object. List is the same thing as List<Object>.
The first example is missing some parenthesis, but it should convoy my point.
The first example is missing some parenthesis, but it should convoy my point.
Assuming we are talking about java, List is not the same as List<Object>.
List<Object> list = new ArrayList<String>(); will fail to compile.
Generic types in Java have no variance by default. The only type that can be assigned to List<String> is exactly List<String>.
Your example would want to be List<? extends Object> foo = new List<String>();
There is a reason for this. Consider:
In general, you typically want to be covarient in type parameters that are used for arguements, and contravariant in type parameters that are used for return values. In the general case, this is unworkable (since types parameters are often used for both), so the only reasonable behaviour is to be invariant.
Raw arrays in Java are actually covariant - which is not reasonable - and quickly result in a runtime type exception when you get to the "bar[0] = new Integer(0)" portion of the above example.
Scala allows you to explicitly mark type parameters as covariant or contravariant, as bellow:
List<Object> list = new ArrayList<String>(); will fail to compile.
Generic types in Java have no variance by default. The only type that can be assigned to List<String> is exactly List<String>.
Your example would want to be List<? extends Object> foo = new List<String>();
There is a reason for this. Consider:
List<String> foo;
List<Object> bar = foo;
bar.add(new Integer(0));
In general, you typically want to be covarient in type parameters that are used for arguements, and contravariant in type parameters that are used for return values. In the general case, this is unworkable (since types parameters are often used for both), so the only reasonable behaviour is to be invariant.
Raw arrays in Java are actually covariant - which is not reasonable - and quickly result in a runtime type exception when you get to the "bar[0] = new Integer(0)" portion of the above example.
Scala allows you to explicitly mark type parameters as covariant or contravariant, as bellow:
class Foo[+T] //Foo is covariant in T
class Foo[-T] //Foo is contravarient in T
class Foo[T] //Foo is invarient in T
Did you often run into proper components ? beside Eclipse layer-0 and academic research I've never seen one (but I stopped caring around 2010 to be honest).
The majority of OO around is very mutation based.
The majority of OO around is very mutation based.
But programming in that OOP definition today effectively means you're using a functional language, erlang, pretending processes are objects, or using a dead language.
I think late architecture is orthogonal to functional, imperative and its solution is higher level than even functional programming and more abstract. I think the solution requires a new class of programming that is beyond APIs and functions. It's the realm of relationships and pluralities, scheduling, understanding of meaning. (The realm of LLMs)
I want software architecture to be cheap and easy to change without breaking any existing behaviours. I don't know much research on this subject.
I think APIs are the wrong thing to code against, because APIs are tied to architecture.
Some codebases are millions of lines big, it is extremely difficult to change a codebase's architecture when it is that big. It's either a lot of work or an extremely complicated program to rewrite sourcecode of that magnitude that isn't throwaway and not general purpose transformations.
If you have a codebase that has a client per thread and you want to refactor it to multiplex clients over threads. Or you want to add load shedding or backpressure to your microservice architecture, you have a large amount of work to do. If you want to add a field to a message or underlying library that your every microservice architecture uses, you have a lot of pain to go through. If you want to change the workflow of a digitised business procedure you have a LOT of refactoring to do.
My ideas on the solution to this problem include:
- Vector programming: represent software architecture as vector embeddings, code against sequences of vectors that are arranged into relationships and then the space where the vectors reside is concretised
- Compositional behavioural typing: We need some way of combining behaviours together, composing the code of functions such that code is interleaved rather than chaining together functions. In most programming languages I know, data is typed, not the behaviour, and behaviour cannot be changed without changing the code. I think behaviour is more useful to type than data. Composing behaviour is not g(f(x)), it's taking arbitrary parts of the innerworkings of f and g and applying behaviours to their mechanisms. Inheritance doesn't do it properly, because you have to rewrite functions to replicate the original behaviour with a tweak. React hooks don't do it. It's the missing idea of computer science. My dream is that classes are compatible if you can provide a mapping from the innerworkings of one class to another class and the code can seamlessly compose without rewriting. And it be open to extension. It's like metaprogramming for behaviour rather than types.
Code that that matches a "dispatcher" interface, automatically inferred by the compiler, and I can can insert a Load balancer or an ACL arbitrarily in the stack, anywhere.
I can change the plurality (one-to-many, many-to-one) of architectural relationships, such as insert a hierarchy of load balancers here between these objects. I can make my software internet scalable, multiregion, multithreaded by adding to the diagram. I can see the architecture as a giant relational diagram data structure and mutate it directly and cheaply and without breaking any existing behaviour.
I've been journalling about this on github.com/samsquire/ideas4
I also wrote https://devops-pipeline.com which is an dataflow/diagram based perspective of software infrastructure.
I want software architecture to be cheap and easy to change without breaking any existing behaviours. I don't know much research on this subject.
I think APIs are the wrong thing to code against, because APIs are tied to architecture.
Some codebases are millions of lines big, it is extremely difficult to change a codebase's architecture when it is that big. It's either a lot of work or an extremely complicated program to rewrite sourcecode of that magnitude that isn't throwaway and not general purpose transformations.
If you have a codebase that has a client per thread and you want to refactor it to multiplex clients over threads. Or you want to add load shedding or backpressure to your microservice architecture, you have a large amount of work to do. If you want to add a field to a message or underlying library that your every microservice architecture uses, you have a lot of pain to go through. If you want to change the workflow of a digitised business procedure you have a LOT of refactoring to do.
My ideas on the solution to this problem include:
- Vector programming: represent software architecture as vector embeddings, code against sequences of vectors that are arranged into relationships and then the space where the vectors reside is concretised
- Compositional behavioural typing: We need some way of combining behaviours together, composing the code of functions such that code is interleaved rather than chaining together functions. In most programming languages I know, data is typed, not the behaviour, and behaviour cannot be changed without changing the code. I think behaviour is more useful to type than data. Composing behaviour is not g(f(x)), it's taking arbitrary parts of the innerworkings of f and g and applying behaviours to their mechanisms. Inheritance doesn't do it properly, because you have to rewrite functions to replicate the original behaviour with a tweak. React hooks don't do it. It's the missing idea of computer science. My dream is that classes are compatible if you can provide a mapping from the innerworkings of one class to another class and the code can seamlessly compose without rewriting. And it be open to extension. It's like metaprogramming for behaviour rather than types.
Code that that matches a "dispatcher" interface, automatically inferred by the compiler, and I can can insert a Load balancer or an ACL arbitrarily in the stack, anywhere.
I can change the plurality (one-to-many, many-to-one) of architectural relationships, such as insert a hierarchy of load balancers here between these objects. I can make my software internet scalable, multiregion, multithreaded by adding to the diagram. I can see the architecture as a giant relational diagram data structure and mutate it directly and cheaply and without breaking any existing behaviour.
I've been journalling about this on github.com/samsquire/ideas4
I also wrote https://devops-pipeline.com which is an dataflow/diagram based perspective of software infrastructure.
I like where you're going with this. There is certainly room for improvement in programming in general, beyond just the new LLM applications that using existing programming paradigms.
It sounds like what you're suggesting is that we try to develop tools that look to an LLM to figure out which "functions" to call, and to write them if they don't already exist. In this way, a corpus of tokenized behaviors could be composited by an LLM to produce an on-the-fly program based in intent rather than syntax.
Is that a fair summary?
It sounds like what you're suggesting is that we try to develop tools that look to an LLM to figure out which "functions" to call, and to write them if they don't already exist. In this way, a corpus of tokenized behaviors could be composited by an LLM to produce an on-the-fly program based in intent rather than syntax.
Is that a fair summary?
Thanks for your lovely comment.
Yes you understand the thought! The thought isn't use ChatGPT to code existing programming languages but to take advantage of LLMs ability to traverse complicated relationships that result in logically composed behaviours.
A load balancer, access control list, pairwise zip, fork/join, map, reduce, interleaving, LIFO, FIFO, dispatcher, scheduling, binpacking, control loops, transformation, coordination, orchestration are all behaviours that could be useful in designing an architecture. An LLM probably knows the arguments they take and how to apply them to another scenario at an abstract level. They can match up the variables in each behaviour with what they map to.
For example, I don't need to write code that glues things together if the LLM knows where a load balancer should fit on a dispatcher
Yes you understand the thought! The thought isn't use ChatGPT to code existing programming languages but to take advantage of LLMs ability to traverse complicated relationships that result in logically composed behaviours.
A load balancer, access control list, pairwise zip, fork/join, map, reduce, interleaving, LIFO, FIFO, dispatcher, scheduling, binpacking, control loops, transformation, coordination, orchestration are all behaviours that could be useful in designing an architecture. An LLM probably knows the arguments they take and how to apply them to another scenario at an abstract level. They can match up the variables in each behaviour with what they map to.
For example, I don't need to write code that glues things together if the LLM knows where a load balancer should fit on a dispatcher
Indeed. On the front end side, there are many scaffolding steps required to even display a simple web page. Certainly all of those inner components and their functions could be subbed out with different appearance or different data using an LLM.
Today, that process is easier than ever but still very manual.
Today, that process is easier than ever but still very manual.
Great comment!
>I think late architecture is orthogonal to functional, imperative
Absolutely. From a truly architectural view, procedural, functional, and method-oriented (current OO) are really only variations on the call/return architectural style. Good and sometimes important distinctions, but not really that far apart. They are very much about computing, results from inputs. That is an appropriate architecture for fewer and fewer programs.
See Why Architecture Oriented Programming matters
https://blog.metaobject.com/2019/02/why-architecture-oriente...
and
Can Programmers Escape the Gentle Tyranny of call/return?
https://2020.programming-conference.org/details/salon-2020-p...
> its solution is higher level than even functional programming
Yes. Well, functional actually gets most of its utility from being lower level as far as paradigms go (less powerful). But yes.
> and more abstract
No. Well, yes, if expressed with current programming languages. But that's part of the problem set, not part of the solution set. We should be able to express our architectures less abstractly, more concretely, but for that we need linguistic support. Which is why I am working on that:
http://objective.st
> I want software architecture to be cheap and easy to change without breaking any existing behaviours. I don't know much research on this subject.
There was quite a bit of research at CMU, for example on packaging mismatch. Famous paper Architectural Mismatch, Why Reuse is so hard, and the 10 year follow up in 2009: Architectural Mismatch: Why Reuse is Still So Hard
https://repository.upenn.edu/cgi/viewcontent.cgi?article=107...*
Not much has changed since.
> https://github.com/samsquire/ideas4
> https://devops-pipeline.com
Will check those out. Dataflow is definitely a big part of it, with the extension of dataflow constraints (make, spreadsheets, "FRP"/"Rx"). But so is in-process REST with Storage Combinators!
And breaking down barriers between scripting and "real" programming.
>I think late architecture is orthogonal to functional, imperative
Absolutely. From a truly architectural view, procedural, functional, and method-oriented (current OO) are really only variations on the call/return architectural style. Good and sometimes important distinctions, but not really that far apart. They are very much about computing, results from inputs. That is an appropriate architecture for fewer and fewer programs.
See Why Architecture Oriented Programming matters
https://blog.metaobject.com/2019/02/why-architecture-oriente...
and
Can Programmers Escape the Gentle Tyranny of call/return?
https://2020.programming-conference.org/details/salon-2020-p...
> its solution is higher level than even functional programming
Yes. Well, functional actually gets most of its utility from being lower level as far as paradigms go (less powerful). But yes.
> and more abstract
No. Well, yes, if expressed with current programming languages. But that's part of the problem set, not part of the solution set. We should be able to express our architectures less abstractly, more concretely, but for that we need linguistic support. Which is why I am working on that:
http://objective.st
> I want software architecture to be cheap and easy to change without breaking any existing behaviours. I don't know much research on this subject.
There was quite a bit of research at CMU, for example on packaging mismatch. Famous paper Architectural Mismatch, Why Reuse is so hard, and the 10 year follow up in 2009: Architectural Mismatch: Why Reuse is Still So Hard
https://repository.upenn.edu/cgi/viewcontent.cgi?article=107...*
Not much has changed since.
> https://github.com/samsquire/ideas4
> https://devops-pipeline.com
Will check those out. Dataflow is definitely a big part of it, with the extension of dataflow constraints (make, spreadsheets, "FRP"/"Rx"). But so is in-process REST with Storage Combinators!
And breaking down barriers between scripting and "real" programming.
Thanks for such a useful, good reply.
Thank you for the links!
Thank you for the links!
> Vector programming: represent software architecture as vector embeddings, code against sequences of vectors that are arranged into relationships and then the space where the vectors reside is concretised
Sounds wonderful!
(What does it mean?)
Sounds wonderful!
(What does it mean?)
In programming languages with generics you can substitute an implementation that is compatible.
I want to program my code that it can interoperate with many other pieces of software, without too much deliberate effort.
It means that the computer can work out how to wire one class with another class based on the desired behaviours of each class.
I want to program my code that it can interoperate with many other pieces of software, without too much deliberate effort.
It means that the computer can work out how to wire one class with another class based on the desired behaviours of each class.
I believe Mike Sperber is talking about Haskell when he says "advanced abstraction", but most regular line-of-business software can reap significant benefits with just simple FP principles: mostly immutable data, record types, sum types, and a vocabulary of small decomposed functions that can parse, transform, and combine this data, together building to larger wholes.
You do need a language with types and first-class functions, and the most common vehicle for that currently is TypeScript. Combine that with a book like Grokking Simplicity by Eric Normand, especially the fundamental idea of separating data, computation, and action, and you have a software system that is supple, simple, and amenable to continuous growth and refactoring over a long period of time.
Grokking Simplicity, IMO, is a severely under-discussed book when it comes to FP. Its only sticking point for me is its lack of static types, but otherwise it is the first and only book that I've seen that distils the functional way of thinking for the working programmer in an accessible way, without having to resort to the all-or-nothing proposition of completely pure FP.