Completely agree with this view. I wish I had seen this thesis earlier when developing redun (https://github.com/insitro/redun/). Looks like a lot of very useful ideas for defining such a system.
We based redun's execution model on very similar ideas of functional programming and graph reduction. In addition, we made it work as an embedded DSL within Python, so one can easily use all the typical data science and ML libraries in a workflow. This has been very helpful for building biotech workflows (genomics, imaging, chem).
I am a bit surprised why many workflow systems shy away from full turing-complete. You usually don't need to trade that away for automatic parallelism, caching, etc.
If you are interested in knowing more about the differences of a pull-style workflow engine like Snakemake which is geared towards Bioinfo problems vs a push-style workflow engine which is geared towards data engineering, you might find our write up helpful: https://insitro.github.io/redun/design.html#influences
There are other important dimensions on which workflow engines differ, such as reactivity, file staging, and dynamism.
We based redun's execution model on very similar ideas of functional programming and graph reduction. In addition, we made it work as an embedded DSL within Python, so one can easily use all the typical data science and ML libraries in a workflow. This has been very helpful for building biotech workflows (genomics, imaging, chem).
I am a bit surprised why many workflow systems shy away from full turing-complete. You usually don't need to trade that away for automatic parallelism, caching, etc.