HackerTrans
TopNewTrendsCommentsPastAskShowJobs

bafe

no profile record

comments

bafe
·3 anni fa·discuss
Thank you for the extensive reply! The first part you mentioned is exactly what we were missing back then, we wanted to dynamically generate workflows starting from a configuration selected by the user. This wasn't really possible unless we would generate the YAML openflow specification ourselves. At the end we gave up and rolled our own simple tool that just does what little we need. This said, it is cool that you are considering offering a more code-friendly way to define workflows. I still think this doesn't offer the same level of dynamism of dragster, where you could easily design branching/conditional workflows. I suppose your considerations regarding the decorators/compilers go exactly in that direction
bafe
·3 anni fa·discuss
It was several months ago, so it is entirely possible. Back then I got the feeling that windmill tried to be more of a low code business/internal tool platform than a data/ETL workflow tool, I especially missed an expressive way to define workflows programmatically (i think you had a JSON schema but nothing as powerful as dragster where you can define a whole workflow in pure python)
bafe
·3 anni fa·discuss
I briefly tried windmill for a project that involved creating custom workflows on-demand from a configuration file. I can't recommend windmill for data pipelines, it is meant to be more of a low-code internal app platform like retool or budibase. It is meant for a relatively static workflow that requires human intervention, like a simple business process involving some API calls and humans approvals in the loop. For complex (and potentially reconfigurable) data pipelines, dragster is a much better choice
bafe
·3 anni fa·discuss
Looks like the use case for BPMN and an engine that can execute processes described in that notation.