A lot of valid concerns against arb code execution in prod for security, performance, auditability etc.
However, I do resonate somewhat with the post if I think about some accounting processes.
Accounting is where I came from, and a lot of data processing we do is mostly determinstic, with some "smartness" or judgement sprinkled in. Take for example bank reconciliation, the basic process is to match bank statement lines with accounting entry lines. In practice, dates, descriptions, and amounts often mismatch between the 2 for various reasons (typos, grouped bookings, value date vs transaction date differences, truncated values). This impacts a lot of SME's and these basic accounting processes are still manual as you need eyeballing. You look at a typical back office excel spreadsheet and will understand this.
You can pre-program the matching rules up to a certain point until it becomes unmaintainable. Or you can use LLM to generate data-dependent matching logic on the fly. I think there is a space for the latter approach, if we keep the scope tight and well contained. As with all engineering, it's about the trade-offs.
Useful targets for LLM to generate can be subsets of sql statements (create views and selects) or pure functions (haskell?), where side effects are strictly limited and there is only data in - data out. I am toying with SQL idea myself (GH: https://github.com/spoj/taskgraph).
However, I do resonate somewhat with the post if I think about some accounting processes.
Accounting is where I came from, and a lot of data processing we do is mostly determinstic, with some "smartness" or judgement sprinkled in. Take for example bank reconciliation, the basic process is to match bank statement lines with accounting entry lines. In practice, dates, descriptions, and amounts often mismatch between the 2 for various reasons (typos, grouped bookings, value date vs transaction date differences, truncated values). This impacts a lot of SME's and these basic accounting processes are still manual as you need eyeballing. You look at a typical back office excel spreadsheet and will understand this.
You can pre-program the matching rules up to a certain point until it becomes unmaintainable. Or you can use LLM to generate data-dependent matching logic on the fly. I think there is a space for the latter approach, if we keep the scope tight and well contained. As with all engineering, it's about the trade-offs.
Useful targets for LLM to generate can be subsets of sql statements (create views and selects) or pure functions (haskell?), where side effects are strictly limited and there is only data in - data out. I am toying with SQL idea myself (GH: https://github.com/spoj/taskgraph).