OCaml as the Universal Translator – DkSDK Merry Christmas 2023(gitlab.com)
gitlab.com
OCaml as the Universal Translator – DkSDK Merry Christmas 2023
https://gitlab.com/diskuv/samples/merrychristmas2023#ocaml-as-the-universal-translator-dksdk-merry-christmas-2023
8 comments
Author here. In the article I highlighted how easy it is now to translate OCaml line-by-line into other languages while at the same time preserving exact semantics. That was the cool news that deserved its own callout. But as you noticed I didn't spend any time talking about how translating OCaml helps solve the problem of managing types "all with slightly different variations".
Briefly: atd, Cap n' Proto, Protobuf and many others occupy the role of representing the same datatype in different languages and transmitting values of that datatype across the wire and/or persisting in storage. But I'm finding most programs have a need for multiple variations to that core record datatype. For example, when you persist an OCaml/Java/etc record datatype to a database, you don't use the OCaml/Java/etc record datatype. You use the data types provided by the database (VARCHAR, BLOB, etc). You can adopt an ORM solution, use an object database, or write a pair of type-conversion functions from OCaml/Java/etc to your database datatype. That is one example of a "variation" that you must spend engineering time on.
I mentioned in the article that my user needed 5 different variations. For example, with the sqlite3 database they were using, the database table variation required writing a pair of datatype-conversion functions to and from the sqlite3 database. What if there was a way where we could write these datatype-conversion functions in one place (cough cough: OCaml with a line-by-line semantics preserving feature) and have bindings to/from the database, the view layer, the controller and all the other variations generated in the appropriate Java/Swift/SQL/etc languages? And what if we could deploy modifications to that datatype (ex. adding a database field) in the same uniform way we do all other minor variations to a datatype? That is the "drastic QOL (quality of life) improvement" win my users asked for.
That QOL win deserves its own article. Yes, I'll post it when done.
Briefly: atd, Cap n' Proto, Protobuf and many others occupy the role of representing the same datatype in different languages and transmitting values of that datatype across the wire and/or persisting in storage. But I'm finding most programs have a need for multiple variations to that core record datatype. For example, when you persist an OCaml/Java/etc record datatype to a database, you don't use the OCaml/Java/etc record datatype. You use the data types provided by the database (VARCHAR, BLOB, etc). You can adopt an ORM solution, use an object database, or write a pair of type-conversion functions from OCaml/Java/etc to your database datatype. That is one example of a "variation" that you must spend engineering time on.
I mentioned in the article that my user needed 5 different variations. For example, with the sqlite3 database they were using, the database table variation required writing a pair of datatype-conversion functions to and from the sqlite3 database. What if there was a way where we could write these datatype-conversion functions in one place (cough cough: OCaml with a line-by-line semantics preserving feature) and have bindings to/from the database, the view layer, the controller and all the other variations generated in the appropriate Java/Swift/SQL/etc languages? And what if we could deploy modifications to that datatype (ex. adding a database field) in the same uniform way we do all other minor variations to a datatype? That is the "drastic QOL (quality of life) improvement" win my users asked for.
That QOL win deserves its own article. Yes, I'll post it when done.
> And what if we could deploy modifications to that datatype (ex. adding a database field) in the same uniform way we do all other minor variations to a datatype?
If you mean propagating changes to a schema across all layers of the stack, that may not necessarily be a good idea. In my experience we want to keep the datatype definitions at different layers somewhat independent of each other, to allow evolving separately. For example, the database table could have a `more JSON` column to allow adding data that was not foreseen in the original table design; but the application's datatype would decode that JSON into specific fields to show the user perhaps.
If you mean propagating changes to a schema across all layers of the stack, that may not necessarily be a good idea. In my experience we want to keep the datatype definitions at different layers somewhat independent of each other, to allow evolving separately. For example, the database table could have a `more JSON` column to allow adding data that was not foreseen in the original table design; but the application's datatype would decode that JSON into specific fields to show the user perhaps.
I'd consider that an extension of the "object database" approach mentioned. Just as you mention the application needs to decode (and encode and deal with missing fields) that JSON. And probably in any sufficient size of application you'd have to do this decode/encode in more than one place. That JSON encoding/decoding is a "variation" (more explicitly, a reshaping operation). The engineering costs of having variations strewn across your source code hasn't disappeared.
Regardless of how awesome this may be, I cannot get over how there are 6 (!) Levels of nesting in the code sample. I hope it's simply for the sake of keeping the sample short, but this kind of "Chevron programming" is generally hard to read.
That's fair. Assuming I introduced a new macro "compactexpr" it could have been written:
Of course, there is a disadvantage. I had to use global variables in the above "compactexpr". Global variables mimic the C language (ie. C functions are globally-scoped or file-scoped functions) but wouldn't be a good fit when translating that code to functional languages. Additionally, the OCaml idiom `fun arr n ->` was lost, but that seems minor.
Do you and others find the above readable? (If so, perhaps it would be best to give some flexibility for the coding style.)
[%%compactexpr
"SumArStaged";
[ arr; n ];
let sum = ref 0 in
for i = 0 to 3 do
for j = i to min (i + 3) (n - 1) do
sum := !sum + arr.(j)
done
done;
!sum]
;;
print_c "sum_ar" (module SumArConv) SumArStaged.expr;;
Of course, there is a disadvantage. I had to use global variables in the above "compactexpr". Global variables mimic the C language (ie. C functions are globally-scoped or file-scoped functions) but wouldn't be a good fit when translating that code to functional languages. Additionally, the OCaml idiom `fun arr n ->` was lost, but that seems minor.
Do you and others find the above readable? (If so, perhaps it would be best to give some flexibility for the coding style.)
IshKebab(1)
https://atd.readthedocs.io/en/latest/atdgen.html