Zeta: Functional Neural Networks in Ocaml(github.com)
github.com
Zeta: Functional Neural Networks in Ocaml
https://github.com/liaopeiyuan/zeta
9 comments
I had been messing around with a similar concept not too long ago in OCaml- as it turns out, strong, static typing can be very nice to have in writing ML models (and code in general). I find it reduces some of the cognitive overhead that comes from having to keep track of how the functions interact in the program (or perhaps, it reduces/clarifies the potential ways the functions can interact).
I'll be very interested to see where this project goes.
I'll be very interested to see where this project goes.
It's always been baffling to me that certain problem domains with extremely slow feedback loops have standardized on using dynamically typed languages. Machine learning and Python. Video game scripting and Lua. A lot of scientific programming as well.
How did that happen? Did the trend start because the work was done by domain specialists who were not necessarily expert programmers, and C was too unergonomic for non-experts to use? Is that the reason a Skyrim modder can't get immediate feedback from a compiler in 2020 - because 25 years ago game devs thought that documenting a C API was harder than embedding a scripting language in the game engine? Or perhaps because evaluating scripts was more secure than loading third-party DLLs and exposing the game's innards to them?
Edit: also, if every CSV file came with a schema, that would be great. Even if it says that every column is of type Option<Any> - at least then I know what to expect.
How did that happen? Did the trend start because the work was done by domain specialists who were not necessarily expert programmers, and C was too unergonomic for non-experts to use? Is that the reason a Skyrim modder can't get immediate feedback from a compiler in 2020 - because 25 years ago game devs thought that documenting a C API was harder than embedding a scripting language in the game engine? Or perhaps because evaluating scripts was more secure than loading third-party DLLs and exposing the game's innards to them?
Edit: also, if every CSV file came with a schema, that would be great. Even if it says that every column is of type Option<Any> - at least then I know what to expect.
I think it is more that 'expert programmers' where told to leave 'domain-specific' bits for power-users to deal with.
Therefore,
- game scene development in games, are in scripts. - Equity derivatives and swap contracts are in Excel - scientific modeling is in python
I also agree with your other though that, dynamic loading enables 're-use' of the core framework, so those frameworks/engines become marketable technology assets so to speak.
The programmers, then in some instances, created DSLs to then auto-generate performant framework-specific code from the DSL-written user specs.
But those, usually, are quite limited and also some power users migrated to be descent (not the expert engine/core-level, perhaps) programmers
Therefore,
- game scene development in games, are in scripts. - Equity derivatives and swap contracts are in Excel - scientific modeling is in python
I also agree with your other though that, dynamic loading enables 're-use' of the core framework, so those frameworks/engines become marketable technology assets so to speak.
The programmers, then in some instances, created DSLs to then auto-generate performant framework-specific code from the DSL-written user specs.
But those, usually, are quite limited and also some power users migrated to be descent (not the expert engine/core-level, perhaps) programmers
ML in ML?
Jokes aside, see:
https://github.com/LaurentMazare/tensorflow-ocaml
Also
http://ryanrhymes.blogspot.com/2017/03/build-neural-network-...
for how you can very quickly write something from scratch with "Ocaml's numpy", owl.
Jokes aside, see:
https://github.com/LaurentMazare/tensorflow-ocaml
Also
http://ryanrhymes.blogspot.com/2017/03/build-neural-network-...
for how you can very quickly write something from scratch with "Ocaml's numpy", owl.
I'm also working on something like this with F# and TensorFlow:
https://youtu.be/GUMTpTqzsJA
https://devpost.com/software/sylvester-tf
using typed natural number dimensions for arrays, vectors, matrices, tensors et.al
https://youtu.be/GUMTpTqzsJA
https://devpost.com/software/sylvester-tf
using typed natural number dimensions for arrays, vectors, matrices, tensors et.al
This looks awesome. I’m an F# developer and I agree with your intro paragraph 100%:
“One of the ironies of today's computer programming landscape is that functional languages directly inspired by the declarative languages for expressing abstractions and equations of logic and mathematics, have been sidelined for mathematical and scientific computing in favor of imperative, dynamically-typed languages like Python and Julia.”
Sadly, I’ve found that every road leads back to Python when it comes to deep neural nets. I tried using TensorFlowSharp from F# but found it very frustrating. Only Python seems to have full bindings to the TF API. Would love to give your wrapper a try.
“One of the ironies of today's computer programming landscape is that functional languages directly inspired by the declarative languages for expressing abstractions and equations of logic and mathematics, have been sidelined for mathematical and scientific computing in favor of imperative, dynamically-typed languages like Python and Julia.”
Sadly, I’ve found that every road leads back to Python when it comes to deep neural nets. I tried using TensorFlowSharp from F# but found it very frustrating. Only Python seems to have full bindings to the TF API. Would love to give your wrapper a try.
I've created a Jupyter notebook here: http://ec2-52-15-66-200.us-east-2.compute.amazonaws.com:6677...
that demos my ongoing progress with Sylvester and I'd love your feedback. It's also available on NuGet: https://www.nuget.org/packages/Sylvester.tf/0.2.3.4 with a native TF 2 package: https://www.nuget.org/packages/Sylvester.tf.Native.Win/.
It's still WIP but coming along pretty well.
that demos my ongoing progress with Sylvester and I'd love your feedback. It's also available on NuGet: https://www.nuget.org/packages/Sylvester.tf/0.2.3.4 with a native TF 2 package: https://www.nuget.org/packages/Sylvester.tf.Native.Win/.
It's still WIP but coming along pretty well.
It's great to hear that people are trying to use ocaml for ML :)
To me it seems like owl is the most mature ocaml project in this area. I recall they even had some GPU support. Unfortunately, I didn't have a chance to try it out so I can't comment first hand on how "ready" it is.
https://github.com/owlbarn/owl
https://github.com/owlbarn/owl
[0]: https://futhark-lang.org/student-projects/duc-bsc-thesis.pdf