Ferret: A functional, lazy language for realtime embedded control systems(ferret-lang.org)
ferret-lang.org
Ferret: A functional, lazy language for realtime embedded control systems
https://ferret-lang.org/
32 comments
Ferret is a literate program! Check out the source/manual, it's quite a trip:
- https://github.com/nakkaya/ferret/blob/master/ferret.org
- https://en.wikipedia.org/wiki/Literate_programming
- https://github.com/nakkaya/ferret/blob/master/ferret.org
- https://en.wikipedia.org/wiki/Literate_programming
The problem with these alternative programming environments is that the developers push off everything other than compiling code. Then the community is left to struggle for years with:
1. How to debug in the language that they write in.
2. How to write unit tests.
3. How to debug unit tests.
4. Common IDE and code-editor integration for 1, 2 and 3.
5. What's the package management story?
For example, the newer programming environment that's released in most recent history that attempts to do an excellent job at this, Deno, still struggles with basics such as IDE integration for debugging unit tests. It's a struggle.
I've heard the creator of Clojure all but say he has put no thought into package management. Nice language, but it's not worth the cost of having to change how the rest of us work when there's existing environments that address the very basics that a developer needs.
For example, the newer programming environment that's released in most recent history that attempts to do an excellent job at this, Deno, still struggles with basics such as IDE integration for debugging unit tests. It's a struggle.
I've heard the creator of Clojure all but say he has put no thought into package management. Nice language, but it's not worth the cost of having to change how the rest of us work when there's existing environments that address the very basics that a developer needs.
> I've heard the creator of Clojure all but say he has put no thought into package management. Nice language, but it's not worth the cost of having to change how the rest of us work when there's existing environments that address the very basics that a developer needs.
Clojure relies on maven for this. Maven is a pretty accomplished package manager in the Java ecosystem. If Rich Hickey said that I assume he meant that he did not _have_ to invent a package manager as it comes for free from the underlying system.
Clojure relies on maven for this. Maven is a pretty accomplished package manager in the Java ecosystem. If Rich Hickey said that I assume he meant that he did not _have_ to invent a package manager as it comes for free from the underlying system.
To add, on Clojure you can also easily use git repos and local directories specified in deps.edn - these tend to get used mostly for private/internal deps.
ClojureScript side mostly uses the Maven mechanism for getting ClojureScript deps from Clojars (a Maven repo), but can also import Node modules for interop.
ClojureScript side mostly uses the Maven mechanism for getting ClojureScript deps from Clojars (a Maven repo), but can also import Node modules for interop.
When I see "lazy" and "real-time" together, I have a cognitive dissonance. Laziness is often unpredictable.
How does Ferret address that?
How does Ferret address that?
Clojure is evaluated eagerly but has lazy data structures. That means you can reason about performance, but you get to work with data in a more functional way than you otherwise could. I imagine ferret follows that example.
Why would laziness be "unpredictable"?
Also I'm pretty sure Ferret is strict, since it is not mentioned on their web page.
edit: it's mentioned in the "overview" section, probably the OP link pointed there[0].
[0] : https://ferret-lang.org/#outline-container-sec-2
Also I'm pretty sure Ferret is strict, since it is not mentioned on their web page.
edit: it's mentioned in the "overview" section, probably the OP link pointed there[0].
[0] : https://ferret-lang.org/#outline-container-sec-2
A good dive into of why lazy IO is a problem are these Oleg slides: https://okmij.org/ftp/Haskell/Iteratee/IterateeIO-talk-notes...
My favorite slide:
I can talk a lot how disturbingly, distressingly wrong lazy IO is theoretically, how it breaks all equational reasoning. Lazy IO entails either incorrect results or poor optimizations. But I won’t talk about theory. I stay on practical issues like resource management. We don’t know when a handle will be closed and the corresponding file descriptor, locks and other resources are disposed. We don’t know exactly when and in which part of the code the lazy stream is fully read: one can’t easily predict the evaluation order in a non-strict language. If the stream is not fully read, we have to rely on unreliable finalizers to close the handle. Running out of file handles or database connections is the routine problem with Lazy IO. Lazy IO makes error reporting impossible: any IO error counts as mere EOF. It becomes worse when we read from sockets or pipes. We have to be careful orchestrating reading and writing blocks to maintain handshaking and avoid deadlocks. We have to be careful to drain the pipe even if the processing finished before all input is consumed. Such precision of IO actions is impossible with lazy IO. It is not possible to mix Lazy IO with IO control, necessary in processing several HTTP requests on the same incoming connection, with select in-between. I have personally encountered all these problems. Leaking resources is an especially egregious and persistent problem. All the above problems frequently come up on Haskell mailing lists.
My favorite slide:
I can talk a lot how disturbingly, distressingly wrong lazy IO is theoretically, how it breaks all equational reasoning. Lazy IO entails either incorrect results or poor optimizations. But I won’t talk about theory. I stay on practical issues like resource management. We don’t know when a handle will be closed and the corresponding file descriptor, locks and other resources are disposed. We don’t know exactly when and in which part of the code the lazy stream is fully read: one can’t easily predict the evaluation order in a non-strict language. If the stream is not fully read, we have to rely on unreliable finalizers to close the handle. Running out of file handles or database connections is the routine problem with Lazy IO. Lazy IO makes error reporting impossible: any IO error counts as mere EOF. It becomes worse when we read from sockets or pipes. We have to be careful orchestrating reading and writing blocks to maintain handshaking and avoid deadlocks. We have to be careful to drain the pipe even if the processing finished before all input is consumed. Such precision of IO actions is impossible with lazy IO. It is not possible to mix Lazy IO with IO control, necessary in processing several HTTP requests on the same incoming connection, with select in-between. I have personally encountered all these problems. Leaking resources is an especially egregious and persistent problem. All the above problems frequently come up on Haskell mailing lists.
> Why would laziness be "unpredictable"?
Well, it's an empirical observation that Haskell programmers, however experienced they may be, are constantly surprised by the memory footprint of their programs — IIRC sometimes even the GHC's core developers get blindsided.
Well, it's an empirical observation that Haskell programmers, however experienced they may be, are constantly surprised by the memory footprint of their programs — IIRC sometimes even the GHC's core developers get blindsided.
Memory footprint doesn't have much to do with realtime though.
Of course it does, asymptotic memory behavior is the #1 reason for temporal instability.
That's only for long running programs generally with a GC. Ferret does not have a GC by default and furthermore can run without heap allocation if you want.
One of the ways to get lower latency is in fact to never deallocate and just ensure you won't run out of memory until the task is done.
One of the ways to get lower latency is in fact to never deallocate and just ensure you won't run out of memory until the task is done.
You just described practical examples of how to lower temporal instability by controlling memory allocation behavior :)
Yes I am. And the ones I'm referencing clearly show that ballooning memory is not equal to higher latency, as you claimed. You might say there is a correlation but those are due to other factors and not amount of memory used.
Ballooning memory can be best for real time, if you can afford it: allocation becomes trivial, reclamation is absent.
I remember a story about a rocket which had a range limitation due to the amount if RAM onboard because it never freed any memory. When the engine of the rocket was upgraded, they simply doubled the RAM to let the firmware work for a longer time.
I remember a story about a rocket which had a range limitation due to the amount if RAM onboard because it never freed any memory. When the engine of the rocket was upgraded, they simply doubled the RAM to let the firmware work for a longer time.
I've worked on a real-time system in Haskell and it is possible but iirc you do avoid laziness.
It's not "unpredictable" as in "stochastic", but rather as in "hard to reason about".
This may be compiler-specific. Say, Haskell does not give you much heads-up on laziness: whether stream fusion actually kicks in where you expect it to, or are you encountering quadratic behavior due to laziness [1].
If Ferret dives you decent visibility into issues like these, laziness may be usable even in hard-realtime setting, because it would become obvious and predictable for the developer.
[1]: https://accidentallyquadratic.tumblr.com/post/114149142587/r...
This may be compiler-specific. Say, Haskell does not give you much heads-up on laziness: whether stream fusion actually kicks in where you expect it to, or are you encountering quadratic behavior due to laziness [1].
If Ferret dives you decent visibility into issues like these, laziness may be usable even in hard-realtime setting, because it would become obvious and predictable for the developer.
[1]: https://accidentallyquadratic.tumblr.com/post/114149142587/r...
> whether stream fusion actually kicks in where you expect it to
An aside, but I remembered seeing something that let you test for that, likely https://hackage.haskell.org/package/inspection-testing
An aside, but I remembered seeing something that let you test for that, likely https://hackage.haskell.org/package/inspection-testing
Seems like there has been no development since 2020 - https://github.com/nakkaya/ferret
That means it’s stable, not dead.
Maybe, though I'd say this is definitely the exception rather than the rule by a wide margin.
Last issue opened was over a year ago. It is still open, no replies.
Last issue opened was over a year ago. It is still open, no replies.
check roadmap section. if someone was trying to follow that road map there should be at least minimal activity.
Does anyone know how it compares to ulisp? http://www.ulisp.com/
Looks like memory pooling is the interesting feature it has over ulisp. But Ferret is compiled while ulisp is interpreted.
I'm confused. Is Ferret a Clojure compiler? Because the code is Clojure, and the file extensions are CLJ, but it's able to compile Clojure into C++? Why is it called a language?
> Ferret is a functional, lazy language designed to be used in real time embedded control systems. It is heavily inspired by Clojure both syntactically and semantically. Functions / Macros that are present in both Ferret and Clojure should mimic their Clojure counter parts. If they don't it is considered a bug. (or not possible to implement with the current implementation.)
> This document is not intended to be a full lisp tutorial. It is a specification of the subset of lisp implemented by Ferret, and the particular workings of the Runtime and Core library. Any getting started guide for Clojure should get you upto speed on Ferret.
From the outline on the site: https://ferret-lang.org/#outline-container-sec-2
> This document is not intended to be a full lisp tutorial. It is a specification of the subset of lisp implemented by Ferret, and the particular workings of the Runtime and Core library. Any getting started guide for Clojure should get you upto speed on Ferret.
From the outline on the site: https://ferret-lang.org/#outline-container-sec-2
I remember reading about this years ago but I don't remember the "realtime" aspect. I remembered it just being a general Clojure-to-C++ compiler. I wonder if the authors "pivoted"?
I'm curious about the real-time aspect as well, especially for a lazy language. I didn't know that that was possible.
I couldn't find any documentation in its runtime about how it plans to handle interrupts so I would wager it's probably "soft" real-time. Also not sure if the generated code can throw exceptions but if it can then hard realtime is out.
Nuru, the developer for this is an absolute 10x god.
for something similar with same RAM requirements see ulisp but using common lisp as a reference subset
www.ulisp.com
www.ulisp.com
- Eowyn stole computer parts in anti-static bags and kept them in the cage. I gave up trying to keep SATA cables anywhere but her stash, because she was better at finding them in boxes than I was. Wouldn't bother other stuff, but if it was in an anti-static bag, it was hers.
- Feromir and Eowyn just disagreed on where the scarf went - downstairs, or up under the couch. Unfortunately for Eowyn, mass matters for traction, so when the game got less fun, it went under the couch.
- Blanco... stole potpourri. Not even sure where he found some of it, but we found a couple little stashes of it when we moved.
- Frodo had a thing for stuffed animals, and would either relocate them under a corner table if they were small enough, or to a different corner if they were too large. It's really quite funny watching a ferret struggle with a stuffed animal several times their size.
They're wonderfully quirky little animals, but definitely not a name I'd use for a realtime language. If it involved finding and hiding things, though, totally a suitable name!