Clojure: `every-pred` and `some-fn`(lambdaisland.com)
lambdaisland.com
Clojure: `every-pred` and `some-fn`
https://lambdaisland.com/blog/2019-12-03-advent-of-parens-3-some-fn-every-pred
43 comments
Thank you for sharing this! Another useful library with "missing" functions is Medley[0] by James Reeves.
[0] https://github.com/weavejester/medley
[0] https://github.com/weavejester/medley
The thing to understand about Clojure is that its creator/maintainer believes software tools should almost never deprecate features. Because of this, there are a few esoteric functions and features that were created in the language along the way that will likely always remain in the language- The two examples given by OP arguably fall into that category, though I'm sure someone in this thread will attempt to argue that every-pred & some-fn are essential features for the core language.
Other oddities include juxt, areduce, fnil, nthnext, derive, seque, and by now arguably the whole "ref" memory system (since Software Transactional Memory has pretty much gone by the wayside as a desirable programming feature)
Of course, Rich Hickey has also been very good at deciding which features to put into the language in the first place, so the number of legacy oddities is pretty tiny anyway, fewer than most languages.
Other oddities include juxt, areduce, fnil, nthnext, derive, seque, and by now arguably the whole "ref" memory system (since Software Transactional Memory has pretty much gone by the wayside as a desirable programming feature)
Of course, Rich Hickey has also been very good at deciding which features to put into the language in the first place, so the number of legacy oddities is pretty tiny anyway, fewer than most languages.
> Rich believes software tools should almost never deprecate features
This is pretty severely mis-stated
He believes the core of a dynamic programming language (specifically the core) has a very high bar for stability in order to avoid a number of problems that other dynamic languages have. This intersects with a number of other ideas, for example Lisp macros can extend syntax from a library and thus keep the change-restricted surface area small since almost anything can be implemented in a library.
I think, don't recall a smoking gun quote for this, correct me if I'm wrong.
This is pretty severely mis-stated
He believes the core of a dynamic programming language (specifically the core) has a very high bar for stability in order to avoid a number of problems that other dynamic languages have. This intersects with a number of other ideas, for example Lisp macros can extend syntax from a library and thus keep the change-restricted surface area small since almost anything can be implemented in a library.
I think, don't recall a smoking gun quote for this, correct me if I'm wrong.
Have you watched his "spec-ulation" talk? It makes a very strong case against EVER removing a feature from an API. https://www.youtube.com/watch?v=oyLBGkS5ICk
If you check his remarks from Spec-ulation (https://www.youtube.com/watch?v=oyLBGkS5ICk), he also thinks just plain old libraries should not remove or change functionality, but only accrete. If you want to make a breaking change, you should change the name of the library. You can see this philosophy in practice with clojure.spec.alpha-2: https://github.com/clojure/spec-alpha2
That's SemVer with the version in the name.
I don't see why these are so weird, logical composition of predicates is really common.
Hmm... I guess that's a valid point, though I would never use them for logical composition simply because the names of these functions really hurt readability: They are basically higher-order versions of "and" and "or", maybe if they had those words in their name they would be less cumbersome.
I think 'every' and 'some' are reasonable analogues, but the choice to make the implementations and hence their suffixes asymmetric wrt 'truthiness handling' is confusing to me, especially since these functions have existed (symmetrically) for a while in various schemes under the names 'conjoin' and 'disjoin'
I agree that I'd be far more likely to use these if they were named "conjoin" and "disjoin".
These names are even worse. I know Clojure likes this style, but to me, "conjoin" is just a fancy way of writing "join" if you want another distinct symbol naming the same, very generic concept.
"some" and "every" create nicely readable code. "every-pred" and "some-fn" could be called "satisfies-every" and "satisfies-some", respectively.
"some" and "every" create nicely readable code. "every-pred" and "some-fn" could be called "satisfies-every" and "satisfies-some", respectively.
I wrote both of these functions and conjoin and disjoin were definitely considered but were ultimately dismissed to avoid confusion with `conj` and `disj`. The chosen names were meant to echo `some?` and `every?`.
I personally don't find it that hard remembering them. I am sure all of Clojure's names could be more regimented, but I will still never forget how much I laughed when someone tried to tell me how well designed the Haskell lens operators were:
https://hackage.haskell.org/package/lens-4.18.1/docs/Control...
https://hackage.haskell.org/package/lens-4.18.1/docs/Control...
This article explains the rationale behind the naming scheme. It's quite regular.
https://gist.github.com/ChrisPenner/1f7b6923448b3396a45d04a2...
https://gist.github.com/ChrisPenner/1f7b6923448b3396a45d04a2...
Yes, I understand it's regular. It doesn't stop it being an absolute abomination. My point is that having regularity in naming doesn't stop an API being horrible, and not having regularity in naming doesn't _make_ an API horrible.
digging into the clojure core implementation, it looks like this asymmetry originates with 'every?' versus 'some', which test predicates against collections, the former being boolean, and the later returning the first non-falsy element. again, this feels a little non-committal; it's true that the not-strictly-boolean behavior of the disjunctive forms is handier than the conjunctive ones, but the latter can still come in handy.
The irregular naming of "some" vs "some?" has been the source of several tricky bugs for me in the past, and I suspect many other Clojurians would report this as well.
Indeed this is one of the first steps Paul Graham goes through in On Lisp.
Are there more recent features of Clojure that you think have made every-pred and some-fn less useful? I'm curious what they are if that is the case.
So, for the every-pred example, I'd prefer to use clojure piping for better readability:
For the some-fn example, I would argue that the usage in OP is a very idealized and tuned example... how often do you really want to say "try this one predicate and if that doesn't work, try this other one"? I only need to do that maybe a couple of times a year (when parsing some sort of irregular raw data) so it's rare enough that I'd prefer to write the predicate explicitly instead of using this sort of succinct predicate composition that could become a "head-scratcher" at a future date.
(->> people (filter :admin?) (filter :mod?))
For the some-fn example, I would argue that the usage in OP is a very idealized and tuned example... how often do you really want to say "try this one predicate and if that doesn't work, try this other one"? I only need to do that maybe a couple of times a year (when parsing some sort of irregular raw data) so it's rare enough that I'd prefer to write the predicate explicitly instead of using this sort of succinct predicate composition that could become a "head-scratcher" at a future date.
If you’re going to do this, transducers let you avoid the intermediate sequences and are nearly just as readable:
(into []
(comp (filter :admin?)
(filter :mod?))
people)
So I find very few examples of people using transducers in production code these days. I agree they are very compelling from a theoretical programming standpoint, and I used them heavily for about a year for that reason. However, it clearly took me far longer to debug transducing code compared to traditional code, so these days I only use them when absolutely necessary for optimization purposes.
Interesting, I find the ergonomics of transducers a lot better than the threading macros, especially because it’s a more generic interface. Also, they seem to be an ideal point in the lazy/strict evaluation continuum: they compose like lazy functions (I.e. if you have (take 3), you’ll only get three elements even with an infinite input stream but they avoid the unpredictability of lazy functions like map because the actual process is run eagerly (unless this doesn’t make sense for the output datatype e.g. a core async channel.).
I’d also be a bit careful about judging popularity from open source codebases too: they are used heavily in all the production code based I maintain :) and I get the sense that they’re pretty widely used by the “silent majority” of clojure programmers.
My debugging strategy for them is basically what I do for threading macros anyways: (into [] (comp ... (map #(doto % (->> (println :it))) ....) inp)
I’d also be a bit careful about judging popularity from open source codebases too: they are used heavily in all the production code based I maintain :) and I get the sense that they’re pretty widely used by the “silent majority” of clojure programmers.
My debugging strategy for them is basically what I do for threading macros anyways: (into [] (comp ... (map #(doto % (->> (println :it))) ....) inp)
Nice feature of doing it like in the post:
(filter (every-pred :admin? :mod?) people)
(let [acl [:admin? :mod?]]
(filter (apply every-pred acl) people))
I admit a case can be made for having every-pred in the language, since from a mathematical standpoint predicate composition is an innate low-level feature (putting the questionable naming issue aside)
> this arguably conses less, by building only one results list
Aren't lazy sequences idiomatic in Clojure?
Aren't lazy sequences idiomatic in Clojure?
This is great and now these functions are now square in my radar, as well.
As far as his example goes, this IS how you model data:
It will forever confound me why some developers/PLs wish to model data in static, inflexible, a priori structures -- given the overtly clear development efficiency and simplicity of NOT doing so (and not HAVING to do so.)
As far as his example goes, this IS how you model data:
(def people [{:name "Elsie"
:admin? true
:mod? true}
{:name "Lin Shiwen"
:nickname "Rocky"
:mod? true}])
It will forever confound me why some developers/PLs wish to model data in static, inflexible, a priori structures -- given the overtly clear development efficiency and simplicity of NOT doing so (and not HAVING to do so.)
I find one's point of view on the matter is largely colored by one's experience.
For example, I personally love static types. In your example I would prefer modelling people as a list of structs, each one with a required "name" (string), an optional "nickname" (string option), and required boolean flags for mod and admin.
Why would I prefer it to be that way? Because a huge number of typos I might introduce in my project are caught before I even try to run something.
I've been bitten once too many times by a typo somewhere (like looking up the "nick" property instead of "nickname") and having the program raise an error after it's already ran for a while. I can't stand that.
For example, I personally love static types. In your example I would prefer modelling people as a list of structs, each one with a required "name" (string), an optional "nickname" (string option), and required boolean flags for mod and admin.
Why would I prefer it to be that way? Because a huge number of typos I might introduce in my project are caught before I even try to run something.
I've been bitten once too many times by a typo somewhere (like looking up the "nick" property instead of "nickname") and having the program raise an error after it's already ran for a while. I can't stand that.
It is probably true that a majority today favors static types, but I don't think this a subjective issue if the problem is maximizing developer throughput over time.
Working on statically typed code bases that model things the way you are describing substantially increases the effort and time it takes to iterate. So while you may prefer them (per your experience or what else), there is still the fact that this way of development is suboptimal with respect to delivery throughput over time.
Of course this is arguable (and is a very heavily argued topic) and the common retort is "to each is own" that isn't sufficient to me.
One might prefer to work in assembly language (per experience or predisposition) but there is still a general statement that can be made that this is intrinsically a suboptimal way to work if delivery throughput is your goal.
I believe that a similar claim can be made to differentiate working in static ADT v. dynamic Map types when modeling business domains.
More food for thought. The typo problems you mentioned are solved by static types but at what cost? And is the CBA there? Especially considering that there are other ways to "solve" the typo problem that do not come with the heavy tax burden of statically typed ADTs.
Working on statically typed code bases that model things the way you are describing substantially increases the effort and time it takes to iterate. So while you may prefer them (per your experience or what else), there is still the fact that this way of development is suboptimal with respect to delivery throughput over time.
Of course this is arguable (and is a very heavily argued topic) and the common retort is "to each is own" that isn't sufficient to me.
One might prefer to work in assembly language (per experience or predisposition) but there is still a general statement that can be made that this is intrinsically a suboptimal way to work if delivery throughput is your goal.
I believe that a similar claim can be made to differentiate working in static ADT v. dynamic Map types when modeling business domains.
More food for thought. The typo problems you mentioned are solved by static types but at what cost? And is the CBA there? Especially considering that there are other ways to "solve" the typo problem that do not come with the heavy tax burden of statically typed ADTs.
In my experience static types increase my development velocity.
I can make changes and find many bugs at compile time without writing tests. I still write tests of course but I have to write fewer, and I rarely waste time fixing trivial things that raise errors at runtime.
It is also much quicker for me to navigate the code. I can find definitions for identifiers instantly and get my mental model refreshed easily every time I switch contexts.
I use dynamic languages too, and I find myself spending more time on things that static languages do for me.
I can make changes and find many bugs at compile time without writing tests. I still write tests of course but I have to write fewer, and I rarely waste time fixing trivial things that raise errors at runtime.
It is also much quicker for me to navigate the code. I can find definitions for identifiers instantly and get my mental model refreshed easily every time I switch contexts.
I use dynamic languages too, and I find myself spending more time on things that static languages do for me.
This can all be covered by more abstractions though via some schema/spec/type validation process. Ide integration is the interesting problem here (and typescript does this beautifully) but I agree horses for courses (and experience). I think the idea for using the keyword to hold the nullability of a field interesting, I thought this was more of a meta attribute in clj? In saying that I actually find this style of keywording way easier to read.
Last time I did a Clojure project, we've constrained the hell out of such maps with Clojure's Spec feature.
Despite spending my last few years working with Common Lisp and a bit of Clojure, I've grown fond again of type constraints. Typing your data prevents a lot of silly and easy-to-make mistakes. Spec in Clojure is doubly useful, because specs can serve both as tests and generators. So after you define how your data is supposed to look like, you can make Clojure start pumping out randomly-generated examples, which you can then use to test your code.
(Note that typing maps with Spec doesn't mean they're inflexible, a priori structures. You just pin down what you know. In this case, you could define that a "person" needs a :name key, and can optionally have :admin?, :mod? and :nickname fields. If it also has some other fields, no harm done.)
Despite spending my last few years working with Common Lisp and a bit of Clojure, I've grown fond again of type constraints. Typing your data prevents a lot of silly and easy-to-make mistakes. Spec in Clojure is doubly useful, because specs can serve both as tests and generators. So after you define how your data is supposed to look like, you can make Clojure start pumping out randomly-generated examples, which you can then use to test your code.
(Note that typing maps with Spec doesn't mean they're inflexible, a priori structures. You just pin down what you know. In this case, you could define that a "person" needs a :name key, and can optionally have :admin?, :mod? and :nickname fields. If it also has some other fields, no harm done.)
Spec can only say what is included in a map, it can't exclude. And the spec is late-bound by the caller. Different callers mandate different specs for the same entity. I see that you know this but I object to the wording "constrained".
Spec is supremely flexible, it isn't hard to enforce a map having a specific set of keys even without explicit support.
(ns user
(:use clojure.set)
(:require [clojure.spec.alpha :as s]))
(s/def ::map-closed #(-> %
keys
set
(difference #{:a :b :c})
empty?))
(s/valid? ::map-closed {:a 1, :c 2})
=> true
(s/valid? ::map-closed {:a 1, :c 2, :e 3})
=> falseSpec alpha 2 supports closed maps:
https://github.com/clojure/spec-alpha2/wiki/Differences-from...
Fair enough. It's entirely unlike constraining in statically-typed languages, but in practice it can still work as constraints if you verify the specs as pre/post conditions of your functions (for which Clojure has some nice syntactic goodies available by default).
The way we've used it, it worked almost as if the maps were statically-typed, and it saved my bacon quite a few times.
The way we've used it, it worked almost as if the maps were statically-typed, and it saved my bacon quite a few times.
clojure.spec + orchestra for nicely integrated pre/post spec checking
https://github.com/jeaye/orchestra
https://github.com/jeaye/orchestra
> Clojure has a bunch of functions with every and some in their names, which can be a bit confusing. In some? and some-> it refers to a value being something other than nil, whereas in some and some-fn it means “the first one that applies”
I haven’t used any of the “some” functions much, but it seems to me you could read this was “the first one that is not nil” which would be consistent.
I haven’t used any of the “some” functions much, but it seems to me you could read this was “the first one that is not nil” which would be consistent.
Manning’s “Clojure. The essential reference”[1] has quite good coverage of these functions.
[1] https://www.manning.com/books/clojure-the-essential-referenc...
[1] https://www.manning.com/books/clojure-the-essential-referenc...
For interested - this blog is a part of Planet Clojure: http://planet.clojure.in/
As a personal name convention, I use '|' (the piping character) as a suffix to name any function returning a function, which includes function combinators. Thus the naming of these functions becomes straighforward: 'every-pred' becomes 'and|' while 'some-fn' turns into 'or|'.
Example:
More: https://gist.github.com/TristeFigure/acd689f3c57e840ebb9f8a6...