Clojure at Netflix (2013) [slides](speakerdeck.com)

Clojure at Netflix (2013) [slides]

https://speakerdeck.com/daveray/clojure-at-netflix

306 comments

Would be interested to hear their experiences with Clojure at Netflix for the 5 years that have followed since then.

Did they keep writing more Clojure? If so, how much of their code is now in Clojure compared to Java? How much more did they rewrite from Java to Clojure? Do they use Clojure rather than Java for new code?

What other languages do they use? Python? Erlang? Rust? How much, in terms of functionality, is written in each language they use? How many lines of code do they have in each language? To what extent did they find existing libraries for the different languages usable for what they are doing?

Among the things that seemed great with Clojure in 2013, did they find that some of these were not so great after all once the codebase grew? Any other problems?

diab0lic8 anni fa

I am very glad you asked!

I wrote and deployed (to production) some Clojure code at Netflix just yesterday. Among other things at Netflix the Mantis Query Language (MQL an SQL for streaming data) which ferries around approximately 2 trillion events every day for operational analysis (SPS alerting, quality of experience metrics, debugging production, etc) is written entirely in Clojure.

This runs in nearly every critical service, ~3000 ASGs and easily > 100k servers and Clojure allows us to also compile it for our NodeJS services as well.

puredanger8 anni fa

Hey, would be great to have this on our success stories page at https://clojure.org/community/success_stories. If you would be able to do that, please contact me at [email protected]. Thanks!

shadowmint8 anni fa

Thats an interesting point that doesn’t answer any of the questions the parent asked. :)

Perhaps you can answer this simpler one:

Netflix has been using clojure for a long time now; has that been a positive experience broadly speaking, that means clojure is still being used for new projects, or not?

Having a large successful project in clojure is lovely, but much of the community’s concern around it is that its hard to maintain, and falling in popularity, broadly speaking.

It would be very nice indeed to see those points addressed by a large scale user of clojure.

davidwparker8 anni fa

> Did they keep writing more Clojure?

> I wrote and deployed (to production) some Clojure code at Netflix just yesterday.

Seems like he answered at least one.

diab0lic8 anni fa

Hey, sorry I saw that and typed a quick response just as I woke up. I'm not usually at a computer so early in the day. I'll address these now that I'm in front of a machine. :)

> Did they keep writing more Clojure?

Yes but it has never been the primary language at Netflix.

> How much more did they rewrite from Java to Clojure?

Very little, if any was rewritten from Java.

> If so, how much of their code is now in Clojure compared to Java?

A very small amount given that it isn't the primary language and Clojure code bases tend to be much smaller than Java.

> Do they use Clojure rather than Java for new code?

This is a personal choice each engineer makes when they write new code. Those who like Clojure might reach for it more often. Clojure is also easy to use within the environment at Netflix since everything was JVM based.

> What other languages do they use? Python? Erlang? Rust?

NodeJS and Javascript, Python, Ruby all have a seat at the table but the majority of back-end code at Netflix is on the JVM, the majority of that is Java.

> Among the things that seemed great with Clojure in 2013, did they find that some of these were not so great after all once the codebase grew? Any other problems?

I've always found larger Clojure code bases to be a bit unwieldy. Fortunately you can usually continue to abstract and keep the size small. If you choose your abstractions carefully you can get a lot of mileage out of this.

I've found the lack of static typing to be a bit of a pain at times especially when refactoring. My safety net for this in the project mentioned in the GP post is to have comprehensive unit tests. If I were to initiate this project today I'd likely explore using Spec to make type assertions.

vram228 anni fa

>Clojure code bases tend to be much smaller than Java.

What are the reasons for this? FP language vs. OOP? Less boilerplate (again maybe due to FP)? Higher-level abstractions in the language or libraries?

I have seen that F# code (another FP language, although I've read F# is more from the ML family via OCaml, vs. Clojure being from the Lisp family) can be significantly shorter than equivalent C# code, for example, as shown in some comparisons on the fsharpforfunandprofit.com site.

Interested to know.

wild_preference8 anni fa

I worked at a shop that used Clojure and Java.

One big difference is that Java APIs tend to require the collaboration of various class instances to get something done, things that you would implement as a single function + options object on your own.

Bouncy Castle is a good example. You may need a Hasher, HasherStrategy, ASNEncoder, DERParameters, and ASNSerializerStrategy instances to execute what you would've implement as `(asn-encode thing)` otherwise, maybe even having to subclass some of them to change some behavior you'd expect an option flag for.

Clojure's own Java-helper macros will compact your 1:1 Java interop code as well, so you have fewer lines even when writing Java from Clojure. Also, short-cuts like ad-hoc reification in Clojure will spare you LoC where you might otherwise have created a whole file for a class with custom interface implementation in Java.

Of course, line-to-line code is also just more compact in Clojure, but ecosystem/api difference is one I don't see mentioned as often.

I don't think this is just good vs bad, though. There are certainly upsides to the more rigid everything-in-its-right-place code you tend to have in Java which has been making big strides in improving itself over the past decade.

vram228 anni fa

Interesting, thanks.

I agree with your last point.

Barrin928 anni fa

one thing for sure is the higher level abstractions you tend to use in functional languages, but a big thing about clojure is that it's very flat and generic in terms of datastructures.

You basically have maps, lists, vectors, functions and not much in terms of hierarchy. A lot of code in Object Oriented languages simply exists to manage the hierarchies and structures you build and that's something that clojure largely avoids.

There's also of course the macro capabilities of lisp that can save you a lot of boilerplate if utilized correctly.

vram228 anni fa

Thanks.

>You basically have maps, lists, vectors, functions and not much in terms of hierarchy. A lot of code in Object Oriented languages simply exists to manage the hierarchies and structures you build and that's something that clojure largely avoids.

It's similar for Python's built-in data structures: tuples, lists, dicts, sets, frozensets, with their built-in features and methods, including slicing for lists and strings (even without using, say, the collections module of the stdlib). (And of course including building up nested structures from the same.) Had read early on in my use of Python and later experienced for myself, a good amount, when doing work with it, that those structures are fairly powerful and for many apps, you do not even need OOP structures and hierarchies.

List, dict and set comprehensions are great for that, too.

Although Clojure may have some additional ones that Python does not, not sure, since I haven't used it.

diab0lic8 anni fa

The two sibling replies to my own do a good job of enumerating the reasons;

- Macros allow you to greatly reduce boilerplate. - Lots of built in functions that operate on very few types. - Compact intertop

And I agree with wild_preference that this isn't necessarily a good vs. bad debate. It is just a matter of fact that Clojure is concise.

Thaxll8 anni fa

Apparently Go as well:

https://www.reddit.com/r/golang/comments/765izv/what_cool_pr...

5M req/sec is pretty cool.

cutler8 anni fa

What is stopping you adding Spec to your existing code?

diab0lic8 anni fa

The thing I would be getting from spec is confidence when modifying the code base, which I get from the unit tests I wrote pre-spec. If I were to initiate the project today I'd probably have less tests and more spec.

bm13628 anni fa

I’ve found that overly using spec leads to more maintenance than upside. We went all in using spec and generators when they were released and ended up having to debug the specs themselves. 2c

jwhitlark8 anni fa

I think of spec like seasoning; if you put lots on everything, you ruin the dish.

brogrammer58 anni fa

What's the condescending tone for?

diab0lic8 anni fa

And now that I've responded to the GP, to address your points:

> that means clojure is still being used for new projects, or not?

This has been / will always be a professional choice of the engineer(s) starting a new project at Netflix. Clojure is great for a lot of reasons and lets you target JVM/NodeJS at the same time (our two largest backend languages) but as a LISP most people aren't going to be excited about using it.

> Having a large successful project in clojure is lovely, but much of the community’s concern around it is that its hard to maintain, and falling in popularity, broadly speaking.

In my personal experience maintenance has been a breeze. I had a meeting at noon yesterday where a data scientist wanted a new feature in the query language and we had it shipping to production by 3:00pm. If the code base were say 10x as large I'm not sure I'd have the same opinions about ease of maintenance but I haven't leveraged spec and I've been able to continually increase abstraction to keep the code size small as more features came in.

As for falling in popularity that is my perception as well though its current level of popularity still seems sustainable. I'd imagine this has to do with it being a LISP, with STM not being as popular as anticipated and with spec taking longer than anticipated.

kbp8 anni fa

> but as a LISP most people aren't going to be excited about using it.

I agree that Clojure has a lot of strong points and that s-expressions probably put a lot of people off, but as a Lisp programmer, I was very disappointed in Clojure's debugging/interactive development story (and I've heard that from a lot of others). It feels more like using a typical scripting language compared to the traditional Lisp/Smalltalk experience, and even there, a typical scripting language would at least give useful backtraces. As it stands, I think a decent number of conventional Lisp programmers would also worry about large Clojure programs being unmaintainable unless they're superbly written.

flavio818 anni fa

> I was very disappointed in Clojure's debugging/interactive development story (and I've heard that from a lot of others). It feels more like using a typical scripting language compared to the traditional Lisp/Smalltalk experience

Common Lisp user here. I was also disappointed in the same way when trying Clojure.

Other minor things i didn't like was the noisy [] on the syntax, and the fact that for practical purposes you're fully tied to the JVM and the java runtime libs.

kamaal8 anni fa

>>for practical purposes you're fully tied to the JVM and the java runtime libs.

That's more like a positive thing about Clojure. Java inter-op and targeting the JVM gives Clojure a great chance of adoption at large enterprises.

These days no one really has a issue installing jars on a production machine.

gleenn8 anni fa

The latest Clojure in the pipe, 1.10, is supposed to have a lot of work done fixing stacktraces FWIW.

diab0lic8 anni fa

I'm very happy to be reading this!

fulafel8 anni fa

It's a big exaggeration to say Clojure stack traces aren't useful. They have some extra noise but they do the job of pointing out the call chain and the exception value. Tooling (CIDER at least) can automatically hide the frames about the Java runtime and highlight the Clojure info & line numbers.

diab0lic8 anni fa

Oh, yes this is another excellent example of pain while building Clojure code bases. Though I'll say I only agree with half of your statement. I've found the interactive development story to be great with nrepl/fireplace/vim but the debugging is downright terrible... this is the single biggest blocker to me using it for larger systems.

michaelmrose8 anni fa

Debugging seems acceptable on emacs with cider. Sayid seems interesting as well.

fiddlerwoaroof8 anni fa

Yeah, cider has an amazing form by form debugger for Clojure. And the way cider uses overlays to surface this is quite nice too.

My general experience of cider is that for a certain set of tasks it is much better than slime. But, the problem is, most of my day-to-day coding tasks are hampered by the language: e.g. if I have a web server running and I want to change a request handler, you can’t just recompile the handler, you also have to restart the server.

fiddlerwoaroof8 anni fa

(Although, if you anticipate this, you can call the ref (e.g. `(#’foo arg)`) rather than the function `(foo arg)`. But this means that you have to plan the dynamically modifiable parts out ahead of time.

unlogic8 anni fa

That's not true. You only have to prepend #' if you pass the function by value (as in your web server handler example).

If you call the function by name yourself, like (foo arg), and you recompile foo, any code that called foo will see the new version.

fiddlerwoaroof8 anni fa

Hmm, I’ll try it again, but I remember issues with the function call syntax too. Iirc, it had to do with the fact that redefinitions don’t count if the use site is running in another thread. But I’ll double-check this when I get a chance.

blain_the_train8 anni fa

> Having a large successful project in clojure is lovely, but much of the community’s concern around it is that its hard to maintain, and falling in popularity, broadly speaking.

What community? The larger software community or the clojure community. I personally see no logical reason why a "large" clojure project would be harder to maintain then say a java, python, ruby, javascript, etc.. project. If anything, the guiding principles that make Clojure a well designed language at the micro level should have an exponential effect the same way poor decisions do.

alextheparrot8 anni fa

Do you know offhand how MQL compares to the Apache Calcite [0] extension of SQL that some big data platforms are using for streaming SQL?

[0] https://calcite.apache.org/docs/stream.html

diab0lic8 anni fa

Hey Alex,

I explored potentially using Calcite when we initiated this project. The syntax is very similar because both are an SQL dialect. Some differences are that MQL largely targets unstructured data (there is no schema for the streams it operates on - they're just streams of JSON blobs).

In addition to that one of the goals of MQL was to have different compiler backends which allows different call sites to operate on different levels. For example the code that runs in our API/proxy/other large services will look at an entire query and only evaluate the WHERE / SAMPLE clauses expecting something down the stream to complete the query. Conversely the client side is a full SQL -> RxJava implementation for the data stream. I can explain more of how this works if there is interest -- it allows us to only egress data that is actively used in queries so devs can log every request and only pay the cost of those for which they are querying.)

Doing our own implementation also allows us more customization, and to compile to a NodeJS backend as well which is a critical ingestion point for our operational data. We support everything on the linked page except hopping windows, subqueries, and DML (MQL is query only, we assume no structure which was true of the streams before we ever wrote a query language for it). Of course we have to implement a lot more of this ourselves which is pretty in line with Netflix's Freedom and Responsibility. We had the freedom to implement our own query language but have the responsibility to maintain it (Calcite would have been sacrificing some freedom to avoid responsibility.)

alextheparrot8 anni fa

Thanks for the detailed answer - as you likely already know many projects have taken the approach you detailed in your past paragraph, it makes sense if your goal is more freedom for sure.

I am wondering how you do client side data egress filtering - does each event need to get materialized, assessed for certain fields or structure, and then sent once for each outgoing stream on your sender? Seems like a good strategy for reducing network bandwidth, but it might reduce your throughout moderately (Due to serialization and analysis costs) or cause hotspots (If you have a distributed stream that is partioned in a particular way)? These are normally problems each consumer individually faces, where now it compounds where the producer does that work for each of the N readers. I appreciate the idea a lot, just wondering if you’ve had any issues making it scale nicely for highly subscribed streams.

diab0lic8 anni fa

The events are already materialized in memory as part of whatever is recording them, thankfully.

The MQL that runs in the services checks the WHERE clause of every query running and then projects a superset of all the fields necessary for every part of the query for all matching queries (including other parts that the server won't be processing: group by, order by, having, etc...). It then tags the event with all of the matching queries. This way we only need to egress a single event.

What happens next requires a lot more context about Mantis, which is a reactive stream processor. The data is egressed from the service to what is referred to as a source job in Mantis... it is the source job's responsibility to multiplex this data to any of the consuming jobs. Jobs can be subscribed to one another so this is a natural fit for Mantis.

As for scaling everything is round robin after it leaves the service and enters Mantis but obviously busier services will pay more cost because they're processing more events. Regular expressions in WHERE clauses has been a pain point for scale, and absurdly heavy queries such as `SELECT * FROM STREAM` result in data dropping as most single machine consumers can't keep up with the large streams. We have different methods of providing different delivery semantics but in the base case we're at most once and will drop data if the client can't keep up.

We have some streams that exceed 1 million RPS at certain times of the day, and last we checked the system moves around 2 trillion events per day so we've managed to scale it to the needs of the company pretty well. It has been a while since I performance tested it, but it's always been a goal to have it push as close as possible to saturating the gigabit connection on the boxes on which it runs.

diab0lic8 anni fa

I came back to add: One of the things that just popped into my mind that Calcite has that we don't is query planning. Obviously since MQL was developed in house we didn't get any optimizations out of the box. We've since added a layer that processes the parse trees looking for optimizations and while it isn't quite query planning it has yielded some significant performance gains (or rather poor performance mitigations, haha).

I'd imagine getting this for free from Calcite would have been really nice.

saosebastiao8 anni fa

I noticed a lot of clojure codebases at amazon in 2013 (I was looking because I was learning clojure at the time). It took a look about three years later and most of them were either ported to java or scala, or dead. There were a few still running, but it looked like they were not actively developed or maintained. In my experience at amazon that likely meant they were zombie services (running but nobody used them and nobody thought to take them down), which is surprisingly prevalent at amazon. But I guess it could also mean that they were running and were so amazingly stable that they didn't need maintenance, which would be a good thing.

CyberSun8 anni fa

Our team has been using it pretty consistently for the last three years. There's a few other teams I know of that uses it, but I wouldn't say it's common.

We rewrote more than half our systems from Java to it. We've kept the Java code that was already in good shape, moved the rest to Clojure.

Most of the new systems we've built in the last three years were done in Clojure. Though it's up to the engineer in charge of the project, eventually most everyone on the team has embraced Clojure, and willfully chooses it as their JVM language of choice when starting a new project.

Maintenance wise it has been way better then Java. Even with our team being around 10 devs, which is pretty big. We've had no issues extending our systems, adding features or fixing bugs. We have more stable systems actually, and our tickets count has drastically gone down. Though it's a little unfair, in that some of those systems are rewrites from Java to Clojure, thus our second attempt, and so we might have learned some things from that also.

Most everyone eventually seems to come around to loving it. Like once they are familiar with it enough. At least with regards to Java they like it better, and some who've had Scala experience also seem to prefer it.

All in all, I don't think anyone on the team has any regrets in choosing Clojure, and are actually quite pleased. We're planning to continue to use it, and hopefully have the whole code base be in Clojure.

The weird thing is, you always feel like you wish you had types to help a little with the data flow, and knowing what part to update if you ever change the data model. As well as miss a bit of the auto refactoring tools java provided. But, when we retrospect, it's never really because it's causing us issues like the code base being unmanageable, or refactoring taking us longer to do. It is much more of a feeling, than an actual practical issue. Akin to how driving without your seat-bell on gives you that constant unease of not having it there just in case. That said, we love not having types everywhere adding noise to the code, and the ease of interactivity it offers from the REPL, and being able to focus more on our data model in terms of data (as opposed to types), and focus on our code's behavior (as opposed to types). So its a trade off, that for the last three years, hasn't caused us any issues.

jatins8 anni fa

> running but nobody used them and nobody thought to take them down

Experienced the same, and these services just kept hoping through the teams.

And why is it so hard in Amazon to know if someone is using a service? At one point our manager asked us to shut down a service to see if someone was using it.

CyberSun8 anni fa

Don't know how long it has been that you've left, but this is no longer an issue. There's a discovery mechanism now that keeps track of all service to service interactions.

Also, that's a strange suggestion, couldn't you have just looked at request count metrics?

jhall14688 anni fa

> And why is it so hard in Amazon to know if someone is using a service? At one point our manager asked us to shut down a service to see if someone was using it.

Don't collect any usage statistics works. In our org we're switching a lot of smaller services to lambda and using fewer and fewer EC2 instances to do our work. But then, we actually track usage on every service so there's that.

Scarbutt8 anni fa

Did they keep writing more Clojure?

Since Netflix likes to constantly write about their tech (videos, blog), my guess would be no, but interested in an official answer too.

They are now heavy users of nodejs though.

I personally ran out of reasons to prefer another dynamically-typed language over Javascript on the server (not a hard rule of course).

Scarbutt8 anni fa

To clarify, the reason I prefer nodejs (for web apps) over clojure these days is the vast amount of libs and documentation for almost all your needs. Clojure has the java ecosystem, but it was very tedious (more cognitive load) investing lots of time doing interop instead of just focusing on the problem at hand. Interop is both a feature and a curse.

Of course, as a language I find Clojure superior in almost every way compared to JS.

CyberSun8 anni fa

> Clojure has the java ecosystem, but it was very tedious (more cognitive load) investing lots of time doing interop instead of just focusing on the problem at hand.

I suspect you have no JVM/Java experience? Because I do, and I don't in NodeJS, and I find the reverse to be true in my case.

paulie_a8 anni fa

Honestly the vast amount of libs is the reason I don't like nodejs. It's a lot of shoddy duplicates.

[deleted]8 anni fa

yogthos8 anni fa

You can use ClojureScript on top of Node. My team started using Node for some services, and I published Macchiato based on that https://macchiato-framework.github.io/

cutler8 anni fa

I like the idea behind Macchiato but unlike it's Clojure cousin, Luminus, there doesn't seem to be any documentation on database access. I especially like HugSQL in Luminus. Is it possible to use it with Macchiato?

yogthos8 anni fa

There is macchiato-sql [1] which is modeled on HugSQL. I've used it for a couple of projects, but YMMV. I tend to use sequelizejs since it works with most databases and it's well documented.

[1] https://github.com/macchiato-framework/macchiato-sql

cutler8 anni fa

Bejesus, Sequelize.js is a LOT more work than HugSQL. I keep looking at Macchiato but without a Clojurescript database layer I can't see it being useful.

yogthos8 anni fa

I mean if you're on Node, it's not going to be any better with plain Js either. Stuff like this is precisely why I prefer the JVM ecosystem myself.

abritinthebay8 anni fa

We’re heavy users of LOTS of languages at Netflix to be fair. Clojure included.

lvh8 anni fa

Latacora has its token Clojure person: me. I write Clojure every occasion I get, we've shipped Clojure code to customers, et cetera et cetera.

Most of the stuff I'm doing in Clojure would be possible in other languages, but in a sense that's only one notch below "Turing tarpit" possible. So, in the only sense that a startup should care (would you f'ing ship it?): only possible in Clojure or a handful of close cousins.

I'm hearing less about it on HN these days? But it was a great language yesterday, remains one today, and there's constant development of tooling and community :)

jwhitlark8 anni fa

I think some of that is due to the nature of the people it attracts, namely, grumpy, older, I've been burned before, programmers. I use Clojure constantly and have for 10 years, but I've never felt a strong need to proselytize. I just make money and enjoy coding, day in and day out.

m12k8 anni fa

I'd like to learn a modern production-ready functional language (I have some academic experience with SML), since they seem like a good way to grow as a programmer. Main contenders so far are Haskell, Scala and Clojure (Reason might go on the list soon too) - but the fact that Clojure is dynamically typed is a bit of a turn off for me. My experience with dynamic vs static typing is that as a system grows in size and complexity, the value of static typing goes up. Doing a major refactor is one of those things that I find especially error-prone in dynamic languages, while statically typed languages make this much less daunting (of course they come with the opposite issue that you might not be able to compile for a day until you've fixed all the type errors - but I'd take this trade-off any day of the week). Can anyone with experience comment whether Clojure suffers from this too or if it somehow handles the lack of static typing better than other dynamic languages?

jwr8 anni fa

Background: I have built and now maintain/extend a fairly complex application in Clojure/ClojureScript with 120k lines of Clojure code (for non-Clojure people: that is quite a bit, because of Clojure being so expressive).

I think the whole "lack of static typing" thing is a red herring, for two reasons:

1. Large-scale code is more about contracts that specifically "static typing". There are many contracts which you can't express using a type system.

2. Clojure doesn't "lack" anything. You can implement contracts for functions using pre/post conditions, use clojure.spec to implement arbitrary contracts over your data (and I do mean arbitrary), or even go for core.typed if that is your thing. I started to rely on spec more and more and I believe that coupled with pre/post conditions, it is way more useful for larger systems than static typing.

One other factor which is often forgotten in these discussions, because so few languages get there: please remember when "choosing" a language, that Clojure can be paired with ClojureScript on the frontend side. That means you can use the same data structures, same transport format (no JSON or XML quirks, ship EDN over websockets using transit and sente) and much of the same data model code on both sides. This is a huge advantage if you can make use of it. And guess what, your spec definitions work just as well on the client side, too.

KingMob8 anni fa

I use Clojure in the large, and the lack of static typing isn't something I miss. FWIW, spec allows you make similar guarantees if you use it (technically more, since you can express more than HM type systems).

I think the static/dynamic divide speaks to deep divisions in programmer personalities, but if you're open to suggestion, I would put it like this: As Haskell/OCaML are to static langs like Java, Clojure is to dynamic langs like Python/Javascript.

Clojure has a deep vein of pragmatic simplicity throughout its core libraries and community that I think does more for taming large-scale development than static typing does, but that just my $.02.

jwhitlark8 anni fa

In my experience, static typing was somewhat helpful in monolithic projects, but when I saw it used in heterogeneous distributed systems, it seemed to lose much of its benefit and impose new costs.

I'm curious if we're going to see a big new synergy between Kubernetes (represent everything as data) and Clojure (great with data).

tanin8 anni fa

Is Spec simply a test? Or is it something more? I keep hearing about it, but it seems to be more than just a test case.

Let's take a simple example. I have an object with the method named "get". But I call "fetch" in my code. When will I see this error? During compile time or run time?

lgessler8 anni fa

Think of it this way: spec gives you an easy way to ensure that your data conforms to arbitrary predicate functions.

Suppose you have an XML format, <library> that has books and authors[1]. The elements look something like this: <book author-id="0000"/>, <author id="0000" name="Plath"/>. Obviously, you want to make sure that book's author-id attribute will always refer to an author that actually exists. This is something a (Java-style) static type system can't do: it doesn't know at compile-time what the contents of a variable will be. But spec can do this because it is a runtime check[2]: you'd just write a function that ensures all author-ids refer to extant authors and register it with spec, telling it that this must be true for valid <library>s.

As for your example, I don't think that is a use-case that spec was intended to handle. You could write a spec to ensure that an object has certain properties/methods, but I'm not spec would be too useful for a function invocation on that object.

[1]: For a worked out example of this in spec, see a blog post I wrote: https://lgessler.com/posts/2018-07-12-choosing-the-right-too...

[2]: It sounds expensive, but there's a compiler flag that lets you turn off all spec checks, so you can have them only run in dev builds if you like.

matt-noonan8 anni fa

That blog post was a nice read, thanks!

I realize this is just one example, but you might be interested to know that I've done some work making this kind of invariant enforceable via static types (though in practice, you need a combination of features that aren't found in many mainstream language besides Haskell afaik).

See the README here for motivation and some examples (in Haskell, but hopefully the idea is still clear): https://github.com/matt-noonan/justified-containers

Or the tutorial module here: https://hackage.haskell.org/package/justified-containers-0.3...

Or this paper, if you really want to go off the deep end: http://kataskeue.com/gdp.pdf

CMCDragonkai8 anni fa

I found your explanation of the relationship between an existentially quantified type and rank 2 polymorphism to be quite illuminating. Do you have any ideas why Haskell didnt have the direct way to express existential quantification in the type signature? Is it a technical limitation or something else?

matt-noonan8 anni fa

I'm not sure about the history. It looks like UHC supported existential types at some point, but I'm not sure how that would interact with what I am presuming is a more sophisticated system in GHC Haskell.

This post had some interesting tidbits about technical challenges, but I didn't fully grasp everything on first read: http://blog.ezyang.com/2016/04/hindley-milner-with-top-level...

I really like tome's observation in the other comment: universally-quantified types have a direct encoding in System F as type abstractions, but there is no such direct encoding for existentially-quantified types.

tome8 anni fa

I don't have a good answer but I did write something about this once http://h2.jaguarpaw.co.uk/posts/exists/

vram228 anni fa

>Suppose you have an XML format, <library> that has books and authors[1]. The elements look something like this: <book author-id="0000"/>, <author id="0000" name="Plath"/>. Obviously, you want to make sure that book's author-id attribute will always refer to an author that actually exists. This is something a (Java-style) static type system can't do: it doesn't know at compile-time what the contents of a variable will be. But spec can do this because it is a runtime check[2]: you'd just write a function that ensures all author-ids refer to extant authors and register it with spec, telling it that this must be true for valid <library>s.

So is it similar to or different from assertions?

tanin8 anni fa

Thank you. I think I have a good idea what it is now. It's a constraint of a field. But it goes a bit further to ensures that, for example, an author id refers to a valid entity.

fazzone8 anni fa

In Clojure unresolved names are a compile failure, and Spec would not be involved. Spec is just a nice library for writing runtime assertions about data. An example would be calling a third-party API and then validating the response with Spec to make sure it matches your expectations, or even validating the responses you are about to return from your own API. Similar to a type system the Spec can also serve as a sort of documentation about what's in the data structure.

CyberSun8 anni fa

You'd see it at run-time, but most likely at test-time. Which is not a "time" we often think about.

Spec is designed so that you setup automated generative tests on your specced functions. These perform brute force search of the input space.

So, they won't give you FOR ALL guarantees, but will still catch quite a lot. For functions with small input domains, it would actually prove FOR ALL.

The trade off is that, you can test for much more. You can test for properties of the values, not just type. Like say making sure that the output is always smaller than the input. Or that the input never is a blank or empty string.

A downside, it doesn't work well for unpure functions. Since the generative brute force doesn't have a way to brute force the side effect, or assert properties about it. For those, you'd need to write your own tests.

All in all, don't expect it to be at all like a static type checker. It's a very different beast, which you'll want to use very differently, and which offers a very different value proposition. For example, you could want to use Spec even if you had a static type system. Just like people still write tests. Spec is a new kind of tool that can be leveraged to mitigate software defects.

ahjones8 anni fa

This is spec: https://clojure.org/about/spec

All spec errors turn up at run time. See the guide for more details: https://clojure.org/guides/spec.

You can combine spec with test.check (Clojure's quick check library) https://clojure.org/guides/spec#_generators

fulafel8 anni fa

It's a way to describe the shape of data, a bit like schema but focus on non-exhaustive description and composability. The practical uses can be generative tests, is-a queries, conformity tests, "describe deviations from spec" queries, etc. You can also write spec for diffs in data structures, so you can use it for state changes. It's pretty open-ended.

jwhitlark8 anni fa

At its core, spec lets you ask "Do I have what I need to do my job?", and if you have more, spec doesn't care. Type systems answer the question "Do I have exactly the machine primitives I expected, no less and no more".

It's about inspecting and verifying minimums at runtime.

jrochkind18 anni fa

I believe you (I'm a ruby programmer, so you know I'm not strict about typing), but can you clarify -- `spec` is runtime, not statically typed, true?

KingMob8 anni fa

Yes, spec is a run-time tool. It's capable of multiple things, but it's mostly used at dev-time to validate fn inputs and auto-generate spec-valid data for testing. It's usually disabled on production except at the edges of the system, where it frequently validates external input (e.g., making sure a JSON input conforms to an expected spec).

Clojure's Schema is similar to spec, if a little different. There's also Typed Clojure, which allows you to introduce gradual typing to a Clojure code base and get compile-time static typing (though I believe it's still alpha/beta quality).

bgorman8 anni fa

I used to be a professional clojure developer, now I write in ReasonML (syntax change of Ocaml) for work.

I would say that that the reason type errors do not hurt Clojure as much as say Javascript is because Clojure data is immutable, and all the common functions operate on collections or sequences. Basically, if you write idiomatic code, you constantly use datatypes that implement the same interface which higher order functions (map, filter, reduce, etc) expect. In contrast, in OCaml for example you have to think whether you are dealing with an Array or a List. Clojure also provides mechanisms to enforce that functions (Macros too with clojure.spec) are applied the correct data parameters (clojure.spec + pre/post conditions). Spec shines when you are integrating with "foreign" data like the results of a HTTP request.

In my experience, it is faster to make changes to a project, then test quickly with the clojure repl than it is to spend a lot of time refactoring types. This is especially true when you are experimenting with how to process data a different way... Having to have all types work out at compile time is a real time sink, when mentally you already know how you want to process data.

blandflakes8 anni fa

There are a lot of people who claim they're more productive in Clojure, in spite of or because of its dynamic typing.

I'll just throw my anecdatum out here: I found that dynamic typing was a major pain even on my own personal projects. I also never found REPL-driven development to mesh well with my workflow.

In Scala, for example, entire classes of error that I just shouldn't be able to make don't exist. Maybe I'm just the kind of person who works better with static typing.

Clojure overall is an excellent language with really good features. It just won't get out of my way sometimes.

xapata8 anni fa

Try to grok REPL-driven development before dynamic typing. You won't like the latter until you love the former.

jerf8 anni fa

Haskell does REPL-driven development fairly well. It's not the best REPL, but it works, especially if you take a moment to learn a few of the commands to drive it. Dynamic types are not necessary for REPL-driven development.

yogthos8 anni fa

Haskell REPL is a completely different animal from the one in Clojure, and it's a stretch to say that you can do REPL driven development in Haskell.

What people mean by REPL driven development in context of Clojure is that the REPL is running within the context of the application you're developing. The editor is connected to the REPL, and you can modify any part of the application at runtime. You can even connect the editor to a REPL on a remote machine such as a production server and inspect its state from your editor.

CMCDragonkai8 anni fa

I do all my Haskell development in a repl driven way. It's even better than Python's repl due to the way it loads modules and allows reloading quickly and automatically checks the type signatures.

Having the entire application loaded into your repl is even more powerful. I did it once in PHP using psych and was able to make use of any controller and try parts of it out. The main thing that is necessary is the ability of the REPL to either drive change to the code, or react to changes on file (the former is temporal, the latter is more permanent) and perform the necessary incremental computation to become eventually consistent.

xapata8 anni fa

Yes, that's why becoming happy with REPL-driven development is more important than becoming happy with dynamic typing. I think much of the productivity boost attributed to dynamic typing is in fact caused by heavy use of a REPL, and not dynamic typing.

blandflakes8 anni fa

I'd maybe consider that perhaps becoming happy with REPL-driven development isn't a universal solution. It's not like I didn't understand the concepts, the tools, or the process. I just didn't find that it made me any more proficient at building software.

xapata8 anni fa

Of course that's possible, but the evidence suggests that most people become more productive when using the REPL to experiment while developing large projects.

One issue with trying to adopt the habit midway through a project is having a design that makes it easy to load small portions of the project with test data into the REPL. It's not quite the same as having code that's easy to test.

Do you use an interactive debugger? It's the same principle, but earlier in the lifecycle of development.

blandflakes8 anni fa

I appreciate your earnest approach, but I also feel somewhat patronized. I've been doing this for a while. I used Clojure extensively for years, was productive and found much to like. The REPL wasn't part of it, and I don't miss it. I still use Clojure on occasion.

This response is pretty common: "You probably weren't doing it right. Trust me, this is the way!" While I agree that suspending beliefs/old practices and trying new things is important when using new technology, I'm also pretty confident that I gave Clojure a more than fair shake, and just find that I don't get much out of the technology surrounding the REPL in that environment.

I was more productive in Scala in days than I was after years of using Clojure, and the difference between what I'm saying and your response is that I'm not claiming my experience is universal.

xapata8 anni fa

No condescension meant. It's hard to estimate how experienced someone is from a few sentences in the forum. I'm generally careful to make assertions about most and in general rather than all or universally.

blandflakes8 anni fa

None taken personally - I just find it valuable when I read comments against the grain, and had an experience that I know many Clojurists find atypical and wanted to share that my style of work just doesn't lend itself to Clojure as supported today.

bribri8 anni fa

Building up complex types in the REPL is hard compared to using a map...

ken8 anni fa

It sounds like Clojure was too much "out of your way", and you prefer a language like Scala that gets in your face about possible issues sooner.

blandflakes8 anni fa

I can get on board with this phrasing.

Rapzid8 anni fa

Never did much clojure outside Riemann "configs" but I found the experience of light table's insta-repl fantastic. Scratch pad type workflows are a fav of mine(linqpad is amazing) and insta-repl blew me away.

keymone8 anni fa

> I also never found REPL-driven development to mesh well with my workflow.

> In Scala, for example, entire classes of error that I just shouldn't be able to make don't exist.

guess what programming practice allows to almost entirely avoid those classes of error?

CyberSun8 anni fa

Have been using Clojure for three years on a dev team of around 10 people. We haven't had any issue with maintenance and extensibility of the code base over time. I'd say it is as easy to maintain and extend as it was in the beginning.

That said, we have a micro-service architecture. Thus the overall system grows horizontally, in that more and more components are built and developed. So individual components don't really grow that large. This would be different say to building a giant monolith like a AAA game, or a massive application like Photoshop. So I can't speak to how such a code base would scale.

Also, this might sound strange, but I don't think we've ever had to refactor the code base in those three years. The paradigms of Clojure (data driven/functional/immutable/meta), combined with our architecture, it just doesn't really need you to refactor things to add features.

Most adding of features tend to be... Write functions for it. Plug the functions inside the outer orchestration chains of function composition which dictates code flow. Add some more keys to our data-structures. Write tests. Release. Where as in Java, we used to have to shuffle all the Class/Interface arrangements around all the time to make room for new features.

saosebastiao8 anni fa

I've used all three, and at least in my opinion, Scala is the most production ready by miles.

Clojure is great if you like dynamic typing...by far the best dynamically typed functional language out there. Beware of claims that clojurescript and clojure are the same language. While both are dynamically typed, clojure's type discipline is strongly enforced (type errors will throw exceptions) while clojurescript will fail silently. I personally know of a team that experienced a million dollar bug because of this subtle distinction...you absolutely have to be prepared for it if you plan on using both. If you like gradual typing, there are some great options.

Haskell is good but the language's strictness (especially around effects) is very demanding. This can be both a good thing and a bad thing. In my experience/opinion, it's a good thing in some use cases and a bad thing in other use cases...but there's no way to opt out of it when it's a bad thing. Further, the ecosystem is far less useful than the jvm ecosystem.

Scala definitely has its faults, but its type system is extremely expressive without being extremely restrictive. I've never felt restricted by it (like I did with Haskell), but it regularly blows me away how easy it is to refactor and fix errors. The Java interop is far more intuitive than Clojure. Drawbacks: The only really useable IDE is Intellij. Some libraries tend to form an ecosystem of dependencies which don't play well with other ecosystems (ie some libraries are "Scalaz-only", some are "Cats-only", etc), which can be really annoying at times. Java interop isn't extremely straightforward in the case of collections (if that is a big concern, go with Kotlin which is phenomenal).

didibus8 anni fa

ClojureScript is strongly typed. I'm curious to know the specific case they ran into? I suspect it might have been interop with JavaScript, because you shouldn't have weak types in ClojureScript.

darksaints8 anni fa

No, it's not. Try `(+ 1 "1")` in clojure and clojurescript. In clojure, it produces a runtime exception. In clojurescript it silently errs, resulting in a "11".

The compiler correctly infers and warns of the problem, but that is of no use if your inputs change types on you. In the case of the service I'm referring to, an external service started serving up json strings for large numbers, whereas it had previously been json numbers. Since there was no exception produced, it silently corrupted data for a couple weeks before it was caught.

didibus8 anni fa

I asked about this here: https://clojureverse.org/t/clojurescript-is-weakly-typed/311...

And documented to what extent types are automatically coerced.

Also got the rational behind this: performance.

And that every new release of the compiler performs smarter and smarter type inference and can warn about more of these cases. Though it still misses some as of now.

ajss8 anni fa

People were running a real money service that consumed JSON from another service and it didn't validate the incoming JSON? Wow.

saosebastiao8 anni fa

The problem comes from the fact that clojure and clojurescript are two different type disciplines, but people approach them with the same expectations. That exact problem would have errored out if they were running clojure, and a thrown exception is good enough validation for many developers. But because they were duped into a false safety net by thinking clojure == clojurescript, it slipped right underneath their radar.

didibus8 anni fa

Woa, you're right. I had never realized they were compiler warnings and not JS errors. Thanks for the tip.

justinhj8 anni fa

Scala is sometimes referred to as the Haskellator, in that once you get far enough down the pure functional programming road, you may find yourself wanting to switch to Haskell. Scala doesn't enforce pure functional programming as much as Haskell. On the other hand I believe that Scala is much more widespread in industry because it's easy to sneak it in the door as an OOP/imperative language that just happens to have good support for higher kinded types, immutable data and other fp features. My advice, if it's for your own personal learning and possibly for some small in-house projects learn Haskell. If you want to use it for a large project with developers that are interested in learning fp but not yet fully committed learn Scala.

Also, yes, any dynamic language suffers from difficulties with refactoring, runtime errors, needing lots of test coverage and not scaling well to large teams and codebases. IMHO

jwhitlark8 anni fa

After my time with Scala, I think that if I need a heavy type system, I'd look to Haskell. And there are certainly classes of problems that I would choose a type system, but the general case isn't nearly as strong as I though it was before seeing them used in anger.

So, in the following, I'm not ranting at you, but I am saying I see lots of arguments for types that jump to QED without nearly enough evidence, with the "everybody knows" kinda explanation. And you happen to have put a handy list right in front of me when I had time to respond.

> ... difficulties with refactoring, runtime errors, needing lots of test coverage and not scaling well to large teams and codebases.

I was really impressed with Scala when learning or toy projects, but every time I saw it in production it was pretty horrid. Small, technical refactoring was taken care of by the IDE, but large scale refactoring takes the same amount of thought as before, but now there's 10x as much code to understand first.

There were less runtime errors, but not much less, and I saw a lot of code that didn't error, but didn't get done what it needed to.

When it comes to scaling to large teams, I saw lots of "The compiler says it's ok, and the tests pass, so commit!" where the code didn't make sense! The old thing about write code for programmers to understand first, and the machine to understand second...

Outside of certain specific situations, I just haven't seen the benefits outweigh the increased code weight and slower time to market.

justinhj8 anni fa

Just like your response, my evidence is the anecdata of my own experience. I’ve built a large system that 9 engineers use, all of different experience levels. Functional programming gives more understandability not less, you can always understand the input and our output of a function from the types. I really can’t see how anyone can argue about types and refactoring ... even without any tools you can just change the function signature and get everything compiling again. You know you didn’t miss any callers. Even better one knows when you can change a data structure or effect type from one to another because the compiler will tell you. If you use scala correctly you can eliminate whole classes of runtime errors and i’ve experienced that too both in a good way and a bad way when we let nulls creep in.

Whats even better is higher kinded types let you assert semantics to engineers that are hard to do in Clojure.. this type must be appendable or foldable, this type must handle async, this type maybe missing and this list cannot be empty.

All these things are a joy not a burden once they start working for you

bribri8 anni fa

This is the number one opposition to Clojure I hear. I was strongly in the static typing camp and rejected Clojure at first, even compared to Java and C++. I tried Haskell and found it amazing, but still too complex. After knowing more about functional programming I returned to Clojure and it finally clicked. I feel writing extremely simple code quickly and reinvesting that time into writing tests was a better workflow than any typed language I've used. I've only had a good experience in the REPL using emacs, but when you set it up properly, it's amazing.

didibus8 anni fa

That's my experience as well.

Static types are overrated. Okay, so you caught the obvious errors a few minutes earlier, but while you were wrestling with types, I ran 30 quick tests validating the behavior, and made them unit tests.

I've only ever had trivial type error bugs, all the nasty ones are behavioral or related to race conditions, or at the boundaries where types don't exist.

zumu8 anni fa

> Main contenders so far are Haskell, Scala and Clojure (Reason might go on the list soon too)

How about plain old OCaml? Jane Street seems to find some success with it.

nestorD8 anni fa

Or, also in the ML family, F# ?

bgorman8 anni fa

Reason and OCaml are the same language

gregwebs8 anni fa

You can't refactor Clojure without fear like in Haskell. It is actually possible in Haskell now to defer type errors to runtime, but I haven't heard of a practice of using that, even to aid in refactoring.

See this comment: https://news.ycombinator.com/item?id=18345672

akmiller8 anni fa

I'd disagree with this.

Primarily been developing with Clojure for 5 years now with some pretty large codebases. It does depend on how you write your code but favoring pure functions, pushing immutability to the edges of your programs allows you to refactor without fear.

Clojure also has many things to aid in this such as pre/post conditions, clojure.spec (which allows you to build complex type definitions), and of course test.check (property based testing).

lgas8 anni fa

Having spent the last two years working professionally in Haskell after having spent the previous two years working professionally in Clojure, I'd disagree with this. Haskell is roughly infinitely better than Clojure for meaningful refactoring. Clojure gives you (great) tools for using your own brain to make sure the refactor goes well. Haskell replaces your brain almost entirely in the process and just gives you a precise list of the things have to fix. The difference in experience is night and day. When you're done with a Haskell refactoring (even a major one) and it compiles again you will be completely done 99% of the time. With Clojure there's not even a "compiles again" moment to look for, and even when you think you're done you're refactoring there's a good chance that there's something that will pop up at runtime (and some point arbitrarily far in the future) and tell you otherwise.

ken8 anni fa

Just as with the word "developer" [1], I'm beginning to think that the word "refactor" is a dirty word. Are we even talking about the same activity? How is it people are alternately claiming that Haskell and Clojure (virtual opposites on the language spectrum) are infinitely superior to each other at the same thing?

Wikipedia defines it as "the process of restructuring existing computer code without changing its external behavior", which sounds about right, but it's also so completely generic that it could mean almost anything.

Claiming that Clojure can't do 'meaningful refactoring' sounds to me about like claiming that Kanji is bad for transcribing Welsh, or that sign language doesn't work well for audiobooks. They're technically languages but the fundamentals are so different that all the comparisons are talking right past each other.

The style of refactoring that one does in Haskell (disclaimer: it's been many years since I've written any) is not really possible in Clojure, but it's not really necessary, either.

[1]: https://news.ycombinator.com/item?id=18310958

didibus8 anni fa

> The style of refactoring that one does in Haskell (disclaimer: it's been many years since I've written any) is not really possible in Clojure, but it's not really necessary, either.

You're spot on. It just isn't really necessary to refactor Clojure code. In my many years using it professionally, I didn't have to refactor it once. The code just stays clean, it doesn't need cleaning up. So I find the whole refactoring argument moot. Why does your language lend itself to code that needs to be refactored? Sounds like a disadvantage to me.

lgas8 anni fa

I never claimed that Clojure can't do meaningful refactoring, and in fact I acknowledged that Clojure gives you great tools for using your own brain to do refactoring. Clojure is my favorite dynamic language and I heartily recommend it to anyone looking for something in that space. It's just that the experience of refactoring (or anything like it -- even just making large changes) in Haskell is massively better than the experience of doing it in Clojure.

wild_preference8 anni fa

I'd say it comes down to the more standard fare of dynamic vs static typing where the latter has a general advantage in refactoring.

It's true that Clojure has its own perks that aid in refactoring, like how you may have less code altogether. But in my experience it's still a drop in the bucket compared to the zoomed out view of dynamic vs static typing.

For example, could you give some examples that make Clojure particularly good in this regard? I'd have trouble coming up with many that quantify favorably against compile-time analysis. And it would be unfair to take this reality and say "Clojure sucks at refactoring." I think that people do say that is what clouds these discussions where one then needs to point out that Clojure has advantages over other dynamically typed languages which is definitely true.

Jach8 anni fa

"Refactoring" has a definitive origin, which is Martin Fowler's Refactoring book from 1999.

Fowler wanted to write the book using Smalltalk, but because the techniques he wanted to write about were fairly language agnostic, he did it in Java, as that was more popular. Unfortunately I think a lot of programmers missed the point, and now think refactoring is only something that can be done well in languages like Java (static typing) and with IDE support.

He's announced a second edition[0] in which he'll use JavaScript, to again prove the point that the techniques matter less than the language (other than sometimes techniques that work for class-based designs aren't as relevant as for function-based designs) and because JS is so popular. I'm not optimistic it will help anyone see the underlying point, but at the very least it might kill the idea that refactoring has to be hard in non-static-typed languages.

[0] https://martinfowler.com/articles/201803-refactoring-2nd-ed....

akmiller8 anni fa

To be fair, I wasn't comparing the two. I've never used Haskell (for more than just learning/tutorials).

I would suggest that if you have runtime bugs popping up in Clojure programs then that would suggest the inputs to functions (since they should be primarily pure) are not being validated which can easily be accomplished. I would imagine this needs to be done in Haskell as well since just verifying types does not indicate valid data.

matt-noonan8 anni fa

In Haskell designs, developers tend to be quite careful about setting things up so that verifying types does indicate valid data. Coming from other languages (C++, Common Lisp, and Python in my case), it can be a little surprising just how often and how easily you can make this happen.

akmiller8 anni fa

You can do similar things with spec for Clojure. As I stated in another response that is opt-in obviously so it requires more discipline perhaps but it is definitely available.

willtim8 anni fa

Null checks are also opt-in, that's why Tony Hoare calls null his "Billion Dollar Mistake". Explicit runtime checks are in no way "similar" to static typing! Static typing is a machine-checked proof of correctness for a certain class of properties.

matt-noonan8 anni fa

I'd rather have the compiler ensure that my discipline (and my team's) never slips, and to get guarantees about all possible executions instead of just those executions that have been tested.

For these reasons and others, I don't personally find run-time checks to be an adequate replacement for compile-time checks. But there is no really convincing research on the subject, and I certainly don't begrudge your preference here.

lgas8 anni fa

> To be fair, I wasn't comparing the two.

Well, the statement you were disagreeing with from the post you responded to was:

> You can't refactor Clojure without fear like in Haskell.

You go on to suggest you can achieve a similar experience in Clojure by "depending on how you write your code". This simply hasn't been my experience. Just "writing your code the right way" solves almost every problem that arises in programming, but just isn't always feasible in practice (on a team of developers with mixed skill levels, operating under deadlines, etc).

Re. validation, as Matt Noonan mentioned, in Haskell the goal is often to build/leverage correct-by-construction data types which obviate the need for any validation.

A simple example of this would be the `NonEmpty` (list) type.

If you have a function that pulls a list out of some key in a clojure map and it's intended that it always be a non-empty list you still need to check if actually is or not before using it because you have no control over what the caller passes to you. If the caller never sends an empty list you're fine, but if they do and you don't check for it, you've got a bug. Even if you do check for it, there's often nothing sensible you can do at that point since the local function shouldn't know anything about it's calling context, so you have to raise an error or return a nil or something.

On the flip side, in Haskell instead of using a map you would be likely to create a specific data type, and in that data type you would declare the non-empty field to be of the `NonEmpty` type. The first immediate benefit you get is that you no longer have to do any of these checks for the list being empty (or nil, or something else instead of a list) and instead just write your algorithm over the list. Among other things this results in cleaner, simpler, and less code in your immediate function.

But there's another benefit which is now anyone that calls your function has to have constructed a `NonEmpty` list before they call your function. The impact of this essentially naturally propagates the need to construct that `NonEmpty` list to the right place in the code. Maybe it really is just the caller to your your function that needs to take a regular (possibly empty) list that it has and package it up into a `NonEmpty` to call your function... and in that case you still get the benefits of code that shorter, simpler and more clearly communicates it's intention, but where this really shines is when that requirement to pass a non-empty list makes you realize something about the nature of your problem, and you let that `NonEmpty` propagate all the way out towards the boundaries of your application.

Then you end up in a situation where a) all of the code that touches that field anywhere is simpler, clearer, etc. but more importantly b) if someone does send you malformed data with an empty list (say via JSON over a web API or similar) then the code that deserializes the JSON into the `NonEmpty` will fail and you will get an error that says something like "Couldn't decode a YourCustomType from {whatever it was trying to decode}" at the very moment that the bad data tried to enter your system -- instead of just reading the JSON into a map because it was well formed and then letting the record with the empty list in it bounce around until it hits a function that assumes it's non empty at which point you may have little to no information about the provenance of the data or other details that would make easier to solve the problem.

akmiller8 anni fa

Exactly, and I was responding to the fact that I never fear refactoring my Clojure code.

Your examples of the type safety can all be mimicked with spec in Clojure. Granted spec is opt-in (but I'm guessing so is some of the more detailed type safety attributes you are talking about like NonEmpty).

Anyhow, to each their own and one persons experience isn't likely to be the same as the others so I'd encourage everyone to try out many languages. Some languages click with people more than others do so it's always worthwhile to experiment.

lgas8 anni fa

The opt-in vs opt-out nature is definitely key. As an example, if you write a small function to operate on a non-empty list and use it on a non-empty list in exactly one place, everything is fine in either language. But in Haskell if someone else tries to use the function passing in an empty list, it won't compile, they will encounter the problem right away. In Clojure if someone sees the function and says "oh good, someone's already written this function, I'll just use it" and doesn't realize that it expects a non-empty list, then you now have another situation where something is going to explode when data that contains an empty list arrives. And the error is again going to be far (in the code and in time) from the source of the problem. The difference between the experience of "I have good tools that I can use effectively to arrange to avoid bugs" and "I have tools that simply will not allow entire (large) classes of bugs at all" is what I'm really trying to get at. This is just an area where Clojure shines compared to most dynamic languages but Haskell shines compared to nearly all languages.

willtim8 anni fa

Explicit runtime checks that must be manually added are not a substitute for static typing! You should follow your own advice and try out a statically typed language with good inference!

Every tool has a sweet spot and large code bases and maintenance is well outside the sweet spot of any dynamic language.

jwhitlark8 anni fa

Well, there are certainly classes of problems that I'd use static typing for, but in my experience, static typing != software that gets the job done. That doesn't mean that it holds things back, though.

Many static typing arguments remind me of the Air Force's old "We'll bomb them so hard we won't have to send in ground troops." I just haven't seen it in practice, and the few studies that have looked at it empirically haven't seen a clear advantage either. If you know of such a study, please point it out!

In the end, all sorts of combinations have succeeded or failed, to the point where now when people start talking about "the right tool for the job", I add in "the right tool for the right people in the right environment for the right job..."

matt-noonan8 anni fa

> I just haven't seen it in practice, and the few studies that have looked at it empirically haven't seen a clear advantage either. If you know of such a study, please point it out!

This is basically right; there are not a ton of studies, and the ones that exist mostly have pretty bad methodologies. Dan Luu summarized a bunch of them, circa about 2014. [1]

Since then, there has been one study in this area that I think has a solid, well-defined, and plausible methodology [2]. Plus a "Threats to Validity" section, sorely missing from many other papers in this area. They work from a corpus of real-world public bugs in Javascript programs and quantify how many are detected by simple type annotations via TypeScript and Flow. They cap the amount of time for trying to resolve a bug with type annotations at 10 minutes. The result is that over a corpus of 400 bugs, they were able to resolve about 60 using either of TypeScript or Flow, suggesting that 15% of Javascript bugs can be eliminated by using either type system.

That's not a huge difference, but it isn't trivial either. The authors quote an engineering manager at Microsoft: "That’s shocking. If you could make a change to the way we do development that would reduce the number of bugs being checked in by 10% or more overnight, that’s a no-brainer. Unless it doubles development time or something, we’d do it."

I suspect with languages that are more amenable to static types the results would be even better, but there is no solid research that I know of to back that up.

[1] https://danluu.com/empirical-pl/

[2] http://ttendency.cs.ucl.ac.uk/projects/type_study/documents/...

jwhitlark8 anni fa

My gut feeling is the problem matters more than the language. I'm hopeful with the push to microservices we'll be able to mix and match these things where we think they'll work best.

tanin8 anni fa

> It does depend on how you write your code but favoring pure functions, pushing immutability to the edges of your programs allows you to refactor without fear.

Dynamic typing is punitive if we make mistake (e.g. using the wrong architecture, now we have to refactor), and we'll make mistakes.

Writing perfect code with perfect architecture and perfect test has been the argument "for" dynamic typing, but I don't think that's a realistic assumption.

yogthos8 anni fa

My experience is that static typing tends to lead to higher coupling and monolithic design, while you tend to break things up more aggressively in a dynamic language like Clojure.

tanin8 anni fa

I understand what you mean, and it's definitely a trade off.

Static typing requires all types to make sense together no matter how small the addition is.

In dynamic typing, we can just add a class/object/field here and there because it'll only be used narrowly anyway. So, it's often fine.

This would be great if we can think of a small example to illustrate this trade off.

yogthos8 anni fa

One example I can think of is how Ring middleware works. The request map is passed through a series of functions, and each one can modify it in its own way. For example, you could have an authentication library that adds an :identity key, and another that adds a :session key. These libraries can work together without having to know anything about each other.

darksaints8 anni fa

Clojure isn't nearly as bad as most would have you believe. I still prefer static strong typing, but since Clojure is strongly typed usually you'll find most of your errors within a few rounds of execution. And they have an excellent implementation gradual typing if that's your thing. If you value dynamic types, this is about as good as you can get.

ken8 anni fa

For some reason I find it funny that all the comments here about static typing are written in English, a language lacking not only static typing, but a syntax that's possible to automatically check. It lets one say things like "I still static strong typing" (which you've since corrected)!

One could write a comment here in Haskell describing why static typing is better, yet nobody ever does. I think even the most ardent static typing proponents are implicitly admitting that dynamic typing is fine, and other concerns can be more important, in some contexts.

Jach8 anni fa

You would probably enjoy this talk by Guy Steele where he discusses the language of mathematics that computer science people use in their papers (Computer Science Metanotation) to talk about other languages. Funnily it's untyped and lacking a rigorous specification... https://www.youtube.com/watch?v=dCuZkaaou0Q

jwhitlark8 anni fa

There's a reason why we teach math in kindergarten and proofs in college. There's lots of things that are useful to do without the rigors of rigorous proof.

bunderbunder8 anni fa

You can't necessarily do it in Haskell, either, on account of the whole partial functions thing.

preordained8 anni fa

Or lack of great IDE...the best type system seems a waste without a right-click refactor option to show for it. I'm not going to chase compilation errors file by file...

jose_zap8 anni fa

It has a language server which works with many good editors like vscode, that’s something at least!

udfalkso8 anni fa

Another option worth considering is Elixir. It's also dynamically typed, but the pattern matching helps mitigate dynamic type issues if used correctly. It's a pleasure to work with.

jb36898 anni fa

I love Elixir but it's worth noting that I'd be hard-pressed to recommend it for every project because of the performance and base OS integration characteristics of the BEAM. The BEAM is a truly amazing piece of software and I do think that Erlang and Elixir are the true microservices dream we all wanted, but the BEAM uses preemptive scheduling and doing things like spinning up a new OS process can be a lot more work in Erlang than you'd deal with in other languages

jimbokun8 anni fa

Why in the world would you spin up a new OS process in Erlang? Having its own process model is at the very core of Erlang's design, so it seems very odd to want to bypass it.

dnautics8 anni fa

What are the performance issues? People (e.g. Instagram) deploy scaled apps in Python, which is far less performant.

Bleacher report obsesses about information push latency, and they use an all elixir stack that deploys to at least a million users.

Not to mention WhatsApp.

lostcolony8 anni fa

CPU bound loads won't benefit from Elixir.

If most of what you're doing is a little minimal data transformation, but otherwise shuffling data to different network locations, and you want consistent latency/throughput, Elixir is a dream. If you're doing some tight number crunching, then not so much.

dnautics8 anni fa

Exactly. Who in sv is actually doing tight number crunching all the time? I think most of what we do is shuffling data around. If you need it, you can always also just orchestrate a container with a computational task anyways. And then someone's gotta marshal the data and metadata to that container... Might as well be something dreamy.

lostcolony8 anni fa

I'll also point out, I'd much prefer a default of Elixir, than of Java.

Why? Because most of my problems are things Elixir can solve well, and Java can solve moderately well, after a bunch of pain and tweaking. Default to the thing that can solve most of your problems well, and pick a different tool when needed, rather than default to the thing that can solve all of your problems moderately well, and you never pick anything else, and so never have any elegant, quickly built yet well designed, solutions.

udfalkso8 anni fa

Yup, it depends what you're building I suppose. For an Elixir/Phoenix web application or api I've found the performance to be very impressive.

dmead8 anni fa

as a long time haskell user (but no longer) i disagree that haskell is production ready.

yogthos8 anni fa

Worked with static typing for about a decade primarily with Java in the enterprise. However, I've also used Haskell and Scala which have advanced type systems. I moved to work with Clojure about 8 years ago, and I don't miss types. If I did, I would've gone back to a typed language a long time ago.

My experience is that dynamic typing is problematic in imperative/OO languages. One problem is that the data is mutable, and you pass things around by reference. Even if you knew the shape of the data originally, there's no way to tell whether it's been changed elsewhere via side effects. Keeping track of that quickly gets out of hand.

What I find to be of highest importance is the ability to reason about parts of the application in isolation, and types don't provide much help in that regard. When you have shared mutable state, it becomes impossible to track it in your head as application size grows. Knowing the types of the data does not reduce the complexity of understanding how different parts of the application affect its overall state.

My experience is that immutability plays a far bigger role than types in addressing this problem. Immutability as the default makes it natural to structure applications using independent components. This indirectly helps with the problem of tracking types in large applications as well. You don't need to track types across your entire application, and you're able to do local reasoning within the scope of each component. Meanwhile, you make bigger components by composing smaller ones together, and you only need to know the types at the level of composition which is the public API for the components.

REPL driven development [1] also plays a big role in the workflow. Any code I write, I evaluate in the REPL straight from the editor. The REPL has the full application state, so I have access to things like database connections, queues, etc. I can even connect to the REPL in production. So, say I'm writing a function to get some data from the database, I'll write the code, and run it to see exactly the shape of the data that I have. Then I might write a function to transform it, and so on. At each step I know exactly what my data is and what my code is doing.

Where I typically care about having a formalism is at component boundaries. Spec [2] provides a much better way to do that than types. The main reason being that it focuses on ensuring semantic correctness. For example, consider a sort function. The types can tell me that I passed in a collection of a particular type and I got a collection of the same type back. However, what I really want to know is that the collection contains the same elements, and that they're in order. This is difficult to express using most type systems out there, while trivial to do using Spec.

[1] https://vvvvalvalval.github.io/posts/what-makes-a-good-repl.... [2] http://danboykis.com/?p=2293

cutler8 anni fa

This nails it for me. Clojure has unique advantages over other, less composable languages and the design of clojure.spec is where you see it most clearly. Rich Hickey's presentation on clojure.spec demonstrates how you can define contracts for your functions which go well beyond what type systems have to offer.

kachnuv_ocasek8 anni fa

Some implementations of SML are definitely production-ready.

plinkplonk8 anni fa

which implementaions of SML are the production ready ones?

dmead8 anni fa

clojure has an out of band type checker. have another look

nerdponx8 anni fa

What happened to Clojure? Was it just a fad, or are people still using it in their day jobs? Are people still hiring Clojure devs? If you write Clojure at work, are you happy with it?

tombert8 anni fa

I work for one of the big "brand name" companies (I don't want to say which, but it's on the "companies using Clojure page"), and on my team we try and start all new projects to use Clojure.

Frankly, I love it. After the "zen" of lisp clicked for me, Clojure gave me an OS/2-style "Better Java Than Java". Suddenly I was able to compose standard JVM library stuff in a more-natural and easy way than I could in vanilla Java, I was able to easily define relatively-well-optimized persistent data structures, and I could quickly provide new semantics to the language as I saw fit. Also, the Clojure REPL provides an amazing level of increased productivity.

Do I miss the Hindley-Milner type system from my last job? A bit, but surprisingly it doesn't bother me as much as I thought with Clojure.

I don't know if Clojure was just a fad, but I think that it'll always have a loyal fanbase, like mosts Lisps.

sunshinerag8 anni fa

Care to mention what is the language you used in your last job that you are comparing clojure with.

tombert8 anni fa

I was using F#, and before that Haskell, and before that Scala.

anon12538 anni fa

We've been using a full Clojure stack (front-end + backend) in production for nearly 4 years and very happy with it (6-8 people full time). But that is very much anecdotal. I guess one thing to note is that the language is very stable, there are very few (if any) breaking changes and a 4 year old library will "just work" most of the time. Even more amazing is that it's trivial to port 40 year old ideas from (Common)Lisp if you're so inclined, or interface with a known good Java implementation. To me that is a big plus, personally I wouldn't even want a ton of extra features in Clojure ... the language is fine and libraries fill in the rest. It does give the impression that not a lot of happening and that it might be fading or boring, but in a lot of settings boring is actually a good thing. The amount of times some NodeJS or Python thing changed from underneath me and I needed to rework an entire codepath or build infrastructure just because the language changed their mind on how things should be done are gladly behind me with Clojure!

jimbokun8 anni fa

"...personally I wouldn't even want a ton of extra features in Clojure ... the language is fine and libraries fill in the rest."

And for anything else there's macros.

Naomarik8 anni fa

My feeling (and personal experience) is that the state Clojure is in now is so solid there's not a lot to talk about. Every time I take a peek at the JS ecosystem there's a new flavor or build tool.

I've been making a living using Clojure for a few years now as a solo dev and I don't feel the language is lacking in anything I would care much about. I started with reagent not knowing anything about react or having any Clojure experience and was immediately productive. I was able to augment part of a webapp I had in production in no time due instant feedback loops from the REPL and figwheel. The language is being iterated on but it has already been in such a state in that I'm not keeping up with what's new because I'm able to solve any problem I need to already.

I'm extremely happy with it. I have a lot of interests other than programming and I feel that Clojure allows me to deliver complex projects on time without driving me insane or having to keep up with new trends.

tribaal8 anni fa

We are a mostly Clojure shop and are very happy with it.

It's very expressive, functional, and allows you to tap into the Java ecosystem if you need to.

Might not be as hyped today as it was some years ago, but the language itself is still a joy to use.

Edit: SWE at exoscale.com here. We're hiring, also for full remote positions!

catbodi8 anni fa

Dang thats some expensive cloud pricing.

Your cheapest plan offers 512MB of RAM for 5 Euros a month. There's well over 5 competitors who can get you quadruple that amount of RAM and maybe even double the storage for half that price per month. This is due to renting out space at datacenters, no? I'm no hosting expert.

pyritschard8 anni fa

Hey, Exoscale CTO here,

Thanks for looking into the service. We're certainly not the cheapest in terms of pricing but I would also warn against using RAM prices as the sole way to gauge a service.

To drive low prices a lot of VPS providers tend to have a very liberal way of playing with overcommit, which we don't do. The same goes for CPU.

Our instances should be considered to higher-end families such as the C3 and here you'll see we're playing in the same field in terms of prices.

tribaal8 anni fa

It's due to us offering some guarantees on what you get, and therefore not over provisioning our hardware :)

Many of our clients choose us for performance (IOPS, memory guarantees, etc...), and our geographic and legal situation (no connection to 5 eyes countries, strong privacy laws).

calgoo8 anni fa

That 512mb server can cost several thousands at enterprise datacenters so its not that bad. It all comes down to who their customer base is.

jakebasile8 anni fa

The industry can rip Clojure from my cold, dead hands. It's still used in smaller teams at a lot of places including some of the BigCos. It's a simple joy to use and it is a damn shame it's not more popular than it is, but I don't think it is a fad.

jb19918 anni fa

The cancellation of a few major Clojure conferances hasn’t helped. I’m guessing it speaks to demand.

I used Clojure for a long time, professionally. Not much any more. Main issue is that the language is not very readable (a pain point mentioned in these slides) and very hard to find people to work in it. Also more than any other language Clojure is very sensitive to typos in your code. A misspelled keyword results in nil accidentally flowing through your app. It was frustrating to spend time on scaffolding everywhere to catch these things.

sdegutis8 anni fa

Also the Clojure language had a huge growth period around 1.1-1.5 which sparked a lot of excitement and spurred a lot of community growth. But since then, most of the exciting features have been adapted to other mainstream languages either as features or libraries, and most senior developers I know prefer to take the path of least resistance, which includes avoiding less-mainstream languages and frameworks. So instead of using Clojure with Compojure, it's common to move to Java with Dropwizard, or slightly less analogous, JavaScript with Express.js. And instead of using ClojureScript with Om, people mostly jumped to JavaScript + React.

Edit: Removed comment about Om, since I must be misremembering something. Om uses React and has since the first commit.

KingMob8 anni fa

What are you referring to by "instead of using ClojureScript with Om, people mostly jumped to JavaScript + React, which the Om team later praised as being the same concept they had in mind, but with a simpler and better API"?

Om uses React, it didn't predate it or anything. Many of the things Om tried are more relevant to things like Redux, not React proper, so I'm not sure what your comment is referring to.

And as a counterpoint, React just introduced the State and Effect Hooks, but those are both things that Om and reagent (the most-used Cljs React lib) have been able to do for years.

sdegutis8 anni fa

> What are you referring to by

Apparently I was remembering Pedestal. See my other comment.

> Many of the things Om tried are more relevant to things like Redux

> And as a counterpoint, React just introduced the State and Effect Hooks, but those are both things that Om and reagent (the most-used Cljs React lib) have been able to do for years.

These are great evidence for my point, that Clojure has worked as a testing ground for features which more mainstream languages and frameworks then adopted.

KingMob8 anni fa

I don't think I was clear enough. Yes, React (and others) has adopted things from Clojure/script, but to me, that's a sign you're using inferior tools that are several years behind.

E.g., I used to do a lot of PHP, and my last PHP project was in Laravel, which is heavily influenced by Rails, and PHP 7. Many people touted "Modern PHP is just as good as Python/Ruby/Js!" but missed the point that they'd been using tools with inferior capabilities for years.

harperlee8 anni fa

Re. your edit, which erased "instead of using ClojureScript with Om, people mostly jumped to JavaScript + React, which the Om team later praised as being the same concept they had in mind, but with a simpler and better API": I’m pretty sure that the coment you mention referred to reagent (which builds on top of React as much as om).

sdegutis8 anni fa

My memory is apparently failing me, but I remember it being one of the very first "reactive" web frameworks, before React came out, and being made by the makers of either Clojure or Datomic (before they were Cognitect) and possibly also David Nolan. And I remember them discontinuing it after React came out, saying something along the lines of "We had this idea before React came out, but our method involved explicitly changing UI when state was updated, whereas React does that automatically for you and in hindsight that was a great improvement over our idea. So we're discontinuing this and going to change the focus of the project to work with React instead." I'm pretty sure the project had a slightly longer name than Om, and was a big thing for a total of a year, where everyone jumped on board and jumped off again. Unfortunately I think I mixed this up with Om / OmNext.

---

EDIT: Oh! Just figured it out: Pedastal. Yep, originally made by Relevance Inc.

Here's the link of its discontinuation which mentioned React as a big motivator: https://groups.google.com/d/msg/pedestal-users/jODwmJUIUcg/6...

> "But unlike in Pedestal App, you don't have to worry about how fine-grained changes to your App model map to the UI."

That's the part I was remembering.

> "So there is clearly conceptual overlap between Pedestal and ReactJS. Further, David Nolen's work on Om has shown that ReactJS and ClojureScript work very well together."

That's probably why I got this mixed up with Om. This all happened about 4 years ago so I guess my memory isn't as fresh.

shady-lady8 anni fa

> he Om team later praised as being the same concept they had in mind, but with a simpler and better API.

is this in public? cursory search did not return any hits.

hellofunk8 anni fa

Um, Om is React. React came first and Om interfaced with it.

smneiswrngonthi8 anni fa

kamaal8 anni fa

>>A misspelled keyword results in nil accidentally flowing through your app.

Some dynamic languages, worked around this issue in the 90's. Perl for example, if you use 'use strict; use warnings' will catch most of these misadventures.

Having said that, only the most novice programmers make these kind of mistakes. Like the absolute beginners, in the first day or two of exposure to programming.

As a rule almost any shop these days write unit test cases, so that should flush out these kind of bugs.

EDIT: Why the downvotes? People seriously work with programmers whose biggest incompetence at work is typos? Or you think writing unit test cases is bad?

nosefrog8 anni fa

With all due respect, I think your argument is a little silly. Typos are the one mistake that programmers never stop making no matter how experienced they are, in my experience. For the same reason, you can pick up a book by an experienced, established writer and find typos in it as well.

It's useful for your tools to help you catch basic mistakes like that instead of just saying you should work with programmers who are competent enough to never write typos.

kamaal8 anni fa

I don't think we are talking about that kind of typos. But there is a often a big dip in quality of people, who routinely struggle to write a half decent working program without making these kind of mistakes.

Sure every once in a while its possible for any one to drop in a spelling mistake, but those are rare situations in code. Mostly because code is often not a dead piece of documentation, which is read and auto corrected in the human brain. Code works on a computer with well defined interfaces and protocols.

Think of it this way, would you hire a programmer who often makes typos, to write shell scripts that have 'rm -rf'?

If you wouldn't, how often do you trust these people to execute DROP DATABASE <typo_database_name_which_happens_to_be_real>?

There is a big difference between programmers who can put in typos in a sentence in a code comment, and the kind of ones we are talking about.

hellofunk8 anni fa

I've deployed tens of thousands of lines of Clojure for a company over several years, and I still find that I occasionally destructure a map with the wrong keyword and I end up with nil accidentally. It's frustrating as much now as it has ever been. Because the app just crashes instead of a simple compiler check that most languages would offer to capture simple stuff like this.

jurassic8 anni fa

You don’t have to be an absolute beginner to make this mistake. You just need to be on a project with inadequate test coverage.

StreamBright8 anni fa

They just disagree without explaining why. Anyways, I sometimes have typos in my Clojure code, but once I follow through with your mental model and just reproduce the problem in REPL it is easy to find these sort of bugs.

kamaal8 anni fa

I'm not sure, but if people have problems, as bad as things like hiring programmers who struggle with indenting code, making typos in variable names, write functions hundreds of lines long, can't modularize code, write 3 nested for loops, or write nested if's where '&&' suffices.

Then you are having far bigger problems related to project management, hiring and overall direction in which you are running your team/company.

Clojure as a tool doesn't aim to fix these kind of problems. And it can be argued no tool will fix these problems.

tome8 anni fa

> if people have problems, as bad as things like hiring programmers who struggle with indenting code, making typos in variable names

Those are the kind of things that the computer is best suited to. It's better if the programmer's brain is left to other, more challenging tasks.

kamaal8 anni fa

One wonders what kind of challenging tasks a programmer who can't even indent code, can solve.

By the way indenting code happens to be the most basic of code review checks.

tome8 anni fa

Why waste human time on what can be done by a machine?

cmoscoso8 anni fa

In my experience, a compiler would be wasting my time interrupting me with errors that are easily catch later, playing with the clojure REPL.

200px8 anni fa

Code indentation has been automated away since like 10 years or ago or maybe even more.

Vim, Emacs, Atom, Sublime Text, Visual Studio, Eclipse, IntelliJ, NetBeans can all auto-indent code for you. Combined with linters they can also automate stylistic checks and coding convention.

I don't know anyone who indents code themselves. The editor/IDE does it for them.

Indenting code is certainly not a code review check. No sane workplace should waste a senior developer's precious time commenting about indentation problems during code reviews. It is a tooling check that is part of CI/CD.

yogthos8 anni fa

Clojure use has been growing steadily, and lots of companies are hiring. In a recent JVM survery of 10K people Clojure came in as the second most used language after Java https://snyk.io/blog/jvm-ecosystem-report-2018

My team has been working with Clojure for the past 8 years, and we're extremely happy with it.

cutler8 anni fa

These stats look highly suspect with Clojure:Scala = 1.6. Here in London the jobs boards Angel.co and Indeed.com have Scala:Clojure at 3.6 and 19.6 respectively.

yogthos8 anni fa

It's one of the most comprehensive surveys to date. Meanwhile, the job numbers can relate to many factors. For example, there could be more turn around for Scala jobs.

cutler8 anni fa

The survey opens with:

"Welcome to the largest survey ever of Java developers ... The survey was conducted by publishing its availability to the Java community at large (via social media primarily), to Java User Groups around the world, including the Virtual JUG, and to subscribers of Java Magazine, the Java bimonthly publication from Oracle Corp."

So the sample is heavily biased and doesn't include the population of JVM users who have never touched Java.

yogthos8 anni fa

That's very likely the case, but presumably you'd get an even sampling from the users of other languages. It would be interesting to see some more surveys focusing on the larger JVM ecosystem though.

cutler8 anni fa

As far as jobs go let's face it - Clojure has never seen anywhere near the adoption of Scala.

mecameron8 anni fa

We've been using Clojure for 3 years and ClojureScript for 1.5 years. We're actively converting more of our legacy codebase to Clojure/Script.

It's been a great choice for us and even the skeptics have become converts over time. I think it's a joy to work with and allows us to reason about our problems and solve them better than many alternatives.

pixelperfect8 anni fa

Clojure is still around and a great language for experienced devs. But the experience for newcomers is much less attractive than for other languages, which is preventing growth. Also, the largest Clojure community is the Slack channel, which is hidden from the rest of the world.

jakebasile8 anni fa

I really wish the community would move away from using Slack. It's completely opaque to most people and anything past the last couple days gets lost to the ether.

Yes, I know you can find it on some archiving site, but it's non-intuitive. Wish they'd move to Discord or Matrix or... pretty much anything else.

StreamBright8 anni fa

I regularly use Clojure at work, even though I am usually a sole contributor because other people are not willing to use anything other than Python, Go, Java. The primitives for doing concurrent and parallel programming is just so nice that I can't think of living without it. Go has similar solutions though. The biggest advantage is to have a concise small code base that does exactly you want without dancing around language related boilerplate.

swah8 anni fa

While other posters comment that "Clojure still works", IIRC the "promise" was that Clojure/Lisp would be much, much more productive, and that claim I feel like must been disproved in daily work, otherwise Clojure would still a main topic around here.

Reminds me of the classic Erann Gatt post: https://news.ycombinator.com/item?id=2308370

(I loved to play with Elisp, Clojure, Lua, Factor, Ion, Ioke and hope dynamic languages become fashionable again someday...)

empthought8 anni fa

I think individual developers are most likely to become much, much more productive when using Clojure/Lisp vs. Java, C#, and the like, but software development teams are unlikely to experience the same benefits.

And it's not even because Clojure is bad for teams, it's because teams often form because there are nontechnical stakeholders, and suddenly an individual contributor's code-writing productivity is a much smaller factor when compared to the communication practices and other parts of the development process outside of writing code.

lukev8 anni fa

It is (sadly) not clear at all that programming language popularity (or rather, "online buzz" around a language which is all you can measure) correlates strongly with productivity.

KingMob8 anni fa

I can only speak for myself, but I'm definitely much, much more productive using Clojure. I think its absence from HN is more reflective of a less-than-stellar onboarding experience for newcomers, and HN's general neophilia.

StreamBright8 anni fa

If there is one think about humanity that is worth to note is that the shittier alternative always wins. This is true from money through computer architectures to programming languages.

jimbokun8 anni fa

"hope dynamic languages become fashionable again someday..."

Python, Javascript and Ruby are very dynamic.

quantisan8 anni fa

We use Clojure for the whole backend, Python for machine learning, and JS for frontend at Motiva AI. It's the fourth company that I worked at (UK and US) with Clojure as one of the main languages. I'd say hiring Clojure devs has been one of the best perks for working with Clojure. Consistently high quality people that I've had the honour to work with.

At Motiva, we've dabbled in other languages like Haskell, Scala, Rust, Go, etc. but Clojure has been what sticks for us on the backend team https://github.com/Motiva-AI. We are a fully distributed remote team and we are hiring. Drop me an email.

tosh8 anni fa

According to the JVM Ecosystem report/survey Clojure is the 2nd main language on the JVM (after Java):

https://snyk.io/blog/jvm-ecosystem-report-2018

ClojureScript also seems to gain traction. Quite impressive when you consider that Clojure is a language/ecosystem not driven/backed by one of the larger tech companies.

cutler8 anni fa

See my earlier post in response to yogothos. On jobs boards Scala adoption is orders of magnitude higher than Clojure.

ljosa8 anni fa

We're using it happily at Yieldbot. We're an ad network, and the core matching technology (which finds relevant ads based on the contents of the page and decides which one to show and how much to bid) is all Clojure. All the data science stuff is Clojure (it's been great with Spark and pretty good with Storm). The data pipeline is all Clojure now. The central database for ad campaigns and such is Datomic. The APIs for the frontends are moving to Clojure.

ClojureScript never caught on for the front-end work. There are some substantial services for which Elixir was chosen instead of Clojure.

eigenhombre8 anni fa

I've been doing Clojure professionally happily for five years. My company/team in Chicago uses Clojure and is hiring (onsite). https://hire.withgoogle.com/public/jobs/opploanscom/view/P_A...

abc_lisper8 anni fa

Yes, we are using it at a BigCo. I am extremely happy with Clojure. I wouldn't think Clojure is a fad, it never was that popular.

joseakle8 anni fa

We use Clojure at Nubank.

We’re actively hiring Clojure devs or we can teach those who don’t know it yet.

https://nubank.workable.com

We’re very happy with it.

z3t48 anni fa

It's interesting that in the current state of software development, a programming language that have been stable for not even a year might be considered obsolete.

KingMob8 anni fa

A perennial problem in Clojure is libraries that reach a mostly "done" state tend to look inactive to outsiders, because the language is so stable there's almost no maintenance work.

So inevitably, someone will ask "Is lib foo still being used? The last commit was over 8 months ago" and then people have to chime in, "yes, it's still in heavy use".

nerdponx8 anni fa

Several languages have this problem, e.g. Common Lisp. It would be nice if developers of libraries like this at least updated their copyright annually, as an indicator that they and the project are still alive.

Jach8 anni fa

Who is the indicator for though? Wouldn't users of that language quickly come to expect that inactive is not necessarily dead or useless? (Especially with Common Lisp, stuff decades old is still useful.) And I find outsiders unlikely to care that much, it's just a nit random passerbys will make when they think they need to comment on something, rather than actually informing a serious decision.

It's also kind of pointless to "update the copyright year" for individual works tied to your name, since the copyright lasts x years after your death. If anonymous/pseudonymous, then it's like 95 years after initial publication date, and you can't just extend that arbitrarily by updating the base year and nothing else every year..

imhoguy8 anni fa

Damn, you have just brought me an idea for next weekend project. Copyright-year updater which submits PRs to GH projects :) In Clojure of course!

jwhitlark8 anni fa

I think there needs to be something like the Apache foundation for Clojure libraries, not handling major dev work, but maintaining working libraries over time.

mayneack8 anni fa

We're still using Clojure at Factual and are hiring. https://www.factual.com/company/careers/?gh_src=1pt2v8

dgritsko8 anni fa

Barely hanging onto the top 50 at #49 according to the TIOBE index (Java is still #1). Make of that what you will. https://www.tiobe.com/tiobe-index/

yen2238 anni fa

That index lists Ladder Logic (#45) as being ahead of Bash (#48). Make of that what you will.

jcelerier8 anni fa

well... yeah ? there are entire companies of thousand of electrical engineers working in LL. They just don't come and post about it to hackernews and make PLC projects in github, they do their 9-5 job and that's all.

Compare https://www.indeed.com/jobs?q=PLC+engineer and https://www.indeed.com/jobs?q=bash+programmer

ilikehurdles8 anni fa

>They just don't come and post about it to hackernews and make PLC projects in github

Same logic applies to clojure. I think that's the point he's making. Everyone and their 12 year old nephew rockstar developer has questions about java. I also find that clojure questions nowadays are mainly asked in the clojurians slack organization. Immediate feedback is more attractive than waiting around hoping for an answer in stackoverflow.

nogridbag8 anni fa

Personally I prefer StackOverflow. SO is a wiki and it's likely my question and others' comments and answers will help someone else in the future.

With any chat interface, it feels like your question may be lost if it's not immediately answered and it's often hard to follow discussions with multiple people chatting. With SO I will spend time to try and formulate a clear question. In general with slack or any other chat I may feel out the room first as it will be a waste of my time writing a detailed question if no one is around to answer.

Perhaps some sort of SO integration into Slack would be the best of both worlds?

KingMob8 anni fa

While Slack is the most popular forum, there's also Clojureverse.org, which aims to be a more threaded and permanent repository of Clojure discussions and knowledge.

kamaal8 anni fa

That's because TIOBE derives metrics from search results. Which means every time some one searches 'Python, the snake', that also gets counted as 'Python, the programming language'.

Most people looking for help on the internet are searching:

"How do I do X in shell?"

No of people looking for help with the word 'Bash' are likely to be lower.

That's one reason why TIOBE is sometimes way off the real trends.

steveklabnik8 anni fa

I don't like TIOBE, and think it's a pretty bad index, but they search for "$LANG programming", not just $LANG.

https://www.tiobe.com/tiobe-index/programming-languages-defi...

200px8 anni fa

> Which means every time some one searches 'Python, the snake', that also gets counted as 'Python, the programming language'.

This is false.

phaedryx8 anni fa

What I make of it: "I'm quietly being productive with $LANG in my day-to-day job and loving it" doesn't move a language up the ranking.

zmmmmm8 anni fa

I like the CloudFoundry report [1] better as a reflection of what businesses are actually using rather than search engine metrics (TIOBE) which are overly influenced by controversy and search engine methodologies than actual usage. However it's no prettier a picture, almost no functional languages in the list at all, only Scala making a significant dent (and you can debate whether that is functional or not). Other JVM languages like Groovy are significantly ahead of Clojure in all rankings.

[1] https://www.cloudfoundry.org/wp-content/uploads/Developer-La...

vorg8 anni fa

> Other JVM languages like Groovy are significantly ahead of Clojure in all rankings.

Apache Groovy fluctuates on TIOBE a lot. Six months ago it was at the same 50-ish position as Clojure, which is its usual ranking. It's shot up only in the past 6 months, which has also happened a lot in the past and is always followed by a sudden fall -- perhaps someone's gaming the search engine rankings. On Redmonk, Groovy and Clojure have ranked equal-ish for a long time. In the CloudFoundry report, Groovy is at the same very low rank as Others which would include Clojure if present.

So Groovy and Clojure are at the same ranking in all rankings I've looked at.

cutler8 anni fa

TIOBE is a joke.

jcadam8 anni fa

Not a fad. Clojure 1.9 was just released. I can only use it on personal projects, as employers hiring for Clojure are few and far between.

imglorp8 anni fa

One datapoint if it helps someone: these guys are full clojure. https://www.relaynetwork.com/careers/

Dirlewanger8 anni fa

CircleCI uses ClojureScript last I read.

shady-lady8 anni fa

think they stopped: https://www.reddit.com/r/Clojure/comments/9ho97m/is_circleci...

smoe8 anni fa

That link only refers to the frontend where they used to have ClojureScript.

Judging from their backend job postings they seem to use Clojure and Go. .

projectileboy8 anni fa

This is a bit tangential to the post, but I've seen a couple comments in this thread that amount to "Clojure is hard to read", but I think that's a really unfair comparison, because people are unwittingly comparing apples and oranges. It seems to me the reason people think Clojure is hard to read is that the language is so powerful and expressive, that when you're reading Clojure code, you're typically trying to understand the actual problem that's being solved, as opposed to just reading a bunch of code that's necessitated by the lack of power afforded by the language. For example, I can easily "read" a bunch of nested for loops in Java, but then I still have to try and understand what the actual problem is that's being solved. People seem to confuse those two things when comparing Clojure with other languages.

ronnier8 anni fa

I worked in a large company where clojure was used. After the original developers moved on from that project (they always move on), it was a giant struggle to get people who could work in the code base. They ended up rewriting it in something more standard. Whatever benifit clojure provided was outweighed by the high cost of maintaining the code base and the inability to find devs who wanted to work on it.

That’s my only data point with clojure.

TeMPOraL8 anni fa

It saddens me to hear such stories as they show up from time to time. I can't help but feel that a lot of programmers just decide to stop learning at some arbitrary point in their career (my observations suggest that point is around the end of their university term or their first job). The industry too seems biased heavily in favor of using lowest-common-denominator, dumbest possible solutions they can get. And not "dumbest" in the sense of "simple therefore robust" - but in the sense of "barely suitable for the job at hand". It's like wanting to dig a pool, and hiring a lot of people with spades, even though you can get excavators for free - but what, you couldn't be bothered to find a few drivers? And the spade crew comes and is all, "no way we're going to learn to operate an excavator, that's too much, let's dig by hand".

I can't explain it any other way. Learning Clojure, or another Lisp, to the point of being comfortable around a codebase, is a matter of weeks for an experienced programmer. It's comparable to the time you need to even figure out how to rewrite an existing mid-sized codebase, and since you have to read it to understand it, why not learn the language along the way? Then suddenly the rewrite isn't needed. But it feels like a concept of learning a new language on the job isn't even considered by people.

lliamander8 anni fa

> I can't help but feel that a lot of programmers just decide to stop learning at some arbitrary point in their career (my observations suggest that point is around the end of their university term or their first job).

Indeed.

I had the frustrating, but ultimately fortuitous, situation of having my first job be one where I was asked to build a solution with woefully under-powered tools. I'm not even talking about the difference between hand tools and power tools - I was essentially asked to use kid toys.

That situation made me very angry, and I was constantly working overtime to make up for the lack of productivity of my tools - even then we were going to miss our deadline.

Finally, I managed to convince management to let me and my team use real tools - nothing super advanced, but well suited for the job. Very quickly we turned around the release, I started working normal hours, and when I left that job shortly after the release my employer found that they didn't need to hire a replacement because the workload was so much less.

This experience taught me that the power of your tools does matter, and that the initial pain of technology transition can easily be outweighed by the benefits of using something better.

For me, the quality I look for in my tools is: does it reward mastery? I'm not (primarily) looking the initial growth curve and how long it takes before I get to my current productivity. Rather, I'm looking at where the ceiling is.

pyrhho8 anni fa

Funny, I have the same story from the other side. They could have chosen to not alienate and lay off the clojure team. We were pretty happy, working on interesting projects, independent enough to get things done. But, turns out paying more for a smaller team of experts who know what they’re doing isn’t compatible with Quarterly-Earnings-Driven-Development.

nogridbag8 anni fa

If I were seeking a Clojure job I would probably avoid companies that only use it for a single app while 90% of the active development is in more mainstream languages, especially if at any point during the interview process I get a feeling like I was being hired because all the other devs on the project jumped ship.

jwhitlark8 anni fa

I think this is a valid point. We talk about the right tool for the job, but forget that the evaluation needs to include the people using the tool, the people paying for the job to be done, and other elements of the environment.

anon12538 anni fa

Excellent points! From my personal experience I'd also like to add that there is a bit of preference involved. I find Lisp like languages much much easier on the eyes compared to Javascript or other C-like syntax. This is also partially due to the excellent support of s-expressions in editors with things like paredit [1], parinfer [2], or clever-parens [3]. With that functionality the syntax becomes nearly invisble and you're editing a /tree/ rather than characters. Coupled with autocomplete and auto-indent [e.g. 4], the code is much easier to manipulate in my experience (but again, might be preference)

[1] http://danmidwood.com/content/2014/11/21/animated-paredit.ht... (also http://emacsrocks.com/e14.html)

[2] https://shaunlebron.github.io/parinfer/

[3] https://github.com/luxbock/evil-cleverparens (Vim with Emacs via Evil)

[4] https://github.com/Malabarba/aggressive-indent-mode

vindarel8 anni fa

Damn lispy is like all of them in one !

http://oremacs.com/lispy/

rubber_duck8 anni fa

Not my experience - for eg. I remember reading trough implementation of a clojure standard lib core.async (this was years ago) https://github.com/clojure/core.async/blob/master/src/main/c...

It took me a long time to figure out what's going on in that code - I remember realizing how it would have been much simpler to read in a popular statically typed language like C#, types are a lot like compiler verified documentation and in undocumented code they help a lot. To be fair this is not the norm - I've seen many cases where Clojure removed incredible amounts of boilerplate and made reading the code straightforward, and the nicest thing about it was that it made simple things simple.

I haven't worked with Clojure in couple of years, the reason I left it is because I felt it wouldn't scale well in the real world (in terms of developers and teams) but it really helped me "get" LISP and FP so I don't regreat learning it

simongray8 anni fa

That source file you're referencing is a fairly low-level library doing lots of interop with Java. That's not really what most Clojure code looks like.

And furthermore: it's just a library, but this library is adding the concept of go blocks (as seen in Go) to the Clojure language. Are you even able to add a new control structure like go blocks to the C# langauge by including a library?

rubber_duck8 anni fa

It's not really about Java interop, it's about low level procedural code - Clojure doesn't really look nice when doing it. It does a lot of stuff nicely but there are cases where it just feels wrong to use.

But your second point is pretty much the reason why I wouldn't use Clojure in a bigger environment.I had the misfortune of working C# on a project where some smartass decided it was cool to use Maybe monad in C# using LINQ (something like https://github.com/louthy/csharp-monad) First it provided no actual value as the code was harder to read, but ignoring that - most of the developers didn't get it and just used it enough to get shit working, it was a hell to debug and maintain.

core.async is public and in core - but what about when your team meber X decides to implement that - and now he sprinkles that shit all over your codebase - people keep using it because they don't want to argue. Clojure really lends it self to that kind of a thing where you do that "smart thing" to solve "that one problem" that ends up being a pointless mess that's hard to maintain and reason about.

jb36898 anni fa

I think my issue with Clojure is that it's hard to learn how to write good Clojure. There is very clearly a way that it should be written (to keep the code clean, readable, and concise) but unless you are very familiar with working with Lisps then it's easy to create a giant mess

You could say that for a lot of other languages too, but you usually have some familiar base to work off of. That's less true with Lisps as they are so syntactically different than more mainstream languages

Royalaid8 anni fa

This is getting better but unfortunately your best bet is still to read some examples of good open source projects. If you or anyone interested in some good starting places for good Clojure see:

  - https://github.com/gothinkster/clojurescript-reframe-realworld-example-app for a great webapp example
  - All of https://github.com/ztellman work but specifically:
    - https://github.com/ztellman/manifold for more advanced Clojure
  - https://github.com/reagent-project/reagent for interop with JS and advanced Clojurescript usage

kickopotomus8 anni fa

This is my experience as well. I started working pretty heavily with Clojure about 2 years ago and just now feel that I am getting the hang of it in terms of style, yet I still feel like I have a ways to go.

This has not been my experience with languages like Java, Python, Go, etc. where I feel that I can write good, clean code in a matter of weeks from reading through style guides and understanding language conventions.

udfalkso8 anni fa

I just inherited a Clojure project, one which I'm quite well acquainted with the business logic, and I find it extremely hard to read. It's very different from the many other languages that I'm comfortable with already. I'm sure with sufficient learning I can get comfortable parsing what's going on, but it is a problem.

jwhitlark8 anni fa

If you can, I'd suggest use spec to improve your understanding and verify behavior. I certainly do know that Clojure can get very dense if you're not careful.

http://blog.cognitect.com/blog/2017/6/19/improving-on-types-...

mmartinson8 anni fa

Just out of curiosity, do you have any previous experience or familiarity with FP? Is it the List syntax, the project organization, or the immutable constraints that are the most challenging?

udfalkso8 anni fa

I've built some pretty large apps with Elixir (FP and also has a pipe operator like Clojure). I guess it's the lisp syntax, the deeply nested parenthesis, the prefix notation, etc.

PopsiclePete8 anni fa

I don't think it's hard to read, I think it's extremely difficult to analyze and almost impossible to debug.

Lazyness is one of those "cool" features, IMO, that have more cons than pros. Yes, certain algorithms look more "elegant", but it's also harder to reason about/debug.

Most of the time, you can ignore the fact that this awesome language runs on the JVM, until you see a barely-comprehensible stack trace containing nothing but `org.clojure.Iseq` something-or-other and no user code in sight.

Looking at how a "real" LISP displays errors and allows you to debug it and comparing it to Clojure makes me feel really sad inside.

Sadly, these are the reasons I tend to favor Go for work-related projects. It's less "fun" to write than a LISP, but makes pinpointing and fixing errors a breeze in comparison. And when my job is on the line, that's the killer language feature for me.

nybble418 anni fa

I don't think laziness is really that much harder to reason about, unless you're trying to fit it into a mental model based on eager evaluation. Evaluation occurs when the result is needed; that's so hard about that? And it's not really about making algorithms look "elegant" but rather about composiblity and reuse: it's much easier to add eager evaluation to non-strict interfaces than it is to get non-strict evaluation from strict interfaces. In the former case you're adding additional behavior, which can often be accomplished with a wrapper; in the latter case you're faced with removing something baked in to the API.

Eager evaluation is merely a performance optimization, one which assumes that the data being evaluated will actually be used. When the data is not used it ceases to be an optimization and can even result in incorrect behavior (unnecessary exceptions / non-termination). Either way, making eager evaluation the default is a form of premature optimization.

PopsiclePete8 anni fa

What makes lazyness difficult for me isn't the mental model, it's the fact that when you get an error, the error is far from its call-site, and the truly helpful information that I could've had in the stack trace is gone. I'm looking at the error - but which lazy operation in the sequence caused it? Where? When?

I by no means claim to be a Clojure guru, and I'm sure people who have more experience can just look at it and know, but for me, the time lost to pinpoint that error is too much. And when I have something in production, I need to be able to pinpoint the error in minutes and deploy the fix quickly.

>Either way, making eager evaluation the default is a form of premature optimization.

I don't see how that's true - it's the default way CPU's operate on the silicon level. Millions upon millions of lines of non-lazy code have been written and put into production and they perform real, valuable work.

I'm not against lazy evaluation or against Clojure - all I'm saying is - it's hard for me to reason about it when things go wrong.

If they don't go wrong (almost never for me) - it truly is a thing of beauty.

nybble418 anni fa

> when you get an error, the error is far from its call-site, and the truly helpful information that I could've had in the stack trace is gone

I'll grant you that one. Languages with non-strict evaluation (or tail calls, or inlining, or extensive use of macros) rarely have useful stack traces. You can see what was being evaluated but not necessarily how it got there. The ability to leverage the stack as a debugging aid is a feature of certain runtime conventions more applicable to imperative programming paradigms than to functional ones. In modern Haskell there is a way for functions to receive extra information about their call site as an implicit parameter, which helps with error reporting. I'm not sure what tools Clojure might offer along these lines.

I will point out that tracing the source of incorrect or corrupted data is no less of a problem for programs written in strict, imperative languages. Functional programming adds to this by emphasising functions as a form of data, and non-strict evaluation carries that still further by treating all data as closures. On the other hand, strong static typing limits the kinds of runtime errors which lay hidden within those closures to a manageable set (assuming a pure functional language like Haskell).

>> Either way, making eager evaluation the default is a form of premature optimization.

> I don't see how that's true - it's the default way CPU's operate on the silicon level.

That's not really true, given the existence of speculative and out-of-order execution. In any case, how the CPU operates at the silicon level has very little to do with whether a high-level language optimization is premature.

> Millions upon millions of lines of non-lazy code have been written and put into production and they perform real, valuable work.

I'm not disputing that, but it's beside the point. Code which has been prematurely optimized can still "perform real, valuable work". It just may take a bit longer to run or require more resources to develop (for example by being harder to reuse, requiring more code to be written to accomplish the same result).

draw_down8 anni fa

tosh8 anni fa

via tonsky: https://twitter.com/nikitonsky/status/1057317735766593536

  - Migrate as you go
  - 1/10th the code that does 10× as much
  - Life with existing Java codebase is better with Clojure macros 
  - Eliminate boilerplate with custom DSLs
  - REPL to live servers

mikeflynn8 anni fa

We have used Clojure in our data processing and API code for five years now at Studio71, and it's been a huge success for us. We have much smaller dev teams than our competitors so Clojure has helped us keep pace with a smaller code base and fast data processing, but it has been a challenge to hire new developers when we need one, but with patience it has always worked out.

auganov8 anni fa

This talk goes into detail how Clojure plays out in larger teams, particularly how they worked to maintain REPLability for a huge monolith: https://www.youtube.com/watch?v=BThkk5zv0DE

That said, I think a lot of the preoccupation with rewrites is just premature optimization. Just a while ago I noticed some open source Clojure code of a startup acquired by FB got rewritten in Haskell. Was very glad to see founders don't seem to be working on this new code-base. Probably moved on to do higher value things.

truculent8 anni fa

FWIW I think this is the startup code referenced: https://github.com/wit-ai/duckling_old

auganov8 anni fa

Yep, that's the one

TheMagicHorsey8 anni fa

I love Lisp, so I thought Clojure would be a great productivity booster. And it is, if you are a one-man or one-woman shop. But try to build a team around a Clojure project and it's another story. The language basically begs you to make "magic" happen with domain-specific constructs. That might make you feel powerful as a programmer, but it's also a nightmare for new team members to get up to speed on.

And then, as others have mentioned, it's hard to find Lisp programmers. Most programmers are turned off by the syntax and/or immutability. I happened to take a Lisp class as an undergrad, but I would otherwise probably be in the same boat.

jimbokun8 anni fa

"Most programmers are turned off by the syntax and/or immutability."

Programmers who don't see the value of immutability or can't understand it are not programmers you really want to work with.

jcadam8 anni fa

> Programmers who don't see the value of immutability or can't understand it are not programmers you really want to work with.

I'm not disagreeing with you, but since you just described easily 90%+ of all working programmers, it's kind of unavoidable.

One place I worked, one of the lead engineers knew I was into functional programming and asked me if I thought it would be a good idea to introduce Scala or Clojure into our pure Java code base. I said "No. The majority of the programmers here couldn't make the adjustment."

Put another way, if I could assemble a hand-picked team of all of the engineers I would rate as "great" who I've worked with over my 12+ year career thus far -- I'd have a team of about 7 people.

TheMagicHorsey8 anni fa

I know plenty of productive programmers who aren't even computer science majors, and just work in Javascript and Python.

And I also know people who code in Rust and Haskell, and can never meet a deadline.

Theoretical chops doesn't make a good programmer. Knowing that immutability is a good thing and abandoning the imperative style, or even being comfortable with a functional style, isn't a prerequisite to being productive in most organizations.

kbp8 anni fa

> And then, as others have mentioned, it's hard to find Lisp programmers. Most programmers are turned off by the [...] immutability.

Most Lisps have very little in the way of immutability (except for C-style "undefined behaviour if you modify it"). They tend to be very dynamic, living systems, and making everything mutable is one way of enabling that. Some people favour a functional style, and a functional style is more common in some dialects like Scheme, but it's almost always essentially "imperative code with good support for higher order functions," and most Lisp programs have always mutated state freely.

There are exceptions, but they are exceptions.

iLemming8 anni fa

It's a known fact that it's possible to build and maintain serious projects having only a few Clojure developers. And either because of that or for some other reason, it's been decided that it's hard to build large codebases in Clojure. That's not true though. I personally worked in a large, distributed team on quite large Clojure projects.

rb8088 anni fa

Is Clojure still a thing? It seems to have fallen off my radar. Java seems resurgent after 8, even Scala seems to have lost momentum.

mschaef8 anni fa

Yes. It's still a thing... but I do think Clojure has lost mindshare. Just some impressions:

* It's natural for the development of a language to slow down after an initial period of churn. So Clojure has reached a certain level of maturity where there's less 'new cool stuff'.

* There are other areas of industry where there's more 'new cool stuff' and attention naturally shifts there.

* The Honeymoon period is over, and anybody that had the unreasonable expectations the Clojure was the magical solution to all their problems has been disappointed and probably moved on. (Don't laugh. This happens.)

* People are better seeing other potential costs to Clojure code bases. I know of at least one organization looking into moving away from Clojure for the purpose of having a more attractive portfolio of assets to potential outside investors.

* My understanding (hearsay at best) is that Rich initially self-funded Clojure development for a couple years, and then got understandably tired of taking the hit and shifted his primary attentions to more commercial endeavors.

* As Java evolves >=8, it's gained features that incorporate some of the benefits of Clojure. Java is clearly not turning into Clojure, but it is getting development such that it's nibbling away at Clojure's technical advantages a bit.

None of this should take away from the fact that Clojure is a well-designed and very pragmatic language for the JVM. I personally love writing code in the language and find it the most effective way to use my time. What I hope the above means it that the language is settling into a more long-term sustainable niche.

For context: I use Clojure in several personal projects, and have done so for a while. I've also slipstreamed it into a commercial app, for automating management tasks, etc. Overall, I've been very happy with the platform, although there are still some startling gaps ( https://www.reddit.com/r/Clojure/comments/9p4tto/best_way_to... )

the_af8 anni fa

> I know of at least one organization looking into moving away from Clojure for the purpose of having a more attractive portfolio of assets to potential outside investors.

Out of curiosity, without naming the organization, can you elaborate on this? I don't understand what kind of portfolio you're talking about here, the codebase? Why would investors care whether it's done in $LANG? I thought they cared more about the business and business-related concerns. Maybe they perceive Clojure as being an impediment for fast delivery of features?

mschaef8 anni fa

> Why would investors care whether it's done in $LANG? I thought they cared more about the business and business-related concerns.

I think the concern is around how hard long term sustainability of the codebase. If it's built in a relatively minor tool, there's merit to the belief that it'll be hard to find people that know it well. (And this is discounting the possibility that a dynamically typed language might be less suitable than programming in the large than something that has more static checking.)

When I was 23 and fresh out of school, I'd probably called this line of reasoning ill-informed at best... after all, a good developer can learn any language, and you'd only want to hire _good_ developers anyway. Twenty years later, I'm a lot more sympathetic. If you're trying to deliver business value under time/budget/risk constraints, adding a known, short-term risk (a minority language) for an unknown, longer-term benefit is a hard sell.

I think the Lisp-like languages are particularly prone to this in that they derive so much of their power by providing lots of mechanisms for abstraction. (Higher-order functions, syntactic macros, multi-methods, etc.) If you have the right abstractions in your code it can be great, but getting to that point and maintaining it can be expensive. (And in the meantime, you also have a great deal of power for shooting yourself in the foot.)

My general sense of languages like Clojure is they are very good for people with experience and time to very carefully consider their problem space. However, that's not necessarily a great description of the modern IT profession.

the_af8 anni fa

Thanks for the answer. What you say makes a lot of sense to me. I prefer functional but statically typed languages for many of the same reasons.

I just find it surprising that investors care about this :)

agumonkey8 anni fa

It's a lot less trendy (no more R.Hickey church talks these days), but that's about it. Clojure is still lively, they're pushing a new release soon, they're exploring spec for contracts, typing, and a lot of other things it seems. There's also new nRepl.

systems8 anni fa

to talk about something new, you need to have something new to talk about

i dont think that scala or clojure have lost momentum i think both clojure and scala, stopped adding new features that are worthy of blogging about

you cant just add feature, just to keep people talking about you

clojure, is what it is, a nice language, a very smart leader (rick hickey), and a nice vibrant community (check #clojure on twitter)

the most talked about language nowadays is actually c++, and i think this mainly because they suddenly decided to have a very ambitious roadmap, and keep adding features, so there is always something new to talk about

does this make c++ better than clojure, well, i dont think they are even comparable, but it just make c++ more talked about than clojure

kamaal8 anni fa

>>i dont think that scala or clojure have lost momentum i think both clojure and scala, stopped adding new features that are worthy of blogging about

More like Clojure is in Cathedral mode of open source development. Most of the development is in private discussions, implementation and prototyping at Cognitect, and once they have something to share, they release it to the world.

That's more a communication and marketing thing. For example, you can try Googling but you won't be able to see anything on the Clojure roadmap. No one knows what's the plan even year or two from now.

But thanks to smart leadership, and brilliant development team they have, its mostly a non issue if you don't really care about the everyday minutiae. You can bet the top dollar that they are working on something good, and it will be eventually released to the world.

To people coming from other open source tools, that can sound as strange.

puredanger8 anni fa

https://clojureverse.org/t/roadmap-to-1-10/1581/8?u=alexmill... is a pretty good overview of where Clojure has been going and will continue to go.

arximboldi8 anni fa

Interesting to hear about C++ in this context, since of my goals is to make C++ more like Clojure:

  https://www.youtube.com/watch?v=sPhpelUfu8Q
  https://www.youtube.com/watch?v=vohGJjGxtJQ

There is a lot of I could say about the relationship between C++ and Clojure, but I'll leave it at that for now :-)

mattdeboard8 anni fa

Very well said!

[deleted]8 anni fa

finalfantasia8 anni fa

As the leader in online affiliate marketing, we’ve just launched our analytics and insights platform that is written in Clojure [0]. If you like Clojure, come work with us [1].

[0] http://junction.cj.com/article/introducing-insights-cj-affil...

[1] https://engineering.cj.com

vbuwivbiu8 anni fa

amazed to see there are people who find

myObject.myMethod(x, y, z);

easier to read than

(my-function x y z)

- that's 2 delimiters in Clojure vs 6 in your C-style language

zeveb8 anni fa

In fairness, it'd actually be:

    (my-method my-object x y z)

But yes, the Lisp version is so much clearer to me. I just don't get why it's not more popular.

trevor-e8 anni fa

How does this work with autocomplete if you need to know the method name before the object you're working with? Maybe I'm not getting something but that sounds incredibly painful to use in practice.

jimbokun8 anni fa

Because idiomatic Clojure code is based on functions and not methods.

The method call syntax is just there to support interop with Java.

zeveb8 anni fa

In Common Lisp, it works because symbols are scoped by package, not by object. So if you're in the package foo and start typing 'bar,' then your editor would autocomplete symbols in the foo package starting with 'bar'; likewise if you typed 'baz:quux' your editor autocomplete would autocomplete symbols in the baz package starting with 'quux.'

Common Lisp most definitely doesn't have one namespace; this is really nice in practice.

trevor-e8 anni fa

OK so my confusion was from thinking Clojure is statically typed, which apparently is not true. I was basically asking the same question as https://www.reddit.com/r/Clojure/comments/6f6cq2/how_to_comp...

vbuwivbiu8 anni fa

or

(object-as-namespace/my-function x y z)

still fewer delimiters and less noise

davnicwil8 anni fa

To me the delimiters help with readability by breaking up the statement into its component parts in a visual way that I can immediately parse.

Before even reading the variable names I understand the role of each in the statement.

vbuwivbiu8 anni fa

you only used 2 delimiters to say that

kazinator8 anni fa

TXR Lisp:

  myobject.(mymethod x y.w z.bar)

Notations can be useful. Most Lispers reach for 'expr rather than (quote expr). Rational numbers could look like #R(1 2) but CL has 1/2 instead.

I decided on a dot notation for object access because OOP is used to organize the program as a whole; so it's not some small thing like arithmetic.

The dot notation integrates into Lisp; it doesn't disturb the syntax with ambiguities and reads/prints cleanly, and corresponds to structure in several straightforward ways:

   a.b.c -> (qref a b c)
  .a.b.c -> (uref a b c)

There must not be whitespace: a . b is the consing dot, which can only appear at the end of the list. Of course, if a and b are 3 and 2, we get 3.2 which is a floating-point token; and that mustn't be glued to something else.

  1> '(qref a b c)
  a.b.c
  2> '(qref a b (qref c))
  (qref a b (qref c))
  3> '(qref a (qref b c))
  (qref a b.c)
  4> 3.2.a
  expr-4:1: trailing junk in floating-point literal: 3.2.a
  ** syntax error
  5> '(qref a (b c) d (e) (f g))
  a.(b c).d.(e).(f g)

There are qref and uref macros which do useful things with the notation. E.g.

  14> (stat "args.h")
  #S(stat dev 2049 ino 670288 mode 33204 nlink 1 uid 500 gid 500 rdev 0
          size 5628 blksize 4096 blocks 16 atime 1540848391 mtime 1524635813
          ctime 1524635813 path "args.h")
  15> (mapcar [chain stat [juxt .path .size]] (glob "*.h"))
  (("args.h" 5628) ("arith.h" 2270) ("buf.h" 3712) ("cadr.h" 2515)
   ("combi.h" 1517) ("config.h" 2644) ("debug.h" 3762) ("dict.h" 4373)
   ("eval.h" 3898) ("ffi.h" 5146) ("filter.h" 2298) ("ftw.h" 1492)
   ("gc.h" 2370) ("glob.h" 1485) ("hash.h" 3412) ("itypes.h" 3238)
   ("lib.h" 37442) ("lisplib.h" 1680) ("match.h" 2031) ("parser.h" 4597)
   ("rand.h" 1798) ("regex.h" 3204) ("signal.h" 7954) ("socket.h" 1460)
   ("stream.h" 8553) ("struct.h" 3475) ("strudel.h" 1486) ("sysif.h" 2747)
   ("syslog.h" 1998) ("termios.h" 1436) ("txr.h" 1885) ("unwind.h" 9880)
   ("utf8.h" 2590) ("vm.h" 1681) ("vmop-ORIG.h" 1660) ("vmop.h" 2008)
   ("y.tab.h" 4542))

Example OOP code that is full of the notation is found in the compiler: http://www.kylheku.com/cgit/txr/tree/share/txr/stdlib/compil...

I feel I have come up with a successful design: a way of integrating the dot selection notation into a Lisp dialect without losing "Lispiness".

666lumberjack8 anni fa

Interesting stuff. I'd be curious to know how Netflix are getting on with it now.

dr_teh8 anni fa

I love Clojure! It's honestly the best language ever.

[deleted]8 anni fa

truculent8 anni fa

I always see loads of Static vs Dynamic and Functional vs OO opinions breaking out in the comments. While the discussion is often interesting and insightful, it would be nice to see some empirical analysis of the trade-offs here. Stuff like development time, error rates, maintainability (no idea how you'd measure this one...).

Anyone know if such studies exist? Feels like they should be out there given the amount of discussion the topics generate

yogthos8 anni fa

There is some research available on the topic. There's a good summary of it here https://danluu.com/empirical-pl/

truculent8 anni fa

Lovely, thank you!

jimbokun8 anni fa

I would like to know the words spoken for this slide:

https://speakerdeck.com/daveray/clojure-at-netflix?slide=29

systems8 anni fa

is clojure still used at netflix [2018]?

choot8 anni fa

One strategy which i learnt from a tech company executive is that if you want to hire best of best at low cost, develop products in interesting new programming languages as the most curious programmers who are very serious about programming care less about money and more about work in a particular language of their interest.

No wonder the company had successful projects in every asoteric languages.

Now, I come from executive background in corporate job where going with anything other than Java is considered risky. This came off as suprise to me.

lemming8 anni fa

...if you want to hire best of best at low cost, develop products in interesting new programming languages as the most curious programmers who are very serious about programming care less about money and more about work in a particular language of their interest.

The problem is that the particular language of their interest probably changes every 6 months.

narwally8 anni fa

>The problem is that the particular language of their interest probably changes every 6 months.

As a language nerd, one of the nice things about Lisp is that the language is easily extended into other paradigms. I've stuck with Clojure for a long time because of this. It had Go style CSP-concurrency as the core.async library within a few years of release. I can use types by just loading core.typed, or I can play around with logic programming with core.logic. Whenever I find a cool feature in another language, I usually find out that it's already available in Clojure as a library.

Royalaid8 anni fa

It's all tradeoffs, all the way down. The good thing about Java is most problems are solved or have easy paths forward by dipping into the massive development talent pool.