Julia 1.11 Highlights(julialang.org)
julialang.org
Julia 1.11 Highlights
https://julialang.org/blog/2024/10/julia-1.11-highlights/index.html
22 comments
> I also feel that Julia managed to achieve "forward interoperability" between libraries that is almost unparalleled in my experience: It is often possible to just pass data structures across library boundarie
Never coded in Julia, how does this work?
Never coded in Julia, how does this work?
Caveat: This is in my view not just a technical achievement, it also works because some library authors play nice. And because things where standardized from the get-go, instead of evolving organically/provided much later by users.
I think the main thing that Julia gets right is that there are de-facto standardized interfaces for a lot of things that are actually followed/used.
E.g. in C++ on the other hand, you have a bunch of libraries that bring their own primitives, like all the Matrix classes-- their is no interface for those, and even if their was, it would be a pita to write C++ code that was able to work with a bunch of difference primitives here. So the problem is not only that it would be a lot of work to write C++ code to be "primitive-agnostic" (=> you would basically have to soak your codebase in templates), there are not even common expectations for you to build upon (like: does a matrix class provide rowncnt()? or rowCount()? or size(1)?, or nrow()? => if there is no common ground not even extensive template magic is gonna save your day).
Julia also makes it super easy to write type-agnostic code in general ("dynamically typed"), which is simply not the default and/or extra effort in many other languages.
I think the main thing that Julia gets right is that there are de-facto standardized interfaces for a lot of things that are actually followed/used.
E.g. in C++ on the other hand, you have a bunch of libraries that bring their own primitives, like all the Matrix classes-- their is no interface for those, and even if their was, it would be a pita to write C++ code that was able to work with a bunch of difference primitives here. So the problem is not only that it would be a lot of work to write C++ code to be "primitive-agnostic" (=> you would basically have to soak your codebase in templates), there are not even common expectations for you to build upon (like: does a matrix class provide rowncnt()? or rowCount()? or size(1)?, or nrow()? => if there is no common ground not even extensive template magic is gonna save your day).
Julia also makes it super easy to write type-agnostic code in general ("dynamically typed"), which is simply not the default and/or extra effort in many other languages.
There is a talk on how this is possible due to multiple dispatch: https://www.youtube.com/watch?v=kc9HwsxE1OY
Perl started that with CPAN.
Julia seems to have fallen out of the mindshare completely in my industry (finance). It's a huge shame because I really like the language and what they are trying to do.
Interchangeable cookie-cutter coder availability is Finance's number one priority. Definitely no room for critical pricers or infra written in languages that they can't frictionlessly slot someone else into as people leave. So python.
Also, arguably, Julia, while fantastic, just didn't do that much that Python didn't do already. It's main argument outside of tidier ergonomics was basically "speed without leaving Julia" but with Numpy and Pandas being essentially stdlib, that just wasn't a very powerful argument. Julia was basically too incremental to be worth switching to. It seems to have found its niche elsewhere though with the optimization people?
Also, arguably, Julia, while fantastic, just didn't do that much that Python didn't do already. It's main argument outside of tidier ergonomics was basically "speed without leaving Julia" but with Numpy and Pandas being essentially stdlib, that just wasn't a very powerful argument. Julia was basically too incremental to be worth switching to. It seems to have found its niche elsewhere though with the optimization people?
One of the main things I like about Julia is that all the libraries are just built on their core Array type. In Python, there’s torch tensors, numpy arrays, and I think Jax has their own array type too. Also, Julia has a really beautiful distributed computing paradigm, whereas python needs to use libraries like Ray, which have their own quirks and documentation and community
General scientific computing is pretty good across the Julia ecosystem, from optimization, to ODE and now PDE solver libraries, to various statistics and inference packages, etc. It lacks the deep NN tooling or breadth of ML libraries of Python, and nothing matches R for breadth of stats libraries, but for most other scientific computing it is really great at this point.
I am by no means an expert, but I used Flux.jl for a convolutional neural net in electromagnetics for my latest paper and it was such a breath of fresh air compared to Python and PyTorch. (I'm an EE and not great at programming, so I found a lot of frustration in PyTorch). Even though the Keras library in Python is pretty nice, even then I got myself into some odd pickles when trying to do some custom layers which used FFT processing as it relates to gradient computation. Things are much smoother in Julia, and that doesn't even count how much easier the Plots library is! I'm ashamed to admit that I have no idea how to manipulate the figures and axes in Matplotlib without extensive googling.
I am really waiting for the day Julia ML ecosystem improves and I can jump ship to it.
Scientific/Numeric/Data-Python is essentially a DSL around C-API which creates friction (try for example to map a custom function over a Pandas column). Whereas in Julia, it's just Julia. It's liberating to just extend and use a library written in the same language. It leads to surprising synergies.
numpy is only fast if the computation does not escape it. There are plenty of cases where execution ping-pongs (if that's a verb) between python and the C(++) wrapper numpy actually is. Then everything becomes quite slow.
Anyway, I see data scientists and statisticians (at least 100% of the ones I know) completely ignoring Julia, just because they only have been exposed to Python and R in their education. The quality of the programming language/ecosystem seems to be irrelevant.
Anyway, I see data scientists and statisticians (at least 100% of the ones I know) completely ignoring Julia, just because they only have been exposed to Python and R in their education. The quality of the programming language/ecosystem seems to be irrelevant.
I’m trying to influence this from my small corner of the world here: https://juliaactuary.org/
Come chat in the Julia slack #finance or #actuarial channels!
Come chat in the Julia slack #finance or #actuarial channels!
Is Julia at the level of stability to be useable by large projects? I've tried to follow and it always looked promising but there were a lot of reports about things being subtly broken in hard-to-debug ways . Anyone have more recent experience?
I did a bunch of contract work last year at a company that was all-in on Julia and it was a really pleasant experience.
IMO one of the issues with Julia is that it’s easy to get nerd-sniped trying to do clever things with the type system and to make as much of your code as possible statically-inferable. Code and libraries that rely heavily on type dispatch ends up throwing MethodErrors deep into the call stack, far away from your code, which makes it harder to debug them.
More mature Julia developers tend to keep things simpler, and make better use of dynamic types instead of contorting to treat it like a statically-typed language.
IMO one of the issues with Julia is that it’s easy to get nerd-sniped trying to do clever things with the type system and to make as much of your code as possible statically-inferable. Code and libraries that rely heavily on type dispatch ends up throwing MethodErrors deep into the call stack, far away from your code, which makes it harder to debug them.
More mature Julia developers tend to keep things simpler, and make better use of dynamic types instead of contorting to treat it like a statically-typed language.
Have been writing all my numerical code in Julia for the last 1.5 years, haven’t run into any issues at all.
Actively use things like Trixi.jl (CFD framework), Jump/IPOPT/PRIMA for numerical optimisation, OrdinaryDifferentialEquations for ODE solutions; a few colleagues actively use Gridap.jl for their work.
Not sure how large-scale these libraries can be considered, but they all seem to work fine, fast, and stable (potential issues with LoopVectorization.jl in upcoming versions of Julia are somewhat concerning, but IIRC, for 1.11 this has been resolved)
Julia’s been stable for my work, from what I understand, things have gotten better over the years on that front. The biggest issue Julia gives me is packaging, just because two libraries that you import can use different versions of the same dependency (among other things)
Stable enough for these folks, https://info.juliahub.com/case-studies
I think it is better than R/Matlab/.... combined.
[deleted]
I also feel that Julia managed to achieve "forward interoperability" between libraries that is almost unparalleled in my experience: It is often possible to just pass data structures across library boundaries, and in MANY other languages this is absolutely not the case; consider e.g. C++, where you might have like five different "Matrix" classes/types between GUI library, linear algebra package, image processing toolkit etc., and the code you write has to convert those types at every boundary by hand.
The one thing the language is bad at (but this also improved a lot over time!) is the suitability as conventional scripting language, where you run some source code through a cold-started interpreter (you get somewhat railroaded into having an open interpreter instance, that you then run your scripts on instead).