Jax is composable. In fact it’s a core design goal. Jax arrays implement the Numpy API. I routinely drop Jax arrays into other python libraries designed for Numpy. It works quite well. It’s not effortless 100% of the time but no library interop is (including Julia multiple dispatch).
I can introspect Jax. Until you wrap your function with jit(foo) it’s as introspectable as any other Python code, at least if I’m understanding what you mean by introspection.
Jax has implemented most of the Numpy functions, certainly most of the ones anyone needs to use on a regular basis. I rarely find anything missing. And if it is, I can write it myself, in python, and have it work seamlessly with the rest of Jax (autodiff, jit, etc)
Thanks Viral. To be clear, I’m a python user who’s cheering for Julia, because I live the problems of python and do see the potential of Julia as a better path. But unfortunately I’m not prepared to be the early adopter (at least in my day job), and will wait until other, braver users have sanded off the rough edges. God speed and good luck.
> 2. I don't entirely follow this point. Perhaps using PyArrow's parser would be faster than what is timed here, but is that what the typical Python data science user would do?
I am a Python data science user. If data gets big enough such that loading time is a bottleneck, I use parquet files instead of CSV, and PyArrow to load them into pandas. It’s a one line change. The creator of Pandas is now leading the Arrow project. It’s very seamless. Don’t know if I’m typical but that’s me.
I don’t really care much about syntax choices, but my small complaint about “end” is that it takes up a line which reduces the amount of business-logic code I can fit on one screen, especially if you ever get into lots of nested loops and conditionals.
Thanks. I watched the JuliaCon state of Julia presentation. As I wrote in my original post, I appreciate the investments the Julia core developers are making, that have improved but not eliminated this problem. I wish them luck.
I’m cautiously optimistic that JAX (or something like JAX) can save the python programming language from stagnation by essentially building a feature-complete reimplementation of the language with JIT and autograd baked into the core. I’m praying that Google diverts like 10% of TF’s budget to JAX.
That way I don’t have to learn to love a bunch of unnecessary semi colons and “end”s littering up my beautiful zero-indexed code ;-)
Let me second GP’s sentiment. I find Julia really slow for my purposes. I don’t know his reasoning, but I will explain mine. None of this is surprising and is oft discussed.
Julia (at least by default) is constantly recompiling everything. This is a huge pain in a REPL style setup where you want to tweak one thing and see the changes, again and again. I know the Julia ecosystem is working on better caching etc to fix this problem but it’s a problem.
Also, despite the marketing claims around the language, expertly crafted C usually beats Julia in performance. So if your “python” program is spending most of its time in Numpy/PyTorch/etc, it will beat Julia, unless you’re writing a fancy “put a differential equation in a neural network in a Bayesian Monte Carlo” program that benefits from cross compiling across specialized libraries.
Finally, the Julia libraries are just not as mature as python’s. Armies of developers and larger armies of users have battle tested and perfected python’s crown jewel libraries over many years. Often when someone posts a bad benchmark to the Julia forums they can “fix” it in the library implementation, proving the correctness of the theoretical case for Julia. But in reality many such problems remain to be fixed.
Julia is really cool and does have many inherent advantages over python. But it’s not the silver bullet many of its proponents suggest it to be. At least not yet. Every few years I check out Julia and I hope one day it does become that perfect language. I think it will. I just fear it will take longer than many others hope.
FWIW I agree with you. I’ve always found cython easier than Numba. And more performant.
I think Numba has a lot of potential and will improve as they fill out remaining language coverage and finalize the API. The idea of a LLVM JIT compiler for python makes a ton of sense.
In the uncommon event I need to write a loop from scratch, and I need it to be really fast, I just rewrite that one jupyter cell in cython or numba. But that is a small piece of my codebase.
I agree that Julia code is aesthetically superior to a long chain of Pandas code. But at this point I’m used to reading a bunch of chained pandas code. Often I think of myself as more of a Pandas programmer than a Python programmer.
Honest question from a heavy python user who would switch if it made sense:
Are there any comprehensive benchmarks that show Julia outperforming Pandas or PyTorch or SciKit?
Obviously pure Python is terrible. But the library algorithms written in C seem fairly competitive.
I’m a fairly boring user who doesn’t do new science, and is fine just composing existing boring algorithms to solve problems in my subject matter domain.
I’m fascinated by Julia and have test driven it before but it didn’t click for me. Maybe I was doing it wrong and/or the ecosystem has matured since I last looked.
I guess I generally do like the pythonic paradigm of an interpreted glue language orchestrating precompiled functions written in other languages. I don’t need or want to compile the entire pipeline end to end after every edit, that slows my development iteration cycle times.
I just want to write my own fast compiled functions to insert into the pipeline on the rare occasions I need something bespoke that doesn’t already exist in the extended python ecosystem. It seems like a lower level language would be optimal for that?
It’s honestly exhausting arguing with all you Julia boosters. You can down vote me to hell, I don’t care. I’m done engaging with this community.
You all are not winning over any market share from Python with your dismissive, arrogant, closed minded culture.