JNumPy: Writing high-performance C extensions for Python in minutes(github.com)
github.com
JNumPy: Writing high-performance C extensions for Python in minutes
https://github.com/Suzhou-Tongyuan/jnumpy
65 comments
> Also why should I pick this over cython, pythran or numba?
One scenario would be that there is code already written in Julia, which you would like to expose to a broader audience. Although overall Julia's ecosystem is smaller than Python's, there are some niches within which it is highly developed. Library code accessible from the alternatives you listed is mainly limited to existing Python and C code. See the demo with `ParallelKMeans.jl`, which wraps a high performance Julia library performing K-means clustering, which could be a useful addition to the Python ecosystem.
Another scenario is that you want to write some custom code which makes use of the Julia ecosystem and then expose it to Python.
One scenario would be that there is code already written in Julia, which you would like to expose to a broader audience. Although overall Julia's ecosystem is smaller than Python's, there are some niches within which it is highly developed. Library code accessible from the alternatives you listed is mainly limited to existing Python and C code. See the demo with `ParallelKMeans.jl`, which wraps a high performance Julia library performing K-means clustering, which could be a useful addition to the Python ecosystem.
Another scenario is that you want to write some custom code which makes use of the Julia ecosystem and then expose it to Python.
Title says “C extension” which is accurate, not “extension written in C”. You can write C extensions for CPython in non-C languages with a bridge, e.g. PyO3 for Rust, but the extension ultimately has to use the C API and ABI.
This is why it's a C extension: https://github.com/Suzhou-Tongyuan/jnumpy/blob/main/TyPython...
The title is correct because it is a Julia numpy interface. Inside it seems is another project, TyPython, that provides an efficient Julia-Python-Numpy bridge, via C extensions.
The title is correct because it is a Julia numpy interface. Inside it seems is another project, TyPython, that provides an efficient Julia-Python-Numpy bridge, via C extensions.
I would argue that it might technically be correct (and admittedly that's the best type of correct) in the strict sense, it is also misleading. If I read that I'm writing C extensions in minutes I at least expect that the extension compiles to C (which the Julia code doesn't).
Definitely misleading given all the bloat and baggage adding Julia as a dependency to a project can have. It's the heaviest glue ever.
Agreed that the title is technically accurate. Julia should also be in the title, but this is possibly an oversight. It seems a bit unnecessary for the GP to assume that this is a bad faith attempt to mislead and insert Julia where it is not welcome rather than a potentially useful extra tool for writing high performance code. Note that the author/poster is probably not a native English speaker.
I wish they would stop overselling too. There are so many problems with trying to adopt it for real projects it isn't even funny... Good for researchers sometimes, but even then... It's got issues. Pretty toxic community too.
> There are so many problems with trying to adopt it for real projects it isn't even funny... Good for researchers sometimes, but even then... It's got issues.
Can you expand on that, so that it's possible to improve on them?
Can you expand on that, so that it's possible to improve on them?
It seems to use a lot of memory which can be unacceptable for many people and can become impractical if you plan on running several programs simultaneously. For several of the problems at https://benchmarksgame-team.pages.debian.net/benchmarksgame/... on the Computer Language Benchmarks Games, it shows the Julia programs as using several times more memory than similar Java programs. It's even worse when comparing to C, C++, Rust, etc...
Remember that for those benchmarks, C, C++, and Rust compile the binaries and then run the benchmark, while Julia compiles as part of the benchmark. The memory usage is dramatically decreased if you do a workflow similar to the other languages using something like PackageCompiler.jl to build a binary that is benchmarked. If you treat the other languages as "JIT" and include those factors in the benchmarking process, it's a lot closer.
Thanks, I hadn't noticed that Julia was also doing compilation during the benchmarking and I'm curious if it would be hard to get those benchmarks to be done with PackageCompiler.jl. With that being said, the memory usage still seems very high. For many of the problems, Julia is often using the second most memory of all the programming language implementations being benchmarked and several of those other programming languages implementations (Node.js, OpenJDK, .NET, etc...) also are doing some sort of compilation during the benchmarking as well. I also tried compiling several of the C programs with GCC and GCC usually only used ~30 MB of additional memory so even if you add that to the the memory usage of the C programs, it's still much less than the Julia programs. Using PackageCompiler.jl does look promising but that does add another step which looks to be slightly more involved (looks a bit comparable to using profile-guided optimization) than compilation steps for other programming languages and some could see that as another problem for using Julia in real projects.
Yes, the size in memory is definitely an issue. Do keep in mind that right now, the julia executable more or less just has to load all of LLVM as a shared object into memory, since we might compile stuff at any time. That alone adds ~84M of overhead on my machine for the current libLLVM14 shared object. There's probably some other stuff I haven't thought of OTOH. The (non-minified) runtime itself is 244K at the moment, which can probably be slimmed down further. There are currently efforts underway to statically compile more stuff and hopefully introduce a more traditional static linking approach, but it takes time & lots of effort.
Still, a large chunk of the memory being spent is not in the code that's produced or in the allocations happening in the code, but libraries that are loaded but not required. It's one of the downsides of focusing on interactivity first. The benchmarks in benchmarksgame don't reflect that, which I guess is up to interpretation/what's required - if the total amount of memory is a concern it's an important figure, if you only care about your core algorithm not having inherent allocation/memory problems, you probably won't care about the compiler chain as much.
Still, a large chunk of the memory being spent is not in the code that's produced or in the allocations happening in the code, but libraries that are loaded but not required. It's one of the downsides of focusing on interactivity first. The benchmarks in benchmarksgame don't reflect that, which I guess is up to interpretation/what's required - if the total amount of memory is a concern it's an important figure, if you only care about your core algorithm not having inherent allocation/memory problems, you probably won't care about the compiler chain as much.
In the past the benchmarkgame people haven't let us use PackageCompiler. The big memory gain would be that you could skip loading things like LinearAlgebra and the other standard libraries that aren't being used (but which you are still loading code for).
I think some more effort should be made in getting the default memory usage down to a more reasonable level. This site https://programming-language-benchmarks.vercel.app/problem/h... shows a simple Hello World Julia program as using 169 MB of memory (and I saw similar memory usage on my computer running '/usr/bin/time julia -e 'println("Hello World!")'') which was the third worse of all the programming language implementations that were tested.
Oh definitely agreed. A lot of this is because Julia always loads BLAS and LLVM, so it's like doing `import numpy; import numba; print("Hello World")` and then noticing most of the startup time is loading numpy and numba. But it's a hard question because if BLAS isn't always loaded with Julia, its previously core scientific computing case is impacted. As it has been growing to be more widely used in a general purpose case, these kinds of assumptions are having to be revisited.
One can of course remove this in the compilation stage of PackageCompiler because PackageCompiler builds a new system image, and where BLAS is loaded is in the system image, so you can create a new from-scratch system image that is more lean. However, the tooling isn't quite there yet: right now the main way that's documented is something that extends the default system image, hence the large binaries. There's StaticCompiler.jl which does tree shaking so it makes small binaries, but it doesn't support most of the Julia runtime right now so it's limited in the codes it can handle. So right now the foundation all exists and it's at a usable state but definitely needs to improve.
One can of course remove this in the compilation stage of PackageCompiler because PackageCompiler builds a new system image, and where BLAS is loaded is in the system image, so you can create a new from-scratch system image that is more lean. However, the tooling isn't quite there yet: right now the main way that's documented is something that extends the default system image, hence the large binaries. There's StaticCompiler.jl which does tree shaking so it makes small binaries, but it doesn't support most of the Julia runtime right now so it's limited in the codes it can handle. So right now the foundation all exists and it's at a usable state but definitely needs to improve.
Absolutely! This isn't high priority but there is some very low hanging fruit. It's not that reducible though since llvm is 84 of those megs and they have to go into memory if you want to be able to generate new code.
While this is true, it's also true that Julia currently brings more into ram than it should (and brings stuff in earlier than it should). The best example of this is that we currently allocate buffers ahead of time for matrix multiplication temporaries (even if matrix multiplication is never performed in the program). We do need to fix that type of thing eventually.
> Pretty toxic community too.
Quite the opposite according to my own experience.
Quite the opposite according to my own experience.
Do you have any source on well written julia being any slower than well written C?
I did a comparison of Julia vs numpy, cython and pythran [1] some time ago, for a typical dsp routine we use in our work, and Julia was quite a bit slower than the alternatives. Now I'm by no means a Julia expert so I might have missed an optimisation opportunity (although I posted this and nobody could point to something obvious) , however the whole advertisement behind Julia is that one gets essentially C speed with the ease of Python. That's actually my main gripe with Julia, the community advertises it as something that it isn't. I think otherwise some really cool stuff is happening in Julia Land.
[1] https://jochenschroeder.com/blog/articles/DSP_with_Python2/
[1] https://jochenschroeder.com/blog/articles/DSP_with_Python2/
> (although I posted this and nobody could point to something obvious)
For someone outside the field of DSP, please note that your post gives no context to the problem, nor to the input data. It's pretty hard to optimize something when we don't understand the structure of the problem or the input, and what exactly the code is trying to accomplish.
I remember taking a look at the post when it was posted some time ago, and having to give up pretty quickly because it was hard to make sense of it (in any language), and it seemed like I had to install specific Python libraries to (- hopefully, if everything works -) generate the input data. That takes it beyond "fun little optimization diversion to the day" to something that feels like work.
(Also, to anyone who hasn't read the post, I'll note that somebody did point to something obvious, as mentioned in the update to the post - a single suggestion that only needed superficial understanding of the code, that lead to a 3x speedup from OP's original Julia code.)
FWIW, someone with some domain knowledge on the Julia Discourse [1] was able to generate the input data, and posted code that they claim should be faster. They only mention the run time of the new code - 9 ms - so I don't know if, and by how much, it is faster in comparison. (Note that the code uses global variables at the end - which is fine there since they're using `@btime` and `$`s, but if using `@timed` instead, those should be enclosed in a `main` function or a `let` block.)
[1] https://discourse.julialang.org/t/this-post-claims-that-juli...
For someone outside the field of DSP, please note that your post gives no context to the problem, nor to the input data. It's pretty hard to optimize something when we don't understand the structure of the problem or the input, and what exactly the code is trying to accomplish.
I remember taking a look at the post when it was posted some time ago, and having to give up pretty quickly because it was hard to make sense of it (in any language), and it seemed like I had to install specific Python libraries to (- hopefully, if everything works -) generate the input data. That takes it beyond "fun little optimization diversion to the day" to something that feels like work.
(Also, to anyone who hasn't read the post, I'll note that somebody did point to something obvious, as mentioned in the update to the post - a single suggestion that only needed superficial understanding of the code, that lead to a 3x speedup from OP's original Julia code.)
FWIW, someone with some domain knowledge on the Julia Discourse [1] was able to generate the input data, and posted code that they claim should be faster. They only mention the run time of the new code - 9 ms - so I don't know if, and by how much, it is faster in comparison. (Note that the code uses global variables at the end - which is fine there since they're using `@btime` and `$`s, but if using `@timed` instead, those should be enclosed in a `main` function or a `let` block.)
[1] https://discourse.julialang.org/t/this-post-claims-that-juli...
wxy[:,:,k] += mu*conj(err[i,k])*X
wxy[:,:,k] .+= mu*conj(err[i,k]).*X
Why are you trying to optimize for performance if you don't have a goal in mind?
> the whole advertisement behind Julia is that one gets essentially C speed with the ease of Python. That's actually my main gripe with Julia, the community advertises it as something that it isn't.
I partially agree with this - that statement is true some of the time with some code, but in general there's a lot of nuance to that. (Though, I haven't seen that as a general claim much apart from in the initial "Why we created Julia" post describing what they were hoping to achieve; I more often see specific claims where people were able to port their scientific code into Julia that looks like Python and runs like C. But that may just be us having different bubbles online.)
My partial disagreement is from the fact that oftentimes, the "ease of Python" seems to get interpreted by Python coders as "as easy as Python is to me today" rather than "as easy as it was when I was a beginner" i.e. they expect to be able to port their Python knowledge directly somehow. Python has its own quirks and pitfalls, and over the first few months of people learn to work with/around them and write idiomatic Python code (and then it becomes second nature, and we often even forget that we had to learn to do this). Given a similar on-ramp, people can easily write Julia code that's at least comparable to C in terms of speed, even if it doesn't always match it.
But when you do need to match or exceed C, Julia starts asking for more in-depth knowledge of allocations, types and their performance characteristics, and the code starts deviating away from looking quite as readable as Python. Hopefully, you only need that in some deep internal functions or packed away in a library, but it _is_ a big caveat to that statement.
I partially agree with this - that statement is true some of the time with some code, but in general there's a lot of nuance to that. (Though, I haven't seen that as a general claim much apart from in the initial "Why we created Julia" post describing what they were hoping to achieve; I more often see specific claims where people were able to port their scientific code into Julia that looks like Python and runs like C. But that may just be us having different bubbles online.)
My partial disagreement is from the fact that oftentimes, the "ease of Python" seems to get interpreted by Python coders as "as easy as Python is to me today" rather than "as easy as it was when I was a beginner" i.e. they expect to be able to port their Python knowledge directly somehow. Python has its own quirks and pitfalls, and over the first few months of people learn to work with/around them and write idiomatic Python code (and then it becomes second nature, and we often even forget that we had to learn to do this). Given a similar on-ramp, people can easily write Julia code that's at least comparable to C in terms of speed, even if it doesn't always match it.
But when you do need to match or exceed C, Julia starts asking for more in-depth knowledge of allocations, types and their performance characteristics, and the code starts deviating away from looking quite as readable as Python. Hopefully, you only need that in some deep internal functions or packed away in a library, but it _is_ a big caveat to that statement.
I only had a quick look, but I would guess the slicing is what makes it slower than you would expect. Julia copies on an array slice, but you can use the @view macro or do some loops with a preallocated array to make it faster.
Thanks I'll have a look and post and update sometime later.
Don't be unreasonable. Numerical Julia code is just as fast, precisely because it's easy to design efficient high-level APIs and let the compiler optimise it.
Try timing Julia's IO, or it's string processing, or it's hash tables, or its array allocation or it's GC - you know, nearly everything other that mathematical operations on floats or ints. Nearly everything leaves performance on the table.
Sure Julia is fast, but not on a C level in these areas. There is still lots of room for optimisation, though.
Try timing Julia's IO, or it's string processing, or it's hash tables, or its array allocation or it's GC - you know, nearly everything other that mathematical operations on floats or ints. Nearly everything leaves performance on the table.
Sure Julia is fast, but not on a C level in these areas. There is still lots of room for optimisation, though.
What’s your point?
The point is that these things (string processing/hash tables) are typically done using custom/tuned code in C and so would be faster than the generic, and not particularly carefully tuned Julia implementations.
However, given this library seems to be about passing NumPy arrays to/from Julia, the application is almost certainly to expose numerical stuff from Julia to Python. Basically, the use case is similar to Numba, but it works by producing a C-extension from a Julia system image (if I understand correctly). The main advantage over something like Numba would be access to the Julia ecosystem.
However, given this library seems to be about passing NumPy arrays to/from Julia, the application is almost certainly to expose numerical stuff from Julia to Python. Basically, the use case is similar to Numba, but it works by producing a C-extension from a Julia system image (if I understand correctly). The main advantage over something like Numba would be access to the Julia ecosystem.
The point is that when adgjlsfhk1 doubts that Julia is slower than C, he does so from a background where "data" is a synonym for arrays of numbers, and "computing" means processing these arrays. Julia shines here, precisely because the people developing Julia have a similar focus.
In contrast, if your performance bottlenecks are IO, parsing and hash table operations, then Julia's performance will be in par with Python and get absolutely crushed by Rust (I have less experience with C, but would imagine Rust and C are about equally fast).
The major difference with Python is that Julia's implementation of these things _could_ be fast in a way that Python's can't. They just aren't.
In contrast, if your performance bottlenecks are IO, parsing and hash table operations, then Julia's performance will be in par with Python and get absolutely crushed by Rust (I have less experience with C, but would imagine Rust and C are about equally fast).
The major difference with Python is that Julia's implementation of these things _could_ be fast in a way that Python's can't. They just aren't.
Some of these are just safety-by-default things. For example, IO in Julia is thread-safe by default, which I think is a good idea because safety-first programming is good for say throwing `print` into a threaded loop written in the REPL. Here for example, https://github.com/aaronang/stl-benchmark/pull/3, was a case where Julia saw a performance hit from C++ and I was curious and tracked it down to this locking-by-default behavior. I'm not sure of a better way of handling it: the C/C++ behavior of not locking by default would make doing things correctly simply would be very hard to use (and is very hard in those languages).
Though I agree parsers haven't gotten much love in Julia. That said, this repo is saying it's for implementing NumPy extensions, and I don't think NumPy has many parsers it's using.
Though I agree parsers haven't gotten much love in Julia. That said, this repo is saying it's for implementing NumPy extensions, and I don't think NumPy has many parsers it's using.
in posix (so in any reasonable C implementation) printf is thread-safe by default, so your comment doesn't make much sense.
on par with python is way off. Julia in these areas definitely isn't fully optimized, but the performance will be much closer to something like Java for these cases. a dict in Julia will still have 1 pointer indirect compared to the python one where all your objects have 16 bytes overhead and an extra pointer indirection.
Python's dicts are way more optimised and have an implementation closer to Dictionaries.jl.
Anyway, I think it might be a good exercise for me to make a repo with a few test cases of where I think Julia could use some optimisations, and compare to Rust and Python.
Anyway, I think it might be a good exercise for me to make a repo with a few test cases of where I think Julia could use some optimisations, and compare to Rust and Python.
It says right there: "Always look at the source code"
fannkuch-redux, n-body, spectral-norm, reverse-complement: the fastest C wins by a large margin because it's an unreadable mess of vector intrinsics. If you really wanted to, you can do that in Julia too (and it might read better). Julia looks pretty even with C in the ones that were faithfully translated.
fannkuch-redux, n-body, spectral-norm, reverse-complement: the fastest C wins by a large margin because it's an unreadable mess of vector intrinsics. If you really wanted to, you can do that in Julia too (and it might read better). Julia looks pretty even with C in the ones that were faithfully translated.
Python itself has to become faster.
For the love of god. Make it as fast as PHP.
My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness.
Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do!
The result would be heaven.
For the love of god. Make it as fast as PHP.
My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness.
Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do!
The result would be heaven.
It is happening: https://github.com/faster-cpython/ideas
It seems to indicate a 30% performance boost?
That would be a step into the right direction.
6 more 30% improvements and it is on par with PHP.
That would be a step into the right direction.
6 more 30% improvements and it is on par with PHP.
It will yield a small speedup (< 50%) at best, probably for selected benchmarks that will be widely promoted.
Microsoft is misallocating its resources. The scientific ecosystem should be ported to .NET, with first class support for F#.
Microsoft is misallocating its resources. The scientific ecosystem should be ported to .NET, with first class support for F#.
.NET would be the perfect ecosystem. And for those who love their dynamic there is support for that too.
They already try it like 2/3 times and it didnt take off so why should it be different today?
Everytime they did it, it was half baked at best. Scientific libraries need driver support, Microsoft always takes the happy path and ignores all the other hardware permutations. If you are not running Intel/Nvidia in a Win32 environment you are out of luck. WSL2, ARM, AMD, etc. don't have much synergy in terms of product support.
> ... for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness.
Good news: there are more than two programming languages!
But seriously, what situation are you in where your only choices are Python and PHP?
Good news: there are more than two programming languages!
But seriously, what situation are you in where your only choices are Python and PHP?
Wait, you are using PHP for scientific computing? I would love to hear more about that part.
That could actually work out nicely with the right libraries, just haven't yet heard of anyone doing that.
I think Python is inherently difficult to optimize because it allows some dynamic trickery that PHP does not. Additionally being large and complex does not help. You can't really pull an LuaJit equivalent. There sure are still many quick wins to be made for Python performance but not sure if it will ever compete with PHP/Lua/Julia and others on the performance front.
That could actually work out nicely with the right libraries, just haven't yet heard of anyone doing that.
I think Python is inherently difficult to optimize because it allows some dynamic trickery that PHP does not. Additionally being large and complex does not help. You can't really pull an LuaJit equivalent. There sure are still many quick wins to be made for Python performance but not sure if it will ever compete with PHP/Lua/Julia and others on the performance front.
> Wait, you are using PHP for scientific computing? I would love to hear more about that part.
Not OP but I wish PHP was usable for scientific computing.
If I had better C/Rust/PHP internals insane macro skills I would integrate a dataframe library (Pola.rs?) into PHP to give us the basics for manipulating data.
Plenty more to add after that for matrix multiplication and the rest but I haven't thought that far ahead!
Not OP but I wish PHP was usable for scientific computing.
If I had better C/Rust/PHP internals insane macro skills I would integrate a dataframe library (Pola.rs?) into PHP to give us the basics for manipulating data.
Plenty more to add after that for matrix multiplication and the rest but I haven't thought that far ahead!
would python being 3x faster make a difference? you'd still have so call out to an actually fast language when you want your code to be fast.
A 3x faster Python would make it about 2x slower than PHP.
That is way easier to digest than the current ~5x performance penalty for using it.
I would probably not consider PHP anymore if Python was 50% of its speed.
I would never consider anything else than Python or PHP for web projects. Developer time is more precious than CPU time.
That is way easier to digest than the current ~5x performance penalty for using it.
I would probably not consider PHP anymore if Python was 50% of its speed.
I would never consider anything else than Python or PHP for web projects. Developer time is more precious than CPU time.
You can have small developer time with other technologies too (JavaScript, Go, Elixir etc) and get some additional benefits compared to PHP. Honestly I don't see the reason to use PHP today unless you have to;
1. Maintain existing PHP codebase
2. Your team only knows PHP
1. Maintain existing PHP codebase
2. Your team only knows PHP
Except for speed, PHP has another awesome thing going for it: Statelessness.
You throw some PHP files on a server and have a working web application. As soon as you change a PHP file, the change is in effect on next reload of the page. You might be able to do the same in Python, but everybody in Python world is doing the long running thread approach, so you would be on uncharted territory.
You throw some PHP files on a server and have a working web application. As soon as you change a PHP file, the change is in effect on next reload of the page. You might be able to do the same in Python, but everybody in Python world is doing the long running thread approach, so you would be on uncharted territory.
If you want to optimize for developer time you should look at statically typed languages with rich type systems that have little overhead.
(Writing Python extensions in Julia)
..which could be a heckuva lot friendlier and often safer than C!
The requisite python boilerplate is pretty ugly, though:
..which could be a heckuva lot friendlier and often safer than C!
The requisite python boilerplate is pretty ugly, though:
init_jl()
include_src('example.jl', __file__)
exec_julia('example.init()')
jl_mat_mul = Pyfunc(jl_mat_mul)
I had a look at the demo folder, that's just a typo.
It should be:
jl_mat_mul = Pyfunc(mat_mul)
which makes a lot more sense.
jl_mat_mul = Pyfunc(mat_mul)
which makes a lot more sense.
Yeah, the boilerplate is rough, but python's import system is magic. There's no reason why you couldn't just do
from example.jl import mat_mul
So, the example is about using Julia code bindings instead of numpy's BLAS bindings for "C-level performance"... that's a misguided optimization attempt by the book.
Should change title to at least hint this is a c-interface to Julia. The issue is not that you cannot use c. The key is if you have problem you have to know Julia.
> import julia_numpy as np
I don't get it. Does this override python's numpy library with a julia native? If so does it implement all of numpy functionality? Or does this piggyback on normal numpy, redirecting accordingly, but also allows plugging extensions written in julia? The documentation doesn't shed any light at all. Neither do the demos.
Also, is this simply a python julia interface and the "high-performance" part of the description is just part of the usual juliaspeak? Or is there an actual "high-performance" component there that makes this more than a simple python julia interface?
I don't get it. Does this override python's numpy library with a julia native? If so does it implement all of numpy functionality? Or does this piggyback on normal numpy, redirecting accordingly, but also allows plugging extensions written in julia? The documentation doesn't shed any light at all. Neither do the demos.
Also, is this simply a python julia interface and the "high-performance" part of the description is just part of the usual juliaspeak? Or is there an actual "high-performance" component there that makes this more than a simple python julia interface?
Coincidentally I used PyJulia last week to call some numerical code I translated from Python/Numpy to Julia.
The actual translation was straightforward and done in under one hour, but I needed an additional day for fine tuning and profiling to actually make it fast.
Does anyone know, how JNumPy compares to PyJulia? Calling my Julia code via PyJulia seems to increase the startup time of Julia even more and it would be great, if JNumpy would do better here.
Does anyone know, how JNumPy compares to PyJulia? Calling my Julia code via PyJulia seems to increase the startup time of Julia even more and it would be great, if JNumpy would do better here.
Are you aware of the juliacall package? [1]
The announcement post [2] says "Calling Julia from Python, juliacall seems to have smaller overhead (in time) than julia". The benchmarks below that show the overhead on calling `identity` with `juliacall` being less than half of what it is with PyJulia.
[1] https://docs.juliahub.com/PythonCall/WdXsa/0.9.4/juliacall/ [2] https://discourse.julialang.org/t/ann-pythoncall-and-juliaca...
The announcement post [2] says "Calling Julia from Python, juliacall seems to have smaller overhead (in time) than julia". The benchmarks below that show the overhead on calling `identity` with `juliacall` being less than half of what it is with PyJulia.
[1] https://docs.juliahub.com/PythonCall/WdXsa/0.9.4/juliacall/ [2] https://discourse.julialang.org/t/ann-pythoncall-and-juliaca...
Great to see contributions from a fellow Suzhou Julia enthusiast.
for C++ folks i can only recommend pybind11 which provides a numpy interface.
it's a breeze to use, header only.
it's a breeze to use, header only.
Very cool and nice work.
Also why should I pick this over cython, pythran or numba? with those I don't need to learn how to optimise another language (and no just writing Julia code does not necessarily get you large speed ups and certainly not the speed of C in many cases).
I really wish the Julia community would stop overselling the language.