Extreme Multi-Threading: C++ and Julia 1.9 Integration(scientificcoder.com)
scientificcoder.com
Extreme Multi-Threading: C++ and Julia 1.9 Integration
https://scientificcoder.com/extreme-multi-threading-c-and-julia-19-integration
37 コメント
I was confused about why this doesn't mention libraries like CxxWrap, and how many rough edges there seem to be, but then I realized this isn't the common case of calling out to a C++ library from Julia. It's about compiling Julia code into a library that can then be linked to from C++ - which I didn't even realize was possible!
Cxxwrap was broken for a while there, maybe a few years? Maybe it was cxx... I forget, whichever one let you have a c++ repl. But yea I'd argue that it's a pretty common use case to embed Julia in c++. This has been possible for a long time, Julia is written in C after all, but it's hard to do meaningfully because of Julia's GC. You have to do a lot of sketchy stuff to ensure values don't get deleted as you call for them via ffi. This might surprise you but you can do this with pretty much any other languages, usually with far less RAM consumption and JIT interruptions though...
> Cxxwrap was broken for a while there, maybe a few years? Maybe it was cxx
Yes, Cxx: https://github.com/JuliaInterop/Cxx.jl -- it uses Clang to generate JIT'd interop thunks, and does neat things to enable that, like cross-language type inference and inlining (in addition to the REPL).
CxxWrap is a different thing, which AFAIK has been actively maintained for > 5 years now: https://github.com/JuliaInterop/CxxWrap.jl
It is similar to Boost.Python/pybind11/nanobind: bindings are written in C++ and compiled ahead of time into a module that defines Julia entrypoints. Those entrypoints take care of signature selection, translation to/from Julia objects, and lifetime bookkeeping.
Yes, Cxx: https://github.com/JuliaInterop/Cxx.jl -- it uses Clang to generate JIT'd interop thunks, and does neat things to enable that, like cross-language type inference and inlining (in addition to the REPL).
CxxWrap is a different thing, which AFAIK has been actively maintained for > 5 years now: https://github.com/JuliaInterop/CxxWrap.jl
It is similar to Boost.Python/pybind11/nanobind: bindings are written in C++ and compiled ahead of time into a module that defines Julia entrypoints. Those entrypoints take care of signature selection, translation to/from Julia objects, and lifetime bookkeeping.
There is also a new player in the game of C++ wrapping Julia: https://github.com/Clemapfel/jluna
Good project, but note that it has its own (pre 1.9) threading model.
Classic Julia. Every minor version bumps you gotta rewrite your code.
If you're using plain-old-C, don't overlook the excellent CBinding.jl:
https://github.com/analytech-solutions/CBinding.jl
https://github.com/analytech-solutions/CBinding.jl
[deleted]
This is great! What feels like ages ago I attempted to include the Julia runtime in a C++ application and it was quite difficult, as only one thread was allowed to call the runtime (not just one thread at a time, one particular thread; if it was called by any other thread than that one particular thread it or the GC would get corrupted somehow and crash).
Yes that is precisely what was fixed, essentially the thread local storage that Julia was expecting were not setup and thus calling the runtime from a foreign thread would cause a crash.
This now enables to dynamically add and remove threads.
This now enables to dynamically add and remove threads.
Nice! I'm really enjoying your blogposts, and it's very cool to see julia being used at ASML for stuff like this.
What is extreme about this multi-threading?
Because it demonstrates a Julia library using threads created in the calling C++ code.
That passing of threads between languages is highly uncommon (because many things could fail and make you crash / slow along the way and because if your language can create threads then it likely already has C-like performance so you will not introduce parallel libraries written in another language for performance reasons).
That passing of threads between languages is highly uncommon (because many things could fail and make you crash / slow along the way and because if your language can create threads then it likely already has C-like performance so you will not introduce parallel libraries written in another language for performance reasons).
Without knowing anything about julia; is thread creation/management better on the C++-side or why wouldn't you just create those threads on the julia side?
Anyways, it is cool if it's stable and works well. I guess there's a use case for such feature, even if it sounds strange to an outsider.
Anyways, it is cool if it's stable and works well. I guess there's a use case for such feature, even if it sounds strange to an outsider.
It sounds like they already had multithreaded code in C++.
One possible use case is that you didn't write/control the C-++ side, yet you can still extend it with Julia.
> Julia library using threads created in the calling C++ code.
I understand Julia is much more dynamic language. But Java JNI can do this too. You have to attach the thread manually, and take care of pinned references.
I understand Julia is much more dynamic language. But Java JNI can do this too. You have to attach the thread manually, and take care of pinned references.
java jni is also really slow. it causes a bunch of copying and pointer indirections. Julia's C interop is zero cost.
> it causes a bunch of copying and pointer indirections.
That's correct. As you said elsewhere, lot of it pins down (!) to design of Java. There's direct byte buffer, but you have to architect your code around nio APIs to utilize it.
But we were talking about working with native threads.
That's correct. As you said elsewhere, lot of it pins down (!) to design of Java. There's direct byte buffer, but you have to architect your code around nio APIs to utilize it.
But we were talking about working with native threads.
> java jni is also really slow. it causes a bunch of copying and pointer indirections.
Are there any safety reasons for it or is it just a sign of its age?
Are there any safety reasons for it or is it just a sign of its age?
it's less safety and more design. java uses a moving GC, so there need to be indirections to allow the GC to move objects without C trying to access them and segfaulting. also, java only has objects, so all the data structures are wrong for C which means most uses in practice will end up copying all the memory anyway. Julia's structs are compatible with C structs, and the GC is non-moving, so you don't have random indirections. another advantage is that Julia generates machine code and then doesn't do all the fancy JIT stuff java does with multiple rounds of optimization and deoptimization, so once you've generated a function in Julia, you can just cal the pointer from C and it won't disappear from under you.
Yea but for tabular data you can just use apache arrow to get zero copy transfers from java to any language. Avoiding C entirely if you want. I wouldn't talk about Julia's GC like it's great, tried 1.85 the other day and single thread had the GC fail single thread no FFI. Something about a the GC not being able to switch tasks. And yea the GC will absolutely sweep away your memory, you have to write code very carefully in Julia to do FFI. It's like writing abstracted C with a GC that frees data from under your feet at random times.
aren't you just embedding julia into cpp?
I consider multi-threaded multi-language embedding an extreme sport. Maybe you are a software athlete and it's easy for you, but I think it's pretty scary.
maybe i am missing something but this is old news for embeddable common lisp. see threading section in this article from 1994[0], or is this extreme-threading different somehow? there is also a recent llvm-based implementation [1] that aims for direct lisp-cpp interface.
i know my usage of the word "just" might suggest otherwise but i am not trying to deminish this feat. i use common lisp for scientific work and whenever i hear about something cool in another language i ask myself can i do this with my toolset. this is the reason i am asking this.
[0] https://dl.acm.org/doi/pdf/10.1145/224139.1379849
[1] https://clasp-developers.github.io/
i know my usage of the word "just" might suggest otherwise but i am not trying to deminish this feat. i use common lisp for scientific work and whenever i hear about something cool in another language i ask myself can i do this with my toolset. this is the reason i am asking this.
[0] https://dl.acm.org/doi/pdf/10.1145/224139.1379849
[1] https://clasp-developers.github.io/
I think it’s just that this is possible in (the recently released) Julia 1.9 and wasn’t possible in earlier versions due to how thread-local storage and the GC worked, and doing this can be useful sometimes.
Your link 0 is cool, but is fairly different from the new stuff here. Some key differences are that the paper uses a conservative garbage collector (so enjoy random memory leaks) as a way to get around the problems of interop with C. Also, while that paper supports multithreading, as far as I can tell, it meant n-thread to 1 core (i.e. concurrent) multithreading (which is what the term usually referred to before multicore cpus were common), while Julia supports n threads to m cores (see Java's long awaited project Loom) which has a bunch of additional challenges. The last thing to mention is that the paper required a near complete rewrite of the lisp compiler. While that's an option for a simple language like Lisp, it's not as feasible for more complicated languages.
Multithreading is handled by C so N-M should be no problem because the key thing is N. If threads are managed from ECL instead then, if available, it will use pthreads [0]
ECL is not a rewrite of some THE compiler, it is a whole implementation. This actually goes toward your point that Julia cant do this because the language is not seperate from its compiler, but it is not because julia is more complicated than common lisp
Common Lisp has been called many things, rarely simple. Its spec is over 1000 pages
[0] https://ecl.common-lisp.dev/static/manual/Native-threads.htm...
ECL is not a rewrite of some THE compiler, it is a whole implementation. This actually goes toward your point that Julia cant do this because the language is not seperate from its compiler, but it is not because julia is more complicated than common lisp
Common Lisp has been called many things, rarely simple. Its spec is over 1000 pages
[0] https://ecl.common-lisp.dev/static/manual/Native-threads.htm...
I think the title was intended to be more playful and fun than a serious take on the most technically intense applications of threading in C++.
This might seem scary to you but people have been doing things like this for decades now in other languages. Not saying you shouldn't post your blog, you should, but yea...
I recognized your nick from another Julia article. Why do you always have that condescending tone when commenting on something related to Julia?
Very cool! thanks for sharing
hsjqllzlfkf(2)