Graal: How to use the new JVM JIT compiler in real life [video](youtube.com)
youtube.com
Graal: How to use the new JVM JIT compiler in real life [video]
https://www.youtube.com/watch?v=yhtrRhNUHvQ
41 comments
Graal dev here. We're testing on jdk 8,9,10 and 11. We are packing graalvm with jdk 8 atm to keep variability reasonable. Graal will be integrated into the JVM soon with project Metropolis: http://openjdk.java.net/projects/metropolis/
Graal is used for many more things than dynamically compiling Java. Besides others it's used for SubstrateVM (closed world AOT compilation) and Truffle languages like TruffleRuby, Graal.JS, FastR, Sulong(llvm integration) and our newest member Graal.Python.
Graal comes out of Oracle Labs. The build tool mx is short for Maxine, the spiritual predecessor of the Graal project. If you don't know mx it's quite intimidating and ugly. It's Python written by Java developers. But it does it's job coping with our not so standard building and testing requirements. No we don't run mx on Graal.Python yet, but we are working on it for full build tool metacircularity ;).
Graal is used for many more things than dynamically compiling Java. Besides others it's used for SubstrateVM (closed world AOT compilation) and Truffle languages like TruffleRuby, Graal.JS, FastR, Sulong(llvm integration) and our newest member Graal.Python.
Graal comes out of Oracle Labs. The build tool mx is short for Maxine, the spiritual predecessor of the Graal project. If you don't know mx it's quite intimidating and ugly. It's Python written by Java developers. But it does it's job coping with our not so standard building and testing requirements. No we don't run mx on Graal.Python yet, but we are working on it for full build tool metacircularity ;).
Vendoring the JDK is a much better use of resources than chasing compatibility, esp for a research project.
Is Graal.Python related to Zippy [0,1] ?
[0] https://github.com/securesystemslab/zippy
[1] http://socalpls.github.io/archive/2013nov/slides/zippy.pdf
Is Graal.Python related to Zippy [0,1] ?
[0] https://github.com/securesystemslab/zippy
[1] http://socalpls.github.io/archive/2013nov/slides/zippy.pdf
It started as fork of Zippy. It changed very much since then. You can download the binary as part of graalvm. It's not yet open source, but it will be. It's just not ready yet.
I've been watching zippy for a while, unfortunately it relies heavily on Jython which is dead. Would be very interested to see progress on this! (And maybe contribute as well!)
I'm sure you will hear about us when we go open source with Python.
Hi - thank you so much for posting here.
Please please get rid of MX. Please move to a better build tool - you will pull the ecosystem with you. And people will help you with that.
Secondly - I have not seen Graal Python anywhere. Is there a repo ? The GitHub project seems centred around Truffleruby. Graal Python would be awesome!
Secondly - I have not seen Graal Python anywhere. Is there a repo ? The GitHub project seems centred around Truffleruby. Graal Python would be awesome!
We have lots of infrastructure depending on mx. Infrastructure that no other build tool provides as far as I can tell.
In the end every build tool sucks. Sometimes you just need to be pragmatic. Mx integrates with maven pretty well and it generates ide config for intellij, eclipse and netbeans. Moving to a different system is probably not worth the cost atm. And it's not complicated to use at all. E.g mx build builds and mx gate builds and tests. Mx unittest runs unit tests.
As written below. Graal.Python is not yet open source. But you will hear from us soon.
In the end every build tool sucks. Sometimes you just need to be pragmatic. Mx integrates with maven pretty well and it generates ide config for intellij, eclipse and netbeans. Moving to a different system is probably not worth the cost atm. And it's not complicated to use at all. E.g mx build builds and mx gate builds and tests. Mx unittest runs unit tests.
As written below. Graal.Python is not yet open source. But you will hear from us soon.
Any story behind why it's not open-source ? I mean Ruby, Js, R seem to have been for a long time.
Any splashy announcements expected?
p.s. sorry to be pushy. I have a long standing python-and-reactjs-on-java-in-2018 bet !
Any splashy announcements expected?
p.s. sorry to be pushy. I have a long standing python-and-reactjs-on-java-in-2018 bet !
They tend to work in private then open source later.
Why not Graal.py?
For me this would mean a python project (graal bindings within CPython) and not a Python implementation.
CPython Pypy Jython IronPython
I would now go with GraalPython (GrPython or GPython)
CPython Pypy Jython IronPython
I would now go with GraalPython (GrPython or GPython)
Good idea. The name is not final.
Nothing, beyond the herculean effort that is to add a new JIT compiler, while keeping up everything else running without regressions.
Graal's work is well known since the days of MaxineVM.
Hopefully eventually everything will be sorted out.
For similar comparison, Microsoft is also having multiple year/releases effort with RyuJIT, .NET Native and CoreRT.
Thankfully Bazel is mostly restrained to Googleplex stuff.
It is already a pain on Android, measured in cups of coffee/tea per developer, that they went with Gradle instead of Maven.
Graal's work is well known since the days of MaxineVM.
Hopefully eventually everything will be sorted out.
For similar comparison, Microsoft is also having multiple year/releases effort with RyuJIT, .NET Native and CoreRT.
Thankfully Bazel is mostly restrained to Googleplex stuff.
It is already a pain on Android, measured in cups of coffee/tea per developer, that they went with Gradle instead of Maven.
bazel is not used on android because it came later. However, lots of startups are using it to build both their android+ios apps in production in CI/CD. here's the android toolchain - https://docs.bazel.build/versions/master/tutorial/android-ap...
that's a bit of fud that it is restricted to googleplex, not sure why you would say that.
https://github.com/bazelbuild/bazel/wiki/Bazel-Users - bazel is being used at tons of places including Dropbox, Asana, Etsy, Huawei, Pinterest, Uber, etc. for multiple languages - Java, Rust, Golang, C++, etc.
Again - I'm not sure why you are mentioning about a "new JIT compiler". Graal on Java 9 works - if you check a ton of documentation, everyone is talking about using Graal on Java 9 (self-built). E.g. http://chrisseaton.com/rubytruffle/jokerconf17/
I'm talking about Oracle + Graal project not putting out Java 9 based releases because of a co-ordination effort.
that's a bit of fud that it is restricted to googleplex, not sure why you would say that.
https://github.com/bazelbuild/bazel/wiki/Bazel-Users - bazel is being used at tons of places including Dropbox, Asana, Etsy, Huawei, Pinterest, Uber, etc. for multiple languages - Java, Rust, Golang, C++, etc.
Again - I'm not sure why you are mentioning about a "new JIT compiler". Graal on Java 9 works - if you check a ton of documentation, everyone is talking about using Graal on Java 9 (self-built). E.g. http://chrisseaton.com/rubytruffle/jokerconf17/
I'm talking about Oracle + Graal project not putting out Java 9 based releases because of a co-ordination effort.
Add to that list SpaceX. Guys from SpaceX showed in Bazel conference they are using Bazel to build their software.
> However, lots of startups are using it to build both their android+ios apps in production in CI/CD.
Keyword here is startups, with freedom to use alternative tech just because.
Apparently Android Studio is work in progress, and doesn't support all the features, like the NDK.
https://ij.bazel.build/docs/bazel-plugin.html
I really don't see the need to use anything else other than Maven, unless it really substantially improves the build speed and IDE tooling experience.
If it wasn't for Google's imposition of Gradle on us, I wouldn't bother with it.
However if in an imaginary future, they replace Gradle with Bazel and those points are met (build speed and tooling) then by all means.
> I'm talking about Oracle + Graal project not putting out Java 9 based releases because of a co-ordination effort.
It is already there on Linux x64 for 9, with macOS and Windows on 10, as mentioned on the talk, so I really don't follow.
Keyword here is startups, with freedom to use alternative tech just because.
Apparently Android Studio is work in progress, and doesn't support all the features, like the NDK.
https://ij.bazel.build/docs/bazel-plugin.html
I really don't see the need to use anything else other than Maven, unless it really substantially improves the build speed and IDE tooling experience.
If it wasn't for Google's imposition of Gradle on us, I wouldn't bother with it.
However if in an imaginary future, they replace Gradle with Bazel and those points are met (build speed and tooling) then by all means.
> I'm talking about Oracle + Graal project not putting out Java 9 based releases because of a co-ordination effort.
It is already there on Linux x64 for 9, with macOS and Windows on 10, as mentioned on the talk, so I really don't follow.
Even if we agree it's true, it's still more than mx. And that was the whole point. MX is a pain to setup and use - if you don't want Bazel, then fine. But then use cmake or waf or scons or gradle or one of the many build tools that make this possible...and more importantly are more widely used and have a ton of help threads and documentation already.
AFAIK Bazel is the best fit because of automatic download of dependencies from many source repos.
About java 9 - it's not officially supported. You have to build your own. Please read the GitHub link I posted in my original comment.
AFAIK Bazel is the best fit because of automatic download of dependencies from many source repos.
About java 9 - it's not officially supported. You have to build your own. Please read the GitHub link I posted in my original comment.
That is about OpenJDK, the official Linux x64 binaries have it.
As mentioned on the talk, all the binary downloads with AOT support, have Graal.
When the commercial JDK moves into OpenJDK as planned, that will most certainly no longer be an issue.
As mentioned on the talk, all the binary downloads with AOT support, have Graal.
When the commercial JDK moves into OpenJDK as planned, that will most certainly no longer be an issue.
not sure what you mean by that.
All official Graal releases are based on top of JDK8 - http://www.oracle.com/technetwork/oracle-labs/program-langua...
JDK 9 supports it - however the build and release needs some kind of Oracle-level coordination. I'm not sure if this has anything to do with OpenJDK. But happy to be corrected.
All official Graal releases are based on top of JDK8 - http://www.oracle.com/technetwork/oracle-labs/program-langua...
JDK 9 supports it - however the build and release needs some kind of Oracle-level coordination. I'm not sure if this has anything to do with OpenJDK. But happy to be corrected.
Video, at minute 5, "Where do I get it":
JEP 295: Ahead-of-Time Compilation http://openjdk.java.net/jeps/295
"The project will merge Graal core into the JDK, and deliver it in Linux/x64 builds."
OpenJDK is not the JDK one downloads from http://www.oracle.com/technetwork/java/javase/downloads/jdk9....
Also there are many Graals so to say, the research being done at Oracle labs, the rewriting of the JVM replacing C++ with Java (Metropolis) and the piecewise integration into the official JDK.
So I don't find it strange to have some kind of Oracle-level coordination.
Yes I do agree it is annoying, but from my humble point of view it is a situation that when everything has moved into the OpenJDK, with AOT compilation supported across all three major desktop OSes, the situation will be sorted out in the meantime.
And we are only speaking of Oracle here, there a few other dozen vendors that most likely will never have something like Graal on their JVMs.
JEP 295: Ahead-of-Time Compilation http://openjdk.java.net/jeps/295
"The project will merge Graal core into the JDK, and deliver it in Linux/x64 builds."
OpenJDK is not the JDK one downloads from http://www.oracle.com/technetwork/java/javase/downloads/jdk9....
Also there are many Graals so to say, the research being done at Oracle labs, the rewriting of the JVM replacing C++ with Java (Metropolis) and the piecewise integration into the official JDK.
So I don't find it strange to have some kind of Oracle-level coordination.
Yes I do agree it is annoying, but from my humble point of view it is a situation that when everything has moved into the OpenJDK, with AOT compilation supported across all three major desktop OSes, the situation will be sorted out in the meantime.
And we are only speaking of Oracle here, there a few other dozen vendors that most likely will never have something like Graal on their JVMs.
> I really don't see the need to use anything else other than Maven, unless it really substantially improves the build speed and IDE tooling experience.
If all you need is build speed why not fall back to ANT scripts?
I am sorry to disappoint you but If it wasn’t for ANT, I’d say Maven is one of the worst build tools out there (at least as far as Java ecosystem is concerned).
Build speed of any decent Maven project is nothing to sneeze at. Grade, while slightly slower at startup is consistently faster at dev time because it can actually do proper incremental build. And it is miles easier for dev tools to integrate with because its build model is actually immutable, which is a major point why Gradle IDE integrations don’t actually suck when compared to Maven.
It’s funny how Maven is considered to be more declarative because of the XML syntax of the POM files while the underpinning technology is ultimately much more dynamic and mutable than Gradle’s. While Gradle which chose dynamic and malleable language like Groovy for the syntactic front end is much more declarative and immutable of the two.
If all you need is build speed why not fall back to ANT scripts?
I am sorry to disappoint you but If it wasn’t for ANT, I’d say Maven is one of the worst build tools out there (at least as far as Java ecosystem is concerned).
Build speed of any decent Maven project is nothing to sneeze at. Grade, while slightly slower at startup is consistently faster at dev time because it can actually do proper incremental build. And it is miles easier for dev tools to integrate with because its build model is actually immutable, which is a major point why Gradle IDE integrations don’t actually suck when compared to Maven.
It’s funny how Maven is considered to be more declarative because of the XML syntax of the POM files while the underpinning technology is ultimately much more dynamic and mutable than Gradle’s. While Gradle which chose dynamic and malleable language like Groovy for the syntactic front end is much more declarative and immutable of the two.
I've use the unofficial Android Maven plugin before the official Gradle plugin was released. When the Gradle plugin came out, it made my builds so much simpler than it used to be with Maven. As much as I like XML, having 4 lines per dependency is way too much. And having to write a custom plugin every time you want some custom logic is terrible. The Maven ecosystem may have increased since then, but I'm very glad they chose Gradle over Maven.
The beauty of XML is that is designed for tooling, not to be written by hand.
So for me adding a dependency is just a matter of using the graphical tooling in Eclipse and Netbeans.
All major headaches I "enjoyed" with Maven was custom logic coming from Ant scripts, from team not willing to adopt a more declarative way of designing their projects.
With Gradle, I get the same mess Ant used to be, just slower.
So for me adding a dependency is just a matter of using the graphical tooling in Eclipse and Netbeans.
All major headaches I "enjoyed" with Maven was custom logic coming from Ant scripts, from team not willing to adopt a more declarative way of designing their projects.
With Gradle, I get the same mess Ant used to be, just slower.
I shouldn't be getting nerd sniped by comment about Maven but here goes. Maven doesn't compose because it is an ad-hoc lisp where every element has different semantics, every element is a different language. This isn't an XML problem, but a way that XML fragments in Maven map to the interpreters of those fragments. Maven could have been an amazingly compact expressive combination of a logic/constraint solver with a declarative language, instead it is PHP written in XML. Maven is declarative only in that it has a textual interface.
My disclaimer: my coworkers are using Scala and SBT. Gradle + Java isn't slower than Ant or Maven. Gradle + Kotlin is a little slower, they just need to make the annotation processors incremental.
But for Java, incremental builds with compiler avoidance combined with the build cache, parallel builds (incubating features close to maturity) mean I write more sophisticated, easier to maintain builds than I ever have with Maven.
But for Java, incremental builds with compiler avoidance combined with the build cache, parallel builds (incubating features close to maturity) mean I write more sophisticated, easier to maintain builds than I ever have with Maven.
Something awesome would be to have a S-expression to Maven XML transpiler. Parens instead of xml tags. Is there something like that?
Not a transpiler, but we use something similar all the time in Clojureland.
https://leiningen.org
It can easily be used as a Java build tool.
https://leiningen.org
It can easily be used as a Java build tool.
You could do something like that with polygot-maven.
https://github.com/takari/polyglot-maven
https://github.com/takari/polyglot-maven
The encoding (XML) isn't Maven's problem, the semantics are.
Does Graal support tail call optimization? It has been announced for hotspot +10 years ago but still not implemented as today.
edit: https://blogs.oracle.com/jrose/tail-calls-in-the-vm
edit: https://blogs.oracle.com/jrose/tail-calls-in-the-vm
When executing on Java we are dependent on hotspot to support it in the native bytecode interpreter. In truffle interpreters we made some successful experiments with tail calls emulated using exceptions. It's pretty slow in the interpreter but as soon as graal optimizes it, it just turns into a tailcall loop.
Here is some more info:
http://epub.jku.at/obvulihs/content/pageview/508465
You may find some blog posts about truffle and tailcalls as well.
You may find some blog posts about truffle and tailcalls as well.
[deleted]
You may see support for tail calls added as part of project loom.
Does Graal VM work on Windows yet? That was the main factor that held me back from using it the last time I evaluated it.
Windows support will be made available on Java 10.
https://bugs.openjdk.java.net/browse/JDK-8172670
https://bugs.openjdk.java.net/browse/JDK-8172670
Weird I remember running Graal on Windows a couple of years ago, was Windows support dropped along the way?
Graal works on Windows since a long time,
it's the AOT with Graal that works only since Java 10.
[deleted]
It looks like a bureaucratic struggle of some kind - I wonder what's the behind the scenes story at Oracle.
Additionally, "mx" : the build system that Graal uses is fairly funky - and everyone including Chris (the engineer that made this video) is trying to get the team to move on
https://twitter.com/christhalinger/status/958824900004753408...
I wish they would use Bazel or something