Just to complement the previous answer. It varies from kernel to kernel. Regarding reductions, the strategy implemented in the TornadoVM JIT compiler allows us to execute reductions offloaded from Java sequential code within 85% of the hand-written OpenCL Kernels:
To quote Gary Frost (creator of Aparapi), TornadoVM is the state-of-the-art right now. He mentioned this at JVMLS 2023. Hopefully the videos will be available soon from this link:
https://openjdk.org/projects/mlvm/jvmlangsummit/
Totally agree. OpenCL code is portable, but performance is not. That's why TornadoVM specializes the OpenCL code depending on the target device. For FPGAs we do a lot more optimizations compared to GPUs, such as tuning the thread-scheduling, better loop unrolling and loop flattening, use of local memory, etc. All of these optimizations are automatically performed in the compiler-IR (GraalIR) before generating the actual OpenCL C code.
With those compiler specializations, we aim to close the performance gap between hand-tuned code and generated code.
The VM name came because TornadoVM implements its own set of bytecodes for handling heterogeneous execution. These bytecodes are used for handling JIT compilation, device exploration, data management and live task-migration for heterogeneous devices (multi-core CPUs, GPUs, and FPGAs). We sometimes refer to a VM inside a VM (nested VM). The main VM is the Java Virtual Machine, and TornadoVM sits on top of that.
We are integrating TornadoVM with Apache Flink. So the idea is to execute unmodified map/reduce Flink applications automatically on heterogeneous hardware by using TornadoVM in the task-managers. This is part of a current European project (https://e2data.eu/).
That's correct. TornadoVM specializes the OpenCL code depending on the target device to increase performance. Also, as you pointed out, it supports some objects through the partial escape analysis by applying scalar replacements.
Also, TornadoVM supports method calls and invitations to native code (e.g., Math library). The examples in the presentations are for simplicity but we have some use cases, such as KFusion Kinet with ~7k lines of Java code for computer vision (https://github.com/beehive-lab/kfusion-tornadovm).
Aparapi is a direct translation from Java bytecode to OpenCL. To do so, Aparapi provides a compiler and a runtime system to automatically handle data and execute the generated OpenCL Kernel.
TornadoVM compiles from Java bytecode to OpenCL as well. But additionally, it optimizes and specializes the code by interleaving Graal compiler optimizations, such as partial escape analysis, canonicalization, loop unrolling, constant propagation, etc) with GPU/CPU/FPGA specific optimizations (e.g., parallel loop exploration, automatic use of local memory, parallel skeletons exploration
such as reductions).
TornadoVM generates different OpenCL code depending on the target device, which means that the code generated for GPUs is different for FPGAs and multi-cores. This is because of OpenCL code is portable across devices, but performance is not portable. TornadoVM addresses this challenge by applying compiler specialization depending on the device.
Additionally, TornadoVM performs live task migration between devices, which means that TornadoVM decides where to execute the code to increase performance (if possible). In other words, TornadoVM switches devices if it knows the new device offers better performance. As far as we know, this is not available in Aparapi (in which device selection is static). With the task-migration, the TornadoVM's approach is to only switch device if it detects application can be executed faster than the CPU execution using the code compiled by C2 or Graal-JIT, otherwise it will stay on CPU. So TornadoVM can be seen as a complement to C2 and Graal. This is because there is no single hardware to best execute all workloads efficiently. GPUs are very good at exploiting SIMD applications, and FPGAs are very good at exploiting pipeline applications. If your applications follow those models, TornadoVM will likely select heterogeneous hardware. Otherwise, it will stay on CPU using the default compilers (C2 or Graal).
Just to complement the previous answer. It varies from kernel to kernel. Regarding reductions, the strategy implemented in the TornadoVM JIT compiler allows us to execute reductions offloaded from Java sequential code within 85% of the hand-written OpenCL Kernels:
https://www.researchgate.net/publication/327871451_Using_Com...