JEP draft: Ahead of Time Compilation for the Java Virtual Machine

Summary

Enhance the Java Virtual Machine with the ability to load Java applications and libraries compiled to native code for faster startup and baseline execution.

Goals

Enable the Java Virtual Machine to load Java code compiled Ahead of Time to native code, if it was compiled by a compiler from a matching Java Virtual Machine. Allow C1, C2, and JVMCI Compilers to compile native code Ahead of Time for a Java Virtual Machine to use. "Profiling" code is code compiled by C1 in tiers 2 and 3 for use as baseline execution and profiling, while "Optimized" code is heavily optimized code produced by C1 in tier 1, C2, or any compatible JVMCI compiler for use as permanent, always optimized code that executes outside the Tiered Compilation model and is hence not subject to optimization and deoptimization cycles.

Non-Goals

It is not a goal to support this feature for code that violates the constraints that Leyden places on its closed world binaries counterpart.

Success Metrics

Java applications, libraries (including Java standard libraries), and any pluggable Java Virtual Machine components written in Java (such as any JVMCI Compilers) should be able to be compiled Ahead of Time to native code, in either profiling or optimized modes, and be loaded in a matching Java Virtual Machine with high baseline performance, bypassing the Interpreter and, based on the mode it was compiled in, C1 or C2 compilation entirely. libgraal seems to already be loadable as a shared library within HotSpot, as such the existing technology will also be a focus of this JEP.

Motivation

Java applications and libraries are currently executed in a customizable three stage model, first at Tier 0 in the Interpreter, at Tier 3 as C1 compiled code, and at Tier 4 as C2 compiled code, with Tiers 1 and 2 being special-cased C1 compiled code. This process is very dynamic, involving many iterations of optimization and deoptimization cycles over the course of the code's lifetime. Additionally, this also means that code warmup (where execution of code instruments compiling a highly optimized version) can take a long time, causing performance issues in applications, some of which can be critical, meaning such issues are not acceptable.

A good example is the Graal Compiler, which used to be an available replacement for C2 prior to Java 17. The bootstrapping stage when the compiler was started up affected the entire application negatively, as Graal itself had to be compiled before it could operate reasonably, something which negated the possible performance gains it brought significantly.

Additionally, the deoptimization process that occurs when C2 compiled code hits an incorrect optimization assumption can also be costly, given that the profiling C1 variant has to again be recompiled when this happens, only to be discarded again once C2 recompiles the same method. This, combined with the issues described above, can be wasteful, and is a significant use of runtime resources and source of overhead. A good solution to this would be to have two modes of compilation: "Profiling", which is a permanent Ahead of Time C1 (At Tiers 2 or 3) compiled variant of the method replace the Interpreter and all runtime C1 compilation, such that on startup, execution begins with the precompiled C1 code and then immediately progresses to C2 compilation, bypassing some of the warmup steps, as well as the Interpreter, entirely, and "Optimized", which are methods that at compilation of the Java application are heavily optimized by either C1 in Tier 1, C2, or a JVMCI Compiler. These methods exist outside the Tiered Compilation scheme as they have already been optimized as much as the Java developer has deemed necessary, and as such will not be subject to deoptimization.

This also means that deoptimization can immediately fall back to the C1 compiled code rather than all the way down to the Interpreter, and that no C1 nmethods have to be discarded once the C2 variant is installed, instead remaining as a deoptimization target should it happen.

With Galahad once again reintroducing potential Ahead of Time Compilation for the JIT Compiler it plans to integrate into the JDK, it is also advantageous to consider such a feature, for Galahad to have a ready platform for loading a precompiled Compiler into the VM.

Description

This section is currently a Work in Progress.

Alternatives

Currently, the only Ahead of Time Compiler alternatives are GraalVM native-image, which is coming to the JDK with the advent of Galahad, and Leyden's closed world Java executables. Both produce closed world, standalone binaries which are not in any way related to the execution of Java applications by the Java Virtual Machine, HotSpot, and hence do not have any bearing on this JEP. The jaotc compiler was another Ahead of Time Compiler produced by the GraalVM, but has since been removed in Java 17 due to lack of use. We do not wish to make this feature specific to Graal, as is the case with the jaotc compiler, instead the goal is to make code compiled by C1, C2, or any JVMCI Compiler available for use by the Java VM Ahead of Time.

The work on Coordinated Restore at Checkpoint is another alternative, however the approach it uses of caching previous executions may be quickly invalidated by source code changes and recompilation of the Java application, and it also does not solve the issue of the heavily dynamic optimization and deoptimization cycles that the Java VM tends to be vulnerable to. Additionally, it is also rather situational and does not provide much tuning ability to Java developers, as most, if not all of the performance benefits occur on the device executing the Java application, rather than in a fashion that the developer can control.

Testing

Some extra testing on whether code compiled without an Ahead of Time library executes the same as with said library will be required.

Risks and Assumptions

Special care has to be taken to ensure that only valid code produced by a compatible JVM can be loaded Ahead of Time, as well as security measures to prevent malicious code from a JVMCI Compiler from being installed into the Java VM

Dependencies

This JEP does not yet depend on any other JEPs or Enhancements.