JEP 544: Ahead-of-Time Code Compilation

Summary

Improve startup and warmup clip by making optimized autochthonal codification for an exertion instantly disposable erstwhile the HotSpot Java Virtual Machine starts. Achieve this by compiling exertion codification to autochthonal codification successful a training run, storing the autochthonal codification successful the AOT cache for usage successful consequent accumulation runs. If the workload changes successful production, regenerate autochthonal codification dynamically for continued highest performance, providing the champion of some ahead-of-time (AOT) and just-in-time (JIT) compilation.

Goals

• Enable applications to execute highest capacity much quickly.

• Enable applications to prolong highest capacity moreover arsenic workloads change.

• Do not require immoderate alteration to the codification of applications, libraries, aliases frameworks.

• Do not require immoderate alteration to the configuration of HotSpot, beyond requesting the usage of the AOT cache.

• Continue to support the Serial, Parallel, G1, and ZGC garbage collectors.

• Do not present caller AOT workflows, but, rather, widen the existing AOT cache creation workflow.

• Ensure that shifting from AOT-compiled codification to JIT-compiled codification is invisible to applications.

• Support the AArch64 and x64 processor architectures.

Non-Goals

• It is not a extremity to supply an AOT-only mode. Applications will usage some AOT-compiled codification and JIT-compiled codification successful the aforesaid run, automatically transitioning betwixt them arsenic needed.

• It is not a extremity to support cross-compilation. Code compiled successful a training tally must tally connected the aforesaid CPU architecture, pinch the aforesaid group of features, successful consequent accumulation runs.

• It is not a extremity to support each CPU architectures presently supported by HotSpot. We expect normal porting activities to yet adhd support for each awesome architectures.

Motivation

When a Java exertion is tally by the HotSpot JVM, it passes done 3 phases: It starts up, past it warms up, and past it reaches highest performance.

During startup, HotSpot invokes the application's main method and loads, links, and initializes classes connected demand. Initially, it runs some exertion and JDK room codification via the bytecode interpreter, which is slow. Inside the interpreter, HotSpot profiles the application's behaviour by counting events specified arsenic method invocations and loop iterations. It uses the floor plan information to prime frequently-invoked methods, aliases hot spots, which it compiles to autochthonal codification via the basal C1 compiler. This autochthonal codification is only modestly optimized.

During warmup, the exertion settles into its workload and the loading, linking, and initialization of classes ramps down. HotSpot continues to floor plan the application, some successful the bytecode expert and via instrumentation codification inserted by C1. It collects richer floor plan information, including not conscionable method-invocation and loop-iteration counts but besides the types of objects encountered. As the floor plan information accumulates complete time, it becomes much statistically useful. Eventually, HotSpot uses the information to prime the hottest methods, which it compiles to autochthonal codification via the precocious C2 compiler. This autochthonal codification contains nary instrumentation and is highly optimized.

Profiling the exertion and generating autochthonal codification is not free. Not only is the bytecode expert slow, but instrumented autochthonal codification is slower than uninstrumented autochthonal code. Compiling methods to autochthonal codification requires some CPU clip and representation that could different beryllium utilized by the application, moreover though HotSpot only compiles methods to autochthonal codification erstwhile floor plan information indicates that doing truthful will beryllium worthy the effort. Gradually, however, JIT compilation catches up to the application's emerging basking spots and the exertion runs faster. Eventually, each basking methods are compiled to afloat optimized autochthonal codification and the compilers spell idle.

The exertion remains successful this authorities of peak performance arsenic agelong arsenic its basking spots do not change. The application's basking spots whitethorn change, however, successful consequence to changes successful its workload. When that happens, HotSpot tin dynamically deoptimize, by discarding antecedently generated autochthonal codification arsenic needed, and reoptimize, by generating caller autochthonal codification for newly-hot methods. For example, if an exertion initially receives 2 types of requests past HotSpot dynamically optimizes the codification for those 2 petition types. If the exertion starts receiving a 3rd type of request, HotSpot tin dynamically deoptimize and past reoptimize the codification for each 3 types of requests. The exertion tin pass, successful effect, done different warmup phase, sustaining capacity arsenic the application's workload changes.

What astir fixed compilation?

Static compilation has sometimes been projected arsenic an replacement to the move compilation of Java code. A fixed compiler converts full applications to autochthonal codification ahead-of-time, anterior to tally time.

Static compilation has immoderate advantages complete move compilation. A statically compiled exertion starts up and reaches highest capacity immediately, without a warmup phase. At tally clip location is nary request for a bytecode interpreter, profiling, aliases compilation. Peak capacity tin moreover beryllium competitory pinch HotSpot if the fixed compiler's optimization activity is guided by meticulous profiles gathered during anterior runs.

Dynamic compilation, however, has 3 cardinal advantages complete fixed compilation.

First, move compilation makes applications agile because it responds to changes successful the application's basking spots. It deoptimizes and reoptimizes arsenic needed, sustaining capacity arsenic the application's workload changes. A statically compiled exertion cannot respond successful this measurement — by its nature, it tin beryllium optimized for only 1 group of basking spots.

Second, move compilation makes applications portable crossed varying hardware and package because it generates autochthonal codification astatine tally clip that is circumstantial to the run-time environment. If an exertion is redeployed connected a different processor architecture, a processor pinch a different characteristic set, a different operating system, aliases a different type of the JDK, HotSpot will execute highest capacity for that situation without requiring immoderate alteration to the application. A statically compiled exertion must beryllium recompiled successful the look of specified changes.

Finally, move compilation is compatible pinch the move quality of the Java Platform. Features specified arsenic move people loading, move linkage, move dispatch, and move reflection bring immense expressive power, and person been basal to the platform's success. HotSpot handles these features naturally, while fixed compilers struggle pinch them. Even heroic amounts of fixed study cannot dress up for the truth that these features require galore decisions to beryllium made astatine tally time. Implementors of fixed compilers for Java codification person truthful resorted to incompatible constraints, specified arsenic closed-world assumptions, and to putting important burdens connected developers, specified arsenic having to place successful beforehand the classes eligible for reflection.

Shifting compilation activity to training runs

Throughout the startup and warmup phases, HotSpot continuously juggles aggregate balls: It runs exertion and JDK room code; it loads, links, and initializes classes connected demand; it profiles the application's execution; and it compiles basking methods to autochthonal codification pinch varying degrees of optimization, guided by the floor plan data.

The thesis of Project Leyden is that the cardinal to improving startup and warmup clip is to do immoderate of this activity earlier, up of time, alternatively than conscionable successful time. We displacement activity earlier successful clip by doing it successful a training run, storing the results of the activity successful the AOT cache for instant usage successful consequent production runs.

We shifted people loading and linking activity earlier successful clip via JEP 483, delivered successful JDK 24. The AOT cache stores the loaded and linked forms of classes from the training run, thereby improving startup time.

We shifted profiling activity earlier successful clip via JEP 515, delivered successful JDK 25. The AOT cache stores the execution profiles of methods invoked successful the training run, enabling the C2 compiler to tally instantly astatine the commencement of accumulation runs, thereby improving warmup time.

These improvements laid the instauration for our eventual goal, which is to displacement compilation and optimization activity earlier successful time. The AOT cache will shop optimized autochthonal codification compiled successful the training run, enabling HotSpot to load that codification instantly, alternatively than having to recompile it astatine the commencement of each accumulation run. This will amended some startup and warmup time.

HotSpot will not ever usage the cached code; if the application's workload changes past HotSpot can, arsenic usual, deoptimize and reoptimize, generating caller autochthonal codification for newly-hot methods successful bid to prolong performance. Thus Java applications will summation immoderate of the benefits of fixed compilation while retaining the agility, portability, and compatibility of move compilation.

Description

We widen the existing AOT cache to shop optimized autochthonal codification generated successful a training run. Such cached codification is known arsenic AOT code. During a accumulation run, a petition for optimized codification for a method tin beryllium fulfilled instantly if matching AOT codification is recovered successful the cache. If AOT codification is unavailable, incompatible, different unsuitable, aliases later deoptimized, execution falls backmost to the existing expert and JIT mechanisms. AOT codification and JIT codification tin coexist and are wholly interoperable since they are created by the aforesaid compilers, C1 and C2.

To create an AOT cache, usage the AOTCacheOutput action to do a training tally of your exertion and make AOT code:

$ java -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

This workflow is unchanged from erstwhile releases. The AOT cache successful the record app.aot, however, now contains not conscionable pre-linked classes and profiling information but besides AOT codification for selected basking methods. Subsequently, successful production, you tin tally the exertion pinch the cache:

$ java -XX:AOTCache=app.aot -cp app.jar com.example.App ...

No further options aliases settings are required to make aliases usage AOT code. HotSpot creates AOT codification and stores it successful the cache by default. It continues to shop floor plan information successful the cache arsenic well, to beryllium utilized for sequencing the loading of AOT codification and for guiding the consequent procreation of JIT code.

Performance

To measure the startup use of AOT code, we ran 5 benchmark applications built pinch celebrated Java frameworks. We ran them connected a two-core Linux/x64 strategy truthful arsenic to emulate a microservice mounting successful which the JIT compiler is apt to compete pinch the exertion for CPU time, thereby expanding startup time:

Without AOT code, the AOT cache reduces the startup clip of these applications by astir 50% to 70%; pinch AOT code, the cache reduces their startup clip by astir 65% to 80%.

To measure the warmup use of AOT code, we ran a javac benchmark exertion which many times compiles the aforesaid 50 root files, 20 times, measuring the clip required for each iteration:

In each curve, the first loop shows the startup-time improvement: The AOT cache without AOT codification improves startup clip by astir 30%; adding AOT codification brings an further 45% improvement, for a full of astir 75%. Successive iterations show the warmup phase, during which HotSpot compiles the hottest methods: The loop clip tends to alteration and past scope a dependable authorities arsenic the value of the autochthonal codification improves and the compilers decorativeness their work. The curve for the AOT cache without AOT codification decreases much quickly than the curve for nary AOT cache, yet reaching astir the aforesaid dependable state. The curve for the cache pinch AOT codification is already adjacent to the dependable authorities by the 4th iteration. The area betwixt the apical curve and the bottommost curve represents the full warmup-time improvement.

Information connected each these benchmarks, including tally instructions and links to root code, is disposable here.

Differences betwixt AOT codification and JIT code

AOT codification and JIT codification tin beryllium different, since training runs and accumulation runs tin beryllium different.

One root of differences is the truth that the bid successful which classes are initialized tin disagree betwixt training and accumulation runs, particularly if the workload differs. A method that accesses a fixed section aliases invokes a fixed method successful different people must guarantee that the people is initialized. When generating AOT codification pinch C2, HotSpot truthful compiles 2 versions of specified methods: A slow type contains other codification to guarantee the initialization of referenced classes, while a accelerated type does not incorporate that codification and frankincense tin beryllium amended optimized. HotSpot uses the slow type initially, and past switches to the accelerated type erstwhile each referenced classes are initialized.

Another root of differences is the truth that a fixed last field's worth tin alteration from tally to run; it might, for example, beryllium initialized pinch the existent day and time. When just-in-time compiling a method that refers to specified a field, the people containing the section will person been initialized, truthful the field's worth will beryllium known and C2 tin dainty it arsenic a compile-time constant, embedding it straight successful autochthonal code. When ahead-of-time compiling the aforesaid method, however, nary classes will person been initialized, truthful the field's worth will not beryllium known and C2 cannot dainty it arsenic a compile-time constant; it must make codification that explicitly loads the field.

Despite these differences, AOT codification still delivers important capacity benefits successful a mode that is transparent to applications. As always, astatine tally clip the compilers tin make JIT codification to switch AOT codification that fails to property well.

Consistency of training and accumulation runs

To bask the benefits of the AOT cache generated successful a training run, the training tally and each consequent accumulation runs must beryllium fundamentally similar, arsenic described successful JEP 483.

If an AOT cache contains AOT code, that codification is utilized erstwhile 2 further constraints are met:

• All runs usage CPUs of the aforesaid architecture and pinch the aforesaid features. For example, AOT codification generated for an x64 CPU pinch the AVX-512 vector-instruction characteristic will not tally connected an x64 CPU without that feature.

• All runs usage the aforesaid garbage collector, since AOT codification contains GC-specific read/write barriers.

If these constraints are not met past HotSpot issues a informing connection and does not load the AOT code, falling backmost to the accustomed expert and JIT mechanisms. It still uses the different accusation successful the AOT cache, namely the loaded and linked classes and the profiling data. In that case, the exertion whitethorn commencement up and lukewarm up much slowly, but its execution will still beryllium correct and it will still, eventually, execute highest performance.

Observing AOT cache usage successful production

You tin observe whether AOT codification is loaded successful a accumulation tally via the existing HotSpot action PrintCompilation, which now reports connected some AOT codification loading and JIT compilation:

$ java -XX:+PrintCompilation \ -XX:AOTCache=app.aot -cp app.jar com.example.App ...

You tin besides cheque whether an AOT cache containing AOT codification is usable successful a peculiar accumulation environment. The action AOTMode=required causes HotSpot to study an correction and exit if the AOT cache violates immoderate constraint:

$ java -XX:AOTMode=required \ -XX:AOTCache=app.aot -cp app.jar com.example.App ...

(This action was primitively AOTMode=on, later renamed for clarity to AOTMode=required.)

Finally, you tin disable the loading of AOT codification via the diagnostic action AOTCodeCaching:

$ java -XX:+UnlockDiagnosticVMOptions -XX:-AOTCodeCaching \ -XX:AOTCache=app.aot -cp app.jar com.example.App ...

You tin usage this action to measure the capacity effects of AOT codification successful a accumulation run, aliases to debar constraint-violation errors erstwhile utilizing AOTMode=required and the AOT codification is unusable but the different accusation successful the cache is usable.

Controlling the procreation of AOT codification successful training

We have, frankincense far, spoken of an AOT cache being created successful a training run, successful a azygous measurement via the AOTCacheOutput action arsenic shown above. In fact, creating an AOT cache takes two chopped steps: HotSpot runs erstwhile successful grounds mode to shop observations of your application's behaviour successful an AOT configuration, and past again successful create mode to combine an AOT cache from that configuration, which includes compiling AOT code. Using the AOTCacheOutput action runs the 2nd measurement transparently, but you tin invoke each measurement explicitly via the AOTMode and AOTConfiguration options:

$ java -XX:AOTMode=record -XX:AOTConfiguration=app.aotconf \ -cp app.jar com.example.App ... $ java -XX:AOTMode=create -XX:AOTConfiguration=app.aotconf \ -XX:AOTCache=app.aot

HotSpot provides many options to power the behaviour of its compilers, C1 and C2. These options uniformly govern the procreation of some JIT code, astatine tally time, and AOT code, successful the assembly step. For example, this bid statement reports each JIT compilation activity during the training tally due and each AOT compilation activity arsenic the cache is created:

$ java -XX:+PrintCompilation \ -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

To spot AOT compilation activity but not JIT compilation activity, usage the existing situation adaptable JDK_AOT_VM_OPTIONS to walk the PrintCompilation action only to the assembly step:

$ JDK_AOT_VM_OPTIONS='-XX:+PrintCompilation' \ java -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

Alternatively, you tin tally the 2 steps yourself, specifying the PrintCompilation action only successful the 2nd step.

Finally, you tin disable the creation of AOT codification successful a training tally via the diagnostic action AOTCodeCaching:

$ java -XX:+UnlockDiagnosticVMOptions -XX:-AOTCodeCaching \ -XX:AOTCacheOutput=app.aot -cp app.jar com.example.App ...

You tin usage this action to measure the size effect of AOT codification connected the AOT cache — cached codification tin make the AOT cache importantly larger.

For much accusation connected each of the AOT-related command-line options, spot the manual page for the java command.

Future Work

• Investigate minimizing bytecode mentation and JIT compilation successful favour of near-total reliance connected AOT code. Initial experiments propose that minimizing expert usage results successful overly ample AOT cache files, which tin return much clip to load than conscionable moving the interpreter. Likewise, minimizing JIT compilation often leads to little highest performance. This attack whitethorn person constricted applicability if, for illustration fixed compilation, it fails to unrecorded up to personification expectations.

• HotSpot has a wide scope of fine-grained options for controlling its compilers. Based connected acquisition pinch this feature, tune the default values of existing options to use much gracefully to AOT code. Also see defining caller options, specified arsenic options for definitive AOT cache size management.

• Consider an action that would alteration giving up immoderate performance, aliases accepting larger AOT cache files, aliases both, successful bid to summation portability crossed processors of the aforesaid architecture but pinch different characteristic sets.

Testing

• We will create caller portion tests for this feature. They will guarantee that AOT code, if present, behaves correctly.

• We will tally existing AOT cache tests pinch this characteristic enabled and guarantee that they pass.

• Initially, only AArch64 and x64 processors will beryllium supported. Unit tests will beryllium adjusted appropriately to let for the absence of AOT codification connected different architectures.

Risks and Assumptions

• There are nary caller risks beyond those already noted successful JEP 483.

• We presume that HotSpot's organizing rule is still sound: A Java exertion should beryllium compiled, astatine tally time, to favour the application's existent behavior, exploiting each of the processor features available. Because of this principle, Java codification automatically runs amended connected caller hardware.

• The guidelines presumption of the AOT cache besides remains operative: A training tally is assumed to beryllium a bully root of observations that, erstwhile passed done an AOT cache to a accumulation run, will use the capacity of the accumulation run. This presumption applies afloat to AOT code, which benefits akin accumulation runs while not doing harm to divergent accumulation runs, which tin usage JIT compilation to make different code.