The JIT Compiler

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

JIT compiler

Just-in-time compiler: it turns bytecode into machine code while the program runs, for the parts that run often.

Interpreter

The part of the JVM that runs bytecode one instruction at a time, without compiling it. Slow, but starts instantly.

C1 and C2

HotSpot's two JIT compilers. C1 compiles quickly with simple optimisations; C2 compiles slowly but produces much faster code.

Tiered compilation

Running code first in the interpreter, then C1, then C2 as it gets hotter.

Profile

Statistics the JVM collects while code runs: how often branches are taken, which classes appear at each call.

Inlining

Copying a called method's body into the caller, so there is no call left and both can be optimised together.

Escape analysis

Checking whether an object can be seen outside the method that created it. If not, the JIT may skip creating it on the heap.

Deoptimisation

Throwing away compiled code whose assumptions turned out wrong and going back to the interpreter.

OSR (on-stack replacement)

Switching a method that is in the middle of a long loop from interpreted to compiled code without waiting for it to return.

Warm-up

The early period after start-up when code is still interpreted or lightly compiled, so the program runs slower.

1. 1

   See the tiers for real

   Run the first example on your own JDK with java -XX:+PrintCompilation Main.java | grep Main::. Find the lines for Main::square and note which tiers appear. Then run with -Xint and with -XX:TieredStopAtLevel=1 and see what changes in the compilation log.

2. 2

   Make a call site megamorphic

   Add a fourth shape, record Circle(int r), to "What the profile sees at a call site", and call totalArea 100,000 times with only squares first, then once with a circle. Run with -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining -XX:+PrintCompilation and look for made not entrant after the circle appears.

3. 3

   Watch escape analysis

   Make n 50,000,000 in "Objects that escape and objects that don't", call only sumLocal, and run with -Xlog:gc. Then add -XX:-DoEscapeAnalysis and run again. Predict which run shows more young GCs (the second: every Point becomes a real allocation).

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

long start = System.nanoTime();
for (int i = 0; i < 100_000_000; i++) {
    Math.sqrt(i);                      // result thrown away
}
System.out.println((System.nanoTime() - start) / 1_000_000 + " ms");

1. 1

   javac produces portable bytecode and does almost no optimisation. The JVM's Just-In-Time (JIT) compiler does the optimising at run time, when it can see the real CPU and the real behaviour of the program. HotSpot has two JIT compilers: C1 (the "client" compiler) compiles fast with simple optimisations, and C2 (the "server" compiler) compiles slowly but produces highly optimised code. Tiered compilation (the default since Java 8) uses both.

2. 2

   The tiers: 0 interpreter, 1 C1 without profiling (for trivial methods), 2 C1 with light profiling, 3 C1 with full profiling, 4 C2. A typical hot method goes 0 -> 3 -> 4. Promotion is driven by counters of invocations and loop back-edges (jumps back to a loop's start), with defaults such as Tier3InvocationThreshold=200 and Tier4InvocationThreshold=5000, adjusted for how busy the compiler threads are. A long-running loop inside a method that is called once is compiled while it runs, by on-stack replacement (OSR).

3. 3

   Inlining is the most important optimisation, because it opens the door to all others: the callee's body is copied into the caller, removing the call and letting the compiler optimise both together. C2 inlines small methods always (bytecode under MaxInlineSize, 35 bytes) and hot methods up to FreqInlineSize (325 bytes on x86-64), up to a nesting depth. Small methods such as getters cost nothing in hot code; giant methods don't get inlined and are harder to optimise.

4. 4

   Java calls are virtual by default, so how can C2 inline shape.area()? With the profile: if the call site has only ever seen Square (monomorphic), C2 inlines Square.area() behind a cheap class check; with two types (bimorphic) it inlines both; with three or more (megamorphic) it usually makes a real virtual call. If the check ever fails, the compiled code hits an uncommon trap, the method is deoptimised (marked "not entrant", execution continues in the interpreter) and later recompiled with the new profile. Class Hierarchy Analysis (CHA) also lets C2 inline a method that no loaded subclass overrides, and deoptimise if one is loaded later.

5. 5

   Escape analysis asks whether an object can be seen outside the method (or thread) that created it. If it can't escape, C2 can scalar-replace it (keep its fields in registers, so no heap allocation at all), and remove locks on it (lock elision). Other C2 optimisations include constant folding, dead-code elimination, loop unrolling, range-check elimination, auto-vectorisation with SIMD instructions, and intrinsics: hand-written machine code for methods such as System.arraycopy, Math functions, String.equals and Arrays.fill.

6. 6

   Consequences for you: a Java program has a warm-up period during which it's slower; measuring with System.nanoTime around a single call gives wrong answers (use JMH, the Java Microbenchmark Harness); compiled code lives in the code cache (Topic 14.2); and -Xint (interpreter only) or -XX:TieredStopAtLevel=1 (C1 only, faster start-up) change the trade-off. GraalVM offers an alternative JIT written in Java and native-image ahead-of-time compilation; the experimental Graal JIT that shipped inside OpenJDK 10-16 was removed in JDK 17.

• ↔

  JIT compilation gives peak performance tuned to real behaviour, at the cost of warm-up time and CPU at start-up; AOT (native images) flips that trade-off.

• ↔

  Speculative optimisation is fast in the common case but causes deoptimisation hiccups when behaviour changes.

• ↔

  C1-only (TieredStopAtLevel=1) suits short-lived processes; full tiered compilation suits long-running servers.

• Done when you can describe tiers 0 to 4 and what moves code between them.

• Done when you can explain inlining, monomorphic/bimorphic/megamorphic call sites and deoptimisation.

• Done when you can explain escape analysis and scalar replacement.

• Done when you can read -XX:+PrintCompilation output.

• Done when you can explain why benchmarks need JMH and warm-up.