One Spring Boot jar on five JVMs: Temurin, Corretto, Zulu, Semeru and GraalVM under load

The FROM line of a Spring Boot Dockerfile is usually copied from somewhere and never changed. eclipse-temurin is the common choice, but Amazon, Azul, IBM and Oracle’s GraalVM team all publish Java 25 images that run the same jar. I built one notes API, put it in five images that differ only in that line, and ran the same k6 tests against each one on the same machine.

Short version: the gap between the best and worst of Temurin, Corretto and Zulu was under 5% in each of the three tests. Semeru started first and used about a third less memory, but it had the lowest throughput in all three tests. GraalVM Community, used as a normal JIT JVM, finished fourth in all three tests.

Results

Requests per second are k6’s http_reqs rate over each 10-minute test. Latency is k6’s http_req_duration. Every one of the 23,651,594 requests across the 15 runs returned the expected status.

Startup, memory and GC:

These hold for this app on a 2 vCPU machine that the load kept at close to full CPU, with no JVM flags. A bigger machine, a tuned heap or a different workload can reorder the list.

The five runtimes

Four of them are builds of the same OpenJDK source with HotSpot inside. They differ in who builds and tests them, how long updates come, the base OS of the image, and the support you can buy. Semeru keeps the OpenJDK class library but swaps HotSpot for OpenJ9, which has its own JIT and garbage collectors and was designed for a smaller footprint. GraalVM Community runs on HotSpot with C1 as the first tier; its top tier is Graal, a JIT compiler written in Java, in place of C2. Native image is a separate way to run Spring Boot and is not part of this test.

The GraalVM image is a JDK image (jdk-community) on Oracle Linux, and at 1.16 GB it is twice the size of the Temurin JRE image.

The app

A small Spring Boot 4.1.2 notes API: Spring Web, Spring Data JPA, Flyway, Hikari, Actuator with the Prometheus registry. The table and its one index:

create table notes (
  id bigserial primary key,
  title text not null,
  body text not null,
  tag text not null,
  created_at timestamptz not null
);
create index notes_tag_id on notes (tag, id desc);

The endpoints the tests call:

@GetMapping("/{id}")
ResponseEntity<Note> get(@PathVariable long id) {
    return ResponseEntity.of(notes.findById(id));
}

@GetMapping
List<Note> byTag(@RequestParam String tag) {
    return notes.findByTagOrderByIdDesc(tag, PageRequest.of(0, 20));
}

@PostMapping
@ResponseStatus(HttpStatus.CREATED)
Note create(@Valid @RequestBody NoteRequest r) {
    return notes.save(new Note(r.title(), r.body(), r.tag()));
}

@PutMapping("/{id}")
@Transactional
ResponseEntity<Note> update(@PathVariable long id, @Valid @RequestBody NoteRequest r) {
    return notes.findById(id).map(n -> {
        n.setTitle(r.title());
        n.setBody(r.body());
        n.setTag(r.tag());
        return ResponseEntity.ok(n);
    }).orElse(ResponseEntity.notFound().build());
}

application.yaml sets a pool of 20 and puts the JVM name on every metric, so one Grafana query can split by runtime:

spring:
  datasource:
    url: ${DB_URL:jdbc:postgresql://localhost:5432/notes}
    hikari:
      maximum-pool-size: 20
  jpa:
    open-in-view: false
    hibernate:
      ddl-auto: validate
management:
  server:
    port: 9091
  endpoints:
    web:
      exposure:
        include: health,prometheus
  metrics:
    tags:
      jvm: ${JVM_NAME:unknown}

One Dockerfile, five images

The jar is built once. The base image is a build argument, so the five images contain the same notes.jar byte for byte:

ARG BASE
FROM ${BASE}
ARG JVM
ENV JVM_NAME=${JVM}
WORKDIR /app
COPY target/notes.jar notes.jar
EXPOSE 8080 9091
ENTRYPOINT ["java", "-jar", "notes.jar"]

./mvnw -q package -DskipTests
docker build --build-arg BASE=eclipse-temurin:25.0.5_8-jre-noble --build-arg JVM=temurin -t notes-temurin .
docker build --build-arg BASE=amazoncorretto:25.0.5-al2023-headless --build-arg JVM=corretto -t notes-corretto .
# … same for semeru, zulu, graalvm

No -Xmx, no GC choice, no JAVA_TOOL_OPTIONS. Each JVM picks its own heap size and collector from what it sees: 2 CPUs and 4 GB, the whole app machine.

The setup

Three DigitalOcean CPU-Optimized droplets in London, on one private network:

The load generator and the monitoring stack sit on their own machine, so they take no CPU from the app.

For each JVM, one script on the load machine does the same steps:

1. Remove the previous container, drop the page cache on the app machine, drop the notes table, checkpoint Postgres.
2. Start the container and poll GET /notes?tag=java every 50 ms until it returns 200. That time is the startup number.
3. Insert 200,000 notes with generate_series, then vacuum analyze.
4. Wait 60 seconds.
5. Run three k6 tests, 10 minutes each, 100 virtual users, with 15 seconds between them: read, then write, then mixed.

The order matters. The mixed test always starts after 20 minutes of load, so every JVM has had time to compile its hot code by then.

The k6 script

constant-vus with 100 users and no sleep, so each user sends its next request as soon as the last one returns. The three modes:

export const options = {
  scenarios: { [MODE]: { executor: 'constant-vus', vus: 100, duration: '10m' } },
  tags: { jvm: __ENV.JVM, mode: MODE },
  discardResponseBodies: true,
  summaryTrendStats: ['avg', 'p(50)', 'p(95)', 'p(99)', 'max'],
  // no `url` tag: every /notes/{id} would become its own Prometheus series
  systemTags: ['status', 'method', 'name', 'scenario', 'expected_response', 'check'],
};

function read() {
  if (Math.random() < 0.7) check(http.get(`${BASE}/notes/${id()}`, { tags: { name: 'GET /notes/{id}' } }), { ok: (r) => r.status === 200 });
  else check(http.get(`${BASE}/notes?tag=${tag()}`, { tags: { name: 'GET /notes?tag' } }), { ok: (r) => r.status === 200 });
}

k6 streams its metrics to Prometheus with remote write (-o experimental-prometheus-rw) and also writes a summary file per test with --summary-export. The tables above come from those summary files.

The read test: a race against the JIT

The 10-minute average hides most of what happened. Here is the first minute against the last minute of the read test:

Every JVM more than doubled its throughput during the test, and the code and machine stayed the same. During the read test the app container used about 1.75 of its 2 cores the whole time. Postgres read every row from its cache (blks_read stayed at 0) and its CPU stayed under 16%. The limit was the app’s own CPU.

Micrometer’s jvm_compilation_time_ms_total shows where that CPU went. As a rate it reads as milliseconds of compiling per second of wall time:

Around 1,000 ms per second means a compiler thread was busy almost all the time on a machine with two cores. On the three HotSpot JVMs the JIT kept compiling through the entire read test and only settled near the end of the write test. In the mixed test they were close to idle after two minutes, and that is where the three of them reached their highest numbers.

GraalVM’s Graal compiler kept working through the write test and most of the mixed test. It reached 3,669 req/s in the last minute of the mixed test, close to the others, but its 10-minute averages stayed fourth. Semeru’s JIT followed a similar pattern through the write test, then went quiet in the mixed test, at a lower level than the HotSpot three.

The Semeru read cell is empty because Prometheus missed most actuator scrapes on that run while both cores were busy, so there aren’t enough points for that window. The k6 numbers come from the load machine and are complete.

Writes: the pool is the queue

In every test, Hikari’s hikaricp_connections_pending averaged 55 to 65 waiting requests, and 71 to 73 in the write test. 100 virtual users share 20 connections, so most requests spend part of their time waiting for a connection. That queue looks the same for every JVM, and in the write test the gap between Semeru and the HotSpot JVMs got much smaller: 1,958 req/s against 2,745–2,861, with a p99 of 226 ms against 194–205 ms.

Memory and GC

The memory column is the container’s RSS from cAdvisor, averaged over the mixed test. The four HotSpot-based JVMs sat at 413–421 MB. Semeru used 280 MB with the same jar and the same load, about a third less.

The cost showed up in GC: Semeru spent 27.7 ms of every second in GC pauses (jvm_gc_pause_seconds), against 7.7–9.6 ms for the others. OpenJ9’s default GC policy, gencon, kept the used heap smaller (71 MB average used heap against 87–98 MB) and collected more often.

What I’d take from it

• If you run on Temurin, Corretto or Zulu, this test gives you no reason to switch. Pick between them on update schedule, support, licence terms and base image.
• Semeru is worth a test where memory is the limit, for example many small containers per node. On a CPU-bound 2 vCPU box it was the slowest.
• Run as a JIT JVM, GraalVM had the highest compile time of the five, and its averages trailed Temurin, Corretto and Zulu in read, write and mixed. Its native image mode is a different trade-off and needs its own test.
• Warm-up dominated everything on a small, busy machine. If your pods restart often or scale from zero, the first minutes matter more than the 10-minute average.

What this doesn’t cover

One app, one machine size, default flags, one run per JVM. No -XX:+UseZGC, no CDS or AOT cache, no CRaC, no native image, no -Xmx tuning, no larger instances where the JIT has spare cores. Any of those can change the ranking. The numbers here say what each image does out of the box with this jar on two busy cores.

Sources

• Eclipse Temurin (Adoptium)
• Amazon Corretto
• Azul Zulu builds of OpenJDK
• IBM Semeru Runtimes
• Eclipse OpenJ9
• GraalVM
• k6: constant VUs executor
• HikariCP