JEP draft: Prepare for Native Memory Tracking in the JDK

Summary

Native Memory Tracking (NMT) is a feature of the Hotspot JVM that enables the user to separate native memory allocations into categories and aids in diagnosing memory issues via reports. Extend NMT to support attributing native memory allocations from JDK core libraries to dedicated categories. This will enable better diagnosis of native memory issues for users.

Goals

• Lay the foundation for JDK core libraries to adopt NMT.

Non-Goals

• It is not a goal for any particular JDK core native library to adopt NMT. Integration into the core libraries is future work.

Motivation

The JVM provides automatic heap memory management for objects using garbage collection. Each newly allocated object is tracked by the GC so that its memory can be reclaimed when the object is no longer reachable. GC has freed Java programmers from needing to explicitly track their memory allocations and object lifecycles thereby preventing entire classes of memory related bugs - such as memory leaks - by design.

The overall memory usage of a Java application cannot be understood solely in terms of the size of the Java heap as the JVM also allocates native memory to support its operation and to hold its data structures. Java applications, including the JDK core class libraries, also allocate native memory through the use of native libraries. Many important core libraries - such as those providing file I/O, networking, compression and cryptography - use native memory. Diagnosing memory-related issues in a Java program benefits from having a complete picture of the total memory usage of the program, including both Java heap and native memory.

The Java ecosystem provides extensive tooling to analyze and categorize heap memory, but tools for understanding native memory allocations are more limited. Since Java 8, the JVM has included the NMT subsystem for tracking and understanding the JVM's native memory allocations. NMT categorizes the native memory used by different JVM components, such as class metadata, thread stacks, and the code cache, and reports the current amount of memory being used as well as the number of allocations, and the categories' peak memory usage.

By categorizing and reporting on these native memory details, NMT can help developers diagnose the source of native memory leaks and high memory usage from the JVM in their applications. However, as NMT is only available to the JVM, all native allocations done by Java class libraries in the JDK is invisible to NMT. Unfortunately, this means that diagnosing memory issues in Java applications today is more difficult than it needs to be. Extending NMT to the JDK core libraries would help alleviate this pain point.

Description

We extend NMT to enable the core libraries to register their own memory categories, and have all their native allocations tracked with those categories. As core libraries integrate with NMT, reports from jcmd and JFR will show new categories for the native memory usage of those JDK libraries. No user changes are required - users will simply benefit from enhanced visibility into the source of native allocations in their application.

NMT's current lack of insight in native allocations done by the core libraries can be observed by running a Java program that performs native allocations with NMT enabled. An example of such a program is the HTTP server available in the jdk.httpserver module. Under the hood, the server is using NIO, which performs native allocations. Because the allocation request comes from a Java program, NMT will correctly account that a native allocation has taken place, but is unable to categorize it correctly. By running the following program under JDK25 using java -XX:NativeMemoryTracking=summary BasicHttpServer.java, we can produce an NMT report which let's us see this problem.

import module java.base;
import module jdk.httpserver;

void main() throws IOException {
    var port = 8000;
    var server = HttpServer.create(new InetSocketAddress(port), 0);
    server.createContext("/", new HelloHandler());
    server.setExecutor(null);
    var pid = ProcessHandle.current().pid();
    IO.println("PID: " + pid);
    IO.println("Server started at http://localhost:" + port);
    server.start();
}

class HelloHandler implements HttpHandler {
    public void handle(HttpExchange exchange) throws IOException {
        var response = "hello world!";
        exchange.sendResponseHeaders(200, response.length());
        try (var os = exchange.getResponseBody()) {
            os.write(response.getBytes());
        }
    }
}

Take note of the PID of the program, and now, in a separate terminal, you can request an NMT report.

term0 $ java -XX:NativeMemoryTracking=summary BasicHttpServer
PID: 12345
Server started at http://localhost:8000

term1 $ jcmd 12345 VM.native_memory summary
...
-                    Symbol (reserved=1379KB, committed=1379KB)
                            (malloc=923KB #3166) (at peak)
                            (arena=456KB #1) (at peak)
-                     Other (reserved=10KB, committed=10KB)
                            (malloc=10KB #2) (at peak)
...

In this heavily trimmed example, there are two categories: the Symbol category and the Other category. The Symbol category contains Java strings which have been interned by the VM. It has 3166 allocations made by malloc and these allocations are spread over a total usage of 923KB. When an allocation has not had a category provided, it is placed into the Other category. The Other category has 10KB allocated over 2 allocations, but where these allocations come from is not clear. In other words, NMT can see the allocations, but is unable to categorize them correctly. In this example, we can assume that it comes from the HTTP server, but in a real application this category can be filled with memory from different sources, and be much larger. If we can ensure that all allocations may be provided with a category, then this Other category will cease to exist and we will be left with only detailed native allocation categories.

With the JEP implemented, the HTTP server module can now integrate with NMT. After doing so, an NMT report will show the HTTP server's native memory allocations in the correct categories rather than in the Other category. Let's assume that the HTTP server uses a single category, named httpserver, then its native memory usage will be correctly attributed as shown in the trimmed report below:

$ java -XX:NativeMemoryTracking=summary HelloWorldLoop.java &
[1] 12345
$ jcmd 12345 VM.native_memory summary
Native Memory Tracking:

(Omitting categories weighting less than 1KB)

Total: reserved=9760301KB, committed=614769KB
       malloc: 24293KB #43040
       mmap:   reserved=9736008KB, committed=590476KB
-                      Code (malloc=264KB #2372) (peak=264KB #2373)
-                    Symbol (malloc=3442KB #26691) (at peak)
-                httpserver (malloc=10KB #2) (at peak)
-                     Other (malloc=0KB #0) (at peak)

Future Work

• Integrate relevant core libraries with NMT. We expect this work to happen incrementally.
• Consider extending NMT to all libraries, not only JDK ones.