JEP draft: provide stable USDT probe points on JVM compiled methods

FIXME: ADD JEP STRUCTURE

Patchable probe points in user code are an increasingly important "hook" for on-line, at-scale debugging and telemetry in cloud settings. Standard mechanisms like USDT enable compilers to insert probes for method entry and exits, which on-line tools (dtrace, BPF) can discover and manipulate. The handshake between compilers and tools is often special sections or other metadata in ELF files, which are static artifacts (hence the S in USDT).

The JVM does not cooperate with these technologies at present, but it can and should.

The problem with the JVM is mainly that it generates hot compiled code dynamically, which means that there is no statically generated ELF file for tools to consult. In addition, the JVM does its own patching frequently and freely over code it generates, which means that, even if an external patching tool could discover the address of a JVM compiled method, installing an external patch on any instruction is certain to fail badly. To allow the JVM to interoperate with instruction patching tools, it must manage all patching of compiled methods. To allow external tools to do their work, the JVM must be able to create (on demand) patch points which are safely patchable by external tools.

Therefore, we need to adapt or extend JVMTI-type APIs to allow probe tools to perform suitable operations on hot compiled code, including some combination of the following:

• enumerating of current compiled methods
• notifying on load of compiled methods (and all other state changes)
• querying over current compiled methods
• querying over features inside those methods, e.g. inlined callees.
• adding patch points for method entry (and all other state changes)
• adding patch points to particular frames for method return (etc.)
• filtering by caller and by argument or return value
• collecting partial backtraces
• collecting statistics (counts, histograms, etc.)
• turning families of trace points on and off
• inspecting chunks of compiled method code

Many of these features are supplied by ad hoc coding in the tool which uses the provided USDT hooks. For example, BPF has a whole language for examining live data at a function entry or exit point, and deciding whether to post and event or collect a statistic, based on that data. Thus, the JVM does not need to support every one of the above features directly if it can provide the right low-level hooks.

The minimal set of hooks required from the JVM is something like this:

• A concept of "patch point", a set of tool-supplied instructions (often a single "nop") through which control passes when a defined event occurs.
• APIs to create (on the fly) patch points for relevant events: compiled method load, method entry, frame exit, frame throw, frame deopt, frame OSR (one for each distinct compiled method)
• A discovery API to find potential and existing patch points.
• A patch point API to safely hook, unhook, enable, disable, and discard patch points or groups of patch points.
• Utility functions (usable by patch code) to parse relevant JVM structures (frames, objects, code blocks, etc.)

Two typical events would be "method Foo::bar was called", or "method Foo::baz was called". Note that these are independent events with separate patch points. Patch points are not created eagerly, but only on a request from an external tool. Types of events would include all kinds of entries and exits from JVM compiled code blobs. Events in the interpreter are covered by pre-existing APIs, not compiled method patch points.

This JEP does not attempt to deal with Java methods per se, only with with the large composite "blobs" of code called compiled methods (class CompiledMethod) which reside in the code cache and are responsible for execution at speed and at scale. Higher-level tracing facilities already exist, which allow users to set breakpoints and tracepoints on specific methods and lines of source code. These techniques do not always apply to optimized compiled methods, because the first thing they do is discard the methods and recompile them with some optimizations turned off and special calls inserted. This is fine for individual debugging but too disruptive for tracing large cloud workloads.

The overheads associated with the USDT are on the order of subroutine calls and fast traps, and generally only are paid, in small increments, when tracing is actually being used. The JVM does not yet have a tracing mechanism for compiled methods that fits this profile.

Making it work is relatively straightforward, given some effort and close cooperation between the right parties. Few or no changes are needed to the shape of the compiled code, since it is already patchable. (For example, deoptimizing a method requires an entry-point patch.) The USDT API will have to "bend" a little to take account of the unusual conditions within the JVM. The JVM will have to set aside parts of the code for patch points, and set up the temporary extra control transfers between method entry points and patch points.

There is a variant of USDT called "dynamic UDST" which is used to perform method instrumentation for Node.js. It is likely that many of the special issues discussed here are already addressed there.