JEP 491: Synchronize Virtual Threads without Pinning

Summary

Improve the scalability of Java code that uses synchronized methods and statements by arranging for virtual threads that block in such constructs to release their underlying platform threads for use by other virtual threads. This will eliminate nearly all cases of virtual threads being pinned to platform threads, which severely restricts the number of virtual threads available to handle an application's workload.

Goals

• Enable existing Java libraries to scale well with virtual threads without having to change them not to use synchronized methods and statements.

• Improve the diagnostics that identify the remaining situations in which virtual threads fail to release platform threads.

Motivation

Virtual threads, which were introduced in Java 21 via JEP 444, are lightweight threads that are provided by the JDK rather than the operating system (OS). Virtual threads significantly reduce the effort of developing, maintaining, and observing high-throughput concurrent applications by enabling applications to use huge numbers of threads. The basic model of virtual threads is as follows:

• To do useful work, a thread must be scheduled, that is, assigned for execution on a processor core. For platform threads, which are implemented as OS threads, the JDK relies on the scheduler in the OS. For virtual threads, by contrast, the JDK has its own scheduler. Rather than assign virtual threads to processor cores directly, the JDK's scheduler assigns virtual threads to platform threads, which are then scheduled by the OS as usual.

• To run code in a virtual thread, the JDK's scheduler assigns the virtual thread for execution on a platform thread by mounting the virtual thread on the platform thread. This makes the platform thread become the carrier of the virtual thread. Later, after running some code, the virtual thread can unmount from its carrier. At that point the platform thread is released so that the JDK's scheduler can mount a different virtual thread on it, thereby making it a carrier again.

• A virtual thread unmounts when performing a blocking operation such as I/O. Later, when the blocking operation is ready to complete because, e.g., bytes were received on a socket, the operation submits the virtual thread back to the JDK's scheduler. The scheduler mounts the virtual thread on a platform thread to resume running code.

Virtual threads are mounted and unmounted frequently and transparently, without blocking any platform threads.

Virtual threads are pinned in synchronized methods

Unfortunately, a virtual thread cannot unmount when it runs code inside a synchronized method. Consider the following synchronized method, which reads bytes from a socket:

synchronized byte[] getData() {
    byte[] buf = ...;
    int nread = socket.getInputStream().read(buf);    // Can block here
    ...
}

If the read method blocks because there are no bytes available, we would like the virtual thread that is running getData to unmount from its carrier. This would release a platform thread so that the JDK's scheduler can mount a different virtual thread on it. Unfortunately, because getData is synchronized, the JVM pins the virtual thread that is running getData to its carrier. Pinning prevents the virtual thread from unmounting. Consequently, the read method blocks not only the virtual thread but also its carrier, and hence the underlying OS thread, until bytes are available to read.

The reason for pinning

The synchronized keyword in the Java programming language is defined in terms of monitors: Every object is associated with a monitor that can be acquired (i.e., locked), held for a time, and then released (i.e., unlocked). Only one thread at a time may hold an object's monitor. For a thread to run a synchronized instance method, the thread first acquires the monitor associated with the instance; when the method is finished, the thread releases the monitor.

To implement the synchronized keyword, the JVM tracks which thread currently holds an object's monitor. Unfortunately, it tracks which platform thread holds the monitor, not which virtual thread. When a virtual thread runs a synchronized instance method and acquires the monitor associated with the instance, the JVM records the virtual thread's carrier platform thread as holding the monitor — not the virtual thread itself.

If a virtual thread were to unmount inside a synchronized instance method, the JDK's scheduler would soon mount some other virtual thread on the now-free platform thread. That other virtual thread, because of its carrier, would be viewed by the JVM as holding the monitor associated with the instance. Code running in that thread would be able to call other synchronized methods on the instance, or release the monitor associated with the instance. Mutual exclusion would be lost. Accordingly, the JVM actively prevents a virtual thread from unmounting inside a synchronized method.

More pinning

If a virtual thread invokes a synchronized instance method and the monitor associated with the instance is held by another thread, then the virtual thread must block since only one thread at a time may hold the monitor. We would like the virtual thread to unmount from its carrier and release that platform thread to the JDK scheduler. Unfortunately, if the monitor is already held by another thread then the virtual thread blocks in the JVM until the carrier acquires the monitor.

Moreover, when a virtual thread is inside a synchronized instance method and it invokes Object.wait() on the object, then the virtual thread blocks in the JVM until awakened with Object.notify() and the carrier re-acquires the monitor. The virtual thread is pinned because it is executing inside a synchronized method, and further pinned because its carrier is blocked in the JVM.

The foregoing discussion applies, with appropriate changes, to synchronized static methods, which synchronize on the monitor associated with the Class object for the method's class, and to synchronized statements, which synchronize on the monitor associated with a specified object.

Overcoming pinning

Frequent pinning for long durations can harm scalability. It can lead to starvation or even deadlock, when no virtual threads can run because all of the platform threads available to the JDK's scheduler are either pinned by virtual threads or blocked in the JVM. To avoid these problems, the maintainers of many libraries have modified their code to use java.util.concurrent locks — which do not pin virtual threads — instead of synchronized methods and statements.

It should not, however, be necessary to abandon synchronized methods and statements in order to enjoy the scalability benefits of virtual threads. The JVM's implementation of the synchronized keyword should allow a virtual thread to unmount when inside a synchronized method or statement, or when blocked on a monitor. This would enable the broader adoption of virtual threads.

Description

We will change the JVM's implementation of the synchronized keyword so that virtual threads can acquire, hold, and release monitors, independently of their carriers. The mounting and unmounting operations will do the bookkeeping necessary to allow a virtual thread to unmount and re-mount when inside a synchronized method or statement, or when waiting on a monitor.

Blocking to acquire a monitor will unmount a virtual thread and release its carrier to the JDK's scheduler. When the monitor is released, and the JVM selects the virtual thread to continue, the JVM will submit the virtual thread to the scheduler. The scheduler will mount the virtual thread, perhaps on a different carrier, to resume execution and try again to acquire the monitor.

The Object.wait() method, and its timed-wait variants, will similarly unmount a virtual thread when waiting and blocking to re-acquire a monitor. When awakened with Object.notify(), and the monitor is released, and the JVM selects the virtual thread to continue, the JVM will submit the virtual thread to the scheduler to resume execution.

Diagnosing remaining cases of pinning

A jdk.VirtualThreadPinned event is recorded by JDK Flight Recorder (JFR) whenever a virtual thread blocks inside a synchronized method. This event has been useful to identify code that would benefit from being changed to make less use of synchronized methods and statements, to not block while inside such constructs, or to replace such constructs with java.util.concurrent locks.

This JFR event will no longer be needed for that purpose once the synchronized keyword no longer pins virtual threads, but we will retain it for other pinning situations. In particular, if a virtual thread calls native code, either through a native method or the Foreign Function & Memory API, and that native code calls back to Java code that performs a blocking operation or blocks on a monitor, then the virtual thread will be pinned. We will therefore change the JVM to issue a jdk.VirtualThreadPinned event in these cases, and we will enhance the event itself to convey both the reason why the virtual thread is pinned and the identity of the carrier thread.

We will also change the times at which the JVM records jdk.VirtualThreadPinned events. Originally, the JVM recorded this event only when a virtual thread continued execution after pinning its carrier for more than 20 milliseconds. This is unsatisfactory for cases in which a virtual thread pins its carrier indefinitely, since no event will ever be recorded. We will therefore change the JVM to always record this event. In a future release we may change this event so that it is only recorded when sampled.

The system property jdk.tracePinnedThreads is no longer needed

The system property jdk.tracePinnedThreads, introduced by JEP 444, causes a stack trace to be printed whenever a virtual thread blocks inside a synchronized method, though not when a virtual thread blocks to acquire a monitor or wait in Object.wait().

This system property will no longer be needed once the synchronized keyword no longer pins virtual threads. It has, in addition, proved to be problematic since the stack traces are printed while executing critical code. We will therefore remove this system property; setting it on the command line will have no effect.

Choosing between synchronized and java.util.concurrent.locks

Once the synchronized keyword no longer pins virtual threads, you can choose between synchronized and the APIs in the java.util.concurrent.locks package based solely upon which best solves the problem at hand.

As background, the java.util.concurrent.locks package defines APIs for locking and waiting that are distinct from, and more flexible than, the built-in synchronized keyword. The ReentrantLock API behaves the same as synchronized. The Condition API is the equivalent of the Object.wait() and Object.notify() methods. Other APIs in the package provide greater power and finer control for advanced cases that require fairness, concurrent access to shared data with read-write locks, timed or interruptible lock acquisition, or optimistic reading.

The flexibility of the java.util.concurrent.locks APIs comes at the expense of more awkward syntax. The APIs should generally be used with the try-finally construct in order to ensure that locks are released appropriately; this is, of course, not necessary with synchronized. The java.util.concurrent.locks APIs also have different performance characteristics than synchronized methods or statements.

We previously recommended solving frequent and long-lived pinning problems by migrating code from using synchronized to using ReentrantLock. Once the synchronized keyword no longer pins virtual threads, such migration will no longer be necessary. You need not revert code that has been migrated to use ReentrantLock back to using synchronized.

If you are writing new code, we agree with the recommendation in Java Concurrency in Practice §13.4: Use synchronized where practical, since it is more convenient and less error prone, and use ReentrantLock and the other APIs in java.util.concurrent.locks when more flexibility is required. Either way, reduce the potential for contention by narrowing the scope of locks and avoid, where possible, doing I/O or other blocking operations while holding locks.

Future Work

There are a few remaining cases, unrelated to the synchronized keyword, in which a virtual thread cannot unmount when blocking:

• When resolving a symbolic reference (JVMS §5.4.3) to a class or interface and the virtual thread blocks while loading a class. This is a case where the virtual thread pins the carrier due to a native frame on the stack.

• When blocking inside a class initializer. This is also a case where the virtual thread pins the carrier due to a native frame on the stack.

• When waiting for a class to be initialized by another thread (JVMS §5.5). This is a special case where the virtual thread blocks in the JVM, thus pinning the carrier.

These cases should rarely cause issues but we will revisit them if they prove to be problematic.

Alternatives

• Compensate for pinning by temporarily expanding the parallelism of the virtual-thread scheduler. The scheduler already does this for Object.wait(), by ensuring that a spare platform thread is available while a virtual thread is waiting.

  Increasing parallelism would help with some cases, but it does not scale. The maximum number of platform threads available to the scheduler is limited, with a default limit of 256 threads. If many virtual threads were to block inside a synchronized method then no value of parallelism would help.

• Rewrite the bytecode of each class, as the JVM loads it, to replace each use of synchronized with an equivalent use of ReentrantLock.

  The synchronized statement can be used with any object, so this would require maintaining a mapping from objects to locks, a significant overhead.

  There are cases where the transformation would not be fully transparent, in particular for synchronized methods, since JVMS §2.11.10 requires acquiring the monitor before invoking the method.

  There are also many challenges with this approach in areas such as JNI locking, several features of JVM TI, and the JVMS requiring that a monitor be automatically released in all cases. This approach would also require the re-implementation of many serviceability features.

Risks and Assumptions

The performance of some code may be different when virtual threads are used in place of platform threads. When a thread exits a monitor it may have to queue a virtual thread to the scheduler. This is currently not as efficient as the case where exiting a monitor unparks a platform thread.

Dependencies

The changes we propose here depend upon a change to the specification of the JVM TI function GetObjectMonitorUsage in Java 23. This function no longer supports returning information about monitors owned by virtual threads. Doing so would have required significant bookkeeping to find the monitors owned by unmounted virtual threads.