JEP draft: Adapt Object Monitors for Virtual Threads

OwnerAlan Bateman
TypeFeature
ScopeImplementation
StatusSubmitted
Componenthotspot / runtime
EffortM
Reviewed byDaniel Daugherty, Vladimir Kozlov
Created2024/07/29 17:09
Updated2024/09/06 19:39
Issue8337395

Summary

Change the HotSpot VM implementation of object monitors to allow a virtual thread unmount from its carrier thread when inside a synchronized block or method, when blocking to gain ownership of a monitor, or when waiting in Object.wait.

Goals

Motivation

JEP 425 added virtual threads to the Java Platform in JDK 19. Virtual threads are a lightweight implementation of threads that is provided by the JDK rather than the operating system (OS). Virtual threads can significantly reduce the effort of developing, maintaining, and observing high-throughput concurrent applications by allowing for a great number of threads.

To do useful work a thread needs to be scheduled, that is, assigned for execution on a processor core. For platform threads, which are implemented as OS threads, the JDK relies on the scheduler in the OS. For virtual threads, by contrast, the JDK has its own scheduler. Rather than assigning virtual threads to processor cores directly, the JDK's scheduler assigns virtual threads to platform threads. The platform threads are then scheduled by the OS as usual. The platform thread to which the JDK's virtual thread scheduler assigns a virtual thread is called the virtual thread's carrier.

To run code in a virtual thread, the JDK's virtual thread scheduler assigns the virtual thread for execution on a platform thread by mounting the virtual thread on a platform thread. This makes the platform thread become the carrier of the virtual thread. Later, after running some code, the virtual thread can unmount from its carrier. At that point the platform thread is free so the scheduler can mount a different virtual thread on it, thereby making it a carrier again.

Typically, a virtual thread will unmount when it blocks on I/O or some other blocking operation in the JDK, such as BlockingQueue.take(). When the blocking operation is ready to complete (e.g., bytes have been received on a socket, or an element becomes available on a queue), it submits the virtual thread back to the scheduler, which will mount the virtual thread on a carrier to resume execution.

The mounting and unmounting of virtual threads happens frequently and transparently, and without blocking any OS threads.

The vast majority of blocking operations in the JDK unmount the virtual thread, freeing its carrier and the underlying OS thread to take on new work. However, some blocking operations in the JDK do not unmount the virtual thread, and thus block both its carrier and the underlying OS thread. These cases arise due to either limitations at the OS level or in the JDK.

In the case of synchronization, which is implemented using object monitors, a virtual thread cannot be unmounted when it blocks entering or inside a synchronized block or method. It also cannot be unmounted when it blocks in Object.wait. These cases pin the carrier.

Frequent pinning for long durations with object monitors can harm the scalability of an application by capturing carriers. In some cases, the pinning can lead to starvation, even deadlock, where no virtual threads can execute because all platform threads available to the JDK's virtual thread scheduler are pinned by a virtual thread.

The JDK includes several diagnostic features to identify cases where a virtual thread cannot unmount because it is pinned to its carrier. These diagnostic features can be used to identify code that would benefit from reduced use of locking with object monitors, changed to avoid blocking while inside a synchronized method, or migrated from synchronized to java.util.concurrent locks. Unlike synchronized, a virtual thread that blocks on a lock from java.util.concurrent can unmount from its carrier.

Since virtual threads were introduced in JDK 19 many libraries have been migrated to use java.util.concurrent locks to avoid pinning issues, particularly libraries that do I/O in synchronized methods. Although the migration is mostly mechanical, it is tedious to identify the problematic cases, and isn't practical when the problematic usage is in a library that is maintained elsewhere.

It would greatly benefit the Java Platform, and the adoption of virtual threads, if the VM implementation of object monitors allowed a virtual thread to unmount from its carrier when blocking inside a synchronized block or method, or when blocking on an object monitor.

Description

Change the HotSpot VM implementation of object monitors to allow a virtual thread unmount from its carrier when it executes code that:

  1. Blocks while inside a synchronized block or method.
  2. Blocks entering a synchronized block or method because the object monitor is owned by another thread.
  3. Waits in Object.wait() or Object.wait(timeoutMillis).

These changes remove the so-called pinning issues with object monitors and allow more existing Java libraries be used with virtual threads without needing to modify code.

In the HotSpot VM, only the "lightweight locking mode" is changed. This locking mode was introduced as an experimental locking mode in JDK 21 (see JDK-8305999). It became the default locking mode in JDK 23. The default locking mode prior to JDK 23 is now known as "legacy locking mode" and was deprecated in that release. Legacy locking mode will continue to pin carriers if selected.

Diagnosability

JEP 425 introduced the JDK Flight Recorder (JFR) event jdk.VirtualThreadPinned that is recorded when a virtual thread parks pinning its carrier. This event has been useful to identify code that would benefit from reduced use of locking, changed to avoid blocking while inside a synchronized method, or migrated to java.util.concurrent locks.

Removing the pinning issues with object monitors removes some of the motivation for this JFR event. There are, however, other cases where a virtual thread cannot unmount from its carrier. In particular, a virtual thread cannot unmount when calling through a native method or foreign function.

In JDK 24, this event is changed to be recorded for more cases when a virtual thread parks or blocks pinning its carrier. This includes being pinned by a native frame or for reasons listed in the Future Work section below. The event is changed to include a useful reason and the identity of the carrier thread. When initially introduced, the event was only recorded when a virtual thread continued execution after pinning its carrier for more than 20ms. This proved unsatisfactory for cases where a virtual thread pinned its carrier indefinitely as no event was recorded. The event is changed to be always recorded (a future release may change this further so that it is only recorded when sampled).

The system property jdk.tracePinnedThreads, introduced in JEP 425, to trigger a stack trace when a thread blocks while pinned, is removed. While easy to use, this diagnostic feature has proven to be problematic as it triggers the printing of a stack trace while executing critical code. This diagnostic feature is also limited in that it doesn't trigger a stack trace when a virtual thread blocks entering a monitor, or when it waits in Object.wait. Removing the pinning issues with object monitors also removes most of the original need for this diagnostic option.

synchronized vs. java.util.concurrent locks

The java.util.concurrent API was added in JDK 5 to enhance support for concurrent programming. It defines APIs and a framework in java.util.concurrent.locks for locking and waiting for conditions that are distinct from the synchronized keyword and provide much greater flexibility at the cost of using APIs over the convenience of synchronized blocks and methods. In java.util.concurrent.locks, the ReentrantLock API provides the same behavior and semantics as the built-in synchronization and object monitors. Condition provides the equivalent of the Object.wait and Object.notify methods. Although semantically equivalent, their usages and implementations are different:

With object monitors no longer causing pinning, the choice of whether to use synchronized or an API from java.util.concurrent.locks can be made on the merits of each. Java Concurrency in Practice 13.4 advises to use synchronized where practical as it is more convenient and less error prone. Use ReentrantLock and the other APIs in java.util.concurrent.locks for more advanced cases or where more flexibility is required.

The general advice for new code is to reduce the potential for contention by narrowing the scope of locks (object monitors or java.util.concurrent locks), and avoid, where possible, doing I/O or blocking operations while holding locks.

Since JDK 19, many libraries have migrated from synchronized to java.util.concurrent locks to avoid pinning issues. The modified code will continue to work and should not need to be reverted to use synchronized.

The guidance in JEP 425 (and later in JEP 444) was to avoid frequent and long-lived pinning issues by migrating the problematic cases from using synchronized to using ReentrantLock instead. The changes to object monitors will remove the need to do further migration and should allow more existing code be executed in virtual threads.

Future Work

There are a few remaining cases where a virtual thread cannot be unmounted when blocking:

  1. When resolving a symbol reference (JVMS 5.4.3) to a class or interface and the virtual thread blocks while loading a class. This is a case where the virtual thread pins the carrier due to a native frame on the stack.

  2. When blocking inside a class initializer. This is also a case where the virtual thread pins the carrier due to a native frame on the stack.

  3. When waiting for a class to be initialized by another thread (JVMS 5.5). This is a special case where a virtual thread blocks in the VM, thus pinning the carrier.

These cases should rarely cause issues but may be examined for a future release.

Alternatives

Risks and Assumptions

Dependencies