JEP draft: Adapt Object Monitors for Virtual Threads
Owner | Alan Bateman |
Type | Feature |
Scope | Implementation |
Status | Submitted |
Component | hotspot / runtime |
Effort | M |
Reviewed by | Daniel Daugherty, Vladimir Kozlov |
Created | 2024/07/29 17:09 |
Updated | 2024/09/06 19:39 |
Issue | 8337395 |
Summary
Change the HotSpot VM implementation of object monitors to allow a virtual thread unmount from its carrier thread when inside a synchronized
block or method, when blocking to gain ownership of a monitor, or when waiting in Object.wait
.
Goals
-
Eliminate the so-called pinning issues with object monitors that can currently harm scalability and performance when used with virtual threads.
-
Expand the set of existing Java libraries that can be used with virtual threads.
-
Improve the diagnostics that identify the remaining cases where a virtual thread blocks when pinned to its carrier.
Motivation
JEP 425 added virtual threads to the Java Platform in JDK 19. Virtual threads are a lightweight implementation of threads that is provided by the JDK rather than the operating system (OS). Virtual threads can significantly reduce the effort of developing, maintaining, and observing high-throughput concurrent applications by allowing for a great number of threads.
To do useful work a thread needs to be scheduled, that is, assigned for execution on a processor core. For platform threads, which are implemented as OS threads, the JDK relies on the scheduler in the OS. For virtual threads, by contrast, the JDK has its own scheduler. Rather than assigning virtual threads to processor cores directly, the JDK's scheduler assigns virtual threads to platform threads. The platform threads are then scheduled by the OS as usual. The platform thread to which the JDK's virtual thread scheduler assigns a virtual thread is called the virtual thread's carrier.
To run code in a virtual thread, the JDK's virtual thread scheduler assigns the virtual thread for execution on a platform thread by mounting the virtual thread on a platform thread. This makes the platform thread become the carrier of the virtual thread. Later, after running some code, the virtual thread can unmount from its carrier. At that point the platform thread is free so the scheduler can mount a different virtual thread on it, thereby making it a carrier again.
Typically, a virtual thread will unmount when it blocks on I/O or some other blocking operation in the JDK, such as BlockingQueue.take()
. When the blocking operation is ready to complete (e.g., bytes have been received on a socket, or an element becomes available on a queue), it submits the virtual thread back to the scheduler, which will mount the virtual thread on a carrier to resume execution.
The mounting and unmounting of virtual threads happens frequently and transparently, and without blocking any OS threads.
The vast majority of blocking operations in the JDK unmount the virtual thread, freeing its carrier and the underlying OS thread to take on new work. However, some blocking operations in the JDK do not unmount the virtual thread, and thus block both its carrier and the underlying OS thread. These cases arise due to either limitations at the OS level or in the JDK.
In the case of synchronization, which is implemented using object monitors, a virtual thread cannot be unmounted when it blocks entering or inside a synchronized
block or method. It also cannot be unmounted when it blocks in Object.wait
. These cases pin the carrier.
Frequent pinning for long durations with object monitors can harm the scalability of an application by capturing carriers. In some cases, the pinning can lead to starvation, even deadlock, where no virtual threads can execute because all platform threads available to the JDK's virtual thread scheduler are pinned by a virtual thread.
The JDK includes several diagnostic features to identify cases where a virtual thread cannot unmount because it is pinned to its carrier. These diagnostic features can be used to identify code that would benefit from reduced use of locking with object monitors, changed to avoid blocking while inside a synchronized
method, or migrated from synchronized
to java.util.concurrent
locks. Unlike synchronized
, a virtual thread that blocks on a lock from java.util.concurrent
can unmount from its carrier.
Since virtual threads were introduced in JDK 19 many libraries have been migrated to use java.util.concurrent
locks to avoid pinning issues, particularly libraries that do I/O in synchronized
methods. Although the migration is mostly mechanical, it is tedious to identify the problematic cases, and isn't practical when the problematic usage is in a library that is maintained elsewhere.
It would greatly benefit the Java Platform, and the adoption of virtual threads, if the VM implementation of object monitors allowed a virtual thread to unmount from its carrier when blocking inside a synchronized
block or method, or when blocking on an object monitor.
Description
Change the HotSpot VM implementation of object monitors to allow a virtual thread unmount from its carrier when it executes code that:
- Blocks while inside a
synchronized
block or method. - Blocks entering a
synchronized
block or method because the object monitor is owned by another thread. - Waits in
Object.wait()
orObject.wait(timeoutMillis)
.
These changes remove the so-called pinning issues with object monitors and allow more existing Java libraries be used with virtual threads without needing to modify code.
In the HotSpot VM, only the "lightweight locking mode" is changed. This locking mode was introduced as an experimental locking mode in JDK 21 (see JDK-8305999). It became the default locking mode in JDK 23. The default locking mode prior to JDK 23 is now known as "legacy locking mode" and was deprecated in that release. Legacy locking mode will continue to pin carriers if selected.
Diagnosability
JEP 425 introduced the JDK Flight Recorder (JFR) event jdk.VirtualThreadPinned
that is recorded when a virtual thread parks pinning its carrier. This event has been useful to identify code that would benefit from reduced use of locking, changed to avoid blocking while inside a synchronized
method, or migrated to java.util.concurrent
locks.
Removing the pinning issues with object monitors removes some of the motivation for this JFR event. There are, however, other cases where a virtual thread cannot unmount from its carrier. In particular, a virtual thread cannot unmount when calling through a native method or foreign function.
In JDK 24, this event is changed to be recorded for more cases when a virtual thread parks or blocks pinning its carrier. This includes being pinned by a native frame or for reasons listed in the Future Work section below. The event is changed to include a useful reason and the identity of the carrier thread. When initially introduced, the event was only recorded when a virtual thread continued execution after pinning its carrier for more than 20ms. This proved unsatisfactory for cases where a virtual thread pinned its carrier indefinitely as no event was recorded. The event is changed to be always recorded (a future release may change this further so that it is only recorded when sampled).
The system property jdk.tracePinnedThreads
, introduced in JEP 425, to trigger a stack trace when a thread blocks while pinned, is removed. While easy to use, this diagnostic feature has proven to be problematic as it triggers the printing of a stack trace while executing critical code. This diagnostic feature is also limited in that it doesn't trigger a stack trace when a virtual thread blocks entering a monitor, or when it waits in Object.wait
. Removing the pinning issues with object monitors also removes most of the original need for this diagnostic option.
synchronized vs. java.util.concurrent locks
The java.util.concurrent
API was added in JDK 5 to enhance support for concurrent programming. It defines APIs and a framework in java.util.concurrent.locks
for locking and waiting for conditions that are distinct from the synchronized
keyword and provide much greater flexibility at the cost of using APIs over the convenience of synchronized
blocks and methods. In java.util.concurrent.locks
, the ReentrantLock
API provides the same behavior and semantics as the built-in synchronization and object monitors. Condition
provides the equivalent of the Object.wait
and Object.notify
methods. Although semantically equivalent, their usages and implementations are different:
-
The APIs in
java.util.concurrent.locks
require using thetry-finally
construct to ensure structured use. -
The APIs in
java.util.concurrent.locks
provide greater flexibility and control for more advanced cases that require fairness, timed and/or interruptible lock aquisition, concurrent access to shared data (with read-write locks), and very advanced APIs such asStampedLock
. -
The implementations have different performance characteristics.
With object monitors no longer causing pinning, the choice of whether to use synchronized
or an API from java.util.concurrent.locks
can be made on the merits of each. Java Concurrency in Practice 13.4 advises to use synchronized
where practical as it is more convenient and less error prone. Use ReentrantLock
and the other APIs in java.util.concurrent.locks
for more advanced cases or where more flexibility is required.
The general advice for new code is to reduce the potential for contention by narrowing the scope of locks (object monitors or java.util.concurrent
locks), and avoid, where possible, doing I/O or blocking operations while holding locks.
Since JDK 19, many libraries have migrated from synchronized
to java.util.concurrent
locks to avoid pinning issues. The modified code will continue to work and should not need to be reverted to use synchronized
.
The guidance in JEP 425 (and later in JEP 444) was to avoid frequent and long-lived pinning issues by migrating the problematic cases from using synchronized
to using ReentrantLock
instead. The changes to object monitors will remove the need to do further migration and should allow more existing code be executed in virtual threads.
Future Work
There are a few remaining cases where a virtual thread cannot be unmounted when blocking:
-
When resolving a symbol reference (JVMS 5.4.3) to a class or interface and the virtual thread blocks while loading a class. This is a case where the virtual thread pins the carrier due to a native frame on the stack.
-
When blocking inside a class initializer. This is also a case where the virtual thread pins the carrier due to a native frame on the stack.
-
When waiting for a class to be initialized by another thread (JVMS 5.5). This is a special case where a virtual thread blocks in the VM, thus pinning the carrier.
These cases should rarely cause issues but may be examined for a future release.
Alternatives
-
Compensate for pinning (and the capture of an OS thread) by temporarily expanding the parallelism of the scheduler. This is already done for
Object.wait
where it increases parallelism, and ensures that there is a spare platform thread available to the scheduler, while a virtual thread is waiting.Increasing parallelism can help in some cases but it doesn't scale. The maximum number of platform threads available to the scheduler is limited (the default limit is 256). If a huge number of virtual threads need to block inside a
synchronized
method then no value of parallelism will help. -
Transform all uses of
synchronized
tojava.util.concurrent.ReentrantLock
at class load time.Adding a mapping of
Object
to aReentrantLock
would add significant overhead. There are also cases where the transformation would not be fully transparent, in particular with synchronized methods (ACC_SYNCHRONIZED
) where the JVMS requires entering the monitor before invoking the method.There are also many challenges with the approach in areas such as JNI locking, several features of JVM TI, and the JVMS requiring that a monitor be automatically exited in all cases. The approach would also require significant re-implementation of many serviceability features.
Risks and Assumptions
-
Performance. Some scenarios may display different performance characteristics to the equivalent scenario with platform threads, in particular when a virtual thread continues after being blocked on a monitor or waiting in
Object.wait
. -
The changes come with a restriction that
java.lang.Object
cannot be dynamically redefined with the JVM TI functions RetransformClasses or RedefineClasses (or the equivalent methods in java.lang.instrument.Instrumentation with Java agents). Attempting to redefine or retransformjava.lang.Object
will fail withJVMTI_ERROR_UNMODIFIABLE_CLASS
or throwUnmodifiableClassException
. This error (and exception) is allowed by the specification. Testing with early access builds has not encountered any agents that changejava.lang.Object
. The restriction may be re-visited in the future if it proves problematic.
Dependencies
- The changes proposed assume a specification change to the JVM TI function
GetObjectMonitorUsage
that was done in JDK 23 (see JDK-8331422). This specification change "degrades" the function to not support getting information about an object monitor that is owned by a virtual thread. This specification change avoids the bookkeeping overhead that would be required to find object monitors owned by unmounted virtual threads.