JEP draft: 4-byte Object Headers (Experimental)

Summary

Reduce the size of object headers in the HotSpot JVM from between 64 and 128 bits down to 32 bits on 64-bit architectures. This will reduce heap size, improve deployment density, and increase data locality.

Goals

When enabled, this feature

• Must reduce the object header size to 32 bits (4 bytes) on the target 64-bit platforms (x64 and AArch64),
• Should reduce object sizes and memory footprint on realistic workloads,
• Should not introduce more than 5% throughput or latency overheads on the target 64-bit platforms, and only in infrequent cases, and
• Should not introduce measurable throughput or latency overheads on non-target 64-bit platforms.

When disabled, this feature

• Must retain the original object header layout and object sizes on all platforms, and
• Should not introduce measurable throughput or latency overheads on any platform.

This experimental feature will have a broad impact on real-world applications. The code might have inefficiencies, bugs, and unanticipated non-bug behaviors. This feature must therefore be disabled by default and enabled only by explicit user request. We intend to enable it by default in later releases and eventually remove the code for legacy object headers altogether.

Non-Goals

It is not a goal to

• Reduce the object header size below 32 bits on 64-bit platforms,
• Reduce the object header size on non-target 64-bit platforms,
• Change the object header size on 32-bit platforms, or
• Change the encoding of object content (i.e., fields and array elements) or array metadata (i.e., array length).

Motivation

An object stored in the heap has metadata, which the HotSpot JVM stores in the object's header. The size of the header is constant; it is independent of object type, array shape, and content. In the 64-bit HotSpot JVM, object headers occupy between 64 bits (8 bytes) and 128 bits (16 bytes), depending on how the JVM is configured.

Objects in Java programs tend to be small. Experiments conducted as part of Project Lilliput show that many workloads have average object sizes of 256 to 512 bits (32 to 64 bytes). This implies that more than 20% of live data can be taken by object headers alone. Thus even a small improvement in object header size could yield a significant reduction in footprint, data locality, and reduced GC pressure. Early adopters of Project Lilliput who have tried it with real-world applications confirm that live data is typically reduced by 10%–20%.

JEP 450 introduced 'compact object headers' that reduced object header size to 8 bytes. This JEP builds on JEP 450 and goes even further and reduces object header size to just 4 bytes.

Description

Compact object headers is an experimental feature and therefore disabled by default. Compact object headers can be enabled with -XX:+UnlockExperimentalVMOptions -XX:+UseCompactObjectHeaders.

Current object header layouts

TBD

4-byte object header layout

TBD

Compact Identity Hash-Code

TBD

Compact GC Forwarding

TBD

Alternatives

• Use lookup-table for identity hash-code (TBD)
• Use lookup-table for GC forwarding (TBD)

Testing

Changing the header layout of Java heap objects touches many HotSpot JVM subsystems: the runtime, all garbage collectors, all just-in-time compilers, the interpreters, the serviceability agent, and the architecture-specific code for all supported platforms. Such massive changes warrant massive testing.

Compact object headers will be tested by:

Tier 1–4 tests, and possibly more testing tiers by vendors which have them; The SPECjvm, SPECjbb, DaCapo, and Renaissance benchmark suites to test both correctness and performance; JCStress, to test the new locking implementation; and Some real-world workloads. All of these tests will be executed with the feature turned on and off, with multiple combinations of GCs and JIT compilers, and on several hardware targets.

We will also deliver a new set of tests that measure the size of various objects, e.g., plain objects, primitive type arrays, reference arrays, and their headers.

The ultimate test for performance and correctness will be real-world workloads once this experimental feature is delivered.

Risks and Assumptions

• Future runtime features need object header bits — In particular, project Valhalla is known to possibly need some header bits. In order to support such needs, this proposal leaves 4 spare header bits for future use.

• Implementation bugs in feature code — The usual risk for an intrusive feature such as this is bugs in the implementation. While issues in the header layout might be visible immediately with most tests, subtleties in the new locking and GC forwarding protocols may expose bugs only rarely. We mitigate this risk with careful reviews by component owners and by running many tests with the feature enabled. This risk does not affect the product so long as the feature remains experimental and off by default.

• Implementation bugs in legacy code — We try to avoid changing legacy code paths, but some refactorings necessarily touch shared code. This exposes the risk of bugs even when the feature is disabled. In addition to careful reviews and testing, we mitigate this risk by coding defensively and trying to avoid modifying shared code paths, even if it requires more work in feature code paths.

• Performance issues in feature code — The more complex protocols for compact object headers may introduce performance issues when the feature is enabled. We mitigate this risk by running major benchmarks and understanding the feature's impact on their performance. This risk does not affect the product so long as the feature remains experimental and off by default.

• Performance issues in legacy code — There is a minor risk that refactoring the legacy code paths will affect performance in unexpected ways. We mitigate this risk by minimizing the changes to the legacy code paths and showing that the performance of major workloads is not substantially affected.

• Compressed class pointers support — Compressed class pointers are not supported by JVMCI on x64. We mitigate the immediate risk by disabling compact object headers when JVMCI is enabled. The long-term risk is that compact headers are never implemented in JVMCI, which would forever block removing the legacy header implementation. We assign only a minor probability to this risk since other JIT compilers support compact object headers without intrusive changes.

• Compressed class pointers encoding — As stated above, the current implementation of compressed class pointers is limited to about 500,000 classes. Presently, users can work around this limitation by disabling compact object headers and compressed class pointers, but if we remove the legacy header implementation, then that will no longer be possible. We mitigate the immediate risk by providing compact object headers as an experimental feature; in the long term, we intend to work toward more efficient compressed class pointer encoding schemes.

• Changing low-level interfaces — Some components that manipulate object headers directly, notably the Graal compiler as the major user of JVMCI, will have to implement the new header layout. We mitigate the current risk by identifying these components and disabling the feature when those components are in use. Before the feature graduates from experimental status, those components will need to be upgraded.

• Soft project failure — There is a minor risk that the feature has irreconcilable functional regressions compared to the legacy implementation, e.g., limiting the number of representable classes. A related risk is that while the feature provides significant performance improvements on its own, it comes with significant functional limitations, which might lead to an argument for keeping both the new and legacy header implementations forever. Given that the goal of this work is to replace the legacy header implementation eventually, we consider this a soft project failure. We mitigate this risk by carefully examining current limitations, planning future work to eliminate them, and looking to early adopters reports to identify other risks before we invest too much effort.

• Hard project failure — While very unlikely, it may turn out that compact object headers do not yield tangible real-world improvements or that the achievable improvements do not justify their additional complexity. We mitigate this minor risk by gating the new code paths as experimental, thus keeping a path open to removing the feature in a future release should the need arise.

Dependencies

This JEP builds on and extends JEP 450: Compact Object Headers (Experimental)