Java Platform Module System: Issue Summary

Mark Reinhold

This is the issue list for JSR 376, the Java Platform Module System. The issues documented here were collected from the JSR 376 Expert Group mailing list, the JSR 376 comments list, the JSR 376 observers’ list, and the OpenJDK jigsaw-dev list. References to pertinent messages, though not all related messages, are included.

During the course of the work on the JSR this was a living document, updated regularly issues were raised, discussed, and resolved; it is now a permanent record of that work. Each issue was assigned a unique #HashTag for use in the e-mail messages and other textual media in which these issues are discussed so that such discussions could be discovered, tracked, and summarized.

Issues by category

Module declarations

#ModuleNameSyntax — Some people find the use of the same syntax for module names and package names to be confusing. Should the recommended syntax for module names be changed to something else, e.g., lower-case words separated by underscores? Or should the syntax of module declarations be revised to expand requires into requires module, and exports into exports package, in order to disambiguate these two kinds of names? Or is there some other solution? (Cf. #AutomaticModuleNames.) [Rémi Forax, Stephen Colebourne, Paul Benedict]

Resolution The resolution of #AutomaticModuleNames strongly recommends that module names follow the reverse Internet domain-name convention, which has long been used for package names.

#ModuleNameCharacters — Module names are presently constrained to be Java identifiers. Some existing module systems allow additional characters in module names, such as hyphens and slashes. Should this restriction be lifted or, perhaps, should it somehow be made layer-specific? [David Lloyd]

Resolution Do not change the treatment of module names in source code; they will remain qualified names. Revise the encoding of module names in compiled module-declaration class files to lift the current constraints but adopt new, less onerous constraints that still provide for the future evolution of the platform. Revise the format of class files to structure module and package names in a manner consistent with that already used for other kinds of constrained names. [Final proposal; first proposal]

#CompileTimeDependences — Provide a means to specify a module dependence that is mandatory at compile time but optional at run time, for use with libraries that are not strictly necessary but can be leveraged if present at run time. (Detail: If a dependence is “optional at run time” then does the module system try to satisfy it during resolution but fail silently if it cannot, or does it ignore it during resolution but add the corresponding read edge to that module if it was resolved for some other reason?) [Philippe Marschall, Stephen Colebourne, Juergen Hoeller, Stephane Epardaud, Paul Benedict]

Resolution Extend the language of module declarations to allow the static modifier to be used on a requires directive, to indicate that a dependence is mandatory at compile time but optional in later phases. [Proposal]

#ModuleAnnotations — Should it be possible to annotate module declarations? As previously discussed this is technically feasible but it would add significant complexity. [Peter Kriens, Stephane Epardaud, Paul Benedict]

Resolution Allow arbitrary annotations to be placed on module declarations. [Proposal]

#ModuleDeprecation — If module declarations cannot be annotated, should there be some other way to deprecate modules? [Rémi Forax]

Resolution Allow arbitrary annotations to be placed on module declarations, and revise the java.lang.Deprecated annotation so that it can be applied to modules. [Proposal]

#ExportAnnotation — Define a standard, source-retention Export annotation which can be used in a package-info.java file to indicate that the package is to be exported. Tools that create module-info.java files could then interpret this annotation in order to generate corresponding exports clauses. [Paul Benedict]

Resolution Do not implement this feature, since it encourages a practice that would reduce the readability of module declarations. [Proposal]

#CompilationWithConcealedPackages — An exported class can rely upon the name of a concealed class, i.e., a class in a non-exported package, in various ways. An exported class can, e.g., declare that it extends a concealed class, or that one of its public methods has a formal parameter whose type is concealed. When a compiler processes the source code of a module M that refers to an exported class of module N, to what degree must the compiler distinguish N’s concealed class (needed by N’s exported class) from a class in M with the same fully-qualified name? In other words, given that a compiler knows that the “same” package exists in M as well as concealed in N, to what degree must the compiler differentiate the packages? [Markus Keller, Stephan Herrmann]

Resolution Revise the JLS to clarify how a Java language compiler is meant to differentiate between packages with the same name in different modules. [EG discussion]

#ResolutionAtCompileTime — When a compiler processes the source code of a module M, the JLS mandates that the compiler use the Java Platform Module System to “resolve” M’s dependences and ultimately compute the modules that M “reads”. However, the Java Platform Module System is not specified within the JLS, but rather in the java.lang.module API specification. Which parts of the API specification have bearing on the resolution of dependences at compile time, and where is the definition of “reads” in the API specification? [Markus Keller, Jayaprakash Artanareeswaran, Stephan Herrmann]

Resolution Rewrite the specification of the module system’s resolution algorithm, in the java.lang.module package, in order to clarify the definition of readability and the means by which dependences are resolved at compile time. [EG discussion]

#RestrictedKeywords — The grammar of module declarations includes several tokens, such as module and transitive, that were traditionally lexed as identifiers. In support of backward compatibility, the JLS deems these tokens to be lexed as identifiers unless they appear as “restricted keywords” in a module declaration, in which case they are to be lexed as keywords. This rule can, however, be difficult to implement because it is context-sensitive, and hence beyond the capabilities of some parser generators. [Markus Keller, Jayaprakash Artanareeswaran, Stephan Herrmann]

Resolution Revise the JLS to clarify the tokenization issues that arise for the character sequence "requires transitive". Do not adopt an alternative syntax, which would simplify compiler implementations but sacrifice readability. [EG discussion]

Module artifacts

#MultiModuleExecutableJARs — Provide a means to create an executable modular “uber-JAR” that contains more than one module, preserving module identities and boundaries, so that an entire application can be shipped as a single artifact. [This issue has not come up in e-mail but was raised multiple times in person at JavaOne 2015, Devoxx BE 2015, and FOSDEM 2016.]

Resolution Defer to future release. This feature could indeed be useful, but it is not required for the initial release and could be added in a later revision. [Proposal]

#MultiModuleJARs — Provide a means to package more than one module in a single artifact, while preserving module identities and boundaries. (This is a generalization of #MultiModuleExecutableJARs.) [Stefan Fuchs]

Resolution Defer to future release. This feature could indeed be useful, but it is not required for the initial release and could be added in a later revision. [EG discussion, Proposal]

#ReifiedModuleGraphs — Define a means to record, in some type of artifact, the module graph that results from resolving a given main module against a given application module path. The artifact could contain full copies of the corresponding module artifacts, or references to such artifacts, or some combination of the two. At run time such an artifact could be used to create a new layer. [Peter Kriens]

Resolution Defer to future release. This feature could indeed be useful, but it is not required for the initial release and could be added in a later revision. [Proposal]

#AddExportsInManifest — Using a command-line option such as --add-exports to make JDK-internal APIs available to existing code is difficult, if not impossible, for applications that are delivered as executable JAR files. To ease migration, consider allowing such a JAR file to include the equivalent of such options in its MANIFEST.MF file. (Cf. #ReflectiveAccessToNonExportedTypes.) [Simon Nash]

Resolution Define two new JAR-file manifest attributes to allow a package in an explicit module to be exported, or exported privately, at run time. [Proposal]

Module descriptors

#ClassFileModuleName — The name of a module is not a simple UTF-8 string but is, rather, derived from the value of the this_class field of the ClassFile structure, which is awkward. [Rémi Forax]

Resolution Revise the binary form of module declarations to add an explicit module-name field and to zero out the this_class field. [Proposal]

#ClassFileAccPublic — The ACC_PUBLIC constant in a requires_flags should be encoded as 0x0001, as it is elsewhere in the JVMS, rather than as 0x0020, which has different meanings in other contexts. [Rémi Forax]

Resolution Rename this constant to ACC_TRANSITIVE. [Proposal]

#ClassFileAccModule — The ACC_MODULE constant is currently specified to have the value 0x8000. This is the last available bit remaining across all of the various access_flags fields of a class file, and thus should be reserved for some unspecified future purpose where it may be useful to use the same value in all such fields. Alternative candidates for ACC_MODULE include 0x0040 (overlaps with ACC_VOLATILE and ACC_BRIDGE) and 0x0080 (ACC_TRANSIENT and ACC_VARARGS). [Rémi Forax]

Resolution Do not change the value of this constant; there are other, better ways to cope with the future. [Proposal]

#StandardModuleAttributes — The current draft defines a set of optional module-info class-file attributes for extended information not required by the module system but useful to tools: A version string, the name of the module’s main class, the names of the module’s non-exported packages, and, for modules that contain native code, the operating-system name and version and the processor-architecture name. Is this the right set of extended attributes? Should others be considered? Candidates include, but are not limited to: Maintainer contact information, project URL, originating source-code repository URL and changeset identification, and copyright and license notices (or references to same). (Cf. the OS-specific module packaging requirement.) [Peter Kriens]

Resolution Standardize the attributes presently in the specification. A future specification may standardize additional attributes. [EG discussion]

Automatic modules

#CustomizableAutomaticModuleNameMapping — Provide a means to customize the algorithm that computes the names and version strings of automatic modules. [Paul Benedict]

Resolution Extend the java.lang.module.ModuleDescriptor.Builder API so that it can be used to create automatic as well as explicit modules. [Proposal]

#ModuleNameInManifest — For ease of migration, consider allowing the name of a module to be specified in the MANIFEST.MF file of a JAR file. If a JAR file with such a manifest entry is not a modular JAR (i.e., it has no module-info.class file), but is used as an automatic module, then the name of the automatic module is that specified in the manifest rather than that derived from the name of the JAR file itself. This would allow the maintainer of an existing library to specify a preferred module name simply by updating the library’s manifest. If a JAR file with such a manifest entry is a modular JAR then the module name in the module-info.class file determines the module name, and the name in the manifest is expected to match. In both cases, the presence of the module name in the manifest is useful to tools that must run on older versions of the platform, since they can use the existing java.util.jar API to read the module name. [Paul Benedict]

Resolution Implement this suggestion as part of the resolution of #AutomaticModuleNames. [EG discussion; proposal; earlier discussion: Mark Reinhold, Robert Scholte]

#AutomaticModuleNames — Revise the algorithm that computes the names of automatic modules to include the Maven group identifier, when available in a JAR file’s pom.properties file, so that module names are less likely to collide, or else drop the automatic-modules feature entirely, since it may be more trouble than it’s worth. (Cf. #ModuleNameSyntax.) [Robert Scholte]

Resolution Keep automatic modules, implement #ModuleNameInManifest, and strongly recommend that module names follow the reverse Internet domain-name convention. [EG discussion; final proposal; first proposal; discussion here and here; [Stephen Colebourne, Mark Reinhold, Stephen Colebourne]

Module graphs

#CyclicDependences — The current draft disallows cycles when the module graph is initially resolved at compile time, link time, and run time. Cycles can arise later on at run time if readability edges are added for automatic modules, or via reflection. The rationale for disallowing cycles during resolution is that it makes the module graph easier to reason about, it simplifies the module system itself, and that, philosophically, any modules involved in a cycle are logically one module anyway, so they should be defined as such in the first place. This constraint is not, however, a documented requirement, and some have argued that cycles should be permitted. [Jochen Theodorou, Stephane Epardaud]

Resolution Cyclic dependences will not be supported at this time. They could be supported in a later revision, if necessary, after a thorough analysis of the impact of this change on both the specification and the implementation. [EG discussion; Mark Reinhold; later discussion: Robert Scholte, Mark Reinhold; earlier discussion]

#MutableConfigurations — Configurations are immutable: Once created, they cannot be modified. Some have argued that this will make it impossible for container applications to add and remove modules at run time. Should configurations, and hence the aspects of layers that depend upon configurations, be made mutable instead? [David Lloyd]

Resolution Mutable configurations will not be supported. They add complexity and risk, and are not necessary to the achievement of this JSR’s stated goal. [Mark Reinhold; earlier discussion]

#LazyConfigurationAndInstantiation — The construction of a configuration loads all of the relevant module descriptors, and the instantiation of a configuration into a layer defines all of its modules to the virtual machine. There is some concern that this could result in significant startup overhead for layers with thousands or tens of thousands of modules, which some consider to be realistic scenarios. If the startup overhead is actually that bad in such settings, should configuration and layer creation be revised to use incremental algorithms? [David Lloyd]

Resolution Lazy configuration and instantiation will not be supported. This adds complexity and risk, and is not necessary to the achievement of this JSR’s stated goal. [Mark Reinhold; earlier discussion]

Services

#ServiceLoaderEnhancements — The module system encourages the use of services for loose coupling, but the ServiceLoader class is not very flexible. Consider enhancing it so that (1) neither a provider class nor its no-args constructor need be declared public, (2) a provider can be a singleton, or perhaps a collection of singletons, and (3) the classes of the available providers can be inspected and selected prior to instantiation. [Stephen Colebourne]

Resolution Enhance the ServiceLoader API to implement suggestions (2) and (3), but not (1). [Proposal]

Reflection

#ClassFilesAsResources — If a type is visible and was loaded from a class file then it should be possible to read that file by invoking the getResourceAsStream method of the type’s class loader, as it is in earlier releases. [Uwe Schindler, Rafael Winterhalter]

Resolution Special-case resource names ending in ".class" so that they can be located by code in any module. [Final proposal; original proposal]

#ResourceEncapsulation — The Module::getResourceAsStream method can be used to read the resources of any named module, without restriction, which violates the resource encapsulation requirement. This method should be restricted somehow so that only “suitably-privileged” code (for some definition of that term) can access resources in a named module other than its own. An alternative is to drop this requirement (cf. #ClassFilesAsResources). [Mark Reinhold]

Resolution Extend the notion of open modules and open packages, introduced for #ReflectiveAccessToNonExportedTypes, to govern whether a resource defined in a named module can be located by code in some other module. [Final proposal; original proposal]

#ResourceExistenceAndSize — Given a Module object and a resource name, should there be a way to determine whether that resource exists in that module and, if it does exist, what its size is, without actually opening the resource as a stream? One way to do this would be to introduce a new abstraction to represent resources. [David Lloyd]

Resolution Defer to future release. This feature could indeed be useful, but it is not required for the initial release and could be added in a later revision. [Proposal]

#ReflectiveAccessToNonExportedTypes — Some kinds of framework libraries require reflective access to members of the non-exported types of other modules; examples include dependency injection (Guice), persistence (JPA), debugging tools, code-automation tools, and serialization (XStream). In some cases the particular library to be used is not known until run time (e.g., Hibernate and EclipseLink both implement JPA). This capability is also sometimes used to work around bugs in unchangeable code. Reflective access to non-exported packages can, at present, only be enabled via command-line flags, which is extremely awkward (cf. #AddExportsInManifest). Provide an easier way for reflective code to access such non-exported types. [Reinier Zwitserloot, Paul Benedict, Alan Snyder, Rémi Forax, Simon Nash]

Resolution Extend the language of module declarations with the concept of open modules and introduce the new per-package directive open to allow all of the non-public elements of a package, or else such elements of specific packages, to be accessed via the AccessibleObject::setAccessible method of the core reflection API. Rename requires public to requires transitive to avoid confusion. [Final proposal; second proposal; first proposal]

#AwkwardStrongEncapsulation — A non-public element of an exported package can still be accessed via the AccessibleObject::setAccessible method of the core reflection API. The only way to strongly encapsulate such an element is to move it to a non-exported package. This makes it awkward, at best, to encapsulate the internals of a package that defines a public API. [Martin Buchholz & Aleksey Shipilev, private communication, August 2016]

Resolution (see that for #ReflectiveAccessToNonExportedTypes)

#ReflectionWithoutReadability — Having to add read edges dynamically just to enable reflection is painful, and could slow migration and adoption. Consider relaxing the access model so that reflection does not require, or perhaps simply assumes, readability. [Rémi Forax, Peter Levart]

Resolution Adopt the second suggested alternative. Revise the core reflection APIs in the java.lang.reflect package to assume that any module that contains code that invokes a reflective operation can read the module that defines the types that are the subject of that operation. [Proposal]

#ReadabilityAddedByLayerCreator — Provide a means by which the code that creates a layer can add readability edges from the modules in that layer to other modules, whether those modules are in that layer or in other layers. [Thomas Watson]

Resolution Revise java.lang.reflect.Layer so that the multi-parent layer-creation methods added for #NonHierarchicalLayers return a Controller capability object that defines an addReads method. [Proposal]

#IndirectQualifiedReflectiveAccess — Provide a means by which a client module can grant qualified reflective access to a framework module that is not known at compile time, assuming that the name of some other module that represents the framework module is known. A canonical example of this case is a client POJO module compiled against a module that defines the JPA API. At run time the client module, or a container or some other code acting on its behalf, must grant qualified reflective access to its POJO packages to the JPA implementation in actual use. [David Lloyd, Mark Reinhold]

Resolution Revise java.lang.reflect.Module::addOpens so that a module can delegate access to packages that are open to it; extend the layer Controller capability proposed for #ReadabilityAddedByLayerCreator with an addOpens method; and extend the java.lang.invoke API so that framework authors can adopt method handles in preference to the core reflection API. [Proposal]

#MoveModuleAndLayerClasses — Move the Module and Layer classes, and a related exception class, from the java.lang.reflect package up into the java.lang package. This will improve conceptual clarity, be consistent with the past, and leave room for the future.

Resolution [EG discussion; proposal]

Class loaders

#AvoidConcealedPackageConflicts — Provide a simple means to load modules, without using reflection, when they contain non-exported packages of the same name. This could be done by loading them via different class loaders or, alternatively, via #StaticLayerConfiguration. [Karl Sanders]

Resolution Defer a specific solution of this problem to a future release since effective, though crude, solutions already exist and the lack of an immediate solution should not block developers who want to modularize existing applications. [EG discussion; proposal]

#PlatformClassLoader — Add a static method to java.lang.ClassLoader to return a class loader via which all of the built-in Java SE and JDK types are visible. (This is, at present, the loader formerly known as the “extension” class loader.) [Alan Snyder]

Resolution Add the suggested method.

#ClassLoaderNames — Enhance class loaders to have optional names, so that external module systems can provide better diagnostics. When the run-time system generates a stack trace or an exception message that mentions a module name and version, if present, then it should also insert the name of that module’s class loader, if present. [David Lloyd, Mark Reinhold].

Resolution Add a string-returning getName() method to java.lang.ClassLoader and corresponding constructors to that class, java.net.URLClassLoader, and java.security.SecureClassLoader. Arrange for stack traces to convey class-loader names when present. [Proposal]

Versioning

#StaticLayerConfiguration — Layers can, at present, only be created dynamically, via reflection. If there were a way to specify them statically, i.e., at startup time, then they could be used to solve the kinds of version conflicts that are presently addressed via shading or shadowing. [Ron Pressler, Nicolai Parlog]

Resolution Defer a specific solution of this problem to a future release since effective, though crude, solutions already exist and the lack of an immediate solution should not block developers who want to modularize existing applications. [Proposal]

#MultipleModuleVersions — Allow multiple distinct modules of a given name to be loaded in a convenient fashion, without using reflection. This could be done by creating new layers automatically, or by relaxing the constraints on multiple versions within a layer, or by some other means (cf. #StaticLayerConfiguration, #AvoidConcealedPackageConflicts). Addressing this issue may entail reconsidering the multiple versions non-requirement. [Mike Hearn]

Resolution Defer a specific solution of this problem to a future release since effective, though crude, solutions already exist and the lack of an immediate solution should not block developers who want to modularize existing applications. [Proposal]

#VersionsInModuleNames — Some have argued that library maintainers will be tempted to encode major version numbers, or even full version numbers, in module names. Is there some way we can guide people away from doing that? [Mike Hearn, Paul Benedict]

Resolution Abandon the previous proposal to mandate that module names appearing in source-form module declarations must both start and end with “Java letters”. Revise the automatic-module naming algorithm to allow digits at the end of module names. [EG discussion; proposal; discussion: Robert Scholte, Mark Reinhold, Tim Ellison; original proposal]

#VersionedDependences — Consider allowing specific version strings, or perhaps version constraints, as an optional element of requires clauses in module declarations. Failing that, consider allowing specific version strings, or perhaps version constraints, to be added to the dependences recorded in a compiled module descriptor; this would, e.g., allow a compiler or build system to record the versions of the modules against which a particular module was compiled, for use by other tools. In either case, if such version information is merely informative then it will still honor the version selection non-requirement; if such version information is interpreted by the module system then that requirement may come into question. [Cristiano Mariano, Stephane Epardaud]

Resolution When compiling a module that depends on some other modules, record the version strings of those modules, if available, in the resulting module descriptor. [Proposal, as amended per a suggestion by Rémi Forax]

#VersionSyntax — Revise the syntax of module version strings to accommodate a wider variety of existing version-string schemes [David Lloyd]. Alternatively, abandon the requirement to impose a total order on version strings so that we don’t even have to bother to parse them [Rémi Forax].

Resolution The present API is adequate; no further changes are needed at this time. [EG discussion]

#ModuleIdentifiers — To leave open the possibility in a future release of identifying modules not just by their names but also by some sort of versioning information, refactor the java.lang.module API to introduce a ModuleId class and to use instances of that class rather than bare String objects wherever modules are named. This class will, for now, simply wrap a string. In a future release it can be enhanced, if needed, also to carry versioning information. [Mark Reinhold]

Resolution Defer this to a future release. Multiple versions of modules of the same name can, if necessary, be handled in a future revision of this specification almost entirely by overloading the existing String-consuming API methods. [EG discussion here and here]

Layers

#NonHierarchicalLayers — Layers are presently constrained to be hierarchical, i.e., each layer has at most one parent. Should this restriction be relaxed so that a layer can have more than one parent? Some have argued that this will be essential to the adoption of the module system by a future version of the Java EE Platform. It would also enable bidirectional interoperation with existing module systems such as OSGi. [David Lloyd, Mark Reinhold, Thomas Watson]

Resolution Add static methods that take a list of parents, rather than a single parent, to java.lang.module.Configuration and java.lang.reflect.Layer. [Proposal]

#DiscardableModules — Provide a means by which a sub-graph of modules within a layer can be discarded, so that both those modules and their class loaders can be reclaimed. [Thomas Watson]

Resolution Discardable modules will not be supported. They add complexity and risk, and are not necessary to the achievement of this JSR’s stated goal. [Mark Reinhold; earlier discussion]

#LayerPrimitives — Extend the Layer.Controller capability added for #ReadabilityAddedByLayerCreator to expose the underlying run-time and virtual-machine primitives used to implement the module system, for the benefit of other module systems. [David Lloyd]

Resolution Extend the Layer.Controller capability with an addExports method, akin to the method of the same name already in the Module class. This method is strictly less powerful than the existing addOpens method, and thus presents little additional risk, and is potentially useful to frameworks and test harnesses. Do not add any other layer primitives at this time: They add complexity and risk, they are not necessary to the achievement of this JSR’s stated goal, and they could be added in a future release if needed. [EG discussion here and here; Mark Reinhold; earlier discussion]

Tooling

#BootstrapClassLoaderSearchInJVMTI — The JVM TI API defines a function, AddToBootstrapClassLoaderSearch, by which an instrumentation agent can append a file-system path element to the bootstrap class loader’s class path. The bootstrap class path, as such, no longer exists, but this function remains supported for now. Should this function be deprecated, with the intent to remove it in a future release, or should it remain supported? It appears to be critical to the operation of at least one popular instrumentation agent. [Andrew Dinn]

Resolution Retain this method. Do not deprecate it. [Proposal]

#ReflectiveAccessByInstrumentationAgents — As noted in the related issue #ReflectiveAccessToNonExportedTypes, the core reflection API cannot be used to gain access to members of packages that are not exported by their defining modules. Even if that issue is addressed for the use cases cited, a distinct use case is that of instrumentation agents that need to enable injected code to invoke non-public methods and access non-public fields. A possible solution for this use case is specifically to provide instrumentation agents with the ability to use core reflection, or some equivalent, even on elements of non-exported packages. [Andrew Dinn]

Resolution Extend the java.lang.instrument.Instrumentation interface, and the corresponding JVM TI native API, to support a redefineModule method, roughly analogous to the existing redefineClasses method. [Proposal; evaluation by Andrew Dinn]