Iris Clark
The Expert Group met via video conference on Monday, 22 May 2017 at 15:00 UTC in order to work toward closure on various technical issues and prepare to submit a revised Public Review Draft Specification to the JCP by 7 June.
Attendees: Jayaprakash “Jay” Arthanareeswaran (IBM), Alex Buckley (Oracle), Tim Ellison (IBM), Rémi Forax, Brian Fox (Apache Maven, about one hour late), Brian Goetz (Oracle, moderator), David Lloyd (Red Hat), Mark Reinhold (Oracle), and Robert Scholte (Apache Maven).
The intent of these minutes is capture the conversational flow of the discussion and to record decisions. If you are only interested in the latter, search for the word “RESOLUTION” throughout the text.
These minutes were taken during the meeting. All participants had the opportunity to review and correct them for accuracy and completeness prior to publication.
Brian welcomed everybody back. He asked that attendees review the draft minutes of the last meeting which had been sent to the private conference list. These minutes will be sent to the public EG list to form the public record for the meeting. The deadline is noon UTC. After a brief discussion, it was determined that this meeting would run 90 minutes.
This is an update for a pending agenda item from the last meeting.
Mark said that after further study and prototyping by Alan Bateman (Oracle),
it was determined that multiple versions of modules of the same name could be
handled in a future revision of this specification almost entirely by
overloading the existing String-consuming API methods. Mark recommended
moving this issue firmly into the LATER disposition. After Tim confirmed
that a solution had been investigated and potential impact identified, he had
no objections.
RESOLUTION: LATER, since the current specification has the necessary flexibility to extend the module system to handle multiple versions in a later release.
This is an update for a pending agenda item from the last meeting.
Mark reported that an updated working draft of the JLS has been published.
RESOLUTION: PENDING: Eclipse to review and provide feedback (no change).
This is an update for a pending agenda item from the last meeting.
Mark communicated that revisions to the specification were in progress.
Brian noted that for this and the previous issue, there seemed to be no disagreements. Only time was required to produce the solution.
RESOLUTION: PENDING: Oracle to provide an updated specification for review (no change).
This is an open issue.
Discussion of this item is deferred until Tuesday (tomorrow) to allow for key participant attendance.
This is an open issue.
Mark opened this issue saying that there had been a little discussion a long time ago, but not much since then. After reading the brief issue summary, he offered his opinion. Mark noted that he prefers not to do things that are not strictly necessary, but that it was very likely that a total ordering of version strings would be desirable in the future and it would be difficult to impose that restriction in the future. The existing system was designed after studying how Maven, Gradle, Ivy, OSGi, and other popular frameworks and tools handled versioning. The current solution is very general. Some advocates of Semantic Versioning don’t like it because it’s not sufficiently prescriptive. The current design does, however, seem to encompass a wide variety of cases and is good enough for now.
Rémi reminded people that technically there are two “versions”: the “version”
from the API and a “raw version” represented by a String. Code may use
either. Mark agreed saying that on Rémi’s suggestion, a
ModuleDescriptor.Requires::rawCompiledVersion method, had been added;
however, Mark did not recommend its general use. Rémi acknowledged that the
new method was a nice fallback.
David explained that there was a distinction between “version syntax” and “version scheme”. He advocated specifying a syntax instead of applying a scheme. In a syntax, there is a common set of rules that people may take advantage of (primarily for ordering). There is no positional meaning. For every software project there is a different version scheme, typically based on the production process. If a syntax is used, there is a total ordering and everybody will can do what makes sense for them with little impact to others.
Robert said that Maven used major, minor, etc.; however, it wasn’t interesting enough so string comparison was preferred.
Mark agreed with David saying that the API was not intended to be prescriptive and it encompassed many things. There are some differences between this API and OSGi with regard to pre-release strings, but even some OSGi advocates think their system is kind of broken in this respect anyway. This API is consistent with Maven and other systems. It does not try to force the use of Semantic Versioning, though it accommodates it. If six or seven version components are needed, they may be compared in a sensible manner.
In response to Robert’s statement, Mark thought that strings were not
sufficient. As an example, he offered the case of a version which is a
sequence of numbers separated by ‘.’. A sensible comparison via
String::compare is not possible. Parsing is required for a total ordering.
Robert agreed that users are on their own for versions using only numbers
since the API would just be a Comparable.
Mark said that the API intentionally did not provide “major” or “minor” convenience methods. The names of individual components are for somebody else to determine. The API could be extended in the future to return a sequence of tokens.
Brian reviewed, saying that while there was some disagreement, it sounded like people were mostly satisfied with the API and no further changes were necessary.
RESOLUTION: NOW, retaining the existing API.
This issue was included because it was of specific interest to the JCP EC.
Mark summarized this item saying that this comes down to whether there should be a way to load multiple modules when they contain non-exported packages of the same name. When the issue was originally closed, the suggestion was to defer a solution to a future release when there would be more time to design a proper solution. The design was not necessarily straight-forward. Mark confirmed that he understood that some people were concerned about this decision and that it has been raised by the EC. He said he suspects that the near-term impact of not addressing this issue is unlikely to be that great since most code being modularized is currently being run on the class path and is already likely to be free of conflicts. The chances of conflict are further reduced by the common use of the reverse-DNS package-naming convention. Further, while not pretty, there are already standard tools and practices, such as shading, for handling those conflicts. For those reasons, this is not a pressing issue and it could be addressed in a later release.
Rémi concurred, saying that in a perfect world there would be a solution in the initial release. Unfortunately, this was not a problem that could be solved with one day or even one week of work. It should be addressed in the next release.
David provided further background saying that this is a general concern for enterprises when deploying updates. They tend to be stuck on those versions for quite a long time. If a restriction is discovered in a year or two, they often cannot update to the next JDK release without difficulty. He suggested that an interim solution was desirable, even if it was unsupported.
Tim shared David’s concern in allowing people to do something that needed to be tightened later. He expressed his disappointment that this stage was not reached earlier. Tim thought that it should be made very clear to early adopters that this is the first version of the modular platform. They should not be encouraged to re-organize their code around the new architecture, but that new code may be written in a modular fashion. A mature modular platform is expected in a subsequent release. Tim echoed Rémi’s comment that now is not the time to attack this problem.
Tim expanded saying that a temporary solution would still result in a complex discussion with customers. Given the choice, he’d rather have a conversation describing the current limitation than the nature of a temporary solution. David agreed with this conclusion.
Brian recapped, saying that everybody agrees that this is something that the module system should support. Everybody is disappointed that this is not addressed in the initial release. There is broad agreement from Rémi, Mark, and Tim that there are still technical issues that need to be considered and a solution should not be rushed. David was not happy that there was no temporary solution and expected members of the community to share to share his disappointment, but did not have any better ideas.
RESOLUTION: LATER (regretfully), as previously decided.
The previous issue was the final pre-seeded agenda item. After Brian individually surveyed everybody, this is the list of additional items:
This issue was added to the agenda when attendees were surveyed for any additional items.
Jay requested status for the recent proposal sent to the jigsaw-dev list to allow illegal reflective access from code on the class path by default in JDK 9.
Mark addressed this issue shortly after it was added to the agenda. He noted that the related text for the JSR 379 platform specification had not been proposed yet. While the general response on jigsaw-dev was positive, he recognized that there was no decision that would make everybody happy. He expected to send a response and update the reference implementation once these EG meetings were over. Mark could not commit to specific dates.
After Jay pressed for a commitment that there would be no further specification changes, Tim reminded him that as a draft specification, there could be no guarantees that there would not be more changes in the next few weeks.
RESOLUTION: NOW: Oracle to propose appropriate text for JSR 379 and update the reference implementation.
This issue was added to the agenda when attendees were surveyed for any additional items.
David’s concern was the cyclic dependency check at runtime. He began by saying that an essential feature of the module system is that recompilation of everything is not required whenever a dependency is updated. The issue is that cycles may be introduced at runtime as the system evolves. He noted that if no cycles are present at compile time, then there is no benefit to the JVM in detecting them at runtime. He thought that this would most likely be encountered by enterprise systems and the problem should be addressed now because addressing it is easy.
Rémi professed that he had seen such a system and wanted a switch that disabled the runtime circularity check. He would want to fix the circularity, but that would not be practical at deployment. The switch would acknowledge this difficulty. David agreed that by the time the problem is detected at deployment, it is too late. It was not always practical to solve a circularity problem in a short time.
Alex requested verification that the cycle check was not a problem at compile time, only at runtime. David and Rémi agreed, adding that this is essentially how Maven behaved. Alex rephrased saying that the first person who discovers the problem is some unfortunate individual who is the first to run with the circular versions of the two components. Robert said he concurred with David and Rémi’s evaluation.
After understanding the problem, Alex made the broad observation that there were plenty of things that could go wrong when there is independent development. For instance, at runtime there could be a discovery that an automatic module’s name has changed (Robert’s issue), or that a module’s exports have been reduced. He noted that rollback to a previous version of the components, before the cycle was introduced, would solve the problem.
David said that avoiding cycles at runtime is a conceptual problem. He continued saying that cycles are not a good design practice; however, these did not cause failures, problems, or bugs. The prohibition of cyclic dependencies existed purely to encourage good design practice. This made it difficult to explain to management why an upgrade failed.
Mark did not think that the solution to this problem was simply a matter of changing a few lines of code. He cited a few scenarios which would need to be resolved, including specification complications with cycles and the potential for class-loader delegation loops at run time. He also reminded the group of a philosophical point, namely that this restriction is meant to lead developers to better design practices for large systems, so it does have value. Since this restriction could be lifted in a later release, he thought it best to be conservative for now and wait for actual experience.
Rémi said that he mostly agreed with Mark but practically speaking he wanted a disabling flag on the command-line. Without such a flag, he expected people to wait until there was a version of the module system with that capability. Mark reminded Rémi that there was no way to have an “invisible” command-line flag, since it would need to be allowed by the specification.
Alex believed that nobody should be able to pick an arbitrary set of modules and expect everything to just work. David responded saying that deployers do that all the time. They do things that are well beyond what the original developer envisioned. Going back to Mark’s comment regarding cycles complicating the implementation, David wanted to know if this was something theoretical or possible. Mark replied saying it would take serious investigation to ensure that there were no problems. He added that there would be issues with both the specification and with adequate testing.
Brian recapped the discussion saying that everybody seemed to agree that cyclic dependencies could become a problem at run time. In theory there was no aversion to having a different runtime response; however, additional, non-trivial analysis was required. Once there is confidence in a solution, the runtime behavior could be changed at that time.
RESOLUTION: LATER: After additional, thorough analysis of issues around run-time cyclic dependencies, it may be possible to relax the requirement.
Shortly before the end of this topic, Brian Fox joined the call. After the resolution, Brian Goetz summarized the set of new agenda items and asked whether Brian Fox wished to add anything. He declined.
This issue was added to the agenda when attendees were surveyed for any additional items.
Mark read the issue summary from the issue list, adding that while the
requested addExports, addUses, and addPackage primitives exist in the
reference implementation he was reluctant to expose them. They weren’t
designed for general use; they were added to the reference implementation for
whitebox testing and were under consideration for removal since they were of
little value for that purpose.
David said that being time-aware, he wanted to discuss the items individually.
For addExports, he indicated that it would be useful to add an export
without opening a module. The only way to do that currently is via injected
bytecode.
Rémi thought that part of the problem was that the current API was not the
right one. He outlined two aspects of the problem that need to be considered:
overriding the module-info.class file and overriding at arbitrary times
during runtime. He thought the first was just adding new things. The second
could potentially override module-info before creation. He suspected that
there may be problems with the latter.
David thought that this was a bigger problem due to layers and he did not wish
to venture into that “rabbit hole.” For now, it seemed logical to have an
addExports given the existence of addOpens.
Brian noted the time and suggested that addPackage seemed to be the most
controversial. He suggested that the conversation move towards that item.
David begin by saying that the ability to add classes at any time already existed. He thought that adding a package wasn’t that different and the distinction between classes and packages was not one that a user would intuitively understand. Further, he didn’t believe that it would be an unreasonable request since it did not push the boundaries of other dynamic systems. David and Rémi disagreed about the identity of a module. Rémi declared that a module is defined by all of the packages it contains; hence, adding a package changes the identity. David countered saying he defined the module’s identity by the ABI, i.e., by the set of classes that it exposes. Rémi replied saying that private packages were part of the ABI because #AvoidConcealedPackageConflicts had not been implemented yet.
Mark made the general observation that in this situation, the addPackage
method would impose additional constraints that would restrict how the
platform and its implementations can evolve in the future. He said that the
reference implementation added code to the JVM that used package indices for
access control, and requiring every JVM to be prepared to allow those indices
to be updated at any time seemed unwise, particularly since it was not needed
in this module system. Mark thought that adding addPackages would impose
constraints on the future evolution of JVMs without clear justification.
Brian reviewed the situation saying that there appeared to be two strong objections and that the arguments in favor seemed to be of the form of “it would be nice.” He suggested that rational thing to do was to consider this later.
David went back to describe the addExports case, saying that it was a
compatibility breaker. Prior to the current release, ServiceLoader could be
used to statically wire modules based on services. He expected to introduce
tools to mitigate the new problems which could be solved via bytecode
generation (not ideal).
Further explanation and discussion was interrupted due to lack of time.
RESOLUTION:
addExports— PENDING: additional discussion required;addPackage— LATER: if the additional complexity and constraints can be justified;addUses— PENDING: no time for discussion today.
Brian made a time-check. After a brief discussion, a meeting at 15:00 UTC for 90 minutes on Tuesday was agreed upon.
At this point the EG adjourned.