Iris Clark
The Expert Group met via video conference on Tuesday, 23 May 2017 at 15:00 UTC in order to work toward closure on various technical issues and prepare to submit a revised Public Review Draft Specification to the JCP by 7 June.
Attendees: Wayne Beaton (Eclipse, for the first ~10 minutes), Sasikanth Bharadwaj (Eclipse, about 5 minutes late), Alex Buckley (Oracle), Tim Ellison (IBM), Rémi Forax, Brian Goetz (Oracle, moderator), Stephan Herrmann (Eclipse), David Lloyd (Red Hat), Mark Reinhold (Oracle), Robert Scholte (Apache Maven), Tom Watson (IBM).
The intent of these minutes is capture the conversational flow of the discussion and to record decisions. If you are only interested in the latter, search for the word “RESOLUTION” throughout the text.
These minutes were taken during the meeting. All participants had the opportunity to review and correct them for accuracy and completeness prior to publication.
Brian started the meeting by welcoming everybody back and doing a brief roll call. He asked Mark whether the minutes for Thursday’s meeting had been posted. Mark replied saying they’d be sent to the public EG list shortly, and that Monday’s minutes would be posted to the private conference list for review later today.
Today’s agenda:
This issue was added to the agenda when attendees were surveyed for any additional items.
Discussion of this item began at Monday’s meeting (yesterday). There was
resolution for the ModuleLayer.Controller::addPackage method. Since
discussion for the addExports method had started, but did not finish, Brian
suggested that David re-start that topic.
David started by saying that addOpens was already in the layer controller
API. It both exports packages and opens them for deep reflection. He wanted
a means to only achieve the former. David thought that addExports was a
natural and low-risk API which would allow for more dynamic deployments. It
was low risk because addOpens was already supported, and does more. As a
use case, David described a deployment in which an framework needed to access
a module, but deep reflection should not be allowed. He wanted minimal
privilege access.
Brian paused David’s description to note that an additional attendee had dialed in, and requested that the person identify himself. After Sasikanth’s introduction, there was some discussion about whether a third representative from Eclipse was reasonable. Wayne said that it was a coordination issue. He had made an internal request for a team representative and two people came. Mark and Tim shared the opinion that a third person reduced focus and was not generally helpful. Mark noted that the original meeting request was for at most two technical representatives, and that participants were free to discuss the meetings with their colleagues or distribute draft minutes as necessary. The publication delay was not intended to keep things private, but to ensure accurate records. After further discussion, Wayne decided that Stephan and Sasikanth were the appropriate representatives given the agenda and he dropped off the call.
Brian apologized to David and asked him to continue describing his use case.
David declared he was done and asked for comments. Tim was the first to
respond, asking whether the framework would be trusted enough to use the open
API. He thought that the framework already had ultimate rights over the
lifecycle and there was no benefit in using a restricted API. David replied
that frameworks we not always privileged. He also said that part of the
problem was that they were in the prototype stage but addExports seemed to
be needed and it was a low risk addition.
When Brian asked for alternatives if the API was not added, David explained
that addOpens or some kind of bytecode generation and injection would
work. The latter was not attractive due to higher maintenance cost. In
contrast, the requested API was easy to implement and its presence avoided a
possible problem.
Mark said that he found the use case less than convincing. After outlining
what he considered the strong use cases for the existing Controller methods,
Mark said that he was nervous adding an API for a hypothetical use case when
there was a workaround. David responded saying that he did not understand how
addExports would be detrimental unless there was a change in the definition
of addOpens. He considered this drastic change highly unlikely. David
repeated that in his view, the risk was essentially zero.
Tom wondered whether addOpens had the potential to extend the class space.
He was concerned about the introduction of split packages. Mark addressed
Tom’s concern by saying that addOpens and addExports do not change class
loader delegation or class visibility in any way. They can introduce split
packages. As sharp tools, they should be used with care. If code introduces
an inconsistency, that code will have to deal with it. While Tom agreed with
David that the addition sounded reasonable, he did not think that the risk was
zero.
Tim contributed that he was struggling to see what the essential problem was
because the key functionality existed in addOpens. He understood the desire
to minimize surface area.
Rémi described an additional use case related to JUnit. When JUnit (or a similar testing framework) is used, there is often a need to export a specific class to the JUnit module. He solved this problem by generating a class (not bytecode) inside the module containing exactly what is needed for testing. His concern was that the addition could be done at any time, not just at layer creation. Tom countered that if the layer controller was not dynamic, it would not be helpful. Mark stated that the JUnit use case was new to him and that the JDK-specific unit-test framework had already solved this problem in a different way.
Mark asked David whether the deferral of addPackage had any impact on his
case for addExports. David replied that since the addPackage resolution
was anticipated, his resolver does as much as possible up front. The
addExports use case was generally the first related to dynamic deployment
after the fact; however, there were others. Mark said that given that there
were now two use cases, one from David and one from Rémi, he thought it
reasonable to add this method in the current release as long as no issues turn
up in its implementation.
Brian summarized the conversation saying that there seemed to be a variety of
use cases and that the incremental risk over addOpens seemed low.
RESOLUTION:
addExports— NOW, assuming no adverse discoveries during implementation. If problems are discovered, they will be brought back to the EG.
The last sub-issue was the addUses method. It was discussed briefly
yesterday. David explained that the need for this API was discovered during
framework ports. David said that the solution was similar to what was needed
for core reflection and that it would be annoying to require a multi-release
JAR just for this case. He proposed that addUses either be automatic or else
mitigated some other way.
Rémi asserted that if there is no module, an addUses is not necessary. He
expressed concern that frameworks which generate “uses” relationships at
runtime would be precluded from static analysis. David requested
clarification. Did Rémi not like the use case where somebody can load a
service that they did not declare? Rémi replied that if somebody creates a
module, they need to follow the rules of modules. David pointed out that if
an API had providers that are declared in the consumer instead of the module
there would be problems. This was currently solved using a ServiceLoader on
behalf of the caller.
Mark joined in saying that the situation was starting to sound like the path that ended in #IndirectQualifiedReflectiveAccess. He thought it sounded like the problem space was not fully understood and that more experience was necessary. Mark suggested that designing an indirect service-loading facility in the next week sounded risky. David suggested “automatic” uses in the same sense that readability relationships are automatically assumed for reflection.
David asked whether there was any gain in making the user “pause”, in the form
of an addUses call, when they wanted to have access to a service. Alex said
that there was a difference between invoking ServiceLoader and performing
core reflection. Only advanced code is expected to perform core reflection.
For such code, addReads has no value so the need to call it was removed. On
the other hand, ordinary user code is expected to call ServiceLoader::load
against the backdrop of its module declaration containing uses directives.
As such, ServiceLoader’s load method expects that usability has been set
up, either by uses directives (ordinary user code) or calls to addUses
(advanced code). In the corner case of advanced code calling
ServiceLoader::load without uses directives in its module declaration,
the need to call addUses is the price for the readability of ordinary user code
that relies solely on explicit uses directives. David agreed that this
analysis made sense. He was concerned about the potential that nobody
declared a service that was later used. Alex concluded by saying that the
apparent need for addUses was due to things not working cleanly.
As an alternate solution for addUses, Rémi offered a preliminary suggestion
of using the MethodHandles.Lookup API as a means to create indirect calls while
avoiding bytecode generation. He cautioned that he was not sure that this was
a good solution and additional thought was required. David reiterated that the
burden should be on the framework. Since he did not think that this was an
emergency, continued discussion could be on the public EG list.
Brian summarized by saying that there are workarounds and valid concerns about making it too easy for services to be undeclared. He asked David whether a “LATER” disposition would be acceptable, so that this issue can be explored further after use cases without workarounds were identified. David agreed.
RESOLUTION:
addUses— LATER, after additional exploration is done and more use cases are identified.
This is an open issue.
Stephan outlined three concerns. First, as it stands the specification is
unacceptable because it could not classify a token until after the entire
module declaration has been consumed. This could be reduced to the extent of a
single module directive, but anything finer-grained would be a heuristic.
Second, since the Eclipse compiler is at the bottom of a tall stack of plugins
(not a stand-alone application like javac), use of a heuristic had the
potential to disturb the entire ecosystem if there is a breaking change.
Finally, any implementation would be based on very specific heuristics. If
modules were extended in a later iteration, the heuristics would break. He
noted that he had sent a proposal for an escaping mechanism to the
jigsaw-dev
list.
When Stephan finished describing the problem, Alex asked whether the specification was unacceptable or the problem was with the feature. Stephan declared that feature itself was unacceptable.
In Rémi’s opinion, since there are millions of developers and only a few compiler implementors, he preferred to burden the implementors. Stephan observed that perhaps the scale of the impact was under-appreciated. If the burden of the implementation is on the IDE, any problems in that IDE would affect millions of developers. Rémi respectfully disagreed.
Mark said he strongly agreed with Rémi adding that a key value of the Java language is readability. He thought that introducing an obscure rule with an obscure escaping syntax added unnecessary complexity to the language. In his opinion, readability vastly outweighed implementation considerations. Stephan regretted that there was no heuristics to measure user impact. He said that some systems already have such escape mechanisms and that the change was small.
Stephan confirmed Alex’s requested verification that parsing transitive
required one additional token of look-ahead. Stephan added that the scale was
the key concern. David mused that an alternative to escaping was quoting as
for strings. He expected this to be a horribly unpopular suggestion.
Alex pointed out that there was a long list of places in the Java language
where lexing and parsing are difficult. A big example is described at the end
of the Java Language Specification (JLS), section 3.2 for '>' characters.
There were similar issues for the repurposing of this as a name in the JLS
for Java SE 8. Alex noted that restricted keywords are coming anyway due to
var in the proposal for local-variable type inference. Rémi and Alex briefly
discussed a few additional cases such as requires transitive, which is
illegal in the Public Review specification since transitive is specified as a
keyword.
Summarizing a resolution, Alex thought we should keep the restricted keywords feature and clarify their treatment in module declarations. This will be done with the knowledge of future potential impact to ordinary keywords. Rémi said that he found this acceptable.
Stephan disagreed. The required look-ahead of at most one was of concern. He
expected it to break when allowing annotations in more locations in
module-info.java files. Maintainability was also a concern due to the added
heuristic which would likely break during future language changes. He thought
that highlighting tokens would be broken for existing tools, and his tool’s
future development would be harder than necessary.
Mark observed that there was resolution with one objection. The resolution meant “NOW” but there was also agreement that resolution for Stephan’s proposal was “NEVER”, since affirming this choice now would close off the possibility of doing something different in a future release.
Brian reiterated the resolution saying that “NOW” referred to clarifying the specification to reflect what the compiler must parse, and to classify Stephan’s request to avoid context-sensitivity in the parser as “NEVER”. Mark added that the specification and reference implementation are already well down the road and that we think this is the right balance.
RESOLUTION: NOW: Oracle to clarify the specification; NEVER, for the delivery of an alternate solution.
Mark said that he’s glad that the discussion occurred but that this issue was raised extremely late. He encouraged early engagement. David agreed adding that most of the community got involved way too late causing a host of headaches. He hoped that the community would learn from this. While there was quite a bit of effort to encourage early engagement, more could always be done.
Stephan said that he had originally brought this issue to the jigsaw-dev list
this past November, and that he had been principally working from
lang-vm.html which contained the expected JLS changes. These changes were
folded into the JLS draft included in the January 2017 Early Draft Review.
Stephan felt that his feedback had been prompt.
This issue was added to the agenda when attendees were surveyed for any additional items.
Robert wished to provide advice to developers about when to use module descriptors. Automatic modules whose names are derived from their file names should not be referred to by name because their names are unstable, and doing so risks breaking the ecosystem. It is understood that some projects cannot be transformed to use modules because of structural export issues. New projects using Java 9 should follow JPMS rules.
Rémi encouraged library maintainers to convert to modules rapidly.
Mark thought he agreed with Robert but wished to repeat the advice to confirm understanding. If you maintain a library then you should modularize it explicitly only after each of its dependencies has either been modularized explicitly or else enhanced to specify an automatic module name in its JAR-file manifest. If you maintain an application then it’s fine for its modules to depend on automatic modules whose names are derived from their file names, just make sure that your modules are not published for broad use since they will almost certainly need to be revised when the automatic modules upon which they depend are given proper names.
Mark said that he did not expect every existing component to be modularized (e.g. popular, but dead projects). In that case, it might be acceptable to publish a module that depends on an automatic module with a file-derived name. Room for judgement needed to be maintained for such corner cases.
Robert said he agreed with Mark’s summary. Tim agreed and emphasized that things should be designed based on module system design assumptions. He thought that this advice should not be in the specification, but clear communication was needed to describe these assumptions and to guide people in the right direction.
Brian summarized saying that it sounded like everybody was in agreement in terms of specific advice and Tim though this advice should be communicated in a FAQ or other documentation. Mark suggested that the “other documentation” could be a revised State of the Module System document. He anticipated an update of that document soon.
This issue was included because it was of specific interest to the JCP EC.
Mark introduced this issue by saying that it was one of those issues that was closed some time ago on the basis that it could be added later. Tim agreed with the resolution but said he added this item to the agenda because of a JCP EC PR ballot comment. He had not seen any example scenarios to support the ballot comment. Tim thought that without further information, the EG should feel comfortable with the existing resolution.
David believed that it should be easy to implement a prototype. Mark noted that Spring Boot already had an implementation. He agreed with David that this feature could be implemented at the user-level either as a prototype or a working product.
Brian recapped saying that everybody seemed happy with the “LATER” resolution.
RESOLUTION: LATER, as previously decided.
Brian said that the previous issue was the final technical item on the agenda. There were two pending items: #CompilationWithConcealedPackages needed review from Eclipse and #ResolutionAtCompileTime required Oracle to provide an updated specification.
David requested a quick review of #LayerPrimitives since he had audio problems at yesterday’s meeting. He wished to ensure that the EG had properly understood his concerns and proposal. After David reviewed his concerns, Mark assured him that yesterday’s proposal was understood.
Brian moved to the final topic saying that with one exception, #RestrictedKeywords, there were no objections to the resolutions. Almost everybody wished that we could have made more progress for the initial release on some of the items.
Reaffirming what Brian said, Mark enumerated four issues that would be good to investigate for a future release (the “LATER” list):
addUse and addPackageMark said he would attempt to condense the reasoning for why each was considered but not addressed at this time, and include that reasoning in an update to the specification.
The discussion then moved to JCP Public Review (PR) process. David suggested publishing the revised specification with sufficient time for the JCP EC to provide comments prior to the PR Ballot submission. Brian noted that the process included a two week voting period for JCP EC review. Any added delay would lose valuable time. David was concerned that without a specific review period for the JCP EC, the specification may still be rejected. He wanted to ensure that people were minimally happy. Brian said that posting the minutes would provide a window into the specification and the EG’s positions.
Tim asked whether it would be helpful to prioritize the “LATER” list. Mark said that there were limits to what any particular EG could expect a hypothetical future EG to do; however he was open to prioritizing the “LATER” list. Mark expected to have an interim working draft of the 376 specification later this week. He encouraged the JCP EC to look at that material since it would be submitted for reconsideration no later than 7 June.
Brian asked for any final comments saying that there would not be a meeting tomorrow (Wednesday). Tim said that he thought that the meetings had been a very useful process because things proceeded much faster than they would have in e-mail. He thanked Brian for moderating and Iris for taking minutes. Brian thanked everybody for being available on such short notice. Mark echoed this sentiment.
Brian declared the meetings a success, and the EG adjourned.