JEP draft: javadoc tags to distinguish API, implementation, specification, and notes

Summary

This informational JEP describes certain JDK-specific javadoc tags that have been introduced in JDK 8 in order to improve the clarity of the specification and related documentation, and also to improve the inheritability of documentation by subclasses.

Goals

Prior to JDK 8, the "javadoc" consisted mainly of the Java SE API specification, with some documentation about implementation and informative notes mixed in. These different kinds of documentation were not always denoted clearly, and the normative force of statements about implementation was also often unclear. This JEP describes javadoc tags that allow these different kinds of documentation to be placed into separate documentation sections and also that allow subclasses to inherit them selectively.

This JEP is also initially intended to document existing usage of these tags in JDK 8. It is not intended to develop new javadoc tags or other enhancements.

Non-Goals

The new tags are JDK-specific and are not intended to become standard javadoc tags at the present time.

This JEP is not a proposal to develop new javadoc tags, but is limited in scope to documenting the tags that were introduced in JDK 8.

Motivation

What many people refer to as "the javadoc" is officially titled the "Java(tm) Platform, Standard Edition N API Specification". The specification primarily describes what a piece of the API should do (often in terms of preconditions and postconditions) and not how the API is implemented. It's thus tempting to conclude that the specification applies only to the API. However, this is not the case. An abstract class contains an implementation, and subclasses must be able to rely on specific behavior of this implementation (such as when super.method() is called), and also to determine whether they should inherit or override the implementation. This implies that the specification must also include requirements on implementations. With the introduction of default methods on interfaces in Java SE 8, interfaces can also now include implementations. We expect default methods will be used quite frequently; thus the need to specify implementations is much greater in Java SE 8 than it was in previous releases.

For example, consider the specification for AbstractMap.putAll. It says in part,

This implementation iterates over the specified map's entrySet() collection, and calls this map's put operation once for each entry returned by the iteration.

This is essential information for subclassers. It specifies that putAll doesn't bypass the put method, so subclassers need only override put in order to implement some custom behavior, and they need not override putAll. However, if they do override putAll, the overriding method can call super.putAll() and rely on the specified behavior of AbstractMap.putAll. The situation is similar with default methods. Classes that implement interfaces containing a default method will need to know whether they can rely on the default method's implementation, or whether it is useful or necessary to override the default method.

Note the difference between the specification of AbstractMap.putAll and the specification of Map.putAll:

The effect of this call is equivalent to that of calling put(k, v) on this map once for each mapping from key k to value v in the specified map.

The words effect and equivalent to make it clear that the Map.putAll specification is describing the results or postconditions of calling the method, but not how the method does its work. (Other specifications may use other words with similar meaning, such as "The effect is as if ....") By contrast, the specification of AbstractMap.putAll states directly that it calls the put operation: no "as if" or "equivalent" words here. It is clearly a specification of how AbstractMap.putAll must do its work.

The above excerpt from the AbstractMap specification used the phrase "this implementation." In this context, it means "the implementation of the putAll method that resides in the AbstractMap class." Although this describes the implementation, this is just as much a part of the specification of the AbstractMap class as the API specifications. The AbstractMap.putAll method of every release of every Java SE product should behave the same way. We would expect a conformance test suite to test that the putAll method indeed calls the put method for each entry.

By contrast, consider the following excerpt from the Runtime.loadLibrary method:

The details of this process are implementation-dependent.

In this context, "implementation" means the particular implementation of Java SE, such as OpenJDK, the Oracle JDK, an independent implementation of Java SE, or even the "same" version of a JDK running on a different operating system. For example, it should not be surprising if the behavior of Runtime.loadLibrary were to differ between OpenJDK running on Linux and OpenJDK running on Windows. Unlike the AbstractMap example, the behavior of Runtime.loadLibrary is allowed to vary across products and platforms. Thus, the statements about the implementation of Runtime.loadLibrary lack the force of specification of the statements about AbstractMap.

One might argue that that implementation specification example of AbstractMap.putAll given above is really an API specification at an unusually fine-grained level of detail. This is not really the case, and it's illustrated when we consider inheritance.

The API specification for putAll is defined in the Map interface. It applies to all subtypes of Map. It is generally considered a design error for some descendant of Map to have a putAll method that violates assertions made in the Map.putAll specification.

On the other hand, the implementation specification for AbstractMap.putAll applies only to AbstractMap itself and its descendants that have chosen to inherit putAll. It's quite possible for a descendant of AbstractMap to have its own implementation of putAll that bypasses the put method. Of course, that implementation would still need to conform to the requirements laid out by the Map.putAll specification. Thus, there is a need for this descendant class to selectively inherit portions of its ancestors' documentation. It needs to inherit the API specification from Map.putAll, but it needs to avoid inheriting the implementation specification from AbstractMap.putAll.

Specifications typically consist of assertions, preconditions, postconditions, and the like. The documentation, however, contains other information, such as background, rationale, usage considerations, and the like. Standards documents refer to the specification statements as "normative" and other documentation as "non-normative" (sometimes "informative"). For brevity, this document will use the terms "spec" and "notes" for these concepts.

We have laid out two dichotomies: between API and implementation, and between specifications and notes. There are thus four different categories of documentation that appear in "the javadoc." We have further identified the need for subtypes to selectively inherit pieces of documentation from their supertypes. We have therefore introduced several new javadoc tags that can be used to separate documentation into different sections corresponding to these different categories. We have also modified the behavior of the @inheritDoc tag to inherit only the particular section of the documentation in which the tag occurs.

Description

There are four categories of documentation that we need to distinguish using javadoc tags:

1. API Specification – (no tag)
2. API Notes – @apiNote
3. Implementation Specification – @implSpec
4. Implementation Notes – @implNote

The initial text of a javadoc comment is in the API Specification category. When one of these tags is used, the subsequent text is put into the associated category. All of these tags are optional. If none of the tags is provided, all of the documentation is considered to be API Specification.

The category definitions are:

API Specification. This is the most familiar, as most of the javadoc is already API specification. This includes specifications that apply to all valid implementations, including preconditions, postconditions, etc. Documentation is considered to be API specification by default, so there is no tag for this category.

API Notes. This category consists of commentary, rationale, or examples pertaining to the API.

Implementation Specification. This is where the default implementation (or an overrideable implementation in a class) is specified. Interface implementors or class subclassers use the information here in order to decide whether it is sensible or necessary to override a particular method, and what behavior they can rely on if a method is called via super.

Implementation Notes. This section contains informative notes about the implementation, such as advice to implementors, or performance characteristics that are specific to the implementation in this class of this version of the JDK. The information in this section is subject to change from release to release. These characteristics are also allowed to vary across platforms, vendors, and versions.

The @inheritDoc tag's behavior has changed to inherit only the documentation for the section within which it appears. This allows finer-grained control over what portions of documentation are inherited. For example, a default method usually will include both API specification (which should apply to all implementations) and implementation specification (which applies only to the implementation of the default method). If an implementing class were to specify @inheritDoc without specifying any category tags, it would inherit only the API specification and not the default method's implementation specification. This is usually the right behavior for implementing classes that override a default method. If the implementing class wants to inherit the implementation specification as well, it should start an @implSpec section and have another @inheritDoc tag within it.

These tags are of most use in method documentation, but they can also appear in class and field documentation. A typical documentation comment that uses these tags will look like this:

/**
 * ... API specifications ...
 *
 * @apiNote
 * ... API notes ...
 *
 * @implSpec
 * ... implementation specification ...
 *
 * @implNote
 * ... implementation notes ...
 *
 * @param ...
 * @return ...
 * @throws ...
 */

Implementation

Support for these tags was not added to javadoc itself, nor the standard doclet (which is responsible for producing the usual HTML output). Instead, the standard doclet supports a custom tag feature that is configured by supplying the -tag command-line option. The new tags were implemented by adding suitable -tag options to the javadoc command line invocation in the JDK makefiles. See JDK-8008632. An example javadoc command line to support these tags would look something like this:

javadoc -tag 'apiNote:a:API Note:' \
        -tag 'implSpec:a:Implementation Requirements:' \
        -tag 'implNote:a:Implementation Note:'

The standard doclet needed to be enhanced to support the @inheritDoc tag in the desired way. See JDK-8008768.

Examples

A class WombatFactory might have a method documented as follows:

/**
 * Returns a list of Wombat instances retrieved from the platform's
 * Wombat repository. The returned list will not contain any duplicates;
 * that is, the result of calling w1.equals(w2) will always be false, where
 * w1 and w2 are any two different elements of the returned list. The
 * returned Wombat instances match the name given as an argument.
 * Whether name matching is case-sensitive or -insensitive is
 * platform-specific behavior. 
 *
 * @apiNote
 * This method returns a List instead of a Collection or Stream,
 * because processing of multiple Wombats usually involves traversing
 * the list in alternating forward and reverse directions.
 *
 * @implSpec
 * The implementation in this class returns a List where
 * access to elements by index is O(1). The returned List is
 * serializable, but it is not guaranteed to be thread-safe.
 *
 * @implNote
 * The JDK Reference Implementation returns the list of wombats sorted
 * by weight, although this is not required of all implementations.
 */
public List<Wombat> getWombats(String name) { ... }

There might be a LazyWombatFactory subclass that selectively inherits the API specification but not the implementation specification:

/**
 * {@inheritDoc}
 *
 * @implSpec
 * The implementation in this class returns a List that is populated
 * with Wombat instances that might not be fully initialized. If
 * get() is called on a not-initialized Wombat instance, the call
 * blocks until the instance has been initialized; otherwise, it
 * returns immediately. The returned List is not serializable, but
 * it is thread-safe.
 */
@Override
public List<Wombat> getWombats(String name) { ... }

References

The initial proposal was discussed on the JSR-335 (Lambda Libraries) Expert Group mailing list. The discussion continued into February, with messages having the Subject line, "Javadoc conventions in the presence of default methods."

The implementation changes to support these new custom tags are covered by bug JDK-8008632.

The implementation changes necessary to support the new @inheritDoc model are covered by bug JDK-8008768.

Alternatives

An early proposal considered modifying the standard doclet or javadoc itself to support these tags. It was considered premature to "bake" these tags directly into those components without any experience. Instead, these tags were implemented using the standard doclet's custom tags feature.

An alternative would be to do nothing and to just live with the current mechanisms. The issues described above could simply be described in prose, without any formatting or markup changes. While this is true in principle, prior to Java SE 8, the issue of implementation specification occurred relatively infrequently (in abstract classes that provided implementations in support of their subclasses) and so the platform was able to muddle through without any specific markup support. The introduction of default methods on interfaces in Java SE 8 is expected increase the frequency with which these specification issues occur, leading to a greater need for the tools to support these cases more formally.