JEP draft: Keyword Management for the Java Language

OwnerAlex Buckley
Componentspecification / language
Discussionjdk dash dev at openjdk dot java dot net
Created2019/04/26 00:26
Updated2020/01/23 22:55


Evolving the Java language often means new keywords for new features, but new keywords risk breaking old programs. To balance compatibility and readability, a new kind of keyword may be used: a hyphenated keyword that is a compound of pre-existing keywords and identifiers, such as non-final, break-with, and short-circuit.

Note: All examples in this JEP are intended solely to illustrate a syntactic form under discussion. They are not intended to suggest that any particular language feature is being considered for inclusion in Java now or in the future.




A keyword is a sequence of ASCII letters that cannot be used as an identifier in Java programs. Java uses a small set of keywords to denote the most fundamental features of the language:

Over time, Java language designers face a challenge: the keywords conceived for the features of Java 1.0 are rarely suitable for denoting new features. There are several obvious techniques for addressing this problem:

For most new features, all of these techniques are on the table -- but most of the time, none are very good. Given that all of these techniques are problematic, and there is not even a least-problematic technique that works in all situations, it is desirable to try to expand the set of syntactic forms that serve as keywords. Otherwise, the lack of reasonable techniques for extending the syntax of the language will become a significant impediment to language evolution.

In addition, modifiers like static and final make up a quarter of all keywords, but the set of modifiers is not complete; there is no way to say "not static" or "not final". Consequently, it is not possible to create features where variables or classes are final by default, or members are static by default, because there is no way to denote the opt out of "not static" or "not final". Leaving a feature out of Java for reasons of simplicity is fine; leaving it out because there is no way to denote the obvious semantics is not. This is a constant problem in evolving the language, and an ongoing tax paid by every Java developer.


Syntax in feature design

The best syntax for a new feature -- whether declaration, statement, or expression -- is inherently feature-specific.

Some features are denoted best with tokens other than keywords: the operator -> for a lambda expression, the separator :: for a method reference expression, and the separator ... for a varargs parameter declaration. Meanwhile, features that support built-in types tend to find their own syntactic ground independent of keywords: the literals true, false, and null, the prefix 0b for binary literals, the suffixes L, F, D, etc for numeric literals, and the delimiter """ for text blocks.

Most features, though, are denoted best with keywords whose length, alphabet, and tone align with pre-existing keywords. That means 2-20 ASCII letters which spell out a simple noun, verb, or adjective of U.S. English. Traditionally, there were two kinds of keyword that met these constraints:

Each classic and contextual keyword is a unitary keyword, that is, made up of one token. This JEP opens up new syntactic ground by allowing multiple tokens to be joined together by the token -, with no white space between any of the tokens. The result is a hyphenated keyword. The token - which is usually an operator is pressed into service as a hyphen; this has implications for compiler engineers and for language designers, discussed later.

There are two kinds of hyphenated keyword, depending on the tokens being joining together:

Hyphenated keywords

Hyphenation admits a rich array of phrases relevant to current and potential constructs of the Java language:

Hyphenated classic keywords (at least one classic keyword is present)

Hyphenated contextual keywords (no classic keywords are present)

A hyphenated keyword is a terminal symbol of the syntactic grammar of the Java language. This presents a challenge for the lower-level lexical grammar of the Java language, where input characters and line terminators are tokenized into keywords, identifiers, literals, operators, and separators. For example, if a hyphenated keyword starts with a classic keyword (e.g. short-circuit), then after consuming the Java letters that make up the classic keyword, the lexer must not tokenize them as a keyword and proceed to tokenize the - character and further Java letters as an operator and an identifier respectively. Instead, the lexer must look ahead and respect the hyphenation, that is, tokenize the entire sequence of Java letters, -, and more Java letters as one (hyphenated) keyword. White space cannot appear "inside" a hyphenated keyword, so if the lexer consumes any white space in its look ahead, then it can give up on detecting hyphenation and proceed with tokenization as usual.

A future version of this JEP may suggest hyphenation of more than two tokens (e.g. never-short-circuit, non-null-except-local), as well as notions of compounding other than hyphenation (e.g. keywords joined with + or : tokens).

Keyword management

The following policy is commended to Java language designers:

  1. Use a hyphenated classic keyword when you want to introduce a keyword in the middle of code, at a place where an identifier may occur.

  2. Use a hyphenated {classic, contextual} keyword when you want a keyword at a declaration site (class, field, method).

  3. Use a unitary {classic, contextual} keyword only in the most extreme cases where no hyphenated keyword is suitable.

The following subsections provide rationale for this policy.

Avoid classic keywords

While it may be legal for language designers to define i as a keyword in a future version of Java, it would likely break every program in the world, since i is used so commonly as an identifier. (When the assert keyword was added in 1.4, it broke every testing framework.) The cost of remediating the effect of such an incompatible change varies as well: invalidating a name choice for a local variable has a local fix, but invalidating the name of a public type or an interface method might well be fatal.

Additionally, the keywords that language designers are likely to want to reclaim are often those that are popular as identifiers (e.g., value, var, method), making such fatal collisions more likely. In some cases, if the keyword candidate in question is rarely used as an identifier, designers might opt to take the source-compatibility hit -- but candidates that are unlikely to collide (e.g., usually_but_not_always_final) are probably not the keywords anyone is hoping for.

Realistically, the space of identifiers is unlikely to be a well that language designers can draw from very often to find keywords, and the bar must be very high.

As a historical note, const and goto have been keywords since Java 1.0, even though they are not used by any language feature. They were defined as keywords not because a future version of Java was expected to use them, but because it supported a broader goal: migration from the then-preeminent C++ to the then-fledgling Java. Per the Java Language Specification in 1996, it allowed "a Java compiler to produce better error messages if these C++ keywords incorrectly appear in programs". (Namely, if const had been an identifier, then const int x = ... would have been flagged by a Java compiler as "Error, 'const' found where a keyword was expected", which is incongruent to a C++ developer who thinks const is a keyword; by making const a keyword in Java, Java compilers were forced to recognize it and flag "Error, 'const' keyword not allowed here", which is more comprehensible to a C++ developer.) Given the vast amount of code now written in Java, and the source incompatibility of a new classic keyword, there would be no justification for eagerly defining a classic keyword to support migration from another language. For example, it would be unacceptable to reclassify function from an identifier to a keyword in order to improve error messages for code copy-pasted from ECMAScript.

Cautiously consider contextual keywords

At first glance, unitary contextual keywords (and their friends, reserved type names) appear to be a magic wand: they let language designers create the illusion of new keywords without breaking existing programs. However, the positive track record of unitary contextual keywords hides a great deal of complexity and distortion.

The process of introducing a unitary contextual keyword is not a simple matter of choosing a word and adding it to the grammar; each one requires an analysis of potential current and future interactions. Each grammar position is its own story: contextual keywords that might be used as modifiers (e.g., readonly) have different ambiguity considerations than those that might be used in code (e.g., a match expression). While a small number of special situations can be managed in a specification or a compiler, the more heavily that unitary contextual keywords are used, the more likely there would be more significant maintenance costs and longer bug tails.

Beyond specifications and compilers, unitary contextual keywords distort the language for IDEs. IDEs often have to guess whether an identifier is meant to be an identifier or a unitary contextual keyword, and it may not have enough information to make a good guess until it has seen more input. While this is easy to dismiss as “not my problem”, in reality, it results in worse code highlighting, auto-completion, and refactoring abilities for everybody. (IDEs have the same trouble with hyphenated contextual keywords too.)

Finally, each identifier that is a candidate for dual-purposing as a unitary contextual keyword may have its own special considerations. For example, the use of var as a restricted identifier is justified only because the naming conventions for type names are so broadly adhered to. Using a hyphenated contextual keyword rather than a unitary contextual keyword can sidestep these considerations, since the hyphenated phrase has never been used as an identifier, though the ambiguity issue remains.

In summary, unitary contextual keywords are a tool in the language design toolbox, but they should be used with care.

Prefer hyphenated keywords

Hyphenated {classic, contextual} keywords create less trouble than unitary contextual keywords because the lexer can tell with fixed lookahead whether A-B should become three tokens (keyword/identifier, operator, keyword/identifier) or one (hyphenated keyword), whereas arbitrary lookahead may be required to tokenize an identifier as a unitary contextual keyword. There is less trouble for parsing as well; for example, non-null cannot be confused for a subtraction expression. In sum, this gives a lot more room for creating new, less-conflicting keywords. Happily, these new keywords are likely to be good names, as many of the missing concepts that might be added to Java can fundamentally be described by their relationship to pre-existing concepts (e.g., non-null).

There is a technical constraint on the space of hyphenated keywords, because some terms of the form A-B already have semantic meaning as expressions or statements:

These examples show type-correct expressions and statements, but there are also type-incorrect expressions and statements that would clash with hyphenated keywords. That is, some terms of the form A-B are not semantically meaningful, but they are syntactically valid, and overloading them as hyphenated keywords would make lexing and parsing very difficult. In particular, the terms are:

Formally, the hyphenated classic keyword A-B would be problematic if A is {assert, case, class, new, return, this, throw}, or if B is {boolean, byte, char, double, float, int, long, new, short, super, switch, this, void}.


A strategy to mitigate the cost of a new classic keyword would be to have a mechanism that allows the keyword to still be used as an identifier. This would have allowed developers in the Java 1.4 era to fix up their variables called assert so that their programs still compiled. However, any such mechanism would bring its own complexity and interactions with other features, and the idea of asking developers to revisit code in this way is undesirable. As a matter of interest, Kotlin allows a keyword to be used as an identifier by enclosing the keyword in backticks, but the goal is specifically to allow Kotlin code to use Java declarations whose names are identifiers in Java but keywords in Kotlin, such as is and when. General-purpose expansion of the Kotlin keyword space is accomplished with soft keywords, which map to unitary contextual keywords in this JEP.

Reusing the same classic keyword for different features has ample precedent in Java. For example, final is (ab)used to mean "not mutable" and "not overridable" and "not extensible". Using a pre-existing keyword in a new feature is sometimes natural and sensible, but usually it is not the first choice. Over time, as the range of demands placed on the keyword space expands, this may descend into the ridiculous; no one wants to use null final as a way of negating finality. (While one might think such things are too ridiculous to consider, there were serious-seeming suggestions during JEP 325 to use new switch to describe a switch with different semantics, presumably to be followed by new new switch in ten years.)

One way to live without making new keywords is to stop evolving Java entirely. While there are some who think this is a fine idea, doing so because of the lack of available tokens would be a silly reason. Java has a long life ahead, and Java developers are excited about new features that enable to them to write more expressive and reliable code.

Risks and Assumptions

Some Java developers will have a negative reaction to the idea of hyphenated keywords, while others may accept the idea but dislike the hyphenated suggestions that emerge over time for particular language features. However, this risk is likely to diminish over time, because many such reactions are possibly-transient responses to unfamiliarity.

Java has a long tradition of declarations having default properties (e.g., package accessibility for classes, mutability for fields, and concreteness for methods) and then using keywords to modify the properties of a given declaration (e.g., public, final, abstract). Hyphenated keywords could subvert this tradition by "merging" a modifier and the declaration into a single term, such as value-class D {..} rather than value class D {..}. Similarly, a hyphenated keyword could simulate a modifier on a modifier (public-read, non-final, semi-abstract) when it may be better to find a unitary term that describes the desired concept and introduce it as a contextual or even classic keyword.