JEP 330: Launch Single-File Source-Code Programs

Summary

Enhance the java launcher to run a program supplied as a single file of Java source code, including usage from within a script by means of "shebang" files and related techniques.

Non-Goals

It is not a goal to change either the Java Language Specification (JLS) or javac to accommodate shebang files. Likewise, it is not a goal to evolve the Java language into a general purpose scripting language.

It is not a goal of this JEP to change the Java Language Specification to accommodate simpler ways of writing small programs, such as eliminating the need for the standard public static void main(String[] args) method. However, it is expected that any such changes to the Java language will be usable in conjunction with this feature.

Motivation

Single-file programs -- where the whole program fits in a single source file -- are common in the early stages of learning Java, and when writing small utility programs. In this context, it is pure ceremony to have to compile the program before running it. In addition, a single-file program may declare multiple classes and thus compile to multiple class files, which adds packaging overhead to the simple goal of "run this program". It is desirable to be able to run the program directly from source with the java launcher:

java HelloWorld.java

Description

As of JDK 10, the java launcher operates in three modes: launching a class file, launching the main class of a JAR file, or launching the main class of a module. Here we add a new, fourth mode: launching a class declared in a source file.

Source-file mode is determined by considering two items on the command line:

1. The first item on the command line that is neither an option nor part of an option. (In other words, the item that previously has been the class name.)
2. The --source version option, if present.

If the "class name" identifies an existing file with the .java extension, source-file mode is selected, with that file to be compiled and run. The --source option may be used to specify the source version of the source code.

If the file does not have the .java extension, the --source option must be used to force source-file mode. This is for cases such as when the source file is a "script" to be executed and the name of the source file does not follow the normal naming conventions for Java source files. (See "shebang" files below.)

The --source option must also be used to specify the source version of the source code when the --enable-preview option is used. (See JEP 12.)

In source-file mode, the effect is as if the source file is compiled into memory, and the first class found in the source file is executed. For example, if a file called HelloWorld.java contains a class called hello.World, then the command

java HelloWorld.java

is informally equivalent to

javac -d <memory> HelloWorld.java
java -cp <memory> hello.World

Any arguments placed after the name of the source file in the original command line are passed to the compiled class when it is executed. For example, if a file called Factorial.java contains a class called Factorial to calculate the factorials of its arguments, then the command

java Factorial.java 3 4 5

is informally equivalent to

javac -d <memory> Factorial.java
java -cp <memory> Factorial 3 4 5

In source-file mode, any additional command-line options are processed as follows:

• The launcher scans the options specified before the source file for any that are relevant in order to compile the source file. This includes: --class-path, --module-path, --add-exports, --add-modules, --limit-modules, --patch-module, --upgrade-module-path, and any variant forms of those options. It also includes the new --enable-preview option, described in JEP 12.

• No provision is made to pass any additional options to the compiler, such as -processor or -Werror.

• Command-line argument files (@-files) may be used in the standard way. Long lists of arguments for either the VM or the program being invoked may be placed in files which are specified on the command-line by prefixing the filename with an @ character.

In source-file mode, compilation proceeds as follows:

• Any command-line options that are relevant to the compilation environment are taken into account.

• No other source files are found and compiled, as if the source path is set to an empty value.

• Annotation processing is disabled, as if -proc:none is in effect.

• If a version is specified, via the --source option, the value is used as the argument for an implicit --release option for the compilation. This sets both the source version accepted by compiler and the system API that may be used by the code in the source file.

• The source file is compiled in the context of an unnamed module.

• The source file should contain one or more top-level classes, the first of which is taken as the class to be executed.

• The compiler does not enforce the optional restriction defined at the end of JLS §7.6, that a type in a named package should exist in a file whose name is composed from the type name followed by the .java extension.

• If the source file contains errors, appropriate error messages are written to the standard error stream, and the launcher exits with a non-zero exit code.

In source-file mode, execution proceeds as follows:

• The class to be executed is the first top-level class found in the source file. It must contain a declaration of the standard public static void main(String[]) method.

• The compiled classes are loaded by a custom class loader, that delegates to the application class loader. (This implies that classes appearing on the application class path cannot refer to any classes declared in the source file.)

• The compiled classes are executed in the context of an unnamed module, and as if --add-modules=ALL-DEFAULT is in effect (in addition to any other --add-module options that may be have been specified on the command line.)

• Any arguments appearing after the name of the file on the command line are passed to the standard main method in the obvious way.

• It is an error if there is a class on the application class path whose name is the same as that of the class to be executed.

Note that there is a potential minor ambiguity when using a simple command-line like java HelloWorld.java. Previously, HelloWorld.java would have been interpreted as a class called java in a package called HelloWorld, but which is now resolved in favor of a file called HelloWorld.java if such a file exists. Given that such a class name and such a package name both violate the nearly-universally-followed naming conventions, and given the unlikeliness of such a class being on the class path and a like-named file being in the current directory, this seems an acceptable compromise.

Implementation

Source-file mode requires the presence of the jdk.compiler module. When source-file mode for a file Foo.java is requested, the launcher behaves as if the command line were translated to:

java [VM args] \
    -m jdk.compiler/<source-launcher-implementation-class> \
    Foo.java [program args]

The source-launcher implementation class programmatically invokes the compiler, which compiles the source to an in-memory representation. The source-launcher implementation class then creates a class loader to load compiled classes from that in-memory representation, and invokes the standard main(String[]) method of the first top-level class found in the source file.

The source-launcher implementation class has access to any relevant command-line options, such as those to define the class path, module path, and the module graph, and passes those options to the compiler to configure the compilation environment.

If the class that is invoked throws an exception, that exception is passed back to the launcher for handling in the normal way. However, the initial stackframes leading up to the execution of the class are removed from the stacktrace of the exception. The intent is that the handling of the exception is similar to the handling if the class had been executed directly by the launcher itself. The initial stackframes will be visible in any direct access to the stack, including (for example) Thread.dumpStack().

The class loader that is used to load the compiled classes itself uses an implementation-specific protocol for any URLs that refer to resources defined by the class loader. The only way to get such URLs is by using methods like getResource or getResources; creating any such URL from a string is not supported.

Single-file programs are also common when the task at hand needs a small utility program. In this context, it is desirable to be able to run a program directly from source using the "#!" mechanism on Unix-derived systems, such as macOS and Linux. This is a mechanism provided by the operating system which allows a single-file program (such as a script or source code) to be placed in any conveniently named executable file whose first line begins with #! and which specifies the name of a program to "execute" the contents of the file. Such files are called "shebang files".

It is desirable to be able to execute Java programs with this mechanism.

A shebang file to invoke the Java launcher using source-file mode must begin with something like:

#!/path/to/java --source version

For example, we could take the source code for a "Hello World" program, and put it in a file called hello, after an initial line of #!/path/to/java --source 10, and then mark the file as executable. Then, if the file is in the current directory, we could execute it with:

$ ./hello

Or, if the file is in a directory in the user's PATH, we could execute it with:

$ hello

Any arguments for the command are passed to the main method of the class that is executed. For example, if we put the source code for a program to compute factorials into a shebang file called factorial, we could execute it with a command like:

$ factorial 6

The --source option must be used in shebang files in the following situations:

• The name of the shebang file does not follow the standard naming conventions for Java source files.
• It is desired to specify additional VM options on the first line of the shebang file. In this case, the --source option should be specified first, after the name of the executable.
• It is desired to specify the version of the Java language used for the source code in the file.

A shebang file can also be invoked explicitly by the launcher, perhaps with additional options, with a command like:

$ java -Dtrace=true --source 10 factorial 3

The Java launcher's source-file mode makes two accommodations for shebang files:

1. When the launcher reads the source file, if the file is not a Java source file (i.e. it is not a file whose name ends with .java) and if the first line begins with #!, then the contents of that line up to but not including the first newline are ignored when determining the source code to be passed to the compiler. The content of the file that appears after the first line must consist of a valid CompilationUnit as defined by §7.3 in the edition of the Java Language Specification that is appropriate to the version of the platform given in the --source option, if present, or the version of the platform being used to run the program if the --source option is not present.

   The newline at the end of the first line is preserved so that the line numbers in any compiler error messages are meaningful in the shebang file.

2. Some operating systems pass the text on the first line after the name of the executable as a single argument to the executable. With that in mind, if the launcher encounters an option beginning --source and containing whitespace, it is split into a series of words, separated by whitespace, before being further analyzed by the launcher. This allows additional arguments to be put on the first line, although some operating system may impose a limit on the overall length of the line. Using quotes to preserve whitespace in such values is not supported.

No changes to the JLS are required in support of this feature.

In a shebang file, the first two bytes must be 0x23 0x21, the two-character ASCII encoding of #!. All subsequent bytes are read with the default platform character encoding that is in effect.

A first line beginning #! is only required when it is desired to execute the file with the operating system's shebang mechanism. There is no need for any special first line when the Java launcher is used explicitly to run the code in a source file, as in the HelloWorld.java and Factorial.java examples, given above. Indeed, the use of the shebang mechanism to execute files that follow the standard naming convention for Java source files is not permitted.

Alternatives

The status quo has worked for 20+ years; we could continue with that.

Instead of using #!, it would be possible to configure systems that support shebang files to use a different prefix, such as //!. Such a prefix would be seen by javac as a single-line comment and would not need any special treatment to ignore it. However, introducing a new magic number on operating systems like macOS and Linux requires either a manual or automated update to such systems, and is beyond the scope of this JEP.

Instead of using the shebang mechanism, it would be possible to write a shell script that contains Java source code as a here document that can be passed to the Java source launcher. While this is ultimately a more flexible mechanism than the shebang mechanism, it is also more overhead than the use of shebang in simple cases.

We could create a source launcher, but call it something else besides java, such as jrun. Given the number of execution modes the launcher already has, this would likely be perceived as a gratuitous difference.

We could delegate the task of "one-off runs" to the jshell tool. While this may at first seem obvious, this was an explicit non-goal in the design of jshell. The jshell tool was designed to be an interactive shell, and many design decisions were made in favor of providing a better interactive experience. Burdening it with the additional constraints of being the batch runner would detract from the interactive experience.

We could also use the jrunscript tool. However, this tool provides limited facilities for interacting with the runtime environment, and does not address the desire to provide a simple introduction to using Java.