Java™ Compiler API

DRAFT VERSION
By Peter von der Ahé

The Java compiler framework (javax.tools) is an API (Application Program Interface) for running compilers (and other tools) as well as an SPI (Service Provider Interface) by which certain aspects of a compiler can be customized.

Note: Feedback that hasn't been incorporated yet will show in red text. Sometimes the feedback is in a tool-tip so you have to hold the mouse still over the text.

The Java compiler API serves three main goals:

The framework does not provide syntax trees, API for creating source files, or byte code instrumentation. However, javac does provide read-only access to syntax trees but these capabilities are outside the Java Compiler API as defined by the Java Community Process (JCP).

Invoking a Java Compiler

The tool provider is used to get a compiler instance. The simplest way to get such an instance is to use the getSystemJavaCompiler() method:

Tool javac = ToolProvider.getSystemJavaCompiler();

When running on Sun's JDK version 6 or later this method returns a compiler object. Most users will not have a compiler available because Sun's JRE is targeted for desktop users which do not normally need a compiler. In this case, the method will return null to indicate that no compiler is available. The JDK can be downloaded from java.sun.com.

Once a compiler instance has been obtained, it is possible to perform a compilation using the run() method:

javac.run(null, null, null, arguments);

The first three arguments to this method can be used to specify alternative streams to System.in, System.out, and System.err. The first two arguments are ignored by the compiler and the last one control where the output from -help and diagnostic messages will go.

The remaining arguments to the compiler are options and file names as used on the command line interface which is described in the manual pages [ Solaris™ Operating System | Microsoft Windows ].

The method takes advantage of the varargs feature that was added to the Java programming language in JDK version 5.0. It is possible to pass either one String array or an arbitrary number of strings following the first three arguments. If not an array, the remaining arguments will be converted to an array automatically and the following lines are equivalent:

javac.run(null, null, null, "Foo.java", "Bar.java");
javac.run(null, null, null, new String[] {"Foo.java", "Bar.java"});

The effect is the same as running the command line program, javac, like this:

javac Foo.java Bar.java

When running the command line program the operating system will process the command line arguments before passing them to the compiler. For this reason the compiler expect exactly one word per string and no extra spaces as illustrated by these examples:

// BAD: multiple files in one string
javac.run(null, null, null, "Foo.java Bar.java");

// BAD: two words in one string
javac.run(null, null, null, "-cp classes", "Foo.java", "Bar.java");

// GOOD
javac.run(null, null, null, "-cp", "classes", "Foo.java", "Bar.java");

This behavior is required to handle file names with spaces correctly.

Most of the options documented in the manual pages can be passed to the run method except for the -J option. This option is used to specify JVM options and must be specified when starting the JVM.

Finally, here is a complete program that behaves very much like javac:

import javax.tools.Tool;
import javax.tools.ToolProvider;

public class CompilerExample1 {
    public static void main(String... arguments) {
        Tool javac = ToolProvider.getSystemJavaCompiler();
        System.exit(javac.run(null, null, null, arguments));
    }
}

The JavaCompiler Interface

The Tool interface is designed to be the lowest common denominator for all kinds of tools and is suitable only for the simplest of tasks involving a compiler. The method getSystemJavaCompiler() returns an instance of JavaCompiler so to take advantage of its wider range of features, use this instead of the example above:

JavaCompiler javac = ToolProvider.getSystemJavaCompiler();

This interface enables more advanced usages through the CompilationTask interface and provides access to the compiler's standard file manager:

StandardJavaFileManager fileManager =
    javac.getStandardFileManager(null, null, null);

The CompilationTask interface provides more control over the compiler's operation. The interface uses a file abstraction instead of plain file names to allow customization of class name resolution and compilation of compilation units from other sources than regular files, as well as allowing output directly to memory or any storage mechanism.

The example above takes a list of compiler options and file names that are passed to the compiler. To accomplish the same with CompilationTask the options, file names, and class names must be separated:

Options options = Options.parse(javac, fileManager, arguments);

The Options class is provided in Options.java and may be added to the JDK in a future release. The list of file names specified on the command-line can be extracted from the options object. These file names can be converted to file objects that can be passed to the JavaCompiler.getTask method:

Iterable<? extends JavaFileObject> fileObjects =
    fileManager.getJavaFileObjectsFromFiles(options.getFiles());

CompilationTask task =  

Finally, a complete example based on CompilationTask (requires Options.java):

import javax.tools.JavaCompiler.CompilationTask;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

public class CompilerExample2 {
    public static void main(String... arguments) {
        JavaCompiler javac = ToolProvider.getSystemJavaCompiler();
        StandardJavaFileManager fileManager =
            javac.getStandardFileManager(null, null, null);

        Options options = Options.parse(javac, fileManager, arguments);
        System.out.println(options);
        if (!options.getUnrecognizedOptions().isEmpty())
            throw new IllegalArgumentException(
                options.getUnrecognizedOptions().toString());

        Iterable<? extends JavaFileObject> fileObjects =
            fileManager.getJavaFileObjectsFromFiles(options.getFiles());

        CompilationTask task = 

        System.exit(task.call() ? 0 : 1);
    }
}

Locating Other Tools

There may be other compilers or tools available in an installation. It is possible to find installed compilers (or other tools) using the service loader:

ServiceLoader.loadInstalled(JavaCompiler.class);

or

ServiceLoader.loadInstalled(Tool.class);

As an example, the following program will list all installed tools:

import java.util.ServiceLoader;
import javax.tools.Tool;

public class ListTools {
    public static void main(String... args) {
        for (Tool tool : ServiceLoader.loadInstalled(Tool.class)) {
            System.out.println("Found tool: " + tool);
        }
    }
}

This program produces no output on a standard installation of Sun's JDK because it has no additional tools available.

Accessing Diagnostic Messages

In the examples above, the compiler's diagnostic messages are output to the system console (System.err). The Diagnostic interface provides an alternative structural view of diagnostics. The class DiagnosticCollector can be used to obtain a list of diagnostic object from the CompilationTask:

DiagnosticCollector<JavaFileObject> collector =
    new DiagnosticCollector<JavaFileObject>();

The DiagnosticCollector is passed to the JavaCompiler.getTask method and after calling the CompilationTask.call method, the list of diagnostics can be extracted using DiagnosticCollector.getDiagnostics():

CompilationTask task =
    javac.getTask(null, fileManager, collector,
options.getRecgonizedOptions(),
options.getClassNames(), fileObjects);

boolean result = task.call(); List<Diagnostic<? extends JavaFileObject>> diagnostics = collector.getDiagnostics();

Diagnostic Properties

Each diagnostic has a number of (read-only) properties. Depending on the kind of diagnostic and compiler implementation, some of these properties may not be defined for a given diagnostic. Undefined properties are signaled with null or the constant Diagnostic.NOPOS.

Kind

The diagnostic kind is an enum which indicates whether a diagnostic represents an error, a warning, or something else. If the compilation fails, the application developer should take care to at least display the first error to the user. The kind is always defined (non-null).

Source

The diagnostic source is the file object associated with a diagnostic. For example, the source file which had an error. Some diagnostics may not have a source file associated in which case the source is null.

Below is a complete example which when given this file:

class Error {
    Number n = "a number?";
}

produces this output:

Error.java:2: error: incompatible types
found   : java.lang.String
required: java.lang.Number

The DiagnosticListener Interface

When running a compilation task as a batch job in a server environment it may be sufficient to get access to diagnostics after the task has completed. In interactive settings, however, it may be preferred that diagnostics can be accessed as soon as possible while the task is still running. For such situations, the framework defines the DiagnosticListener interface.

Managing Files

The framework defines interfaces for managing files.

Acknowledgments

Jonathan Gibbons provided a lot of feedback on the early drafts.