The Hitchhiker's Guide to javac
DON'T PANIC
The following notes are provided to help newcomers to javac navigate their way around the code base.
See also:
- Compiler Package Overview for an overview of the javac package structure
- Compilation Overview for an overview of the compilation pipeline
Functional blocks
javac can be considered as a number of functional blocks: some go towards the abstraction of a Java compiler, that can transform Java source files into Java class files, while others provide the environmental infrastructure to support the compiler.
| area | block | description |
|---|---|---|
| environment provides compiler support Java Compiler API (JSR 199) |
invocation | how to invoke javac |
| options | option handling | |
| files | file manager | |
| reporting | log, diagnostics, etc | |
| utils | utility classes | |
| Java compiler transforms source files into class files JLS, JVMS, Language Model API, Annotation Processing API (JSR 269), Compiler Tree API |
syntax tree | AST, parser |
| semantics | semantic analysis of AST | |
| processing | annotation processing | |
| simplification | simplify the AST | |
| code generation | generating class files |
Environment
Invocation
| package | classes |
|---|---|
| javax.tools | ToolProvider, JavaCompiler |
| com.sun.source.util | JavacTask |
| com.sun.tools.javac | Main |
| com.sun.tools.javac.api | JavacTool |
| com.sun.tools.javac.main | Main, JavaCompiler |
These are the classes for running javac. You can invoke javac in three ways:
- Main
- Simple, direct command line functionality.
- Java Compiler API (JSR 199)
- Access via classes in the
javax.toolspackage. - Compiler Tree API
- Cast a
javax.tools.CompilationTaskto acom.sun.source.util.JavacTask.
All the invocation paths go through
com.sun.tools.javac.main.JavaCompiler, which is the
main compiler driver.
Option Handling
| package | classes |
|---|---|
| com.sun.tools.javac.main | OptionName, JavacOption, RecognizedOptions |
| com.sun.tools.javac.api | JavacTool |
| com.sun.source.util | Options |
Currently, there are different option decoding paths for Main
and for the Compiler/Tree API. (This is not good). This is being
rationalized, with OptionName,
JavacOption and RecognizedOptions being
combined into a single new enum Option.
FileManager
| package | classes |
|---|---|
| javax.tools | JavaFileManager, FileObject, JavaFileObject |
| com.sun.tools.javac.file | JavacFileManager, Paths |
| com.sun.tools.javac.nio | JavacPathFileManager |
JavaFileManager provides a very simple abstract
file system, sufficient for the needs of javac.
The implementation of JavaFileManager in the
javac.file package uses java.io.File and
java.util.jar API, and is currently the standard
(default) file manager.
The implementation of JavaFileManager in the
javac.nio package uses java.nio.file.Path
and is recommended for future use by anyone wanting to have javac
access files via an abstract file system.
Reporting
| package | classes |
|---|---|
| javax.tools | Diagnostic, DiagnosticListener |
| com.sun.tools.javac.util | AbstractLog, Log, DiagnosticFormatter and subtypes, JCDiagnostic |
The Log provides the primary mechanism for
reporting diagnostics, and these days provides a flexible reporting
mechanism, based on configurable, pluggable formatters.
To facilitate testing, there is a raw diagnostics mode which can print out the basic details of a diagnostic, in a form which is safe for use in golden files.
For more information, see javac diagnostics.
Utils
| package | classes |
|---|---|
| com.sun.tools.javac.util | Context, List, Name, NameTable, ... |
The root of all information for a compilation is a
Context. Originally, it was one per compilation;
currently, if annotation processing is involved, it is one context
per annotation processing round and one for the final compilation,
although an increasing amount of information is migrated between
contexts. It is a goal to get back to one context per
compilation.
javac uses its own implementation of List, which provides simple, fast, immutable lists.
Name and NameTable provide simple
interned strings.
Java Compiler
Syntax Tree
| package | classes |
|---|---|
| com.sun.source.tree | Tree and subtypes |
| com.sun.tools.javac.tree | JCTree and subtypes, TreeInfo |
| com.sun.tools.javac.parser | Scanner, JavacParser, etc |
The Scanner class maps source text into a stream of
tokens. The JavacParser class converts the stream of
tokens into one or more syntax trees.
The classes in the com.sun.source.tree package
provide public read access to the syntax trees.
The trees have a “minimalist” API design. Expressions and declarations point to semantic information derived from the tree. Use utility classes and visitors to process trees.
Semantics – The data model
| package | classes |
|---|---|
| javax.lang.model.element | Element and subtypes |
| javax.lang.model.type | TypeMirror and subtypes |
| javax.lang.model.util | Elements, Types, etc. |
| com.sun.tools.javac.code | Symbol, Type, Annotations, Attribute, TypeAttribute, Types |
A Symbol provides semantic information about a
declaration. Each subtype of Symbol implements a
corresponding subtype of Element.
A Type provides semantic information about an
expression. Each subtype of Type implements a
corresponding subtype of TypeMirror.
An Attribute provides semantic information about an
annotation. Note: They should not
be confused with class file attributes, which are completely
different.
The Types class provides utility methods for types
and symbols.
Semantics – Creating symbol tables
| package | classes |
|---|---|
| com.sun.tools.javac.comp | Enter, MemberEnter, Annotate |
Enter and MemberEnter scan trees for
class and member declarations, and initialize symbols.
Annotate scans trees for annotations on class and
member declarations, to add information to the symbols prior to
annotation processing.
Semantics – Code analysis
| package | classes |
|---|---|
| com.sun.tools.javac.comp | Attr, Check, Infer, Resolve, Flow |
Attr, Check, Infer and
Resolve analyze all names and expressions within the
program.
Flow performs static program flow analysis. It
checks the following:
- Reachability: can a statement be executed?
- Definite Assignment: has a variable been initialized?
- Definite Unassignment: has a variable not been initialized?
Annotation Processing
| package | classes |
|---|---|
| javax.annotation.processing | AbstractProcessor, ProcessingEnvironment, RoundEnvironment |
| com.sun.tools.javac.model | JavacElements, JavacTypes, etc |
| com.sun.tools.javac.processing | JavacProcessingEnvironment |
Annotation processing occurs after symbols have been entered
(Enter, MemberEnter) but before method
analysis (Attr, Flow).
Annotation processing gives users the opportunity to execute code that can analyze method signatures and generate additional types to be used in the compilation.
If new files are generated by the user code, the compilation is conceptually restarted.
Code Simplification
| package | classes |
|---|---|
| com.sun.tools.javac.comp | Lower, TransTypes |
| com.sun.tools.javac.tree | TreeTranslator |
Lower converts “syntactic sugar”
constructions into simpler code. This includes inner classes, class
literals, assertions, foreach loops, strings in switch, etc., which
can all be represented by code not using those constructs.
TransTypes eliminates (erases) generics from the
program.
Lower and TransTypes use
TreeTranslator to rewrite trees.
Code Generation
| package | classes |
|---|---|
| com.sun.tools.javac.jvm | all |
The Gen and Code classes generate
bytecodes for the class file
Code attribute.
The Pool class is used to model the class file
constant pool.
The CRTable and CRTable classes are
used for mapping source positions to bytecodes.
Class files are written out, via the file manager, using
ClassWriter.
Source code
- Master repository for JDK 8
- https://hg.openjdk.org/jdk8/jdk8/langtools
- Integration repository for JDK 8
- https://hg.openjdk.org/jdk8/tl/langtools
Testing
Developer tests are in the langtools repository, test directory. Tests are normally run with the jtreg test harness, although all tests can be run either stand-alone or using TestNG (depending on the nature of the test.)
All the tests should pass, all the time, on all platforms.
See Writing jtreg tests and Guidelines for tests in the “langtools” repository
TCK tests are licensed separately.
For more info …
- OpenJDK Compiler Group
- Related projects: Coin, Jigsaw, Lambda, Type Annotations