JTB GJ Home 

Download 

Documentation 

Examples 

Overview of Generated Files

• The file jtb.out.jj, the original grammar file, now with syntax tree building actions inserted.
• The subdirectory/package syntaxtree which contains a class for each production in the grammar. 
• The subdirectory/package visitor which contains four sets of default visitors:
• Visitor.java interface and DepthFirstVisitor.java class
• GJVisitor.java interface and GJDepthFirst.java class
• GJNoArguVisitor.java interface and GJNoArguDepthFirst.java class
• GJVoidVisitor.java interface and GJVoidDepthFirst.java class
• The subdirectory/package gj which contains gj packages: gj.util and gj.lang.

Let's take a look at all the files and directories JTB GJ generates. 
 

jtb.out.jj 

syntaxtree

Let's examine one of the classes generated from a production.  Take, for example, the following production (from the Java1.1.jj grammar): 

void ImportDeclaration() :
{}
{
  "import" Name() [ "." "*" ] ";"
}

//
// Generated by JTB GJ1.1
//

package syntaxtree;

/**
 * Grammar production:
 * f0 -> "import"
 * f1 -> Name()
 * f2 -> [ "." "*" ]
 * f3 -> ";"
 */
public class ImportDeclaration implements Node {
   public NodeToken f0;
   public Name f1;
   public NodeOptional f2;
   public NodeToken f3;

   public ImportDeclaration(NodeToken n0, Name n1,
   NodeOptional n2, NodeToken n3) {
      f0 = n0;
      f1 = n1;
      f2 = n2;
      f3 = n3;
   }

   public ImportDeclaration(Name n0, NodeOptional n1) {
      f0 = new NodeToken("import");
      f1 = n0;
      f2 = n1;
      f3 = new NodeToken(";");
   }

   public void accept(visitor.Visitor v) {
      v.visit(this);
   }
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu) {
      return v.visit(this,argu);
   }
   public <R> R accept(visitor.GJNoArguVisitor<R> v) {
      return v.visit(this);
   }
   public <A> void accept(visitor.GJVoidVisitor<A> v, A argu) {
      v.visit(this,argu);
   }
}

The first set of comments obviously shows which version of JTB created this file.  The second group of comments is for your benefit, showing the names of the fields of this class (the children of the node), and what parts of the original production they represent.   All parts of a production are represented in the tree, including tokens. 

Notice that this class is in the package "syntaxtree".  The purpose of separating the generated tree node classes into their own package is that it greatly simplifies file organization, particularly when the grammar contains a large number of productions.  If the grammar is stable and not subject to change, once these classes are generated and compiled, it's not necessary to pay them any more attention.  All of the work is to done to the visitor classes. 

Next you'll note that this class implements an interface named Node.  This is one of eight tree node classes and interfaces automatically generated for every grammar.  These classes are as follows: 
 

Next comes the member variables of the ImportDeclaration class.  These are generated based on the RHS of the production.  Their type depends on the various items in the RHS and their names begin with f0 and work their way up.  You may be wondering why these variables are declared as public.  Since the visitors which must access these fields reside in a different package than the syntax tree nodes, package visibility cannot be used.  We decided that breaking encapsulation was a necessary evil in this case. 

The next portion of the generated class is the standard constructor.  It is called from the tree-building actions in the annotated grammar so you will probably not need to use it. 

Following the first constructor is a convenience constructor with the constant tokens of the production already filled-in by the appropriate NodeToken.  This constructor's purpose is to help in manual construction of syntax trees. 

After the constructor are the accept() methods.  These methods are the way in which visitors interact with the node classes. void accept(visitor.Visitor v) works with Visitor, R accept(visitor.GJVisitor<R,A> v, A argu) works with GJVisitor, and so on. 
 

visitor

• Visitor interface and DepthFirstVisitor class
• GJVisitor interface and GJDepthFirst class
• GJNoArguVisitor interface and GJNoArguDepthFirst class
• GJVoidVisitor interface and GJVoidDepthFirst class

With regards to DepthFirstVisitor, our intent is for the programmer to only have to override those methods for which specific actions must be performed.  For example, in a visitor which simply counts the number of assignment statments in a Java source file, only the overloaded method visit(Assignment n) would need to be modified. 

Continuing our above example is the visit(ImportDeclaration n) method of class DepthFirstVisitor: 

/**
 * f0 -> "import"
 * f1 -> Name()
 * f2 -> [ "." "*" ]
 * f3 -> ";"
 */
public void visit(ImportDeclaration n) {
   n.f0.accept(this);
   n.f1.accept(this);
   n.f2.accept(this);
   n.f3.accept(this);
}

The other three sets of defualt visitors provide support for different visit() methods: 

• GJVisitor interface and GJDepthFirst class support visit() methods that have both return value and an additional argument.  For example,

 
public R visit(ImportDeclaration n, A argu)
  
• GJNoArguVisitor interface and GJNoArguDepthFirst class support visit() methods that have return value but no additional argument.  For example,

 
public R visit(ImportDeclaration n)

• GJVoidVisitor interface and GJVoidDepthFirst class support visit() methods that have an additional argument but no return value.  For example,

 
public void visit(ImportDeclaration n, A argu)

gj

The Automatically Generated Classes

Node

public interface Node extends java.io.Serializable {
   public void accept(visitor.Visitor v);
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu);
   public <R> R accept(visitor.GJNoArguVisitor<R> v);
   public <A> void accept(visitor.GJVoidVisitor<A> v, A argu);
}

The Node interface extends java.io.Serializable, meaning that you can now serialize your trees (or subtrees) to an output stream and read them back in.  If you are not familiar with object serialization, see the Java documentation on the java.io.Serializable interface. 
 

NodeListInterface

public interface NodeListInterface extends Node {
   public void addNode(Node n);
   public Node elementAt(int i);
   public gj.util.Enumeration<Node> elements();
   public int size();
}

• addNode() is used by the tree-building code to add nodes to the list. 
• elements() is similar to the method of the same name in Vector<A> in gj.util, returning an Enumeration<Node> of the elements in the list.
• elementAt() returns the node at the ith position in the list (starting at 0, naturally).
• size() returns the number of elements in the list.

NodeChoice

( "abstract" | "final" | "public" )

void ResultType() : {}
{
  "void" | Type()
}

public class NodeChoice implements Node {
   public NodeChoice(Node node, int whichChoice);
   public void accept(visitor.Visitor v);
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu);
   public <R> R accept(visitor.GJNoArguVisitor<R> v);
   public <A> void accept(visitor.GJVoidVisitor<A>, A argu);

   public Node choice;
   public int which;
}

NodeList

( "[" Expression() "]" )+ 

void ArrayDimensions() :
{}
{
  ( "[" Expression() "]" )+ ( "[" "]" )*
}

public class NodeList implements NodeListInterface {
   public NodeList();
   public void addNode(Node n);
   public Enumeration<Node> elements();
   public Node elementAt(int i);
   public int size();
   public void accept(visitor.Visitor v);
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu);
   public <R> R accept(visitor.GJNoArguVisitor<R> v);
   public <A> void accept(visitor.GJVoidVisitor<A>, A argu);

   public Vector<Node> nodes;
}

 
NodeListOptional

( ImportDeclaration() )*

public class NodeListOptional implements NodeListInterface {
   public NodeListOptional();
   public void addNode(Node n);
   public Enumeration<Node> elements();
   public Node elementAt(int i);
   public int size();
   public boolean present();
   public void accept(visitor.Visitor v);
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu);
   public <R> R accept(visitor.GJNoArguVisitor<R> v);
   public <A> void accept(visitor.GJVoidVisitor<A>, A argu);

   public Vector<Node> nodes;
}

NodeOptional

[ "." "*" ]

void ImportDeclaration() : {}
{
  "import" Name() [ "." "*" ] ";"
}

public class NodeOptional implements Node {
   public NodeOptional();
   public void addNode(Node n);
   public void accept(visitor.Visitor v);
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu);
   public <R> R accept(visitor.GJNoArguVisitor<R> v);
   public <A> void accept(visitor.GJVoidVisitor<A>, A argu);
   public boolean present();

   public Node node;
}

NodeSequence

[ "extends" Name() ]

public class NodeSequence implements NodeListInterface {
   public NodeSequence();
   public void addNode(Node n);
   public Node elementAt(int i);
   public Enumeration<Node> elements();
   public int size();
   public void accept(visitor.Visitor v);
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu);
   public <R> R accept(visitor.GJNoArguVisitor<R> v);
   public <A> void accept(visitor.GJVoidVisitor<A>, A argu);

   public Vector<Node> nodes;
}

void foo() : {}
{
   A() ( B() C() ) D()
}

Since we are undecided if this type of construct is necessary, for now these parentheses will be flagged as a warning by the semantic checker.  We may remove this warning if we decide there is a use for this construct.  Note that you may disable the semantic checker by using the -e command-line option if desired. 
 

NodeToken

public class NodeToken implements Node {
   public NodeToken(String s);
   public NodeToken(String s, int kind, int beginLine,
      int beginColumn, int endLine, int endColumn);
   public String toString();
   public void accept(visitor.Visitor v);
   public <R,A> R accept(visitor.GJVisitor<R,A> v, A argu);
   public <R> R accept(visitor.GJNoArguVisitor<R> v);
   public <A> void accept(visitor.GJVoidVisitor<A>, A argu);

   public String tokenImage;

   // -1 for these ints means no position info is available.
   public int beginLine, beginColumn, endLine, endColumn;

   // Equal to the JavaCC token "kind" integer.
   // -1 if not available.
   public int kind;

   // Special Token methods below
   public NodeToken getSpecialAt(int i);
   public int numSpecials();
   public void addSpecial(NodeToken s);
   public void trimSpecials();
   public String withSpecials();

   public Vector<NodeToken> specialTokens;
}

Also available is the kind integer.  JavaCC assigns each type of token a unique integer to identify it.  This integer is now available in each JTB NodeToken.  For more information on using the kind integer, see the JavaCC documentation. 

If the -tk command-line option is used, JTB will also store special tokens in the tree (see the JavaCC documentation for information on special tokens).  Since these tokens have no place in the syntax tree structure, JTB stores the special token in the next regular token which follows it.  If multiple special tokens appear before a regular token, that token's NodeToken object will store the special tokens in the specialTokens Vector. 

• The getSpecialAt(), numSpecials(), and addSpecial() methods are self-explanatory; they function like the similarly-named methods in the Vector class. 
• trimSpecials() calls the trimToSize() method of Vector in order to conserve memory.  It is called by the parser during tree construction.
• withSpecials() acts like the toString() method except that it returns a string with all of the special tokens present (in a somewhat messy, unformatted manner--we recommend you override the visit(NodeToken) method to print special tokens as desired).

JTB GJ Command-Line Options

Toolkit Options

Scheme Tree Builder Toolkit

1. Generates records.scm to the current directory.  This file is analgous to the syntaxtree directory in that it defines the syntax tree.  The difference is that this file contains Scheme record definitions as opposed to Java classes.
2. Generates the visitor SchemeTreeBuilder.java into the visitor directory and package.  This visitor will traverse a syntax tree and output the Scheme equivalent of the tree.  The default output location is standard output, but this can be modified by passing the constructor a Java Writer object.

1. Choices such as ( x | y ) may only occur at the top level of a production.  The choice may only be between two nonterminal symbols.  For example, ( x y | z ) would be illegal.
2. Whatever goes within a list, optional list, or optional, such as ( x )+, ( y )*, or [ z ] may only be a single nonterminal symbol.  For example, [ x y ] would be illegal.

Printer Toolkit

• The visitor TreeDumper, when used on a syntax tree, will output (to standard output or another OutputStream or Writer of your choice) the tree based on the token location variables in each NodeToken.  If you use this visitor on an unmodified syntax tree, it will print the tree out exactly as it was read in.  It contains several methods:

• Using these methods, it is possible to only print certain parts of the tree.  For example, to only print method signatures in the Java grammar,  the following annonymous class could be used:

      root.accept(new DepthFirstVisitor() {
         public void visit(MethodDeclaration n) {
            dumper.startAtNextToken();
            n.f0.accept(dumper);
            n.f1.accept(dumper);
            n.f2.accept(dumper);
            n.f3.accept(dumper);
            // skip n.f4, the method body
            System.out.println();
         }
      });

• Pay attention to the location check in TreeDumper.java.  There is a location check in TreeDumper.java, which may cause problems:
• <EOL> will cause the current location in TreeDumper changed automatically.  Make sure you set the location correctly, when you have <EOL> in your grammar and use TreeFormatter.java or your own class to change the location variables in NodeToken.  Another workaround is to comment the location check in TreeDumper but this is not recommended.
• The TreeFormatter visitor is a skeleton pretty printer template.  It contains convenience methods to modify the token location information in each NodeToken:

• For example, this is how the CompilationUnit visit() method looks in the rewritten Java pretty printer example:

   /**
    * f0 -> [ PackageDeclaration() ]
    * f1 -> ( ImportDeclaration() )*
    * f2 -> ( TypeDeclaration() )*
    * f3 -> <EOF>
    */
   public void visit(CompilationUnit n) {
      if ( n.f0.present() ) {
         n.f0.accept(this);
         add(force(2));
      }

      if ( n.f1.present() ) {
         processList(n.f1, force());
         add(force(2));
      }
      if ( n.f2.present() ) {
         processList(n.f2, force(2));
         add(force());
      }
      n.f3.accept(this);
   }

Common Problems

Errors

Warnings