What's New 

Release Notes 

Download 

Documentation 

Examples 

Tutorial 

Why Visitors? 

Other Tools 

User Comments 

Introduction

If you are converting to a newer version of JTB, please see the release notes for the new version. 
 

Overview of Generated Files

• The file jtb.out.jj, the original grammar file, now with syntax tree building actions inserted.
• The subdirectory/package syntaxtree which contains a java class for each production in the grammar. 
• The subdirectory/package visitor which contains Visitor.java, the default visitor interface, as well as DepthFirstVisitor.java, a default implementation which visits each node of the tree in depth-first order.  In addition, there is ObjectVisitor.java, another default visitor inerface that supports return value and argument. ObjectDepthFirst.java is a defualt implemetation of ObjectVisitor.java.

Let's take a look at all the files and directories JTB generates. 
 

jtb.out.jj 

syntaxtree

Let's examine one of the classes generated from a production.  Take, for example, the following production (from the Java1.1.jj grammar): 

void ImportDeclaration() :
{}
{
  "import" Name() [ "." "*" ] ";"
}

//
// Generated by JTB 1.1.2
//

package syntaxtree;

/**
 * Grammar production:
 * f0 -> "import"
 * f1 -> Name()
 * f2 -> [ "." "*" ]
 * f3 -> ";"
 */
public class ImportDeclaration implements Node {
   public NodeToken f0;
   public Name f1;
   public NodeOptional f2;
   public NodeToken f3;

   public ImportDeclaration(NodeToken n0, Name n1,
   NodeOptional n2, NodeToken n3) {
      f0 = n0;
      f1 = n1;
      f2 = n2;
      f3 = n3;
   }

   public ImportDeclaration(Name n0, NodeOptional n1) {
      f0 = new NodeToken("import");
      f1 = n0;
      f2 = n1;
      f3 = new NodeToken(";");
   }

   public void accept(visitor.Visitor v) {
      v.visit(this);
   }
   public Object accept(visitor.ObjectVisitor v, Object argu) {
      return v.visit(this,argu);
   }
}

The first set of comments obviously shows which version of JTB created this file.  The second group of comments is for your benefit, showing the names of the fields of this class (the children of the node), and what parts of the original production they represent.   All parts of a production are represented in the tree, including tokens. 

Notice that this class is in the package "syntaxtree".  The purpose of separating the generated tree node classes into their own package is that it greatly simplifies file organization, particularly when the grammar contains a large number of productions.  If the grammar is stable and not subject to change, once these classes are generated and compiled, it's not necessary to pay them any more attention.  All of the work is to done to the visitor classes. 

Next you'll note that this class implements an interface named Node.  This is one of eight tree node classes and interfaces automatically generated for every grammar.  These classes are as follows: 
 

Next comes the member variables of the ImportDeclaration class.  These are generated based on the RHS of the production.  Their type depends on the various items in the RHS and their names begin with f0 and work their way up.  You may be wondering why these variables are declared as public.  Since the visitors which must access these fields reside in a different package than the syntax tree nodes, package visibility cannot be used.  We decided that breaking encapsulation was a necessary evil in this case. 

The next portion of the generated class is the standard constructor.  It is called from the tree-building actions in the annotated grammar so you will probably not need to use it. 

Following the first constructor is a convenience constructor with the constant tokens of the production already filled-in by the appropriate NodeToken.  This constructor's purpose is to help in manual construction of syntax trees. 

After the constructor are the accept() methods.  These methods are the way in which visitors interact with the class.  void accept(visitor.Visitor v) works with Visitor, while Object accept(visitor.ObjectVisitor v, Object argu) works with ObjectVisitor. 
 

visitor

To avoid confusion, let's go with Visitor interface and DepthFirstVisitor class in the following paragraphs, and I will explain the difference between Visitor and ObjectVisitor at the end of this documentation. 

The Visitor interface contains one method declaration per production in the grammar, plus one method declaration for each of the six automatically-generated classes.  DepthFirstVisitor is a convenience class which implements the Visitor interface.  Its default methods simply visit each node of the tree, calling the accept() method of each node's children. 

With regards to DepthFirstVisitor, our intent is for the programmer to only have to override those methods for which specific actions must be performed.  For example, in a visitor which simply counts the number of assignment statments in a Java source file, only the overloaded method visit(Assignment n) would need to be modified. 

Continuing our above example is the visit(ImportDeclaration n) method of class DepthFirstVisitor: 

/**
 * f0 -> "import"
 * f1 -> Name()
 * f2 -> [ "." "*" ]
 * f3 -> ";"
 */
public void visit(ImportDeclaration n) {
   n.f0.accept(this);
   n.f1.accept(this);
   n.f2.accept(this);
   n.f3.accept(this);
}

Note: All visitor classes must implement the Visitor or ObjectVisitor interface either directly or by subclassing a class which does so (such as DepthFirstVisitor or ObjectDepthFirst). 
 

The Automatically Generated Classes

Node

public interface Node extends java.io.Serializable {
   public void accept(visitor.Visitor v);
   public Object accept(visitor.ObjectVisitor v, Object argu);
}

Two features presented in JTB 1.2 may be helpful to you.  The first is that Node extends java.io.Serializable, meaning that you can now serialize your trees (or subtrees) to an output stream and read them back in.  If you are not familiar with object serialization, see the Java documentation on the java.io.Serializable interface.  Secondly, there is one accept() method that can take an extra argument and return a value. 
 

NodeListInterface

public interface NodeListInterface extends Node {
   public void addNode(Node n);
   public Node elementAt(int i);
   public java.util.Enumeration elements();
   public int size();
}

• addNode() is used by the tree-building code to add nodes to the list. 
• elements() is similar to the method of the same name in Vector, returning an Enumeration of the elements in the list.
• elementAt() returns the node at the ith position in the list (starting at 0, naturally).
• size() returns the number of elements in the list.

NodeChoice

( "abstract" | "final" | "public" )

void ResultType() : {}
{
  "void" | Type()
}

public class NodeChoice implements Node {
   public NodeChoice(Node node, int whichChoice);
   public void accept(visitor.Visitor v);
   public Object accept(visitor.ObjectVisitor v, Object argu);

   public Node choice;
   public int which;
}

NodeList

( "[" Expression() "]" )+ 

void ArrayDimensions() :
{}
{
  ( "[" Expression() "]" )+ ( "[" "]" )*
}

public class NodeList implements NodeListInterface {
   public NodeList();
   public void addNode(Node n);
   public Enumeration elements();
   public Node elementAt(int i);
   public int size();
   public void accept(visitor.Visitor v);
   public Object accept(visitor.ObjectVisitor v, Object argu);

   public Vector nodes;
}
 

NodeListOptional

( ImportDeclaration() )*

public class NodeListOptional implements NodeListInterface {
   public NodeListOptional();
   public void addNode(Node n);
   public Enumeration elements();
   public Node elementAt(int i);
   public int size();
   public boolean present();
   public void accept(visitor.Visitor v);
   public Object accept(visitor.ObjectVisitor v, Object argu);

   public Vector nodes;
}

NodeOptional

[ "." "*" ]

void ImportDeclaration() : {}
{
  "import" Name() [ "." "*" ] ";"
}

public class NodeOptional implements Node {
   public NodeOptional();
   public void addNode(Node n);
   public void accept(visitor.Visitor v);
   public Object accept(visitor.ObjectVisitor v, Object argu);
   public boolean present();

   public Node node;
}

NodeSequence

[ "extends" Name() ]

public class NodeSequence implements NodeListInterface {
   public NodeSequence();
   public void addNode(Node n);
   public Node elementAt(int i);
   public Enumeration elements();
   public int size();
   public void accept(visitor.Visitor v);
   public Object accept(visitor.ObjectVisitor v, Object argu);

   public Vector nodes;
}

void foo() : {}
{
   A() ( B() C() ) D()
}

Since we are undecided if this type of construct is necessary, and since these extra parentheses caused JTB to behave incorrectly prior to version 1.1, for now these parentheses will be flagged as a warning by the semantic checker.  We may remove this warning if we decide there is a use for this construct.  Note that you may disable the semantic checker by using the -e command-line option if desired. 
 

NodeToken

public class NodeToken implements Node {
   public NodeToken(String s);
   public NodeToken(String s, int kind, int beginLine,
      int beginColumn, int endLine, int endColumn);
   public String toString();
   public void accept(visitor.Visitor v);
   public Object accept(visitor.ObjectVisitor v, Object argu);

   public String tokenImage;

   // -1 for these ints means no position info is available.
   public int beginLine, beginColumn, endLine, endColumn;

   // Equal to the JavaCC token "kind" integer.
   // -1 if not available.
   public int kind;

   // Special Token methods below
   public NodeToken getSpecialAt(int i);
   public int numSpecials();
   public void addSpecial(NodeToken s);
   public void trimSpecials();
   public String withSpecials();

   public Vector specialTokens;
}

Also available is the kind integer.  JavaCC assigns each type of token a unique integer to identify it.  This integer is now available in each JTB NodeToken.  For more information on using the kind integer, see the JavaCC documentation. 

If the -tk command-line option is used, JTB will also store special tokens in the tree (see the JavaCC documentation for information on special tokens).  Since these tokens have no place in the syntax tree structure, JTB stores the special token in the next regular token which follows it.  If multiple special tokens appear before a regular token, that token's NodeToken object will store the special tokens in the specialTokens Vector. 

• The getSpecialAt(), numSpecials(), and addSpecial() methods are self-explanatory; they function like the similarly-named methods in the Vector class. 
• trimSpecials() calls the trimToSize() method of Vector in order to conserve memory.  It is called by the parser during tree construction.
• withSpecials() acts like the toString() method except that it returns a string with all of the special tokens present (in a somewhat messy, unformatted manner--we recommend you override the visit(NodeToken) method to print special tokens as desired).

JTB Command-Line Options

 
Toolkit Options

Scheme Tree Builder Toolkit

1. Generates records.scm to the current directory.  This file is analgous to the syntaxtree directory in that it defines the syntax tree.  The difference is that this file contains Scheme record definitions as opposed to Java classes.
2. Generates the visitor SchemeTreeBuilder.java into the visitor directory and package.  This visitor will traverse a syntax tree and output the Scheme equivalent of the tree.  The default output location is standard output, but this can be modified by passing the constructor a Java Writer object.

1. Choices such as ( x | y ) may only occur at the top level of a production.  The choice may only be between two nonterminal symbols.  For example, ( x y | z ) would be illegal.
2. Whatever goes within a list, optional list, or optional, such as ( x )+, ( y )*, or [ z ] may only be a single nonterminal symbol.  For example, [ x y ] would be illegal.

Printer Toolkit

• The visitor TreeDumper, when used on a syntax tree, will output (to standard output or another OutputStream or Writer of your choice) the tree based on the token location variables in each NodeToken.  If you use this visitor on an unmodified syntax tree, it will print the tree out exactly as it was read in.  It contains several methods:

• Using these methods, it is possible to only print certain parts of the tree.  For example, to only print method signatures in the Java grammar,  the following annonymous class could be used:

      root.accept(new DepthFirstVisitor() {
         public void visit(MethodDeclaration n) {
            dumper.startAtNextToken();
            n.f0.accept(dumper);
            n.f1.accept(dumper);
            n.f2.accept(dumper);
            n.f3.accept(dumper);
            // skip n.f4, the method body
            System.out.println();
         }
      }

• Pay attention to the location check in TreeDumper.java.  There is a location check in TreeDumper.java, which may cause problems:
• <EOL> will cause the current location in TreeDumper changed automatically.  Make sure you set the location correctly, when you have <EOL> in your grammar and use TreeFormatter.java or your own class to change the location variables in NodeToken.  Another workaround is to comment the location check in TreeDumper but this is not recommended.
• The TreeFormatter visitor is a skeleton pretty printer template.  It contains convenience methods to modify the token location information in each NodeToken:

• For example, this is how the CompilationUnit visit() method looks in the rewritten Java pretty printer example:

   /**
    * f0 -> [ PackageDeclaration() ]
    * f1 -> ( ImportDeclaration() )*
    * f2 -> ( TypeDeclaration() )*
    * f3 -> <EOF>
    */
   public void visit(CompilationUnit n) {
      if ( n.f0.present() ) {
         n.f0.accept(this);
         add(force(2));
      }

      if ( n.f1.present() ) {
         processList(n.f1, force());
         add(force(2));
      }
      if ( n.f2.present() ) {
         processList(n.f2, force(2));
         add(force());
      }
      n.f3.accept(this);
   }

ObjectVisitor

public Object visit(XXXX n, Object argu);

• The visit() method in ObjectVisitor takes one more argument as user's parameter.
• The visit() method in ObjectVisitor can return a value.

In a word, ObjectVisitor and ObjectDepthFirst are advanced versions of Visitor and DepthFirstVisitor. Note: any call to a class that implements ObjectVisitor must have two arguments. 
 

Common Problems

Errors

Warnings