This assignment is discussed in your textbook in Chapter 2. Essentially, your job is to construct a scanner using JLex. This involves construction of a token description file that contains regular expressions for each token you want to recognize, necessaryJLex directives and any auxiliary code you wish to include in your scanner. JLex creates a class Yylex which can be instantiated to a scanner object that scans a particular input with a specific error handler. In project 3 we will be using this scanner with a parser to parse a subset of the Tiger language. For now we will concentrate on finding the tokens of the Tiger language and printing them out as a check.

We are making one change to the Tiger language that affects the tokens you must find. We are restricting the definition of string constants (see page 526ff in textbook) to contain only the following special characters:

\n, \t, \" \\

A skeleton of the program is given to you in /usr/local/class/cs415/sp99/tiger/chap2/ and its subdirectories. Your job is to fill in the regular expressions and corresponding actions; leave the other sections of the Tiger.lex file alone; their function will be explained in class and recitation. The files and their function are explained below:

• Parse/sym.java This file contains definitions of the kinds of tokens, identifying each type with a constant. It contains statements like:

  static final int FUNCTION = 42;
  
  Parse/Lexer.java This file contains the definition
  of an interface which the Yylex class implements. It provides a
  nextToken() method which throws an exception if the input
  stream is empty when it looks for a token.
  
  interface Lexer {
      public java_cup.runtime.Symbol nextToken() throws java.io.IOException;
  }
  
  
  Parse/Main.java This file provides a 
   sample test program 
  to use to test your scanner. 
  
  Tiger.lex This file provides the skeleton of your input
  to JLex. You need to fill in more regular expression patterns
  and corresponding actions to complete this file. You should leave this
  edited file in the Parse subdirectory.
  
  ErrorMsg/ This directory contains a package
  ErrorMsg which consists of the class ErrorMsg that
  provides methods for keeping track of where the scanner is looking for
  a token in terms of character position in which line of input.
  
  makefile This file contains dependences between the
  software we are using to construct the scanner and  UNIX shell
  commands. When a dependent part is changed, the make command
  uses the 
  makefile to automatically redo those system commands
  necessary to update all files dependent on that part. (There is a 
  man page for make.)
  

To fully understand this implementation, you will need to familiarize yourself with class methods, interfaces and package usage in Java.

How to proceed?

For organizing your cs415 work, you should create a new subdirectory for every new assignment. All assignment directories and files should be read and write protected so that only you can read or write the directory and its files.

In addition, you will need to add to your CLASSPATH in order to access the packages needed to do this, and subsequent assignments. The easiest way to do this is to add the following line to your .cshrc or .tshrc file:

#a change to access packages in cs415
setenv CLASSPATH .:/ug/s1/class/cs415/sp99/JLex/classes/:/ug/s1/class/cs415/sp99/CUP/classes/

To begin, you should create a working directory proj2 for Assignment 2 and then copy the directory /usr/local/class/cs415/sp99/tiger/chap2 recursively (use -R to get subdirectories) to your working directory.

Augment Tiger.lex to contain the JLex commands you need to recognize tokens. Note: you can do these for a few tokens at a time just as long as you only include recognizable tokens in your input file. The Tiger.lex file stays in your Parse subdirectory.

Make a short file test.tig containing a short program in the Tiger language, or at first, a stream of Tiger tokens.

In your working directory, execute the make command which should echo to your terminal the steps its taking. You should see something like this:

84 remus!proj2> make
cd Parse; javaJLex.Main Tiger.lex; mv Tiger.lex.java Yylex.java
Processing first section -- user code.
Processing second section --JLex declarations.
Processing third section -- lexical rules.
Creating NFA machine representation.
NFA comprised of 28 states.
Creating DFA transition table.
Working on DFA states...........
Minimizing DFA transition table.
7 states after removal of redundant states.
Outputting lexical analyzer code.
javac -g Parse/*.java

Now to execute your parser on input file test.tig in your working directory using the main method in Parse.Main, type: java Parse.Main test.tig

We will be using the handin program for this and all subsequent programming assignments. This time we want you to turn in a hard copy of your Tiger.lex file including a listing of any test data used to test your program. We also require a 1-2 page (not handwritten) summary of what you have done. Make sure you describe any error handling you have implemented and any other special features of your scanner.

Last updated by Barbara Ryder (removed under construction sign) at 3:22pm on February 12, 1999.