198:415 Compilers
Spring 1999
Professor Barbara G. Ryder

Programming Assignment 2
Constructing a Scanner with JLEX

due: 6pm Monday, February 15, 1999

This assignment is discussed in your textbook in Chapter 2. Essentially, your job is to construct a scanner using JLex. This involves construction of a token description file that contains regular expressions for each token you want to recognize, necessaryJLex directives and any auxiliary code you wish to include in your scanner. JLex creates a class Yylex which can be instantiated to a scanner object that scans a particular input with a specific error handler. In project 3 we will be using this scanner with a parser to parse a subset of the Tiger language. For now we will concentrate on finding the tokens of the Tiger language and printing them out as a check.

We are making one change to the Tiger language that affects the tokens you must find. We are restricting the definition of string constants (see page 526ff in textbook) to contain only the following special characters:

\n, \t, \" \\
Note that in Tiger, there are no negative integer literals; therefore, -23 is returned as two separate tokens. Errors you should detect and report include unclosed comments and unclosed strings.

A skeleton of the program is given to you in /usr/local/class/cs415/sp99/tiger/chap2/ and its subdirectories. Your job is to fill in the regular expressions and corresponding actions; leave the other sections of the Tiger.lex file alone; their function will be explained in class and recitation. The files and their function are explained below:

To fully understand this implementation, you will need to familiarize yourself with class methods, interfaces and package usage in Java.

How to proceed?

For organizing your cs415 work, you should create a new subdirectory for every new assignment. All assignment directories and files should be read and write protected so that only you can read or write the directory and its files.

In addition, you will need to add to your CLASSPATH in order to access the packages needed to do this, and subsequent assignments. The easiest way to do this is to add the following line to your .cshrc or .tshrc file:

#a change to access packages in cs415
setenv CLASSPATH .:/ug/s1/class/cs415/sp99/JLex/classes/:/ug/s1/class/cs415/sp99/CUP/classes/

To begin, you should create a working directory proj2 for Assignment 2 and then copy the directory /usr/local/class/cs415/sp99/tiger/chap2 recursively (use -R to get subdirectories) to your working directory.

Augment Tiger.lex to contain the JLex commands you need to recognize tokens. Note: you can do these for a few tokens at a time just as long as you only include recognizable tokens in your input file. The Tiger.lex file stays in your Parse subdirectory.

Make a short file test.tig containing a short program in the Tiger language, or at first, a stream of Tiger tokens.

In your working directory, execute the make command which should echo to your terminal the steps its taking. You should see something like this:

84 remus!proj2> make
cd Parse; javaJLex.Main Tiger.lex; mv Tiger.lex.java Yylex.java
Processing first section -- user code.
Processing second section --JLex declarations.
Processing third section -- lexical rules.
Creating NFA machine representation.
NFA comprised of 28 states.
Creating DFA transition table.
Working on DFA states...........
Minimizing DFA transition table.
7 states after removal of redundant states.
Outputting lexical analyzer code.
javac -g Parse/*.java

Now to execute your parser on input file test.tig in your working directory using the main method in Parse.Main, type: java Parse.Main test.tig

We will be using the handin program for this and all subsequent programming assignments. This time we want you to turn in a hard copy of your Tiger.lex file including a listing of any test data used to test your program. We also require a 1-2 page (not handwritten) summary of what you have done. Make sure you describe any error handling you have implemented and any other special features of your scanner.

Last updated by Barbara Ryder (removed under construction sign) at 3:22pm on February 12, 1999.