stklos-genlex - lexical analyzer generation for STklos


stklos-genlex lex-file output-file constructor

stklos-genlex source-file output-file


The stklos-genlex can be used to make lexical analyzers with STklos. It is a simple front-end to the Danny Dubé SILex tool.

When this command is called with three parameters, it takes a standard SILex input file as input and generate an analyzer. These parameters are described below:

is the input file which contains the lex-like rules describing the analyzer to be built
is a file which will contain the Scheme analyzer produced by the SILex tool.
is the name of the procedure that will be produced in the output-file . This procedure takes one parameter which is the file name (or the port) which is used as the input stream for the lexical analyzer. The value returned by this procedure is a STklos object which contains the automaton. To fetch the next token of the input stream, programs must use the lexer-next-token. procedure. See the example below for details.

Another usage of the stklos-genlex can be to use the define-regular-grammar special form directly in a source file. This form takes the name of the analyzer, the arguments which are passed to the analyzer, the macros part and rules part of the analyzer. In this case, stklos-genlex is called with the source file and the name of the output file. The output file, will contain a copy of the source file where the define-regular-grammar have been replaced by the code of the analyzer.


Suppose that the lexical rules are in the file rules.l. One can built an analyzer in the file out.stkfor this rules using the following command
     stklos-genlex  rules.l out.stk my-make-lex
The following piece of code show how to use the produced analyzer.
      (load "out.stk")
      (let ((lex (my-make-lex "foo")))
	;; Display the first token of file foo
	(display (lexer-next-token lex))

This example uses the second form of stklos-genlex usage. Here is an analyzer which skips lines starting with 2 slashes. Suppose that we have a file lex.stk which is:

(define-regular-grammar skip-//-lines
  ;; The arguments
  ;; The macros
space [ \9]
eol   \n
  ;; The grammar
{space}*//.*{eol}       (yycontinue)
^.*{eol}                yytext
<<EOF>>                 (eof-object)
<<ERROR>>               (error "invalid token")

(define (copy in out)
  (let ((reader (make-regular-reader skip-//-lines in)))
    (let loop ((line (reader)))
      (unless (eof-object? line)
	(display line out)
	(loop (reader))))))
The following command will produce an analyzer in file ana.stk
     stklos-genlex  lex.stk ana.stk


stklos(1) lex(1) and the SILex documentation

Markup created by unroff 1.0sc,    March 04, 2023.