LEX:Lex is a program generator designed for lexical processing of character input streams. It accepts a high-level, problem oriented specification for character string matching, and produces a program in a general purpose language which recognizes regular expressions. The regular expressions are specified by the user in the source specifications given to Lex. The Lex written code recognizes these expressions in an input stream and partitions the input stream into strings matching the expressions. At the boundaries between strings program sections provided by the user are executed. The Lex source file associates the regular expressions and the program fragments. As each expression appears in the input to the program written by Lex, the corresponding fragment is executed.

The general format of Lex source is:

                            {definitions}

                             %%

                            {rules}

                             %%

                            {user subroutines}

where the definitions and the user subroutines are often omitted. The second %% is optional, but the first is required to mark the beginning of the rules. The absolute minimum Lex program is thus

                                     %%

(no definitions, no rules) which translates into a program which copies the input to the output unchanged.

The general form of a Lex source file is:

                             {definitions}

                             %%

                             {rules}

                             %%

                             {user subroutines}

The definitions section contains a combination of

1) Definitions, in the form ``name space translation''.

2) Included code, in the form ``space code''.

3) Included code, in the form

                                       %

                                       code

                                       %}

4) Start conditions, given in the form

                                %S name1 name2 ...

5) Character set tables, in the form

                          %T

                          number space character-string

                          ...

                          %T

6) Changes to internal array sizes, in the form

                                     %x  nnn

where nnn is a decimal integer representing an array size and x selects the parameter as follows:

                        Letter          Parameter

                                positions

                          n      states

                          e      tree nodes

                          a      transitions

                          k      packed character classes

                          o      output array size

Lines in the rules section have the form ``expression action'' where the action may be continued on succeeding lines by using braces to delimit it.

Regular expressions in Lex use the following operators:

               x        the character "x"

               "x"      an "x", even if x is an operator

               \x       an "x", even if x is an operator.

               [xy]     the character x or y.

               [x-z]    the characters x, y or z.

               [^x]     any character but x.

               .        any character but newline.

               ^x       an x at the beginning of a line.

               <y>x     an x when Lex is in start condition y.

               x$       an x at the end of a line.

               x?       an optional x.

               x*       0,1,2, ... instances of x.

               x+       1,2,3, ... instances of x.

               x|y      an x or a y.

               (x)      an x.

               x/y      an x but only if followed by y.

               {xx}     the translation of xx from the

                        definitions section.

               x{m,n}   m through n occurrences of x

The programs generated by Lex handle character I/O only through the routines input, output, and unput. Thus the character representation provided in these routines is accepted by Lex and employed to return values in yytext.

YACC:  Yacc (for "yet another compiler compiler.") is the standard parser generator for the Unix operating system. An open source program, yacc generates code for the parser in the C programming language. The acronym is usually rendered in lowercase but is occasionally seen as YACC or Yacc. The original version of yacc was written by Stephen Johnson at American Telephone and Telegraph (AT&T). Versions of yacc have since been written for use with Ada, Java and several other less well-known programming languages.

yacc is designed for use with C code and generates a parser written in C. The parser is configured for use in conjunction with a lex-generated scanner and relies on standard shared features (token types, yylval, etc.) and calls the function yylex as a scanner coroutine. You provide a grammar specification file, which is traditionally named using a .y extension. You invoke yacc on the .y file and it creates the y.tab.h and y.tab.c files containing a thousand or so lines of intense C code that implements an efficient LALR(1) parser for your grammar, including the code for the actions you specified. The file provides an extern function yyparse.y that will attempt to successfully parse a valid sentence. You compile that C file normally, link with the rest of your code, and you have a parser! By default, the parser reads from stdin and writes to stdout, just like a lex-generated scanner does.

% yacc myFile.y                                                //creates y.tab.c of C code for parser

% gcc -c y.tab.c                                                //compiles parser code

% gcc –o parse y.tab.o lex.yy.o –ll -ly                //link parser, scanner, libraries

                                                 //invokes parser reads from stdin

atOptions = { 'key': 'a900f2dbf175e78754c26c6231a4b673', 'format': 'iframe', 'height': 90, 'width': 728, 'params': {} };