[/==============================================================================
    Copyright (C) 2001-2008 Joel de Guzman
    Copyright (C) 2001-2008 Hartmut Kaiser

    Distributed under the Boost Software License, Version 1.0. (See accompanying
    file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
===============================================================================/]

[section:lexer_static_model The /Static/ Lexer Model]

The documentation of __lex__ so far mostly was about describing the features of
the /dynamic/ model, where the tables needed for lexical analysis are generated
from the regular expressions at runtime. The big advantage of the dynamic model
is its flexibility, and its integration with the __spirit__ library and the C++
host language. Its big disadvantage is the need to spend additional runtime to 
generate the tables, which especially might be a limitation for larger lexical 
analyers. The /static/ model strives to build upon the smooth integration with
__spirit__ and C++, and reuses large parts of the __lex__ library as described
so far, while overcoming the additional runtime requirements by using 
pre-generated tables and tokenizer routines. To make the code generation as 
simple as possible, it is possible reuse the token definition types developed 
using the /dynamic/ model without any changes. As will be shown in this 
section, building a code generator based on an existing token definition type 
is a matter of writing 3 lines of code.

Assuming you already built a dynamic lexer for your problem, there are two more
steps needed to create a static lexical analyzer using __lex__:

# generating the C++ code for the static analyzer (including the tokenization 
  function and corresponding tables), and 
# modifying the dynamic lexical anlyzer to use the generated code.

Both steps are described in more detail in the two sections below (for the full
source code used in this example see the code here:
[@../../example/lex/static_lexer/word_count_tokens.hpp the common token definition],
[@../../example/lex/static_lexer/word_count_generate.cpp the code generator], 
[@../../example/lex/static_lexer/word_count_static.hpp the generated code], and 
[@../../example/lex/static_lexer/word_count_static.cpp the static lexical analyzer]).

[import ../example/lex/static_lexer/word_count_tokens.hpp]
[import ../example/lex/static_lexer/word_count_static.cpp]
[import ../example/lex/static_lexer/word_count_generate.cpp]

But first we provide the code snippets needed to understand the further 
descriptions. Both, the definition of the used token identifier and the of the 
token definition class in this example are put into a separate header file to 
make these available to the code generator and the static lexical analyzer. 

[wc_static_tokenids]

The important point here is, that the token definition class is not different 
from a similar class to be used for a dynamic lexical analyzer. The library 
has been designed in a way, that all components (dynamic lexical analyzer, code 
generator, and static lexical analyzer) can reuse the very same token definition 
syntax.

[wc_static_tokendef]

The only thing changing between the three different use cases is the template
parameter used to instantiate a concrete token definition. Fot the dynamic 
model and the code generator you probably will use the __class_lexertl_lexer__
template, where for the static model you will use the 
__class_lexertl_static_lexer__ type as the template parameter.

This example not only shows how to build a static lexer, but it additionally 
demonstrates, how such a lexer can be used for parsing in conjunction with a 
__qi__ grammar. For completeness we provide the simple grammar used in this 
example. As you can see, this grammar does not have any dependencies on the 
static lexical analyzer, and for this reason it is not different from a grammar
used either without a lexer or using a dynamic lexical analyzer as described 
before.

[wc_static_grammar]


[heading Generating the Static Analyzer]

The first additional step to perform in order to create a static lexical 
analyzer is to create a small standalone program for creating the lexer tables
and the corresponding tokenization function. For this purpose the __lex__ 
library exposes a special API - the function __api_generate_static__. It 
implements the whole code generator, no further code is needed. All what it 
takes to invoke this function is to supply a token definition instance, an 
output stream to use to generate the code to, and an optional string to be used 
as a prefix for the name of the generated function. All in all just a couple 
lines of code.

[wc_static_generate_main]

The shown code generator will generate output, which should be stored in a file
for later inclusion into the static lexical analzyer as shown in the next 
topic (the full generated code can be viewed 
[@../../example/lex/static_lexer/word_count_static.hpp here]).


[heading Modifying the Dynamic Analyzer]

The second required step to convert an existing dynamic lexer into a static one 
is to change your main program at two places. First, you need to change the 
type of the used lexer (that is the template parameter used while instantiating
your token definition class). While in the dynamic model we have been using the
__class_lexertl_lexer__ template, we now need to change that to the 
__class_lexertl_static_lexer__ type. The second change is tightly related to 
the first one and involves correcting the corresponding `#include` statement to:

[wc_static_include]

Otherwise the main program is not different from an equivalent program using 
the dynamic model. This feature makes it really easy for instance to develop 
the lexer in dynamic mode and to switch to the static mode after the code has
been stabilized. The simple generator application showed above enables the 
integration of the code generator into any existing build process. The 
following code snippet provides the overall main function, highlighting 
the code to be changed.

[wc_static_main]


[endsect]
