Character Generators (char_, lit)

| Home | Libraries | People | FAQ | More

##### Character Generators (`char_`, `lit`)

###### Description

The character generators described in this section are:

The `char_` generator emits single characters. The `char_` generator has an associated Character Encoding Namespace. This is needed when doing basic operations such as forcing lower or upper case and dealing with character ranges.

There are various forms of `char_`.

###### char_

The no argument form of `char_` emits any character in the associated Character Encoding Namespace.

    char_               // emits any character as supplied by the attribute

###### char_(ch)

The single argument form of `char_` (with a character argument) emits the supplied character.

    char_('x')          // emits 'x'
    char_(L'x')         // emits L'x'
    char_(x)            // emits x (a char)

###### char_(first, last)

`char_` with two arguments, emits any character from a range of characters as supplied by the attribute.

    char_('a','z')      // alphabetic characters
    char_(L'0',L'9')    // digits

A range of characters is created from a low-high character pair. Such a generator emits a single character that is in the range, including both endpoints. Note, the first character must be before the second, according to the underlying Character Encoding Namespace.

Character mapping is inherently platform dependent. It is not guaranteed in the standard for example that `'A' < 'Z'`, that is why in Spirit2, we purposely attach a specific Character Encoding Namespace (such as ASCII, ISO-8859-1) to the `char_` generator to eliminate such ambiguities.

| Note

Sparse bit vectors

To accomodate 16/32 and 64 bit characters, the char-set statically switches from a `std::bitset` implementation when the character type is not greater than 8 bits, to a sparse bit/boolean set which uses a sorted vector of disjoint ranges (`range_run`). The set is constructed from ranges such that adjacent or overlapping ranges are coalesced.

`range_runs` are very space-economical in situations where there are lots of ranges and a few individual disjoint values. Searching is O(log n) where n is the number of ranges.

###### char_(def)

Lastly, when given a string (a plain C string, a `std::basic_string`, etc.), the string is regarded as a char-set definition string following a syntax that resembles posix style regular expression character sets (except that double quotes delimit the set elements instead of square brackets and there is no special negation ^ character). Examples:

    char_("a-zA-Z")     // alphabetic characters
    char_("0-9a-fA-F")  // hexadecimal characters
    char_("actgACTG")   // DNA identifiers
    char_("\x7f\x7e")   // Hexadecimal 0x7F and 0x7E

These generators emit any character from a range of characters as supplied by the attribute.

###### lit(ch)

`lit`, when passed a single character, behaves like the single argument `char_` except that `lit` does not consume an attribute. A plain `char` or `wchar_t` is equivalent to a `lit`.

| Note
`lit` is reused by the String Generators, the char generators, and the Numeric Generators (see signed integer, unsigned integer, and real number generators). In general, a char generator is created when you pass in a character, a string generator is created when you pass in a string, and a numeric generator is created when you use a numeric literal. The exception is when you pass a single element literal string, e.g. `lit("x")`. In this case, we optimize this to create a char generator instead of a string generator.

Examples:

    'x'
    lit('x')
    lit(L'x')
    lit(c)      // c is a char

###### Header

    // forwards to <boost/spirit/home/karma/char/char.hpp>
    #include <boost/spirit/include/karma_char_.hpp>

Also, see Include Structure.

###### Namespace

Name
`boost::spirit::lit // alias: boost::spirit::karma::lit`
`ns::char_`

In the table above, `ns` represents a Character Encoding Namespace.

###### Model of

> `PrimitiveGenerator`

Notation

`ch`, `ch1`, `ch2`

  Character-class specific character (See Character Class Types), or a Lazy Argument that evaluates to a character-class specific character value

`cs`

  Character-set specifier string (See Character Class Types), or a Lazy Argument that evaluates to a character-set specifier string, or a pointer/reference to a null-terminated array of characters. This string specifies a char-set definition string following a syntax that resembles posix style regular expression character sets (except the square brackets and the negation `^` character).

`ns`

  A Character Encoding Namespace.

`cg`

  A char generator, a char range generator, or a char set generator.

###### Expression Semantics

Semantics of an expression is defined only where it differs from, or is not defined in `PrimitiveGenerator`.

Expression | Description
--- | ---
`ch` | Generate the character literal `ch`. This generator never fails (unless the underlying output stream reports an error).
`lit(ch)` | Generate the character literal `ch`. This generator never fails (unless the underlying output stream reports an error).
`ns::char_` | Generate the character provided by a mandatory attribute interpreted in the character set defined by `ns`. This generator never fails (unless the underlying output stream reports an error).
`ns::char_(ch)` | Generate the character `ch` as provided by the immediate literal value the generator is initialized from. If this generator has an associated attribute it succeeds only as long as the attribute is equal to the immediate literal (unless the underlying output stream reports an error). Otherwise this generator fails and does not generate any output.
`ns::char_("c")` | Generate the character `c` as provided by the immediate literal value the generator is initialized from. If this generator has an associated attribute it succeeds only as long as the attribute is equal to the immediate literal (unless the underlying output stream reports an error). Otherwise this generator fails and does not generate any output.
`ns::char_(ch1, ch2)` | Generate the character provided by a mandatory attribute interpreted in the character set defined by `ns`. The generator succeeds as long as the attribute belongs to the character range `[ch1, ch2]` (unless the underlying output stream reports an error). Otherwise this generator fails and does not generate any output.
`ns::char_(cs)` | Generate the character provided by a mandatory attribute interpreted in the character set defined by `ns`. The generator succeeds as long as the attribute belongs to the character set `cs` (unless the underlying output stream reports an error). Otherwise this generator fails and does not generate any output.
`~cg` | Negate `cg`. The result is a negated char generator that inverts the test condition of the character generator it is attached to.

A character `ch` is assumed to belong to the character range defined by `ns::char_(ch1, ch2)` if its character value (binary representation) interpreted in the character set defined by `ns` is not smaller than the character value of `ch1` and not larger then the character value of `ch2` (i.e. `ch1 <= ch <= ch2`).

The `charset` parameter passed to `ns::char_(charset)` must be a string containing more than one character. Every single character in this string is assumed to belong to the character set defined by this expression. An exception to this is the `'-'` character which has a special meaning if it is not specified as the first and not the last character in `charset`. If the `'-'` is used in between to characters it is interpreted as spanning a character range. A character `ch` is considered to belong to the defined character set `charset` if it matches one of the characters as specified by the string parameter described above. For example

Example | Description
--- | ---
`char_("abc")` | 'a', 'b', and 'c'
`char_("a-z")` | all characters (and including) from 'a' to 'z'
`char_("a-zA-Z")` | all characters (and including) from 'a' to 'z' and 'A' and 'Z'
`char_("-1-9")` | '-' and all characters (and including) from '1' to '9'

###### Attributes

Expression | Attribute
--- | ---
`ch` | `unused`
`lit(ch)` | `unused`
`ns::char_` | `Ch`, attribute is mandatory (otherwise compilation will fail). `Ch` is the character type of the Character Encoding Namespace, `ns`.
`ns::char_(ch)` | `Ch`, attribute is optional, if it is supplied, the generator compares the attribute with `ch` and succeeds only if both are equal, failing otherwise. `Ch` is the character type of the Character Encoding Namespace, `ns`.
`ns::char_("c")` | `Ch`, attribute is optional, if it is supplied, the generator compares the attribute with `c` and succeeds only if both are equal, failing otherwise. `Ch` is the character type of the Character Encoding Namespace, `ns`.
`ns::char_(ch1, ch2)` | `Ch`, attribute is mandatory (otherwise compilation will fail), the generator succeeds if the attribute belongs to the character range `[ch1, ch2]` interpreted in the character set defined by `ns`. `Ch` is the character type of the Character Encoding Namespace, `ns`.
`ns::char_(cs)` | `Ch`, attribute is mandatory (otherwise compilation will fail), the generator succeeds if the attribute belongs to the character set `cs`, interpreted in the character set defined by `ns`. `Ch` is the character type of the Character Encoding Namespace, `ns`.
`~cg` | Attribute of `cg`

| Note
In addition to their usual attribute of type `Ch` all listed generators accept an instance of a `boost::optional<Ch>` as well. If the `boost::optional<>` is initialized (holds a value) the generators behave as if their attribute was an instance of `Ch` and emit the value stored in the `boost::optional<>`. Otherwise the generators will fail.

###### Complexity

> O(1)

The complexity of `ch`, `lit(ch)`, `ns::char_`, `ns::char_(ch)`, and `ns::char_("c")` is constant as all generators emit exactly one character per invocation.

The character range generator (`ns::char_(ch1, ch2)`) additionally requires constant lookup time for the verification whether the attribute belongs to the character range.

The character set generator (`ns::char_(cs)`) additionally requires O(log N) lookup time for the verification whether the attribute belongs to the character set, where N is the number of characters in the character set.

###### Example

| Note
The test harness for the example(s) below is presented in the Basics Examples section.

Some includes:

    #include <boost/spirit/include/karma.hpp>
    #include <boost/spirit/include/phoenix_core.hpp>
    #include <boost/spirit/include/phoenix_operator.hpp>
    #include <boost/fusion/include/std_pair.hpp>
    #include <iostream>
    #include <string>

Some using declarations:

    using boost::spirit::karma::lit;
    using boost::spirit::ascii::char_;

Basic usage of `char_` generators:

    test_generator("A", 'A');
    test_generator("A", lit('A'));

    test_generator_attr("a", char_, 'a');
    test_generator("A", char_('A'));
    test_generator_attr("A", char_('A'), 'A');
    test_generator_attr("", char_('A'), 'B');         // fails (as 'A' != 'B')

    test_generator_attr("A", char_('A', 'Z'), 'A');
    test_generator_attr("", char_('A', 'Z'), 'a');    // fails (as 'a' does not belong to 'A'...'Z')

    test_generator_attr("k", char_("a-z0-9"), 'k');
    test_generator_attr("", char_("a-z0-9"), 'A');    // fails (as 'A' does not belong to "a-z0-9")