The Full Extension Mechanism

| Home | Libraries | People | FAQ | More

### The Full Extension Mechanism

The Fusion library is designed to be extensible, new sequences types can easily be added. In fact, the library support for `std::pair`, `boost::array` and MPL sequences is entirely provided using the extension mechanism.

The process for adding a new sequence type to Fusion is:

1. Enable the tag dispatching mechanism used by Fusion for your sequence type
2. Design an iterator type for the sequence
3. Provide specialized behaviour for the intrinsic operations of the new Fusion sequence

##### Our example

In order to illustrate enabling a new sequence type for use with Fusion, we are going to use the type:

    namespace example
    {
        struct example_struct
        {
            std::string name;
            int age;
            example_struct(
                const std::string& n,
                int a)
                : name(n), age(a)
            {}
        };
    }

We are going to pretend that this type has been provided by a 3rd party library, and therefore cannot be modified. We shall work through all the necessary steps to enable `example_struct` to serve as an Associative Sequence as described in the Quick Start guide.

##### Enabling Tag Dispatching

The Fusion extensibility mechanism uses tag dispatching to call the correct code for a given sequence type. In order to exploit the tag dispatching mechanism we must first declare a new tag type for the mechanism to use. For example:

    namespace example {
        struct example_sequence_tag; // Only definition needed
    }

Next we need to enable the `traits::tag_of` metafunction to return our newly chosen tag type for operations involving our sequence. This is done by specializing `traits::tag_of` for our sequence type.

    #include <boost/fusion/support/tag_of_fwd.hpp>
    #include <boost/fusion/include/tag_of_fwd.hpp>

    namespace boost { namespace fusion { namespace traits {
        template<>
        struct tag_of<example_struct>
        {
            typedef example::example_sequence_tag type;
        };
    }}}

`traits::tag_of` also has a second template argument, that can be used in conjuction with `boost::enable_if` to provide tag support for groups of related types. This feature is not necessary for our sequence, but for an example see the code in:

    #include <boost/fusion/adapted/array/tag_of.hpp>
    #include <boost/fusion/include/tag_of.hpp>

##### Designing a suitable iterator

We need an iterator to describe positions, and provide access to the data within our sequence. As it is straightforward to do, we are going to provide a random access iterator in our example.

We will use a simple design, in which the 2 members of `example_struct` are given numbered indices, 0 for `name` and 1 for `age` respectively.

    template<typename Struct, int Pos>
    struct example_struct_iterator
        : boost::fusion::iterator_base<example_struct_iterator<Struct, Pos> >
    {
        BOOST_STATIC_ASSERT(Pos >=0 && Pos < 3);
        typedef Struct struct_type;
        typedef boost::mpl::int_<Pos> index;
        typedef boost::fusion::random_access_traversal_tag category;

        example_struct_iterator(Struct& str)
            : struct_(str) {}

        Struct& struct_;
    };

A quick summary of the details of our iterator:

1. The iterator is parameterized by the type it is iterating over, and the index of the current element.
2. The typedefs `struct_type` and `index` provide convenient access to information we will need later in the implementation.
3. The typedef `category` allows the `traits::`category_of`` metafunction to establish the traversal category of the iterator.
4. The constructor stores a reference to the `example_struct` being iterated over.

We also need to enable tag dispatching for our iterator type, with another specialization of `traits::tag_of`.

In isolation, the iterator implementation is pretty dry. Things should become clearer as we add features to our implementation.

##### A first couple of instructive features

To start with, we will get the `result_of::value_of` metafunction working. To do this, we provide a specialization of the `boost::fusion::extension::value_of_impl` template for our iterator's tag type.

    template<>
    struct value_of_impl<example::example_struct_iterator_tag>
    {
        template<typename Iterator>
        struct apply;

        template<typename Struct>
        struct apply<example::example_struct_iterator<Struct, 0> >
        {
            typedef std::string type;
        };

        template<typename Struct>
        struct apply<example::example_struct_iterator<Struct, 1> >
        {
            typedef int type;
        };
    };

The implementation itself is pretty simple, it just uses 2 partial specializations to provide the type of the 2 different members of `example_struct`, based on the index of the iterator.

To understand how `value_of_impl` is used by the library we will look at the implementation of `result_of::value_of`:

    template <typename Iterator>
    struct value_of
        : extension::value_of_impl<typename detail::tag_of<Iterator>::type>::
            template apply<Iterator>
    {};

So `result_of::value_of` uses tag dispatching to select an MPL Metafunction Class to provide its functionality. You will notice this pattern throughout the implementation of Fusion.

Ok, lets enable dereferencing of our iterator. In this case we must provide a suitable specialization of `deref_impl`.

    template<>
    struct deref_impl<example::example_struct_iterator_tag>
    {
        template<typename Iterator>
        struct apply;

        template<typename Struct>
        struct apply<example::example_struct_iterator<Struct, 0> >
        {
            typedef typename mpl::if_<
                is_const<Struct>, std::string const&, std::string&>::type type;

            static type
            call(example::example_struct_iterator<Struct, 0> const& it)
            {
                return it.struct_.name;
            }
        };

        template<typename Struct>
        struct apply<example::example_struct_iterator<Struct, 1> >
        {
            typedef typename mpl::if_<
                is_const<Struct>, int const&, int&>::type type;

            static type
            call(example::example_struct_iterator<Struct, 1> const& it)
            {
                    return it.struct_.age;
                }
            };
        };
    }

The use of `deref_impl` is very similar to that of `value_of_impl`, but it also provides some runtime functionality this time via the `call` static member function. To see how `deref_impl` is used, lets have a look at the implementation of `deref`:

    namespace result_of
    {
        template <typename Iterator>
        struct deref
            : extension::deref_impl<typename detail::tag_of<Iterator>::type>::
                template apply<Iterator>
        {};
    }

    template <typename Iterator>
    typename result_of::deref<Iterator>::type
    deref(Iterator const& i)
    {
        typedef result_of::deref<Iterator> deref_meta;
        return deref_meta::call(i);
    }

So again `result_of::deref` uses tag dispatching in exactly the same way as the `result_of::value_of` implementation. The runtime functionality used by `deref` is provided by the `call` static function of the selected MPL Metafunction Class.

The actual implementation of `deref_impl` is slightly more complex than that of `value_of_impl`. We also need to implement the `call` function, which returns a reference to the appropriate member of the underlying sequence. We also require a little bit of metaprogramming to return `const` references if the underlying sequence is const.

| Note
Although there is a fair amount of left to do to produce a fully fledged Fusion sequence, `result_of::value_of` and `deref` illustrate all the signficant concepts required. The remainder of the process is very repetitive, simply requiring implementation of a suitable `xxxx_impl` for each feature `xxxx`.

##### Implementing the remaining iterator functionality

Ok, now we have seen the way `result_of::value_of` and `deref` work, everything else will work in pretty much the same way. Lets start with forward iteration, by providing a `next_impl`:

    template<>
    struct next_impl<example::example_struct_iterator_tag>
    {
        template<typename Iterator>
        struct apply
        {
            typedef typename Iterator::struct_type struct_type;
            typedef typename Iterator::index index;
            typedef example::example_struct_iterator<struct_type, index::value + 1> type;

            static type
            call(Iterator const& i)
            {
                 return type(i.struct_);
            }
        };
    };

This should be very familiar from our `deref_impl` implementation, we will be using this approach again and again now. Our design is simply to increment the `index` counter to move on to the next element. The various other iterator manipulations we need to perform will all just involve simple calculations with the `index` variables.

We also need to provide a suitable `equal_to_impl` so that iterators can be correctly compared. A Bidirectional Iterator will also need an implementation of `prior_impl`. For a Random Access Iterator `distance_impl` and `advance_impl` also need to be provided in order to satisfy the necessary complexity guarantees. As our iterator is a Random Access Iterator we will have to implement all of these functions.

Full implementations of `prior_impl`, `advance_impl`, `distance_impl` and `equal_to_impl` are provided in the example code.

##### Implementing the intrinsic functions of the sequence

In order that Fusion can correctly identify our sequence as a Fusion sequence, we need to enable `is_sequence` for our sequence type. As usual we just create an `impl` type specialized for our sequence tag:

    template<>
    struct is_sequence_impl<example::example_sequence_tag>
    {
        template<typename T>
        struct apply : mpl::true_ {};
    };

We've some similar formalities to complete, providing `category_of_impl` so Fusion can correctly identify our sequence type, and `is_view_impl` so Fusion can correctly identify our sequence as not being a View type. Implementations are provide in the example code.

Now we've completed some formalities, on to more interesting features. Lets get `begin` working so that we can get an iterator to start accessing the data in our sequence.

    template<>
    struct begin_impl<example::example_sequence_tag>
    {
        template<typename Sequence>
        struct apply
        {
            typedef example::example_struct_iterator<Sequence, 0> type;

            static type
            call(Sequence& seq)
            {
                return type(seq);
            }
        };
    };

The implementation uses the same ideas we have applied throughout, in this case we are just creating one of the iterators we developed earlier, pointing to the first element in the sequence. The implementation of `end` is very similar, and is provided in the example code.

For our Random Access Sequence we will also need to implement `size_impl`, `value_at_impl` and `at_impl`.

##### Enabling our type as an associative sequence

In order for `example_struct` to serve as an associative forward sequence, we need to adapt the traversal category of our sequence and our iterator accordingly and enable 3 intrinsic sequence lookup features, `at_key`, __value_atkey_ and `has_key`. We also need to enable 3 iterator lookup features, `result_of::key_of`, `result_of::value_of_data` and `deref_data`.

To implement `at_key_impl` we need to associate the `fields::name` and `fields::age` types described in the Quick Start guide with the appropriate members of `example_struct`. Our implementation is as follows:

    template<>
    struct at_key_impl<example::example_sequence_tag>
    {
        template<typename Sequence, typename Key>
        struct apply;

        template<typename Sequence>
        struct apply<Sequence, fields::name>
        {
            typedef typename mpl::if_<
                is_const<Sequence>,
                std::string const&,
                std::string&>::type type;

            static type
            call(Sequence& seq)
            {
                return seq.name;
            };
        };

        template<typename Sequence>
        struct apply<Sequence, fields::age>
        {
            typedef typename mpl::if_<
                is_const<Sequence>,
                int const&,
                int&>::type type;

            static type
            call(Sequence& seq)
            {
                return seq.age;
            };
        };
    };

Its all very similar to the implementations we've seen previously, such as `deref_impl` and `value_of_impl`. Instead of identifying the members by index or position, we are now selecting them using the types `fields::name` and `fields::age`. The implementations of the other functions are equally straightforward, and are provided in the example code.

##### Summary

We've now worked through the entire process for adding a new random access sequence and we've also enabled our type to serve as an associative sequence. The implementation was slightly longwinded, but followed a simple repeating pattern.

The support for `std::pair`, MPL sequences, and `boost::array` all use the same approach, and provide additional examples of the approach for a variety of types.