| Home | Libraries | People | FAQ | More

## An efficient polymorphic data structure

Suppose we have a `base` abstract class with implementations `derived1`, `derived2` and `derived3`. The memory layout of a `std::vector<base*>` (or similar constructs such as `std::vector<std::unique_ptr<base>>` or `boost::ptr_vector<base>`) looks like the following:

Elements that are adjacent in the vector are not necessarily allocated contiguously, much less so if the vector has undergone mid insertions and deletions. A typical processing operation

    std::vector<base*> v;
    ...
    for(base* b: v){
      ... // access base's virtual interface
    }

is impacted negatively by two factors:

- Scattering of elements throughout memory reduces CPU caching efficiency, which in general favor regular access loops to contiguous memory areas.
- Branch prediction tries to minimize the effect of running conditional code (such as an `if`-`else` statement or the invocation of a `base` virtual function) by speculatively executing a given branch based on past history. This mechanism is rendered mostly useless when `derived1`, `derived2` and `derived3` elements are interspersed along the sequence without a definite pattern.

These limitations are imposed by the very nature of dynamic polymorphism: as the exact types of the elements accessed through `base`'s interface are not known, an indirection through `base*` (a particular form of type erasure) is required. There is however a critical observation: even though derived types are not known when traversing a `std::vector<base*>`, the information is typically available at compile time at the point of insertion in the vector:

    std::vector<base*> v;
    ...
    v.insert(new derived2{...}); // the type derived2 is "forgotten" by v

A suitably designed container can take advantage of this information to arrange elements contiguously according to their exact type, which results in an internal data structure (a map of pointers to `std::type_info` objects, actually) pointing to as many vectors or segments as there are derived classes:

Traversing such a structure reduces to looping over all the segments one after another: this is extremely efficient both in terms of caching and branch prediction. In the process we have however lost the free-order capability of a `std::vector<base*>` (free order can only be retained at the segment level), but if this is not relevant to the user application the potential performance gains of switching to this structure are large.

The discussion has focused on base/derived programming, also known as OOP, but it also applies to other forms of dynamic polymorphism:

- `std::function` abstracts callable entities with the same given signature under a common interface. Internally, pointer indirections and virtual-like function calls are used. Memory fragmentation is expected to be lower than with OOP, though, as implementations usually feature the so-called small buffer optimization to avoid heap allocation in some situations.
- The case of `std::function` can be seen as a particular example of a more general form of polymorphism called duck typing, where unrelated types are treated uniformly if they conform to the same given interface (a specified set of member functions and/or operations). Duck typing provides the power of OOP while allowing for greater flexibility as polymorphic types need not derive from a preexisting base class or in general be designed with any particular interface in mind --in fact, the same object can be duck-typed to different interfaces. Among other libraries, Boost.TypeErasure provides duck typing for C++. Under the hood, duck typing requires pointer indirection and virtual call implementation techniques analogous to those of OOP, and so there are the same opportunities for efficient container data structures as we have described.

Boost.PolyCollection provides three different container class templates dealing with OOP, `std::function`-like polymorphism and duck typing as implemented by Boost.TypeErasure:

- `boost::base_collection`
- `boost::function_collection`
- `boost::any_collection`

The interfaces of these containers are mostly the same and follow the usual conventions of standard library containers.