# Special Method and Operator Support

## Overview

BPL supports all of the standard special method names supported by real Python class instances except `__complex__` (more on the reasons below). In addition, it can quickly and easily expose suitable C++ functions and operators as Python operators. The following categories of special method names are supported:

- Basic Customization
- Numeric Operators
- Sequence and Mapping protocols
- Attribute Getters and Setters

## Basic Customization

Python provides a number of special operators for basic customization of a class. Only a brief description is provided below; more complete documentation can be found here.

`__init__`(self)
  Initialize the class instance. For extension classes not subclassed in Python, `__init__` is defined by

          my_class.def(boost::python::constructor<...>())

  (see section "A Simple Example Using BPL").

`__del__`(self)
  Called when the extension instance is about to be destroyed. For extension classes not subclassed in Python, `__del__` is always defined automatically by means of the class' destructor.
`__repr__`(self)
  Create a string representation from which the object can be reconstructed.
`__str__`(self)
  Create a string representation which is suitable for printing.
`__cmp__`(self, other)
  Three-way compare function, used to implement comparison operators (< etc.) Should return a negative integer if `self < other` , zero if `self == other` , a positive integer if `self > other` .
`__hash__`(self)
  Called for the key object for dictionary operations, and by the built-in function hash(). Should return a 32-bit integer usable as a hash value for dictionary operations (only allowed if __cmp__ is also defined)
`__nonzero__`(self)
  called if the object is used as a truth value (e.g. in an if statement)
`__call__` (self[, args...])
  Called when the instance is ``called'' as a function; if this method is defined, `x(arg1, arg2, ...)` is a shorthand for `x.__call__(arg1, arg2, ...)`.

If we have a suitable C++ function that supports any of these features, we can export it like any other function, using its Python special name. For example, suppose that class `Foo` provides a string conversion function:

>     std::string to_string(Foo const& f)
>     {
>         std::ostringstream s;
>         s << f;
>         return s.str();
>     }

This function would be wrapped like this:

>     boost::python::class_builder<Foo> foo_class(my_module, "Foo");
>     foo_class.def(&to_string, "__str__");

Note that BPL also supports automatic wrapping of `__str__` and `__cmp__`. This is explained in the next section and the Table of Automatically Wrapped Methods.

## Numeric Operators

Numeric operators can be exposed manually, by `def`ing C++ [member] functions that support the standard Python numeric protocols. This is the same basic technique used to expose `to_string()` as `__str__()` above, and is covered in detail below. BPL also supports automatic wrapping of numeric operators whenever they have already been defined in C++.

### Exposing C++ Operators Automatically

Supose we wanted to expose a C++ class `BigNum` which supports addition. That is, in C++ we can write:

>     BigNum a, b, c;
>     ...
>     c = a + b;

To enable the same functionality in Python, we first wrap the `BigNum` class as usual:

>     boost::python::class_builder<BigNum> bignum_class(my_module, "BigNum");
>     bignum_class.def(boost::python::constructor<>());
>     ...

Then we export the addition operator like this:

>     bignum_class.def(boost::python::operators<boost::python::op_add>());

Since BigNum also supports subtraction, multiplication, and division, we want to export those also. This can be done in a single command by ``or''ing the operator identifiers together (a complete list of these identifiers and the corresponding operators can be found in the Table of Automatically Wrapped Methods):

>     bignum_class.def(boost::python::operators<(boost::python::op_sub | boost::python::op_mul | boost::python::op_div)>());

[Note that the or-expression must be enclosed in parentheses.]

This form of operator definition can be used to wrap unary and homogeneous binary operators (a homogeneous operator has left and right operands of the same type). Now suppose that our C++ library also supports addition of BigNums and plain integers:

>     BigNum a, b;
>     int i;
>     ...
>     a = b + i;
>     a = i + b;

To wrap these heterogeneous operators, we need to specify a different type for one of the operands. This is done using the `right_operand` and `left_operand` templates:

>     bignum_class.def(boost::python::operators<boost::python::op_add>(), boost::python::right_operand<int>());
>     bignum_class.def(boost::python::operators<boost::python::op_add>(), boost::python::left_operand<int>());

BPL uses overloading to register several variants of the same operation (more on this in the context of coercion). Again, several operators can be exported at once:

>     bignum_class.def(boost::python::operators<(boost::python::op_sub | boost::python::op_mul | boost::python::op_div)>(),
>                      boost::python::right_operand<int>());
>     bignum_class.def(boost::python::operators<(boost::python::op_sub | boost::python::op_mul | boost::python::op_div)>(),
>                      boost::python::left_operand<int>());

The type of the operand not mentioned is taken from the class being wrapped. In our example, the class object is `bignum_class`, and thus the other operand's type is ```BigNum const&`''. You can override this default by explicitly specifying a type in the `operators` template:

>     bignum_class.def(boost::python::operators<boost::python::op_add, BigNum>(), boost::python::right_operand<int>());

Note that automatic wrapping uses the expression ```left + right`'' and can be used uniformly regardless of whether the C++ operators are supplied as free functions

>     BigNum operator+(BigNum, BigNum)

or as member functions

>     BigNum::operator+(BigNum).

For the Python built-in functions `pow()` and `abs()`, there is no corresponding C++ operator. Instead, automatic wrapping attempts to wrap C++ functions of the same name. This only works if those functions are known in namespace `python`. On some compilers (e.g. MSVC) it might be necessary to add a using declaration prior to wrapping:

>     namespace boost { namespace python {
>       using my_namespace::pow;
>       using my_namespace::abs;
>     }

### Wrapping Numeric Operators Manually

In some cases, automatic wrapping of operators may be impossible or undesirable. Suppose, for example, that the modulo operation for BigNums is defined by a set of functions called `mod()`:

>     BigNum mod(BigNum const& left, BigNum const& right);
>     BigNum mod(BigNum const& left, int right);
>     BigNum mod(int left, BigNum const& right);

For automatic wrapping of the modulo function, `operator%()` would be needed. Therefore, the `mod()`-functions must be wrapped manually. That is, we have to export them explicitly with the Python special name "__mod__":

>     bignum_class.def((BigNum (*)(BigNum const&, BigNum const&))&mod, "__mod__");
>     bignum_class.def((BigNum (*)(BigNum const&, int))&mod, "__mod__");

The third form of `mod()` (with `int` as left operand) cannot be wrapped directly. We must first create a function `rmod()` with the operands reversed:

>     BigNum rmod(BigNum const& right, int left)
>     {
>         return mod(left, right);
>     }

This function must be wrapped under the name "__rmod__" (standing for "reverse mod"):

>     bignum_class.def(&rmod,  "__rmod__");

Many of the possible operator names can be found in the Table of Automatically Wrapped Methods. Special treatment is necessary to export the ternary pow operator.

Automatic and manual wrapping can be mixed arbitrarily. Note that you cannot overload the same operator for a given extension class on both ```int`'' and ```float`'', because Python implicitly converts these types into each other. Thus, the overloaded variant found first (be it ```int``` or ```float`'') will be used for either of the two types.

### Coercion

Plain Python can only execute operators with identical types on the left and right hand side. If it encounters an expression where the types of the left and right operand differ, it tries to coerce these types to a common type before invoking the actual operator. Implementing good coercion functions can be difficult if many type combinations must be supported.

BPL solves this problem the same way that C++ does: with overloading. This technique drastically simplifies the code neccessary to support operators: you just register operators for all desired type combinations, and BPL automatically ensures that the correct function is called in each case; there is no need for user-defined coercion functions. To enable operator overloading, BPL provides a standard coercion which is implicitly registered whenever automatic operator wrapping is used.

If you wrap all operator functions manually, but still want to use operator overloading, you have to register the standard coercion function explicitly:

>     // this is not necessary if automatic operator wrapping is used
>     bignum_class.def_standard_coerce();

If you encounter a situation where you absolutely need a customized coercion, you can still define the "__coerce__" operator manually. The signature of a coercion function should look like one of the following (the first is the safest):

>     boost::python::tuple custom_coerce(boost::python::reference left, boost::python::reference right);
>     boost::python::tuple custom_coerce(PyObject* left, PyObject* right);
>     PyObject* custom_coerce(PyObject* left, PyObject* right);

The resulting `tuple` must contain two elements which represent the values of `left` and `right` converted to the same type. Such a function is wrapped as usual:

>     // this must be called before any use of automatic operator
>     // wrapping or a call to some_class.def_standard_coerce()
>     some_class.def(&custom_coerce, "__coerce__");

Note that the standard coercion (defined by use of automatic operator wrapping on a `class_builder` or a call to `class_builder::def_standard_coerce()`) will never be applied if a custom coercion function has been registered. Therefore, in your coercion function you should call

>     boost::python::standard_coerce(left, right);

for all cases that you don't want to handle yourself.

### The Ternary `pow()` Operator

In addition to the usual binary `pow(x, y)` operator (meaning xy), Python also provides a ternary variant that implements xy mod z, presumably using a more efficient algorithm than concatenation of power and modulo operators. Automatic operator wrapping can only be used with the binary variant. Ternary `pow()` must always be wrapped manually. For a homgeneous ternary `pow()`, this is done as usual:

>     BigNum power(BigNum const& first, BigNum const& second, BigNum const& modulus);
>     typedef BigNum (ternary_function1)(const BigNum&, const BigNum&, const BigNum&);
>     ...
>     bignum_class.def((ternary_function1)&power,  "__pow__");

If you want to support this function with non-uniform argument types, wrapping is a little more involved. Suppose you have to wrap:

>     BigNum power(BigNum const& first, int second, int modulus);
>     BigNum power(int first, BigNum const& second, int modulus);
>     BigNum power(int first, int second, BigNum const& modulus);

The first variant can be wrapped as usual:

>     typedef BigNum (ternary_function2)(const BigNum&, int, int);
>     bignum_class.def((ternary_function2)&power,  "__pow__");

In the second variant, however, `BigNum` appears only as second argument, and in the last one it's the third argument. These functions must be presented to BPL such that that the `BigNum` argument appears in first position:

>     BigNum rpower(BigNum const& second, int first, int modulus)
>     {
>         return power(first, second, modulus);
>     }

>     BigNum rrpower(BigNum const& modulus, int first, int second)
>     {
>         return power(first, second, modulus);
>     }

These functions must be wrapped under the names "__rpow__" and "__rrpow__" respectively:

>     bignum_class.def((ternary_function2)&rpower,  "__rpow__");
>     bignum_class.def((ternary_function2)&rrpower,  "__rrpow__");

Note that "__rrpow__" is an extension not present in plain Python.

## Table of Automatically Wrapped Methods

BPL can automatically wrap the following special methods:

Python Operator Name | Python Expression | C++ Operator Id | C++ Expression Used For Automatic Wrapping with `cpp_left = from_python(left, type<Left>())`, `cpp_right = from_python(right, type<Right>())`, and `cpp_oper = from_python(oper, type<Oper>())`
`__add__, __radd__` | `left + right` | `op_add` | `cpp_left + cpp_right`
`__sub__, __rsub__` | `left - right` | `op_sub` | `cpp_left - cpp_right`
`__mul__, __rmul__` | `left * right` | `op_mul` | `cpp_left * cpp_right`
`__div__, __rdiv__` | `left / right` | `op_div` | `cpp_left / cpp_right`
`__mod__, __rmod__` | `left % right` | `op_mod` | `cpp_left % cpp_right`
`__divmod__, __rdivmod__` | `(quotient, remainder) = divmod(left, right)` | `op_divmod` | `cpp_left / cpp_right` `cpp_left % cpp_right`
`__pow__, __rpow__` | `pow(left, right)` (binary power) | `op_pow` | `pow(cpp_left, cpp_right)`
`__rrpow__` | `pow(left, right, modulo)` (ternary power modulo) | no automatic wrapping, special treatment required
`__lshift__, __rlshift__` | `left << right` | `op_lshift` | `cpp_left << cpp_right`
`__rshift__, __rrshift__` | `left >> right` | `op_rshift` | `cpp_left >> cpp_right`
`__and__, __rand__` | `left & right` | `op_and` | `cpp_left & cpp_right`
`__xor__, __rxor__` | `left ^ right` | `op_xor` | `cpp_left ^ cpp_right`
`__or__, __ror__` | `left | right` | `op_or` | `cpp_left | cpp_right`
`__cmp__, __rcmp__` | `cmp(left, right)` `left < right` `left <= right` `left > right` `left >= right` `left == right` `left != right` | `op_cmp` | `cpp_left < cpp_right` `cpp_right < cpp_left`
`__neg__` | `-oper` (unary negation) | `op_neg` | `-cpp_oper`
`__pos__` | `+oper` (identity) | `op_pos` | `+cpp_oper`
`__abs__` | `abs(oper)` (absolute value) | `op_abs` | `abs(cpp_oper)`
`__invert__` | `~oper` (bitwise inversion) | `op_invert` | `~cpp_oper`
`__int__` | `int(oper)` (integer conversion) | `op_int` | `long(cpp_oper)`
`__long__` | `long(oper)` (infinite precision integer conversion) | `op_long` | `PyLong_FromLong(cpp_oper)`
`__float__` | `float(oper)` (float conversion) | `op_float` | `double(cpp_oper)`
`__str__` | `str(oper)` (string conversion) | `op_str` | `std::ostringstream s; s << oper;`
`__coerce__` | `coerce(left, right)` | usually defined automatically, otherwise special treatment required

## Sequence and Mapping Operators

Sequence and mapping operators let wrapped objects behave in accordance to Python's iteration and access protocols. These protocols differ considerably from the ones found in C++. For example, Python's typical iteration idiom looks like

>     for i in S:

while in C++ one writes

>     for (iterator i = S.begin(), end = S.end(); i != end; ++i)

One could try to wrap C++ iterators in order to carry the C++ idiom into Python. However, this does not work very well because

1. It leads to non-uniform Python code (wrapped sequences support a usage different from Python built-in sequences) and
2. Iterators (e.g. `std::vector::iterator`) are often implemented as plain C++ pointers which are problematic for any automatic wrapping system.

It is a better idea to support the standard Python sequence and mapping protocols for your wrapped containers. These operators have to be wrapped manually because there are no corresponding C++ operators that could be used for automatic wrapping. The Python documentation lists the relevant container operators. In particular, expose __getitem__, __setitem__ and remember to raise the appropriate Python exceptions (`PyExc_IndexError` for sequences, `PyExc_KeyError` for mappings) when the requested item is not present.

In the following example, we expose `std::map<std::size_t,std::string>`:

>     typedef std::map<std::size_t, std::string> StringMap;

>     // A helper function for dealing with errors. Throw a Python exception
>     // if p == m.end().
>     void throw_key_error_if_end(
>             const StringMap& m,
>             StringMap::const_iterator p,
>             std::size_t key)
>     {
>         if (p == m.end())
>         {
>             PyErr_SetObject(PyExc_KeyError, boost::python::converters::to_python(key));
>             throw boost::python::error_already_set();
>         }
>     }

>     // Define some simple wrapper functions which match the Python  protocol
>     // for __getitem__, __setitem__, and __delitem__.  Just as in Python, a
>     // free function with a ``self'' first parameter makes a fine class method.

>     const std::string& get_item(const StringMap& self, std::size_t key)
>     {
>         const StringMap::const_iterator p = self.find(key);
>         throw_key_error_if_end(self, p, key);
>         return p->second;
>     }

>     // Sets the item corresponding to key in the map.
>     void StringMapPythonClass::set_item(StringMap& self, std::size_t key, const std::string& value)
>     {
>         self[key] = value;
>     }

>     // Deletes the item corresponding to key from the map.
>     void StringMapPythonClass::del_item(StringMap& self, std::size_t key)
>     {
>         const StringMap::iterator p = self.find(key);
>         throw_key_error_if_end(self, p, key);
>         self.erase(p);
>     }

>     class_builder<StringMap> string_map(my_module, "StringMap");
>     string_map.def(boost::python::constructor<>());
>     string_map.def(&StringMap::size, "__len__");
>     string_map.def(get_item, "__getitem__");
>     string_map.def(set_item, "__setitem__");
>     string_map.def(del_item, "__delitem__");

Then in Python:

>     >>> m = StringMap()
>     >>> m[1]
>     Traceback (innermost last):
>       File "<stdin>", line 1, in ?
>     KeyError: 1
>     >>> m[1] = 'hello'
>     >>> m[1]
>     'hello'
>     >>> del m[1]
>     >>> m[1]            # prove that it's gone
>     Traceback (innermost last):
>       File "<stdin>", line 1, in ?
>     KeyError: 1
>     >>> del m[2]
>     Traceback (innermost last):
>       File "<stdin>", line 1, in ?
>     KeyError: 2
>     >>> len(m)
>     0
>     >>> m[0] = 'zero'
>     >>> m[1] = 'one'
>     >>> m[2] = 'two'
>     >>> m[3] = 'three'
>     >>> len(m)
>     4

## Customized Attribute Access

Just like built-in Python classes, BPL extension classes support special the usual attribute access methods `__getattr__`, `__setattr__`, and `__delattr__`. Because writing these functions can be tedious in the common case where the attributes being accessed are known statically, BPL checks the special names

- `__getattr__<name>__`
- `__setattr__<name>__`
- `__delattr__<name>__`

to provide functional access to the attribute <name>. This facility can be used from C++ or entirely from Python. For example, the following shows how we can implement a ``computed attribute'' in Python:

>     >>> class Range(AnyBPLExtensionClass):
>     ...    def __init__(self, start, end):
>     ...        self.start = start
>     ...        self.end = end
>     ...    def __getattr__length__(self):
>     ...        return self.end - self.start
>     ...
>     >>> x = Range(3, 9)
>     >>> x.length
>     6

#### Direct Access to Data Members

BPL uses the special `__xxxattr__<name>__` functionality described above to allow direct access to data members through the following special functions on `class_builder<>` and `extension_class<>`:

- `def_getter(pointer-to-member, name)` // read access to the member via attribute name
- `def_setter(pointer-to-member, name)` // write access to the member via attribute name
- `def_readonly(pointer-to-member, name)` // read-only access to the member via attribute name
- `def_read_write(pointer-to-member, name)` // read/write access to the member via attribute name

Note that the first two functions, used alone, may produce surprising behavior. For example, when `def_getter()` is used, the default functionality for `setattr()` and `delattr()` remains in effect, operating on items in the extension instance's name-space (i.e., its `__dict__`). For that reason, you'll usually want to stick with `def_readonly` and `def_read_write`.

For example, to expose a `std::pair<int,long>` we might write:

>     typedef std::pair<int,long> Pil;
>     int first(const Pil& x) { return x.first; }
>     long second(const Pil& x) { return x.second; }
>        ...
>     my_module.def(first, "first");
>     my_module.def(second, "second");

>     class_builder<Pil> pair_int_long(my_module, "Pair");
>     pair_int_long.def(boost::python::constructor<>());
>     pair_int_long.def(boost::python::constructor<int,long>());
>     pair_int_long.def_read_write(&Pil::first, "first");
>     pair_int_long.def_read_write(&Pil::second, "second");

Now your Python class has attributes `first` and `second` which, when accessed, actually modify or reflect the values of corresponding data members of the underlying C++ object. Now in Python:

>     >>> x = Pair(3,5)
>     >>> x.first
>     3
>     >>> x.second
>     5
>     >>> x.second = 8
>     >>> x.second
>     8
>     >>> second(x) # Prove that we're not just changing the instance __dict__
>     8

## And what about `__complex__`?

That, dear reader, is one problem we don't know how to solve. The Python source contains the following fragment, indicating the special-case code really is hardwired:

>     /* XXX Hack to support classes with __complex__ method */
>     if (PyInstance_Check(r)) { ...

Next: A Peek Under the Hood Previous: Inheritance Up: Top

© Copyright David Abrahams and Ullrich Köthe 2000. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided ``as is'' without express or implied warranty, and with no claim as to its suitability for any purpose.

Updated: Nov 26, 2000