[C++-sig] Returning values to Python in C++ reference arguments

[Jason Addison]

How can results be returned in function arguments?

I've include my example C++ extension and Python code below.

I've tried, what I think, are some obvious approaches (Python objects,
ctypes), though none have worked.

It seems like this should be doable.

If the python code owns, say, a c_int, I would thing it is mostly safe
to pass it to c++ to be modified.

This seems like it should be a common question, though I can't find it
being explicitly addressed in the documentation

All of the call policy stuff seems to only apply to the function result.

Can the policy stuff be applied to the function arguments?

If so, how would that work with my example?

Thanks,
Jason


my_module.cpp
/*
clang++ -c -I/usr/include/python2.7 my_module.cpp
clang++ -dynamiclib -o my_module.so -lboost_python -lpython2.7
my_module.o
*/

#include <boost/python.hpp>

struct Foo {
void bar(int &a, double &b) {
a = 12;
b = 6.2832;
}
};

using namespace boost::python;

BOOST_PYTHON_MODULE(my_module)
{
class_<Foo>("Foo", init<>())
.def("bar", &Foo::bar)
;
}
<<<<


byref.py
from ctypes import *
import my_module

f = my_module.Foo()

a = 0
b = 0.

# py objects
'''
f.bar(a, b)

Traceback (most recent call last):
File "byref.py", line 8, in <module>
f.bar(a, b)
Boost.Python.ArgumentError: Python argument types in
Foo.bar(Foo, int, float)
did not match C++ signature:
bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''

x = c_int(0)
y = c_double(0)

# as is
'''
f.bar(x, y)

Traceback (most recent call last):
File "byref.py", line 25, in <module>
f.bar(x, y)
Boost.Python.ArgumentError: Python argument types in
Foo.bar(Foo, c_int, c_double)
did not match C++ signature:
bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''

# ctype.byref
'''
f.bar(byref(x), byref(y))

Traceback (most recent call last):
File "byref.py", line 36, in <module>
f.bar(byref(x), byref(y))
Boost.Python.ArgumentError: Python argument types in
Foo.bar(Foo, CArgObject, CArgObject)
did not match C++ signature:
bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''

# ctype.pointer
'''
f.bar(pointer(x), pointer(y))

Traceback (most recent call last):
File "byref.py", line 48, in <module>
f.bar(pointer(x), pointer(y))
Boost.Python.ArgumentError: Python argument types in
Foo.bar(Foo, LP_c_int, LP_c_double)
did not match C++ signature:
bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''
<<<<

[Nikolay Mladenov]

You have to do something like that

using namespace boost::python;

BOOST_PYTHON_MODULE(my_module)
{
struct wrapper{
static tuple bar(Foo & f){
double a, b;
f.bar(a,b);
return make_tuple(a,b);
}
};
class_<Foo>("Foo", init<>())
.def("bar", &wrapper::bar)
;
}

a,b = Foo().bar()

On Mon, May 25, 2015 at 5:39 AM, Trigve Siver via Cplusplus-sig <

----- Original Message -----

Sent: Saturday, May 23, 2015 6:05 PM
Subject: [C++-sig] Returning values to Python in C++ reference arguments
How can results be returned in function arguments?

I don't think that you can return arguments by reference/pointers in
python. You could theoretically pass a mutable sequence (list) and push the
objects there.
But the common approach is to return tuple.
_______________________________________________
Cplusplus-sig mailing list
https://mail.python.org/mailman/listinfo/cplusplus-sig

[Holger Joukl]

Von: Jason Addison
How can results be returned in function arguments?
I've include my example C++ extension and Python code below.
I've tried, what I think, are some obvious approaches (Python objects,
ctypes), though none have worked.
It seems like this should be doable.

As already mentioned, you can use helper functions and return tuples.

Or you could expose custom "ref-object" classes to Python, s.th. along the
lines of:

// ref_object.hpp
#if !defined REF_OBJECT
#define REF_OBJECT


#include <boost/python.hpp>
#include <algorithm>


namespace bp = boost::python;

namespace refob {


// Basic RefObject
template<typename T>
struct RefObject {
public:
T t;

RefObject();
// callable operator
T operator()();

// conversion functions: implicit castability
operator T&() { return t; }
};


// RefObject for handling C-style arrays
// This provides an operator() that allows access to
// a list of values.
// NOTE: If numElements > the actual stored number of elements then
// THIS WILL CRASH! It is essential to correctly apply set_num() after
// this has been used as a function argument, in some thin wrappers
// There is only some very rudimentary safety in that _numElements is
// initialized to 0
// I really don't see a better way to make this safer
template<typename T>
struct RefObject<T const *> {
T const * t;

RefObject();
// callable operator
bp::list operator()(unsigned int numElements=0);

// conversion functions: implicit castability
operator T const *&() { return t; }

void set_num(unsigned int const & numElements);

private:
unsigned int _numElements;
};


// RefObject specialization for void*
template<>
struct RefObject<void *> {
void* t;

RefObject();
// callable operator
void* operator()();
};


// RefObject specialization for void const* (where we cast away const for
the
// operator() call result to make boost work automagically
template<>
struct RefObject<void const *> {
void const * t;

RefObject();
// callable operator
void* operator()();
};


// RefObject specialization for char const* which boost.python will
automatically
// handle as a python string (necessary to distingish from the C-style
array
// handling RefObject, see above
template<>
struct RefObject<char const *> {
char const * t;

RefObject();
// callable operator
char const* operator()();
};


// Is inlining the way to go to avoid multiply defined symbols here?
// Note: Remember that
// * a specialization is not a template but a concrete (member function in
// this case)
// * as ref_object.hpp are
// #included from every module there will be multiple definitions (one
// in each compilation unit)
// ==> linker chokes
// Using inline this can be avoided
// See
// http://msdn.microsoft.com/en-us/magazine/cc163769.aspx
// http://www.parashift.com/c++-faq-lite/inline-functions.html
// for background

template <typename T>
RefObject<T>::RefObject(): t() {
//std::cout << "RefObject<" << typeid(T).name() << ">::RefObject()" <<
std::endl;
}


template <typename T>
RefObject<T const *>::RefObject(): t(), _numElements(0) {
//std::cout << "RefObject<" << typeid(T).name() << " const
*>::RefObject()" << std::endl;
}


inline RefObject<void *>::RefObject(): t() {
//std::cout << "RefObject<void *>::RefObject()" << std::endl;
}


inline RefObject<void const *>::RefObject(): t() {
//std::cout << "RefObject<void const *>::RefObject()" << std::endl;
}


inline RefObject<char const *>::RefObject(): t() {
//std::cout << "RefObject<char const *>::RefObject()" << std::endl;
}


template <typename T>
T RefObject<T>::operator()()
{
// Do we need to care about pointer data issues here?
return t;
}


template<typename T>
bp::list RefObject<T const *>::operator()(unsigned int numElements)
{
bp::list list_object;

unsigned int maxElements = std::min(numElements, _numElements);
for (unsigned int i=0; i < maxElements; i++) {
bp::object item(*t);
list_object.append(item);
}
return list_object;
}


inline void* RefObject<void const *>::operator()()
{
void* non_const_ptr = const_cast<void*>(t);
//std::cout << "returning non-const void* t" << std::endl;
return non_const_ptr;
}


inline void* RefObject<void *>::operator()()
{
return t;
}


inline char const* RefObject<char const *>::operator()()
{
return t;
}


template<typename T>
void RefObject<T const *>::set_num(unsigned int const & numElements)
{
_numElements = numElements;
}


} // namespace refob

#endif // #if !defined REF_OBJECT


And you might expose those to Python:

// ref_object.cpp
#include <boost/python.hpp>
#include "ref_object.hpp"


namespace bp = boost::python;


namespace refob
{


// dummy function, expose this to force creation of void* converters in
boost registry
void* void_ptr_from_void_ptr(void* void_ptr) {
return void_ptr;
}


void export_reference_type_classes()
{

// // dummy function registration to force creation of void* converters
// // (not necessary if there is an exposed function returning void*)
// bp::def("void_ptr_from_void_ptr", &void_ptr_from_void_ptr,
// bp::return_value_policy<bp::return_opaque_pointer>());

// const-pointer types

// 1. char

// 1.1: const char*
bp::class_<RefObject<const char*>, boost::noncopyable>
("const_char_ptr_ref")
.def(bp::init<>())
.def("__call__", &RefObject<const char*>::operator())
;


// 2. numeric types (float/double/int)

// Note: signed and unsigned char are *numeric* types usage-wise
// const signed char*
// bp::class_<RefObject<const signed char*>, boost::noncopyable>
("const_signed_char_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const signed char*>::operator())
// ;


// // const unsigned char*
// bp::class_<RefObject<const unsigned char*>, boost::noncopyable>
("const_unsigned_char_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const unsigned char*>::operator())
// ;

// const double*
bp::class_<RefObject<const double*>, boost::noncopyable>
("const_double_ptr_ref")
.def(bp::init<>())
.def("__call__", &RefObject<const double*>::operator())
;
bp::implicitly_convertible< RefObject<const double*>, double const * >
();

// // const float*
// bp::class_<RefObject<const float*>, boost::noncopyable>
("const_float_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const float*>::operator())
// ;

// // const short*
// bp::class_<RefObject<const short*>, boost::noncopyable>
("const_short_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const short*>::operator())
// ;

// // const unsigned short*
// bp::class_<RefObject<const unsigned short*>, boost::noncopyable>
("const_unsigned_short_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const unsigned short*>::operator())
// ;

// // const int*
// bp::class_<RefObject<const int*>, boost::noncopyable>
("const_int_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const int*>::operator())
// ;

// // const unsigned int*
// bp::class_<RefObject<const unsigned int*>, boost::noncopyable>
("const_unsigned_int_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const unsigned int*>::operator())
// ;


// // const long long*
// bp::class_<RefObject<const long long*>, boost::noncopyable>
("const_long_long_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const long long*>::operator())
// ;

// // const unsigned long long*
// bp::class_<RefObject<const unsigned long long*>, boost::noncopyable>
("const_unsigned_long_long_ptr_ref")
// .def(bp::init<>())
// .def("__call__", &RefObject<const unsigned long long*>::operator
())
// ;


// 3. void

// const void*
bp::class_<RefObject<const void*>, boost::noncopyable>
("const_void_ptr_ref")
.def(bp::init<>())
.def("__call__", &RefObject<const void*>::operator(),
bp::return_value_policy<bp::return_opaque_pointer>())
;



// NON-const-pointer types:

// void*
bp::class_<RefObject<void*>, boost::noncopyable>("void_ptr_ref")
.def(bp::init<>())
.def("__call__", &RefObject<void*>::operator(),
bp::return_value_policy<bp::return_opaque_pointer>())
;


// Basic types

// 1. char types

// char
bp::class_<RefObject<char>, boost::noncopyable>("char_ref")
.def(bp::init<>())
.def("__call__", &RefObject<char>::operator())
;


// 2. numeric types

// Note: signed and unsigned char are *numeric* types usage-wise
// const signed char*
// signed char
bp::class_<RefObject<signed char>, boost::noncopyable>
("signed_char_ref")
.def(bp::init<>())
.def("__call__", &RefObject<signed char>::operator())
;

// unsigned char
bp::class_<RefObject<unsigned char>, boost::noncopyable>
("unsigned_char_ref")
.def(bp::init<>())
.def("__call__", &RefObject<unsigned char>::operator())
;

// double
bp::class_<RefObject<double>, boost::noncopyable>("double_ref")
.def(bp::init<>())
.def("__call__", &RefObject<double>::operator())
;

// float
bp::class_<RefObject<float>, boost::noncopyable>("float_ref")
.def(bp::init<>())
.def("__call__", &RefObject<float>::operator())
;

// short
bp::class_<RefObject<short>, boost::noncopyable>("short_ref")
.def(bp::init<>())
.def("__call__", &RefObject<short>::operator())
;

// unsigned short
bp::class_<RefObject<unsigned short>, boost::noncopyable>
("unsigned_short_ref")
.def(bp::init<>())
.def("__call__", &RefObject<unsigned short>::operator())
;

// int
bp::class_<RefObject<int>, boost::noncopyable>("int_ref")
.def(bp::init<>())
.def("__call__", &RefObject<int>::operator())
;

// unsigned int
bp::class_<RefObject<unsigned int>, boost::noncopyable>
("unsigned_int_ref")
.def(bp::init<>())
.def("__call__", &RefObject<unsigned int>::operator())
;
bp::implicitly_convertible< RefObject<unsigned int>, unsigned int >();

// long long
bp::class_<RefObject<long long>, boost::noncopyable>("long_long_ref")
.def(bp::init<>())
.def("__call__", &RefObject<long long>::operator())
;

// unsigned long long
bp::class_<RefObject<unsigned long long>, boost::noncopyable>
("unsigned_long_long_ref")
.def(bp::init<>())
.def("__call__", &RefObject<unsigned long long>::operator())
;

};

} // namespace refob


Add to your module (or consider putting into a separate extension module):


namespace refob {
//forward declaration must be in appropriate namespace
void export_reference_type_classes(); // helpers/ref_object.cpp
}
BOOST_PYTHON_MODULE(my_module) {

refob::export_reference_type_classes();
...

You'd then have to use the specialized ref objects as function arguments,
much
like what you tried with the ctypes stuff:

import my_module
f = my_module.Foo()
a = my_module.int_ref(0)
b = my_module.double_ref(0)
f.bar(a, b)

print "a result:", a()

If such effort makes sense depends on how "reference-call-littered"
the C++ API you wrap is.

Note: I hand-edited the original code a little to remove some irrelevant
stuff
and didn't try to compile and run it afterwards. But you get the idea.

Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

[Jason Addison]

On Mon, May 25, 2015 at 3:39 AM, Trigve Siver via Cplusplus-sig

Post by Jason Addison
How can results be returned in function arguments?

I don't think that you can return arguments by reference/pointers in python. You could theoretically pass a mutable sequence (list) and push the objects there.

You can return results via function arguments from C to Python (see code below).

Why is something similar impossible with Boost Python?

my_module2.c
/*
clang -c -I/usr/include/python2.7 my_module2.c
clang -dynamiclib -o my_module2.so -lpython2.7 my_module2.o
*/

void foo_bar(int *a, double *b) {
*a = 12;
*b = 6.2832;
}
<<<<

byref2.py
from ctypes import *
my_module2 = CDLL('my_module2.so')

x = c_int(0)
y = c_double(0)

my_module2.foo_bar(byref(x), byref(y))

assert(x.value == 12)
assert(y.value == 6.2832)
<<<<

But the common approach is to return tuple.

[Holger Joukl]

Hi,

Von: Jason Addison
You can return results via function arguments from C to Python (see code below).
Why is something similar impossible with Boost Python?
my_module2.c
/*
clang -c -I/usr/include/python2.7 my_module2.c
clang -dynamiclib -o my_module2.so -lpython2.7 my_module2.o
*/
void foo_bar(int *a, double *b) {
*a = 12;
*b = 6.2832;
}
<<<<
byref2.py
from ctypes import *
my_module2 = CDLL('my_module2.so')
x = c_int(0)
y = c_double(0)

What does type(x) and type(y) give you?

my_module2.foo_bar(byref(x), byref(y))
assert(x.value == 12)
assert(y.value == 6.2832)
<<<<

(unrelated sidenote: it's probably not the best idea to test float/double
equality)

Best regards
Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart