I was recently attempting to get Botan (http://botan.randombit.net) working with Python 2.6.6 on FreeBSD when it failed to compile, I filled a bug with Botan (http://bugs.randombit.net/show_bug.cgi?id=135) and first thought it was a compiler issue, and then thought it might be a Boost.Python issue.

However after further digging, see comment 3 (http://bugs.randombit.net/show_bug.cgi?id=135#c3) I figured out it had to do with the fix that went in to pyport.h for issues with isspace/toupper/et al. on FreeBSD's libc and also Mac OS X.

As can be seen in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=36519&r2=36793 a change was added to override the original definitions with a new one that used the wide character support. It was modified again in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=77585&r2=80178 to also include Mac OS X for http://bugs.python.org/issue7072.

The issue here is that if this file is included in any C++ code that then also includes <locale> or any of other functions that might pull in localefwd.h the defines will cause actual C++ code functions to be overwritten with invalid data, and the C++ compiler will throw errors such as:

/usr/include/c++/4.2/bits/localefwd.h:58:34: error: macro "isspace" passed 2
arguments, but takes just 1

Putting an #if 0, #endif around that block of code allows Botan's python module to cleanly compile without issues.

I do apologise that I don't have a simple C++ program that reproduces the problem.

I'm upgrading from Python 2.6.5 to 2.7.1, and I'm getting the error below when compiling my code using Boost 1.45 and gcc 4.2.1 in OSX 10.6.6.  The following thread describes similar symptoms related to the ordering of header files and macro definitions:
http://code.google.com/p/unladen-swallow/issues/detail?id=87#c13

The suggested patch to pyport.h (http://codereview.appspot.com/179049/patch/1/2) does not fix my problem.  However, prepending 

#include "Python.h"

to each source file (e.g., bend.cc below) does fix the problem.  Is there a simple way to fix this without adding Python.h to every source file in my codebase?

I've installed Boost and Python from source to non-standard directories, Boost 1.45 by:
cd $(BOOST); ./bootstrap.sh --prefix=$(CURDIR)/$(BOOST) --with-python-root=$(CURDIR)/$(PYTHON_DIR) --with-python-version=2.7 --with-libraries=mpi

and Python 2.7 by:
cd $(PYTHON_SRCDIR) ; ./configure --enable-shared --prefix=$(CURDIR)/$(PYTHON_DIR)
cd $(PYTHON_SRCDIR) ; make -j $(NUMPROC) && make install

Any guidance will be greatly appreciated.

Kind regards,
Nasos

===============
...
darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o
In file included from /usr/include/c++/4.2.1/ios:47,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58:34: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:70:34: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:74:34: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:78:34: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:94:34: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:102:34: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:106:34: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
                 from /usr/include/c++/4.2.1/ios:50,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:242:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:271:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:814:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:847:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4611:44: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4629:44: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4635:44: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4641:44: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4665:44: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4677:44: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4683:44: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/ios:47,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: ‘std::isspace’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: template declaration of ‘bool std::isspace’
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: ‘std::isupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: template declaration of ‘bool std::isupper’
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: ‘std::islower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: template declaration of ‘bool std::islower’
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: ‘std::isalpha’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: template declaration of ‘bool std::isalpha’
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: ‘std::isalnum’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: template declaration of ‘bool std::isalnum’
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: ‘std::toupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: template declaration of ‘_CharT std::toupper’
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: ‘std::tolower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: template declaration of ‘_CharT std::tolower’
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
                 from /usr/include/c++/4.2.1/ios:50,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:227: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:242: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:255: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h:256: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:271: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:287: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towupper(int (*)(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:228: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towlower(int (*)(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:257: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:797: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:814: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:829: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h:830: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:847: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:866: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype<char>::towupper(int (*)(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:798: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype<char>::towlower(int (*)(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:831: error: ‘__c’ was not declared in this scope
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
                 from /usr/include/c++/4.2.1/ios:50,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:4611: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4629: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4635: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4641: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4665: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4677: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4683: error: function definition does not declare parameters

    "g++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Wstrict-aliasing=2 -ffast-math -fno-strict-aliasing -DBOOST_PYTHON_MAX_ARITY=30 -D_REENTRANT  -I"boost" -I"python/include/python2.7" -c -o "bin/darwin-4.2.1/release/threading-multi/Dbend.o" "bend.cc"

...failed darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o...

This doesn't look like it has anything to to with the 'ctypes' library
component (http://docs.python.org/library/ctypes).  So I am removing
'ctypes' from the 'Components' selection.  Please correct me if I am
wrong ...

Meador et al,

Thanks for your attention to this issue.  Just to clarify, I can eliminate
the compile errors by prepending

#include "Python.h"

to any source file with

#include<boost/python.hpp>

Also, my code compiles in Linux, with and without the Python.h header:

$ uname -a
Linux nxx.xx.com 2.6.18-128.1.14.el5 #1 SMP Wed Jun 17 06:38:05 EDT 2009
x86_64 x86_64 x86_64 GNU/Linux

with the same flags as before:

cd $(BOOST); ./bootstrap.sh --prefix=$(CURDIR)/$(BOOST)
--with-python-root=$(CURDIR)/$(PYTHON_DIR) --with-python-version=2.7
--with-libraries=mpi

cd $(PYTHON_SRCDIR) ; ./configure --enable-shared
--prefix=$(CURDIR)/$(PYTHON_DIR)
cd $(PYTHON_SRCDIR) ; make -j $(NUMPROC) && make install

Thanks again,
Nasos

The problem is that pyport tries to replace the ctypes definion of the isascii(ch) and related functions by a replacement that works better with UTF-8 on FreeBSD and OSX. That replacement is incompatible with the localfwd.h header.

The attached patch is a very crude hack that should fix the issue for this. This assumes that Python itself won't be compiled with C++ though, and might cause problems in C++ programs that use locale.h instead of localfwd.h.

As such I won't apply the patch just yet, the patch needs review from someone that's more experienced w.r.t. C++.

Ronald,

Thanks much for your help and insight.  I tried the patch but unfortunately it didn't work for me-- this might be because I have Python extensions written in C++.

Any other suggestions?  Should I wait for the next version of boost.python?

Thanks and regards,
Nasos

Waiting for a new version of boost probably won't help: this is an incompatibility between a redefinition of ctypes macros in pyport.h and definitions in the C++ header <locale>.

I barely use C++ at this time and don't know how to tweak the headers to avoid this problem. It would probably have been better to not redefine the ctype.h definitions, we should have introduced Py_isascii etc. That requires some invasive changes to the Python codebase though, and I'm not sure if that is worth the trouble.

Could someone attach a sample project that suffers from this issue (with instructions on how to build it)? That would make it easier to debug the problem.

(Removing python 2.6 because that version receives only security patches at this time, adding 3.2 and 3.3 because those use the same code in pyport.h and hence should also be affected by this issue)

Ronald, thanks again.  I stripped out as much as I could from the
original code and generated the attached project.  In doing so, I
narrowed the problem down to boost/weak_ptr.hpp.

The sample project consists of:

test-boost-python/Makefile
test-boost-python/src
test-boost-python/src/Jamroot
test-boost-python/src/test
test-boost-python/src/test/Jamfile
test-boost-python/src/test/TestBoostPython.cc
test-boost-python/src/test/TestBoostPython.hh
test-boost-python/tools/Makefile

Before building the project (just type "make" in test-boost-python),
you'll need to download Boost and save it to
test-boost-python/tools/boost/boost.tar.bz2; also, the tools/Makefile
automatically downloads Python 2.7.2, but you can try Python 3.2+ in
its place.

Thanks and kind regards,
Nasos

With attachment--

I am experiencing the same issue. Googled for a bit but couldn't find much with the error, then changed the search context, come across to this thread. 


In file included from /opt/local/include/boost/date_time/gregorian/parsers.hpp:13,
                 from /opt/local/include/boost/date_time/gregorian/gregorian.hpp:34,
                 from /opt/local/include/boost/date_time/posix_time/time_formatters.hpp:12,
                 from /opt/local/include/boost/date_time/posix_time/posix_time.hpp:24,
                 from /opt/local/include/boost/date_time/local_time/local_time.hpp:11,
                 from /opt/local/include/boost/date_time.hpp:15,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:11,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/opt/local/include/boost/date_time/date_parsing.hpp:50:33: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:13:21: error: Devices.h: No such file or directory
In file included from /opt/local/include/boost/date_time/gregorian/parsers.hpp:13,
                 from /opt/local/include/boost/date_time/gregorian/gregorian.hpp:34,
                 from /opt/local/include/boost/date_time/posix_time/time_formatters.hpp:12,
                 from /opt/local/include/boost/date_time/posix_time/posix_time.hpp:24,
                 from /opt/local/include/boost/date_time/local_time/local_time.hpp:11,
                 from /opt/local/include/boost/date_time.hpp:15,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:11,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/opt/local/include/boost/date_time/date_parsing.hpp: In function ‘std::string boost::date_time::convert_to_lower(std::string)’:
/opt/local/include/boost/date_time/date_parsing.hpp:50: error: cannot resolve overloaded function ‘tolower’ based on conversion to type ‘char’
In file included from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:

if I put "#include <cctype>" before boost/date_time, it solves my problem.

And by the way, i can't seem to get to the point where boost/week_ptr.hpp causes the error. I tried to compile Nasos's upload test-boost-python. That codes works on my machine.

Nevermind, putting #include <cctype> doesn't help. I have to put <python.h> on top to get away this.

The only real fix I found is to introduce "Py_" prefixed versions of all definitions in ctypes.h that are used in Python (that is Py_isalnum) and use that throughout the python source tree.

That's a pretty invasive patch though and would probably be off-limit for 2.7 and 3.2.

In my first comment on this bug post I posted what project has issues with this, Botan with Boost.Python on FreeBSD and Mac OS X.

If required I will post how to reproduce this error using that project. If you would prefer a simplified test case, unfortunately I don't have the time to dig into that and find one.

One way to solve it is by including the Python header before any other file, however this is not an option for most 3rd party software that never sees this bug because those functions are not redefined for Linux...

Per Chen Huang's comment, I retested my test case code using several versions of boost and Python 2.7:

boost 1.45: build fails, errors as reported.
boost 1.46: success.
boost 1.47: success.

I've attached a revised test case that automatically downloads and builds boost 1.45.