classification
Title: pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation
Type: compile error Stage: resolved
Components: Build, macOS Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: Chen.Huang, X-Istence, barry-scott, benjamin.peterson, brett.cannon, dim, georg.brandl, gregory.p.smith, koobs, larry, meador.inge, mihaic, ndousis, ned.deily, python-dev, ronaldoussoren
Priority: Keywords: needs review, patch

Created on 2011-01-14 21:21 by X-Istence, last changed 2016-08-15 07:35 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
issue10910.txt ronaldoussoren, 2011-05-07 08:13 review
test-boost-python.tar.gz ndousis, 2011-07-20 03:49
test-boost-python.revised.tar.gz ndousis, 2012-02-05 04:28 revised simplified test case
issue10910-workaround.txt ronaldoussoren, 2013-07-06 12:03 review
issue10910-fix-versionchecks-1.diff dim, 2016-04-03 15:41 Correct FreeBSD version checks for enabling ctype UTF-8 workaround review
Messages (24)
msg126299 - (view) Author: Bert JW Regeer (X-Istence) * Date: 2011-01-14 21:21
I was recently attempting to get Botan (http://botan.randombit.net) working with Python 2.6.6 on FreeBSD when it failed to compile, I filled a bug with Botan (http://bugs.randombit.net/show_bug.cgi?id=135) and first thought it was a compiler issue, and then thought it might be a Boost.Python issue.

However after further digging, see comment 3 (http://bugs.randombit.net/show_bug.cgi?id=135#c3) I figured out it had to do with the fix that went in to pyport.h for issues with isspace/toupper/et al. on FreeBSD's libc and also Mac OS X.

As can be seen in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=36519&r2=36793 a change was added to override the original definitions with a new one that used the wide character support. It was modified again in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=77585&r2=80178 to also include Mac OS X for http://bugs.python.org/issue7072.

The issue here is that if this file is included in any C++ code that then also includes <locale> or any of other functions that might pull in localefwd.h the defines will cause actual C++ code functions to be overwritten with invalid data, and the C++ compiler will throw errors such as:

/usr/include/c++/4.2/bits/localefwd.h:58:34: error: macro "isspace" passed 2
arguments, but takes just 1

Putting an #if 0, #endif around that block of code allows Botan's python module to cleanly compile without issues.

I do apologise that I don't have a simple C++ program that reproduces the problem.
msg130477 - (view) Author: Nasos Dousis (ndousis) Date: 2011-03-09 23:32
I'm upgrading from Python 2.6.5 to 2.7.1, and I'm getting the error below when compiling my code using Boost 1.45 and gcc 4.2.1 in OSX 10.6.6.  The following thread describes similar symptoms related to the ordering of header files and macro definitions:
http://code.google.com/p/unladen-swallow/issues/detail?id=87#c13

The suggested patch to pyport.h (http://codereview.appspot.com/179049/patch/1/2) does not fix my problem.  However, prepending 

#include "Python.h"

to each source file (e.g., bend.cc below) does fix the problem.  Is there a simple way to fix this without adding Python.h to every source file in my codebase?

I've installed Boost and Python from source to non-standard directories, Boost 1.45 by:
cd $(BOOST); ./bootstrap.sh --prefix=$(CURDIR)/$(BOOST) --with-python-root=$(CURDIR)/$(PYTHON_DIR) --with-python-version=2.7 --with-libraries=mpi

and Python 2.7 by:
cd $(PYTHON_SRCDIR) ; ./configure --enable-shared --prefix=$(CURDIR)/$(PYTHON_DIR)
cd $(PYTHON_SRCDIR) ; make -j $(NUMPROC) && make install

Any guidance will be greatly appreciated.

Kind regards,
Nasos

===============
...
darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o
In file included from /usr/include/c++/4.2.1/ios:47,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58:34: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:70:34: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:74:34: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:78:34: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:94:34: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:102:34: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:106:34: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
                 from /usr/include/c++/4.2.1/ios:50,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:242:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:271:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:814:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:847:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4611:44: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4629:44: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4635:44: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4641:44: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4665:44: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4677:44: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4683:44: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/ios:47,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: ‘std::isspace’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: template declaration of ‘bool std::isspace’
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: ‘std::isupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: template declaration of ‘bool std::isupper’
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: ‘std::islower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: template declaration of ‘bool std::islower’
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: ‘std::isalpha’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: template declaration of ‘bool std::isalpha’
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: ‘std::isalnum’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: template declaration of ‘bool std::isalnum’
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: ‘std::toupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: template declaration of ‘_CharT std::toupper’
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: ‘std::tolower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: template declaration of ‘_CharT std::tolower’
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
                 from /usr/include/c++/4.2.1/ios:50,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:227: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:242: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:255: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h:256: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:271: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:287: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towupper(int (*)(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:228: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towlower(int (*)(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:257: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:797: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:814: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:829: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h:830: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:847: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:866: error: expected `;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype<char>::towupper(int (*)(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:798: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype<char>::towlower(int (*)(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:831: error: ‘__c’ was not declared in this scope
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
                 from /usr/include/c++/4.2.1/ios:50,
                 from /usr/include/c++/4.2.1/ostream:45,
                 from /usr/include/c++/4.2.1/iterator:70,
                 from boost/boost/next_prior.hpp:15,
                 from boost/boost/utility.hpp:17,
                 from boost/boost/python/instance_holder.hpp:10,
                 from boost/boost/python/object/pointer_holder.hpp:14,
                 from boost/boost/python/to_python_indirect.hpp:10,
                 from boost/boost/python/converter/arg_to_python.hpp:10,
                 from boost/boost/python/call.hpp:15,
                 from boost/boost/python/object_core.hpp:14,
                 from boost/boost/python/args.hpp:25,
                 from boost/boost/python.hpp:11,
                 from bend.hh:6,
                 from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:4611: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4629: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4635: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4641: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4665: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4677: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4683: error: function definition does not declare parameters

    "g++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Wstrict-aliasing=2 -ffast-math -fno-strict-aliasing -DBOOST_PYTHON_MAX_ARITY=30 -D_REENTRANT  -I"boost" -I"python/include/python2.7" -c -o "bin/darwin-4.2.1/release/threading-multi/Dbend.o" "bend.cc"

...failed darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o...
msg133949 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-04-18 01:20
This doesn't look like it has anything to to with the 'ctypes' library
component (http://docs.python.org/library/ctypes).  So I am removing
'ctypes' from the 'Components' selection.  Please correct me if I am
wrong ...
msg134095 - (view) Author: Nasos Dousis (ndousis) Date: 2011-04-19 20:16
Meador et al,

Thanks for your attention to this issue.  Just to clarify, I can eliminate
the compile errors by prepending

#include "Python.h"

to any source file with

#include<boost/python.hpp>

Also, my code compiles in Linux, with and without the Python.h header:

$ uname -a
Linux nxx.xx.com 2.6.18-128.1.14.el5 #1 SMP Wed Jun 17 06:38:05 EDT 2009
x86_64 x86_64 x86_64 GNU/Linux

with the same flags as before:

cd $(BOOST); ./bootstrap.sh --prefix=$(CURDIR)/$(BOOST)
--with-python-root=$(CURDIR)/$(PYTHON_DIR) --with-python-version=2.7
--with-libraries=mpi

cd $(PYTHON_SRCDIR) ; ./configure --enable-shared
--prefix=$(CURDIR)/$(PYTHON_DIR)
cd $(PYTHON_SRCDIR) ; make -j $(NUMPROC) && make install

Thanks again,
Nasos
msg135414 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2011-05-07 08:13
The problem is that pyport tries to replace the ctypes definion of the isascii(ch) and related functions by a replacement that works better with UTF-8 on FreeBSD and OSX. That replacement is incompatible with the localfwd.h header.

The attached patch is a very crude hack that should fix the issue for this. This assumes that Python itself won't be compiled with C++ though, and might cause problems in C++ programs that use locale.h instead of localfwd.h.

As such I won't apply the patch just yet, the patch needs review from someone that's more experienced w.r.t. C++.
msg140455 - (view) Author: Nasos Dousis (ndousis) Date: 2011-07-15 18:48
Ronald,

Thanks much for your help and insight.  I tried the patch but unfortunately it didn't work for me-- this might be because I have Python extensions written in C++.

Any other suggestions?  Should I wait for the next version of boost.python?

Thanks and regards,
Nasos
msg140665 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2011-07-19 13:31
Waiting for a new version of boost probably won't help: this is an incompatibility between a redefinition of ctypes macros in pyport.h and definitions in the C++ header <locale>.

I barely use C++ at this time and don't know how to tweak the headers to avoid this problem. It would probably have been better to not redefine the ctype.h definitions, we should have introduced Py_isascii etc. That requires some invasive changes to the Python codebase though, and I'm not sure if that is worth the trouble.

Could someone attach a sample project that suffers from this issue (with instructions on how to build it)? That would make it easier to debug the problem.

(Removing python 2.6 because that version receives only security patches at this time, adding 3.2 and 3.3 because those use the same code in pyport.h and hence should also be affected by this issue)
msg140718 - (view) Author: Nasos Dousis (ndousis) Date: 2011-07-20 03:47
Ronald, thanks again.  I stripped out as much as I could from the
original code and generated the attached project.  In doing so, I
narrowed the problem down to boost/weak_ptr.hpp.

The sample project consists of:

test-boost-python/Makefile
test-boost-python/src
test-boost-python/src/Jamroot
test-boost-python/src/test
test-boost-python/src/test/Jamfile
test-boost-python/src/test/TestBoostPython.cc
test-boost-python/src/test/TestBoostPython.hh
test-boost-python/tools/Makefile

Before building the project (just type "make" in test-boost-python),
you'll need to download Boost and save it to
test-boost-python/tools/boost/boost.tar.bz2; also, the tools/Makefile
automatically downloads Python 2.7.2, but you can try Python 3.2+ in
its place.

Thanks and kind regards,
Nasos
msg140719 - (view) Author: Nasos Dousis (ndousis) Date: 2011-07-20 03:49
With attachment--
msg152166 - (view) Author: Chen Huang (Chen.Huang) Date: 2012-01-28 16:00
I am experiencing the same issue. Googled for a bit but couldn't find much with the error, then changed the search context, come across to this thread. 


In file included from /opt/local/include/boost/date_time/gregorian/parsers.hpp:13,
                 from /opt/local/include/boost/date_time/gregorian/gregorian.hpp:34,
                 from /opt/local/include/boost/date_time/posix_time/time_formatters.hpp:12,
                 from /opt/local/include/boost/date_time/posix_time/posix_time.hpp:24,
                 from /opt/local/include/boost/date_time/local_time/local_time.hpp:11,
                 from /opt/local/include/boost/date_time.hpp:15,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:11,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/opt/local/include/boost/date_time/date_parsing.hpp:50:33: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:13:21: error: Devices.h: No such file or directory
In file included from /opt/local/include/boost/date_time/gregorian/parsers.hpp:13,
                 from /opt/local/include/boost/date_time/gregorian/gregorian.hpp:34,
                 from /opt/local/include/boost/date_time/posix_time/time_formatters.hpp:12,
                 from /opt/local/include/boost/date_time/posix_time/posix_time.hpp:24,
                 from /opt/local/include/boost/date_time/local_time/local_time.hpp:11,
                 from /opt/local/include/boost/date_time.hpp:15,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:11,
                 from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/opt/local/include/boost/date_time/date_parsing.hpp: In function ‘std::string boost::date_time::convert_to_lower(std::string)’:
/opt/local/include/boost/date_time/date_parsing.hpp:50: error: cannot resolve overloaded function ‘tolower’ based on conversion to type ‘char’
In file included from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
msg152169 - (view) Author: Chen Huang (Chen.Huang) Date: 2012-01-28 16:40
if I put "#include <cctype>" before boost/date_time, it solves my problem.

And by the way, i can't seem to get to the point where boost/week_ptr.hpp causes the error. I tried to compile Nasos's upload test-boost-python. That codes works on my machine.
msg152201 - (view) Author: Chen Huang (Chen.Huang) Date: 2012-01-28 23:34
Nevermind, putting #include <cctype> doesn't help. I have to put <python.h> on top to get away this.
msg152329 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2012-01-30 16:49
The only real fix I found is to introduce "Py_" prefixed versions of all definitions in ctypes.h that are used in Python (that is Py_isalnum) and use that throughout the python source tree.

That's a pretty invasive patch though and would probably be off-limit for 2.7 and 3.2.
msg152340 - (view) Author: Bert JW Regeer (X-Istence) * Date: 2012-01-30 17:52
In my first comment on this bug post I posted what project has issues with this, Botan with Boost.Python on FreeBSD and Mac OS X.

If required I will post how to reproduce this error using that project. If you would prefer a simplified test case, unfortunately I don't have the time to dig into that and find one.

One way to solve it is by including the Python header before any other file, however this is not an option for most 3rd party software that never sees this bug because those functions are not redefined for Linux...
msg152660 - (view) Author: Nasos Dousis (ndousis) Date: 2012-02-05 04:28
Per Chen Huang's comment, I retested my test case code using several versions of boost and Python 2.7:

boost 1.45: build fails, errors as reported.
boost 1.46: success.
boost 1.47: success.

I've attached a revised test case that automatically downloads and builds boost 1.45.
msg163783 - (view) Author: Barry Alan Scott (barry-scott) Date: 2012-06-24 12:44
I'm happy to review patches or create them for you.

I see a related set of errors on Mac OS X that are down to issues in bytes_methods.py that mirror the pyport.h issues.

In C++ isspace and friends are functions not macros.
You cannot go around defining them.

Here are the lines in python2.7 that cause compilation errors for me.

/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/bytes_methods.h:#undef isspace
/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/bytes_methods.h:#define isspace(c) undefined_isspace(c)
msg192420 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-07-06 12:03
The bytes_methods.h issue is not present in the default branch, but is still present in the 2.7 branch. I'm not sure why those defines are there, I'm adding Gregory P. Smith to the nosy list because he added the defines and might remember why they were added and if it would be save to remove them (from bytes_methods.h).

That is defines like this:

#undef islower
#define islower(c) undefined_islower(c)


I don't know what to do about the defines in pyport.h, one workaround is disable that entire block for C++ compiler, as in the attached patches (the later one is the same as the first one, but adds a comment that explains why the code is disabled for C++)
msg192449 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2013-07-06 15:08
I was merely refactoring for PEP3137, the original version of all that code prior to 2.7 was in Objects/stringobject.c and long predates me.
msg192450 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-07-06 15:11
I've some more digging to do then, although I expect that nobody knows anymore why the #define's are there. It should be safe to remove them by now, and they are not present in the default branch.
msg262828 - (view) Author: Dimitry Andric (dim) * Date: 2016-04-03 15:41
I am a FreeBSD committer, and I recently ran into this issue too, since I am working on an update of libc++ in the FreeBSD base system.  As part of this work, we attempt to recompile all ports with the proposed change (see [1]).  During such a recompile, we get many errors in ports that include pyport.h in C++ mode, similar to (see [2]):

In file included from scipy/interpolate/src/_interpolate.cpp:4:
In file included from scipy/interpolate/src/interpolate.h:3:
In file included from /usr/include/c++/v1/iostream:38:
In file included from /usr/include/c++/v1/ios:216:
/usr/include/c++/v1/__locale:468:15: error: C++ requires a type specifier for all declarations
    char_type toupper(char_type __c) const
              ^
/usr/local/include/python2.7/pyport.h:731:29: note: expanded from macro 'toupper'
#define toupper(c) towupper(btowc(c))
                            ^

I think Ronald's proposed workaround is fine, since messing with ctype macros is risky in C++ mode.  E.g. the libc++ <ctype.h> header explicitly undefines all ctype macros, and replaces them with inline functions.

Additionally, the original issue mentioned in changeset 32950 [3] was fixed in October 2007 for FreeBSD 8.x [4], and subsequently the fix was merged back to FreeBSD 6.x and 7.x [5].

I'm adding an additional diff that corrects the __FreeBSD_version number checks.  This will also be submitted to the FreeBSD ports bug tracker.

[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=208158
[2] http://package18.nyi.freebsd.org/data/headamd64PR208158-default/2016-03-22_18h30m05s/logs/errors/py27-scipy-0.16.1.log
[3] https://hg.python.org/cpython/rev/adfe7d39a049
[4] https://svnweb.freebsd.org/base?view=revision&revision=172619
[5] https://svnweb.freebsd.org/base?view=revision&revision=172929
msg272199 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-08-08 22:17
What do you think, Ned?
msg272234 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2016-08-09 11:05
@Brett / Ned I'm happy to carry the proposed patch in the FreeBSD ports/packages until the next releases if that helps your confidence levels.
msg272727 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-08-15 07:24
New changeset 2f857ac9c7af by Ned Deily in branch '3.5':
Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X.
https://hg.python.org/cpython/rev/2f857ac9c7af

New changeset 27a99a722828 by Ned Deily in branch '3.5':
Issue #10910: Update FreedBSD version checks for the ctype UTF-8 workaround.
https://hg.python.org/cpython/rev/27a99a722828

New changeset 5ca8790f1161 by Ned Deily in branch 'default':
Issue #10910: merge from 3.5
https://hg.python.org/cpython/rev/5ca8790f1161

New changeset e0ec3471cb09 by Ned Deily in branch '2.7':
Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X.
https://hg.python.org/cpython/rev/e0ec3471cb09

New changeset 12a70477db03 by Ned Deily in branch '2.7':
Issue #10910: Update FreedBSD version checks for the ctype UTF-8 workaround.
https://hg.python.org/cpython/rev/12a70477db03
msg272729 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-08-15 07:35
I have no special insight into this one but, since there seems to be general agreement here that this makes things better for those using C++ on OS X and FreeBSD, I guess it's time to try it.  Pushed for release in 2.7.13, 3.5.3, and 3.6.0.

Thanks for the version update, Dimitry.  BTW, it would be good if you signed the PSF Contributor Agreement (https://www.python.org/psf/contrib/contrib-form/).
History
Date User Action Args
2016-08-15 07:35:14ned.deilysetstatus: open -> closed
priority: release blocker ->
versions: - Python 3.2, Python 3.3, Python 3.4
messages: + msg272729

resolution: fixed
stage: patch review -> resolved
2016-08-15 07:24:39python-devsetnosy: + python-dev
messages: + msg272727
2016-08-09 11:05:53koobssetmessages: + msg272234
versions: + Python 3.5, Python 3.6
2016-08-08 22:17:43brett.cannonsetpriority: normal -> release blocker

nosy: + ned.deily, larry, brett.cannon, benjamin.peterson, georg.brandl
messages: + msg272199

assignee: ronaldoussoren -> ned.deily
2016-07-28 18:41:39mihaicsetnosy: + mihaic
2016-04-03 15:41:48dimsetfiles: + issue10910-fix-versionchecks-1.diff
nosy: + dim
messages: + msg262828

2015-02-02 13:04:06koobssetnosy: + koobs
2013-07-06 15:11:24ronaldoussorensetmessages: + msg192450
2013-07-06 15:08:43gregory.p.smithsetmessages: + msg192449
2013-07-06 12:03:03ronaldoussorensetfiles: + issue10910-workaround.txt
versions: + Python 3.4
nosy: + gregory.p.smith

messages: + msg192420
2012-06-24 12:44:58barry-scottsetnosy: + barry-scott
messages: + msg163783
2012-02-05 04:28:15ndousissetfiles: + test-boost-python.revised.tar.gz

messages: + msg152660
2012-01-30 17:52:17X-Istencesetmessages: + msg152340
2012-01-30 16:49:13ronaldoussorensetmessages: + msg152329
2012-01-28 23:34:31Chen.Huangsetmessages: + msg152201
2012-01-28 16:40:25Chen.Huangsetmessages: + msg152169
2012-01-28 16:00:35Chen.Huangsetnosy: + Chen.Huang
messages: + msg152166
2011-07-20 03:49:00ndousissetfiles: + test-boost-python.tar.gz

messages: + msg140719
2011-07-20 03:47:57ndousissetmessages: + msg140718
2011-07-19 13:31:20ronaldoussorensetmessages: + msg140665
versions: + Python 3.2, Python 3.3, - Python 2.6
2011-07-15 18:48:36ndousissetmessages: + msg140455
2011-05-07 08:13:12ronaldoussorensetkeywords: + patch, needs review
files: + issue10910.txt
messages: + msg135414

stage: patch review
2011-05-07 08:00:36ronaldoussorensetfiles: - unnamed
2011-04-19 20:16:19ndousissetfiles: + unnamed

messages: + msg134095
2011-04-18 02:55:14meador.ingesetcomponents: + macOS
2011-04-18 01:20:46meador.ingesetnosy: + meador.inge
messages: + msg133949
components: - Extension Modules, Library (Lib), macOS, Unicode, ctypes
2011-03-09 23:32:31ndousissetnosy: + ndousis
messages: + msg130477
2011-01-14 21:21:37X-Istencecreate