Issue22846
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2014-11-11 08:29 by mark, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Messages (10) | |||
---|---|---|---|
msg231008 - (view) | Author: Mark Summerfield (mark) * | Date: 2014-11-11 08:29 | |
When I try to build APSW (http://rogerbinns.github.io/apsw/index.html) with Python 3.3 or 3.4 on Debian stable 64-bit I get the error output shown below. I dug into the source and it seems that the problem is that distutils/ccompiler.py's gen_preprocess_options() functions expects to get a sequence of macros and each macro *must* be a 1 or 2 item tuple. But for some reason during the APSW build it gets a 2 item list which it then chokes on. Now, the code really does need a tuple because in some cases it uses Python's % print formatting option as in "-D%s=%s" % macro -- and this won't work if macro is a list. I solved this problem for me by adding two lines, shown here in context: pp_opts = [] for macro in macros: if isinstance(macro, list): # NEW macro = tuple(macro) # NEW I don't know how safe or wise a fix this is, but it did work for me for a locally built (from www.python.org tarball) 3.3 and 3.4. $ /home/mark/opt/python34/bin/python3 setup.py fetch --all build --enable-all-extensions install running fetch Getting download page to work out current SQLite version Fetching https://sqlite.org/download.html Version is 3.8.7.1 Getting the SQLite amalgamation Fetching https://sqlite.org/2014/sqlite-autoconf-3080701.tar.gz Length: 1998389 SHA1: 5601be1263842209d7c5dbf6128f1cc0b6bbe2e5 MD5: 8ee4541ebb3e5739e7ef5e9046e30063 Checksums verified Running configure to work out SQLite compilation flags setup.py:53: DeprecationWarning: 'U' mode is deprecated f=open(name, mode) running build running build_ext SQLite: Using amalgamation /home/mark/zip/apsw-3.8.7.1-r1/sqlite3/sqlite3.c setup.py:624: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'> for part in shlex.split(os.popen("icu-config --cppflags", "r").read()): setup.py:637: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'> for part in shlex.split(os.popen("icu-config --ldflags", "r").read()): ICU: Added includes, flags and libraries from icu-config building 'apsw' extension Traceback (most recent call last): File "setup.py", line 862, in <module> 'win64hackvars': win64hackvars} File "/home/mark/opt/python34/lib/python3.4/distutils/core.py", line 148, in setup dist.run_commands() File "/home/mark/opt/python34/lib/python3.4/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/home/mark/opt/python34/lib/python3.4/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/home/mark/opt/python34/lib/python3.4/distutils/command/build.py", line 126, in run self.run_command(cmd_name) File "/home/mark/opt/python34/lib/python3.4/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/home/mark/opt/python34/lib/python3.4/distutils/dist.py", line 974, in run_command cmd_obj.run() File "setup.py", line 661, in run v=beparent.run(self) File "/home/mark/opt/python34/lib/python3.4/distutils/command/build_ext.py", line 339, in run self.build_extensions() File "/home/mark/opt/python34/lib/python3.4/distutils/command/build_ext.py", line 448, in build_extensions self.build_extension(ext) File "/home/mark/opt/python34/lib/python3.4/distutils/command/build_ext.py", line 503, in build_extension depends=ext.depends) File "/home/mark/opt/python34/lib/python3.4/distutils/ccompiler.py", line 566, in compile depends, extra_postargs) File "/home/mark/opt/python34/lib/python3.4/distutils/ccompiler.py", line 341, in _setup_compile pp_opts = gen_preprocess_options(macros, incdirs) File "/home/mark/opt/python34/lib/python3.4/distutils/ccompiler.py", line 1061, in gen_preprocess_options % macro) TypeError: bad macro definition '['_FORTIFY_SOURCE', '2']': each element of 'macros' list must be a 1- or 2-tuple |
|||
msg231011 - (view) | Author: Ned Deily (ned.deily) * | Date: 2014-11-11 09:15 | |
Have you reported this problem to the author of apsw? It seems like figuring out why apsw is apparently creating an incorrect setup.py configuration should be a first step before suggesting a change to Distutils. You might want to supply the values from your system for: icu-config --cppflags icu-config --ldflags FWIW, with a current Debian testing, I get the following failure with either Python 3.4 or 2.7: running build_ext SQLite: Using amalgamation /tmp/e/apsw-3.8.7.1-r1/sqlite3/sqlite3.c ### icu-config: Can't find /usr/lib/i386-linux-gnu/libicuuc.so - ICU prefix is wrong. ### Try the --prefix= option ### or --detect-prefix ### (If you want to disable this check, use the --noverify option) ### icu-config: Exitting. ### icu-config: Can't find /usr/lib/i386-linux-gnu/libicuuc.so - ICU prefix is wrong. ### Try the --prefix= option ### or --detect-prefix ### (If you want to disable this check, use the --noverify option) ### icu-config: Exitting. ICU: Unable to determine includes/libraries for ICU using icu-config ICU: You will need to manually edit setup.py or setup.cfg to set them So there may be issues with icu-config on Debian. (Also, Python 3.3 is now in security-fix only mode.) |
|||
msg231012 - (view) | Author: Ned Deily (ned.deily) * | Date: 2014-11-11 09:25 | |
Another data point: apsw appears to build OK on OS X with a MacPorts-supplied icu, including icu-config. |
|||
msg231013 - (view) | Author: Mark Summerfield (mark) * | Date: 2014-11-11 09:35 | |
The first person I asked was the author of APSW (Roger Binns). He told me: "The ultimate cause of that is some interaction with the compilation environment. Some sort of CFLAGS is ultimately ending up in some Python code like above when it should be [ ('_FORTIFY_SOURCE', '2') ]. Note this is not part of the APSW source - it is something external." "I have seen it before when using the Ubuntu PPA build service. When building locally everything was fine, but the build service injected _FORTIFY_SOURCE like above and got it wrong. I presume your version of Debian is doing something similar. Sadly I have no idea how to fix it." So clearly he believes it is not a problem with his setup.py file. Also, it strikes me as a bit unpythonic that a function should demand a specific type (i.e., tuple) especially when this is just for the convenience of being able to use % formatting. I'm not asking or expecting you to add my change to distutils; but at least now if someone encounters the same problem, they will have a potential fix. |
|||
msg231014 - (view) | Author: Mark Summerfield (mark) * | Date: 2014-11-11 09:36 | |
Here are the flags you asked for: $ icu-config --cppflags -D_FORTIFY_SOURCE=2 -D_REENTRANT -I/usr/include $ icu-config --ldflags -Wl,-z,relro -ldl -lm -L/usr/lib/x86_64-linux-gnu -licui18n -licuuc -licudata -ldl -lm |
|||
msg231016 - (view) | Author: Marc-Andre Lemburg (lemburg) * | Date: 2014-11-11 09:57 | |
Shouldn't this be fixed in the APSW setup.py ? The patch is you are proposing looks harmless, but it can also mask programming errors in setup.py. |
|||
msg231017 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2014-11-11 10:07 | |
> The ultimate cause of that is some interaction with the compilation > environment. Some sort of CFLAGS is ultimately ending up in some > Python code like above when it should be [ ('_FORTIFY_SOURCE', '2') ]. > Note this is not part of the APSW source - it is something external. That's as unhelpful as a bug report can get. If ICU doesn't provide the right types, the setup.py should fix them up. (assuming ICU provides the preprocessor flags here - I haven't check) |
|||
msg231018 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2014-11-11 10:07 | |
I suggest closing. |
|||
msg231019 - (view) | Author: Ned Deily (ned.deily) * | Date: 2014-11-11 10:10 | |
I was able to reproduce the behavior you saw with an older Debian system. The following patch to the apsw setup.py file seems to fix the problem: --- apsw-3.8.7.1-r1/setup.py 2014-11-04 19:23:36.000000000 -0800 +++ apsw-3.8.7.1-r1_PATCHED/setup.py 2014-11-11 02:01:16.000000000 -0800 @@ -628,7 +628,7 @@ elif part.startswith("-D"): part=part[2:] if '=' in part: - part=part.split('=', 1) + part=tuple(part.split('=', 1)) else: part=(part, '1') ext.define_macros.append(part) Also, requiring a tuple is the documented behavior of Distutils: https://docs.python.org/2/distutils/apiref.html#distutils.core.Extension I agree that the issue should be closed and am closing it. |
|||
msg231020 - (view) | Author: Mark Summerfield (mark) * | Date: 2014-11-11 10:18 | |
I've notified APSW's author and I'm sure he'll fix it. Thanks! |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:10 | admin | set | github: 67035 |
2014-11-11 10:18:31 | mark | set | messages: + msg231020 |
2014-11-11 10:10:12 | ned.deily | set | status: pending -> closed messages: + msg231019 stage: resolved |
2014-11-11 10:07:38 | pitrou | set | status: open -> pending nosy: + ncoghlan messages: + msg231018 resolution: not a bug |
2014-11-11 10:07:04 | pitrou | set | nosy:
+ pitrou messages: + msg231017 |
2014-11-11 09:57:28 | lemburg | set | nosy:
+ lemburg messages: + msg231016 |
2014-11-11 09:36:47 | mark | set | messages: + msg231014 |
2014-11-11 09:35:09 | mark | set | messages: + msg231013 |
2014-11-11 09:25:12 | ned.deily | set | messages: + msg231012 |
2014-11-11 09:15:53 | ned.deily | set | nosy:
+ ned.deily messages: + msg231011 versions: - Python 3.3 |
2014-11-11 08:29:28 | mark | create |