classification
Title: Python 2.7: math module fails to build on Solaris 9
Type: compile error Stage:
Components: Extension Modules Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mark.dickinson Nosy List: CoryZ, DNS, Doug.Shea, Jerzy.Kozera, Matt.Selsky, mark.dickinson, mhenriq, python-dev, skrah, srmadsen, trent
Priority: normal Keywords: patch

Created on 2010-09-02 14:35 by mhenriq, last changed 2012-11-17 20:21 by mark.dickinson. This issue is now closed.

Files
File name Uploaded Description Edit
pyconfig.h Doug.Shea, 2010-11-22 16:34 pyconfig.h from Solaris 9 ./configure
Python-2.7.patch srmadsen, 2011-01-10 22:25 Python-2.7-solaris-9-math-module-fix
issue9742.patch mark.dickinson, 2012-10-31 22:19 review
Messages (33)
msg115379 - (view) Author: Misael Henriquez (mhenriq) Date: 2010-09-02 14:35
building 'math' extension$
gcc -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/python/Python-2.7/Include -I/usr/local/src/python/Python-2.7 -c /usr/local/src/python/Python-2.7/Modules/mathmodule.c -o build/temp.solaris-2.9-sun4u-2.7/usr/local/src/python/Python-2.7/Modules/mathmodule.o$
gcc -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/python/Python-2.7/Include -I/usr/local/src/python/Python-2.7 -c /usr/local/src/python/Python-2.7/Modules/_math.c -o build/temp.solaris-2.9-sun4u-2.7/usr/local/src/python/Python-2.7/Modules/_math.o$
gcc -shared build/temp.solaris-2.9-sun4u-2.7/usr/local/src/python/Python-2.7/Modules/mathmodule.obuild/temp.solaris-2.9-sun4u-2.7/usr/local/src/python/Python-2.7/Modules/_math.o -L/usr/local/lib -lm -o build/lib.solaris-2.9-sun4u-2.7/math.so$
*** WARNING: renaming "math" since importing it failed: ld.so.1: python: fatal: relocation error: file build/lib.solaris-2.9-sun4u-2.7/math.so: symbol round: referenced symbol not found$
msg115391 - (view) Author: Misael Henriquez (mhenriq) Date: 2010-09-02 16:01
The problem appears to be the lack of a round function in Solaris 9.  This compiles find in Solaris 10, but I need it in Solaris 9.

configure:11989: checking for round
configure:11989: gcc -o conftest -g -O2   conftest.c -lresolv -lsocket -lnsl -lrt -ldl  -lpthread -lm >&5
/var/tmp//ccWVF7bf.o: In function `main':
/usr/local/src/python/Python-2.7/conftest.c:287: undefined reference to `round'
msg115398 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-09-02 19:40
Hmm.  There should be a configure-time test for 'round'.  Could you look for a line something like:

  checking for round... yes

in the configure output and tell me whether you've got a 'yes' or a 'no' there?
msg115399 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-09-02 19:48
Ah.  Reading your second message more closely, you're presumably getting a 'no' there.  Could you double check that 'HAVE_ROUND' is not defined in pyconfig.h?

In that case, what's supposed to happen is that there's a substitute 'round' function defined in Python/pymath.c that gets used.  That's clearly working okay in the Python core, since it looks like your build doesn't fail until you get to the module build stage, and 'round' is used in Objects/floatobject.c in the core.

But I'm guessing that for some reason your Python executable isn't exporting that 'round' symbol that's defined in pymath.c.  I'm not sure why that would be.
msg115400 - (view) Author: Misael Henriquez (mhenriq) Date: 2010-09-02 19:50
The check for round failed during configure.  I included a bit of that in my second note above.  Here's a bit more of the output:

collect2: ld returned 1 exit status
configure:11989: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define _GNU_SOURCE 1
| #define _NETBSD_SOURCE 1
<snip>
| /* Override any GCC internal prototype to avoid an error.
|    Use char because int might match the return type of a GCC
|    builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char round ();
| /* The GNU C library defines this for functions which it implements
|     to always fail with ENOSYS.  Some functions are actually named
|     something starting with __ and the normal name is an alias.  */
| #if defined __stub_round || defined __stub___round
| choke me
| #endif
|
| int
| main ()
| {
| return round ();
|   ;
|   return 0;
| }
configure:11989: result: no
msg115401 - (view) Author: Misael Henriquez (mhenriq) Date: 2010-09-02 19:55
Thanks for your attention to this issue.  It sounds like you're onto something regarding the stand-in function not getting exported.  I'm not sure what to check next, though.

From pyconfig.h:

/* Define to 1 if you have the `round' function. */
/* #undef HAVE_ROUND */

From Python/pymath.c:

#ifndef HAVE_ROUND
double
round(double x)
{
    double absx, y;
    absx = fabs(x);
    y = floor(absx);
    if (absx - y >= 0.5)
    y += 1.0;
    return copysign(y, x);
}
#endif /* HAVE_ROUND */
msg121573 - (view) Author: Doug Shea (Doug.Shea) Date: 2010-11-19 21:42
Is there perhaps a work-around we could use to get this to compile and have a math module? Force it to export that 'round' symbol in the core, perhaps?
msg121599 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-11-20 09:39
> Force it to export that 'round' symbol in the core, perhaps?

Sure.  That might involve first understanding why it's not being exported.  That's where I'm a bit stuck, especially without a Solaris system to test on.  If someone can figure out why the symbol isn't being exported in the first place, that would be helpful.
msg122135 - (view) Author: Doug Shea (Doug.Shea) Date: 2010-11-22 15:24
I have some knowledge of these things, so I'll try to find out what's going on, but I could also upload output and/or debug files here for you to examine, if that helps. If you give me the files you'd like to see from my build, or the commands you'd like the output of, I'd be happy to oblige. Thanks!
msg122139 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-11-22 15:53
Doug:  thanks.  I think I may just be being stupid here. One thing you might try is replacing the line

  extern double round(double);

in Include/pymath.h, with:

  PyAPI_FUNC(double) round(double);

and see if that helps.
msg122140 - (view) Author: Doug Shea (Doug.Shea) Date: 2010-11-22 16:12
I unpacked a fresh tarball, made this change, then did a ./configure and make as normal. Exact same error as originally reported. :(
msg122143 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-11-22 16:29
Hmm.  That's puzzling indeed.

I made a claim earlier that 'round' is already used in Objects/floatobject.c, but it occurs to me now that that's not true if PY_NO_SHORT_FLOAT_REPR is #defined.

Could you attach the pyconfig.h file produced by configure?
msg122145 - (view) Author: Doug Shea (Doug.Shea) Date: 2010-11-22 16:34
Certainly!
msg122154 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-11-22 18:55
Thanks.  I'm still stuck, though.  Since I'm pretty much at the wild guesses stage, here's one:

<wild idea> Perhaps the pymath.o object file isn't being included in the Python executable at all, because none of its functions are needed.  Now that doesn't seem to make sense, since the round function is needed, and used, in Objects/floatobject.c (that is, if PY_NO_SHORT_FLOAT_REPR is *not* defined;  I'm still not clear on whether that's the case or not---if you can get as far as launching an interpreter, then the result of sys.float_repr_style would be 'short' in that case).  But gcc (depending on the version, I guess) has builtin versions of some standard C library functions, including 'round'.  So if it were using the builtin then there would be no need to link to the pymath.o file, so it might be left out of the python executable.  But I'm not sure why the same doesn't happen when compiling the math module.

Neither am I sure how one might go about testing the above hypothesis.
msg122159 - (view) Author: Doug Shea (Doug.Shea) Date: 2010-11-22 19:29
> ./python
Python 2.7 (r27:82500, Nov 22 2010, 10:06:14)
[GCC 3.3.2] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.float_repr_style
short

So it appears, if I follow you, that PY_NO_SHORT_FLOAT_REPR is not defined.

Glancing through the Makefile, pymath.o is included in the PYTHON_OBJS variable, which is in turn included in the LIBRARY_OBJS variable, which is used in all the libpython* targets.

I'm going to try to turn on some debugging on the commands that build those targets, and see if anything jumps out at me. Maybe it'll be obvious that pymath.o is being left out for one reason or another that'll sync up with your "wild guess". :/
msg122169 - (view) Author: Doug Shea (Doug.Shea) Date: 2010-11-22 21:52
I don't think there's anything wrong with the setup we've been looking at so far, per se. The libpython2.7.a file produced has the 'round' function like it should:

> nm libpython2.7.a | grep round
[116] |      1360|     696|FUNC |GLOB |0    |2      |_Py_double_round
[218] |         0|       0|NOTY |GLOB |0    |UNDEF  |_Py_double_round
[10] |     15268|     236|FUNC |LOCL |0    |2      |builtin_round
[48] |      9912|     198|OBJT |LOCL |0    |4      |round_doc
[8] |         0|      96|FUNC |GLOB |0    |2      |round

However, the python executable itself, compiled against that archive, does *not* have it:

> nm python | grep round
[4805] |    244408|     696|FUNC |GLOB |0    |9      |_Py_double_round
[1735] |    640836|     236|FUNC |LOCL |0    |9      |builtin_round
[1770] |   1533576|     198|OBJT |LOCL |0    |16     |round_doc

So, for some reason, the gcc command that builds that python executable is leaving it out:

gcc -o python \
       Modules/python.o \
       libpython2.7.a -lresolv -lsocket -lnsl -lrt -ldl -lpthread -lm

Just an update. Going to try to debug that command and see if I can find out why.
msg122218 - (view) Author: Jerzy Kozera (Jerzy.Kozera) Date: 2010-11-23 15:06
Running

gcc -Wl,-R/usr/local/lib,-R/usr/lib  -o python Python/pymath.o Modules/python.o libpython2.7.a -lresolv -lsocket -lnsl -lrt -ldl  -lpthread   -lm

mv build/lib.solaris-2.8-sun4u-2.7/math_failed.so build/lib.solaris-2.8-sun4u-2.7/math.so

seems to have made math module import correctly and work:

bash-2.03$ ./python
Python 2.7 (r27:82500, Nov 23 2010, 14:49:30)
[GCC 3.4.6] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import math
>>> math.floor(2.4)
2.0


I suppose it's more a workaround than a solution, but hopefully it makes using math module possible and confirms the suggestion there might be something wrong with ar/gcc linking the .a file.
msg122229 - (view) Author: Doug Shea (Doug.Shea) Date: 2010-11-23 18:37
It's actually not quite a solution, either. Working your changes into the build process, I *do* get a math module... but it does *not* have a round function.

> python
Python 2.7 (r27, Nov 23 2010, 11:54:39)
[GCC 3.3.2] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import math;
>>> print math.round(2.5);
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'round'
>>> print math.floor(2.5);
2.0

A step in the right direction, though, I'd think. Thanks.
msg122234 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-11-23 19:05
> I *do* get a math module... but it does *not* have a round function.

Not a problem:  the math module isn't supposed to have a round function. :-)

The round function is used as part of the calculations that produce the gamma function.  So if the gamma function is working, then all's well:

>>> from math import gamma
>>> gamma(3.1)
2.197620278392477
msg125943 - (view) Author: Reid Madsen (srmadsen) Date: 2011-01-10 22:25
Python support,

This issue with not being able to build on Solaris 9 is easily fixed.  I have attached a patch with the fix for Python 2.7.

When linking with libpython-2.7.a, the linker will only extract modules that satisfy dependencies emanating from python.o.  There may be objects in the archive that are not needed to satisfy any of these dependencies and those WILL NOT be included in the executable.

The GNU linker supports two options that can be use to force the linker to include ALL objects in the archive.  Thus if you change the python link line from:


 $(BUILDPYTHON):  Modules/python.o $(LIBRARY) $(LDLIBRARY)
     $(LINKCC) $(LDFLAGS) $(LINKFORSHARED) -o $@ \
         Modules/python.o \
         $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)

to:

 $(BUILDPYTHON): Modules/python.o $(LIBRARY) $(LDLIBRARY)
     $(LINKCC) $(LDFLAGS) $(LINKFORSHARED) -o $@ \
         Modules/python.o \
         -Wl,--whole-archive $(BLDLIBRARY) -Wl,--no-whole-archive \
         $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)

Then the problem is resolved.

For compiler toolchains that do not support the --whole-library option, you can change the link to link with the individual .o files and not use the archive library at all.

Let me know if I can be of any more help.

Reid Madsen
msg134222 - (view) Author: David Schnur (DNS) Date: 2011-04-21 15:22
I encountered this problem when updating from 2.6.4 to 2.7.1 on Solaris 8, which also doesn't have 'round'.  I tried srmadsen's --whole-archive option, but it wasn't recognized by the (somewhat ancient) tools on that machine.

I solved the problem by just editing the $(BUILDPYTHON) target to include Python/pymath.o, as Jerzy Kozera did.

Should configure perhaps take care of this when it detects no round?  It seems like this has slipped in simply because very few people still use systems without round.
msg161774 - (view) Author: Cory Zito (CoryZ) Date: 2012-05-28 12:42
I hit this issue today with 2.7.3 (Solaris 9, Sun Studio 12).  

Adding Python/pymath.o to $(BUILDPYTHON) worked for me.  

Should a check in configure of is Solaris and no round should just go ahead and add this to $(BUILDPYTHON)?
msg161785 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-05-28 14:07
This is just a suggestion, but the easiest way of getting good support
for a non-mainstream platform is to provide a build slave:

http://www.python.org/dev/buildbot/


Setting up a build slave takes 20 min:

http://bugs.python.org/file24399/buildslave_install.txt


On Unix running a build slave is practically zero-maintenance.


The general problem with exotic platforms is that developers aren't
able to reproduce the issue and are reluctant to commit a fix which
they can't test.
msg174032 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-10-28 11:51
Unassigning, since I don't plan to work on this any time soon.
msg174033 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-10-28 12:08
FWIW, Solaris 9 has EOL status:

http://www.oracle.com/technetwork/server-storage/solaris10/overview/index-138972.html

I suspect we might as well close this issue, since the motivation to work
on it will be pretty low in general.
msg174036 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-10-28 12:17
Thanks, Stefan.  Closing.
msg174038 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-10-28 12:20
Snakebite's got a Solaris 9 SPARC instance up and running.  It's available via the s9 alias.

Regarding EOL, this indicates October 2014 for 9: http://en.wikipedia.org/wiki/Solaris_(operating_system)#Version_history

If it's as simple as suggested, I don't mind taking ownership of this.  I'm going to run into it anyway as soon as I set up a Solaris 9 slave.
msg174044 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-10-28 12:52
Right, they distinguish between EOL and "Extended Support Phase":

http://www.oracle.com/technetwork/server-storage/solaris10/overview/general-137378.html
msg174374 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-10-31 21:47
Looking back at the discussion, I think I got this bit wrong:

"the round function is needed, and used, in Objects/floatobject.c (that is, if PY_NO_SHORT_FLOAT_REPR is *not* defined;"

In fact, the round function is used if PY_NO_SHORT_FLOAT_REPR *is* defined, and is unused otherwise.  So indeed nothing from pymath.o is used in the rest of the Python core.
msg174382 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-10-31 22:19
Reopening because I think I may have an easy solution.  Is there anyone who can reproduce the problem who could test the attached patch (against 2.7) to see if it solves the issue?
msg174417 - (view) Author: Matt Selsky (Matt.Selsky) Date: 2012-11-01 13:34
I tested this patch again python 2.7.3 on Solaris 9 and the math module now builds correctly.  Thanks!

Let me know if you need any output.
msg175804 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-11-17 20:19
New changeset ff300ccd119d by Mark Dickinson in branch '2.7':
Issue #9742: Sneaky fix for build failure on Solaris 9.
http://hg.python.org/cpython/rev/ff300ccd119d
msg175805 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-11-17 20:21
Matt:  thanks for testing.  Patch applied---this should be fixed now.

I'm a little bit nervous about this one, since it may introduce new failures on systems with a buggy round implementation (Linux / alpha, for example).  Still, we have tests for that, so I'll just watch the buildbots for a bit, and possibly revert this fix if it proves necessary.
History
Date User Action Args
2012-11-17 20:21:36mark.dickinsonsetstatus: open -> closed
resolution: fixed
messages: + msg175805
2012-11-17 20:19:00python-devsetnosy: + python-dev
messages: + msg175804
2012-11-01 13:34:51Matt.Selskysetmessages: + msg174417
2012-10-31 22:19:58mark.dickinsonsetstatus: closed -> open
files: + issue9742.patch
messages: + msg174382

assignee: mark.dickinson
resolution: out of date -> (no value)
2012-10-31 21:47:23mark.dickinsonsetmessages: + msg174374
2012-10-28 12:52:51skrahsetmessages: + msg174044
2012-10-28 12:20:39trentsetmessages: + msg174038
2012-10-28 12:17:54mark.dickinsonsetstatus: open -> closed
resolution: out of date
messages: + msg174036
2012-10-28 12:12:17trentsetnosy: + trent
2012-10-28 12:08:38skrahsetmessages: + msg174033
2012-10-28 11:51:09mark.dickinsonsetassignee: mark.dickinson -> (no value)
messages: + msg174032
2012-05-28 14:07:20skrahsetnosy: + skrah
messages: + msg161785
2012-05-28 12:42:14CoryZsetnosy: + CoryZ
messages: + msg161774
2011-04-21 15:22:48DNSsetnosy: + DNS
messages: + msg134222
2011-01-10 22:25:00srmadsensetfiles: + Python-2.7.patch

nosy: + srmadsen
messages: + msg125943

keywords: + patch
2010-12-29 09:20:24Matt.Selskysetnosy: + Matt.Selsky
2010-11-23 19:05:25mark.dickinsonsetmessages: + msg122234
2010-11-23 18:37:33Doug.Sheasetmessages: + msg122229
2010-11-23 15:06:37Jerzy.Kozerasetnosy: + Jerzy.Kozera
messages: + msg122218
2010-11-22 21:52:38Doug.Sheasetmessages: + msg122169
2010-11-22 19:29:20Doug.Sheasetmessages: + msg122159
2010-11-22 18:55:57mark.dickinsonsetmessages: + msg122154
2010-11-22 16:34:40Doug.Sheasetfiles: + pyconfig.h

messages: + msg122145
2010-11-22 16:29:41mark.dickinsonsetmessages: + msg122143
2010-11-22 16:12:09Doug.Sheasetmessages: + msg122140
2010-11-22 15:53:07mark.dickinsonsetmessages: + msg122139
2010-11-22 15:24:21Doug.Sheasetmessages: + msg122135
2010-11-20 09:39:34mark.dickinsonsetmessages: + msg121599
2010-11-19 21:42:21Doug.Sheasetnosy: + Doug.Shea
messages: + msg121573
2010-09-02 19:55:56mhenriqsetmessages: + msg115401
2010-09-02 19:50:17mhenriqsetmessages: + msg115400
2010-09-02 19:48:14mark.dickinsonsetmessages: + msg115399
2010-09-02 19:40:50mark.dickinsonsetassignee: mark.dickinson
messages: + msg115398
2010-09-02 17:53:13ned.deilysetnosy: + mark.dickinson
2010-09-02 16:01:15mhenriqsetmessages: + msg115391
2010-09-02 14:35:33mhenriqcreate