This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: python fails to build when configured with --with-system-libmpdec
Type: behavior Stage: resolved
Components: Build Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, doko, pitrou, skrah
Priority: normal Keywords:

Created on 2013-11-23 08:33 by doko, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (18)
msg203978 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-11-23 08:32
using the only mpdecimal release (2.3):

building '_decimal' extension
creating build/temp.linux-x86_64-3.3/scratch/packages/python/3.3/python3.3-3.Modules/_decimal
x86_64-linux-gnu-gcc -pthread -fPIC -D_FORTIFY_SOURCE=2 -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fprofile-generate -g -flto -fuse-linker-plugin -DCONFIG_64=1 -DASM=1 -I../Include -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/scratch/packages/python/3.3/python3.3-3.3.3/Include -I/scratch/packages/python/3.3/python3.3-3.3.3/build-static -c /scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c -o build/temp.linux-x86_64-3.3/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.o -Wextra -Wno-missing-field-initializers
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c: In function 'dectuple_as_str':
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:2522:47: error: expected ')' before 'PRI_mpd_ssize_t'
         n = snprintf(cp, MPD_EXPDIGITS+2, "%" PRI_mpd_ssize_t, exp);
                                               ^
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:2522:47: warning: spurious trailing '%' in format [-Wformat=]
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:2522:47: warning: spurious trailing '%' in format [-Wformat=]
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c: In function 'dec_str':
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:3075:5: warning: implicit declaration of function 'mpd_to_sci_size' [-Wimplicit-function-declaration]
     size = mpd_to_sci_size(&cp, MPD(dec), CtxCaps(context));
     ^
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c: In function 'dec_format':
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:3243:9: warning: implicit declaration of function 'mpd_validate_lconv' [-Wimplicit-function-declaration]
         if (mpd_validate_lconv(&spec) < 0) {
         ^
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c: In function 'dec_as_long':
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:3340:5: warning: passing argument 1 of 'mpd_qexport_u32' from incompatible pointer type [enabled by default]
     n = mpd_qexport_u32(&ob_digit, 0, PyLong_BASE, x, &status);
     ^
In file included from /scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:34:0:
/usr/include/x86_64-linux-gnu/mpdecimal.h:474:8: note: expected 'uint32_t *' but argument is of type 'digit **'
 size_t mpd_qexport_u32(uint32_t *rdata, size_t rlen, uint32_t base,
        ^
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c: In function 'dec_mpd_to_eng':
/scratch/packages/python/3.3/python3.3-3.Modules/_decimal/_decimal.c:4136:5: warning: implicit declaration of function 'mpd_to_eng_size' [-Wimplicit-function-declaration]
     size = mpd_to_eng_size(&s, MPD(self), CtxCaps(context));
     ^
msg203979 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-11-23 08:34
also, there is no VCS repository and no ML archive for mpdecimal publically available.
msg204242 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-11-24 17:42
Development of libmpdec has effectively happened on hg.python.org
since I included _decimal.  That's also one of the reasons why there
isn't any public VCS.  The mailing list isn't archived because I'm
using ezmlm and ezmlm's retrieval system caused excessive backscatter
of mail (IIRC).

There is also no bug-tracker for the simple reason that no libmpdec
bugs have been reported either here or on bytereef.org.


_decimal should only be built against the upcoming mpdecimal-2.4.
I'm going to name the shared library libmpdec.so.2, which will only
receive bugfixes, since the spec isn't going to change.
msg204243 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-11-24 17:44
Am 24.11.2013 18:42, schrieb Stefan Krah:
> _decimal should only be built against the upcoming mpdecimal-2.4.

is there a schedule for this version?
msg204260 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-11-24 19:23
I have prepared Python-3.3+ for use with libmpdec.so.2:

cbd78679080b
9d07b3eb34e3

Here is a prerelease for mpdecimal:

http://www.bytereef.org/software/mpdecimal/releases/mpdecimal-2.4-rc1.tar.gz

sha256sum:

528a61b3de5c1e553a012d083c4d909391510cc733c1a990131d312bbdf1a717

I think I've addressed all your comments you sent privately earlier.
The "uname -s" in configure.ac is only used for suncc and Darwin.
It should not affect cross building.

I still have to update the docs and a couple of READMEs for the final release.
The code in libmpdec is now the same as on hg.python.org and is highly unlikely
to change.

If you have time and have some final requests for the configury (config.guess,
install-sh?), let me know.  I prefer to run the final tests on the exact tarball
that is ultimately shipped.

The deal for Python-3.3 and Python-3.4 is:  For compatibility with decimal.py
only link against libmpdec.so.2.
msg204263 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-11-24 20:27
2.4~rc1:
 - configure.ac should call AC_CANONICAL_HOST, config,guess and
   config.sub should be included.
 - there are still symbols which exists only for 32/64 bit archs.
   intended?
   (arch=@64@)mpd_qsset_i64@Base 2.3
   (arch=@64@)mpd_qsset_u64@Base 2.3
   (arch=@64@)mpd_sset_i64@Base 2.3
   (arch=@64@)mpd_sset_u64@Base 2.3
msg204326 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-11-25 11:42
Matthias Klose <report@bugs.python.org> wrote:
> 2.4~rc1:
>  - configure.ac should call AC_CANONICAL_HOST, config,guess and
>    config.sub should be included.

Hmm. configure.ac doesn't use any of the variables set by that macro, and this
appears to work:

./configure --host=arm CC=arm-linux-gnueabi-gcc-4.7

>  - there are still symbols which exists only for 32/64 bit archs.
>    intended?
>    (arch=@64@)mpd_qsset_i64@Base 2.3
>    (arch=@64@)mpd_qsset_u64@Base 2.3
>    (arch=@64@)mpd_sset_i64@Base 2.3
>    (arch=@64@)mpd_sset_u64@Base 2.3

That's on purpose:  These are low level functions that are supposed to set
static decimals as fast as possible.  A static mpd_t has a guaranteed
minimum allocation of two words for the coefficient. The 32-bit version
needs three words for UINT64_MAX, so they aren't available.
msg204327 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-11-25 11:45
Am 25.11.2013 12:42, schrieb Stefan Krah:
> 
> Stefan Krah added the comment:
> 
> Matthias Klose <report@bugs.python.org> wrote:
>> 2.4~rc1:
>>  - configure.ac should call AC_CANONICAL_HOST, config,guess and
>>    config.sub should be included.
> 
> Hmm. configure.ac doesn't use any of the variables set by that macro, and this
> appears to work:
> 
> ./configure --host=arm CC=arm-linux-gnueabi-gcc-4.7

sure, but this should work without explicitly setting CC.
msg204344 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-11-25 13:14
Matthias Klose <report@bugs.python.org> wrote:
> > ./configure --host=arm CC=arm-linux-gnueabi-gcc-4.7
> 
> sure, but this should work without explicitly setting CC.

I can't get it to work without passing CC even with config.guess and
config.sub.  If you want it to work, could you submit a patch?  I've
set up a temporary repo:

hg clone http://bytereef.org:8000/ mpdecimal

Also, do you by any chance know a better solution for ldconfig (I'm
calling it unconditionally during install)?  I vaguely remember
that ldconfig does something different on OpenBSD.
msg204354 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-11-25 16:00
Apparently the build failed because only arm-linux-gnueabi-gcc-4.7 was installed
but not arm-linux-gnueabi-gcc.

But the cross build succeeds with the original 2.4-rc1, except that arm-linux-gnueabi-ar
isn't picked up:

$ ./configure --host=arm-linux-gnueabi
checking for arm-linux-gnueabi-gcc... arm-linux-gnueabi-gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether arm-linux-gnueabi-gcc accepts -g... yes
checking for arm-linux-gnueabi-gcc option to accept ISO C89... none needed
checking system as reported by uname -s... Linux
checking how to run the C preprocessor... arm-linux-gnueabi-gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for inttypes.h... (cached) yes
checking for stdint.h... (cached) yes
checking for size_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for __uint128_t... no
checking size of size_t... 4
checking size of __uint128_t... 0
checking for x64 gcc inline assembler... no
checking for x87 gcc inline assembler... no
checking for -O2... yes
checking for glibc _FORTIFY_SOURCE/memmove bug... undefined
configure: creating ./config.status
config.status: creating Makefile
config.status: creating libmpdec/Makefile
config.status: creating tests/Makefile
config.status: creating libmpdec/mpdecimal.h
config.status: creating config.h
$ make
cd libmpdec && make
make[1]: Entering directory `/home/stefan/tmp/mpdecimal-2.4-rc1/libmpdec'
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c basearith.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c context.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c constants.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c convolute.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c crt.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c mpdecimal.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c mpsignal.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c difradix2.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c fnt.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c fourstep.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c io.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c memory.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c numbertheory.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c sixstep.c
arm-linux-gnueabi-gcc -Wall -W -Wno-unknown-pragmas -DCONFIG_32 -DANSI -O2 -fpic  -c transpose.c
ar rc libmpdec.a basearith.o context.o constants.o convolute.o crt.o mpdecimal.o mpsignal.o difradix2.o fnt.o fourstep.o io.o memory.o numbertheory.o sixstep.o transpose.o
ranlib libmpdec.a
arm-linux-gnueabi-gcc   -shared -Wl,-soname,libmpdec.so.2 -o libmpdec.so.2.4.0 basearith.o context.o constants.o convolute.o crt.o mpdecimal.o mpsignal.o difradix2.o fnt.o fourstep.o io.o memory.o numbertheory.o sixstep.o transpose.o -lm
make[1]: Leaving directory `/home/stefan/tmp/mpdecimal-2.4-rc1/libmpdec'
msg204419 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-11-25 21:42
I've fixed cross-compiling and a couple of small issues in the
repo from msg20434.  Cross-compiling seems to work also without
config-guess, but I've included it anyway.
msg205915 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-11 19:26
Antoine, could you add --with-system-libmpdec to my FreeBSD and Fedora
bots? Sorry for bothering you with all these obscure options. :)
msg205924 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-12-11 20:35
> Antoine, could you add --with-system-libmpdec to my FreeBSD and Fedora
> bots? Sorry for bothering you with all these obscure options. :)

Should be done now. Can you check they work fine?
msg205932 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-11 21:58
Antoine Pitrou <report@bugs.python.org> wrote:
> Should be done now. Can you check they work fine?

Thank you! Works OK.
msg205933 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-11 22:09
Matthias, my target is to release mpdecimal-2.4 on January 9th 2014.
I saw that you started to package mpdecimal-2.3:

    http://packages.debian.org/pt/source/sid/misc/mpdecimal

I hope that it can be changed to 2.4:  Packaging 2.3 would create
some confusion, since one or two functions are not binary compatible
with 2.4 and of course 2.3 cannot be used for Python.

Starting from 2.4, the remainder of the 2.x.y series will stay
binary (and decimal.py) compatible.
msg207082 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-12-29 18:07
your current repo doesn't create and install the .so symlink, and thus won't be used for linking.

Also the sphinx docs are missing, which were included in 2.3.
msg207095 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-30 12:16
I wanted to force people to link explicitly with -l:libmpdec.so.2 in
order to avoid picking up a wrong version.  But indeed that won't work
since the GNU linker happily picks up the static lib with -lmpdec if
libmpdec.so is missing.

The docs are part of my website repo and are copied manually as
part of the release process.
msg207398 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-05 20:21
I’ve released mpdecimal-2.4.0:

http://www.bytereef.org/mpdecimal/download.html


sha256sum:
==========

996c73b63868699c4f86694671bd7ff3d1a8a88d8af2c22a5abe0c9a5845cf75  mpdecimal-2.4.0.tar.gz


On obscure architectures I would recommend to run `make check` (leaves asserts in!).


The fastest binaries are generated with:

./configure CFLAGS=-DNDEBUG
make profile


The -flto option appears to slow things down, -O3 instead of -O2
is the same or slightly worse in my experience.
History
Date User Action Args
2022-04-11 14:57:54adminsetgithub: 63931
2014-01-05 20:22:10skrahsetstatus: open -> closed
type: behavior
resolution: fixed
stage: resolved
2014-01-05 20:21:36skrahsetmessages: + msg207398
2013-12-30 12:16:32skrahsetmessages: + msg207095
2013-12-29 18:07:59dokosetmessages: + msg207082
2013-12-11 22:09:12skrahsetmessages: + msg205933
2013-12-11 21:58:59skrahsetmessages: + msg205932
2013-12-11 20:35:27pitrousetmessages: + msg205924
2013-12-11 19:26:25skrahsetnosy: + pitrou
messages: + msg205915
2013-11-25 21:42:04skrahsetmessages: + msg204419
2013-11-25 16:00:38skrahsetmessages: + msg204354
2013-11-25 13:14:08skrahsetmessages: + msg204344
2013-11-25 11:45:06dokosetmessages: + msg204327
2013-11-25 11:42:13skrahsetmessages: + msg204326
2013-11-24 20:27:08dokosetmessages: + msg204263
2013-11-24 19:54:03skrahsetnosy: + Arfrever
2013-11-24 19:23:40skrahsetmessages: + msg204260
2013-11-24 17:44:25dokosetmessages: + msg204243
2013-11-24 17:42:26skrahsetmessages: + msg204242
2013-11-23 08:34:03dokosetnosy: + skrah
messages: + msg203979
2013-11-23 08:33:00dokocreate