Issue460751
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2001-09-11 21:29 by giacometti, last changed 2022-04-10 16:04 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
with-shared_211.cdiff | giacometti, 2001-09-11 21:29 | patch against python 2.1.1 source tree | ||
python-share-build.cdiff | giacometti, 2001-09-17 17:36 | |||
shared.tar | giacometti, 2001-09-25 01:06 | diff file + comment file | ||
shared1.cdiff | giacometti, 2001-09-25 21:51 | |||
shared1.cdiff | giacometti, 2001-09-25 21:53 |
Messages (19) | |||
---|---|---|---|
msg37540 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-11 21:29 | |
As a follow up to patch 400938 (https://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=400938), here is an implementation of the --with-shared configure option for building python on Posix systems as a shared library. Building python as shared library is activated with the --with-shared option on ./configure The following points are drawn to particular attention: - A configbuild.py module is generated and placed in the machine-dependent directory. It records most Python/Makefile variables (CC, CFLAG, LDSHARED...), and make them available in Python. - The build has been tested on: Linux2: OK SunOS5: OK IRIX64: Build OK, crash on test (bus error) - Inference with distutils: I found it very difficult understanding the distutil architecture and finding the right points of change; I minimized my interaction there. Furthermore, the availability of the 'configbuild' module overlaps some of the distutils; but 'configbuild' is very easy to use ... - Unresolved symbols: I turned on the symbol checking flags whenever possible, on behalf the its better generating the error on build than at runtime. - configure.in: I only edited the configure Bourne script, and did not bother with configure.in (actually, I deleted it) and its autoconfig/m4 macro comparses. Frederic Giacometti |
|||
msg37541 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-09-12 02:03 | |
Logged In: YES user_id=6380 Haven't seen the patch yet, but I like the idea -- the command line option suggests that it shouldn't hurt platforms where it can't be done or has to be done differently. Alternative: we could also consider using libtool. I didn't like that much in the past, but maybe it would work. Dunno. Anybody know about it? |
|||
msg37542 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2001-09-12 07:18 | |
Logged In: YES user_id=21627 I recommend to rework this patch to get rid of configbuild. I cannot see what you achieve with the configbuild module that you couldn't do with distutils.sysconfig.get_config_vars (which is capable of parsing the Makefile directly). I also recommend to go through the patch chunk by chunk, asking yourself whether this chunk *really* is needed to achieve the desired functionality. E.g. I cannot see the relationship of the .cvsignore chunk; I also believe that removal of .purify is incorrect. Likewise, while I sympathize with the unresolved symbols change, I believe it is unrelated to the main objective of this patch (building Python shared). What is the rationale for not changing configure.in? As-is, this patch is useless: configure will be overwritten everytime you run autoconf. Please get a copy of autoconf on one of your systems, change configure.in, run autoconf, and then provide us with a patch that only changes configure.in: everybody reviewing this patch should be capable of running autoconf. configure is a generated file and should not be edited manually. What is the rationale for the introduction of the THREADOBJ variable? |
|||
msg37543 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2001-09-12 08:06 | |
Logged In: YES user_id=21627 As for libtool: I strongly advise not to use it. It is broken in too many ways to enumerate. |
|||
msg37544 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-12 18:12 | |
Logged In: YES user_id=93657 - I will remove the configbuild; since Martin pointed to me the existence of distutils.sysconfig.get_config_vars(). I did not know about this function... Thanks! On the fly, you'll also note the following potential problem with using get_config_vars() instead of configbuild: configbuild will report the actual values used for the build, whereas get_config_vars() will report erroneous values whenever the Makefile internal variables were overriden at build time (by either the environment, or by commandline assignment of the type 'make CC=gxx ...'); not to mention that configbuild does not need to find and parse the original Makefile (with all the potential problems associated). - The .purify line should be in the CVSROOT/cvsignore CVS admin file, not in .cvsignore in the root directory of the source distribution... - Autoconf is not installed on the commercial Unix platforms, it is useless for me to learn it, and I was better off directly editing ./configure. However, if others are familiar with the tool, it should be straightforward for them to retrocede the changes from configure to configure.in - I had to introduce THREADOBJ because you will note that, in the present implementation, Modules/thread.o appears multiple times in LIBOBJS, which causes warning at link time. Now, Modules/thread.o appears only once |
|||
msg37545 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-09-12 18:37 | |
Logged In: YES user_id=6380 > Autoconf is not installed on the commercial Unix > platforms, it is useless for me to learn it, and I was > better off directly editing ./configure. However, if others > are familiar with the tool, it should be straightforward for > them to retrocede the changes from configure to configure.in Sure. You'll just have to wait until we have time for that. You can make it easy for us to integrate your changes, or you can make it difficult. Your choice. |
|||
msg37546 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-12 23:06 | |
Logged In: YES user_id=93657 I am not willing to make anything difficult for anybody, but: What is the point of keeping using the autoconf thing when it is easier to maintain configure directly, than going through configure.in ? After configure has been generated once, it makes more sense to me to work directly on configure, and forget about configure.in. configure (pure Bourne shell) is a easier to understand, to troubleshoot, and to maintain that configure.in (a mixture of Bourne shell and autoconf m4 macros). I'll have a second look at the autoconf thing latter on - at present I hardly understand anything to configure.in, and got 'configure --with-shared' working with satisfaction across multiple platforms without undue difficulties. |
|||
msg37547 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-09-13 01:36 | |
Logged In: YES user_id=6380 > What is the point of keeping using the autoconf thing when > it is easier to maintain configure directly, than going > through configure.in ? But it's not. One line of configure.in often expands to dozens (occasionally hundreds) of lines of configure. autoconf has arcane knowledge that gets updated with new autoconf releases. You are entitled to your opinion, but in this case, it's wrong. Go fork your own version of Python if you don't want to learn autoconf. |
|||
msg37548 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-17 17:36 | |
Logged In: YES user_id=93657 I'm attaching an updated patch where, following Martin suggestions: - I removed the configbuild thing, and replaced it with distutils.sysconfig.get_conf_var calls - I manually cleaned up the patch from unrelated changes (.cvsignore, @ Makefile modifiers...) Remains the configure.in thing. If one looks at autoconf: - it takes care of plateform idiosyncracies, for platform which do not follow posix API specs; this was of use in previous generation Unix OS's, and these legacy systems do not evolve anymore. On present Unixes, and with regard to the features used for building the core python engine, nothing moves, and autoconf updates bring nothing. Doesn't the Python 1.5.2's configure still works as it did the first day it was generated ... ? - autoconf is of use for configuring particular packages and libraries (e.g.: for configuring and building python extensions, Tkinter/Tk/TCL...); unfortunately this never worked with python, and the task is now assumed by the distutils. - In the Python core build, autoconf is of no help in anything sensible: thread, shared library, debug build, compiler options... All these were programmed manually in Bourne; and for doing the later, autoconf is a complexifying factor on something already fairly complex - i.e. a handicap -. - not to say, autoconf is of no use on non-posix plateforms (windows...) - last, and the least: autoconf now requires that perl be installed... Frederic Giacometti |
|||
msg37549 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2001-09-18 17:05 | |
Logged In: YES user_id=21627 I've tried testing your patch, but I get numerous failed chunks when applying it against the current CVS. What version does this patch apply to? To simplify the integration, it would be greatly appreciated if you could use the current source base. From reviewing the patch, I still have a number of concerns: - it appears that you change the mechanism by which buildno is incremented. Why? - There appear to extra white space changes (e.g. in the distclean chunk); those are naturally unrelated to the patch - please remove them - it appears that configuration knowledge for some platforms is lost when applying this patch, e.g. the dguxR4 special-casing of passing -pic is gone. Please try to integrate any such knowledge faithfully into your patch, instead of restricting its applicability to just the platforms where you've tested it on. - There is something going on with dropping -lsocket from the patch. What is that, and why does it happen? Instead, it appears that you put 'socket' and 'nsl' into setup.py. According to configure.in, this is wrong on some systems: they may not have the libraries, or the libraries may be broken. - Other libraries appear to suffer as well, such as removing -lmpc. That might mean that you break thread support on some systems. - the chunk to disable fpectl if nothing is known about the systems appears to be unrelated. |
|||
msg37550 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-20 19:16 | |
Logged In: YES user_id=93657 The patch is against Python 2.1.1. I'll try to integrate this with the current source. I'll work with a single checkout across the multiple platforms. Regarding buildno: I was getting recursive dependency links, after I made the shared library dependent upon its .o files. I had to break the recursivity. What is dguxR4 ? Is it OSF1 V4.0, a.k.a. Digital Unix 4.0, a.k.a. Tru64 ? I have now access to an OSF1 v4.0 (a.k.a. DU 4.0). I'll do the build there. uname returns 'OSF1 ...' The -lsocket is needed for building the socket module, and had to be moved there (cf. Python/setup.py, I think). It is not used in the core, so I dropped it there. Dito for -ldsl. No symbols are missing when linking libpython2.1.so, anyway. These libraries just have now to be linked with the modules which use them. The rule I applied for deciding on libraries are: - link with all libraries necessaries so as to suppress all unresolved symbol output - link only with libraries whose absence triggers unresolved symbol outputs - -lmpc: You might be right, but then -lmpc should be present only on systems that need it (and, first of all these systems should be identified/documented). - fpectl: dito FG |
|||
msg37551 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-25 01:06 | |
Logged In: YES user_id=93657 Martin, I'm attaching a tar containing a diff against the CVS mainline (sync last Friday), and the comments on the changes in a different file. I put the changes in configure.in, from configure, but after running autoconf, the --with-shared option has no effect. Hopefully, this is not much. The small changes for building threaded python on OSF/1 are in the same zone as those edited for the change, so I could not separate them out. FG |
|||
msg37552 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2001-09-25 10:37 | |
Logged In: YES user_id=21627 PLEASE do not put unrelated changes into your patches. What does "--with-shared: python with shared lib" have to do with removing @ prefixes in the makefile???? I'm close to giving up reviewing your patches, and waiting for somebody else to reject them. Likewise, why did you remove the makedepend support? It may be broken, but the objective of this patch has nothing to do with this. Same for reconfigure; to re-run configure with the same options, just invoke 'make'. If you think there is a problem, report a bug. The problem with the generated configure not recognizing --with-shared is that you put the AC_ARG_WITH after the check for build_as_shared into configure.in. Just move the AC_ARG_WITH somewhat up. Never use ld to link on Unix, always use $(CC); otherwise, people will complain that linking in C++ modules will fail. To pass options to the linker using the SunPRO compiler, use the -Wl,<option> option; see the description of -W in cc(1). Also, I propose that you put the -zdefs,-no_unresolved in a separate patch, to be applied after this one got integrated. It is not essential for building libpython.so. It appears that your -lsocket changes broke IPv6 support, atleast on Solaris 8: Since -lsocket -lnsl is not linked into programs produced by AC_TRY_{LINK,RUN}, the tests for creating AF_INET6 sockets and for getaddrinfo fail. Please reconsider whether putting the libraries together with the socket module is really essential for this patch. I believe the module will continue to work even if just the Python executable is linked with the socket library (of course, assuming that you get rid of the -z defs requirement). |
|||
msg37553 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-09-25 13:27 | |
Logged In: YES user_id=6380 Martin, feel free to reject and close this. Frederic's attitude has been impossible to deal with, here and in other cases. Frederic, the real issues here are non-technical -- you appear unable to understand and adapt to the conventions of open source development that almost every other contributor picks up without effort. We don't have the time to educate you -- we've tried and failed. We'll continue to review your patches, but we'll be quick to reject them if you continue to ignour our conventions, which have served Python and the Python community well over the years. Feel free to maintain your own fork of the Python sources -- that may be less of a waste of time for all of us. |
|||
msg37554 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-25 21:51 | |
Logged In: YES user_id=93657 Martin, here is a patch update. - I fixed the missing library problem on bulding statically the main Python. The static build work now. I had not tested it yesterday; I was losing myself in patch/autoconf pb instead. - The configure script is now generated from configure.in, with autoconf. Thanks for your help on this. For the patch division issue, I propose to work in two steps: 1) First the global patch working 2) When the global patch works, I'll edit manually the diff file, and create the independent patch files, as well as will remove what I need to troubleshoot the thing. This work is no small piece of cake (at least for me). Please keep in mind that I patch everything from a Python 2.1.1 branch; which is the one in production, and I'd rather keep the manual editing of the patch to the end, as the last step; especially since this is no straightforward business, technically. Otherwise, yes, I've been maintaining my own branch, fortunately. Thanks, FG |
|||
msg37555 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-25 21:53 | |
Logged In: YES user_id=93657 Martin, here is a patch update. - I fixed the missing library problem on bulding statically the main Python. The static build work now. I had not tested it yesterday; I was losing myself in patch/autoconf pb instead. - The configure script is now generated from configure.in, with autoconf. Thanks for your help on this. For the patch division issue, I propose to work in two steps: 1) First the global patch working 2) When the global patch works, I'll edit manually the diff file, and create the independent patch files, as well as will remove what I need to troubleshoot the thing. This work is no small piece of cake (at least for me). Please keep in mind that I patch everything from a Python 2.1.1 branch; which is the one in production, and I'd rather keep the manual editing of the patch to the end, as the last step; especially since this is no straightforward business, technically. Otherwise, yes, I've been maintaining my own branch, fortunately. Thanks, FG |
|||
msg37556 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2001-09-26 10:00 | |
Logged In: YES user_id=21627 I cannot review the patch until I know what chunks of it you mean to submit, and which ones you don't; there are numerous chunks which I will not commit into Python. For the moment, the only thing that is still broken is IPv6 configuration. config.log says configure:4306: checking if --enable-ipv6 is specified configure:4345: gcc -o conftest -g -O2 conftest.c -lpthread -ldl -lthread 1>&5 /var/tmp/ccgOnzLA.o: In function `main': /vol/home-vol1/simulant/loewis/python/dist/src/configure:4337: undefined reference to `socket' So once you've fixed that, please submit a patch that you'd consider final (i.e. completely edited). To simplify further revisions of this final patch, I recommend that you check out a separate Python tree into a fresh directory from CVS, and develop the patch in this directory. This should simplify only including pieces meant to be part of this patch; no editing of the patch would be necessary. If you don't have the disk space for a separate copy, I recommend to use the SourceForge compile farm. |
|||
msg37557 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-09-28 00:37 | |
Logged In: YES user_id=93657 Martin, You should add the following patch, too: =================================================================== RCS file: /mgl/home/fredg/cvs/Python/Lib/distutils/command/build_ext.py,v retrieving revision 1.1.1.1.2.6 retrieving revision 1.1.1.1.2.7 diff -r1.1.1.1.2.6 -r1.1.1.1.2.7 100c100 < self.library_dirs = ['.'] --- > self.library_dirs = ['.', sysconfig.get_config_var( 'LIBDIR')] It is required to build third party modules with distutils. [Actually, I have now 'PLATLIBDIR' instead of 'LIBDIR', so as to differentiate 'platform-independent libdir' from 'platform-specific libdir'.... but I know you don't want to see that here]. Afterhand, I think one would have avoided much of the arguments if the configure.in, Makefile.pre.in and other files associated to the configure/build/install process were commented/documented, structured, and laid out up to "Open Source" standards. Just an idea... Cheers, FG |
|||
msg37558 - (view) | Author: Martin v. Löwis (loewis) * | Date: 2001-10-19 06:22 | |
Logged In: YES user_id=21627 Now that the 2.2b1 branch has been created, we effectively have a feature freeze. That means that this feature will not appear in 2.2. Since the patch isn't ready yet, I'll reject it. If you (or anybody else) revises this patch for use in Python 2.3, please submit a new patch. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-10 16:04:26 | admin | set | github: 35158 |
2001-09-11 21:29:46 | giacometti | create |