classification
Title: Adding _collections to static build
Type: crash Stage: resolved
Components: Build Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: 9546 Superseder:
Assigned To: orsenthil Nosy List: eli.bendersky, flox, loewis, ncoghlan, orsenthil, pitrou, rhettinger
Priority: high Keywords: patch

Created on 2010-08-09 07:01 by orsenthil, last changed 2010-08-15 14:42 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
_collections.patch orsenthil, 2010-08-09 07:01
Messages (12)
msg113392 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-08-09 07:01
As a fix for issue9396 in r83874, functools was included in re module and this caused a build failure.

This was reverted in r83875, with the following message by Raymond:

The problem is that the re module is imported by sysconfig
and re needs functools which uses collections.OrderedDict()
but the _collectionsmodule.c code is not yet constructed
at this point in the build.

The likely best solution will be to include _collections
as part of the static build before the rest of the
boot-strapping.

We discussed it in IRC, I am including the _collections module to the static build.

After this the r83874 can be reapplied.
msg113396 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-08-09 07:32
Fixed in revision 83876.

The reverted commit r83875 is re-applied in r83877.
msg113404 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-09 11:10
Reverted since it broke the buildbots.
msg113407 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-08-09 11:40
This is the message from the buildbot failure:

Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
Usually, copying Modules/Setup.dist to Modules/Setup will work.

I am not sure, if anything more to be done if changes made to Modules/Setup.dist are to be propagated to Modules/Setup *in the buildbots*.

On an svn checkout the compilation is successful.
msg113410 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-08-09 12:52
See report on issue #9546: the "clean" step should remove "Modules/Setup", but it is skipped.
msg113792 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-08-13 16:24
Note that the following sequence should work even without changing the buildbot behaviour:
- in one checkin, modify Modules/Setup.dist. The buildbots will pick up this change on their *subsequent* build (as the build triggered by this checkin, so long as it compiles correctly, will trigger the clean step)
- in a latter checkin, make the changes that depend on the new build order

What breaks at the moment is trying to do both of those things in one step, or trying to do the first step while compilation is broken.

Since Antoine's last reversion, a few successful build have gone through so it should be OK to reapply the original patch again.

Longer term, we probably want to run the clean step both before the configure step and after the compile and test steps (even if the compilation fails).
msg113938 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-08-15 03:53
The reverted patch was reapplied in r83980 by Antoine.
The issue9546 resolves the part of always running the cleanup code on compile error.
msg113943 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-08-15 08:44
Re msg113792: Nick, running the clean step before configure is not possible. It requires a Makefile, which isn't there yet.
msg113946 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-08-15 09:16
On Sun, Aug 15, 2010 at 6:44 PM, Martin v. Löwis <report@bugs.python.org> wrote:
>
> Martin v. Löwis <martin@v.loewis.de> added the comment:
>
> Re msg113792: Nick, running the clean step before configure is not possible. It requires a Makefile, which isn't there yet.

Ah, of course. Still, the fix to run the clean step even if the
compile fails should at least help the issue.
msg113951 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2010-08-15 12:46
Compiling py3k from main repo doesn't work:
ould not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
  File "/home/eliben/python_src/eliben-py3k/Lib/site.py", line 540, in <module>
    main()
  File "/home/eliben/python_src/eliben-py3k/Lib/site.py", line 522, in main
    known_paths = addusersitepackages(known_paths)
  File "/home/eliben/python_src/eliben-py3k/Lib/site.py", line 249, in addusersitepackages
    user_site = getusersitepackages()
  File "/home/eliben/python_src/eliben-py3k/Lib/site.py", line 224, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/home/eliben/python_src/eliben-py3k/Lib/site.py", line 214, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/home/eliben/python_src/eliben-py3k/Lib/sysconfig.py", line 553, in get_config_var
    return get_config_vars().get(name)
  File "/home/eliben/python_src/eliben-py3k/Lib/sysconfig.py", line 435, in get_config_vars
    import re
  File "/home/eliben/python_src/eliben-py3k/Lib/re.py", line 121, in <module>
    import functools
  File "/home/eliben/python_src/eliben-py3k/Lib/functools.py", line 15, in <module>
    from collections import OrderedDict
  File "/home/eliben/python_src/eliben-py3k/Lib/collections.py", line 9, in <module>
    from _collections import deque, defaultdict
ImportError: No module named _collections
[27060 refs]
make: *** [sharedmods] Error 1

----------------------------------------------

I re-ran configure, ran 'make clean' then 'make', doesn't help. The error and the HG log make me suspect it might be this issue's fix that's involved.
msg113954 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2010-08-15 12:56
`make distclean` worked for me (tipped at #python-dev), after it ./configure and make succeeded. I must say it's not 100% intuitive, I'm used to just using `make clean` for a complete cleanup.
msg113962 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-08-15 14:42
Considering the notice

-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
Usually, copying Modules/Setup.dist to Modules/Setup will work.
-----------------------------------------------

also would have solved the problem.
History
Date User Action Args
2010-08-15 14:42:51loewissetmessages: + msg113962
2010-08-15 12:56:27eli.benderskysetmessages: + msg113954
2010-08-15 12:46:53eli.benderskysetnosy: + eli.bendersky
messages: + msg113951
2010-08-15 09:16:54ncoghlansetmessages: + msg113946
2010-08-15 08:44:39loewissetmessages: + msg113943
2010-08-15 03:53:32orsenthilsetstatus: open -> closed
resolution: fixed
messages: + msg113938
2010-08-13 16:25:00ncoghlansetnosy: + ncoghlan, loewis
messages: + msg113792
2010-08-09 12:52:42floxsetnosy: + flox
dependencies: + buildbot: if a compile error occurs, the "clean" step is skipped
messages: + msg113410
2010-08-09 11:40:56orsenthilsetstatus: closed -> open
priority: normal -> high
resolution: fixed -> (no value)
messages: + msg113407
2010-08-09 11:10:23pitrousetnosy: + pitrou
messages: + msg113404
2010-08-09 07:32:21orsenthilsetstatus: open -> closed

assignee: orsenthil
components: + Build
versions: + Python 3.2
nosy: + rhettinger

messages: + msg113396
resolution: fixed
stage: resolved
2010-08-09 07:01:23orsenthilcreate