This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: make install misses test dirs for packaging and email modules
Type: behavior Stage: patch review
Components: Installation Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, barry, eric.araujo, ned.deily, python-dev, r.david.murray, vinay.sajip
Priority: normal Keywords: patch

Created on 2011-06-10 16:08 by eric.araujo, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
add-packaging-makefile.pre.in.diff eric.araujo, 2011-06-10 16:08 review
Messages (15)
msg138096 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-10 16:08
Here’s a patch adding Lib/packaging subdirectories to Makefile.pre.in.  Because of our extensive tests for pypi-related code, the list is huge.  Maybe it should use a call to find(1) instead of using a fragile manually-maintained list, so I’m asking here.
msg138140 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-06-10 22:22
Listing all those directories to the Makefile does seem very brittle and will certainly be a maintenance hassle.  OTOH, there is something to be said for being explicit about what gets installed.  I suppose you could add the find step to ./configure and pass the list into the Makefile.  Dunno if that is really is a good idea, though.

In any case, if a reminder isn't already there, perhaps one should be added to the devguide or packaging docs about this when adding test directories.

Also you may want to add some of the packaging test directories to those excluded in the calls to compileall.py a little further down in the libinstall target.
msg138175 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-11 18:01
> Also you may want to add some of the packaging test directories to
> those excluded in the calls to compileall.py a little further down in
> the libinstall target.

Ah, good one.  I guess doing this could have prevented http://hg.python.org/cpython/rev/9041520be581
msg138352 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2011-06-14 21:42
Nice to see my search didn't find this bug. ;)

I already committed a change to install packaging, but you may want to revert that and commit the patch in this issue.  I'll leave that for you to decide!
msg138365 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-15 13:36
> Nice to see my search didn't find this bug. ;)
Do you remember what search terms you tried?

> I already committed a change to install packaging, but you may want to
> revert that and commit the patch in this issue.
I honestly don’t know.  Do we have a policy about using intermediary variables (like XMLLIBSUBDIRS) or not?  Can we assume find(1) exists on the system and use it?
msg138418 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-06-16 08:24
It's not packaging-related, but other directories are also missing from LIBSUBDIRS:

test/test_email test/test_email/data

IMO it's not ideal to use find(1), as in theory a developer may have some directories with matching names in the source build that they don't want installed - or would that be a no-no?

Ideally at least one buildbot should be set up to install and then test the new installation - I presume this error wasn't caught on the buildbots as they all test source builds?
msg138443 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-16 14:40
Fixing the missing dirs for email has been on my todo list for a while but obviously hasn't gotten done yet.  I talked with Antoine about setting up a buildbot that tested an installed version, but I'm not sure when I'll get around to trying *that* either.  And in any case, even that would not have caught the test_email omission, since rergrtest uses filename-based discovery to select the tests to run, so it would not have noticed that test_email was missing.  I suppose that is a disadvantage of having regrtest recognize test packages.
msg138462 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-06-16 17:37
Perhaps not in the general case, but I found the missing test_email and test_email/data through running regrtest using an installed Python 3.3.
msg138469 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-16 20:44
Ah.  I wouldn't have expected that.  For my information, what was the error you got?
msg138477 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-06-16 23:09
> R. David Murray <rdmurray@bitdance.com> added the  comment:

> 
> Ah.  I wouldn't have expected that.  For my  information, what was the error 
>you  got?

[ 99/356] test_email
test test_email crashed -- Traceback (most recent call last):
  File "regrtest.py", line 1036, in runtest_inner
ImportError: No module named 'test_email'

From the full test results at

https://gist.github.com/1022705/45fa6992c25197a3c9f6281fc645da94dcc5feae

N.B. I have now fixed the error in my fork, you can see the change I made here:

https://bitbucket.org/vinay.sajip/pythonv/compare/..mirror/cpython#chg-Makefile.pre.in

Regards,

Vinay Sajip
msg138483 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-17 00:31
OK, now I'm really confused.  I tried to reproduce this by installing from my checkout into a work dir, and then running regrtest using that installed python, and discovered that test_email and its data directory got copied by the installation process.  (There were test failures, so I'll have to work on that...)  This is without any mention of test_email in the Makefile I used to do the 'make install'.
msg138485 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-06-17 00:56
> OK, now I'm really confused.  I tried to reproduce this by  installing from my 
>checkout into a work dir, and then running regrtest using  that installed 
>python, and discovered that test_email and its data directory got  copied by the 
>installation process.  (There were test failures, so I'll  have to work on 
>that...)  This is without any mention of test_email in the  Makefile I used to 
>do the 'make  install'.

That does seem odd. On my pythonv fork (which follows the cpython repo pretty 
closely), the only current failures are test_lib2to3, test_packaging and 
test_sysconfig - all of which failures are already the subject of tickets on the 
tracker. On my system at least, test_email doesn't show up any failures. See the 
latest version of the test log from the Gist link I posted earlier - the comment 
also references the failure tickets.
msg138517 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-06-17 15:46
New changeset c9d27c63b45c by R David Murray in branch 'default':
#12313: update Makefile.pre.in to account for email tests moving to 'test' dir
http://hg.python.org/cpython/rev/c9d27c63b45c
msg138518 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-17 15:47
Well, regardless of whether or not I understand what's going on, clearly those directories in the makefile needed updating, so I did it.
msg138830 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-06-22 19:10
The changes have been checked in by Barry and David, so I'm closing this issue.
History
Date User Action Args
2022-04-11 14:57:18adminsetgithub: 56522
2011-06-22 19:10:38vinay.sajipsetstatus: open -> closed
resolution: fixed
messages: + msg138830
2011-06-17 15:47:44r.david.murraysetmessages: + msg138518
2011-06-17 15:46:50python-devsetnosy: + python-dev
messages: + msg138517
2011-06-17 00:56:59vinay.sajipsetmessages: + msg138485
2011-06-17 00:31:07r.david.murraysetmessages: + msg138483
2011-06-16 23:09:04vinay.sajipsetmessages: + msg138477
2011-06-16 20:44:54r.david.murraysetmessages: + msg138469
2011-06-16 17:37:02vinay.sajipsetmessages: + msg138462
2011-06-16 14:40:05r.david.murraysetmessages: + msg138443
2011-06-16 08:32:01vinay.sajipsetnosy: + r.david.murray

title: make install misses packaging module -> make install misses test dirs for packaging and email modules
2011-06-16 08:24:42vinay.sajipsetnosy: + vinay.sajip
messages: + msg138418
2011-06-15 13:36:04eric.araujosetmessages: + msg138365
2011-06-14 21:42:16barrysetnosy: + barry
messages: + msg138352
2011-06-14 21:13:22Arfreversetnosy: + Arfrever
2011-06-11 18:01:46eric.araujosetmessages: + msg138175
2011-06-10 22:22:24ned.deilysetnosy: + ned.deily
messages: + msg138140
2011-06-10 16:08:02eric.araujocreate