classification
Title: py_compile cannot create files in current directory
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: meador.inge Nosy List: Arfrever, eric.araujo, eric.snow, georg.brandl, meador.inge, ncoghlan, pitrou, python-dev, sjdv1982
Priority: normal Keywords: easy, patch

Created on 2011-07-23 07:42 by sjdv1982, last changed 2011-11-28 15:41 by meador.inge. This issue is now closed.

Files
File name Uploaded Description Edit
py_compile.py sjdv1982, 2011-07-23 07:42 py_compile.py, fixed in line 133
py_compile.diff sjdv1982, 2011-07-23 09:43 diff with 3.2 release
issue12618-0.patch meador.inge, 2011-10-26 04:29 Patch v0 against tip (3.3.0a0) review
issue12618-1.patch meador.inge, 2011-11-27 01:08 review
Messages (25)
msg140940 - (view) Author: Sjoerd de Vries (sjdv1982) Date: 2011-07-23 07:42
When you specify cfile to be in the current directory, an error occurs (line 133).
I have fixed the file, see attached
msg140941 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-07-23 07:46
Please create a patch in unified format.
msg140945 - (view) Author: Sjoerd de Vries (sjdv1982) Date: 2011-07-23 08:18
The attached file just works.
You can diff with trunk, or wherever python devs store the latest version.
msg140946 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-07-23 08:21
It might work right now, but in case the file changes before your change can be processed, we will lose the previous changes if we just copy in your new file.

IOW, you're making our work much harder and your change is less likely to be applied.
msg140953 - (view) Author: Sjoerd de Vries (sjdv1982) Date: 2011-07-23 09:43
Makes no sense to me: since I don't have the trunk version, I can only diff -c against 3.2 release. Doing a diff against trunk is 1 sec of work for you.

But I am just being a helpful user; so if a diff is what you want, here it is. No trouble for me, since I am on Linux. 

Not a very nice policy to any helpful Windows users, though: they won't have diff installed and they would waste an hour or so on this.
msg140959 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-07-23 09:54
It's a context patch, not a unified patch, and it is reversed :) .
msg140964 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-07-23 10:43
Well, we can work with this patch.  Thanks.

Yes, we can diff against 3.2, but for that you would have at least needed to specify

* that you worked on a released version
* and that that version is 3.2

Otherwise, we cannot know what to diff against.

Note that even Windows users usually use Python from Mercurial for developing, so that they can use "hg diff" to create a patch easily.
msg140990 - (view) Author: Sjoerd de Vries (sjdv1982) Date: 2011-07-23 15:26
Good to hear that the patch is helpful.

Again, I am just trying to be a helpful user, making a (very very little) contribution to make Python better. I am not a Python dev at all: Python is already awesome enough for me, no desire to change it :-) 
I just want to say that a "we only accept patches" policy is not being very nice to similar helpful users on Windows, who don't have diff or Mercurial but who did manage to fix a file themselves. 

I did indicate "Python 3.2" in the bug report, maybe something went wrong.
msg141064 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-07-24 23:41
It's rare to receive fixes from Windows users that don't have a dev environment set up (i.e. they just edited their stdlib code in place, or copied it and created a modified version). Thanks for persisting in the face of invalid assumptions on our part.
msg146098 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-10-21 15:58
Sjoerd, can you paste the code that produces the bug?  It would help create a test.
msg146309 - (view) Author: Sjoerd de Vries (sjdv1982) Date: 2011-10-24 17:00
Hi Éric,

There you go, adapted from http://effbot.org/librarybook/py-compile.htm :

############
# File: py-compile-example-1.py

import py_compile

# explicitly compile this module
py_compile.compile("py-compile-example-1.py","py-compile-example-1.pyc")

############

Also, I tested and this bug is present neither on 3.1 nor on 2.x

cheers
Sjoerd
msg146372 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-10-25 15:16
I can reproduce in 3.2 and 3.3.  I’ll commit a test and patch when I get the time, or another dev can take this over.
msg146410 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-10-26 04:29
I think it might be easier to just always use the absolute path rather than looking at the directory length.  Maybe something like the attached.  I added unit tests as well.
msg148049 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-21 14:55
Thanks for the patch Meador, I hadn’t realized we had no tests for py_compile (it is however used in test_import and test_compileall).  I think it would be nice to commit the tests first (except for the one that’s the object of this bug report) in order to have a baseline, and then see about fixing this bug.  I’ll do that in a few days if nobody objects.

I’m not sure there would be no negative side-effects to using os.path.abspath; we don’t know what people do with symlinks and relative paths out there, so I’d prefer adding a safe special case* rather than always calling abspath.  What do you think?


* Instead of using len, something like this would be clear IMO:

      if parent:  # empty string means current directory, skip creating the dir
          os.makedirs(parent)
msg148189 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-23 16:48
Meador, maybe you would like to commit the tests (except for the one that’s the object of this report) yourself?  I don’t mind doing it, but as you have push rights now maybe you prefer to have your name directly associated with your work.
msg148329 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-11-25 15:58
Éric, sure, I will commit the tests sometime today.  Then I will respond to the 'os.path.abspath' question as well.
msg148380 - (view) Author: Roundup Robot (python-dev) Date: 2011-11-26 05:45
New changeset bcc7bf3963cc by Meador Inge in branch '2.7':
Issue #12618: create unit tests for the py_compile module
http://hg.python.org/cpython/rev/bcc7bf3963cc

New changeset 2be3a2e63683 by Meador Inge in branch '3.2':
Issue #12618: create unit tests for the py_compile module
http://hg.python.org/cpython/rev/2be3a2e63683

New changeset f8f58db0715e by Meador Inge in branch 'default':
Issue #12618: create unit tests for the py_compile module
http://hg.python.org/cpython/rev/f8f58db0715e
msg148400 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-11-26 13:17
The tests break on the Windows buildbots:

======================================================================
ERROR: test_relative_path (test.test_py_compile.PyCompileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_py_compile.py", line 33, in test_relative_path
    py_compile.compile(os.path.relpath(self.source_path),
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\ntpath.py", line 622, in relpath
    raise ValueError(error)
ValueError: path is on mount 'c:', start on mount 'D:'
msg148411 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-11-26 14:52
On Sat, Nov 26, 2011 at 7:17 AM, Antoine Pitrou <report@bugs.python.org> wrote:

> The tests break on the Windows buildbots:

I am investigating now.
msg148418 - (view) Author: Roundup Robot (python-dev) Date: 2011-11-26 18:01
New changeset b23453530d5f by Meador Inge in branch '2.7':
Issue #12618: fix py_compile unit tests to handle different drives on Windows
http://hg.python.org/cpython/rev/b23453530d5f

New changeset 7097d52cacee by Meador Inge in branch '3.2':
Issue #12618: fix py_compile unit tests to handle different drives on Windows
http://hg.python.org/cpython/rev/7097d52cacee

New changeset 5243752e19aa by Meador Inge in branch 'default':
Issue #12618: fix py_compile unit tests to handle different drives on Windows
http://hg.python.org/cpython/rev/5243752e19aa
msg148419 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-11-26 18:06
The tests are fixed now.  A relative path was being computed, but on Windows the current working directory drive and the drive of the relative path we were computing was different (and so this test bug would *not* be seen if running on a Windows box with a single "C: drive" setup).

/me sighs at the concept of Windows drives.  Thanks for the heads up on the test break Antoine.
msg148434 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-11-27 01:08
> we don’t know what people do with symlinks and relative paths out 
> there, so I’d prefer adding a safe special case* rather than always 
> calling abspath.  What do you think?

Éric, I agree.  I didn't know about the strange symlink + relative path 
behavior with 'os.path.normpath', but [1] cleared me up.  With that in
mind the special casing is OK.  I have attached an updated patch with
a unit test.

Also, this is not an issue for Python 2.7.  The 2.7 implementation
assumes any directories mentioned in the path already exist.  So, I
removed 2.7 from the affected versions.

[1] http://mail.python.org/pipermail/python-dev/2005-December/058452.html
msg148479 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-28 13:59
> I have attached an updated patch with a unit test.
LGTM.

> Also, this is not an issue for Python 2.7.  The 2.7 implementation assumes any
> directories mentioned in the path already exist.
Cool.  The test can still be committed in that branch too.
msg148491 - (view) Author: Roundup Robot (python-dev) Date: 2011-11-28 15:39
New changeset 661fb211f220 by Meador Inge in branch '3.2':
Issue #12618: py_compile cannot create files in current directory
http://hg.python.org/cpython/rev/661fb211f220

New changeset e3647275f468 by Meador Inge in branch 'default':
Issue #12618: py_compile cannot create files in current directory
http://hg.python.org/cpython/rev/e3647275f468
msg148492 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-11-28 15:40
> Cool.  The test can still be committed in that branch too.

The regression test for this issue was already committed for 2.7 in bcc7bf3963cc as a part of creating the unit test baseline.  I just committed the bug fix to 3.2 and default.

Thanks for the fix Sjoerd.
History
Date User Action Args
2011-11-28 15:41:28meador.ingesetstatus: open -> closed
stage: patch review -> resolved
2011-11-28 15:40:40meador.ingesetresolution: fixed
messages: + msg148492
2011-11-28 15:39:18python-devsetmessages: + msg148491
2011-11-28 13:59:50eric.araujosetassignee: meador.inge
messages: + msg148479
versions: + Python 2.7
2011-11-27 01:08:18meador.ingesetfiles: + issue12618-1.patch

messages: + msg148434
versions: - Python 2.7
2011-11-26 18:06:28meador.ingesetmessages: + msg148419
2011-11-26 18:01:22python-devsetmessages: + msg148418
2011-11-26 14:52:15meador.ingesetmessages: + msg148411
2011-11-26 13:17:42pitrousetnosy: + pitrou
messages: + msg148400
2011-11-26 05:45:36python-devsetnosy: + python-dev
messages: + msg148380
2011-11-25 15:58:26meador.ingesetmessages: + msg148329
2011-11-23 16:48:34eric.araujosetmessages: + msg148189
2011-11-21 20:12:59eric.snowsetnosy: + eric.snow
2011-11-21 14:55:39eric.araujosetmessages: + msg148049
2011-10-26 04:29:53meador.ingesetfiles: + issue12618-0.patch

nosy: + meador.inge
messages: + msg146410

stage: test needed -> patch review
2011-10-25 15:16:17eric.araujosetmessages: + msg146372
2011-10-24 17:00:49sjdv1982setmessages: + msg146309
2011-10-21 15:58:12eric.araujosetmessages: + msg146098
2011-07-29 16:23:03eric.araujosetkeywords: + easy
nosy: + eric.araujo
stage: test needed

versions: + Python 2.7, Python 3.3
2011-07-24 23:41:21ncoghlansetnosy: + ncoghlan
messages: + msg141064
2011-07-23 15:26:27sjdv1982setmessages: + msg140990
2011-07-23 10:43:12georg.brandlsetmessages: + msg140964
2011-07-23 09:54:16Arfreversetmessages: + msg140959
2011-07-23 09:43:46sjdv1982setfiles: + py_compile.diff
keywords: + patch
messages: + msg140953
2011-07-23 08:21:38georg.brandlsetnosy: + georg.brandl
messages: + msg140946
2011-07-23 08:18:16sjdv1982setmessages: + msg140945
2011-07-23 07:46:23Arfreversetnosy: + Arfrever
messages: + msg140941
2011-07-23 07:42:50sjdv1982create