classification
Title: doctest fails to run file based tests with 8bit paths
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, bear, ilan, jafo, meatballhat, terry.reedy, tim.peters, vstinner
Priority: low Keywords: easy, patch

Created on 2007-10-12 18:29 by bear, last changed 2013-04-02 03:19 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
ensuring-doctest-file-suite-works-with-unicode-paths.patch meatballhat, 2010-07-31 21:08 assert unicode paths are properly handled review
Messages (12)
msg56381 - (view) Author: Mike Taylor (bear) * Date: 2007-10-12 18:31
In our builds the included patch fixes this issue.

Patch by Brian Kirsch, tested at OSAF on Python 2.5.1
msg63766 - (view) Author: Ilan Schnell (ilan) * (Python triager) Date: 2008-03-17 20:41
Bug is most likely platform specific.  Can someone suggest how this
should be handled on multiple platforms?

Mike, can you report on which platform you encountered the bug on?
Can you provide a script that reproduces the bug?

On Mac OS 10.4, Python 2.5 I could not create a file:
>>> f=open('\xed', 'w')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: invalid mode: w

I will submit this as a separate bug because the error
message sould say 'invalid file name' instead of 'invalid mode'.
msg63836 - (view) Author: Mike Taylor (bear) * Date: 2008-03-18 01:09
Hi,

it was running on FC4 with UTF-32 support and was using the Japanese locale.

The bug is reproducible using any doctest that is stored in a mixed
character path.  where it is in the Chandler tree is not easily pulled
apart but if you really need it I can work up a small subset.

On Mon, Mar 17, 2008 at 4:41 PM, Ilan Schnell <report@bugs.python.org> wrote:
>
>  Ilan Schnell <ilanschnell@gmail.com> added the comment:
>
>  Bug is most likely platform specific.  Can someone suggest how this
>  should be handled on multiple platforms?
>
>  Mike, can you report on which platform you encountered the bug on?
>  Can you provide a script that reproduces the bug?
>
>  On Mac OS 10.4, Python 2.5 I could not create a file:
>  >>> f=open('\xed', 'w')
>  Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>  IOError: invalid mode: w
>
>  I will submit this as a separate bug because the error
>  message sould say 'invalid file name' instead of 'invalid mode'.
>
>  ----------
>  assignee:  -> tim_one
>  nosy: +ilan, tim_one
>  priority:  -> low
>  versions: +Python 2.6
>
>
>
>  __________________________________
>  Tracker <report@bugs.python.org>
>  <http://bugs.python.org/issue1274>
>  __________________________________
>
msg63852 - (view) Author: Sean Reifschneider (jafo) * (Python committer) Date: 2008-03-18 02:40
This may be fixed already, or a bug in FC4.  Or perhaps you could
provide more information on how the bug is invoked.  I was able to
successfully execute a doctest with "\xee" in the path on an F8 box:

Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11) 
[GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2

Also: Please do not quote text in the reply.
Type "help", "copyright", "credits" or "license" for more information.
>>> import doctest
>>> doctest.testfile('foo\xeebar/test.txt')
(0, 1)
>>> 
guin:pytest$ cat fo*/test.txt
The ``example`` module
======================

Using ``print``
-------------------

This is a test example.

   >>> print 'Hello world!'
   Hello world!
msg63874 - (view) Author: Mike Taylor (bear) * Date: 2008-03-18 03:42
Not in the system PATH but in the path where the file is stored:

Here is the original traceback from the bug report for Chandler:

Traceback (most recent call last):
  File
"/Development/osaf/chandler_石田リチャード/chandler/release/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/doctest.py",
line 2107, in runTest
    test, out=new.write, clear_globs=False)
  File
"/Development/osaf/chandler_石田リチャード/chandler/release/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/doctest.py",
line 1345, in run
    return self.__run(test, compileflags, out)
  File
"/Development/osaf/chandler_石田リチャード/chandler/release/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/doctest.py",
line 1236, in __run
    got += _exception_traceback(exc_info)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 70:
ordinal not in range(128)

On Mon, Mar 17, 2008 at 10:40 PM, Sean Reifschneider
<report@bugs.python.org> wrote:
>
>  Sean Reifschneider <jafo@tummy.com> added the comment:
>
>  This may be fixed already, or a bug in FC4.  Or perhaps you could
>  provide more information on how the bug is invoked.  I was able to
>  successfully execute a doctest with "\xee" in the path on an F8 box:
>
>  Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11)
>  [GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2
>
>  Also: Please do not quote text in the reply.
>  Type "help", "copyright", "credits" or "license" for more information.
>  >>> import doctest
>  >>> doctest.testfile('foo\xeebar/test.txt')
>  (0, 1)
>  >>>
>  guin:pytest$ cat fo*/test.txt
>  The ``example`` module
>  ======================
>
>  Using ``print``
>  -------------------
>
>  This is a test example.
>
>    >>> print 'Hello world!'
>    Hello world!
>
>  ----------
>  nosy: +jafo
>
>
>
>  __________________________________
>  Tracker <report@bugs.python.org>
>  <http://bugs.python.org/issue1274>
>  __________________________________
>
msg112198 - (view) Author: Dan Buch (meatballhat) Date: 2010-07-31 20:39
just updating the patch
msg112204 - (view) Author: Dan Buch (meatballhat) Date: 2010-07-31 21:08
seems to be okay with py3k's unicode StringIO

adding test to show as much (although I'm sure the test could be more elegant than what I've done.)
msg112224 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-07-31 23:06
I didn't read all comments, but about bug_3740_1.patch: it's not a good idea to fallback to byte strings. If you have an unicode path, you are lucky, and you have to keep all these nice and correctly formatted characters!

I mean: this patch is just an ugly workaround. You should fix the real bug instead.
msg112251 - (view) Author: Dan Buch (meatballhat) Date: 2010-08-01 01:47
@haypo - I'm not in favor of using the attached bug_3740_1.patch but instead adding a test to assert that unicode file paths are now handled correctly.  I can't remove @bear's original patch myself, though, so ... not sure what to do about that :)
msg112256 - (view) Author: Mike Taylor (bear) * Date: 2010-08-01 02:07
wow - that is some old bug history ;)

I'm surprised the patch is even close to being valid

On Sat, Jul 31, 2010 at 21:47, Dan Buch <report@bugs.python.org> wrote:
>
> Dan Buch <daniel.buch@gmail.com> added the comment:
>
> @haypo - I'm not in favor of using the attached bug_3740_1.patch but instead adding a test to assert that unicode file paths are now handled correctly.  I can't remove @bear's original patch myself, though, so ... not sure what to do about that :)
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue1274>
> _______________________________________
>
msg116781 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-09-18 14:10
The patch simply changes test_doctest, is it acceptable or not?
msg185798 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-04-02 03:19
I am closing this for lack of evidence that there is a bug in a current release 2.7.3 or 3.3.0 or release candidate for 2.7.4 or 3.3.1. What would be needed is a self-contained example that shows a problem with one of the above.
History
Date User Action Args
2013-04-02 03:19:31terry.reedysetstatus: open -> closed

assignee: tim.peters ->

nosy: + terry.reedy
messages: + msg185798
resolution: out of date
stage: resolved
2010-09-18 14:10:54BreamoreBoysetnosy: + BreamoreBoy
messages: + msg116781
2010-09-18 14:08:12BreamoreBoysetfiles: - bug_3740_1.patch
2010-08-01 02:07:23bearsetmessages: + msg112256
2010-08-01 01:47:12meatballhatsetmessages: + msg112251
2010-07-31 23:06:38vstinnersetmessages: + msg112224
2010-07-31 21:09:23meatballhatsetfiles: - doctest-support-unicode-file-paths.patch
2010-07-31 21:08:54meatballhatsetfiles: + ensuring-doctest-file-suite-works-with-unicode-paths.patch

messages: + msg112204
2010-07-31 20:40:34eric.araujosetnosy: + vstinner
2010-07-31 20:39:13meatballhatsetfiles: + doctest-support-unicode-file-paths.patch
versions: + Python 3.2, Python 3.3, - Python 2.6, Python 2.5
nosy: + meatballhat

messages: + msg112198
2008-03-18 03:43:00bearsetmessages: + msg63874
2008-03-18 02:40:38jafosetnosy: + jafo
messages: + msg63852
2008-03-18 01:09:15bearsetmessages: + msg63836
2008-03-17 20:41:35ilansetpriority: low
assignee: tim.peters
messages: + msg63766
nosy: + ilan, tim.peters
versions: + Python 2.6
2008-01-12 01:46:50akuchlingsetkeywords: + easy
2007-10-12 22:07:14brett.cannonsetkeywords: + patch
2007-10-12 18:31:59bearsetfiles: + bug_3740_1.patch
nosy: + bear
messages: + msg56381
2007-10-12 18:29:49bearcreate