msg87264 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 18:01 |
I'm getting what looks like a PEP 383-related failure in test_os, on OS
X 10.5.6. Here's the tail end of running Lib/test/test_os.py, on a
standard non-debug build of py3k.
test_setuid (__main__.PosixUidGidTests) ... ok
test_listdir (__main__.Pep383Tests) ... FAIL
test_open (__main__.Pep383Tests) ... ok
test_stat (__main__.Pep383Tests) ... ok
======================================================================
FAIL: test_listdir (__main__.Pep383Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "Lib/test/test_os.py", line 725, in test_listdir
self.assertEquals(found, expected)
AssertionError: Items in the first set but not the second:
'fooöbar'
'foo%F6bar'
Items in the second set but not the first:
'foo\udcf6bar'
'fooöbar'
----------------------------------------------------------------------
Ran 60 tests in 0.079s
FAILED (failures=1)
Traceback (most recent call last):
File "Lib/test/test_os.py", line 758, in <module>
test_main()
File "Lib/test/test_os.py", line 754, in test_main
Pep383Tests
File "/Users/dickinsm/svn/python/py3k/Lib/test/support.py", line 878,
in run_unittest
_run_suite(suite)
File "/Users/dickinsm/svn/python/py3k/Lib/test/support.py", line 861,
in _run_suite
raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
File "Lib/test/test_os.py", line 725, in test_listdir
self.assertEquals(found, expected)
AssertionError: Items in the first set but not the second:
'fooöbar'
'foo%F6bar'
Items in the second set but not the first:
'foo\udcf6bar'
'fooöbar'
|
msg87265 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 18:09 |
One other piece of information that may be relevant:
Macintosh-4:py3k dickinsm$ echo $LANG
en_US.UTF-8
If I unset the LANG environment variable then I get a different failure
(see output below).
I'll investigate this further and see if I can figure out what's going
wrong.
test_setreuid (__main__.PosixUidGidTests) ... ok
test_setuid (__main__.PosixUidGidTests) ... ok
test_listdir (__main__.Pep383Tests) ... FAIL
test_open (__main__.Pep383Tests) ... ok
test_stat (__main__.Pep383Tests) ... ok
======================================================================
FAIL: test_listdir (__main__.Pep383Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "Lib/test/test_os.py", line 758, in <module>
test_main()
File "Lib/test/test_os.py", line 754, in test_main
Pep383Tests
File "/Users/dickinsm/svn/python/py3k/Lib/test/support.py", line 878,
in run_unittest
_run_suite(suite)
File "/Users/dickinsm/svn/python/py3k/Lib/test/support.py", line 853,
in _run_suite
result = runner.run(suite)
File "/Users/dickinsm/svn/python/py3k/Lib/unittest.py", line 1455, in
run
result.printErrors()
File "/Users/dickinsm/svn/python/py3k/Lib/unittest.py", line 1424, in
printErrors
self.printErrorList('FAIL', self.failures)
File "/Users/dickinsm/svn/python/py3k/Lib/unittest.py", line 1431, in
printErrorList
self.stream.writeln("%s" % err)
File "/Users/dickinsm/svn/python/py3k/Lib/unittest.py", line 1340, in
writeln
self.write(arg)
UnicodeEncodeError: 'ascii' codec can't encode character '\u0308' in
position 194: ordinal not in range(128)
|
msg87272 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2009-05-05 19:06 |
OS/X needs to be special cased in the tests, since it transforms the
invalid filenames on write. And if I remember correctly older versions
do something different...throw an error, maybe?
The case where the letter looks the same (at least in my email) but is
apparently different may require some thought, too.
|
msg87273 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 19:11 |
> The case where the letter looks the same (at least in my email) but is
> apparently different may require some thought, too.
In case it's useful, when I do a list(map(ord, ...)) on those two
lookalike strings I get (where 'expected' comes from the explicit .decode,
while 'found' comes from listdir):
expected: [102, 111, 111, 246, 98, 97, 114]
found: [102, 111, 111, 111, 776, 98, 97, 114]
|
msg87275 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 19:42 |
Even better:
>>> for s in map(unicodedata.name, found): print(s)
...
LATIN SMALL LETTER F
LATIN SMALL LETTER O
LATIN SMALL LETTER O
LATIN SMALL LETTER O
COMBINING DIAERESIS
LATIN SMALL LETTER B
LATIN SMALL LETTER A
LATIN SMALL LETTER R
>>> for s in map(unicodedata.name, expected): print(s)
...
LATIN SMALL LETTER F
LATIN SMALL LETTER O
LATIN SMALL LETTER O
LATIN SMALL LETTER O WITH DIAERESIS
LATIN SMALL LETTER B
LATIN SMALL LETTER A
LATIN SMALL LETTER R
>>> found == expected
False
>>> unicodedata.normalize('NFC', found) == unicodedata.normalize('NFC',
expected)
True
So there are two separate issues, both of which just require fixing the
tests: one issue is that OS X transforms invalid filenames; the other is
that it normalizes valid filenames.
|
msg87280 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 21:00 |
Here's a patch that fixes the test failures for me. It's not ideal,
because it no longer tests that on OS X, it's possible to open a file
created with an invalid filename, using that same name.
|
msg87283 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2009-05-05 21:15 |
I think skipping the test entirely would be even better on Darwin. The
test is really meant to pass only on systems that treat file names as
uninterpreted byte sequences (as such, it may also fail on Linux if the
file system driver enforces UTF-8).
To skip it, it probably would be best to move it out of the "not win32"
block, and into a "neither win32 nor darwin" block - or use the skipIf
decorator.
|
msg87286 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 21:28 |
> I think skipping the test entirely would be even better on Darwin.
Sounds good to me! Here's a new (one-line) patch, using skipIf.
|
msg87288 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2009-05-05 21:32 |
Fine with me, please apply.
|
msg87290 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 21:37 |
Committed, r72359.
|
msg364386 - (view) |
Author: Matthias Braun (Matthias Braun) * |
Date: 2020-03-17 02:41 |
I believe my suggested pull request in https://bugs.python.org/issue39986 may solve this issue as a side effect because we no longer list the root directory but a temporary directory with controlled filenames.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:56:48 | admin | set | github: 50194 |
2020-03-17 02:41:24 | Matthias Braun | set | nosy:
+ Matthias Braun messages:
+ msg364386
|
2009-05-05 21:37:13 | mark.dickinson | set | status: open -> closed resolution: accepted -> fixed messages:
+ msg87290
|
2009-05-05 21:32:22 | loewis | set | assignee: mark.dickinson resolution: accepted messages:
+ msg87288 |
2009-05-05 21:28:30 | mark.dickinson | set | files:
+ issue5944_v2.patch
messages:
+ msg87286 |
2009-05-05 21:15:38 | loewis | set | messages:
+ msg87283 |
2009-05-05 21:00:20 | mark.dickinson | set | files:
+ issue5944.patch keywords:
+ patch messages:
+ msg87280
|
2009-05-05 19:42:05 | mark.dickinson | set | messages:
+ msg87275 |
2009-05-05 19:11:28 | mark.dickinson | set | messages:
+ msg87273 |
2009-05-05 19:06:48 | r.david.murray | set | nosy:
+ r.david.murray messages:
+ msg87272
|
2009-05-05 18:09:01 | mark.dickinson | set | messages:
+ msg87265 |
2009-05-05 18:01:17 | mark.dickinson | create | |