➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2010-04-16 16:35 by vstinner, last changed 2022-04-11 14:57 by admin. This issue is now closed.
| Messages (10) | |||
|---|---|---|---|
| msg103340 - (view) | Author: STINNER Victor (vstinner) *  |
Date: 2010-04-16 16:35 | |
http://www.python.org/dev/buildbot/3.x/builders/x86 Tiger 3.x/builds/6/steps/test/logs/stdio test test_pep277 failed -- multiple errors occurred; run in verbose mode for details Re-running test test_pep277 in verbose mode test_directory (test.test_pep277.UnicodeFileTests) ... ok test_failures (test.test_pep277.UnicodeFileTests) ... ok test_listdir (test.test_pep277.UnicodeFileTests) ... FAIL test_normalize (test.test_pep277.UnicodeFileTests) ... ERROR test_open (test.test_pep277.UnicodeFileTests) ... ok test_rename (test.test_pep277.UnicodeFileTests) ... ok test_directory (test.test_pep277.UnicodeNFCFileTests) ... ok test_failures (test.test_pep277.UnicodeNFCFileTests) ... ok test_listdir (test.test_pep277.UnicodeNFCFileTests) ... FAIL test_normalize (test.test_pep277.UnicodeNFCFileTests) ... ERROR test_open (test.test_pep277.UnicodeNFCFileTests) ... ok test_rename (test.test_pep277.UnicodeNFCFileTests) ... ok test_directory (test.test_pep277.UnicodeNFDFileTests) ... ok test_failures (test.test_pep277.UnicodeNFDFileTests) ... ok test_listdir (test.test_pep277.UnicodeNFDFileTests) ... ok test_normalize (test.test_pep277.UnicodeNFDFileTests) ... ERROR test_open (test.test_pep277.UnicodeNFDFileTests) ... ok test_rename (test.test_pep277.UnicodeNFDFileTests) ... ok test_directory (test.test_pep277.UnicodeNFKCFileTests) ... ok test_failures (test.test_pep277.UnicodeNFKCFileTests) ... ok test_listdir (test.test_pep277.UnicodeNFKCFileTests) ... FAIL test_normalize (test.test_pep277.UnicodeNFKCFileTests) ... ERROR test_open (test.test_pep277.UnicodeNFKCFileTests) ... ok test_rename (test.test_pep277.UnicodeNFKCFileTests) ... ok test_directory (test.test_pep277.UnicodeNFKDFileTests) ... ok test_failures (test.test_pep277.UnicodeNFKDFileTests) ... ok test_listdir (test.test_pep277.UnicodeNFKDFileTests) ... ok test_normalize (test.test_pep277.UnicodeNFKDFileTests) ... ERROR test_open (test.test_pep277.UnicodeNFKDFileTests) ... ok test_rename (test.test_pep277.UnicodeNFKDFileTests) ... ok ====================================================================== ERROR: test_normalize (test.test_pep277.UnicodeFileTests) ---------------------------------------------------------------------- test test_pep277 crashed -- <class 'UnicodeEncodeError'>: 'ascii' codec can't encode characters in position 222-239: ordinal not in range(128) Traceback (most recent call last): File "./Lib/test/regrtest.py", line 905, in runtest_inner File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_pep277.py", line 195, in test_main UnicodeNFKDFileTests, File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/support.py", line 1000, in run_unittest _run_suite(suite) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/support.py", line 974, in _run_suite result = runner.run(suite) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/unittest/runner.py", line 158, in run result.printErrors() File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/unittest/runner.py", line 108, in printErrors self.printErrorList('ERROR', self.errors) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/unittest/runner.py", line 116, in printErrorList self.stream.writeln("%s" % err) File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/unittest/runner.py", line 24, in writeln self.write(arg) UnicodeEncodeError: 'ascii' codec can't encode characters in position 222-239: ordinal not in range(128) |
|||
| msg103469 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2010-04-18 08:02 | |
See also #8422. |
|||
| msg105648 - (view) | Author: Florent Xicluna (flox) * |
Date: 2010-05-13 17:56 | |
Some diagnosis information ... - happens for 2 tests inside test_pep277: test_listdir and test_normalize - happens only for 4 weird filenames (added with #8180) - happens only on "x86 Tiger" buildbot ("PPC Tiger" does not report this error) I've added some lines to regrtest.py to identify each platform: "x86 Tiger": Darwin-8.10.1-i386-32bit little-endian (OS X 10.4.10) "PPC Tiger": Darwin-8.11.0-Power_Macintosh-powerpc-32bit big-endian (OS X 10.4.11) I don't know how the endianness may be linked with this issue. Failing files (4): u'\u0385\u03d3\u03d4', u'\u00a8\u0301\u03d2\u0301\u03d2\u0308', # == NFD(u'\u0385\u03d3\u03d4') u'\u0020\u0308\u0301\u038e\u03ab', # == NFKC(u'\u0385\u03d3\u03d4') u'\u1e9b\u1fc1\u1fcd\u1fce\u1fcf\u1fdd\u1fde\u1fdf\u1fed', One solution is to skip these files on all 'darwin' platforms, without losing more time on this issue. |
|||
| msg105651 - (view) | Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2010-05-13 18:43 | |
It prefer if it were clear why these tests fail (but don't have time to look into this at the moment). That is, I don't mind if the tests get disabled on darwin, but only if there is a documented reason for it because otherwise disabling the test might hide a real issue. |
|||
| msg105662 - (view) | Author: Mark Dickinson (mark.dickinson) * |
Date: 2010-05-13 21:47 | |
> It prefer if it were clear why these tests fail I agree. There's little point having tests if we're just going to disable them when they fail without understanding why they're failing first. At that point a green buildbot just means "Hey, we disabled all failing tests!", rather than being an indication that Python's working well on that platform. |
|||
| msg105669 - (view) | Author: Florent Xicluna (flox) * |
Date: 2010-05-13 23:42 | |
The gory details...
>>> import os, unicodedata
>>> tempdir = u'temp_pep277'
>>> name = u'\u0385\u03d3\u03d4'
>>> os.mkdir(tempdir)
>>> with open(os.path.join(tempdir, name), 'w') as f:
... f.write('foo')
>>> unicodedata.normalize('NFD', name)
u'\xa8\u0301\u03d2\u0301\u03d2\u0308'
>>> os.listdir(tempdir)
[u'\xa8\u0301\u03d2\u03d2\u0301\u0308']
The filename should use the Normal Decomposition (NFD) of the original filename. But the decomposition looks wrong on this platform.
The problem is similar for the other special filenames. Examples:
- u'\u0385\u03d3\u03d4'
==> NFD: u'\xa8\u0301\u03d2\u0301\u03d2\u0308'
==> OSX: u'\xa8\u0301\u03d2\u03d2\u0301\u0308'
- u' \u0308\u0301\u038e\u03ab'
==> NFD: u' \u0308\u0301\u03a5\u0301\u03a5\u0308'
==> OSX: u' \u0308\u0301\u03a5\u03a5\u0301\u0308'
- u'\u1e9b\u1fc1\u1fcd\u1fce\u1fcf\u1fdd\u1fde\u1fdf\u1fed'
==> NFD: u'\u017f\u0307\xa8\u0342\u1fbf\u0300\u1fbf\u0301\u1fbf\u0342\u1ffe\u0300\u1ffe\u0301\u1ffe\u0342\xa8\u0300'
==> OSX: u'\u017f\u0307\xa8\u0342\u1fbf\u0300\u1fbf\u0301\u1fbf\u1ffe\u1ffe\u1ffe\xa8\u0342\u0300\u0301\u0342\u0300'
I have no idea how to investigate further.
And the "PPC Tiger" buildbot does not fail here.
Reference about OSX decomposition:
http://developer.apple.com/mac/library/qa/qa2001/qa1173.html
|
|||
| msg105674 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2010-05-14 00:46 | |
> Reference about OSX decomposition: > http://developer.apple.com/mac/library/qa/qa2001/qa1173.html Mac OS X source code: http://www.opensource.apple.com/source/xnu/xnu-1504.3.12/bsd/vfs/vfs_utfconv.c?txt |
|||
| msg114280 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2010-08-18 21:34 | |
#8207 was the same issue and r79426 (for trunk and r79426 in py3k) was supposed to fix it. |
|||
| msg119663 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2010-10-27 00:37 | |
I opened the issue #10209: "Mac OS X: Decompose filenames on encode, and precompose filenames on decode". |
|||
| msg119829 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2010-10-29 00:08 | |
> I opened the issue #10209: "Mac OS X: Decompose filenames on encode, > and precompose filenames on decode". It was a bad idea. I fixed test_pep277 instead with: - r85897 disables the filenames that are normalized differently by Python and by darwin - r85899 disables test_normalize and test_listdir tests And test_pep277 now pass on "x86 Tiger 3.x" buildbot, and so I can close this issue (and issue #10209). |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:57:00 | admin | set | github: 52670 |
| 2010-10-29 00:08:14 | vstinner | set | status: open -> closed resolution: fixed messages: + msg119829 |
| 2010-10-27 00:37:33 | vstinner | set | messages: + msg119663 |
| 2010-09-25 11:12:53 | pitrou | set | nosy:
+ db3l |
| 2010-08-18 21:34:29 | vstinner | set | messages: + msg114280 |
| 2010-05-14 00:46:23 | vstinner | set | messages: + msg105674 |
| 2010-05-13 23:42:20 | flox | set | messages: + msg105669 |
| 2010-05-13 21:47:36 | mark.dickinson | set | messages: + msg105662 |
| 2010-05-13 18:43:31 | ronaldoussoren | set | messages: + msg105651 |
| 2010-05-13 17:56:25 | flox | set | assignee: ronaldoussoren -> messages: + msg105648 |
| 2010-05-13 17:31:02 | flox | set | nosy:
+ ned.deily, flox, mark.dickinson, ronaldoussoren, michael.foord versions: + Python 2.7 assignee: ronaldoussoren components: + macOS, Tests type: behavior stage: needs patch |
| 2010-04-18 08:02:45 | vstinner | set | messages: + msg103469 |
| 2010-04-16 18:59:54 | loewis | set | keywords: + buildbot |
| 2010-04-16 16:35:48 | vstinner | create | |