classification
Title: When the package has non-ascii path and .pyc file, we cannot import them.
Type: behavior Stage:
Components: Unicode Versions: Python 3.0
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Suzumizaki, ezio.melotti, pitrou, terry.reedy
Priority: normal Keywords:

Created on 2009-05-25 01:56 by Suzumizaki, last changed 2009-05-29 19:26 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
testcode.zip Suzumizaki, 2009-05-25 01:56 testcode and screenshots.
Messages (5)
msg88299 - (view) Author: Suzumizaki (Suzumizaki) Date: 2009-05-25 01:56
When the path of the package has non-ascii characters, importing such
packages always fails except the first time (in other words, always 
fails when compiled .pyc file exists).

The problem exists even the names of such modules only consist of
us-ascii characters. Please follow the test code I create, which can
test the problem even with English version Windows(, I expected).

Target: Windows XP (I tested Home Ed., SP3, 32bit, Japanase version)
Python: 3.0.1

The priority of this issue might be CRITICAL or higher by 2 reasons:
1) When the end-users meet this condition, the users cannot find what's
wrong easily, because UnicodeDecodeError is raised (not ImportError).

2) On non-English version Windows, "Desktop", "My Documents" or etc. 
folders have non-English name as default. For example, Katakana is used
in Japanese version. This means all python packages in the desktop meet
 this problem for not a few users.
msg88349 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2009-05-26 10:04
Your problem could be related to #3297, but katakana characters don't
have a codepoint greater than U+FFFF so it could be something else.
(What is the value of sys.maxunicode there though?)

I tried your script on Linux with Python3.1b1 and en_US.UTF-8 but I
wasn't able to reproduce the issue.
I ran the script from the terminal with "python3 test.py" twice and it
worked both the time (is this the correct way to use the script?).
Can you try with Python3.1 and see if it works?

I also tried to create a directory called デスクトップ with an
__init__.py in it, I imported it and reloaded and it worked properly,
even when the .pyc was there.

If you delete the .pyc an reload the script again, does it work?

(Next time upload the files separately, it's easier for us to see them.)
msg88410 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-05-27 10:57
Could you test with the 3.1 beta? There's a Windows installer here:
http://python.org/download/releases/3.1/
msg88414 - (view) Author: Suzumizaki (Suzumizaki) Date: 2009-05-27 11:33
Thanks for reply.

I tried Python 3.1 Beta, and the testcode goes fine.
(and my current work also.) 

I checked with Windows XP, I told before.
My problem seems resolved, and here is answers only to make sure.

>(What is the value of sys.maxunicode there though?)

65535.

> I ran the script from the terminal with "python3 test.py" twice and it
> worked both the time (is this the correct way to use the script?).

Yes.

> If you delete the .pyc an reload the script again, does it work?

You mean "and" insted of "an", right?
The answer is Yes, even Python 3.0.1.

> (Next time upload the files separately, it's easier for us to see them.)

OK, I know.

Thank you very much for all.
msg88523 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2009-05-29 19:26
Am closing based on 'My problem seems resolved'.
I don't know whether it will be fixed in 3.0.2 or not,
but that is pretty much a moot point.
History
Date User Action Args
2009-05-29 19:26:51terry.reedysetstatus: open -> closed

nosy: + terry.reedy
messages: + msg88523

resolution: fixed
2009-05-27 11:33:57Suzumizakisetmessages: + msg88414
2009-05-27 10:57:29pitrousetnosy: + pitrou
messages: + msg88410
2009-05-26 10:04:59ezio.melottisetnosy: + ezio.melotti
messages: + msg88349
components: + Unicode, - None
2009-05-25 01:56:55Suzumizakicreate