classification
Title: Failures in test_macostools for --enable-unicode=ucs4
Type: compile error Stage: patch review
Components: Macintosh, Unicode Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: ajaksu2, brett.cannon, christian.heimes, ezio.melotti, jackjansen, janssen, ronaldoussoren
Priority: low Keywords:

Created on 2003-07-01 06:54 by brett.cannon, last changed 2011-01-26 18:09 by brett.cannon. This issue is now closed.

Files
File name Uploaded Description Edit
issue763708.patch ronaldoussoren, 2010-10-20 13:58 review
Messages (25)
msg16750 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-07-01 06:54
Here are the test results:

test_copy (__main__.TestMacostools) ... ok
test_mkalias (__main__.TestMacostools) ... ERROR
test_mkalias_relative (__main__.TestMacostools) ... ERROR
test_touched (__main__.TestMacostools) ... ok

=====================================
=================================
ERROR: test_mkalias (__main__.TestMacostools)
--------------------------------------------------------------------
--
Traceback (most recent call last):
  File "Lib/test/test_macostools.py", line 69, in test_mkalias
    macostools.mkalias(test_support.TESTFN, TESTFN2)
  File "/Users/drifty/cvs_code/lib/python2.3/plat-mac/
macostools.py", line 46, in mkalias
    File.FSGetResourceForkName())
Error: (-1402, 'Fork name parameter is bad')

=====================================
=================================
ERROR: test_mkalias_relative (__main__.TestMacostools)
--------------------------------------------------------------------
--
Traceback (most recent call last):
  File "Lib/test/test_macostools.py", line 78, in 
test_mkalias_relative
    macostools.mkalias(test_support.TESTFN, TESTFN2, 
sys.prefix)
  File "/Users/drifty/cvs_code/lib/python2.3/plat-mac/
macostools.py", line 46, in mkalias
    File.FSGetResourceForkName())
Error: (-1402, 'Fork name parameter is bad')
msg16751 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-07-01 11:47
Logged In: YES 
user_id=45365

Brett, I've seen this bug once in a while, but whenever I try to 
hunt it it disappears. Could you give me the following info:
- OSX exact version
- Python exact version, plus where you got it from (binary install, 
source tarball install, CVS)
- Type of filesystem (HFS+, UFS, NFS, something else)

Also, if you're building from source, did you run the tests from the 
build tree or from the installed tree? Could you try the other one 
too?
msg16752 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-07-01 20:42
Logged In: YES 
user_id=357491

OK, Jack, here is your info:
  - 10.2.6
  - Python 2.3b2+ straight from CVS (only way to code =)
  - HFS+ I believe (Finder says my HD is Mac OS Extended)

Ran the test from my CVS tree with my installed version of Python 
2.3b2+ (I can't do anything the standard way).  I just ran it with 
the installed copy from the installed test directory and got the 
errors.  I also just ran it with the compiled copy but not installed 
Python executable in my CVS tree and once again got the error.

I noticed this came up, for me at least, when I was checking a 
patch for posixpath.py (which was applied to CVS).  I checked the 
code, though, and didn't find a problem where anything changed to 
posixpath.py would affect it.
msg16753 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-07-09 18:03
Logged In: YES 
user_id=357491

OK, so the problem seems to be coming from Mac/Modules/file/
_Filemodule.c and the File_FSGetResourceForkName function 
(printing the result just prints "ERROR").  The online docs for 
FSGetResourceForkName (which is pretty much all that is called in 
that function) can be found at http://developer.apple.com/
documentation/Carbon/Reference/File_Manager/file_manager/
function_group_20.html#//apple_ref/c/func/
FSGetResourceForkName .

Now it looks like the error is an OS X error and has nothing to do 
with Python.  The error (-1402) means that the fork name is 
syntactically invalid.  I don't see how this can mean anything, 
though, since FSGetResourceForkName's only argument is a 
pointer to store the result of the call into.

The  only thing I can think of that might be causing this error from 
Python's side is that a resource fork does not exist when the call is 
made.  But this is just a guess so I could be wrong.

Jack, I am going to be gone from July 13 until July 21, so if  you 
need any debugging info from me before 2.3 final goes out I am 
afraid it will have to happen soon.
msg16754 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-07-20 00:30
Logged In: YES 
user_id=45365

This seems to be fixed in 2.3c1. Could you please test?
msg16755 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-07-22 06:48
Logged In: YES 
user_id=357491

Tested and still fails with a CVS update on 2003-07-21.
msg16756 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-07-22 15:22
Logged In: YES 
user_id=45365

Brett, please try the following:
>>> import Carbon.File
>>> Carbon.File.FSGetResourceForkName()
u'RESOURCE_FORK'
>>> 

You mention it prints ERROR for you, I'd like to see exactly what it 
prints (u"ERROR"? "ERROR"? ERROR?).

And a bit more information I need to try and duplicate the 
problem:
1. Are you using a framework build or a straight build?
2. Is your /Users/drifty directory in yoor root partition?
3. Could you check that there are no test_support.TESTFN or 
test_support.TESTFN + '2' turds on your system?
msg16757 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-07-22 18:42
Logged In: YES 
user_id=357491

The Carbon.File.FSGetResourceForkName() returns 
u'\U00520045\U0053004f\U00550052\U00430045\U005f0046\U004f
0052\U004b0000\U0001bfff\Uf4100000\U00080040\Ue4589000\U5
334bfff\Uf4200000' .  That can't be printed because of a 
conversion to ASCII error.  If I remember correctly this is what 
was returning the string "ERROR" (or however it was formatted).  
There is also the ``Error: (-1402, 'Fork name parameter is bad'`` 
from the failed tests.

And here is the info you wanted, Jack:

1. My build has the settings --with-pydebug --prefix=$HOME/
cvs_code --enable-ipv6 --enable-unicode=ucs4 ; so it's a straight 
build.
2. I think so.  I also have an OS 9 partition that came with my 
iBook that I never touch.
3. Nope, no stray TESTFN files.  The  only file I have in my CVS 
directory that is not in the repository is Lib/plat-mac/
errors.rsrc.df.rsrc .

Hope that helps.
msg16758 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-07-22 19:35
Logged In: YES 
user_id=45365

Boo, hiss! :-)

You're using --enable-unicode=ucs4 and you didn't tell me that 
when there's a unicode problem:-)

My guess is that I'm somewhere grabbing the pointer from the 
Python unicode object and passing that straight to the Apple 
routines, which will fail miserably because they expect ucs2 (or 
utf16, or whatever it's called). Actually, I wouldn't be surprised if 
there are more places where I do that. Yes, looking at the data 
returned this looks like a very likely scenario.
msg16759 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-07-22 21:27
Logged In: YES 
user_id=357491

Sorry, Jack.  I didn't know that this was a Unicode issue.  Oh well, 
at least the cause has been narrowed down.
msg16760 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2004-12-18 21:40
Logged In: YES 
user_id=357491

This test still fails in 2.4 and so far in 2.5 .  Does this still deserve a 
"later" resolution?
msg16761 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2004-12-19 00:21
Logged In: YES 
user_id=45365

I'd like to keep this bug open, because the underlying problem is serious: 
the Python interfaces to the MacOSX APIs assume that the binary 
representation of "python unicode" is identical to the binary 
representation of "macosx unicode". This needs to be fixed at some 
point, but such a fix requires major surgery (especially if we want things 
to be efficient for the normal case, where the assumption indeed holds).
msg16762 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2007-05-31 22:49
This is still failing in the trunk (r55677).  Any chance this is going to be addressed soon?
msg16763 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2007-06-01 07:23
Hi Brett!
This bug turns in to a once-a-year "how are you " conversation:-)

I haven't checked whether unicode objects have grown the required method in the mean time. If they have this problem could be solved, otherwise it can't (at least, not without serious hacking).
But: even if the APIs are available that woulnd't mean that I have time to look at the problem in the foreseeable future...
msg59206 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-04 01:32
Happy New Year Jack! How are you :)

*scnr*
msg81854 - (view) Author: Daniel Diniz (ajaksu2) Date: 2009-02-13 02:49
Happy new year :)
msg83152 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-04 22:36
IMHO fixing this is not worth the trouble, we should just document that 
the Carbon extensions aren't supported in a UCS4 build (or even explictly 
detect a UCS4 build in setup.py and not compile the Carbon extensions).
msg83157 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2009-03-04 22:55
The code solution is probably the most useful since people are more
likely to just compile Python without reading some text file that will
mention UCS4 builds will not create the Carbon extensions.
msg84646 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-30 21:01
The attached file issue763708.patch changes configure.in and disables the 
toolbox glue when doing a UCS4 build.

To get this behaviour I had to move a couple of sections around inside 
configure.in (--enable-toolbox-glue was checked for before the size of 
unicode characters was known).
msg84667 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2009-03-30 21:53
Patch looks fine to me.
msg84679 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-03-30 22:10
I'm not yet sure that patch is correct, I got some build failure during 
later tests that went away when I reverted this patch. To be continued...
msg108754 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2010-06-26 22:40
Is this still a problem? None of the buildbot are reporting failures in test_macostools.
msg108781 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-06-27 12:01
This still is a problem, the Carbon wrappers assume that Py_UNICODE has the same representation as the Unicode type in Carbon and that is a 16-bit type.
msg119207 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2010-10-20 13:58
I've attached a new patch that works for me.

The new patch doesn't try to warn when running the configure script, but bails out when you run make by using an '#error' in pymacconfig.h.

(Removing 2.6 because that's in security-fix-only mode and 3.1 because this issue only affects 2.x)
msg127126 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-01-26 18:09
Since the whole Mac directory got gutted in the switch over to Python 3, macostools is now gone, making this out of date.
History
Date User Action Args
2011-01-26 18:09:25brett.cannonsetstatus: languishing -> closed

messages: + msg127126
resolution: out of date
nosy: brett.cannon, jackjansen, ronaldoussoren, janssen, christian.heimes, ajaksu2, ezio.melotti
2010-10-20 13:58:47ronaldoussorensetkeywords: - patch, 26backport, needs review
files: + issue763708.patch
messages: + msg119207

versions: - Python 2.6, Python 3.1
2010-10-20 13:56:33ronaldoussorensetfiles: - issue763708.patch
2010-06-27 12:01:21ronaldoussorensetmessages: + msg108781
2010-06-26 23:03:11ezio.melottisetnosy: + janssen
2010-06-26 22:42:44ezio.melottisetstatus: open -> languishing
2010-06-26 22:40:36ezio.melottisetnosy: + ezio.melotti
messages: + msg108754
2009-03-30 22:10:10ronaldoussorensetmessages: + msg84679
2009-03-30 21:53:25brett.cannonsetresolution: later -> (no value)
messages: + msg84667

assignee: jackjansen -> ronaldoussoren
type: behavior -> compile error
stage: test needed -> patch review
2009-03-30 21:01:36ronaldoussorensetkeywords: + 26backport, needs review, patch
files: + issue763708.patch
messages: + msg84646

versions: + Python 3.1, Python 2.7
2009-03-04 22:55:34brett.cannonsetmessages: + msg83157
2009-03-04 22:36:32ronaldoussorensetnosy: + ronaldoussoren
messages: + msg83152
2009-02-13 02:49:37ajaksu2setversions: - Python 2.5
nosy: + ajaksu2
messages: + msg81854
components: + Unicode
type: behavior
stage: test needed
2008-01-04 01:32:32christian.heimessetnosy: + christian.heimes
messages: + msg59206
versions: + Python 2.6, Python 2.5, - Python 2.3
2003-07-01 06:54:15brett.cannoncreate