classification
Title: c_char_p return value returns string, not bytes
Type: behavior Stage: resolved
Components: ctypes Versions: Python 3.1
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: theller Nosy List: benjamin.peterson, georg.brandl, kaizhu, theller, vstinner
Priority: critical Keywords: patch

Created on 2009-06-08 16:07 by georg.brandl, last changed 2009-09-08 16:53 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
c_char_p.patch theller, 2009-07-31 20:11 Patch against py3k branch
_asciiporn.c kaizhu, 2009-08-22 07:20 more motivation to fix bug
_asciiporn.h kaizhu, 2009-08-22 07:22
Messages (13)
msg89086 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-06-08 16:07
If you assign a function a restype of c_char_p, you get back a Unicode
string, but you should get a bytes object.

>>> from ctypes import *
>>> strchr = cdll['libc.so.6'].strchr
>>> strchr.restype = c_char_p
>>> strchr.argtypes = [c_char_p, c_char]
>>> strchr(b'abcde', b'd')
'de'
msg91149 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2009-07-31 20:11
Here's a patch.
msg91581 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-08-14 23:07
I agree that the C type 'char' is a byte, not a character. So Python3
should creates a Python bytes object for the ctypes "c_char_p" type.
Since Python 3.1 is out, is it too late to change it in the 3.x branch?
:-) Maybe for Python 3.2?
msg91600 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2009-08-15 09:55
> I agree that the C type 'char' is a byte, not a character. So Python3
> should creates a Python bytes object for the ctypes "c_char_p" type.
> Since Python 3.1 is out, is it too late to change it in the 3.x branch?
> :-) Maybe for Python 3.2?

I think the current behaviour is clearly a bug, so I would apply
this patch to Python 3.1 as well.  Any comments?  Who will decide that?
msg91608 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-08-15 14:03
I agree that it's a bug.  Letting Benjamin decide whether to put it in
3.1.1 :)
msg91609 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-08-15 14:07
Sounds ok to me.

2009/8/15 Georg Brandl <report@bugs.python.org>:
>
> Georg Brandl <georg@python.org> added the comment:
>
> I agree that it's a bug.  Letting Benjamin decide whether to put it in
> 3.1.1 :)
>
> ----------
> assignee: theller -> benjamin.peterson
> nosy: +benjamin.peterson
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue6239>
> _______________________________________
>
msg91611 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-08-15 17:10
> I agree that it's a bug.
> Letting Benjamin decide whether to put it in 3.1.1 :)

And what about 3.0.x? Is this branch still alive or not?
msg91612 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-08-15 17:14
2009/8/15 STINNER Victor <report@bugs.python.org>:
>
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
>
>> I agree that it's a bug.
>> Letting Benjamin decide whether to put it in 3.1.1 :)
>
> And what about 3.0.x? Is this branch still alive or not?

No, it's as dead as a doornail.
msg91805 - (view) Author: kai zhu (kaizhu) Date: 2009-08-21 07:13
i just found this bug independently, but yes its a bug, which i hope 
gets fixed for sake of extension community:

// test.c beg
char s[4] = "ab\xff";
char *foo() { return s; }
// test.c end

$ gcc -fPIC -g -c -Wall test.c
$ gcc -shared test.o -o test.so
$ python3.1 -c "import ctypes; lib = 
ctypes.cdll.LoadLibrary('./test.so'); lib.foo.restype = ctypes.c_char_p; 
lib.foo()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 2: 
unexpected code byte
msg91856 - (view) Author: kai zhu (kaizhu) Date: 2009-08-22 07:20
wrote an extension application which relies on the patch (works after 
applying patch to python 3.1.1).

it converts png images to colorized ascii-art on ansi-compatible 
terminal.  requires the patch b/c a ctype function returns a c-string w/ 
ansi-escape characters.

>>> import ctypes
>>> lib = ctypes.cdll.LoadLibrary("_asciiporn.so")
>>> lib.img_read(b"foo.png") // load png image
>>> lib.asc_itp(4, 16)       // ascii-rize algorithm
>>> lib.asc_str.restype = ctypes.c_char_p
>>> print( lib.asc_str() )   // prints out ansi-colorized ascii-art

hopefully, this is more motivation to commit the patch to trunk
msg92128 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-09-01 07:59
Assigning back to Thomas for commit.
msg92261 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2009-09-04 18:39
Commited as svn rev 74664 (py3k) and rev 74665 (release31-maint).
msg92424 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-09-08 16:53
> Commited as svn rev 74664 (py3k) and rev 74665 (release31-maint).

@theller: Cool, thanks ;-)
History
Date User Action Args
2009-09-08 16:53:01vstinnersetmessages: + msg92424
2009-09-04 18:39:09thellersetstatus: open -> closed

messages: + msg92261
stage: needs patch -> resolved
2009-09-01 07:59:23georg.brandlsetassignee: benjamin.peterson -> theller
resolution: accepted
messages: + msg92128
2009-08-22 07:22:09kaizhusetfiles: + _asciiporn.h
2009-08-22 07:20:07kaizhusetfiles: + _asciiporn.c

messages: + msg91856
2009-08-21 07:13:55kaizhusetnosy: + kaizhu
messages: + msg91805
2009-08-15 17:14:39benjamin.petersonsetmessages: + msg91612
2009-08-15 17:10:29vstinnersetmessages: + msg91611
2009-08-15 14:07:15benjamin.petersonsetmessages: + msg91609
2009-08-15 14:03:46georg.brandlsetassignee: theller -> benjamin.peterson

messages: + msg91608
nosy: + benjamin.peterson
2009-08-15 09:56:00thellersetmessages: + msg91600
2009-08-14 23:07:09vstinnersetnosy: + vstinner
messages: + msg91581
2009-07-31 20:11:25thellersetkeywords: + patch
files: + c_char_p.patch
messages: + msg91149
2009-06-08 16:07:28georg.brandlcreate