classification
Title: Python 3.x ctypes c_wchar_p return is different from official documentation example
Type: behavior Stage: resolved
Components: ctypes, Documentation Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Alex Wang, berker.peksag, docs@python, eryksun
Priority: normal Keywords: easy

Created on 2016-11-15 17:23 by Alex Wang, last changed 2017-04-26 08:48 by berker.peksag. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1160 merged louielu, 2017-04-16 05:56
PR 1291 merged louielu, 2017-04-26 08:20
PR 1292 merged louielu, 2017-04-26 08:21
Messages (7)
msg280869 - (view) Author: Alex Wang (Alex Wang) Date: 2016-11-15 17:23
- Python Version

Python 3.5.2

- Issue

I found that the c_wchar_p and c_char_p return results behaves different from what it is based on ctypes doc. From the ctypes doc of Python 3.5:

>>> c_wchar_p("Hello, World")
c_wchar_p('Hello, World')

It return the ctypes string. But my results of execute the same cmd in Python3.5 console:

>>> from ctypes import *
>>> c_wchar_p("Hello, World")
c_wchar_p(1374004842736)
>>> c_wchar_p("Hello, World")
c_wchar_p(1374004841680)
>>> c_wchar_p("Hello, World")
c_wchar_p(1374004842736)

So seems like the orignial string part replaced by memory address. Digged in more, and found out if it is Python 2.x, then the return shows the string like the Python 3.5 ctypes doc shows. But in Python 3.x, it always return these numbers. Checked on multiple PCs, all seen the same thing. And understood the part that, we can use .value to return the original string. Same thing observed on create_string_buffer() and create_unicode_buffer(). Meanwhile, for other data types like c_int(), c_bool, c_void_p etc., not see this.

- Question

Can anyone provide a explaination about this behavior of ctypes?
And is there any way to fix the Python3.x return resuls as the same as what is doc wrote? (It seems that when it behave like this, I had issue with passing Python string to C function, when I interact with a C DLL.)

- Repro

This can be reproduce in python.org/shell easily.

Thanks a lot in advance~
msg280875 - (view) Author: Eryk Sun (eryksun) * Date: 2016-11-15 18:53
The repr can't automatically dereference the string at the address because it may be an invalid pointer. ctypes was developed on Windows, for which, in Python 2, it (ab)uses the obsolete IsBadStringPtr[A|W] function [1]. This function should never be used in a multithreaded process. 

On POSIX systems, the repr of c_char_p was special-cased to avoid dereferencing the pointer, but c_wchar_p was overlooked, and you can still easily crash Python 2 like this:

    Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
    [GCC 5.4.0 20160609] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import ctypes
    >>> ctypes.c_wchar_p(1)
    Segmentation fault (core dumped

A while back the bogus use of WinAPI IsBadStringPtr was removed from the Python 3 branch, but apparently the docs weren't updated to reflect this change. I'm changing this to a documentation issue.

[1]: https://msdn.microsoft.com/en-us/library/aa366714
msg280876 - (view) Author: Alex Wang (Alex Wang) Date: 2016-11-15 19:24
Hi Eryk,

Thanks a lot for quick reply~

This is about the bug I filed: http://bugs.python.org/issue28698#

I may still need your help to have a look the original case when I caught
this issue:

​I am writing some test automation which call C DLL from Python, the C
function is something like:

MEASURE_API int InitTester(char *ipAddress)

​So I need to pass an IP address string (for example, 192.168.100.100) from
Python in ctypes to this function. For non-const char in C, I used

c_ipAddress = create_string_buffer(b'192.168.100.100')
lib.InitTester(c_ipAddress)

​But error code returned indicate that the parameter passing is incorrect,
then I traced back and found then reported the c_char_p/c_wchar_p issue.​

Also tried
​c_ipAddress = create_unicode_buffer('192.168.100.100')
c_ipAddress = c_char_p(b'192.168.100.100')
c_ipAddress = c_wchar_p('192.168.100.100')​

​But none of them working... I had called other function to this C DLL
passing c_int(). c_bool(), c_void_p() and etc. they are all working as
expected, only string related have this issue.

Therefore, any idea how write the correct assignment and pass it to ​C DLL
for this case in Python 3.5? Any hint would be great helpful.

Thank you in advance~

BR,
Alex

On Tue, Nov 15, 2016 at 10:57 AM, Eryk Sun <report@bugs.python.org> wrote:

>
> Changes by Eryk Sun <eryksun+pybugs@gmail.com>:
>
>
> ----------
> keywords: +easy
> stage:  -> needs patch
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28698>
> _______________________________________
>
msg292317 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-04-26 08:15
New changeset 0d637e236d7099f7b724026c8cb7bd83d8e12e6b by Berker Peksag (Louie Lu) in branch 'master':
bpo-28698: Fix c_wchar_p doc example (GH-1160)
https://github.com/python/cpython/commit/0d637e236d7099f7b724026c8cb7bd83d8e12e6b
msg292321 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-04-26 08:45
New changeset 9f6828119df23fca9e799f47e02baabe87adca2a by Berker Peksag (Louie Lu) in branch '3.6':
bpo-28698: Fix c_wchar_p doc example (GH-1160)
https://github.com/python/cpython/commit/9f6828119df23fca9e799f47e02baabe87adca2a
msg292322 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-04-26 08:47
New changeset c7b8367076dc7771dabcb9491bd98218c788d489 by Berker Peksag (Louie Lu) in branch '3.5':
bpo-28698: Fix c_wchar_p doc example (GH-1160)
https://github.com/python/cpython/commit/c7b8367076dc7771dabcb9491bd98218c788d489
msg292323 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-04-26 08:48
Thanks for the report, Alex and thanks for the PRs, Louie!
History
Date User Action Args
2017-04-26 08:48:40berker.peksagsetstatus: open -> closed
resolution: fixed
messages: + msg292323

stage: patch review -> resolved
2017-04-26 08:47:05berker.peksagsetmessages: + msg292322
2017-04-26 08:45:33berker.peksagsetmessages: + msg292321
2017-04-26 08:21:28louielusetpull_requests: + pull_request1399
2017-04-26 08:20:18louielusetpull_requests: + pull_request1398
2017-04-26 08:15:09berker.peksagsetnosy: + berker.peksag
messages: + msg292317
2017-04-19 11:07:42berker.peksagsetstage: needs patch -> patch review
2017-04-16 05:56:14louielusetpull_requests: + pull_request1289
2016-11-15 19:53:32eryksunsetmessages: - msg280878
2016-11-15 19:40:47eryksunsetmessages: + msg280878
2016-11-15 19:24:19Alex Wangsetmessages: + msg280876
2016-11-15 18:57:57eryksunsetkeywords: + easy
stage: needs patch
2016-11-15 18:53:12eryksunsetversions: + Python 3.6, Python 3.7
nosy: + docs@python, eryksun

messages: + msg280875

assignee: docs@python
components: + Documentation
2016-11-15 17:23:56Alex Wangcreate