classification
Title: ctypes s_set() uses strlen() and so truncates string at null character
Type: enhancement Stage: patch review
Components: Tests Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: shihai1991, vstinner
Priority: normal Keywords: patch

Created on 2020-02-09 16:17 by shihai1991, last changed 2020-02-11 09:04 by shihai1991.

Pull Requests
URL Status Linked Edit
PR 18424 open shihai1991, 2020-02-09 16:19
Messages (5)
msg361656 - (view) Author: hai shi (shihai1991) * Date: 2020-02-09 16:17
strlen(data) can not be replaced by Py_SIZE(value) in https://github.com/python/cpython/blob/master/Modules/_ctypes/cfield.c#L1297.

victor have give a great example about it in https://github.com/python/cpython/pull/18419

I create this bpo for two reasons:
1. This question info could be removed(some questions info will catch my attention´╝ë
2. Current tests should be enhanced(It can not help me found this backward incompatible:( ).
msg361690 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-02-10 12:57
Using strlen() seems to be as old as ctypes itself.

--

I don't know which behavior is correct. It's a little bit strange that ctypes truncate the string at "\0". A "char" buffer can be an arbitrary array of bytes. It may be a binary format which doesn't use null byte as string terminator, but just to encode a 16-bit integer as two bytes.

My attempt to understand the current behavior:

https://github.com/python/cpython/pull/18419#pullrequestreview-355606890

"size = strlen(data);" instruction was added when the ctypes was added by this commit:

commit d4c9320412177895f598a93d73a0e654db27c351
Author: Thomas Heller <theller@ctypes.org>
Date:   Wed Mar 8 19:35:11 2006 +0000

    Copy ctypes-0.9.9.4 sources from external into the trunk.

Sadly, Thomas Heller no longer contributes to Python since 2011:
https://blog.python.org/2011/04/thomas-heller-steps-down-as-ctypes.html

The original project is hosted at:
https://sourceforge.net/p/ctypes/code/

It seems like s_set() function was added between these source/cfield.c two versions:

revision 1.116
date: 2006/03/15 20:35:55;  author: theller;  state: Exp;  lines: +14 -4
PyString_FromFormat()b understands the C99 "z" qualifier on all
platforms, in Python 2.5.  Patch from Tim Peters, adapted to work with
older Python versions.

revision 1.115
date: 2006/03/03 20:17:15;  author: theller;  state: Exp;  lines: +636 -279
Moving files from branch_1_0 to HEAD.

Sadly, the commit message doesn't say much about the rationale.
msg361782 - (view) Author: hai shi (shihai1991) * Date: 2020-02-11 06:31
releated bpo: issue12769

s_get() in cfield.c function have similar behavior. So far, this is a planned action(lack evidence).
msg361789 - (view) Author: hai shi (shihai1991) * Date: 2020-02-11 08:47
I am not sure it have realtion of libffi's type or not?

`s_set()` and `s_get()`'s type of ffi is `FFI_TYPE_POINTER`;

REF: https://www.manpagez.com/info/libffi/libffi-3.0.13/libffi_6.php#index-ffi_005ftype_005fpointer
msg361790 - (view) Author: hai shi (shihai1991) * Date: 2020-02-11 09:04
I guess the `xx_get()` and `xx_set()` function's process logic have relation with ffi's type, after I checked some `xx_get()` and `xx_set()`'s process logic.
History
Date User Action Args
2020-02-11 09:04:18shihai1991setmessages: + msg361790
2020-02-11 08:47:43shihai1991setmessages: + msg361789
2020-02-11 06:31:28shihai1991setmessages: + msg361782
2020-02-10 12:58:03vstinnersettitle: Adding a unit test of ctypes -> ctypes s_set() uses strlen() and so truncates string at null character
2020-02-10 12:57:06vstinnersetnosy: + vstinner
messages: + msg361690
2020-02-09 16:19:39shihai1991setkeywords: + patch
stage: patch review
pull_requests: + pull_request17801
2020-02-09 16:17:25shihai1991create