Issue23606
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2015-03-08 04:41 by steve.dower, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Messages (25) | |||
---|---|---|---|
msg237510 - (view) | Author: Steve Dower (steve.dower) * | Date: 2015-03-08 04:41 | |
With the changes to the CRT on Windows, it no longer makes any sense to call find_library("c") or ("m") as there is no single C Runtime DLL. The new structure has a grouped and layered approach that is better for versioning, so we now link to "api-ms-win-crt-*-l1-1-0.dll" where "*" is something like "filesystem", "heap", "locale", "math", "string", etc. and the "l1-1-0" part is a version. I don't know what the intended purpose of this function is, so I can't suggest a good replacement other than very fast deprecation in 3.4 and raising an error in 3.5. Any better suggestions? |
|||
msg237511 - (view) | Author: Eryk Sun (eryksun) * | Date: 2015-03-08 05:50 | |
Shouldn't find_library("c") return "ucrtbase.dll" or "ucrtbased.dll" (debug)? Introducing the Universal CRT http://blogs.msdn.com/b/vcblog/archive/2015/03/03/introducing-the-universal-crt.aspx |
|||
msg237533 - (view) | Author: Steve Dower (steve.dower) * | Date: 2015-03-08 13:36 | |
That was my original thought, but it's going to lose all of its named exports and effectively become useless for this. |
|||
msg237687 - (view) | Author: Eryk Sun (eryksun) * | Date: 2015-03-09 18:29 | |
The api-ms-win-crt-* DLLs forward their exports to ucrtbase.dll, which currently uses named exports. When is it planned to remove the named exports? >>> crt = CDLL('ucrtbase') >>> filesystem = CDLL('api-ms-win-crt-filesystem-l1-1-0') >>> math = CDLL('api-ms-win-crt-math-l1-1-0') >>> c_void_p.from_buffer(crt._stat64) c_void_p(8791677890384) >>> c_void_p.from_buffer(filesystem._stat64) c_void_p(8791677890384) >>> c_void_p.from_buffer(crt.fabs) c_void_p(8791678417616) >>> c_void_p.from_buffer(math.fabs) c_void_p(8791678417616) |
|||
msg237689 - (view) | Author: Steve Dower (steve.dower) * | Date: 2015-03-09 18:39 | |
AIUI, by the time Windows 10 or Visual Studio 2015 releases (since it's now a Windows component, it's technically on a different schedule, even though the main developer is still working against Visual Studio's schedule) and probably sooner (VS 2015 RC is most likely). It's always possible that they'll decide not to remove them, but the current plan is that loading ucrtbase.dll directly will not be helpful, since it would prevent Windows from making breaking changes to the API that the api-*.dll files would mask. |
|||
msg237731 - (view) | Author: Eryk Sun (eryksun) * | Date: 2015-03-10 06:53 | |
Will manual forwarding eventually be replaced by the loader's ApiSetMap (in the process PEB), i.e. will ucrtbase join ntdll, sechost, kernelbase, and kernel32 in apisetschema.dll? In this case will the CRT API sets only exist in winsxs, i.e. will they no longer be hard linked in System32? What about debug builds? python35_d.dll links directly to ucrtbased.dll: C:\Program Files\Python35>dumpbin /dependents python35_d.dll | find /i ".dll" Dump of file python35_d.dll WS2_32.dll KERNEL32.dll ADVAPI32.dll VCRUNTIME140D.dll ucrtbased.dll |
|||
msg237773 - (view) | Author: Steve Dower (steve.dower) * | Date: 2015-03-10 15:26 | |
Good question, I'm not sure. Right now, the api-* DLLs use forwarding rather than apisetschema, but that could change. Since they support back to XP though, I suspect it probably won't the api-* DLLs will stay as they are with ucrtbase exporting ordinals rather than names. ucrtbased is probably going to stay as it is. When you install a debug runtime you're voluntarily giving up well-defined versioning. You'll only have that file installed if you have VS 2015 or later though - it won't be part of the updates for everyone. |
|||
msg237780 - (view) | Author: Roundup Robot (python-dev) | Date: 2015-03-10 16:57 | |
New changeset 86c9ef950288 by Steve Dower in branch 'default': Issue #23606: Disable ctypes.util.find_library("c") on Windows so tests are skipped while we figure out how best to approach the CRT change https://hg.python.org/cpython/rev/86c9ef950288 |
|||
msg237781 - (view) | Author: Steve Dower (steve.dower) * | Date: 2015-03-10 17:01 | |
That change should get the buildbots passing again, as it will now skip those tests. The "appcrt%d" return value was blatantly incorrect anyway. |
|||
msg238095 - (view) | Author: Roundup Robot (python-dev) | Date: 2015-03-14 18:49 | |
New changeset fabbe6093567 by Steve Dower in branch 'default': Issue #23606: Temporarily suppress test for CRT name. https://hg.python.org/cpython/rev/fabbe6093567 |
|||
msg238339 - (view) | Author: Eryk Sun (eryksun) * | Date: 2015-03-17 21:16 | |
Say I need to use ctypes to call _wsopen_s to open a file without write sharing. If I read you correctly, you're saying I'll need to know it's exported by api-ms-win-crt-stdio-l1-1-0.dll? Does the 'l1-1-0' suffix reflect a version number that will be incremented over time? If so, how about adding an API that maps 'stdio' to the version that Python is linked against? |
|||
msg238349 - (view) | Author: Steve Dower (steve.dower) * | Date: 2015-03-17 22:09 | |
Pretty much, except the entry point DLL version won't increment unless there's a breaking change to the API. So you have to know where it's from, but (AIUI) the l1-0-0 file will always be available. At some point it may turn into a wrapper rather than a redirect, but that doesn't really matter. I'm not sure there's any better way to provide access to the functions other than adding them to msvcrtmodule. It's more work, but far more friendly. I hope we have a pretty low bar for adding functions to that module. |
|||
msg248838 - (view) | Author: Steve Dower (steve.dower) * | Date: 2015-08-19 13:12 | |
I've been ignoring this because I wasn't assigned... Here's the options. If we make it load ucrtbase.dll directly (which does still have named exports, and also uses the API schema on Windows 10): * some code that uses it will need updating (due to API changes since VC9/VC10) * code may need to change depending on OS updated (which could change the ucrtbase exports) * much existing code using this will probably work If we leave it as is: * all existing uses will obviously fail, nothing subtle * users will either substitute the name themselves, find an equivalent stdlib function, or use a supported Windows API Having just written that out, I still think not supporting it is best, but I do need to write it up in the porting info still. Any other thoughts? |
|||
msg251759 - (view) | Author: Eryk Sun (eryksun) * | Date: 2015-09-28 10:40 | |
> some code that uses it will need updating (due to API changes > since VC9/VC10) For example, I discovered that the stdio exports don't include printf, etc. Using __stdio_common_vfprintf to implement printf relies on calling __acrt_iob_func to get stdout and (in general) dynamically building a va_list argument list. Of course, relying on either operation is a bad idea. |
|||
msg258254 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2016-01-15 00:11 | |
Does it mean `cdll.msvcrt` is not the standard way to access the C symbols anymore? Could you update the docs to reflect the current guidelines? (https://docs.python.org/3/library/ctypes.html) |
|||
msg258255 - (view) | Author: Steve Dower (steve.dower) * | Date: 2016-01-15 00:27 | |
Strictly there's nothing incorrect about the docs, and `cdll.msvcrt` is no more incorrect than it has been since Python 2.4 or so (whenever we stopped using VC6). It will find the DLL and you can call the functions, but it isn't the same DLL as the exports in the `msvcrt` module will call. Might be worth a note pointing this out, but it's probably better off with a different example. I'll try and think of something, but I'm fairly distracted at the moment and would appreciate suggestions. Otherwise I'll get there eventually. |
|||
msg258282 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2016-01-15 09:10 | |
> Strictly there's nothing incorrect about the docs, and `cdll.msvcrt` is no more incorrect than it has been since Python 2.4 or so (whenever we stopped using VC6). It will find the DLL and you can call the functions, but it isn't the same DLL as the exports in the `msvcrt` module will call. Is there a documentation of what functions are exposed through it? |
|||
msg258295 - (view) | Author: Steve Dower (steve.dower) * | Date: 2016-01-15 14:02 | |
The msvcrt module? I don't think so. |
|||
msg260079 - (view) | Author: Eryk Sun (eryksun) * | Date: 2016-02-11 09:32 | |
If the examples continue to use printf, then msvcrt.dll is the best option. The universal CRT exports a single Swiss-Army-knife function, __stdio_common_vfprintf, which requires 5 parameters, including a va_list for the variadic argument list. That's not appropriate for a tutorial. We just need a docs update to warn that msvcrt.dll has its own set of file descriptors, heap, and thread-locale state. > Does it mean `cdll.msvcrt` is not the standard way to access > the C symbols anymore? On a tangent, cdll.msvcrt shouldn't be recommended anywhere in the ctypes docs. All code that uses cdll.msvcrt shares the same restype, argtypes, and errcheck prototypes for each function pointer, which is an unholy mess. The docs should recommend CDLL('mscvrt') or CDLL('msvcrt', use_errno=True). Every module or package should use its own private instance of CDLL. |
|||
msg261659 - (view) | Author: Roundup Robot (python-dev) | Date: 2016-03-12 16:26 | |
New changeset f9dc71b566fb by Steve Dower in branch '3.5': Issue #23606: Adds note to ctypes documentation regarding cdll.msvcrt. https://hg.python.org/cpython/rev/f9dc71b566fb New changeset 6d84fe4d8cb0 by Steve Dower in branch 'default': Issue #23606: Adds note to ctypes documentation regarding cdll.msvcrt. https://hg.python.org/cpython/rev/6d84fe4d8cb0 |
|||
msg261660 - (view) | Author: Steve Dower (steve.dower) * | Date: 2016-03-12 16:30 | |
That's a bit too much of a tangent for me to note in-place in the docs, but it'd be good content for a "Best practices" section (I seem to recall we had one somewhere, that basically started with "avoid ctypes if possible"...) Given we've heard no feedback about this change (well, I've heard no feedback - if anyone else has please let me know) I'm going to consider this closed. |
|||
msg261666 - (view) | Author: Eryk Sun (eryksun) * | Date: 2016-03-12 22:39 | |
I occasionally come across code snippets on Stack Overflow, and projects such as win-unicode-console (IIRC), that use ctypes to work with C stdio FILE streams (sometimes for dubious reasons, such as a DLL API that uses FILE streams). Maybe the _ctypes extension module could provide void pointers for the current C stdin, stdout, and stderr -- as well as stdio functions such as fflush, fopen, and freopen. This is already done with _ctypes._memmove_addr and _ctypes._memset_addr. However, getting the current standard stream pointers would need to use a callable or descriptor. > it'd be good content for a "Best practices" section The tutorial itself is outdated in places and doesn't promote best practices. For example, it assigns a ValidHandle function to windll.kernel32.GetModuleHandleA.restype, which would affect every module that uses windll.kernel32. Also, this ValidHandle example is bogus, as is every example in the tutorial that uses GetModuleHandle without setting restype to a pointer type such as c_void_p. It's truncating 64-bit pointers to 32-bit int values. You just need to try a DLL that loads at a high address: >>> kernel32.GetModuleHandleA(b'advapi32') -27590656 >>> def ValidHandle(value): ... if value == 0: ... raise WinError() ... return value ... >>> kernel32.GetModuleHandleA.restype = ValidHandle >>> kernel32.GetModuleHandleA(b'advapi32') -27590656 >>> hex(kernel32.GetModuleHandleA(b'advapi32') & 2**32-1) '0xfe5b0000' >>> kernel32.GetModuleHandleA.restype = c_void_p >>> hex(kernel32.GetModuleHandleA(b'advapi32')) '0x7fefe5b0000' |
|||
msg289361 - (view) | Author: Alex Gaynor (alex) * | Date: 2017-03-10 13:57 | |
An FYI for the future, it would have been very helpful if this had been documented in the whats-changed file for 3.5. |
|||
msg289365 - (view) | Author: Steve Dower (steve.dower) * | Date: 2017-03-10 14:27 | |
Noted. Did this bite you somehow? Is there something else that should be added/changed? |
|||
msg289366 - (view) | Author: Alex Gaynor (alex) * | Date: 2017-03-10 14:29 | |
Yeah, this got me (happy to explain what I was trying to do in more detail, if it'd be helpful), took me longer to understand why my tests passed on {26,27,33,34} but failed on 35 since the public "what's changed" docs page is where I went to. Ultimately I discovered the root cause when I started reading the find_library() source code, and found this issue :-) |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:13 | admin | set | github: 67794 |
2017-03-10 14:29:01 | alex | set | messages: + msg289366 |
2017-03-10 14:27:33 | steve.dower | set | messages: + msg289365 |
2017-03-10 13:57:44 | alex | set | nosy:
+ alex messages: + msg289361 |
2016-04-10 14:38:31 | steve.dower | link | issue26727 superseder |
2016-03-12 22:39:30 | eryksun | set | messages: + msg261666 |
2016-03-12 16:30:06 | steve.dower | set | status: open -> closed resolution: fixed messages: + msg261660 stage: resolved |
2016-03-12 16:26:50 | python-dev | set | messages: + msg261659 |
2016-02-11 09:32:55 | eryksun | set | messages: + msg260079 |
2016-01-15 14:02:06 | steve.dower | set | messages: + msg258295 |
2016-01-15 09:10:35 | pitrou | set | messages: + msg258282 |
2016-01-15 00:27:25 | steve.dower | set | messages: + msg258255 |
2016-01-15 00:11:07 | pitrou | set | nosy:
+ pitrou messages: + msg258254 |
2015-09-28 10:40:22 | eryksun | set | messages: + msg251759 |
2015-08-19 13:12:49 | steve.dower | set | assignee: steve.dower messages: + msg248838 |
2015-08-19 07:56:38 | cgohlke | set | nosy:
+ cgohlke |
2015-03-17 22:09:39 | steve.dower | set | messages: + msg238349 |
2015-03-17 21:16:03 | eryksun | set | messages: + msg238339 |
2015-03-14 18:49:22 | python-dev | set | messages: + msg238095 |
2015-03-10 17:01:06 | steve.dower | set | messages: + msg237781 |
2015-03-10 16:57:50 | python-dev | set | nosy:
+ python-dev messages: + msg237780 |
2015-03-10 15:26:14 | steve.dower | set | messages: + msg237773 |
2015-03-10 06:53:02 | eryksun | set | messages: + msg237731 |
2015-03-09 18:39:03 | steve.dower | set | messages: + msg237689 |
2015-03-09 18:29:57 | eryksun | set | messages: + msg237687 |
2015-03-08 13:36:53 | steve.dower | set | messages: + msg237533 |
2015-03-08 05:50:05 | eryksun | set | nosy:
+ eryksun messages: + msg237511 |
2015-03-08 04:41:09 | steve.dower | create |