classification
Title: Deprecate and remove ctypes usage in uuid
Type: Stage: patch review
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: corona10, ned.deily, steve.dower, vstinner
Priority: normal Keywords: patch

Created on 2020-05-04 16:34 by steve.dower, last changed 2020-05-12 22:32 by steve.dower.

Pull Requests
URL Status Linked Edit
PR 19948 merged steve.dower, 2020-05-05 23:44
Messages (8)
msg368066 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-04 16:34
The uuid module uses ctypes to try and load likely system libraries to provide some functionality of the uuid module.

This is a security risk (depending on your sensitivity to DLL hijacking), but it also seems to be not very necessary? It would be nice to remove the ctypes usage from an otherwise (almost) pure Python module.
msg368067 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-04 16:42
There are three scenarios where ctypes is used in this module:

* get libuuid.uuid_generate_time_safe and uuid_generate_time (if _uuid was not compiled)
* get rpcrt4.UuidCreateSequential (on Windows, depending on how the libuuid lookup failed)
* GetSystemDirectory (on Windows, safe to replace with alternative approach)

I'll happily move the UuidCreateSequential call into _uuid and add it to the Windows build, but it's not clear whether the libuuid lookup is important or not. If anyone knows of scenarios where you can't compile against libuuid but can load it with ctypes, would be great to hear about it!
msg368068 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-04 16:42
uuid.py has a long history. The recent history is the addition of the _uuid module which exposes libuuid function properly. IMHO it's a better approach than ctypes.

On Windows, it seems like ctypes remains used to get access to Windows function UuidCreateSequential(). It is used by uuid.getnode() for example. We should expose UuidCreateSequential() in _uuid rather than using ctypes for that.

I propose to do that in two steps:

(A) Ensure that _uuid works on macOS, FreeBSD and Linux, especially in the macOS installer of python.org. If yes, remove the ctypes code to access libuuid functions.

(B) Add a function to _uuid to expose Windows UuidCreateSequential(), use it in uuid.py and remove the related ctypes code.

Both steps can be done in parallel.
msg368070 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-04 16:48
Do you think either need a deprecation cycle? I'd say not for the Windows change, but I'm not sure whether people could be relying on libuuid showing up after build.
msg368071 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-04 16:52
> Do you think either need a deprecation cycle? I'd say not for the Windows change, but I'm not sure whether people could be relying on libuuid showing up after build.

If the behavior doesn't change, no deprecation is needed.

For example, if libuuid was already used trough _uuid module, the ctypes code path is basically just dead code. But it should be checked manually.
msg368220 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-05-06 01:09
> (A) Ensure that _uuid works on macOS, FreeBSD and Linux, especially in the macOS installer of python.org.

FWIW, a spot check shows that _uuid builds and test_uuid passes both sets of its tests, e.g. TestUUIDWithExtModule and TestUUIDWithoutExtModule, with current python.org macOS installers on macOS releases of interest.  So, AFAICT, we have no need for the ctypes fallback on at least macOS 10.6 or later and we don't actively support anything older than that.
msg368296 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-06 19:53
Thanks Ned.

There are some platform.version() checks in there that I'm basically ignoring right now - are those unnecessary? Shall I clean them up too?
msg368746 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-12 22:32
New changeset d6b727e2c947240804b8e434b305ba2890122550 by Steve Dower in branch 'master':
bpo-40501: Replace ctypes code in uuid with native module (GH-19948)
https://github.com/python/cpython/commit/d6b727e2c947240804b8e434b305ba2890122550
History
Date User Action Args
2020-05-12 22:32:49steve.dowersetmessages: + msg368746
2020-05-06 19:53:55steve.dowersetmessages: + msg368296
2020-05-06 01:09:44ned.deilysetnosy: + ned.deily
messages: + msg368220
2020-05-05 23:44:13steve.dowersetkeywords: + patch
stage: test needed -> patch review
pull_requests: + pull_request19263
2020-05-04 16:52:40vstinnersetmessages: + msg368071
2020-05-04 16:51:54corona10setnosy: + corona10
2020-05-04 16:48:18steve.dowersetmessages: + msg368070
2020-05-04 16:42:54vstinnersetnosy: + vstinner
messages: + msg368068
2020-05-04 16:42:33steve.dowersetmessages: + msg368067
2020-05-04 16:34:03steve.dowercreate