This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: uuid.uuid1() should use uuid_generate_time_safe() if available
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: barry Nosy List: alex, barry, serhiy.storchaka, taleinat, vila, vstinner
Priority: normal Keywords:

Created on 2014-11-06 20:07 by barry, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 138 merged barry, 2017-02-16 15:47
PR 703 larry, 2017-03-17 21:00
PR 552 closed dstufft, 2017-03-31 16:36
Messages (12)
msg230759 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2014-11-06 20:07
I'm classifying this as a security issue, since using uuid_generate_time() -- i.e. the not _safe() variety -- does return collisions in real world cases that we've seen, and those could have security implications. However, I don't know that this can be exploited in any real world cases, so I'm not making it private or sending to security@.

The basic problem is that uuid.uuid1() uses uuid_generate_time(3), but if the synchronization methods used in that C function's manpage are not used, then two concurrent processes can -- and do in our cases -- return the same UUID.

I would propose that if uuid_generate_time_safe() is available, this should be used instead, and the return value should be checked to see if a safe method was used.  If not, then uuid1() should fall back to the pure-Python approach.
msg230760 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2014-11-06 20:10
FWIW, I'm not convinced the pure python fallback code is sufficient either; time.time() doesn't have the necessary resolution AFAIK? Also clock_seq is generated using the random module's messerne twister, not SystemRandom().
msg230762 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2014-11-06 20:29
On Nov 06, 2014, at 08:10 PM, Alex Gaynor wrote:

>FWIW, I'm not convinced the pure python fallback code is sufficient either;
>time.time() doesn't have the necessary resolution AFAIK? Also clock_seq is
>generated using the random module's messerne twister, not SystemRandom().

Perhaps, but that's a different bug. ;)

-----snip snip-----
from uuid import UUID
import ctypes
import ctypes.util

lib = ctypes.CDLL(ctypes.util.find_library('uuid'))
_ugts = lib.uuid_generate_time_safe

_buffer = ctypes.create_string_buffer(16)
retval = _ugts(_buffer)

# Remember, this is C!
is_safe = (retval == 0)

print('{} is safe? {}'.format(UUID(bytes=_buffer.raw), is_safe))
-----snip snip-----

On Ubuntu 14.10, gives me:

xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is safe? True
msg287950 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-02-16 15:45
I changed my mind on whether this should affect older versions of Python.  I have a branch which adds an UUID.is_safe attribute that relays the platform information about whether the UUID was generated safely or not, if available.  It's an enum named SafeUUID with values .safe, .unsafe, .unknown (the default).
msg288089 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-02-18 20:45
New changeset 8c130d7f8114158f5b94749032ec0c17dba96f83 by GitHub in branch 'master':
bpo-22807: Expose platform UUID generation safety information. (#138)
https://github.com/python/cpython/commit/8c130d7f8114158f5b94749032ec0c17dba96f83
msg288209 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-20 14:21
I don't understand well this change.

What am I supposed to do with an UUID with safe=False? Should I loop on the function until I get safe==True?


"safe for multiprocessing applications"

Does it mean unique on the whole system?

I looked at uuid_generate_time_safe(3) manual page which mention "synchronization mechanisms (see above)" but they are not documented.
http://manpages.ubuntu.com/manpages/zesty/en/man3/uuid_generate.3.html


> I'm classifying this as a security issue, (...)

This issue was only fixed in Python 3.7. Does it mean that it's no more considered as as security vulnerability?
msg288213 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-02-20 15:13
On Feb 20, 2017, at 02:21 PM, STINNER Victor wrote:

>What am I supposed to do with an UUID with safe=False? Should I loop on the
>function until I get safe==True?

It would be an application dependent response.  It might be that you would
check some other attributes of your platform (e.g. are the OS packages that
should be installed to give you safe UUIDs?).  Or your application may not
care that much, or your application may refuse to continue to run on platforms
without safe UUIDs, or you might use some application-level synchronization
methods to guarantee safe UUIDs (e.g. store the unsafe or unknown ones in a
database and check that new ones are not already used).

The point of this change is that it provides information to the application
creating UUIDs that wasn't previously available.

>"safe for multiprocessing applications"
>
>Does it mean unique on the whole system?
>
>I looked at uuid_generate_time_safe(3) manual page which mention
>"synchronization mechanisms (see above)" but they are not documented.
>http://manpages.ubuntu.com/manpages/zesty/en/man3/uuid_generate.3.html

I believe some systems at least use interprocess communication with a daemon
to provide the synchronization.  Yes, it would be system-wide.

>> I'm classifying this as a security issue, (...)  
>
>This issue was only fixed in Python 3.7. Does it mean that it's no more
>considered as as security vulnerability?

I should remove that tag.  While this could have an impact on application
security, it's not a security issue *in Python* itself.
msg288214 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-02-20 15:16
Oh, and because the fix is an API change, I don't believe it should be applied to earlier versions.  So I think adding the API in 3.7 is all the fix needed here.
msg288215 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-20 15:45
>>> import uuid
>>> u=uuid.uuid4()
>>> u.is_safe
<SafeUUID.unknown: None>

Can't we consider that UUID4 is always safe?
msg288216 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-02-20 15:53
On Feb 20, 2017, at 03:45 PM, STINNER Victor wrote:

>Can't we consider that UUID4 is always safe?

It's not a guarantee made by the underlying platform, so I chose to use the
default SafeUUID.unknown value there.
msg324829 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-08 11:02
This breaks pickle compatibility. UUIDs pickled in 3.7 can't be unpickled in older Python versions because they do not have the SafeUUID class. See issue30977 for possible solution.
msg324920 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2018-09-10 12:17
The fix for issue30977 did fix the unpickling in older versions. It was only applied to the master (i.e. 3.8) branch, though.

I've created issue34621 to deal with this separately.
History
Date User Action Args
2022-04-11 14:58:09adminsetgithub: 66996
2018-09-10 12:17:40taleinatsetmessages: + msg324920
2018-09-08 11:02:22serhiy.storchakasetnosy: + taleinat, serhiy.storchaka
messages: + msg324829
2017-03-31 16:36:27dstufftsetpull_requests: + pull_request1010
2017-03-17 21:00:34larrysetpull_requests: + pull_request606
2017-02-20 15:53:06barrysetmessages: + msg288216
2017-02-20 15:45:53vstinnersetmessages: + msg288215
2017-02-20 15:16:11barrysetmessages: + msg288214
2017-02-20 15:15:25barrysetstatus: open -> closed
resolution: fixed
2017-02-20 15:14:27barrysetkeywords: - security_issue
2017-02-20 15:14:00barrysetmessages: + msg288213
2017-02-20 14:21:01vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg288209
2017-02-19 17:03:10barrysetstatus: open -> closed
stage: resolved
2017-02-19 17:03:05barrysetresolution: fixed
2017-02-18 20:45:51barrysetmessages: + msg288089
2017-02-16 15:47:02barrysetpull_requests: + pull_request98
2017-02-16 15:45:10barrysetmessages: + msg287950
versions: - Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6
2017-02-16 03:34:04barrysetassignee: barry
2017-02-11 17:35:47barrysetversions: + Python 3.3, Python 3.6, Python 3.7
2014-11-07 09:53:47vilasetnosy: + vila
2014-11-07 00:23:12pitrousetnosy: + vstinner
2014-11-06 20:29:02barrysetmessages: + msg230762
2014-11-06 20:10:09alexsetnosy: + alex
messages: + msg230760
2014-11-06 20:07:26barrycreate