Calling uuid.uuid4() while using the multiprocessing module leads to the same exact UUIDs being generating in each process.  It is an artifact resulting from the built-in uuid_generate_random() of my underlying platform, Mac OS X 10.6.3.  A Linux machine I have does not exhibit this problem.  I have tested it with both Python 2.5 and 2.6.

For the record, I filed a bug against the underlying platform.  (I wrote a simple program to reproduce this in C and filed a bug report with Apple rdar://7944700.  The OpenRadar page for it is here: http://openradar.appspot.com/radar?id=334401 )

As the bug is in the underlying platform the best we can do is to warn about this in the documentation, as in the attached patch.


BTW. I've updated the title to be slightly more informative.

Why not default to not use the Python implementation on darwin instead of the underlying platform's uuid_generate_random(), until it's proven safe?

Ahem.  Why not use the Python implementation on darwin until its uuid_generate_random() is deemed to be safe?

Because I didn't look closely enough at the source :-(

The attached patch disabled the C implementation on OSX 10.6 or later. I've tested that 10.5 is not affected by the issue.

Also added a testcase that should warn if other unix-y platforms start to suffer from the same issue.

BTW. issue8621.patch uses a runtime test in the uuid module instead of a configure-check because a binary might be created on 10.5 (without the issue) and run on 10.6 (with the issue) and that should not result in a broken library.

Great work!  Very thorough patches.  Strange that it's a regression versus 10.5.

There are way more interesting regressions in OSX, issue8621 is one example: basicly getgroups(2) does not reflect the results of setgroups(2) with the compiler settings we use.

Committed in r80784 (trunk), r80785 (2.6), r80786 (3.2) and r80788 (3.1)

Reopening since test failures are reported on python-dev:

[...]
test_uuid
test test_uuid failed -- Traceback (most recent call last):
  File "/private/tmp/Python-2.7rc2/Lib/test/test_uuid.py", line 472, in testIssue8621
    self.assertNotEqual(parent_value, child_value)
AssertionError: '8395a08e40454895be537a180539b7fb' == '8395a08e40454895be537a180539b7fb'

[...]

This is on an Intel machine running OS X 10.5.8.  I downloaded and built 2.7rc2 from source with "./configure ; make".  I then ran the tests with "make test".  test_uuid fails with this output:


test test_uuid failed -- Traceback (most recent call last):
  File "/private/tmp/Python-2.7rc2/Lib/test/test_uuid.py", line 472, in testIssue8621
    self.assertNotEqual(parent_value, child_value)
AssertionError: '751ca85de22f4450b7f95dd3f82c7e5f' == '751ca85de22f4450b7f95dd3f82c7e5f'

However, when I run the test standalone with this command-line, it passes:

% ./python.exe -Wd -3 -E -tt ./Lib/test/regrtest.py -l test_uuid
test_uuid
1 test OK.

Not sure what's going on.

Odd, unless someone already checked in a fix on the trunk. 

I currently have a clean test run on the trunk on OSX 10.6.4 (intel). I haven't checked this on my 10.5 VM yet.

It is disputed on http://openradar.appspot.com/radar?id=334401 that
this is an OS bug. If I understand correctly, the test program is
not guaranteed to work if threads are involved. See also:

http://webcache.googleusercontent.com/search?q=cache:hJo0u_Lc8wkJ:www.opengroup.org/onlinepubs/000095399/functions/fork.html+http://www.opengroup.org/onlinepubs/009695399/functions/fork.html&hl=en&strip=1


"A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called. [THR] [Option Start]  Fork handlers may be established by means of the pthread_atfork() function in order to maintain application invariants across fork() calls."


This could explain why running the complete test suite fails but
running the individual test does not.

Stefan: we already new that, see msg105018.

This issue was closed as fixed because the uuid module contains a workaround for this issue (by not using the broken C API on OSX 10.6).

It seems that OSX 10.5.8 and 10.4 is also affected by this issue. I'll test on 10.5, and if I can reproduce the issue there I'll adjust the workaround for this.

Bill, could you try to add this to the tests and see if they
also fail when you run them standalone?


Index: Lib/test/test_uuid.py
===================================================================
--- Lib/test/test_uuid.py       (revision 82109)
+++ Lib/test/test_uuid.py       (working copy)
@@ -479,4 +479,7 @@
     test_support.run_unittest(TestUUID)
 
 if __name__ == '__main__':
+    import threading
+    t = threading.Thread(target=lambda: None)
+    t.start()
     test_main()

Bill, please test the patch below on a 10.5 system:


Index: Lib/uuid.py
===================================================================
--- Lib/uuid.py	(revision 82148)
+++ Lib/uuid.py	(working copy)
@@ -416,7 +416,7 @@
     import sys
     if sys.platform == 'darwin':
         import os
-        if int(os.uname()[2].split('.')[0]) >= 10:
+        if int(os.uname()[2].split('.')[0]) >= 9:
             _uuid_generate_random = _uuid_generate_time = None
 
     # On Windows prior to 2000, UuidCreate gives a UUID containing the


This extends the workaround for the bug in the system uuid libraries to OSX 10.5.

Ronald Oussoren <report@bugs.python.org> wrote:
> Stefan: we already new that, see msg105018.
> 
> This issue was closed as fixed because the uuid module contains a workaround for this issue (by not using the broken C API on OSX 10.6).

Ok, my comment was partly meant to give an explanation why the test suite failed
but the individual tests did not.

If uuid_generate_random() is specified as async-signal-safe, then yes, the C API
would be broken on OSX.

Stefan, I tried your suggestion of starting threading.  Test still succeeds on my 10.5.8 system when test_uuid is run separately.

Ronald, your fix works on my 10.5.8 system.  Why not check it in, and let's see if the buildbots turn green again?

Committed a fix in r82276 (2.7), r82277 (2.6), r82278 (3.2) and r82279 (3.1).

Please reopen the issue if you notice that the problem does not go away on the buildbots.