Demo:

Run the Python library's test_random.py under the Python debugger and check the
generator at the start of test_shuffle():

C:\bin\Python36>python -m pdb Lib\test\test_random.py
> c:\bin\python36\lib\test\test_random.py(1)<module>()
-> import unittest
(Pdb) break 61
Breakpoint 1 at c:\bin\python36\lib\test\test_random.py:61
(Pdb) continue
.............................> c:\bin\python36\lib\test\test_random.py(61)test_shuffle()
-> shuffle = self.gen.shuffle
(Pdb) list
 56             # randomness source is not available.
 57             urandom_mock.side_effect = NotImplementedError
 58             self.test_seedargs()
 59
 60         def test_shuffle(self):
 61 B->         shuffle = self.gen.shuffle
 62             lst = []
 63             shuffle(lst)
 64             self.assertEqual(lst, [])
 65             lst = [37]
 66             shuffle(lst)
(Pdb) p self.gen.getrandbits(31)
2137781566
(Pdb) p self.gen.getrandbits(31)
2137781566
(Pdb) p self.gen.getrandbits(31)
2137781566
(Pdb) p self.gen.getrandbits(31)
2137781566
(Pdb) p self.gen.getrandbits(31)
2137781566

That's not random.


Diagnosis:

The order in which test functions run is the lexicographic order of their names. Thus unittest ran test_setstate_middle_arg() before running test_shuffle(). test_setstate_middle_arg() did some failed calls to _random.Random.setstate(), which raised exceptions as planned, but also trashed the state of the generator. test_random.py continues to use the same instance of _random.Random after setstate() raises exceptions.

The documentation for Random.setstate() does not specify what happens to the state of the generator if setstate() raises an exception. Fortunately the generator recommended for secure applications, SystemRandom, does not implement setstate(). 


Solution:

The fix I prefer is a small change to random_setstate() in _randommodule.c, so that it does not change the state of the generator until the operation is sure to succeed.

I'm going through https://docs.python.org/devguide/pullrequest.html and would like to be assigned this issue.

New changeset 9616a82e7802241a4b74cf7ae38d43c37bf66e48 by Serhiy Storchaka (bladebryan) in branch 'master':
bpo-29960 _random.Random corrupted on exception in setstate(). (#1019)
https://github.com/python/cpython/commit/9616a82e7802241a4b74cf7ae38d43c37bf66e48

Mariatta, can you update your PRs?

New changeset 440bc4f4b2690b99541e87bedfdb0dc4abbc0501 by Mariatta in branch '3.5':
[3.5] bpo-29960 _random.Random corrupted on exception in setstate(). … (#1288)
https://github.com/python/cpython/commit/440bc4f4b2690b99541e87bedfdb0dc4abbc0501

New changeset 1626a479e22db3d44bcd6736d571243433cb6d0e by Mariatta in branch '2.7':
[2.7] bpo-29960 _random.Random corrupted on exception in setstate(). … (#1289)
https://github.com/python/cpython/commit/1626a479e22db3d44bcd6736d571243433cb6d0e

New changeset 94d8261d1ca3f0c551d9d43a4db342d806af7a6a by Mariatta in branch '3.6':
[3.6] bpo-29960 _random.Random corrupted on exception in setstate(). … (#1287)
https://github.com/python/cpython/commit/94d8261d1ca3f0c551d9d43a4db342d806af7a6a

Thanks Bryan and Serhiy.