classification
Title: BlockingIOError during system startup
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Python 3.6 on Windows doesn't seed Random() well enough
View: 29085
Assigned To: Nosy List: auscompgeek, haypo, martin.panter, virtuald
Priority: normal Keywords: 3.6regression

Created on 2017-01-08 16:50 by virtuald, last changed 2017-01-10 15:13 by virtuald. This issue is now closed.

Messages (7)
msg284996 - (view) Author: Dustin Spicuzza (virtuald) Date: 2017-01-08 16:50
I haven't dug into this very deeply yet, so I'm seeking some clarity on this issue before doing so.

According to my reading of PEP 522 (https://www.python.org/dev/peps/pep-0522/), BlockingIOError will be raised in security sensitive APIs, but not when just importing the random module. However, it seems that this is not actually the case:

BlockingIOError: [Errno 11] Resource temporarily unavailable 
  
 The above exception was the direct cause of the following exception: 
  
 Traceback (most recent call last): 
   File "/home/lvuser/py/robot.py", line 15, in <module> 
     import wpilib 
   File "/usr/local/lib/python3.6/site-packages/wpilib/__init__.py", line 72, in <module> 
     from ._impl.main import run 
   File "/usr/local/lib/python3.6/site-packages/wpilib/_impl/main.py", line 7, in <module> 
     from pkg_resources import iter_entry_points 
   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 36, in <module> 
     import email.parser 
   File "/usr/local/lib/python3.6/email/parser.py", line 12, in <module> 
     from email.feedparser import FeedParser, BytesFeedParser 
   File "/usr/local/lib/python3.6/email/feedparser.py", line 27, in <module> 
     from email._policybase import compat32 
   File "/usr/local/lib/python3.6/email/_policybase.py", line 9, in <module> 
     from email.utils import _has_surrogates 
   File "/usr/local/lib/python3.6/email/utils.py", line 28, in <module> 
     import random 
   File "/usr/local/lib/python3.6/random.py", line 742, in <module> 
     _inst = Random() 
 SystemError: <class 'random.Random'> returned a result with an error set 


Background: I'm the maintainer of the RobotPy project, which allows students to use Python to control their robots in the FIRST Robotics competition. The robot controller we use is an NI roboRIO, which is powered by an ARM cortex-a9.

This error was encountered when the robot is booting up with a python program. Presumably the program is started early enough in the boot process where the urandom blocking issue mentioned in the PEP occurs.

I'd love some help figuring out what's going on here, and the best way to fix it. Thanks!

I'm also tracking this issue at https://github.com/robotpy/robotpy-wpilib/issues/243
msg284998 - (view) Author: Dustin Spicuzza (virtuald) Date: 2017-01-08 17:03
Crap, after reading more carefully, I see that PEP was rejected in favor of PEP 524.

According to that PEP, then I must implement the wait_for_system_rng() function? It's a bit weird, because I'm not explicitly using the random module and don't care to use it, I'm using pkg_resources... which is importing the email parser which imports random which creates a random instance which causes this bug.

It seems to me then that the stdlib should be modified to avoid importing random unless they actually need it, otherwise this gets triggered. Or am I missing something here?
msg285007 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-08 21:20
It looks like the logic for handling an error seeding from urandom is reversed: <https://hg.python.org/cpython/rev/45fc0c83ed42#l6.66>. Random_seed_urandom() actually returns -1 if is an exception set, and 0 if it was successful.

The result would be a normal working urandom setup ignores urandom and seeds from the time and PID. Not a big deal, since the old code always seeded from a (lower resolution) time anyway. But if urandom failed, we get the SystemError.

The fix should be simple, in Modules/_randommodule.c change the random_seed() function to read

if (arg == NULL || arg == Py_None) {
    if (random_seed_urandom(self) < 0) { // was >= 0!
        PyErr_Clear();
        
        /* Reading system entropy failed, fall back on the worst entropy:
           use the current time and process identifier. */
        random_seed_time_pid(self);
    }
    Py_RETURN_NONE;
}
msg285010 - (view) Author: David Vo (auscompgeek) Date: 2017-01-08 23:07
It looks like this was already fixed in issue 29085.
msg285011 - (view) Author: Dustin Spicuzza (virtuald) Date: 2017-01-08 23:12
I'll try applying that fix tomorrow and see if the issue is addressed.
msg285012 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-01-09 00:14
David you are right, thanks. 

Dustin: the exact patch you want is revision 0a55e039d25f.
msg285125 - (view) Author: Dustin Spicuzza (virtuald) Date: 2017-01-10 15:13
I'm able to confirm that the patch does indeed fix the problem. Thanks everyone!
History
Date User Action Args
2017-01-10 15:13:52virtualdsetmessages: + msg285125
2017-01-09 00:14:06martin.pantersetstatus: open -> closed
superseder: Python 3.6 on Windows doesn't seed Random() well enough
resolution: duplicate
messages: + msg285012
2017-01-08 23:12:35virtualdsetmessages: + msg285011
2017-01-08 23:07:08auscompgeeksetnosy: + auscompgeek
messages: + msg285010
2017-01-08 21:20:02martin.pantersetnosy: + haypo, martin.panter
messages: + msg285007

keywords: + 3.6regression
type: behavior
2017-01-08 17:03:53virtualdsetmessages: + msg284998
2017-01-08 16:50:12virtualdcreate