Currently if /dev/urandom does not provide any data, unradom() call is just stuck infinitely waiting for data.

I actually faced this issue when /dev/urandom was empty regular file (due to bug in pbuilder, but I don't think it matters how it did happen) and urandom() call just hang. I think it would be much saner in such case to throw an error and let user know that something it wrong than waiting infinitely.

This would change semantics of the library call, though.
I see two possible solutions:
- do nothing and let users tackle the issue if necessary (e.g. by calling open() themselves and using select() on the resulting fd)
- add an optional "blocking" parameter to os.urandom() that, if False, would return None when no data is available

Well in this particular case (/dev/urandom is regular file), it might make sense to simply raise NotImplementedError

> Well in this particular case (/dev/urandom is regular file), it might
> make sense to simply raise NotImplementedError

IMO it would be quite fragile to try to detect regular files vs special
files. I suppose you noticed the issue quite quickly anyway, since you
had the blocking issue.

Is it really necessary to do something about this?  /dev/urandom being a regular file is clearly a bug in your system configuration, and I don't want to know what all the other programs will do that rely on it...

From the documentation:
"This function returns random bytes from an OS-specific randomness source."

In your case, this problem shows up because of an OS misconfiguration : in that case, the behaviour is undefined (not much Python can do about it). Note that since /dev/urandom is used, with a properly configured system, this should never block (contrarily to /dev/random which might block until enough entropy has been gathered).

Yes, it was blocking, but deep in some program (which was actually called by dpkg postinst script), so it took some time to figure out.

I don't think it's that fragile to figure out whether it is regular file using os.path.isfile.

Indeed it was bug in a system, but actually this was only thing which got stuck because of it.

> Note that since /dev/urandom is used, with a properly configured
> system, this should never block

Ok, suggesting closing then.

I wonder why reading from /dev/urandom has a loop in the first place, though - isn't it guaranteed that you can read as many bytes as you want in one go? This goes back to #934711, and apparently, even the original patch had the loop - for reasons that got never questioned.

Agreed that the original issue is invalid.  So either the title should be changed so it can be used to address Martin's question, or it should be closed.

Closing it. It seems that people feel more safe when urandom loops until it has enough data (see http://stackoverflow.com/questions/4598507/dev-urandom-maximum-size/4598534#4598534). I might still pursue this idea, but in a different issue.

> Martin v. Löwis <martin@v.loewis.de> added the comment:
>
> I wonder why reading from /dev/urandom has a loop in the first place, though - isn't it guaranteed that you can read as many bytes as you want in one go? This goes back to #934711, and apparently, even the original patch had the loop - for reasons that got never questioned.
>

I found surprising that a read from /dev/urandom would be
uninterruptible, so I digged a little, and found this mail from 1998:

[patch] fix for urandom read(2) not interruptible
http://marc.info/?l=bugtraq&m=91495921611500&w=2

"It's a bug in random.c that doesn' t check for signal pending inside the
read(2) code, so you have no chance to kill the process via signals until
the read(2) syscall is finished, and it could take a lot of time before
return, if the buffer given to the read syscall is very big..."

I've had a quick look at the source code, and indeed, read(2) from
/dev/urandom can now be interrupted by a signal, so looping seems to
be justified.

> ----------
> nosy: +loewis
> status: pending -> open
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue10824>
> _______________________________________
>

> "It's a bug in random.c that doesn' t check for signal pending inside the
> read(2) code, so you have no chance to kill the process via signals until
> the read(2) syscall is finished, and it could take a lot of time before
> return, if the buffer given to the read syscall is very big..."
> 
> I've had a quick look at the source code, and indeed, read(2) from
> /dev/urandom can now be interrupted by a signal, so looping seems to
> be justified.

No: if read(2) is interrupted, no data is returned, and exception is
raised. So it won't loop in that case, but raise the exception out of
urandom also (which is the right thing to do).

> Martin v. Löwis <martin@v.loewis.de> added the comment:
>
>> "It's a bug in random.c that doesn' t check for signal pending inside the
>> read(2) code, so you have no chance to kill the process via signals until
>> the read(2) syscall is finished, and it could take a lot of time before
>> return, if the buffer given to the read syscall is very big..."
>>
>> I've had a quick look at the source code, and indeed, read(2) from
>> /dev/urandom can now be interrupted by a signal, so looping seems to
>> be justified.
>
> No: if read(2) is interrupted, no data is returned, and exception is
> raised. So it won't loop in that case, but raise the exception out of
> urandom also (which is the right thing to do).
>

(Sorry for being a little off-topic, but since there's not dedicated thread)
Try with this:
dd if=/dev/urandom of=/dev/null bs=100M count=1

Then, in another terminal:
pkill -USR1 -xn dd

You'll see that read returns less that 100M bytes when interrupted.
You can also try with the following python code:

---
import os

d = os.open('/dev/urandom', os.O_RDONLY)
data = os.read(d, 1 << 28)
os.close(d)
print('read %d bytes' % len(data))
---

and in another terminal
pkill -STOP -xn python
then
pkill -CONT -xn python

Same thing, read returns less bytes than requested.
Anyway, since /dev/urandom is not part of any standard (AFAIK), it's
probably better to use the common idiom
while len(data) < expected:
    read(expected - len(data))

So we're sure it won't break under some systems/conditions.

Cheers

> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue10824>
> _______________________________________
>

> You'll see that read returns less that 100M bytes when interrupted.

I see.

> while len(data) < expected:
>     read(expected - len(data))
> 
> So we're sure it won't break under some systems/conditions.

I think this is not quite the idiom we should use if we want to deal
with signals: if read() returns an empty string, we have hit end-of-file.

If there is a signal before anything is read, we should catch the
exception and continue reading (which the loop doesn't do, either).

OTOH, if the signal was due to a user interrupt, we should raise an
exception, anyway, IMO - even if we've read some data already.