classification
Title: bad seed in python 2.3 random
Type: Stage:
Components: Extension Modules Versions: Python 2.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: barry, blaforge, jackjansen, rhettinger, tim.peters
Priority: normal Keywords:

Created on 2003-07-28 14:47 by blaforge, last changed 2003-08-10 07:10 by blaforge. This issue is now closed.

Messages (17)
msg17388 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-28 14:47
class crjentPje(PJE):
    def __init__
(self,uuri,headline,actorName,jentProxy,lelProxy=None,p
op='',op='',jelType='L1PJENT.PJENT',ts='NoName'):
        Cache.PJE.__init__(self,uuri,actorName,'*',pop+' 
|'+headline+'|',ts=ts)

class StartOfJel(PJE):
    def __init__
(self,uuri,jelName,actorName,partOfProxy=None,jelType
='L1JelTypes.JEL',ts='NoName'):
        Cache.PJE.__init__
(self,uuri,actorName,'`','StartOfJel',ts=ts)
        self['*JelName']=jelName
        self['*JelType']=jelType
        if partOfProxy:
            self['*PartOf']=partOfProxy.uuri

About 1 time in 10 when running 2.3c2, when creating 
a StartOfJel, the partOfProxy ended up being passed as 
the uuri parameter to crjentPje!!! (I was passing None 
as the value for the uuri parameter.)

The problem vanishes when I fall back to 2.2.
msg17389 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-07-28 15:15
Logged In: YES 
user_id=45365

My gut feeling says that if something this basic isn't working there 
is probably something in your setup wrong, or something in the 
way you built Python, or some such. The reasoning behind this is 
that something so essential as parameters being mixed up would 
have shown up earlier.

But: there is also the possibility that you have found a genuine 
bug, and then it is very serious. Unfortunately the bug report as it 
is now doesn't allow for easy duplication. We can try if you can 
provide the following:
- an isolated runnable script that shows the problem
- exact details on your Python version, OS version, any other 
details that may be pertinent
- a recipy for running the script to see the problems.

As Python 2.3 is scheduled for tomorrow there is a bit of a hurry 
here...
msg17390 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-29 03:06
Logged In: YES 
user_id=22406

I'm running MS Windows 2000 and did the standard python 
windows install. So you can't blame my build! Its real.

And yes, this is a pretty basic flaw, and intermittent at that!
(about 1 out of ten runs failing.)

I'll see what I can do about isolation. (Being in India has its 
advantages.)

My guess is that a small change to the signiture will fix this, 
which doesn't help anyone else, as this bug will likely haunt 
2.3 if not caught.

blaforge
msg17391 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-29 04:31
Logged In: YES 
user_id=22406

Bad news.

I'm running 2000 with service pack 3. I moved the code to 
another 2000 system (with no service pack). Code still runs 
under 2.2 but not under 2.3. ;-(

blaforge
msg17392 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-07-29 04:39
Logged In: YES 
user_id=12800

The most important thing right now is to get an isolated,
runnable script that we can use to try to reproduce the
problem.  Without this, we're at a loss, and time is rapidly
running out for us to be able to do anything for Python 2.3
final (if it hasn't already run out).
msg17393 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-29 07:09
Logged In: YES 
user_id=22406

Lets say that its too late. This bug is more obscure than I 
had thought. It also can not be isolated.

I tracked the problem down to my UURI generator, which 
returns nice random 128 bit base-32 encoded strings. In one 
test case only it returns the same UURI as passed by that 
other parameter. Note that the other parameter is NOT 
passed to the UURI generator.

I'm saying that a code fragment can not be generated 
because one of my regression tests checks to make sure the 
UURI generator doesn't return duplicate values. And that test 
still works fine.

I'm finding this all very peculilar. One other thing I'll note in 
passing is that the more print statements I put in the code, 
the less likely any given test is to fail. With no print 
statements, failure occurs about half the time. With lots of 
print statements, failure is about 1 in 20.

The whole program is about 30K (compressed). Its also hard 
to spot when the error occurs, as the error destroys data 
which causes a subsequent failure. (Though knowing the 
error, I can now add additional checks.)

As the code is confidential and propriatary, I'll need 
permission from the client to send it, as well. 

blaforge
msg17394 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-07-29 09:22
Logged In: YES 
user_id=45365

The fact that you can't easily isolate it strengthens my hunch that 
this is not a general issue. The code looks distinctly fishy, and the 
description also doesn't match the code fragement (although that 
could well be the result of you pulling this out of a large package). 
Note that you talk about parameters being passed to crjentPje, but 
that isn't a baseclass of StartOfJel.

I suggest you single-step the code in a debugger, to check that 
what really happens actually matches what you think is 
happening...
msg17395 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-29 09:40
Logged In: YES 
user_id=22406

Then by all means close this bug report. My sincere desire is 
that this is NOT a general issue.

Also my hands are a bit tied as the client is NOT willing that 
I release this code. 

All the best, and thanks to everyone for the great software--
Python's the Best!

blaforge
msg17396 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-29 10:31
Logged In: YES 
user_id=22406

I reopened this because I learned something that might be 
helpful--I only have problems with 2.3c2 when invoked via a 
bat script. When run from a DOS command prompt, I have 
no problem.

blaforge
msg17397 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-29 10:49
Logged In: YES 
user_id=22406

Solved!

It looks very much like I'm getting a bad seed for random 
when running from a batch file. (My code depends on unique 
128-bit random numbers for proper functioning--I've written 
my own secure random method that sits on top of random.)

Anyway, the fix for me that makes 2.3c work fine is to 
provide my own seed to random:

    random.Random(time.time())

blaforge
msg17398 - (view) Author: Bill la Forge (blaforge) Date: 2003-07-29 12:14
Logged In: YES 
user_id=22406

Stranger and stranger

While setting the seed fixes it for me, I still can't reduce the 
script to recreate the problem!

(frustrated)
msg17399 - (view) Author: Bill la Forge (blaforge) Date: 2003-08-09 11:40
Logged In: YES 
user_id=22406

Identified the problem in python 2.3 and have test script!

It looks like the default seed in python2.3 is int(time()),
which yields the same random numbers if a program is run 
twice in the same second.

This is not true of python 2.2.

Bill la Forge
msg17400 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2003-08-09 15:29
Logged In: YES 
user_id=31435

That is indeed a silent change in strategy.  Assigned to 
Raymond for pondering.

Making it act more like previous Pythons would involve 
changing Random.seed() to replace a when None with long
(time.time() * 256) before calling super(Random, self).seed
(a).  This is much harder to do in C because getting fractional 
seconds is a cross-platform mess (Python's time.time() hides 
that mess); then random_seed() should probably insist on 
having a real argument.  Alternatively, random_seed() could 
invoke Python's time.time() to get at fractional seconds 
datetimemodule.c's time_time() is one way to do that).
msg17401 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2003-08-09 18:32
Logged In: YES 
user_id=80475

Fixed using Tim's first listed approach.  Avoided the second 
approach because datetime is an optional build component.

See Lib/random.py 1.51.8.1 and 1.53.
msg17402 - (view) Author: Bill la Forge (blaforge) Date: 2003-08-10 04:46
Logged In: YES 
user_id=22406

I'll note that this (released version 2.3) is a fairly substantial 
security flaw, as it makes it much easier to guess someone's 
seed.
msg17403 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2003-08-10 06:00
Logged In: YES 
user_id=31435

Multiplying by 256 is a trivial improvement if you're a Security 
Geek.  If you care about that, you'll need to tap a source of 
strong randomness to feed the seed.
msg17404 - (view) Author: Bill la Forge (blaforge) Date: 2003-08-10 07:10
Logged In: YES 
user_id=22406

OK, yes. Further, I'll admidt to using such code myself.

b-)
History
Date User Action Args
2003-07-28 14:47:23blaforgecreate