classification
Title: test_io hangs on 3.1.3rc1
Type: crash Stage:
Components: Tests Versions: Python 3.1
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: ned.deily, pitrou, r.david.murray, ronaldoussoren, vdupras
Priority: release blocker Keywords:

Created on 2010-11-14 14:13 by vdupras, last changed 2010-11-14 17:28 by vdupras. This issue is now closed.

Messages (8)
msg121194 - (view) Author: Virgil Dupras (vdupras) (Python triager) Date: 2010-11-14 14:13
I downloaded Python 3.1.3rc1 this morning to do my civic duty of testing it. I don't know what I'm doing wrong, but for me, test_io hangs and never completed. I'm on OS X 10.6.5. I ran it with:

$ ./python.exe Lib/test/regrtest.py test_io

And I got:

test_io

Testing large file ops skipped on darwin.
It requires 2147483648 bytes and a long time.
Use 'regrtest.py -u largefile test_io' to run it.

Testing large file ops skipped on darwin.
It requires 2147483648 bytes and a long time.
Use 'regrtest.py -u largefile test_io' to run it.
python.exe(12242) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
python.exe(12242) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
python.exe(12242) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
<hangs at this point>

I build Python with this configuration:

$ ./configure --enable-universalsdk=/ --enable-framework --with-universal-archs=intel

Oh, and it's being run as 64-bit.
By looking at the size being allocated, by guess that it's some kind of overflow thing or something.
msg121198 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-14 15:52
Can you run regrtest with the -v option, to know which test case fails?

(we have several OS X buildbots which run this test fine, by the way, including a Snow Leopard instance)

Note: 9223372036854775808 is 2**63 or 0x8000000000000000.
msg121199 - (view) Author: Virgil Dupras (vdupras) (Python triager) Date: 2010-11-14 16:04
I ran the test with the -v option flag. The malloc error don't happen at the same place the hang up happens. The first one happens at:

test_readline (test.test_io.PyIOTest) ... ok
test_unbounded_file (test.test_io.PyIOTest) ... skipped 'test can only run in a 32-bit address space'
test_with_open (test.test_io.PyIOTest) ... ok
test_buffering (test.test_io.CBufferedReaderTest) ... ok
test_constructor (test.test_io.CBufferedReaderTest) ... python.exe(24083) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
ok
test_context_manager (test.test_io.CBufferedReaderTest) ... ok
test_detach (test.test_io.CBufferedReaderTest) ... ok

And the second one happens at:

test_readlines (test.test_io.PyBufferedReaderTest) ... ok
test_repr (test.test_io.PyBufferedReaderTest) ... ok
test_threads (test.test_io.PyBufferedReaderTest) ... skipped "resource 'cpu' is not enabled"
test_constructor (test.test_io.CBufferedWriterTest) ... python.exe(24083) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
ok
test_context_manager (test.test_io.CBufferedWriterTest) ... ok
test_destructor (test.test_io.CBufferedWriterTest) ... ok

And finally, the hang:

test_io_after_close (test.test_io.CMiscIOTest) ... ok
test___all__ (test.test_io.PyMiscIOTest) ... ok
test_abc_inheritance (test.test_io.PyMiscIOTest) ... ok
test_abc_inheritance_official (test.test_io.PyMiscIOTest) ... ok
test_abcs (test.test_io.PyMiscIOTest) ... ok
test_attributes (test.test_io.PyMiscIOTest) ... ok
test_blockingioerror (test.test_io.PyMiscIOTest) ... ok
test_io_after_close (test.test_io.PyMiscIOTest) ... ok
test_interrupted_write_buffered (test.test_io.CSignalsTest) ... <hang>
msg121200 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-14 16:24
> I ran the test with the -v option flag. The malloc error don't happen
> at the same place the hang up happens. The first one happens at:

The malloc error is normal, then, it's part of that test. It's a pity OS
X dumps something on stderr, though.

> And finally, the hang:
> 
> test_io_after_close (test.test_io.CMiscIOTest) ... ok
> test___all__ (test.test_io.PyMiscIOTest) ... ok
> test_abc_inheritance (test.test_io.PyMiscIOTest) ... ok
> test_abc_inheritance_official (test.test_io.PyMiscIOTest) ... ok
> test_abcs (test.test_io.PyMiscIOTest) ... ok
> test_attributes (test.test_io.PyMiscIOTest) ... ok
> test_blockingioerror (test.test_io.PyMiscIOTest) ... ok
> test_io_after_close (test.test_io.PyMiscIOTest) ... ok
> test_interrupted_write_buffered (test.test_io.CSignalsTest) ... <hang>

Ok. I guess you would need to investigate what happens in that
particular sub-test.
msg121202 - (view) Author: Virgil Dupras (vdupras) (Python triager) Date: 2010-11-14 16:44
Nobody else can reproduce the bug? I'm not sure I can fix this (although I can try).

I tried to re-compile 3.1.2 with the same flags and run test_io and it passes, so something happened between 3.1.2 and 3.1.3rc1.

So, I'll give it a look, but if someone more experienced can reproduce it...
msg121203 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-11-14 16:58
Ronald hasn't replied yet and he's the most likely to be in a position to try to reproduce it.

Even if you can't fix it, figuring out more about how that individual test is arriving at the hang could be useful.  If you've got the chops to look at the hang in a debugger that would be even more helpful.
msg121204 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-11-14 17:19
Be careful when running a framework build from the build directory without installing it.  Chances are you are actually dynamically linking to an older existing framework in /Library/Frameworks.  (Try looking at sys.modules to see what I mean.)  You'll either need to temporarily move the old fw out of the way and do the install or use a different fw path to configure or don't use a fw build.
msg121205 - (view) Author: Virgil Dupras (vdupras) (Python triager) Date: 2010-11-14 17:28
Ooh, darn, that was it. I installed it and afterwards, the tests passed.

Sorry for the fuss, I'll keep that gotcha in mind next time.
History
Date User Action Args
2010-11-14 17:28:08vduprassetstatus: open -> closed
resolution: not a bug
messages: + msg121205
2010-11-14 17:19:33ned.deilysetmessages: + msg121204
2010-11-14 16:58:34r.david.murraysetnosy: + r.david.murray, ned.deily
messages: + msg121203
2010-11-14 16:44:57vduprassetmessages: + msg121202
2010-11-14 16:24:37pitrousetmessages: + msg121200
2010-11-14 16:04:48vduprassetmessages: + msg121199
2010-11-14 15:52:23pitrousetnosy: + ronaldoussoren
messages: + msg121198
2010-11-14 15:50:51r.david.murraysetnosy: + pitrou
2010-11-14 14:13:33vduprascreate