classification
Title: Occasional failure in test_ioctl when run parallel with test_gdb
Type: behavior Stage: resolved
Components: Library (Lib), Tests Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, ezio.melotti, martin.panter, neologix, pitrou, python-dev, serhiy.storchaka
Priority: low Keywords: patch

Created on 2012-05-10 18:19 by pitrou, last changed 2016-01-16 05:24 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
gdb-stdin.patch martin.panter, 2015-12-07 05:08 review
Messages (19)
msg160357 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-05-10 18:19
This happens from time to time on my desktop computer:

[151/354/1] test_ioctl
test test_ioctl failed -- Traceback (most recent call last):
  File "/home/antoine/cpython/32/Lib/test/test_ioctl.py", line 36, in test_ioctl
    self.assertIn(rpgrp, ids)
AssertionError: 25242 not found in (23865, 23615)
msg171360 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-09-26 20:19
Can you still reproduce this issue? Do you use any specific flag to run the tests? What OS is this?
I tried on linux, solaris, and aix with ``./python -m test -F test_ioctl`` for about 1k loops, and couldn't reproduce the issue.
msg180467 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-23 16:13
Third argument of ioctl should be not "    ", but a bytes object with platform dependent length sizeof(pid_t).

Beside this, the code at the top of the module checks the same condition and skip the tests if it is wrong. Race condition happened -- the process has been put into the background between executing these two codes. What command line you run the test?
msg180468 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-23 16:26
> Beside this, the code at the top of the module checks the same
> condition and skip the tests if it is wrong. Race condition happened
> -- the process has been put into the background between executing
> these two codes. What command line you run the test?

It happens (or happened) quite often when using the -j option to regrtest.
msg180470 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-23 16:45
Do you use -r option?
msg180472 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-23 16:47
> Serhiy Storchaka added the comment:
> 
> Do you use -r option?

No. Usually just -j<something>
msg180475 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-23 16:59
What is <something>? Can you please expose several lines of regrtest output before and after this error? When "/1" appeared?

I suspect test_gdb:

$ ./python -m test.regrtest -j 2 test_gdb test_gdb

[1]+  Stopped                 ./python -m test.regrtest -j 2 test_gdb test_gdb
msg180476 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-23 17:01
$ ./python -m test.regrtest -j 2 test_gdb test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl
[ 1/10] test_ioctl
[ 2/10] test_ioctl
[ 3/10] test_ioctl
[ 4/10] test_ioctl
[ 5/10] test_ioctl
[ 6/10] test_ioctl
[ 7/10] test_ioctl
[ 8/10] test_ioctl
test_ioctl skipped -- Neither the process group nor the session are attached to /dev/tty
[ 9/10] test_ioctl
test_ioctl skipped -- Neither the process group nor the session are attached to /dev/tty
[10/10] test_gdb
8 tests OK.
2 tests skipped:
    test_ioctl test_ioctl
msg180477 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-23 17:03
$ ./python -m test.regrtest -j 4 test_gdb test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl
[ 1/20] test_ioctl
[ 2/20] test_ioctl
[ 3/20] test_ioctl
[ 4/20] test_ioctl
[ 5/20] test_ioctl
[ 6/20] test_ioctl
[ 7/20] test_ioctl
[ 8/20] test_ioctl
[ 9/20] test_ioctl
[10/20] test_ioctl
[11/20] test_ioctl
[12/20] test_ioctl
[13/20] test_ioctl
[14/20] test_ioctl
[15/20] test_ioctl
[16/20] test_ioctl
[17/20] test_ioctl
[18/20] test_ioctl
test_ioctl skipped -- Neither the process group nor the session are attached to /dev/tty
[19/20] test_ioctl
[20/20/1] test_gdb
test test_gdb failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_gdb.py", line 716, in test_threads
    cmds_after_breakpoint=['thread apply all py-bt'])
  File "/home/serhiy/py/cpython/Lib/test/test_gdb.py", line 170, in get_stack_trace
    self.assertEqual(err, '')
AssertionError: 'Error occurred in Python command.\n' != ''
- Error occurred in Python command.
18 tests OK.
1 test failed:
    test_gdb
1 test skipped:
    test_ioctl
msg180478 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-23 17:05
Bingo!

$ ./python -m test.regrtest -j 4 test_gdb test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl test_ioctl
[ 1/20] test_ioctl
[ 2/20] test_ioctl
[ 3/20] test_ioctl
[ 4/20] test_ioctl
[ 5/20] test_ioctl
[ 6/20] test_ioctl
[ 7/20] test_ioctl
[ 8/20] test_ioctl
[ 9/20] test_ioctl
test_ioctl skipped -- Neither the process group nor the session are attached to /dev/tty
[10/20] test_ioctl
[11/20/1] test_ioctl
test test_ioctl failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_ioctl.py", line 66, in test_ioctl_mutate_2048
    self._check_ioctl_mutate_len(2048)
  File "/home/serhiy/py/cpython/Lib/test/test_ioctl.py", line 54, in _check_ioctl_mutate_len
    self.assertIn(rpgrp, ids)
AssertionError: 14270 not found in (14225, 4030)
[12/20/1] test_ioctl
test_ioctl skipped -- Neither the process group nor the session are attached to /dev/tty
[13/20/1] test_ioctl
[14/20/1] test_ioctl
[15/20/1] test_ioctl
[16/20/1] test_ioctl
[17/20/1] test_ioctl
test_ioctl skipped -- Neither the process group nor the session are attached to /dev/tty
[18/20/1] test_ioctl
[19/20/1] test_ioctl
[20/20/1] test_gdb
16 tests OK.
1 test failed:
    test_ioctl
3 tests skipped:
    test_ioctl test_ioctl test_ioctl
msg180479 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-23 17:11
Nice catch, Serhiy. I wouldn't have suspected test_gdb.
msg180481 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-23 17:40
Perhaps we need a possibility to mark a test that it can't run in parallel. 
I suspect some multiprocess tests fail for same reason.
msg180494 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-23 19:08
> Perhaps we need a possibility to mark a test that it can't run in parallel. 
> I suspect some multiprocess tests fail for same reason.

Well, first I would like to know what the underlying problem is.
-j doesn't use multiprocessing, it spawns standalone processes using the subprocess module.
msg180496 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-23 19:20
Ah, I see, that's because of the "Skip if another process is in foreground" thing?
msg180497 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-23 19:34
Given what _check_ioctl_mutate_len() seems to be designed to check, perhaps we should relax:

        self.assertIn(rpgrp, ids)

into:

        self.assertNotEqual(rpgrp, fill)

or perhaps simply:

        self.assertGreater(rpgrp, 0)
msg180498 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-23 19:54
Ah, and the test_ioctl() method needs to be fixed too.
msg221850 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-29 15:41
@Antoine/Serhiy I believe that you'd want to follow up on this.
msg256049 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-07 05:08
I found that if you redirect GDB’s stdin if fixes all the problems. I don’t fully understand what GDB is doing to change the terminal’s foreground process group.

In my patch I set stdin to a pipe. It is like using DEVNULL, but should be compatible with Python 2 as well.
msg258358 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-01-16 05:21
New changeset 8b8ac7adbf49 by Martin Panter in branch '2.7':
Issue #14771: Redirect GDB's stdin to avoid messing the terminal settings
https://hg.python.org/cpython/rev/8b8ac7adbf49

New changeset c87cc05af8e7 by Martin Panter in branch '3.5':
Issue #14771: Redirect GDB's stdin to avoid messing the terminal settings
https://hg.python.org/cpython/rev/c87cc05af8e7
History
Date User Action Args
2016-01-16 05:24:04martin.pantersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2016-01-16 05:21:43python-devsetnosy: + python-dev
messages: + msg258358
2015-12-07 05:08:07martin.pantersetfiles: + gdb-stdin.patch

versions: + Python 3.5, Python 3.6, - Python 3.2, Python 3.3, Python 3.4
keywords: + patch
nosy: + martin.panter

messages: + msg256049
stage: needs patch -> patch review
2014-06-29 15:41:55BreamoreBoysetnosy: + BreamoreBoy
messages: + msg221850
2013-01-23 19:54:20pitrousetmessages: + msg180498
2013-01-23 19:34:46pitrousetmessages: + msg180497
2013-01-23 19:20:40pitrousetmessages: + msg180496
2013-01-23 19:08:37pitrousetmessages: + msg180494
2013-01-23 17:40:46serhiy.storchakasetmessages: + msg180481
2013-01-23 17:11:50pitrousettitle: Occasional failure in test_ioctl -> Occasional failure in test_ioctl when run parallel with test_gdb
stage: needs patch
versions: + Python 3.4
2013-01-23 17:11:29pitrousetmessages: + msg180479
2013-01-23 17:05:44serhiy.storchakasetmessages: + msg180478
2013-01-23 17:03:51serhiy.storchakasetmessages: + msg180477
2013-01-23 17:01:14serhiy.storchakasetmessages: + msg180476
2013-01-23 16:59:52serhiy.storchakasetmessages: + msg180475
2013-01-23 16:47:11pitrousetmessages: + msg180472
2013-01-23 16:45:40serhiy.storchakasetmessages: + msg180470
2013-01-23 16:26:28pitrousetmessages: + msg180468
2013-01-23 16:13:51serhiy.storchakasetstatus: pending -> open
nosy: + serhiy.storchaka
messages: + msg180467

2013-01-23 09:15:39ezio.melottisetstatus: open -> pending
2012-09-26 20:19:26ezio.melottisetnosy: + ezio.melotti
messages: + msg171360
2012-05-10 18:19:12pitroucreate