This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Popen of python3.6 hangs on os.read(errpipe_read, 50000)
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: iritkatriel, liuxu1005, vstinner
Priority: normal Keywords:

Created on 2020-01-07 01:35 by liuxu1005, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg359486 - (view) Author: Xu (liuxu1005) Date: 2020-01-07 01:35
I have a piece code hangs on os.read(errpipe_read, 50000)
So I compared the python3.6 with python2.7 on _execute_child, I saw:

for python2.7 we create the errpipe_read/write with pipe_cloexec()

1213             # For transferring possible exec failure from child to parent
1214             # The first char specifies the exception type: 0 means
1215             # OSError, 1 means some other error.
1216             errpipe_read, errpipe_write = self.pipe_cloexec()


while for python3.6 we create the errpipe_read/write with pipe()

1251             # For transferring possible exec failure from child to parent.
1252             # Data format: "exception name:hex errno:description"
1253             # Pickle is not used; it is complex and involves memory allocation.
1254             errpipe_read, errpipe_write = os.pipe()

Does that mean python3.6 doesn't set the the flag FD_CLOEXEC on the pipe ?
msg359587 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-08 12:50
os.pipe() creates non-inheritable file descriptors by default, see PEP 446:

$ python3.6
Python 3.6.9 (default, Jul 26 2019, 00:00:00) 
>>> import os
>>> a,b=os.pipe()
>>> os.get_inheritable(a)
False
>>> os.get_inheritable(b)
False
msg359741 - (view) Author: Xu (liuxu1005) Date: 2020-01-10 17:13
Thanks Stinner.

Do you have some clues about this issue ?
In my case, our test system use Popen to start a few infrastructure processes for tests going. After tests done we will kill all processes and start again for new round.

Most of time it works fine.  However we have some chances where one Popen hangs on "os.read(errpipe_read, 50000) forever.

The close_fd is always true.

I guess there is some subtle issue here.
msg359743 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-10 17:21
I have no clue. Try to attach a debugger and try to inspect the Python frames. You can use gdb with python-gdb.py for that, for example. Or play with faulthandler, especially faulthandler.dump_traceback_later().
msg400916 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-09-02 11:02
I'm closing this because 

1. both 2.7 and 3.6 are no longer maintained
2. This is not an actual bug report (we don't know what the hanging code was doing) so there is no chance of reproducing it.

If you are having problems with this in a current version (>= 3.8) please create a new issue with a reproducible example of failing code.

If you need debugging assistance, python-list might be a more appropriate place than the bug tracker to ask for that.
History
Date User Action Args
2022-04-11 14:59:25adminsetgithub: 83422
2021-09-02 11:02:41iritkatrielsetstatus: open -> closed

nosy: + iritkatriel
messages: + msg400916

resolution: not a bug
stage: resolved
2020-01-10 17:21:08vstinnersetmessages: + msg359743
2020-01-10 17:13:12liuxu1005setmessages: + msg359741
2020-01-08 12:50:37vstinnersetnosy: + vstinner
messages: + msg359587
2020-01-07 01:35:08liuxu1005create