classification
Title: multiprocessing calls flush on sys.stdout at exit even if it is None (pythonw)
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder: multiprocessing.Process depends on sys.stdout being open
View: 28326
Assigned To: Nosy List: Pox TheGreat, davin, pitrou, terry.reedy
Priority: normal Keywords: patch

Created on 2017-10-17 14:59 by Pox TheGreat, last changed 2018-01-12 12:55 by Pox TheGreat.

Files
File name Uploaded Description Edit
process.py.patch Pox TheGreat, 2017-10-17 14:59 patch file with trivial bugfix
Messages (8)
msg304512 - (view) Author: Pox TheGreat (Pox TheGreat) * Date: 2017-10-17 14:59
If you start Python by pythonw then sys.stdout and sys.stderr are set to None. If you also use multiprocessing then when the child process finishes BaseProcess._bootstrap calls sys.stdout.flush() and sys.stderr.flush() finally. This causes the process return code to be not zero (it is 1).
msg304678 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-10-20 20:39
This looks to be a duplicate of https://bugs.python.org/issue28326
msg309708 - (view) Author: Pox TheGreat (Pox TheGreat) * Date: 2018-01-09 16:11
Unfortunately this is NOT a duplicate of https://bugs.python.org/issue28326. That issue is about a closed output stream. In that case sys.stdout and sys.stderr are file like objects which have been closed.

This issue is about sys.stdout and sys.stderr being None! This is because pythonw was used not python.
msg309716 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-01-09 16:51
@Pox, nevertheless, the fix committed in https://github.com/python/cpython/pull/4073 should also fix this issue. Do you disagree?
msg309721 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-01-09 17:38
Pox, please retest with either 3.6.4 or 3.7.0a3 (or .0a4 when released, soon).  Both were released after the merge that should fix this.
msg309759 - (view) Author: Pox TheGreat (Pox TheGreat) * Date: 2018-01-10 09:33
Retested it with a freshly installed 3.6.4 version. Used the following code to test:

import sys
import multiprocessing


def foo():
    return 'bar'


if __name__ == '__main__':
    proc = multiprocessing.Process(target=foo)
    proc.start()
    proc.join()
    with open('process_exit_code.txt', 'w') as f:
        f.write(sys.version)
        f.write('\nprocess exit code: ')
        f.write(str(proc.exitcode))

It is very important to run the script with pythonw, not just with python. This is the content of the resulting process_exit_code.txt file on my machine:
3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 bit (Intel)]
process exit code: 1

As it can be seen the problem was not fixed. The process exit code should be 0. By default the new multiprocessing process created uses the same interpreter as the creator process, so it uses pythonw too.
msg309762 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-01-10 10:59
I'm not using Windows, so I'm unable to test using pythonw.  You'll have to provide a fix, or someone else will have to.
msg309853 - (view) Author: Pox TheGreat (Pox TheGreat) * Date: 2018-01-12 12:55
I have already uploaded a patch file but it is not in the required format. Also I realize that most of the confusion was because I forgot to provide the OS version. Perhaps it would be good to have a separate field for that.

I will upload a patch as it is described in the developer guide.
History
Date User Action Args
2018-01-12 12:55:47Pox TheGreatsettype: crash -> behavior
messages: + msg309853
2018-01-10 10:59:53pitrousetmessages: + msg309762
2018-01-10 09:33:29Pox TheGreatsetmessages: + msg309759
2018-01-09 17:38:16terry.reedysetmessages: + msg309721
2018-01-09 16:51:06pitrousetmessages: + msg309716
2018-01-09 16:11:04Pox TheGreatsetstatus: closed -> open
resolution: duplicate ->
messages: + msg309708
2017-10-20 20:39:55pitrousetstatus: open -> closed
superseder: multiprocessing.Process depends on sys.stdout being open
messages: + msg304678

resolution: duplicate
stage: resolved
2017-10-20 18:47:31terry.reedysetversions: + Python 2.7, Python 3.6, Python 3.7, - Python 3.5
2017-10-20 18:47:06terry.reedysetnosy: + terry.reedy, pitrou, davin
2017-10-17 14:59:34Pox TheGreatcreate