classification
Title: when forking, buffered output is not flushed first.
Type: behavior Stage:
Components: None Versions: Python 2.7, Python 2.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: gregory.p.smith, jort.bloem, r.david.murray, vstinner
Priority: normal Keywords:

Created on 2013-02-18 23:12 by jort.bloem, last changed 2013-02-19 06:37 by gregory.p.smith. This issue is now closed.

Files
File name Uploaded Description Edit
test.py jort.bloem, 2013-02-19 00:24
Messages (9)
msg182346 - (view) Author: jort bloem (jort.bloem) Date: 2013-02-18 23:12
When calling os.fork() without a tty, a process reporting the parent's pid runs code BEFORE the fork().

When running on a tty, it behaves as expected: both parent and child continue running from statement immediately after os.fork()

See attached test.py, compare running it with tty vs. without.

to run without tty: ssh localhost `pwd`/test.py
msg182347 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-02-18 23:15
There is no attached file.

I don't think that it's a bug that the child starts before the parent. It depends on the OS, OS version and many other things. You should use a synchronization mechanism to ensure the execution order.
msg182349 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-02-19 00:14
Haypo: I think he's saying that a statement in the source before the os.fork call is executed in the child, which seem rather unlikely.  Your suggestion may be what is happening to confuse him into thinking that, but without the sample program we can't evaluate this any further.
msg182350 - (view) Author: jort bloem (jort.bloem) Date: 2013-02-19 00:24
Try attachment again.
msg182351 - (view) Author: jort bloem (jort.bloem) Date: 2013-02-19 00:30
haypo: I understand that, after a fork, parent and child instructions are run in parallel; which one prints first is a matter of chance.

However, commands BEFORE THE FORK should not be re-run.

See test script. I would expect one "Start <pid>", followed by a "parent <pid>" and a "child <pid>". I would not expect to see (as I do) a second "Start <pid>".

(The original program was long and complex, with numerous forks; this is the smallest program I could write to show the problem).
msg182353 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-02-19 00:41
I only see Start printed once (on linux).  What OS are you running this on?
msg182354 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-02-19 00:49
I can reproduce the issue using "python test.py|cat". The problem is that sys.stdout is buffered and the buffer is flushed twice: once in the parent, once in the child. Just call sys.stdout.flush() before os.fork() should fix your issue.

I don't think that Python should flush buffers of all streams before fork, so I propose to close this issue. Except if you see something interesting to add to Python documentation.
msg182363 - (view) Author: jort bloem (jort.bloem) Date: 2013-02-19 06:03
I agree that it is reasonable NOT to flush stdout on fork().

I don't think the outcome is reasonable. 

What about voiding all buffers after the fork for the child?
msg182366 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2013-02-19 06:37
os.fork() is a low level system call wrapper.  Anyone using it needs to deal with flushing whatever buffers their application has before forking among many many other things.  There is a reason it lives in the os module.

It is already a dangerous system call to use from Python (ie: your child is likely to lock up if your parent had any threads).  There really is nothing we can or should do to make it better.
History
Date User Action Args
2013-02-19 06:37:50gregory.p.smithsetstatus: open -> closed
title: when forking without tty, code is run from start -> when forking, buffered output is not flushed first.
nosy: + gregory.p.smith

messages: + msg182366

resolution: wont fix
2013-02-19 06:03:15jort.bloemsetmessages: + msg182363
2013-02-19 00:49:11vstinnersetmessages: + msg182354
2013-02-19 00:41:03r.david.murraysetmessages: + msg182353
2013-02-19 00:30:21jort.bloemsetmessages: + msg182351
2013-02-19 00:24:48jort.bloemsetfiles: + test.py

messages: + msg182350
2013-02-19 00:14:43r.david.murraysetnosy: + r.david.murray
messages: + msg182349
2013-02-18 23:15:18vstinnersetnosy: + vstinner
messages: + msg182347
2013-02-18 23:12:53jort.bloemcreate