classification
Title: Interpreter not closing stdout/stderr on exit
Type: behavior Stage: resolved
Components: IO Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: filip.zyzniewski, neologix
Priority: normal Keywords:

Created on 2012-12-07 13:33 by filip.zyzniewski, last changed 2013-08-02 09:30 by neologix. This issue is now closed.

Messages (4)
msg177090 - (view) Author: Filip Zyzniewski (filip.zyzniewski) Date: 2012-12-07 13:33
When using a Python script as a unix pipe filter with its stdout redirected to a file:

python script.py > /nfs/foo

user is not notified of some writing problems on NFS, because these are sometimes reported on close(), and the interpreter never does neither close(1) nor close(2):

$ strace -eclose python -c '' 2>&1 | grep  'close([12])'
$
msg177093 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2012-12-07 13:52
stdout and stderr are not closed, because in 99.9999% of cases it's useless: they are closed upon the program exit.
Also, imagine what would happend if an error occured after closing them, we would be unable to log it.

If you want to avoid this kind of problem, you should call:
"""
os.fsync(1)
os.fsync(2)
"""

when you're done.

Note that you'll have the same problem with shell scripts and any other executable...
msg177094 - (view) Author: Filip Zyzniewski (filip.zyzniewski) Date: 2012-12-07 14:08
If stdout was closed before closing stderr, then stdout problems could be reported, and that is what I would expect when using Python this way.

os.fsync(1) helps, but only if preceeded by sys.stdout.flush() and it seems a bit cumbersome.

Is there any downside to doing close(1) explicitely?

cat, grep, dd and echo close their stdout, why couldn't Python do this?
msg177097 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2012-12-07 14:57
> If stdout was closed before closing stderr, then stdout problems
> could be reported, and that is what I would expect when using Python
> this way.

To be consistent, we should also close stderr, and then we couldn't
report any subsequent error.

> os.fsync(1) helps, but only if preceeded by sys.stdout.flush() and it
> seems a bit cumbersome.

You should complain to the POSIX committee ;-)
The thing is simply that if you want to make sure that data is
committed (or get an error otherwise), you should call fsync() on the
file descriptor. Just having close return 0 doesn't guarantee that
data is committed, in the general case (NFS is a special case with its
close-to-open cache consistency).

> cat, grep, dd and echo close their stdout, why couldn't Python do this?

bash, dmesg, ifconfig and many others don't.

It could probably be possible, but I don't think the benefit is worth
the trouble.
History
Date User Action Args
2013-08-02 09:30:27neologixsetstatus: open -> closed
resolution: rejected
stage: resolved
2012-12-07 14:57:56neologixsetmessages: + msg177097
2012-12-07 14:08:03filip.zyzniewskisetmessages: + msg177094
2012-12-07 13:52:51neologixsetnosy: + neologix
messages: + msg177093
2012-12-07 13:33:50filip.zyzniewskicreate