Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

popen spawned process may not write to stdout under windows #45707

Closed
pmezard mannequin opened this issue Oct 31, 2007 · 6 comments
Closed

popen spawned process may not write to stdout under windows #45707

pmezard mannequin opened this issue Oct 31, 2007 · 6 comments
Labels
OS-windows type-bug An unexpected behavior, bug, or error

Comments

@pmezard
Copy link
Mannequin

pmezard mannequin commented Oct 31, 2007

BPO 1366
Nosy @birkenfeld

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-12-05.09:46:32.846>
created_at = <Date 2007-10-31.09:59:43.695>
labels = ['type-bug', 'OS-windows']
title = 'popen spawned process may not write to stdout under windows'
updated_at = <Date 2008-12-05.09:46:32.845>
user = 'https://bugs.python.org/pmezard'

bugs.python.org fields:

activity = <Date 2008-12-05.09:46:32.845>
actor = 'georg.brandl'
assignee = 'none'
closed = True
closed_date = <Date 2008-12-05.09:46:32.846>
closer = 'georg.brandl'
components = ['Windows']
creation = <Date 2007-10-31.09:59:43.695>
creator = 'pmezard'
dependencies = []
files = []
hgrepos = []
issue_num = 1366
keywords = []
message_count = 6.0
messages = ['56991', '57201', '57259', '57261', '63687', '69829']
nosy_count = 5.0
nosy_names = ['georg.brandl', 'jafo', 'ggenellina', 'pmezard', 'pythonmeister']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1366'
versions = ['Python 2.5']

@pmezard
Copy link
Mannequin Author

pmezard mannequin commented Oct 31, 2007

Let child.py be:
"""
import sys

sys.stdout.write('1:stdout\n')
sys.stdout.flush()
sys.stderr.write('2:stderr\n')
sys.stderr.flush()
sys.stdout.write('3:stdout\n')
sys.stdout.flush()
"""

and parent.py:
"""
import os

cmd = 'python child.py'
for l in os.popen(cmd):
    print l,
"""

Then running it:
"""

python parent.py
1:stdout

python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
(Intel)] on win32
"""

I would have expected at least:
"""
1:stdout
3:stdout
"""
to be output, which actually happens if the stderr is nullified like
"python child.py 2>nul" in parent.py call.

Am I wrong ?

@pmezard pmezard mannequin added OS-windows type-bug An unexpected behavior, bug, or error labels Oct 31, 2007
@ggenellina
Copy link
Mannequin

ggenellina mannequin commented Nov 7, 2007

(I think the title you meant was
"popen spawned process may not
write to stderr under windows")

The child is dying with IOError:
[Errno 22] Invalid argument
at the sys.stderr.flush() call.

Neither the docs for os.popen nor
the Linux man page for popen(3)
say that stderr is redirected, so
one would expect the handle to be
inherited; the IOError looks like
a bug.
Try using os.popen4 or
popen2.popen4 or -the recommended
choice- the subprocess module.
Using the latter, this is the
modified parent.py:

"""
import subprocess
cmd = 'python child.py'
p = subprocess.Popen(cmd,
stdout=subprocess.PIPE)
for line in p.stdout:
print ">>>", line,
print p.wait()
"""

and this is the output, as
expected:

"""
2:stderr
>>> 1:stdout
>>> 3:stdout
0
"""

Note the 2:stderr line lacking the

>>, because it was printed
directly by the child process onto
the stderr handle inherited from
its parent.

@pythonmeister
Copy link
Mannequin

pythonmeister mannequin commented Nov 8, 2007

the popen call does not redirect stderr.
If you do something like 2>null (windows) or 2>/dev/null (*nix) it will
_never_ get printed.
If you want to have stderr & stdout getting in via popen and thus stdout,
under *nix and windows you would do that:

command 2>&1

It is not popen to blame.
See this for reference:
http://netbsd.gw.com/cgi-bin/man-cgi?popen++NetBSD-current

@pmezard
Copy link
Mannequin Author

pmezard mannequin commented Nov 8, 2007

pythonmeister: I never expected stderr to be redirected, just *all
stdout* to be captured. But...

gagenellina: you are completely right about the failure. Still, this
issue happened with a real world application written in C, and
redirecting manually stderr to :NUL: solved it unexpectedly. I did not
expect spawned process behaviour to differ when its stderr is being
redirected.

@jafo
Copy link
Mannequin

jafo mannequin commented Mar 17, 2008

We've discussed this at the PyCon sprints, and here's the concensus:

os.popen inherits the parents stderr, and on Windows there is not an
existing valid stderr by default. So the parent should, to be
compatible with the Windows environment, create stderr or use popen2.

However, popen is deprecated. The best solution would be to use
subprocess module which is defined to do the right thing in this case.
popen is not.

Because popen is deprecated, we are going to leave this behavior and
documentation as it is.

@birkenfeld
Copy link
Member

Did you want to close this, Sean?

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OS-windows type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

1 participant