classification
Title: os.popen() negative error code IOError
Type: Stage:
Components: Windows Versions: Python 2.6, Python 2.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: mhammond Nosy List: asmodai, facundobatista, gvanrossum, mhammond, pmoore, skip.montanaro, tcdelaney
Priority: normal Keywords:

Created on 2002-08-30 06:25 by tcdelaney, last changed 2008-01-19 11:21 by facundobatista. This issue is now closed.

Files
File name Uploaded Description Edit
error.py skip.montanaro, 2002-08-30 12:38
using_subprocess.py facundobatista, 2008-01-19 11:08 Same example but using subprocess module
Messages (13)
msg12199 - (view) Author: Tim Delaney (tcdelaney) Date: 2002-08-30 06:25
Windows 2000, Python 2.0.1, 2.1.1, 2.2.

When a negative return code is received by the os.popen
() family, an IOError is raised when the last pipe from the 
process is closed.

The following code demonstrates the problem:

import sys
import os
import traceback

if __name__ == '__main__':

    if len(sys.argv) == 1:
        
        try:
            r = os.popen('%s %s %s' % (sys.executable, 
sys.argv[0], -1,))
            r.close()
        except IOError:
            traceback.print_exc()

        try:
            w, r = os.popen2('%s %s %s' % 
(sys.executable, sys.argv[0], -1,))
            w.close()
            r.close()
        except IOError:
            traceback.print_exc()

        try:
            w, r, e = os.popen3('%s %s %s' % 
(sys.executable, sys.argv[0], -1,))
            w.close()
            r.close()
            e.close()
        except IOError:
            traceback.print_exc()

    else:
        sys.exit(int(sys.argv[1]))

---------- Run ----------
Traceback (most recent call last):
  File "Q:\Viper\src\webvis\tests\test.py", line 11, in ?
    r.close()
IOError: (0, 'Error')
Traceback (most recent call last):
  File "Q:\Viper\src\webvis\tests\test.py", line 18, in ?
    r.close()
IOError: (0, 'Error')
Traceback (most recent call last):
  File "Q:\Viper\src\webvis\tests\test.py", line 26, in ?
    e.close()
IOError: (0, 'Error')
msg12200 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2002-08-30 12:38
Logged In: YES 
user_id=44345

Attached Tim's code so it's more readily accessible to testers.
BTW, works fine for me on Linux.
msg12201 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-08-30 14:05
Logged In: YES 
user_id=6380

I suppose this is one for Mark Hammond...
msg12202 - (view) Author: Mark Hammond (mhammond) * (Python committer) Date: 2004-05-05 12:57
Logged In: YES 
user_id=14198

I'm not sure what to do about this.  Python internally
checks for -1 from the close function.  The safest thing to
do would be to translate the return code from -1 to
something else.  Changing the fclose semantics to check for
PyErr_Occurred is another alternative, but doesn't seem that
reasonable just for this.  Any thoughts?
msg12203 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2004-05-05 13:18
Logged In: YES 
user_id=6380

How about changing the line "result = exit_code;" in
_PyPclose() to truncate the value to the lower 16 bits,
ensuring it is always >= 0? I think that should do it.
msg12204 - (view) Author: Mark Hammond (mhammond) * (Python committer) Date: 2004-05-09 23:45
Logged In: YES 
user_id=14198

I tried truncating as you suggest, and this works fine -
although it gives different results than Linux.  Digging
deeper, I found for linux:
"""
The value from the wait() system call is split into three parts
mask 0xff00 is the exit status
mask 0x007f is the signal causing the exit
mask 0x0080 is the cord dumped flag in combination with a signal
"""

To get the same results as Linux for negative exit codes, it
seems:
         result = (int)((exit_code & 0x000000FF) << 8);

Will do the job.  Counter-intuitively, this will make
sys.exit(-1) yield an exit code of 65280, but this exactly
what happens on Linux.

Should we try and strive for compatibility within Python, or
compatibility with the host OS?  I lean towards
compatibility with Linux - if they want win32 specific
stuff, use win32 specific code.

Thoughts?
msg12205 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2004-05-12 03:54
Logged In: YES 
user_id=6380

Yeah, that behavior is not just Linux, but any sane Unix
(although the POSIX standard abstracts it away in macros). I
think it's right to do that.
msg12206 - (view) Author: Paul Moore (pmoore) * Date: 2004-05-12 08:32
Logged In: YES 
user_id=113328

Do you propose to implement all of the WIFEXITED,
WEXITSTATUS series of functions for Windows as well?
Expecting people to unpack the return value manually -
especially Windows users who won't be aware of the Unix
heritage of the format - is probably unreasonable.

If you implement the W... functions, I'm OK with this
change. Otherwise I'm -1. After all, many uses of popen are
probably to run non-portable programs anyway, so portability
is only one aspect of the issue... 
msg60140 - (view) Author: Jeroen Ruigrok van der Werven (asmodai) * (Python committer) Date: 2008-01-19 10:11
Current output on XP with 2.5.1:

D:\>test.py
Traceback (most recent call last):
  File "D:\test.py", line 12, in <module>
    r.close()
IOError: [Errno 0] Error
Traceback (most recent call last):
  File "D:\test.py", line 20, in <module>
    r.close()
IOError: [Errno 0] Error
Traceback (most recent call last):
  File "D:\test.py", line 29, in <module>
    e.close()
IOError: [Errno 0] Error
msg60148 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-01-19 11:07
os.popen* are deprecated, we should use subprocess now... could you
please try what happens with this alternate example I'm attaching?

Thank you!
msg60149 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-01-19 11:08
Boo, I forgot the attach, :p
msg60150 - (view) Author: Jeroen Ruigrok van der Werven (asmodai) * (Python committer) Date: 2008-01-19 11:12
Using Facundo's attachment:

D:\>using_subprocess.py

D:\>
msg60152 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-01-19 11:21
Great! So it's fixed, :)
History
Date User Action Args
2008-01-19 11:21:50facundobatistasetstatus: open -> closed
resolution: out of date
messages: + msg60152
2008-01-19 11:12:09asmodaisetmessages: + msg60150
2008-01-19 11:08:16facundobatistasetfiles: + using_subprocess.py
messages: + msg60149
2008-01-19 11:07:36facundobatistasetnosy: + facundobatista
messages: + msg60148
2008-01-19 10:11:08asmodaisetnosy: + asmodai
messages: + msg60140
2008-01-05 18:12:50christian.heimessetversions: + Python 2.6, Python 2.5
2002-08-30 06:25:49tcdelaneycreate