Created on 2011-04-10 07:45 by Thekent, last changed 2012-05-27 21:28 by roger.serwy.
|msg133452 - (view)||Author: kent (Thekent)||Date: 2011-04-10 07:45|
attempting to run an os.system command under the idle 3 shell swallows the out put. Idle 3 is running on a 32 bit kde mandriva linux. >>> import os >>> os.system('ls') 0 >>> os.system('pwd') 0 as you can see it returns a 0 indicating successful completion, but no output. However os.getcwd works perfectly. >>> os.getcwd() '/home/kent/Documents' running the same code from python in an xwindow terminal works fine. apparently the idle shell does not echo the the standard output or error output as the python interpreter does.
|msg133453 - (view)||Author: kent (Thekent)||Date: 2011-04-10 08:02|
running it as a file from idle gives the same result. import os print (os.system('pwd')) 0
|msg133493 - (view)||Author: kent (Thekent)||Date: 2011-04-11 01:07|
When starting idle from a terminal the output from the command is sent to the terminal. When starting idle from the desktop, the output disappears except for the exit status. Same behavior with 2.65
|msg133873 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2011-04-15 20:47|
I am not sure if this should be called a bug or feature request, but that does not matter so much with IDLE. Os.system is documented as executing in a subshell and returning the exit code, which is does. The doc also says "If command generates any output, it will be sent to the interpreter standard output stream." IDLE tries to imitate the interpreter, but it is not the interpreter, and I am not sure if that is always possible. The problem is that IDLE sends code (or, I presume, a filename) to a windowless interpreter (via socket or pipe) and receives and displays whatever is sent back. So I suspect the problem and fix is and would have to be with how a windowless interpreter executes os.system (in a third process). But for all I know, it may be the OS that decides not to hook the output of a system process to a no-window process that calls it. On Windows, os.system('dir') ('dir' == 'ls') within IDLE pops up a command window to display the output, which immediately disappears. The same within the interactive interpreter (in a Command Prompt window) displays the output, just as with your XTerminal case. Os.getcwd is documented as returning a string, so of course it does. It is not relevant to this issue. Because of problems with os.system, the docs end with a suggestion to use subprocess instead. So there may be reluctance to 'fix' os.system calls. The subprocess doc has an example for 'ls'. For 3.2 it is >>> subprocess.check_output(["ls", "-l", "/dev/null"]) b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n' but the quotes and \n suggest that multiline output would not display properly. On Windows with 3.2, the following works >>> print(subprocess.check_output(['dir'], shell=True).decode()) Volume in drive C is HP_PAVILION Volume Serial Number is 6C44-B700 Directory of C:\Programs\Python32 shell=True is needed for 'dir' to be recognized. Both print and .decode() are needed for proper line breaks. The same info for the current directory is also available in an Open File or Save File dialog, so the ls/dir is really not needed.
|msg133877 - (view)||Author: kent (Thekent)||Date: 2011-04-16 01:52|
I tried using subprocess.Popen and subprocess.call, both of which did the same behavior. Under the interpreter I get the desired string output: >>> subprocess.call('ls') bin Documents eclipse local Pictures tmp workspace Desktop Downloads hamlib Music Templates Videos 0 Under Idle: >>> subprocess.call('ls') 0 Thus the subprocess.call method provides the string output. Why should I have to use the subprocess.check_output('command').decode() when subprocess.call() gives me what I want? Thus it does not seem to be an os.system issue, but a failure of Idle to capture the sysoutput string as the interpreter does The xterminal example highlights this. The sysoutput is echoed at the xterminal not in idle. You are correct there are other ways to get the specific information for ls. I was using that as test command. I was experimenting with the commands in order to a write a program which will run an executable under either windows or linux. I wanted to see the output in the interpreter in order to test it. I was using Idle for the testing. It does NOT work the same as the interpreter. If Idle is to be useful as an IDE, shouldn't it's shell work the same as the interpreter? If it doesn't why use Idle ?
|msg133881 - (view)||Author: Ned Deily (ned.deily) *||Date: 2011-04-16 06:17|
I agree that it is kind of odd behavior and, after a quick look back through open issues, I was a bit surprised to not find an open issue about it (although I may have overlooked one). "Thus it does not seem to be an os.system issue, but a failure of Idle to capture the sysoutput string as the interpreter does The xterminal example highlights this. The sysoutput is echoed at the xterminal not in idle." The reason this is an issue is because the interpreter does *not* do anything special to capture stdout from subprocesses. If you're running the interpreter in a terminal window both stdout from the interpreter process and, by default, stdout from any child processes, like those created by os.system or subprocess (if you don't override it), inherit the same stdout and so the output from both processes show up in the same place (i.e. terminal session) automatically. When running under IDLE though, IDLE uses its own proxy output object to intercept the output from the interpreter but does not do anything special for stdout from subprocesses so that output ends up in unexpected places: the terminal session from which IDLE was launched or the transient window on Windows or into the system log with the OS X IDLE.app. I think IDLE could do a better job of intercepting stdout for these cases but it might be a bit tricky to make it work across all platforms.
|msg133885 - (view)||Author: kent (Thekent)||Date: 2011-04-16 07:32|
I had kind of figured it might be something like this. I ran the following code in the xterm interpreter: >>> x=subprocess.call('ls') bin Documents eclipse local Pictures tmp workspace Desktop Downloads hamlib Music Templates Videos >>> print(x) 0 It does not capture the output.
|msg133886 - (view)||Author: kent (Thekent)||Date: 2011-04-16 07:50|
The getoutput and getstatusoutput provide the expect output which can be captured >>> x=subprocess.getoutput('ls') >>> print(x) hs_err_pid28274.log LP4E-examples mydir.pth mydir.pth~ PP4E-Examples-1.2 ProgMan Python_dir Would it be a good thing to have the interpreter capture the sysout of all child processes automatically?
|msg161717 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2012-05-27 19:18|
After thinking some more and re-reading the os.system doc, I am closing this. I think things are working exactly as documented, and in a way that will not be changed. The os.system doc initially says 1. "Execute the command (a string) in a subshell." It does that. 2. "This is implemented by calling the Standard C function system(), and has the same limitations." Read Standard C for more on that. Nearly everything in os is a thin wrapper of operating system data and functions. Enhanced wrappers that add user-friendliness, go elsewhere. 3. "Changes to sys.stdin, etc. are not reflected in the environment of the executed command." The sys module is Python-specific and has no effect on system functions. In particular, whether sys.stdout, initially copied from the process, is initally null or the terminal, Idle changing sys.stdout has no effect on os.system output, and it will not see it. This is, obviously, a known and expected limitation. 4. "If command generates any output, it will be sent to the interpreter standard output stream." The interpreter process stdout is not sys.stdout. If it is null, bye-bye output. The doc ends with 5. "The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function." In other words, the limitations of os.system are known, and it is semi-deprecated, especially when it does not work as one would wish, and a replacement has been provided. The subprocess module is an example of an 'enhanced wrapper'. 6 "See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes." These recipes are replacements for things that work. The .check_output method enables things that one could not very easily do before. I already gave examples of how to use it.
|msg161720 - (view)||Author: Ned Deily (ned.deily) *||Date: 2012-05-27 20:12|
I disagree. I think you are confusing Python's sys.std* objects with the std* file descriptors of a process. The important points here are that IDLE is effectively substituting its rpcproxy for the std* file descriptors of the IDLE shell process that it creates *and* that, in POSIX world, a forked subprocess is required to inherit copies of the file descriptors (in particular, sys.std*) of its parent (the IDLE shell subprocess). So, if the IDLE shell subprocess displays the output written to the stdout and stderr in the Python Shell window and directs typed-in input from that window to the stdin file descriptor, it is reasonable to assume that the same thing should happen by default to processes launched within that IDLE shell subprocess, be they by os.system or by subprocess.Popen. subprocess.Popen lets you substitute sys* file associations when creating the subprocess but that is irrelevant for the default case. The default for subprocess.Popen is exactly the same as os.system: "With the default settings of None, no redirection will occur; the child’s file handles will be inherited from the parent."
|msg161724 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2012-05-27 20:40|
I thought I was un-confusing sys.std* Python objects (such as idlelib.rpc.RPCProxy) from std* integer file descriptors, whereas you seem to say they are the same: "copies of the file descriptors (in particular, sys.std*)". The rest of your message seems to imply that subprocess.check_output should not work, but it does, at least on Windows. And subprocess *is* a recommended replacement for os.system. Roger, do you have any thoughts on this issue?
|msg161732 - (view)||Author: Ned Deily (ned.deily) *||Date: 2012-05-27 21:12|
I didn't mean to imply anything about subprocess.check_output(). It works as expected, that is by explicitly intercepting anything written to the stdout of the forked subprocess and returning it as the call result. What doesn't work is if you try a default subprocess.call(), for example. The results are the same as os.system and for the same reason. In a Posix terminal session: $ python3.3 >>> import subprocess >>> subprocess.call('pwd') /Users/nad 0 >>> But in an IDLE shell window: >>> import subprocess >>> subprocess.call('pwd') 0 >>> That's confusing to users. IDLE should handle that.
|msg161737 - (view)||Author: Roger Serwy (roger.serwy) *||Date: 2012-05-27 21:28|
It might be possible to fix this problem by creating pipes to redirect stdout and stderr to the text widget in PyShell. The ShellWindow.py demo in Demo/tkinter/guido may be helpful.
|2012-05-27 21:28:00||roger.serwy||set||messages: + msg161737|
|2012-05-27 21:12:31||ned.deily||set||messages: + msg161732|
messages: + msg161724
|2012-05-27 20:12:16||ned.deily||set||status: closed -> open|
resolution: invalid ->
messages: + msg161720
stage: committed/rejected -> needs patch
|2012-05-27 19:18:49||terry.reedy||set||status: open -> closed|
messages: + msg161717
stage: needs patch -> committed/rejected
|2011-04-16 07:50:46||Thekent||set||messages: + msg133886|
components: + Interpreter Core
messages: + msg133881
components: - Interpreter Core
stage: needs patch
|2011-04-16 01:52:28||Thekent||set||messages: + msg133877|
+ Python 3.2, Python 3.3, - Python 3.1|
nosy: + terry.reedy
messages: + msg133873
components: + Interpreter Core
|2011-04-11 01:07:19||Thekent||set||messages: + msg133493|
|2011-04-10 08:02:32||Thekent||set||messages: + msg133453|