attempting to run an os.system command under the idle 3 shell swallows the out put.
Idle 3 is running on a 32 bit kde mandriva linux.

>>> import os
>>> os.system('ls')
0
>>> os.system('pwd')
0


as you can see it returns a 0 indicating successful completion, but no output. However os.getcwd works perfectly.

>>> os.getcwd()
'/home/kent/Documents' 

running the same code from python in an xwindow terminal works fine.

apparently the idle shell does not echo the the standard output or error output as the python interpreter does.

running it as a file from idle gives the same result.
import os
print (os.system('pwd'))

0

When starting idle from a terminal the output from the command is sent to the terminal.  When starting idle from the desktop, the output disappears except  for the exit status.  Same behavior with 2.65

I am not sure if this should be called a bug or feature request, but that does not matter so much with IDLE. 

Os.system is documented as executing in a subshell and returning the exit code, which is does. The doc also says "If command generates any output, it will be sent to the interpreter standard output stream."

IDLE tries to imitate the interpreter, but it is not the interpreter, and I am not sure if that is always possible. The problem is that IDLE sends code (or, I presume, a filename) to a windowless interpreter (via socket or pipe) and receives and displays whatever is sent back. So I suspect the problem and fix is and would have to be with how a windowless interpreter executes os.system (in a third process). But for all I know, it may be the OS that decides not to hook the output of a system process to a no-window process that calls it.

On Windows, os.system('dir') ('dir' == 'ls') within IDLE pops up a command window to display the output, which immediately disappears. The same within the interactive interpreter (in a Command Prompt window) displays the output, just as with your XTerminal case.

Os.getcwd is documented as returning a string, so of course it does. It is not relevant to this issue.

Because of problems with os.system, the docs end with a suggestion to use subprocess instead. So there may be reluctance to 'fix' os.system calls.

The subprocess doc has an example for 'ls'. For 3.2 it is

>>> subprocess.check_output(["ls", "-l", "/dev/null"])
b'crw-rw-rw- 1 root root 1, 3 Oct 18  2007 /dev/null\n'

but the quotes and \n suggest that multiline output would not display properly.

On Windows with 3.2, the following works

>>> print(subprocess.check_output(['dir'], shell=True).decode())
 Volume in drive C is HP_PAVILION
 Volume Serial Number is 6C44-B700

 Directory of C:\Programs\Python32

shell=True is needed for 'dir' to be recognized.
Both print and .decode() are needed for proper line breaks.

The same info for the current directory is also available in an Open File or Save File dialog, so the ls/dir is really not needed.

I tried using subprocess.Popen and subprocess.call, both of which did the same behavior. Under the interpreter I get the desired string output:
>>> subprocess.call('ls')
bin      Documents  eclipse  local  Pictures   tmp     workspace
Desktop  Downloads  hamlib   Music  Templates  Videos
0

Under Idle:

>>> subprocess.call('ls')
0
Thus the subprocess.call method provides the string output.  Why should I have to use the subprocess.check_output('command').decode() when subprocess.call() gives me what I want?

Thus it does not seem to be an os.system issue, but a failure of Idle to capture the sysoutput string as the interpreter does  The xterminal example highlights this.  The sysoutput is echoed at the xterminal not in idle.

You are correct there are other ways to get the specific information for ls.  I was using that as test command.  I was experimenting with the commands in order to a write a program which will run an executable under either windows or linux.  I wanted to see the output in the interpreter in order to test it.  I was using Idle for the testing.  It does NOT work the same as the interpreter.  If Idle is to be useful as an IDE, shouldn't it's shell work the same as the interpreter? If it doesn't why use Idle ?

I agree that it is kind of odd behavior and, after a quick look back through open issues, I was a bit surprised to not find an open issue about it (although I may have overlooked one).

"Thus it does not seem to be an os.system issue, but a failure of Idle to capture the sysoutput string as the interpreter does  The xterminal example highlights this.  The sysoutput is echoed at the xterminal not in idle."

The reason this is an issue is because the interpreter does *not* do anything special to capture stdout from subprocesses.  If you're running the interpreter in a terminal window both stdout from the interpreter process and, by default, stdout from any child processes, like those created by os.system or subprocess (if you don't override it), inherit the same stdout and so the output from both processes show up in the same place (i.e. terminal session) automatically.  When running under IDLE though, IDLE uses its own proxy output object to intercept the output from the interpreter but does not do anything special for stdout from subprocesses so that output ends up in unexpected places: the terminal session from which IDLE was launched or the transient window on Windows or into the system log with the OS X IDLE.app.  I think IDLE could do a better job of intercepting stdout for these cases but it might be a bit tricky to make it work across all platforms.

I had kind of figured it might be something like this.  I ran the following code in the xterm interpreter:
>>> x=subprocess.call('ls')
bin      Documents  eclipse  local  Pictures   tmp     workspace
Desktop  Downloads  hamlib   Music  Templates  Videos
>>> print(x)
0
It does not capture the output.

The getoutput and getstatusoutput provide the expect output which can be captured

>>> x=subprocess.getoutput('ls')
>>> print(x)
hs_err_pid28274.log
LP4E-examples
mydir.pth
mydir.pth~
PP4E-Examples-1.2
ProgMan
Python_dir


Would it be a good thing to have the interpreter capture the sysout of all child processes automatically?

After thinking some more and re-reading the os.system doc, I am closing this. I think things are working exactly as documented, and in a way that will not be changed. The os.system doc initially says

1. "Execute the command (a string) in a subshell."

It does that.

2. "This is implemented by calling the Standard C function system(), and has the same limitations."

Read Standard C for more on that.

Nearly everything in os is a thin wrapper of operating system data and functions. Enhanced wrappers that add user-friendliness, go elsewhere.

3. "Changes to sys.stdin, etc. are not reflected in the environment of the executed command."

The sys module is Python-specific and has no effect on system functions. In particular, whether sys.stdout, initially copied from the process, is initally null or the terminal, Idle changing sys.stdout has no effect on os.system output, and it will not see it. This is, obviously, a known and expected limitation.

4. "If command generates any output, it will be sent to the interpreter standard output stream."

The interpreter process stdout is not sys.stdout. If it is null, bye-bye output.

The doc ends with

5. "The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function."

In other words, the limitations of os.system are known, and it is semi-deprecated, especially when it does not work as one would wish, and a replacement has been provided. The subprocess module is an example of an 'enhanced wrapper'.

6 "See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes."

These recipes are replacements for things that work. The .check_output method enables things that one could not very easily do before. I already gave examples of how to use it.

I disagree. I think you are confusing Python's sys.std* objects with the std* file descriptors of a process.  The important points here are that IDLE is effectively substituting its rpcproxy for the std* file descriptors of the IDLE shell process that it creates *and* that, in POSIX world, a forked subprocess is required to inherit copies of the file descriptors (in particular, sys.std*) of its parent (the IDLE shell subprocess).  So, if the IDLE shell subprocess displays the output written to the stdout and stderr in the Python Shell window and directs typed-in input from that window to the stdin file descriptor, it is reasonable to assume that the same thing should happen by default to processes launched within that IDLE shell subprocess, be they by os.system or by subprocess.Popen.  subprocess.Popen lets you substitute sys* file associations when creating the subprocess but that is irrelevant for the default case.  The default for subprocess.Popen is exactly the same as os.system: "With the default settings of None, no redirection will occur; the child’s file handles will be inherited from the parent."

I thought I was un-confusing sys.std* Python objects (such as idlelib.rpc.RPCProxy) from std* integer file descriptors, whereas you seem to say they are the same: "copies of the file descriptors (in particular, sys.std*)". The rest of your message seems to imply that subprocess.check_output should not work, but it does, at least on Windows. And subprocess *is* a recommended replacement for os.system.

Roger, do you have any thoughts on this issue?

I didn't mean to imply anything about subprocess.check_output().  It works as expected, that is by explicitly intercepting anything written to the stdout of the forked subprocess and returning it as the call result.  What doesn't work is if you try a default subprocess.call(), for example.  The results are the same as os.system and for the same reason.  In a Posix terminal session:

$ python3.3
>>> import subprocess
>>> subprocess.call('pwd')
/Users/nad
0
>>> 

But in an IDLE shell window:
>>> import subprocess
>>> subprocess.call('pwd')
0
>>> 

That's confusing to users.  IDLE should handle that.

It might be possible to fix this problem by creating pipes to redirect stdout and stderr to the text widget in PyShell. 

The ShellWindow.py demo in Demo/tkinter/guido may be helpful.

The same lack of output occurs from processes started via the multiprocessing module.

#22664 (closed as duplicate of this) has ppperry's multiprocessing test example.

Thank you Roger for your suggestion. Here is a patch which handless subprocess output (it is more complicated than Guido's demo because we need Unicode strings, not bytes).

It looks like the issue can be reduced to whether or not to show this
output:

  >>> import os
  >>> os.write(1, b'should we see this in idle?\n')
  should we see this in idle?
  28

assuming sys.__stdout__.fileno() == fileno(stdout) == 1.

If "should we see this in idle?" is shown in idle then the output that
writes to the inherited C stdout such as os.system('ls'), 
subprocess.call('ls') will also be shown automatically.

And in reverse If "should we see this in idle?" should not be shown 
then other C stdout output such as produced by os.system('ls') should 
not be redirected too. The output written to the same file should be
shown/not shown consistently.

  os.write(2, b'the same goes for stderr\n')

A quick smoke test of the patch (solely on OS X) looks like it works pretty well.  Not surprisingly, it doesn't fix anything when using the deprecated -n ("no subprocess") option but that's not a problem.  One more serious issue is that, depending on the length of the output produced, the IDLE shell prompt ('>>>') for the next command may appear out of sync, interspersed within the displayed stdout rather than at the end.  And, yes, the os.write(1, ...) to stdout in the same process is an equivalent way to look at the issue and the patch seems to handle that as well.

Nice work, Serhiy!

What do you think about this patch Terry?

Currently, when Idle is started from a command line or console interpreter, "import os; os.system('dir')" produces the listing in the console window, as expected.

Reading the patch, it *augments* one-channel socket communication, which properly* combines and serializes stdout and stderr messages, with two-channel pipe communication.  This is rather messy and leads to inter-mixed displays, as reported.  If sensibly possible, the process stdout/err (fileno 1&2) written to by the os functions in question should forward bytes to the idle process via the rpc channel, properly labelled as stdout or stderr.

* There is a bug where the second prompt of a shell session gets intermixed with previous output or clipboard content, but I do not have a repeatable test case yet. 

Issue #18823 is about *replacing* the socket with pipes (whether using subprocess or multi-processing or whatever.).  (There should still be just one channel, not two, from user process to Idle process to keep the rpc protocol in control of what the Idle process receives.)  About last August, Roger sent at least a few of us somewhat large proof of concept patch, which I have not reviewed yet.

The patch uses Tk.createfilehandler, which seems not to exist on Windows, so I cannot review further.

  File "F:\Python\dev\34\lib\tkinter\__init__.py", line 1932, in __getattr__
    return getattr(self.tk, attr)
AttributeError: 'tkapp' object has no attribute 'createfilehandler'
  File "F:\Python\dev\34\lib\tkinter\__init__.py", line 1932, in __getattr__
    return getattr(self.tk, attr)
AttributeError: 'tkapp' object has no attribute 'createfilehandler'  File "F:\Python\dev\34\lib\tkinter\__init__.py", line 1932, in __getattr__
    return getattr(self.tk, attr)
AttributeError: 'tkapp' object has no attribute 'createfilehandler'

Closed #13220 as duplicate of this.