Title: sys.stdout problems with pythonw.exe
Type: Stage:
Components: Interpreter Core Versions:
Status: closed Resolution: duplicate
Dependencies: Superseder: print in pythonw raises silent exception when no console available
View: 706263
Assigned To: Nosy List: giampaolo.rodola, manlioperillo, pfremy, techtonik, tim.peters
Priority: normal Keywords:

Created on 2004-06-15 20:34 by manlioperillo, last changed 2008-11-17 16:23 by amaury.forgeotdarc. This issue is now closed.

Messages (8)
msg60503 - (view) Author: Manlio Perillo (manlioperillo) Date: 2004-06-15 20:34
>>> sys.version
'2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit
>>> sys.platform
>>> sys.getwindowsversion()
(5, 1, 2600, 2, '')

I have written this script for reproducing the bug:

import sys

class teeIO:
    def __init__(self, *files):
        self.__files = files

    def write(self, str):
        for i in self.__files:
            print >> trace, 'writing on %s: %s' % (i, str)

        print >> trace, '-' * 70

def tee(*files):
    return teeIO(*files)

log = file('log.txt', 'w')
err = file('err.txt', 'w')
trace = file('trace.txt', 'w')

sys.stdout = tee(log, sys.__stdout__)
sys.stderr = tee(err, sys.__stderr__)

def write(n, width):
    sys.stdout.write('x' * width)
    if n == 1: return

    write(n - 1, width)
    write(1, 4096)

[output from err.log]
Traceback (most recent call last):
  File "", line 36, in ?
    write(1, 4096)
  File "", line 28, in write
    sys.stdout.write('x' * width)
  File "", line 10, in write
IOError: [Errno 9] Bad file descriptor

TeeIO is needed for actually read the program output,
but I don't know if the problem is due to teeIO.

The same problem is present for stderr, as can be seen
by swapping sys.__stdout__ and sys.__stderr__.

As I can see, 4096 is the buffer size for sys.stdout/err.
The problem is the same if the data is written in
chunks, ad example: write(2, 4096/2).

The bug isn't present if I use python.exe or if I write
less than 4096 bytes.

Thanks and regards   Manlio Perillo
msg60504 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-06-15 23:09
Logged In: YES 

Ya, this is well known, although it may not be documented.  
pythonw's purpose in life is *not* to create (or inherit) a 
console window (a "DOS box").  Therefore stdin, stdout, and 
stderr aren't attached to anything usable.  Microsoft's C 
runtime seems to attach them to buffers that aren't 
connected to anything, so they complain if you ever exceed 
the buffer size.

The short course is that stdin, stdout and stderr are useless 
in programs without a console window, so you shouldn't use 
them.  Or you should you install your own file-like objects, 
and make them do something useful to you.

I think it would be helpful if pythonw did something fancier 
(e.g., pop up a window containing attempted output), but 
that's in new-feature terrority, and nobody has contributed 
code for it anyway.
msg60505 - (view) Author: Manlio Perillo (manlioperillo) Date: 2004-06-16 17:05
Logged In: YES 

The problem with this bug is that I have a script that can
be executed both with python.exe that with pythonw.exe!
How can I know if stdout is connected to a console?

I think a 'patch' would be to replace sys.stdout/err with a
null stream instead of using windows stdout/err implementation.
If fileno can't be implemented, it should not be a problem.
msg60506 - (view) Author: Manlio Perillo (manlioperillo) Date: 2004-06-18 16:42
Logged In: YES 

I have found a very simple patch.

First I have implemented this function:

import os

def isrealfile(file):
    Test if file is on the os filesystem
    if not hasattr(file, 'fileno'): return False

    try: tmp = os.dup(file.fileno())
    except: return False
    else: os.close(tmp); return True

Microsoft implementation of stdout/err/in when no console is
created (and when no pipes are used) actually are not 'real'

Then I have added the following code in

import sys

class NullStream:
    A file like class that writes nothing
    def close(self): pass
    def flush(self): pass
    def write(self, str): pass
    def writelines(self, sequence): pass

if not isrealfile(sys.__stdout__): 
   sys.stdout = NullStream()

if not isrealfile(sys.__stderr__):
   sys.stderr = NullStream()

I have tested the code only on Windows XP Pro.

isrealfile could be added in os module.

Regards   Manlio Perillo
msg60507 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-06-20 03:13
Logged In: YES 

Just noting that "the usual" way to determine whether you're 
running under pythonw is to see whether


The usual way to get a do-nothing file object on Windows is 
to open the special (to Windows) file named "nul" (that's akin 
to opening the special file /dev/null on Unixish boxes).

Note that file('nul').fileno() does return a handle on Windows, 
despite that it's not a file in the filesystem.
msg60508 - (view) Author: Manlio Perillo (manlioperillo) Date: 2004-06-20 13:39
Logged In: YES 

Thanks for sys.executable and 'nul' hints!

I only want to add two notes:

1) isrealfile(file('nul')) -> True

So 'nul' has a 'real' implementation

2) sys.executables isn't very useful for me, since I can do:
    pythonw > afile

In this case sys.stdout is a 'real file', so I don't want to
redirect it to a null device.

In all cases, isrealfile work as I want.
msg60509 - (view) Author: Bluebird (pfremy) Date: 2004-12-23 15:19
Logged In: YES 

Manlio, thanks a lot for the tip. I ran into the same
problem (a program that can both be used with python.exe and
pythonw.exe). I will apply your fix.

I think that the fix should be applied somehow to
pythonw.exe, so that it does something more understandable
to the user.
msg75959 - (view) Author: anatoly techtonik (techtonik) Date: 2008-11-17 13:12
Duplicate of #706263
Date User Action Args
2008-11-17 16:23:27amaury.forgeotdarcsetstatus: open -> closed
resolution: duplicate
superseder: print in pythonw raises silent exception when no console available
2008-11-17 13:32:06giampaolo.rodolasetnosy: + giampaolo.rodola
2008-11-17 13:12:27techtoniksetnosy: + techtonik
messages: + msg75959
2004-06-15 20:34:40manlioperillocreate