classification
Title: Windows bug picking up stdin from a pipe
Type: behavior Stage: resolved
Components: Build, Windows Versions: Python 3.3, Python 2.7, Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, eryksun, oeffner, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2013-01-08 13:44 by oeffner, last changed 2018-09-03 22:15 by zach.ware. This issue is now closed.

Files
File name Uploaded Description Edit
testsubproc.py oeffner, 2013-01-08 13:44 python script demonstrating the problem
Messages (4)
msg179352 - (view) Author: Robert Oeffner (oeffner) Date: 2013-01-08 13:44
Hi,

This is a bug that seems to exist on python 2.7, python 3.3 on Windows versions XP, Vista, 7 and 8 and has been around for some years, presumably also in other python versions. It is only recently I have managed to better isolate it although not completely.

My Windows PC is set up with some doskey macros that are loaded at each
instance of the commandline interpreter cmd.exe with the registry key
HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun\mymacros.cmd
Unfortunately this seems to delete stdin when stdin is piped to python scripts that runs another executable invoked with subprocess. In this case the command:

python32 testsubproc.py < mytextfile.txt

will not work. The textfile mytextfile.txt will be ignored. If subprocess is invoked with shell=False the textfile does get piped into stdin. This is however not an option as our porgrams are distributed on other platforms. It would also give an inconsistent experience on the same OS (Windows).

If mymacros.cmd is called interactively from the command prompt the problem does not exist and stdin can be piped into the python script with no problem.

I have attached a working script below which includes a comment section with a small C++ program and the registry key in question to demonstrate the problem.

I would be grateful if there are people who know how to get to the bottom of this bug or even fix it.


Many thanks,


Robert
msg223203 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-16 13:58
@Robert we're sorry about the delay in getting back to you.  Sorry Windows gurus this is over my head :(
msg223341 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-07-17 15:42
Doskey is a command-line interface for a subset of the Win32 console API. Macros are implemented as console input aliases by calling AddConsoleAlias. 

http://msdn.microsoft.com/en-us/library/ms681935

You can load macros from a text file that defines aliases for multiple EXEs:

    doskey /macrofile=doskey.cfg

Create this file from your current macros: 

    doskey /macros:all > doskey.cfg

You can also use ctypes to add an alias. 

    #!python3
    import ctypes
    import subprocess
    
    AddConsoleAliasW = ctypes.windll.kernel32.AddConsoleAliasW
    AddConsoleAliasW.argtypes = [ctypes.c_wchar_p] * 3
    
    # $db => import builtins;dir(builtins)
    AddConsoleAliasW("$db",
                     "import builtins;dir(builtins)",
                     "python.exe")

    #subprocess.call("doskey")
    subprocess.call("stdintest.exe", shell=True)

The "bug" occurs if I uncomment the line to run doskey. 

When stdin is a file, calling sys.stdin.tell() before and after running doskey shows that it's reading standard input, for whatever reason. That's its prerogative, not a bug. Calling sys.stdin.seek(0) after running doskey works, but that's not a possibility when doskey is run via the cmd shell's AutoRun.

By the way, redirection to a file is not a pipe. It actually works for a pipe such as

    type mytxtfile.txt | testsubproc.py

The good news is that you only need to load doskey macros once for a given console window. You can use an AutoRun script that tests and sets an environment variable:

    @echo off
    if "%AUTORUN_DOSKEY%"=="No" goto end
    echo %0: setting doskey macros
    "%SystemRoot%\System32\doskey.exe" /macrofile="%AppData%\doskey.cfg"
    set AUTORUN_DOSKEY=No

    :end

This should take care of cases where you're redirecting standard input to a file, assuming the shell wasn't started with the /d option to skip AutoRun. 

On Posix systems /bin/sh -c doesn't execute .profile, .bashrc, etc. So maybe on Windows Popen should use %ComSpec% /d /c. You'd still have this problem with os.system, however.
msg324543 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2018-09-03 22:15
Eryk's diagnosis sounds like this is not a bug in Python or something that Python needs to or even should fix.
History
Date User Action Args
2018-09-03 22:15:29zach.waresetstatus: open -> closed
resolution: not a bug
messages: + msg324543

stage: resolved
2014-07-17 15:42:08eryksunsetnosy: + eryksun
messages: + msg223341
2014-07-16 13:58:13BreamoreBoysetnosy: + tim.golden, BreamoreBoy, zach.ware, steve.dower
messages: + msg223203
2013-01-08 13:44:24oeffnercreate