Issue11990
Created on 2011-05-03 21:27 by Jimbofbx, last changed 2012-01-09 07:12 by astrobuf.
| Messages (9) | |||
|---|---|---|---|
| msg135076 - (view) | Author: James Hutchison (Jimbofbx) | Date: 2011-05-03 21:27 | |
In windows, 64-bit, python *mostly* writes only a \n to stdout even though it's mode is 'w'. However it sometimes writes a \r\n on certain print statements and erratically when I have multiple processes writing to stdout.
Output looks fine in console, in IDLE, and using v.3.1
Example with multiple processes writing to stdout out using the same code: print("TESTCODE"); (note that in windows, the naked \n is ignored):
TESTCODETESTCODE
TESTCODE
TESTCODE
TESTCODETESTCODETESTCODE
TESTCODE
Windows program that calls python and receives its piped output is a C# .NET program.
|
|||
| msg135078 - (view) | Author: James Hutchison (Jimbofbx) | Date: 2011-05-03 21:28 | |
Sorry there isn't more info but I'm really busy right now In fact a workaround would be appreciated if known. |
|||
| msg135081 - (view) | Author: James Hutchison (Jimbofbx) | Date: 2011-05-03 21:52 | |
Nevermind, I have a workaround that didn't require rewriting all the print statements but its in the C# code not the python code |
|||
| msg135104 - (view) | Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * ![]() |
Date: 2011-05-04 09:29 | |
Do you have a test scipt for us to reproduce the issues? |
|||
| msg135141 - (view) | Author: James Hutchison (Jimbofbx) | Date: 2011-05-04 16:18 | |
Yes and no, I can give you a single process single child example that just shows that python 3.2 uses binary output while python 3.1 used system default when piping, but trying to reproduce the multiprocessing output inconsistencies would be... difficult. Unfortunately the software I used to spot the \n, \r\n inconsistency with is proprietary. After creating a sample case in only python I couldn't reproduce the inconsistent \r\n issue in python 3.2 so I cannot say for certain that it wasn't caused by my C# program. I wouldn't worry about the inconsistent endlines for now.
Note that I don't see in the "what's new" documentation it mentioning that 3.2 changed the behavior of piped output. Kind of a big deal.
Sample code to compare 3.1 and 3.2 piped output:
import sys;
import os;
import subprocess;
from time import sleep;
python31loc = r"C:\python31\python.exe";
python32loc = r"C:\python32\python.exe";
myname = "IOPipetest.py";
def main(args):
if(len(args) == 1):
# main code
print("Testing python 3.1 endlines.", end='\r\n');
output = subprocess.check_output("%s %s -output" % (python31loc, myname));
print(output);
print("Testing python 3.2 endlines.", end='\r\n');
output = subprocess.check_output("%s %s -output" % (python32loc, myname));
print(output);
sleep(7);
else:
for i in range(4):
print("TESTING DEFAULT"); # endline is supposed to be automatic
print("TESTING SLASH-EN\n", end='');
print("TESTING WINDOW-RETURN\r\n", end='');
if __name__ == "__main__":
main(sys.argv);
--------------------------------------
sample output:
Testing python 3.1 endlines.
b'TESTING DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\nTESTING DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\nTESTING DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\nTESTING DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\n'
Testing python 3.2 endlines.
b'TESTING DEFAULT\nTESTING SLASH-EN\nTESTING WINDOW-RETURN\r\nTESTING DEFAULT\nTESTING SLASH-EN\nTESTING WINDOW-RETURN\r\nTESTING DEFAULT\nTESTING SLASH-EN\nTESTING WINDOW-RETURN\r\nTESTING DEFAULT\nTESTING SLASH-EN\nTESTING WINDOW-RETURN\r\n'
|
|||
| msg136118 - (view) | Author: James Hutchison (Jimbofbx) | Date: 2011-05-16 18:48 | |
I would like to add in windows, "input" now adds a \r at the end which wasn't in 3.1. It doesn't do it in idle. This is using just the regular console window that opens up when you double click. I'm guessing this is related to the issue I saw earlier:
code:
from time import sleep
item = input("Input something\n");
print("%s %i" % (item, len(item)));
sleep(15);
in Idle:
Input something
something
something 9
in console window:
Input something
something
10ething
ouch
|
|||
| msg136119 - (view) | Author: Brian Curtin (brian.curtin) * ![]() |
Date: 2011-05-16 18:49 | |
That's already fixed, it'll be in 3.2.1 |
|||
| msg143076 - (view) | Author: Vinay Sajip (vinay.sajip) * ![]() |
Date: 2011-08-27 15:19 | |
So is this now just a documentation issue, about the changed behaviour of pipes in 3.2? |
|||
| msg150921 - (view) | Author: Peter Csapo (astrobuf) | Date: 2012-01-09 07:12 | |
I have having the same issues as Jimbofbx. This seems to stem from changes due to issue 10841. All stdio is now opened in binary mode, in consideration that it is the TextIOWrapper's job to do endline translation. The problem here is that the newline mode = '\n' for the TextIOWrapper created for stdout. ( see pythonrun.c: create_stdio() ). For windows, the newline mode for stdin is already set to null enabling universal newline translation on input, and it should be set to null for newline transation on output as well. OLD CODE newline = "\n"; #ifdef MS_WINDOWS if (!write_mode) { /* translate \r\n to \n for sys.stdin on Windows */ newline = NULL; } #endif FIXED??? CODE newline = "\n"; #ifdef MS_WINDOWS /* translate \r\n to \n for sys.stdin on Windows */ /* translate \n to \r\n for sys.stdout on Windows */ newline = NULL; #endif |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2012-01-09 07:12:53 | astrobuf | set | nosy:
+ astrobuf messages: + msg150921 |
| 2011-11-15 19:37:26 | ezio.melotti | set | nosy:
+ haypo |
| 2011-08-27 15:19:43 | vinay.sajip | set | nosy:
+ vinay.sajip messages: + msg143076 |
| 2011-05-16 18:49:29 | brian.curtin | set | nosy:
+ brian.curtin messages: + msg136119 |
| 2011-05-16 18:48:56 | Jimbofbx | set | messages: + msg136118 |
| 2011-05-04 16:18:35 | Jimbofbx | set | messages: + msg135141 |
| 2011-05-04 09:29:06 | amaury.forgeotdarc | set | nosy:
+ amaury.forgeotdarc messages: + msg135104 |
| 2011-05-03 21:52:09 | Jimbofbx | set | messages: + msg135081 |
| 2011-05-03 21:28:02 | Jimbofbx | set | messages: + msg135078 |
| 2011-05-03 21:27:09 | Jimbofbx | create | |
