classification
Title: subprocess should open stdin in mode w+b on windows
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, Jason.Gross, cheryl.sabella, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2013-05-19 20:29 by Jason.Gross, last changed 2019-03-19 22:59 by cheryl.sabella. This issue is now closed.

Messages (4)
msg189624 - (view) Author: Jason Gross (Jason.Gross) Date: 2013-05-19 20:29
Files opened on Windows in mode 'wb' have a limit on the number of characters that can be written to them (at once?); attempting to write too many bytes gives the confusing error “IOError: [Errno 22] Invalid argument”.  See http://support.microsoft.com/default.aspx?scid=kb;en-us;899149 and http://stackoverflow.com/questions/4226941/python-ioerror-errno-22-invalid-argument-when-using-cpickle-to-write-large.  I have gotten this error when using subprocess.communicate(input=<large string>).  Please fix the subprocess library (and any other library that uses mode 'wb') to use mode 'w+b', at least on Windows.
msg222892 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-12 22:45
@Jason I'm very sorry about the delay in getting back to you.

Can our Windows gurus shed any light on this one?
msg223117 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2014-07-15 16:15
With 2.7 and 3.4 (same for 32- and 64-bit):

>>> f = open('test.bin', 'wb')
>>> f.write(b' ' * (1024*1024*100))
104857600
>>> f.close()
>>> import os
>>> os.stat('test.bin').st_size
104857600

The linked KB only applies to VS 2003 and VS 2005 (VC7 and VC8), so I'm not entirely surprised that this doesn't repro with VC9 or VC10.

For the subprocess case (on 3.4):

>>> import sys, subprocess
>>> p=subprocess.Popen([sys.executable, '-c', 'print(len(input()))'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> p.communicate(b' ' * (1024*1024*100))
(b'104857600\r\n', b'')

I'm inclined to close this as no repro unless we get an actual repro.
msg338414 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2019-03-19 22:59
Since Steve was inclined to close this in 2014 and there haven't been any additional comments since then, I am going to close this now.
History
Date User Action Args
2019-03-19 22:59:02cheryl.sabellasetstatus: open -> closed

nosy: + cheryl.sabella
messages: + msg338414

resolution: works for me
stage: resolved
2014-07-15 16:15:14steve.dowersetmessages: + msg223117
2014-07-12 22:45:24BreamoreBoysetversions: + Python 3.4, Python 3.5
nosy: + BreamoreBoy, tim.golden, zach.ware, steve.dower

messages: + msg222892

type: behavior
2013-05-19 20:29:13Jason.Grosscreate