This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eryksun
Recipients eryksun, ezio.melotti, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Date 2020-09-24.00:33:08
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1600907590.14.0.954461587522.issue41849@roundup.psfhosted.org>
In-reply-to
Content
io._WindowsConsoleIO reads from the console via ReadConsoleW in line-input mode, which limits the line length to the maximum of 256 and the size of the client buffer, including the trailing CRLF (or just CR if processed-input mode is disabled). Text that's typed or pasted beyond the length limit is ignored. The call returns when a carriage return ("\r") is read or the user types the enter key anywhere on the line.

Currently the buffer that _WindowsConsoleIO passes to ReadConsoleW is capped at 512 wide characters, based on the C runtime's BUFSIZ (512) macro. This is too small. Legacy mode (i.e. PYTHONLEGACYWINDOWSSTDIO) uses io.FileIO with an 8 KiB buffer, which is 8K characters when the console input codepage is a legacy single-byte encoding. _WindowsConsoleIO should support at least this much. 

I'd prefer that it allowed up to 32K characters, which is the upper limit for a process command line or for a long filepath. By way of comparison, input(), which calls _PyOS_WindowsConsoleReadline if stdin is a console file, is currently capped at 16K characters.

To be able to read up to 32K characters also requires increasing the BufferedReader default buffer size and TextIOWrapper chunk size to 96 KiB (BMP codes in the range 0x0800-0xFFFF encode as a 3-byte UTF-8 sequence) in order to ensure that f.readline() and f.buffer.readline() request the maximum size. This would require changing _io__WindowsConsoleIO___init___impl to set `self->blksize = 96 * 1024` when `console_type == 'r'`, as well as changing _io_open_impl to manually set the _CHUNK_SIZE of the TextIOWrapper to 96 KiB for console input (type 'r'). Maybe TextIOWrapper should just query the raw _blksize as the initial chunk size. That would remove the need to manually set _CHUNK_SIZE.
History
Date User Action Args
2020-09-24 00:33:10eryksunsetrecipients: + eryksun, paul.moore, vstinner, tim.golden, ezio.melotti, zach.ware, steve.dower
2020-09-24 00:33:10eryksunsetmessageid: <1600907590.14.0.954461587522.issue41849@roundup.psfhosted.org>
2020-09-24 00:33:09eryksunlinkissue41849 messages
2020-09-24 00:33:08eryksuncreate