This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: shutil.get_terminal_size() throws ValueError if stdout is detached, no fallback
Type: behavior Stage: resolved
Components: IO Versions: Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Fix shutil.get_terminal_size() to catch AttributeError
View: 26801
Assigned To: Nosy List: eryksun, iritkatriel, r.david.murray, rsc1975
Priority: normal Keywords:

Created on 2015-08-30 22:59 by rsc1975, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (13)
msg249372 - (view) Author: Roberto Sánchez (rsc1975) Date: 2015-08-30 22:59
After the stdout stream has been overwritten and detached, the method shutils.get_terminal_size throws a ValueError exception and no fallback value is returned.

Code to reproduce it:

    >>> import os, sys, codecs, shutils
    >>> sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach())
    >>> os.get_terminal_size()
    os.terminal_size(columns=160, lines=37)
    >>> shutil.get_terminal_size((200, 30))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module> 
      File "/usr/lib64/python3.4/shutil.py", line 1057, in get_terminal_size
        size = os.get_terminal_size(sys.__stdout__.fileno())
    ValueError: underlying buffer has been detached

Is this the expected behavior ? IMHO, the given fallback values should be returned.

The official doc: "If the terminal size cannot be successfully queried, either because the system doesn’t support querying, or because we are not connected to a terminal, the value given in fallback parameter is used. fallback defaults to (80, 24) which is the default size used by many terminal emulators"


BTW, the function os.get_terminal_size() returns the correct size when it's invoked without parameters, maybe It could be a good fallback if sys.__stdout__ is not available and there isn't any user fallback values.
msg249374 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-08-31 00:03
See also issue 24920.
msg249375 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-08-31 00:05
In this case it might be better to leave the error.  The buffer being detached would seem to be a programming error.  What would be the use case for using get_terminal_size in a program that detaches the standard stream buffer?
msg249387 - (view) Author: Roberto Sánchez (rsc1975) Date: 2015-08-31 07:35
In my case, I modify the stdout but the error was thrown by a third party (click lib), I need to overwrite the stdout because the program has a CLI and the locale env in the terminal seems to be wrong:
 codecs.lookup(locale.getpreferredencoding()).name == 'ascii'
 sys.getdefaultencoding() == 'utf-8'

For these cases I try to fix it forcing the the IO encoding to the value given by sys.getdefaultencoding(), otherwise the script in a CLI won't work properly. 

I can apply a monkey-patch on the third party method that return "os.get_terminal_size()" if a ValueError exception is thrown, actually, The script is working now with the monkey-patch, but I thought that a bit of more "magic" in get_terminal_size would help to deal with the encoding issues in python3 CLI programs.
msg249391 - (view) Author: Roberto Sánchez (rsc1975) Date: 2015-08-31 08:44
I've added a new issue: http://bugs.python.org/issue24968 that is related with this one, because is the root cause to modify the stdin/stdout streams.
msg249405 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-08-31 15:40
So, the problem is needing to compensate for a broken (according to posix) locale setup.  I don't think quieting get_terminal_size is necessarily the answer here.  See issue 19846 for more than you want to know about the ASCII locale issue.

See also issue 15216, which is probably a solution to your problem, but is an open issue.
msg249411 - (view) Author: Roberto Sánchez (rsc1975) Date: 2015-08-31 16:27
Thank you for the references, changing the encoding of an open stream sounds interesting, but It's targeted for v. 3.6 so It seems a bit far away.

Anyway, as I said I fixed on my side with monkey-patching on the third-party lib, the broken locale is just the context, just to explain how I got this issue, I opened this because I honestly though that could be better to return the fallback value that raise an exception, but you know better the "big picture" so, if It's better as is, It's ok for me.

BTW, I don't find the opened issue to use utf8 as default instead ascii (except for Windows)
msg249423 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-08-31 18:59
Yeah, I think I was thinking of issue 19846, which was fixed by using surrogateescape on the standard streams instead of "guessing" utf-8.

I'm not utterly opposed to turning the ValueError into a default return value, but as you can see, the fact that the ValueError was returned pointed toward the deeper problem (needing to have a way to change the encoding of an already-existing TextIO stream).
msg249435 - (view) Author: Roberto Sánchez (rsc1975) Date: 2015-08-31 22:34
I totally agree, on the other hand, if the detach() method can cause that some core methods like shutil.get_termina_size() raise an exception then I think that should be warned in the doc of detach().

In all places that I've seen, the detach() method is used to overwrite the stdout in py3, so a sort of advice about the using of detach() It'd save some problems.
msg249443 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-09-01 01:42
"After the raw stream is detached, the buffer is in an unusable state" pretty much says everything that needs to be said, doesn't it?  It certainly seems like anything more does not belong in the detach docs.  A mention could possibly be added to the sys.stdin/etc docs, though.
msg249462 - (view) Author: Roberto Sánchez (rsc1975) Date: 2015-09-01 07:18
Yeah, sorry, actually I was thinking in the stdin/out section, the detach() doc itself is Ok, the problem comes when is called in stdin/out.
msg407554 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-02 23:05
I am unable to reproduce this, was it fixed?

>>> import os, sys, codecs, shutil
>>> sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach())
>>> os.get_terminal_size()
os.terminal_size(columns=79, lines=36)
>>> shutil.get_terminal_size((200, 30))
os.terminal_size(columns=200, lines=30)
>>>
msg407556 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-12-02 23:22
This was fixed in 3.5.
History
Date User Action Args
2022-04-11 14:58:20adminsetgithub: 69154
2021-12-02 23:22:51eryksunsetstatus: open -> closed

superseder: Fix shutil.get_terminal_size() to catch AttributeError

nosy: + eryksun
messages: + msg407556
resolution: duplicate
stage: resolved
2021-12-02 23:05:57iritkatrielsetnosy: + iritkatriel
messages: + msg407554
2015-09-01 07:18:24rsc1975setmessages: + msg249462
2015-09-01 01:42:35r.david.murraysetmessages: + msg249443
2015-08-31 22:34:33rsc1975setmessages: + msg249435
2015-08-31 18:59:44r.david.murraysetmessages: + msg249423
2015-08-31 16:27:07rsc1975setmessages: + msg249411
2015-08-31 15:40:53r.david.murraysetmessages: + msg249405
2015-08-31 08:44:05rsc1975setmessages: + msg249391
2015-08-31 07:35:08rsc1975setmessages: + msg249387
2015-08-31 00:05:13r.david.murraysetmessages: + msg249375
2015-08-31 00:03:38r.david.murraysetnosy: + r.david.murray
messages: + msg249374
2015-08-30 22:59:38rsc1975settitle: shutil.get_terminal_size() throws ValueError is stdout is detached, no fallback -> shutil.get_terminal_size() throws ValueError if stdout is detached, no fallback
2015-08-30 22:59:13rsc1975create