This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Odd crashes/freezes when sys.stdout.shell.console is typed
Type: crash Stage: needs patch
Components: IDLE Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: Harrison Chudleigh, iritkatriel, terry.reedy
Priority: normal Keywords:

Created on 2018-09-17 02:53 by Harrison Chudleigh, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
fail.py Harrison Chudleigh, 2018-09-17 02:53 After running this file, IDLE hangs or crashes, depending on version.
Messages (5)
msg325509 - (view) Author: Harrison Chudleigh (Harrison Chudleigh) Date: 2018-09-17 02:53
If sys is imported and sys.stdout.shell.console is typed, IDLE does not return to the prompt. Ctrl-C has no effect and after a restart with Ctrl-F6, IDLE does not react after a command is typed in. This also occurs when other variables such as sys.stdout.shell.stdout are used, but not with non-existent variables such as sys.stdout.shell.not_a_variable. No such issue occurs using the command line; however, sys.stdout.shell is not defined in the command line. This behaviour exists in at least Python 3.6.1 and 3.7.0 on Mac OS X 10.9.

This might be related to a similar issue that occurs in at least Python 3.2 on Windows 2000. Entering sys.stdout.shell.console causes IDLE to crash immediately; a crash also occurs when you type sys.stdout.shell.console. and wait for IDLE to bring up the list of attributes.

I know that this bug shouldn't be encountered unless you're using the IDLE PyShell object for some reason, but it still seems weird that getting the value of a variable causes crashes and hangs, especially when I know that the variable exists from looking at idlelib.

This is marked as Python 3.4 to 3.7 because I tested with 3.7, 3.6 and 3.2 and I don't think that this would have been fixed and then broken again. I don't know about 3.8, though.
msg325510 - (view) Author: Harrison Chudleigh (Harrison Chudleigh) Date: 2018-09-17 02:57
Edit: I checked Python 3.4 (Windows XP SP3). It hangs in the same way as Python 3.6/3.7.
msg325569 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-09-17 19:38
3.4 and 3.5 only get security fixes and I doubt this qualifies.

I reproduced the described behavior with 3.7 on Win 10, so it is not Mac-specific.

Python sys.stdout: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
IDLE's sys.stdout: <idlelib.run.PseudoOutputFile object at 0x00000246821B1E10>
The PseudoFiles are unusual in being internal objects accessible from user code.

Comparing attributes with public names (not starting with single or double underscore) PseudoOutputFile lacks buffer, line_buffering, mode, reconfigure, and write_through.  #21995 is about fixing or documenting each omission for all 3 streams.

PseudoFiles add shell and tags.  Both should have private names that warn users that their behavior is undocumented and usage is at one's own risk.  'shell' should be double underscored. This minimal fix would be sufficient to close this issue.  Some manual testing should be sufficient for such a change.

'Tags' should be singular, as it is one of 'stdin', 'stdout', or 'stderr'.  The tag is used in the write method to tell Shell whether to display with the configured stdout or stderr colors.

Shell is an idlelib.rpc.RPCProxy object.  As an object in itself, in the run process, it has 2 public attributes: oid ('console') and sockio (an  idlelib.run.MyHandler object).  But as a proxy representing the Shell console object in the IDLE process, it supposedly 'has' all the attributes of the latter.  These are accessed with a custom __getattr__.

    def __getattr__(self, name):
        if self.__methods is None:
            self.__getmethods()
        if self.__methods.get(name):
            return MethodProxy(self.sockio, self.oid, name)
        if self.__attributes is None:
            self.__getattributes()
        if name in self.__attributes:
            value = self.sockio.remotecall(self.oid, '__getattribute__',
                                           (name,), {})
            return value
        else:
            raise AttributeError(name)

Note that self.__attributes is mangled to shell._RPCProxy__attributes when accessed externally.  (Also, it is a set still implemented in this ancient code as a dict with int 1 values.)

More importantly, remote calls to the Shell console can only pass and return objects that can be pickled, and there is currently no provision for graceful failure.  Hence sys.stdout.shell.width is (for me, currently) 80, but .console or .stdout are impossible to return.

I am extremely dubious about trying to fix this in the IDLE side of the link.  Instead, the proxy should be initialized with sets of known usable attributes and methods.  I think the underlying problem is trying to be unnecessarily and impossibly generic.  Run only calls the .readline, .write, and .close methods of the console proxy.  It only calls stack viewer setup on the interp proxy.  A third proxy, for flist, is passed to the stack viewer (I have not yet checked what is needed for that).  Restricting proxies to things that are needed and *should* work would make it more feasible to test them.  But setting up automated tests would still require some work.
msg404215 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-10-18 20:10
Terry, I don't see any PseudoOutputFile in the current codebase. Is this out of date?
msg404224 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-10-18 21:38
Unfortunately, no.  The standard stream replacements were renamed to StdOutputFile, etc, and slightly revised, but the buggy behavior remains.  The worst is that shell restart does not completely restart the shell.  It has to be closed instead.

I would like to fix this first by adding 'provision for graceful failure'.  'try' is now nearly free when there is no exception.
History
Date User Action Args
2022-04-11 14:59:06adminsetgithub: 78889
2021-10-18 21:41:12iritkatrielsetversions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.7, Python 3.8
2021-10-18 21:38:39terry.reedysetmessages: + msg404224
2021-10-18 20:10:47iritkatrielsetnosy: + iritkatriel
messages: + msg404215
2018-12-11 22:37:16terry.reedysetversions: - Python 3.6
2018-09-17 19:38:09terry.reedysetstage: needs patch
messages: + msg325569
versions: + Python 3.8, - Python 3.4, Python 3.5
2018-09-17 02:59:15Harrison Chudleighsettype: crash
2018-09-17 02:57:19Harrison Chudleighsetmessages: + msg325510
2018-09-17 02:53:01Harrison Chudleighcreate