classification
Title: can cmd.py's API/docs for the use of an alternate stdin be improved?
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: postponed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, Jack.Andrews, ajaksu2, draghuram, eric.araujo, hoffman, rickbking, ron_adam, terry.reedy
Priority: low Keywords: easy

Created on 2008-04-07 18:58 by rickbking, last changed 2015-05-17 22:49 by terry.reedy. This issue is now closed.

Messages (20)
msg65094 - (view) Author: Richard King (rickbking) Date: 2008-04-07 18:58
The module global value use_rawinput is initialized to 1 but not reset
when stdin is replaced with a passed-in value.
msg65107 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2008-04-07 19:48
I don't think it should stop using raw_input just because you changed
stdin, as you can change it to something that will work with raw_input.
Consider:
>>> import sys
>>> sys.stdin = open("/dev/tty")
>>> raw_input()
a
'a'

You can tie it to any object (e.g. a GUI input) that supports the file
protocol and keep using raw_input. Or change Cmd.use_rawinput to 0 to
use stdin.readline directly.

On a related issue. Cmd.use_rawinput should be "True", not 1...
msg65120 - (view) Author: Raghuram Devarakonda (draghuram) (Python triager) Date: 2008-04-07 20:54
The doc for "cmd" at
http://docs.python.org/dev/library/cmd.html#module-cmd says:

"Instances of Cmd subclasses have some public instance variables:
.
.
.
Cmd.use_rawinput¶
    A flag, defaulting to true. If true, cmdloop() uses raw_input() to
display a prompt and read the next command; if false, sys.stdout.write()
and sys.stdin.readline() are used. (This means that by importing
readline, on systems that support it, the interpreter will automatically
support Emacs-like line editing and command-history keystrokes.)"

So it is for the user to modify use_rawinput as required. This flag has
been introduced in #405952. BTW, this one and other similar variables
are at class level and are not instance variables. Isn't it?
msg65200 - (view) Author: Richard King (rickbking) Date: 2008-04-08 17:37
(this is really 2 mails because my home email address was not registered so they were rejected at first)
Right - I wasn't too clear. The module stashes stdin, whether from sys 
or passed in, in self.stdin. When it reads input it uses a flag 
"raw_input" to determine whether to use raw_input or 
self.stdin.readline(), but the flag is not reset when a different stdin 
is passed in, so raw_input is always true.

The flag should be True/False, and I didn't think of setting it directly 
to be honest because it never occurred to me that I should have to do that to get a cmd class that i just instantiated with a different input object to use the one it was created with. I think the flag should be eliminated and replaced 
with the test self.stdin == sys.stdin anyway.

I also entered a feature request to add a stack of stdin's which are 
stacked when you want to process lines in a file, and then pop off the 
stack automatically at end of file. This would make it easy to write a 
command-line tool, like i'm doing, so that any input object could enter commands that change to other input objects and then restore the previous input object....this would allow for nesting of command files. There would be special conditions for 
sys.stdin (sys.stdin can only be used if there are no items on the 
stack). This could all be done outside the module, but it's so easy when 
it's integrated right in there.

----
I think I understand better what you are getting at, but it makes more 
sense to me to be explicit in the code and not take advantage of the 
fact the raw_input always works off sys.stdin. Also, I see now that 
maybe the idea was to have raw_input be changeable so that you could 
switch back and forth between "stdin" (whatever that is), and some other 
input object - I'm having a hard time seeing the usefulness of that, 
though. Anyway, instantiating a cmd class with a non-stdin input object 
and then having to set raw_input to False to get it to use that input 
object seems wrong.
----
does this make sense?
-Rick King

Daniel Diniz wrote:
> Daniel Diniz <ajaksu2@users.sourceforge.net> added the comment:
>
> I don't think it should stop using raw_input just because you changed
> stdin, as you can change it to something that will work with raw_input.
> Consider:
>   
>>>> import sys
>>>> sys.stdin = open("/dev/tty")
>>>> raw_input()
>>>>         
> a
> 'a'
>
> You can tie it to any object (e.g. a GUI input) that supports the file
> protocol and keep using raw_input. Or change Cmd.use_rawinput to 0 to
> use stdin.readline directly.
>
> On a related issue. Cmd.use_rawinput should be "True", not 1...
>
> ----------
> nosy: +ajaksu2
>
> __________________________________
> Tracker <report@bugs.python.org>
> <http://bugs.python.org/issue2571>
> __________________________________
>
>
>
msg68364 - (view) Author: Raghuram Devarakonda (draghuram) (Python triager) Date: 2008-06-18 14:28
Richard, I see the following very clearly mentioned in the doc:

"If you want a given stdin to be used, make sure to set the instance’s
use_rawinput attribute to False, otherwise stdin will be ignored."

Even though this seems like unnecessary, at least it is documented. If
you want to push for automatically setting use_rawinput when 'stdin' is
not None, you will need to submit a patch and convince some core
developer to agree with you.
msg68390 - (view) Author: Richard King (rickbking) Date: 2008-06-19 01:28
There were some other things I wanted too so I just made my own cmd.py.
-Rick

Raghuram Devarakonda wrote:
> Raghuram Devarakonda <draghuram@gmail.com> added the comment:
>
> Richard, I see the following very clearly mentioned in the doc:
>
> "If you want a given stdin to be used, make sure to set the instance’s
> use_rawinput attribute to False, otherwise stdin will be ignored."
>
> Even though this seems like unnecessary, at least it is documented. If
> you want to push for automatically setting use_rawinput when 'stdin' is
> not None, you will need to submit a patch and convince some core
> developer to agree with you.
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue2571>
> _______________________________________
> ------------------------------------------------------------------------
>
>
> No virus found in this incoming message.
> Checked by AVG. 
> Version: 8.0.100 / Virus Database: 270.4.0/1506 - Release Date: 6/17/2008 4:30 PM
>
msg68410 - (view) Author: Raghuram Devarakonda (draghuram) (Python triager) Date: 2008-06-19 14:39
On Wed, Jun 18, 2008 at 9:28 PM, Richard King <report@bugs.python.org> wrote:
>
> Richard King <rickbking@comcast.net> added the comment:
>
> There were some other things I wanted too so I just made my own cmd.py.

Yes. Lot of people seem to use their own versions of cmd.py. Recently,
I also implemented another class on top of cmd.Cmd in order to have
more useful functionality. I have seen at least three implementations
('cmdln' on googlecode, 'CommandLoop' and 'cmd2' on pypi). I hope that
after some heavy use, I will be able to submit some patches to the
standard library module. There is already one in #1294.
msg86737 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2009-04-28 13:45
Changing into a RFE: "automatically set use_rawinput when 'stdin' is
not None". Will be closed unless someone voices interest.
msg104553 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-04-29 18:09
Too late for new 2.7 features.
msg118078 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-06 20:29
Can we reopen this as a feature request for 3.2?
msg118093 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-10-07 02:57
Since 'we' can reopen any closed issue, I will try to answer what I think you might be asking.

I closed this because of Daniel's suggestion coupled with the Richard disclaiming further interest and neither Raghuram nor any new responder saying anything more.

The issue title is a misstatement arising from the OP not noticing how to change the behavior. Anyone reopening this (or opening a new issue) would need to change the title to reflect what he thinks should be done. I will not do that because the naive change proposed in the thread seems unnecessary, while further investigation suggests that it may be wrong or insufficient.

The original post refers to use_rawinput as a 'module global value'. It is not that but a cmd.Cmd class data attribute that can be changed either for the class or an instance. It is one of 9 that might be so changed.

Looking at the patch with #405952 shows that Cmd.__init__ had no parameters and Cmd.cmdloop always read input with raw_input(), now renamed just input() (making the attribute name a bit obsolete). The replacement then was something like

                    if self.use_rawinput:
                        try:
                            line = input(self.prompt)
                        except EOFError:
                            line = 'EOF'
                    else:
                        sys.stdout.write(self.prompt) # note sys.,
                        sys.stdout.flush()            # not self. here
                        line = sys.stdin.readline()
                        if not len(line):
                            line = 'EOF'
                        else:
                            line = line[:-1] # chop \n

The reason for this patch, which *almost* replicates raw_input(), was that raw_input and/or readline swallowed an exception, whereas the replacement code does not. I wonder:
1. Does input() still do the same and should it?
2. Is there something about cmd processing that this is the only use of input() where this exception swallowing is a problem?
3. If so, why not always use the replacement? Speed?
4. I am sort of asking whether this was/is really the right hack.

Someone later added completekey as an initialization parameter (rather than as a date attribute) and the following code

        if self.use_rawinput and self.completekey:
            try:
                import readline
                self.old_completer = readline.get_completer()
                readline.set_completer(self.complete)
                readline.parse_and_bind(self.completekey+": complete")
            except ImportError:

I know nothing about readline and why it (apparently) requires the use of input(), but the point here is that setting use_rawinput to False disables completekey. This should be documented but I did not see such.

At the same or later time, someone added stdin and stdout parameters and change 'sys' to 'self' in the first snippet above. Given that these parameters appears useless when use_rawinput is True, why was use_rawinput not automatically set to false then? Blunder or subtle reason? Someone who proposes auto-resetting should try to find the name of the patch and/or commit author and ask.

It seems to me that all the process parameters should be handled uniformly. Make then all class attributes and let any be changed for instances as keyword attributes to __init__(). Given the conflict noted above, perhaps raise ValueError if someone makes contradictory changes.

So, Éric, if your question was academic, I would leave this closed.
If it was not, and you want to reopen, retitle, and assign it to yourself, go ahead.
msg119035 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-18 16:35
You could say my question was half-academic.  I read your closing message and thought “this feature request has been closed because of the version, not really rejected”, so I asked about reopening.  On a second level, it appears from your detailed message (thanks for writing it) that the situation is still unclear, so I do think that a review of docs and tests is needed, and maybe an API cleanup.  I’m assigning this to myself as an opportunity to learn the insides of cmd in some months.
msg133740 - (view) Author: Jack Andrews (Jack.Andrews) Date: 2011-04-14 14:10
hi guys,

this makes Cmd a bit more useful.

my use case is talking to pdb via pipe
(ie. subprocess module).  pdb doesn't 
behave very well if stdin is not a tty.


===================================================================
--- cmdpy.orig/cmd.py   2011-04-14 23:55:01.102867999 +1000
+++ cmdpy/cmd.py        2011-04-14 23:55:16.272868002 +1000
@@ -92,6 +92,8 @@
             self.stdin = stdin
         else:
             self.stdin = sys.stdin
+        if not stdin.isatty():
+            self.use_rawinput = 0
         if stdout is not None:
             self.stdout = stdout
msg133752 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-04-14 17:15
Jack, several questions.
Are you saying that when stdin is a pipe and not a tty, pdb works better with use_rawinput set False? 
Are you sure that the auto setting is correct in all use cases?
Are you aware that you can set use_rawinput 'manually'?
Have you read my previous response msg118093?
Are you willing to help sort out the questions and issues therein?
msg133778 - (view) Author: Jack Andrews (Jack.Andrews) Date: 2011-04-15 02:39
> Terry J. Reedy <tjreedy@udel.edu> added the comment:
>
> Jack, several questions.
> Are you saying that when stdin is a pipe and not a tty, pdb works better with use_rawinput set False?

yes.  well, at least i thought so.  today, pdb works fine
with no patch.  i must have stumbled across my bug
in the middle of patching cmd.py.

if i notice anything strange, i'll let you know.
msg221763 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-28 01:48
Does somebody want to propose a patch to take this forward, or can it be closed again, or what?
msg221777 - (view) Author: Jack Andrews (Jack.Andrews) Date: 2014-06-28 06:56
I'm no longer working on this, but IIRC, my patch is not necessary and
there is no deficiency.

Ta, Jack

On Saturday, June 28, 2014, Mark Lawrence <report@bugs.python.org> wrote:

>
> Mark Lawrence added the comment:
>
> Does somebody want to propose a patch to take this forward, or can it be
> closed again, or what?
>
> ----------
> nosy: +BreamoreBoy
> versions: +Python 3.5 -Python 3.3
>
> _______________________________________
> Python tracker <report@bugs.python.org <javascript:;>>
> <http://bugs.python.org/issue2571>
> _______________________________________
>
msg221786 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-28 13:21
Given "my patch is not necessary and there is no deficiency." from Jack Andrews in msg221777 please close as "not a bug".
msg221798 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-06-28 15:43
Since Eric assigned this to himself, I will give him a chance to answer. I removed the 'easy' tag because it is not clear to me what the remaining issue is.
msg221899 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-06-29 20:26
I won’t have the time to do the docs/tests inspection I wanted to do.
History
Date User Action Args
2018-06-03 22:45:42r.david.murraylinkissue33749 superseder
2015-05-17 22:49:14terry.reedysetstatus: open -> closed
resolution: remind -> postponed
stage: resolved
2014-06-29 20:26:09eric.araujosetassignee: eric.araujo ->
messages: + msg221899
2014-06-28 15:43:21terry.reedysetmessages: + msg221798
2014-06-28 13:21:02BreamoreBoysetmessages: + msg221786
2014-06-28 06:56:25Jack.Andrewssetmessages: + msg221777
2014-06-28 01:48:17BreamoreBoysetnosy: + BreamoreBoy

messages: + msg221763
versions: + Python 3.5, - Python 3.3
2011-04-15 02:39:54Jack.Andrewssetmessages: + msg133778
2011-04-14 17:15:54terry.reedysetmessages: + msg133752
2011-04-14 14:10:49Jack.Andrewssetnosy: + Jack.Andrews
messages: + msg133740
2011-02-18 02:26:01r.david.murraysetnosy: terry.reedy, hoffman, draghuram, rickbking, ajaksu2, ron_adam, eric.araujo
versions: + Python 3.3, - Python 3.2
2011-02-18 02:25:52r.david.murraysetnosy: terry.reedy, hoffman, draghuram, rickbking, ajaksu2, ron_adam, eric.araujo
title: cmd.py always uses raw_input, even when another stdin is specified -> can cmd.py's API/docs for the use of an alternate stdin be improved?
2011-02-17 22:35:07ron_adamsetnosy: + ron_adam
2010-11-19 17:02:45eric.araujolinkissue10396 superseder
2010-11-19 15:44:37eric.araujounlinkissue10396 superseder
2010-11-15 01:38:16eric.araujosetstatus: closed -> open
type: enhancement
versions: + Python 3.2, - Python 2.7
2010-11-13 18:09:47hoffmansetnosy: + hoffman
2010-11-13 04:22:15r.david.murraylinkissue10396 superseder
2010-10-18 16:35:08eric.araujosetresolution: out of date -> remind
messages: + msg119035

assignee: eric.araujo
type: enhancement -> (no value)
stage: test needed -> (no value)
2010-10-07 02:57:10terry.reedysetmessages: + msg118093
2010-10-06 20:29:35eric.araujosetnosy: + eric.araujo
messages: + msg118078
2010-04-29 18:09:04terry.reedysetstatus: pending -> closed

nosy: + terry.reedy
messages: + msg104553

resolution: out of date
2009-04-28 13:45:51ajaksu2setstatus: open -> pending
priority: low
type: behavior -> enhancement

components: + Library (Lib), - Extension Modules
versions: + Python 2.7, - Python 2.4
keywords: + easy
nosy: draghuram, rickbking, ajaksu2
messages: + msg86737
stage: test needed
2008-06-19 14:39:54draghuramsetmessages: + msg68410
2008-06-19 01:28:13rickbkingsetmessages: + msg68390
2008-06-18 14:28:39draghuramsetmessages: + msg68364
2008-04-12 18:29:32georg.brandllinkissue1156280 superseder
2008-04-08 17:37:45rickbkingsetmessages: + msg65200
2008-04-07 20:54:39draghuramsetnosy: + draghuram
messages: + msg65120
2008-04-07 19:48:58ajaksu2setnosy: + ajaksu2
messages: + msg65107
2008-04-07 18:58:21rickbkingcreate