This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author terry.reedy
Recipients louielu, mdcowles, terry.reedy
Date 2017-05-15.06:31:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1494829916.06.0.832731681025.issue26143@psf.upfronthosting.co.za>
In-reply-to
Content
Thank you for making a start on code for this issue.  After thinking about this a few hours, here is what I currently think.

I don't want a tiny module for one function.  I would like a module with objects common to the idle and user processes, in particular things needed by both pyshell and run modules. This would include things now imported into pyshell from run. (These were initially imported in the other direction, but switching sped up run startup.)  I would like to refactor and include the near duplicate warning functions.  The imports of io, linecache, and maybe warnings in the new module would come after the new function is called in the module.  At least initially, I would not include anything dependent on another IDLE module. 

Possible names: runutil, runcom, runshell, ??? (and test_...).

Call the new function fix_path (nearly everything in idlelib is private already).  Add "def unfix_path(): if [path[0] != '': <insert ''>.  It is possible that we will need to return and pass back what, if anything, is removed.  I am not sure yet.

When one runs 'python <path>/file.py', sys.path[0] is <path>, not ''.  So one can start idlelib with, for instance, "python <path-to_idlelib>/idle.py" and sys.path[0] will be <idledir>. It slows imports slightly and could be removed (and never re-added). I need to check for other instances of sys.path[0] != '' and <idledir> additions.

Here is one. Add "print(sys.path, file=sys.__stdout__)" to run.py and run it and the result in Shell is
['C:\\Programs\\Python36\\lib\\idlelib', 'C:\\Programs\\Python36\\Lib\\idlelib', 'C:\\Programs\\Python36\\python36.zip', ...

If I start IDLE from the console, path (in console) starts with '' instead of idledir x 2.  It appears twice because of the tkinter input.  The original sys.path is the same in both processes. If I then run a file with 'print sys.path', user-process sys.path begins with filedir, consoledir. I am not quite how '' is replaced by consoledir, but the presence of consoledir is wrong (explained elsewhere).

If I start from the icon, output never appears. The result needs to be saved and displayed otherwise, such as in a tkinter messagebox. If I then run the same file, path (displayed in Shell) begins with just the file directory.

More experiments are needed, such as starting from Explorer [Edit with IDLE].  We will need some repeated on Linux and MAC

In run.py, would "unfix()" be needed after imports?  I think not when running a file, because pyshell.ModifiedInterpreter.prepend_syspath and MI.run_command together prepend usercodedir to sys.path.  I believe this is true when running a startup file.  So we only need to be concerned about restoring '' (or anything else deleted) sometime before running user code from the shell.  

Within run, all but one of the delayed non-main imports are either duplicates that can be removed or can be moved to the top, where they will surely run before sys.path is augmented.

The un-movable delayed pydoc import is in the handle method of run.MyHandler(RPCHandler(socketserver.BaseRequestHandler)) and is called in the base .__init__.  This must happen *before* run_command sends the rpc request to modify sys.path.  So it should be safe if we do not previously unfix sys.path.

My conclusion: We may sometimes need to restore something so imports in Shell work the same as in Python REPL. The last possibility is in run.main just before 'while 1:'. 

Importing in the IDLE process and pyshell are messy.  As a temporary partial fix, we could unfix() after the top imports in pyshell, or after the top imports in pyshell.main, or at the bottom of pyshell.main just before entering the loop.  But if there are any further delayed non-idlelib imports, the IDLE process remains vulnerable to shadow files.  But I don't yet know if this is possible.

As with run, do we need to explicitly unfix in pyshell?  In normal mode, 
no user code runs in the idle process. So unfix is not needed.  With -n set, I think unfix is still not needed for user files, as MI.run_command runs the same userdir prepend in self.locals, where user code is exec'ed.  Indeed, in order to protect all IDLE imports (in the IDLE process), sys.path should be left reduced *except while running user code in the IDLE process, in -n mode*.

The irreducible problem for -n is that once user code *is* run, the fact that IDLE simulates python -i and the fact that -n means no Shell restart, the directory cannot be removed.  If there is a conflict, we cannot help it.  The oddity of -n mode is that one can run additional files from other directories and make sys.path grow indefinitely.  Thus files can interfere with each other as well as with IDLE.   In python, one can run dir1/file1.py with -i and 'exec(open(dir2/file2.py).read)', and so on, but exec does not prepend dir2 to sys.path.

What about the pyshell.main section "# process sys.argv and sys.path"?  I think the sys.path parts are wrong, at least for protecting IDLE imports, and should be removed.

    for i in range(len(sys.path)):
        sys.path[i] = os.path.abspath(sys.path[i])

Everything is already absolute except for '' and changing the '' added by python to abspath('') == getcwd() seems wrong.  It makes the IDLE shell unnecessarily different from the Python REPL.  If I run
  path-a> python -i path-b/tem.py
I cannot import files in path-a.  The same should be true if I run
  path-a> python -m idlelib
and then edit and run path-b/tem.py.

    elif args:
        enable_edit = True
        pathx = []
        for filename in args:
            pathx.append(os.path.dirname(filename))
        for dir in pathx:
            dir = os.path.abspath(dir)
            if not dir in sys.path:
                sys.path.insert(0, dir)

These insertions should only happen in -n mode and will happen if and when run, not before being opened to be edited (which could even fail).

    else:
        dir = os.getcwd()
        if dir not in sys.path:
            sys.path.insert(0, dir)

With '' changed (wrongly) to getcwd, this will usually do nothing. If I run 'python', sys.path start with '', not os.getcwd.  If I os.chdir(newdir), then '' refers to newdir, not the startig dir.  However, in -n mode

Conclusion: delete the above and add
    if not use_subprocess (and other conditions?):
        sys.path.insert(0, '')
at the last possible moment, just before
History
Date User Action Args
2017-05-15 06:31:56terry.reedysetrecipients: + terry.reedy, mdcowles, louielu
2017-05-15 06:31:56terry.reedysetmessageid: <1494829916.06.0.832731681025.issue26143@psf.upfronthosting.co.za>
2017-05-15 06:31:56terry.reedylinkissue26143 messages
2017-05-15 06:31:53terry.reedycreate