This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Idle: use pipes instead of sockets to talk with user subprocess
Type: behavior Stage: needs patch
Components: IDLE Versions: Python 3.10
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, ezio.melotti, serhiy.storchaka, terry.reedy, tshepang
Priority: normal Keywords:

Created on 2013-08-23 23:35 by terry.reedy, last changed 2022-04-11 14:57 by admin.

File name Uploaded Description Edit terry.reedy, 2014-03-28 18:30
Messages (6)
msg196055 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-23 23:35
Idle once used os.spawnv to open the user subprocess and 2.7 still does. Sockets were then used to interact with the subprocess. Since rev71746, 2011-8-3, 3.2+ use subprocess.Popen but still use sockets. A rare but continual problem is failure to make the socket connection.

In #16123, Amaury Forgeot d'Arc suggested "pipes?" The idea resurfaced again in current Idle-sig thread "Idle does not open on mac" (because socket connection timed out). Guido, who designed the current 2.x system, chimed in with "I would recommend trying to use the subprocess module."

A current python-list thread indicates that there can be problems with
"Running a command line program and reading the result as it runs"

In particular, "for line in p:" gets lines delayed (buffered) while
Peter Otten discovered that "for line in iter(p.stdout.readline, ''):" gets them as they are produced.

We first need to experiment running a simple echo server with python(w).exe.  I have not succeeded yet on Windows.
msg215062 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-28 18:30
In the thread that started here
in this message.
Josiah Carlson posted a pastebin link about how to actually use the pipes. Guido recommended that the he post a revised version to a new issue, as a patch to subprocess, but I am uploading it here to make sure it does not get lost.
msg215069 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-28 19:08
from Victor Stinner gives example code using asyncio
msg215123 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-29 08:44
Pydev, Re: Status of PEP 3145 - Asynchronous I/O for subprocess.popen; Antoine Pitrou:
> Why don't you use multiprocessing or concurrent.futures? They have
> everything you need for continuous conversation between processes.
msg336154 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-02-20 23:54
#34313 reports problems with multiprocessing when using at least some versions of tkinter on some versions of macOS.
msg370832 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-06-06 16:26
Problems with multiprocessing and tkinter on Mac might have been due to using the default start method, which used to be fork and is now spawn.  This should be retested sometime.
Date User Action Args
2022-04-11 14:57:50adminsetgithub: 63023
2020-06-06 17:06:56terry.reedyunlinkissue16123 dependencies
2020-06-06 16:26:52terry.reedysetnosy: - kbk, roger.serwy

messages: + msg370832
versions: + Python 3.10, - Python 3.8
2019-02-20 23:54:25terry.reedysetmessages: + msg336154
versions: + Python 3.8, - Python 3.4, Python 3.5
2014-07-13 17:40:31terry.reedylinkissue16123 dependencies
2014-03-29 08:44:01terry.reedysetstage: needs patch
messages: + msg215123
versions: + Python 3.5, - Python 3.3
2014-03-28 19:08:19terry.reedysetmessages: + msg215069
2014-03-28 18:30:24terry.reedysetfiles: +

messages: + msg215062
2014-02-02 05:44:00kbksetnosy: + kbk
2013-09-04 18:44:57ezio.melottisetnosy: + ezio.melotti
2013-08-31 16:19:15serhiy.storchakasetnosy: + serhiy.storchaka
2013-08-30 22:23:12tshepangsetnosy: + tshepang
2013-08-26 06:19:44asvetlovsetnosy: + asvetlov
2013-08-23 23:35:04terry.reedycreate