classification
Title: Idle: use pipes instead of sockets to talk with user subprocess
Type: behavior Stage: needs patch
Components: IDLE Versions: Python 3.5, Python 3.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, ezio.melotti, kbk, roger.serwy, serhiy.storchaka, terry.reedy, tshepang
Priority: normal Keywords:

Created on 2013-08-23 23:35 by terry.reedy, last changed 2014-03-29 08:44 by terry.reedy.

Files
File name Uploaded Description Edit
Carlson.py terry.reedy, 2014-03-28 18:30
Messages (4)
msg196055 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-23 23:35
Idle once used os.spawnv to open the user subprocess and 2.7 still does. Sockets were then used to interact with the subprocess. Since rev71746, 2011-8-3, 3.2+ use subprocess.Popen but still use sockets. A rare but continual problem is failure to make the socket connection.

In #16123, Amaury Forgeot d'Arc suggested "pipes?" The idea resurfaced again in current Idle-sig thread "Idle does not open on mac" (because socket connection timed out). Guido, who designed the current 2.x system, chimed in with "I would recommend trying to use the subprocess module."

A current python-list thread indicates that there can be problems with
"Running a command line program and reading the result as it runs"
http://mail.python.org/pipermail/python-list/2013-August/654265.html

In particular, "for line in p:" gets lines delayed (buffered) while
Peter Otten discovered that "for line in iter(p.stdout.readline, ''):" gets them as they are produced.
http://mail.python.org/pipermail/python-list/2013-August/654330.html

We first need to experiment running a simple echo server with python(w).exe.  I have not succeeded yet on Windows.
msg215062 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-28 18:30
In the thread that started here
https://mail.python.org/pipermail/python-dev/2014-March/133492.html
in this message.
https://mail.python.org/pipermail/python-dev/2014-March/133671.html
Josiah Carlson posted a pastebin link about how to actually use the pipes. Guido recommended that the he post a revised version to a new issue, as a patch to subprocess, but I am uploading it here to make sure it does not get lost.
msg215069 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-28 19:08
https://mail.python.org/pipermail/python-dev/2014-March/133641.html
from Victor Stinner gives example code using asyncio
msg215123 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-29 08:44
Pydev, Re: Status of PEP 3145 - Asynchronous I/O for subprocess.popen; Antoine Pitrou:
> Why don't you use multiprocessing or concurrent.futures? They have
> everything you need for continuous conversation between processes.
History
Date User Action Args
2014-07-13 17:40:31terry.reedylinkissue16123 dependencies
2014-03-29 08:44:01terry.reedysetstage: needs patch
messages: + msg215123
versions: + Python 3.5, - Python 3.3
2014-03-28 19:08:19terry.reedysetmessages: + msg215069
2014-03-28 18:30:24terry.reedysetfiles: + Carlson.py

messages: + msg215062
2014-02-02 05:44:00kbksetnosy: + kbk
2013-09-04 18:44:57ezio.melottisetnosy: + ezio.melotti
2013-08-31 16:19:15serhiy.storchakasetnosy: + serhiy.storchaka
2013-08-30 22:23:12tshepangsetnosy: + tshepang
2013-08-26 06:19:44asvetlovsetnosy: + asvetlov
2013-08-23 23:35:04terry.reedycreate