Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Idle: use pipes instead of sockets to talk with user subprocess #63023

Open
terryjreedy opened this issue Aug 23, 2013 · 7 comments
Open

Idle: use pipes instead of sockets to talk with user subprocess #63023

terryjreedy opened this issue Aug 23, 2013 · 7 comments
Labels
3.10 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@terryjreedy
Copy link
Member

BPO 18823
Nosy @terryjreedy, @ezio-melotti, @asvetlov, @serhiy-storchaka
Files
  • Carlson.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2013-08-23.23:35:04.085>
labels = ['expert-IDLE', 'type-bug', '3.10']
title = 'Idle: use pipes instead of sockets to talk with user subprocess'
updated_at = <Date 2020-06-06.16:26:52.685>
user = 'https://github.com/terryjreedy'

    bugs.python.org fields:

    activity = <Date 2020-06-06.16:26:52.685>
actor = 'terry.reedy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['IDLE']
creation = <Date 2013-08-23.23:35:04.085>
creator = 'terry.reedy'
dependencies = []
files = ['34654']
hgrepos = []
issue_num = 18823
keywords = []
message_count = 6.0
messages = ['196055', '215062', '215069', '215123', '336154', '370832']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'ezio.melotti', 'asvetlov', 'tshepang', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue18823'
versions = ['Python 3.10']
    @terryjreedy
    Copy link
    Member Author

    Idle once used os.spawnv to open the user subprocess and 2.7 still does. Sockets were then used to interact with the subprocess. Since rev71746, 2011-8-3, 3.2+ use subprocess.Popen but still use sockets. A rare but continual problem is failure to make the socket connection.

    In bpo-16123, Amaury Forgeot d'Arc suggested "pipes?" The idea resurfaced again in current Idle-sig thread "Idle does not open on mac" (because socket connection timed out). Guido, who designed the current 2.x system, chimed in with "I would recommend trying to use the subprocess module."

    A current python-list thread indicates that there can be problems with
    "Running a command line program and reading the result as it runs"
    http://mail.python.org/pipermail/python-list/2013-August/654265.html

    In particular, "for line in p:" gets lines delayed (buffered) while
    Peter Otten discovered that "for line in iter(p.stdout.readline, ''):" gets them as they are produced.
    http://mail.python.org/pipermail/python-list/2013-August/654330.html

    We first need to experiment running a simple echo server with python(w).exe. I have not succeeded yet on Windows.

    @terryjreedy terryjreedy added topic-IDLE type-bug An unexpected behavior, bug, or error labels Aug 23, 2013
    @terryjreedy
    Copy link
    Member Author

    In the thread that started here
    https://mail.python.org/pipermail/python-dev/2014-March/133492.html
    in this message.
    https://mail.python.org/pipermail/python-dev/2014-March/133671.html
    Josiah Carlson posted a pastebin link about how to actually use the pipes. Guido recommended that the he post a revised version to a new issue, as a patch to subprocess, but I am uploading it here to make sure it does not get lost.

    @terryjreedy
    Copy link
    Member Author

    https://mail.python.org/pipermail/python-dev/2014-March/133641.html
    from Victor Stinner gives example code using asyncio

    @terryjreedy
    Copy link
    Member Author

    Pydev, Re: Status of PEP-3145 - Asynchronous I/O for subprocess.popen; Antoine Pitrou:

    Why don't you use multiprocessing or concurrent.futures? They have
    everything you need for continuous conversation between processes.

    @terryjreedy
    Copy link
    Member Author

    bpo-34313 reports problems with multiprocessing when using at least some versions of tkinter on some versions of macOS.

    @terryjreedy terryjreedy added the 3.8 only security fixes label Feb 20, 2019
    @terryjreedy
    Copy link
    Member Author

    Problems with multiprocessing and tkinter on Mac might have been due to using the default start method, which used to be fork and is now spawn. This should be retested sometime.

    @terryjreedy terryjreedy added 3.10 only security fixes and removed 3.8 only security fixes labels Jun 6, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @arhadthedev
    Copy link
    Member

    Problems with multiprocessing and tkinter on Mac might have been due to using the default start method, which used to be fork and is now spawn.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.10 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error
    Projects
    IDLE Issues
    Status: No status
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @arhadthedev @terryjreedy