Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIPE_BUF is invalid on AIX #54071

Closed
sable mannequin opened this issue Sep 15, 2010 · 7 comments
Closed

PIPE_BUF is invalid on AIX #54071

sable mannequin opened this issue Sep 15, 2010 · 7 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@sable
Copy link
Mannequin

sable mannequin commented Sep 15, 2010

BPO 9862
Nosy @amauryfa, @bitdancer
Files
  • patch_broken_pipe_buf.diff
  • patch_broken_pipe_buf_updated.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/bitdancer'
    closed_at = <Date 2010-10-16.00:49:07.914>
    created_at = <Date 2010-09-15.16:30:33.584>
    labels = ['type-bug', 'library']
    title = 'PIPE_BUF is invalid on AIX'
    updated_at = <Date 2010-10-18.09:24:10.790>
    user = 'https://bugs.python.org/sable'

    bugs.python.org fields:

    activity = <Date 2010-10-18.09:24:10.790>
    actor = 'sable'
    assignee = 'r.david.murray'
    closed = True
    closed_date = <Date 2010-10-16.00:49:07.914>
    closer = 'r.david.murray'
    components = ['Library (Lib)']
    creation = <Date 2010-09-15.16:30:33.584>
    creator = 'sable'
    dependencies = []
    files = ['18937', '18944']
    hgrepos = []
    issue_num = 9862
    keywords = ['patch']
    message_count = 7.0
    messages = ['116467', '116932', '116952', '117031', '117040', '118849', '118998']
    nosy_count = 3.0
    nosy_names = ['amaury.forgeotdarc', 'sable', 'r.david.murray']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue9862'
    versions = ['Python 2.7', 'Python 3.2']

    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Sep 15, 2010

    On AIX, the test test_communicate_pipe_buf in test_subprocess will hang forever (in py3k and py27):

    test_communicate_pipe_buf (main.ProcessTestCase) ...

    File "Lib/test/test_subprocess.py", line 386, in test_communicate_pipe_buf
    (stdout, stderr) = p.communicate(string_to_write)
    File "/san_cis/home/cis/data/bamboo-home-agent-runtime/xml-data/build-dir/RTAIX30-SUP/Python-2.7-svn/Lib/subprocess.py", line 740, in communicate
    return self._communicate(input)
    File "/san_cis/home/cis/data/bamboo-home-agent-runtime/xml-data/build-dir/RTAIX30-SUP/Python-2.7-svn/Lib/subprocess.py", line 1257, in _communicate
    stdout, stderr = self._communicate_with_poll(input)
    File "/san_cis/home/cis/data/bamboo-home-agent-runtime/xml-data/build-dir/RTAIX30-SUP/Python-2.7-svn/Lib/subprocess.py", line 1320, in _communicate_with_poll
    input_offset += os.write(fd, chunk)
    KeyboardInterrupt

    The comment in this test indicates:
    # communicate() with writes larger than pipe_buf
    # This test will probably deadlock rather than fail, if
    # communicate() does not work properly.

    So I guess it means communicate() does not work properly on AIX.

    @sable sable mannequin added the stdlib Python modules in the Lib dir label Sep 15, 2010
    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Sep 20, 2010

    The problem does not happen with Python 2.6.
    The difference is that:

    • in Python 2.6, the subprocess module would try to write at most 512 bytes
      cf L1221 in subprocess.py
      chunk = input[input_offset : input_offset + 512]

    • in Python 2.7 and py3k, the subprocess module will try to write at most _PIPE_BUF bytes
      cf L1319 in subprocess.py
      chunk = input[input_offset : input_offset + _PIPE_BUF]

    When forcing PIPE_BUF to 512 in selectmodule.c, the test will pass.
    PIPE_BUF seems to be broken on AIX.

    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Sep 20, 2010

    PIPE_BUF in unistd.h is defined to 32768.

    The test works with PIPE_BUF changed up to 6144 but won't work with 7168.

    We could probably use 4096 as a safe value for faster result if needed.
    I just do not define the value and leave the default of 512 be used.

    Here is my patch which has been verified to work OK and should be applied to py3k and py27.

    @sable sable mannequin changed the title test_subprocess hangs on AIX PIPE_BUF is invalid on AIX Sep 20, 2010
    @amauryfa
    Copy link
    Member

    Removing select.PIPE_BUF does not seem a good idea to me because this breaks compatibility. I suggest to simply set it to 512 on AIX.

    (An ideal solution would be to really determine the actual buffer size in ./configure; this is probably overkill)

    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Sep 21, 2010

    OK for me.
    Here is a new patch that defines PIPE_BUF to 512 instead of removing select.PIPE_BUF.

    @bitdancer bitdancer self-assigned this Oct 15, 2010
    @bitdancer bitdancer added the type-bug An unexpected behavior, bug, or error label Oct 15, 2010
    @bitdancer
    Copy link
    Member

    Committed to py3k in r85554, 2.7 in r85556. Sébastien, from what you say it sounds like this does not apply to 3.1, so I blocked it there. If this is incorrect let me know and I'll backport it.

    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Oct 18, 2010

    Thanks R. David.

    I checked in 3.1 and PIPE_BUF is not defined in the select module, so the default value of 512 is used in subprocess.
    So no correction is needed for that version.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants