classification
Title: sys.stdout.write on Python 2.7 is not EINTR safe
Type: Stage: resolved
Components: IO Versions: Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: handle EINTR in the stdlib
View: 18885
Assigned To: Nosy List: gregory.p.smith, jstewmon, ned.deily, neologix, piotr.dobrogost, vstinner
Priority: normal Keywords:

Created on 2014-07-18 21:10 by jstewmon, last changed 2014-07-30 17:07 by piotr.dobrogost. This issue is now closed.

Messages (10)
msg223435 - (view) Author: Jonathan Stewmon (jstewmon) Date: 2014-07-18 21:10
Writing to sys.stdout on OS X can fail with IOError: [Errno 4] Interrupted system call.

I have observed this while trying to write to sys.stdout when SIGCHLD is received. The script below consistently reproduces the problem with python 2.7.2 on OS X 10.9.3.

import sys
import os
import signal
import subprocess
import time


children = {}
def claim_child():
    pid, status = os.wait()
    p = children.pop(pid)

def trap(sig, frame):
    claim_child()

signal.signal(signal.SIGCHLD, trap)

running = 0
max_procs = 70
program = [sys.executable, '-c', 'import sys, time; print sys.version; time.sleep(3)']
f = sys.stdout # crashes with: IOError: [Errno 4] Interrupted system call
# f = file('/tmp/eintr.log', 'w') # works just fine
while True:
    while len(children) < max_procs:
        f.write('starting program: {}\n'.format(program))
        p = subprocess.Popen(program)
        children[p.pid] = p
    time.sleep(0.05)
msg223441 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-18 22:00
There have been a number of EINTR-releated issues reported in the past, some unique to BSD-based systems and/or OS X; see for example, Issue9867 and Issue12268.  I don't know that any of them specifically addressed problems with writes to Python 2 file objects.  FWIW, with your test case, I'm not able to reproduce a failure on 10.9.3 using python.org 2.7.8 or 2.7.3 or with the system 2.7.5.  And, with a modified print(), I didn't see a failure with python.org 3.4.1, either.  Can you say more about the origins of your Python 2.7.2?  Have you tried with a current 2.7.8?  How quickly does the test fail, e.g. after how many iterations?
msg223442 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-18 22:06
This issue is just an example of issue addressed by the issue #18885.
msg223444 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-18 22:14
Yes, an issue not likely to be addressed in Python 2.7.  I'm still curious as to why I'm not able to reproduce the problem, though.  I suppose it could just come down to differences in the system it is running on, like workload, amount of memory, and/or number of processors.
msg223445 - (view) Author: Jonathan Stewmon (jstewmon) Date: 2014-07-18 22:23
Sorry, I had a typo in the original report - I am actually using Python 2.7.7 installed with homebrew.

The script crashes for me on the first iteration every time using iTerm as my terminal. I just tried it in Terminal, and it doesn't crash.

Maybe it's actually an issue with iTerm?
msg223446 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-18 22:32
I can reproduce the issue on Linux:

Traceback (most recent call last):
  File "x.py", line 26, in <module>
    f.write('starting program: {}\n'.format(program))
IOError: [Errno 4] Interrupted system call
msg223469 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-19 16:47
> I can reproduce the issue on Linux:

Sorry, I forgot to mention that I reproduced the issue on Fedora 20 with Python 2.7.

I cannot reproduce with Python 3.3.
msg223527 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-20 19:01
I also can reproduce this on Linux with Python 2.7 but not with 3.4 which uses a different io system.  So the question is: is anyone interested in addressing this for Python 2 file objects?
msg223531 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-20 19:16
I'm interested to work on all EINTR issues, including Python 2.

I'm working on a new PEP with Charles-François Natali to address *all* EINTR at once in Python 3.5. If the PEP is accepted, we may fix some EINTR issues on Python 2. Some changes change the behaviour and so cannot we done in Python 2. Well, the PEP is still a draft, we will see that later.
msg224326 - (view) Author: Piotr Dobrogost (piotr.dobrogost) Date: 2014-07-30 17:07
@hypo
Is there any reason you keep this PEP "secret" :) by not mentioning it on bug 18885?
History
Date User Action Args
2014-07-30 17:07:37piotr.dobrogostsetnosy: + piotr.dobrogost
messages: + msg224326
2014-07-22 20:16:05neologixsetstatus: open -> closed
superseder: handle EINTR in the stdlib
resolution: duplicate
stage: needs patch -> resolved
2014-07-20 19:16:25vstinnersetnosy: + neologix
messages: + msg223531
2014-07-20 19:01:35ned.deilysetnosy: + gregory.p.smith
title: sys.stdout.write on OS X is not EINTR safe -> sys.stdout.write on Python 2.7 is not EINTR safe
messages: + msg223527

stage: needs patch
2014-07-19 16:47:15vstinnersetmessages: + msg223469
2014-07-18 22:32:19vstinnersetmessages: + msg223446
2014-07-18 22:23:47jstewmonsetmessages: + msg223445
2014-07-18 22:14:01ned.deilysetmessages: + msg223444
2014-07-18 22:06:56vstinnersetnosy: + vstinner
messages: + msg223442
2014-07-18 22:00:00ned.deilysetnosy: + ned.deily
messages: + msg223441
2014-07-18 21:10:00jstewmoncreate