classification
Title: pipe.read hang, when calling commands.getstatusoutput in multi-threading code of python 2.4
Type: behavior Stage: resolved
Components: Library (Lib) Versions: 3rd party
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: denny, r.david.murray, rosslagerwall
Priority: normal Keywords:

Created on 2010-08-06 09:30 by denny, last changed 2011-01-15 16:13 by r.david.murray. This issue is now closed.

Messages (4)
msg113086 - (view) Author: denny (denny) Date: 2010-08-06 09:30
Hi all

My environment is python 2.4.4 on linux. I am encountering hang of pipe.read() in commands.getstatusoutput for multi-threading code.

I have spawned several threads which will call commands.getstatusoutput to run cli. However, pipe.read may hang sometimes.
From lsof, we know some pipe handles are not closed in the parent process, after the child process is stopped.

;; -------------------------- Reproduce steps --------------------------
Below are reproduce steps.
# Create a python script of /tmp/hang.py, whose content is given below.
# Run "service crond stop; python /tmp/hang.py" several times.
# The script may probably hang. From lsof of main.py and crond service, we may find one pipe existing in both processes.
# If we stop crond to close the pipe, the hang of hang.py will be resolved.

;; -------------------------- Code of hang.py --------------------------
#!/usr/bin/python
import commands
import datetime
import time
import os
import thread
import threading

def thread_run1():

    cmd = "hostname"
    print "begin to run command:%s" % (cmd)
    status, output = commands.getstatusoutput(cmd)
    print "pid:%d, name:%s, status:%d, output:%s" % \
        (os.getpid(), threading.currentThread().getName(), status, output)

    cmd = "ifconfig eth0"
    print "begin to run command:%s" % (cmd)
    status, output = commands.getstatusoutput(cmd)
    print "pid:%d, name:%s, status:%d, output:%s" % \
        (os.getpid(), threading.currentThread().getName(), status, output)

    cmd = "ifconfig eth1"
    print "begin to run command:%s" % (cmd)
    status, output = commands.getstatusoutput(cmd)
    print "pid:%d, name:%s, status:%d, output:%s" % \
        (os.getpid(), threading.currentThread().getName(), status, output)

    # cmd = "sh /tmp/subprocess.sh"
    cmd = "echo here1; sleep 2; echo here2; sleep 5"
    print "begin to run command:%s" % (cmd)
    status, output = commands.getstatusoutput(cmd)
    print "pid:%d, name:%s, status:%d, output:%s" % \
        (os.getpid(), threading.currentThread().getName(), status, output)

def thread_run2():

    cmd = "service crond start"
    print "begin to run command:%s" % (cmd)
    status, output = commands.getstatusoutput(cmd)
    print "pid:%d, name:%s, status:%d, output:%s" % \
        (os.getpid(), threading.currentThread().getName(), status, output)

if __name__=='__main__':
    print "main function begins."
    thread_list = []
    for i in xrange(1, 10):
        my_thread = threading.Thread(target = thread_run1)
        thread_list.append(my_thread)

    my_thread = threading.Thread(target = thread_run2)
    thread_list.append(my_thread)

    for t in thread_list:
        t.start()

    time.sleep(10)

    for t in thread_list:
        t.join()

    print "main function ends."
msg113139 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-08-06 21:42
2.4 is no longer maintained by the CPython team.  Can you reproduce this in 2.7?
msg113384 - (view) Author: denny (denny) Date: 2010-08-09 05:16
Hi David

I have tried in another testbd with python 2.6.5, and the problem of hang doesn't reproduce, after retrying for several times.

The original hang happens in pipe.read of commands module.

After comparing the code of python 2.4.4 and python 2.6.5, I noticed two enhancements in posimodule.c:posix_read.

These defensive coding add precheck, before invoking read function.

David, without these enhancements, would it cause the potential hang problem, in your opinion?
Recompiling python 2.4.4 with some manual changes is a little scaring to me.

,----------- python 2.4.4 posixmodule.c
| static PyObject *
| posix_read(PyObject *self, PyObject *args)
| {
|     int fd, size, n;
|     PyObject *buffer;
|     if (!PyArg_ParseTuple(args, "ii:read", &fd, &size))
|         return NULL;
|     buffer = PyString_FromStringAndSize((char *)NULL, size);
|     if (buffer == NULL)
|         return NULL;
|     Py_BEGIN_ALLOW_THREADS
|     n = read(fd, PyString_AsString(buffer), size);
|     Py_END_ALLOW_THREADS
|     if (n < 0) {
|         Py_DECREF(buffer);
|         return posix_error();
|     }
|     if (n != size)
|         _PyString_Resize(&buffer, n);
|     return buffer;
| }
`-----------

,----------- 2.6.5 posixmodule.c
| static PyObject *
| posix_read(PyObject *self, PyObject *args)
| {
|     int fd, size, n;
|     PyObject *buffer;
|     if (!PyArg_ParseTuple(args, "ii:read", &fd, &size))
|         return NULL;
|     if (size < 0) {
|         errno = EINVAL;
|         return posix_error();
|     }
|     buffer = PyString_FromStringAndSize((char *)NULL, size);
|     if (buffer == NULL)
|         return NULL;
|     if (!_PyVerify_fd(fd))
|         return posix_error();
|     Py_BEGIN_ALLOW_THREADS
|     n = read(fd, PyString_AsString(buffer), size);
|     Py_END_ALLOW_THREADS
|     if (n < 0) {
|         Py_DECREF(buffer);
|         return posix_error();
|     }
|     if (n != size)
|         _PyString_Resize(&buffer, n);
|     return buffer;
| }
`-----------
msg126317 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-01-15 07:59
After trying to reproduce this bug in 2.7.1 & 3.2b2 and failing, I think this should be closed (even the OP couldn't reproduce it in anything other than 2.4).
History
Date User Action Args
2011-01-15 16:13:54r.david.murraysetstatus: open -> closed
nosy: r.david.murray, denny, rosslagerwall
resolution: out of date
stage: resolved
2011-01-15 07:59:57rosslagerwallsetnosy: + rosslagerwall
messages: + msg126317
2010-08-09 05:16:21dennysetmessages: + msg113384
2010-08-06 21:42:30r.david.murraysetnosy: + r.david.murray
messages: + msg113139
2010-08-06 09:30:10dennycreate