classification
Title: multiprocessing expects sys.stdout to have a fileno/close method.
Type: Stage:
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: multiprocessing.process using os.close(sys.stdin.fileno) instead of sys.stdin.close()
View: 5313
Assigned To: Nosy List: Trundle, asksol, ikanobori, jnoller, mher.movsisyan, pitrou
Priority: normal Keywords:

Created on 2010-10-22 23:22 by ikanobori, last changed 2011-01-08 11:33 by pitrou. This issue is now closed.

Messages (5)
msg119408 - (view) Author: Simon de Vlieger (ikanobori) Date: 2010-10-22 23:22
When I have replaced sys.stdin with my own file-like object and I try to do a multiprocessing.Pool(processes=x) I get errors about sys.stdin not having a fileno or close method.

For at least fileno it is described in the docs (http://docs.python.org/library/stdtypes.html#file-objects) that if your object is not a real file you should not implement it.

This happens to me on Mac OS X, I will add the traceback a bit later as I am currently not on my Mac.
msg119422 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2010-10-23 10:29
Please add the traceback,  I can't seem to find any obvious places where this would happen now.

Also, what version are you currently using?


I agree with the fileno, but I'd say close is a reasonable method to implement, especially for stdin/stdout/stderr
msg125718 - (view) Author: Mher Movsisyan (mher.movsisyan) Date: 2011-01-07 21:52
The reported error is only reproducible in 2.6

Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import multiprocessing
>>> class A:pass
... 
>>> sys.stdin=A()
>>> p=multiprocessing.Pool(processes=2)
Process PoolWorker-1:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/multiprocessing/process.py", line 223, in _bootstrap
Process PoolWorker-2:
Traceback (most recent call last):
    os.close(sys.stdin.fileno())
AttributeError: A instance has no attribute 'fileno'
>>>   File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/multiprocessing/process.py", line 223, in _bootstrap
    os.close(sys.stdin.fileno())
AttributeError: A instance has no attribute 'fileno'
msg125720 - (view) Author: Mher Movsisyan (mher.movsisyan) Date: 2011-01-07 22:02
This bug was fixed in #5313

--- a/multiprocessing/process.py
+++ b/multiprocessing/process.py
@@ -225,7 +225,8 @@ class Process(object):
             self._children = set()
             self._counter = itertools.count(1)
             try:
-                os.close(sys.stdin.fileno())
+                sys.stdin.close()
+                sys.stdin = open(os.devnull)
             except (OSError, ValueError):
                 pass
             _current_process = self
msg125779 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-08 11:33
Ok, closing then.
History
Date User Action Args
2011-01-08 11:33:02pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg125779

superseder: multiprocessing.process using os.close(sys.stdin.fileno) instead of sys.stdin.close()
resolution: duplicate
2011-01-07 22:02:52mher.movsisyansetnosy: jnoller, Trundle, mher.movsisyan, asksol, ikanobori
messages: + msg125720
2011-01-07 21:52:26mher.movsisyansetnosy: + mher.movsisyan
messages: + msg125718
2010-10-23 10:29:53asksolsetmessages: + msg119422
2010-10-23 09:56:14pitrousetnosy: + asksol, jnoller

components: + Library (Lib), - None
versions: + Python 3.1, Python 3.2, - Python 2.6
2010-10-23 00:42:19Trundlesetnosy: + Trundle
2010-10-22 23:22:17ikanoboricreate