This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author amrith
Recipients amrith
Date 2014-07-31.11:58:59
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1406807939.82.0.812223355091.issue22114@psf.upfronthosting.co.za>
In-reply-to
Content
Environment:
- Linux (Ubuntu 14.04 LTS, x64) running Python 2.7.
- Code uses eventlet.green.subprocess

Code establishes subprocess as:

   subprocess.Popen(command,
                    stdin=subprocess.PIPE,
                    stdout=subprocess.PIPE,
                    stderr=subprocess.PIPE, ...)

It then calls communicate()

communicate() throws EAGAIN

The code calls communicate() again

This fails with "ValueError: I/O operation on closed file"

This exception comes from eventlet/greenio.py because an operation (flush) has been attempted on a closed file.

The complete explanation of this failure is this.

When communicate() is called in this way (WITH NO INPUT but with all three handles stdin, stdout, stderr), code in communicate() bypasses the "1 handle optimization" and goes directly to _communicate()

_communicate() will first send any input on stdin along, and flush and close stdin(). 

From that point forward, if anything to do with stderr and stdout returns EAGAIN, an attempt to call communicate() again will fail because stdin has now been closed().

This is because the code in _communicate() preemptively closes stdin if there is no input but does not reset stdin.

       def _communicate(self, input):
            if self.stdin:
                # Flush stdio buffer.  This might block, if the user has
                # been writing to .stdin in an uncontrolled fashion.
                self.stdin.flush()
                if not input:
                    self.stdin.close()

The fix for this is relatively straightforward (conceptually).

       def _communicate(self, input):
            if self.stdin:
                # Flush stdio buffer.  This might block, if the user has
                # been writing to .stdin in an uncontrolled fashion.
                self.stdin.flush()
                if not input:
                    self.stdin.close()
                    self.stdin = None # This should take care of it.

However, a partial workaround from the client perspective is this.

1. If you have no input, set stdin to None
2. If you do have input and you get EAGAIN, you cannot safely retry because your input may have already been completely or partially sent to the subprocess; you have to assume failure.

--------------------
Backtrace:

Traceback (most recent call last):
	File "/opt/stack/trove/trove/tests/unittests/guestagent/test_mongodb_manager.py", line 58, in test_prepare_from_backup
		self._prepare_dynamic(backup_id='backup_id_123abc')
	File "/opt/stack/trove/trove/tests/unittests/guestagent/test_mongodb_manager.py", line 91, in _prepare_dynamic
		backup_info=backup_info)
	File "trove/guestagent/datastore/mongodb/manager.py", line 66, in prepare
		operating_system.update_owner('mongodb', 'mongodb', mount_point)
	File "trove/guestagent/common/operating_system.py", line 109, in update_owner
		run_as_root=True, root_helper="sudo")
	File "trove/common/utils.py", line 278, in execute_with_timeout
		return execute(*args, **kwargs)
	File "trove/openstack/common/processutils.py", line 186, in execute
		result = obj.communicate()
	File "/usr/lib/python2.7/subprocess.py", line 799, in communicate
		return self._communicate(input)
	File "/usr/lib/python2.7/subprocess.py", line 1396, in _communicate
		self.stdin.flush()
	File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/eventlet/greenio.py", line 419, in _operationOnClosedFile
		raise ValueError("I/O operation on closed file")

ValueError: I/O operation on closed file
History
Date User Action Args
2014-07-31 11:58:59amrithsetrecipients: + amrith
2014-07-31 11:58:59amrithsetmessageid: <1406807939.82.0.812223355091.issue22114@psf.upfronthosting.co.za>
2014-07-31 11:58:59amrithlinkissue22114 messages
2014-07-31 11:58:59amrithcreate