This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: subprocess - uncaught PermissionError in send_signal can cause hang
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: subprocess.run timeout does not function if shell=True and capture_output=True
View: 37424
Assigned To: gregory.p.smith Nosy List: gregory.p.smith, iritkatriel, wesinator
Priority: low Keywords:

Created on 2019-05-29 22:52 by wesinator, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13669 closed wesinator, 2019-05-29 22:52
Messages (4)
msg343939 - (view) Author: wesinator (wesinator) * Date: 2019-05-29 22:52
Python 3.7.3
Ubuntu 18.10 Cosmic

https://github.com/python/cpython/pull/13669

Encountered a condition where uncaught PermissionError caused a hang
running a subprocess command with sudo -u

Traceback (most recent call last):
  File "/usr/lib/python3.7/subprocess.py", line 474, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.7/subprocess.py", line 939, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.7/subprocess.py", line 1707, in _communicate
    self.wait(timeout=self._remaining_time(endtime))
  File "/usr/lib/python3.7/subprocess.py", line 990, in wait
    return self._wait(timeout=timeout)
  File "/usr/lib/python3.7/subprocess.py", line 1616, in _wait
    raise TimeoutExpired(self.args, timeout)
subprocess.TimeoutExpired: Command '['sudo', '-u', 'chrome', '/snap/bin/chromium', '--headless', '--disable-gpu', '--hide-scrollbars', '--ignore-certificate-errors', '--enable-sandbox', '--incognito', '--mute-audio', '--disable-databases', '--enable-strict-powerful-feature-restrictions', '--no-pings', '--no-referrers', '--timeout=30000', '--window-size=1280,1000', '--screenshot=[REDACTED]_screenshot.png', 'https://[REDACTED]']' timed out after 59.9998986274004 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/subprocess.py", line 476, in run
    process.kill()
  File "/usr/lib/python3.7/subprocess.py", line 1756, in kill
    self.send_signal(signal.SIGKILL)
  File "/usr/lib/python3.7/subprocess.py", line 1746, in send_signal
    os.kill(self.pid, sig)
PermissionError: [Errno 1] Operation not permitted
msg344014 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-05-31 00:53
Clarifying: The problem is that the TimeoutExpired exception that is desired was replaced with a PermissionError that the caller isn't expecting to catch and handle.
msg345061 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-06-08 18:47
even with that initially proposed patch, the subprocess.run code is going to hang if the .kill() fails without raising an exception:

The run() TimeoutExpired path calls communicate() after the kill to consume all remaining output: 
  https://github.com/python/cpython/blob/master/Lib/subprocess.py#L478

and the __exit__ code path calls wait().

both assume that the kill() succeeded because the normal case is not someone launching a child process that they are unable to kill (as happens when sudo or similar is used).

Today's workarounds: Don't use timeout when launching a child you cannot kill.  OR  launch it via an intermediate process that you can kill (shell=True or equivalent).

To address this issue, catching a PermissionError exception around both of those kill() calls and preventing it from calling either communicate() or wait() as we cannot force the child process to exit makes sense.  With some indication that this happened left as a hint to the user on the TimeoutExpired itself.
msg411015 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-20 09:49
I think this was fixed in Issue37424.
History
Date User Action Args
2022-04-11 14:59:15adminsetgithub: 81272
2022-01-30 16:51:48iritkatrielsetstatus: pending -> closed
stage: resolved
2022-01-20 09:49:05iritkatrielsetstatus: open -> pending

nosy: + iritkatriel
messages: + msg411015

superseder: subprocess.run timeout does not function if shell=True and capture_output=True
resolution: duplicate
2019-09-11 14:44:55gregory.p.smithsetpriority: normal -> low
2019-06-08 18:47:49gregory.p.smithsetmessages: + msg345061
2019-05-31 00:53:38gregory.p.smithsetversions: + Python 3.8
2019-05-31 00:53:28gregory.p.smithsetmessages: + msg344014
2019-05-31 00:43:58gregory.p.smithsetassignee: gregory.p.smith
2019-05-30 05:50:57xtreaksetnosy: + gregory.p.smith
2019-05-29 22:52:02wesinatorcreate