Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(106)

Delta Between Two Patch Sets: Lib/subprocess.py

Issue 12085: subprocess.Popen.__del__ raises AttributeError if __init__ was called with an invalid argument list
Left Patch Set: Created 6 years, 6 months ago
Right Patch Set: Created 4 years, 1 month ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
Left: Side by side diff | Download
Right: Side by side diff | Download
« no previous file with change/comment | « no previous file | no next file » | no next file with change/comment »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
LEFTRIGHT
1 # subprocess - Subprocesses with accessible I/O streams 1 # subprocess - Subprocesses with accessible I/O streams
2 # 2 #
3 # For more information about this module, see PEP 324. 3 # For more information about this module, see PEP 324.
4 # 4 #
5 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se> 5 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
6 # 6 #
7 # Licensed to PSF under a Contributor Agreement. 7 # Licensed to PSF under a Contributor Agreement.
8 # See http://www.python.org/2.4/license for licensing details. 8 # See http://www.python.org/2.4/license for licensing details.
9 9
10 r"""subprocess - Subprocesses with accessible I/O streams 10 r"""subprocess - Subprocesses with accessible I/O streams
11 11
12 This module allows you to spawn processes, connect to their 12 This module allows you to spawn processes, connect to their
13 input/output/error pipes, and obtain their return codes. This module 13 input/output/error pipes, and obtain their return codes. This module
14 intends to replace several other, older modules and functions, like: 14 intends to replace several other, older modules and functions, like:
15 15
16 os.system 16 os.system
17 os.spawn* 17 os.spawn*
18 18
19 Information about how the subprocess module can be used to replace these 19 Information about how the subprocess module can be used to replace these
20 modules and functions can be found below. 20 modules and functions can be found below.
21 21
22 22
23 23
24 Using the subprocess module 24 Using the subprocess module
25 =========================== 25 ===========================
26 This module defines one class called Popen: 26 This module defines one class called Popen:
27 27
28 class Popen(args, bufsize=0, executable=None, 28 class Popen(args, bufsize=-1, executable=None,
29 stdin=None, stdout=None, stderr=None, 29 stdin=None, stdout=None, stderr=None,
30 preexec_fn=None, close_fds=True, shell=False, 30 preexec_fn=None, close_fds=True, shell=False,
31 cwd=None, env=None, universal_newlines=False, 31 cwd=None, env=None, universal_newlines=False,
32 startupinfo=None, creationflags=0, 32 startupinfo=None, creationflags=0,
33 restore_signals=True, start_new_session=False, pass_fds=()): 33 restore_signals=True, start_new_session=False, pass_fds=()):
34 34
35 35
36 Arguments are: 36 Arguments are:
37 37
38 args should be a string, or a sequence of program arguments. The 38 args should be a string, or a sequence of program arguments. The
(...skipping 10 matching lines...) Expand all
49 the first item specifies the command string, and any additional items 49 the first item specifies the command string, and any additional items
50 will be treated as additional shell arguments. 50 will be treated as additional shell arguments.
51 51
52 On Windows: the Popen class uses CreateProcess() to execute the child 52 On Windows: the Popen class uses CreateProcess() to execute the child
53 program, which operates on strings. If args is a sequence, it will be 53 program, which operates on strings. If args is a sequence, it will be
54 converted to a string using the list2cmdline method. Please note that 54 converted to a string using the list2cmdline method. Please note that
55 not all MS Windows applications interpret the command line the same 55 not all MS Windows applications interpret the command line the same
56 way: The list2cmdline is designed for applications using the same 56 way: The list2cmdline is designed for applications using the same
57 rules as the MS C runtime. 57 rules as the MS C runtime.
58 58
59 bufsize, if given, has the same meaning as the corresponding argument 59 bufsize will be supplied as the corresponding argument to the io.open()
60 to the built-in open() function: 0 means unbuffered, 1 means line 60 function when creating the stdin/stdout/stderr pipe file objects:
61 buffered, any other positive value means use a buffer of 61 0 means unbuffered (read & write are one system call and can return short),
62 (approximately) that size. A negative bufsize means to use the system 62 1 means line buffered, any other positive value means use a buffer of
63 default, which usually means fully buffered. The default value for 63 approximately that size. A negative bufsize, the default, means the system
64 bufsize is 0 (unbuffered). 64 default of io.DEFAULT_BUFFER_SIZE will be used.
65 65
66 stdin, stdout and stderr specify the executed programs' standard 66 stdin, stdout and stderr specify the executed programs' standard
67 input, standard output and standard error file handles, respectively. 67 input, standard output and standard error file handles, respectively.
68 Valid values are PIPE, an existing file descriptor (a positive 68 Valid values are PIPE, an existing file descriptor (a positive
69 integer), an existing file object, and None. PIPE indicates that a 69 integer), an existing file object, and None. PIPE indicates that a
70 new pipe to the child should be created. With None, no redirection 70 new pipe to the child should be created. With None, no redirection
71 will occur; the child's file handles will be inherited from the 71 will occur; the child's file handles will be inherited from the
72 parent. Additionally, stderr can be STDOUT, which indicates that the 72 parent. Additionally, stderr can be STDOUT, which indicates that the
73 stderr data from the applications should be captured into the same 73 stderr data from the applications should be captured into the same
74 file handle as for stdout. 74 file handle as for stdout.
(...skipping 22 matching lines...) Expand all
97 SIG_IGN are restored to SIG_DFL in the child process before the exec. 97 SIG_IGN are restored to SIG_DFL in the child process before the exec.
98 Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This 98 Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99 parameter does nothing on Windows. 99 parameter does nothing on Windows.
100 100
101 On POSIX, if start_new_session is True, the setsid() system call will be made 101 On POSIX, if start_new_session is True, the setsid() system call will be made
102 in the child process prior to executing the command. 102 in the child process prior to executing the command.
103 103
104 If env is not None, it defines the environment variables for the new 104 If env is not None, it defines the environment variables for the new
105 process. 105 process.
106 106
107 If universal_newlines is false, the file objects stdin, stdout and stderr
108 are opened as binary files, and no line ending conversion is done.
109
107 If universal_newlines is true, the file objects stdout and stderr are 110 If universal_newlines is true, the file objects stdout and stderr are
108 opened as a text files, but lines may be terminated by any of '\n', 111 opened as a text files, but lines may be terminated by any of '\n',
109 the Unix end-of-line convention, '\r', the old Macintosh convention or 112 the Unix end-of-line convention, '\r', the old Macintosh convention or
110 '\r\n', the Windows convention. All of these external representations 113 '\r\n', the Windows convention. All of these external representations
111 are seen as '\n' by the Python program. Note: This feature is only 114 are seen as '\n' by the Python program. Also, the newlines attribute
112 available if Python is built with universal newline support (the 115 of the file objects stdout, stdin and stderr are not updated by the
113 default). Also, the newlines attribute of the file objects stdout, 116 communicate() method.
114 stdin and stderr are not updated by the communicate() method.
115 117
116 The startupinfo and creationflags, if given, will be passed to the 118 The startupinfo and creationflags, if given, will be passed to the
117 underlying CreateProcess() function. They can specify things such as 119 underlying CreateProcess() function. They can specify things such as
118 appearance of the main window and priority for the new process. 120 appearance of the main window and priority for the new process.
119 (Windows only) 121 (Windows only)
120 122
121 123
122 This module also defines some shortcut functions: 124 This module also defines some shortcut functions:
123 125
124 call(*popenargs, **kwargs): 126 call(*popenargs, **kwargs):
(...skipping 34 matching lines...) Expand 10 before | Expand all | Expand 10 after
159 getoutput(cmd): 161 getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell. 162 Return output (stdout or stderr) of executing cmd in a shell.
161 163
162 Like getstatusoutput(), except the exit status is ignored and the return 164 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example: 165 value is a string containing the command's output. Example:
164 166
165 >>> subprocess.getoutput('ls /bin/ls') 167 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls' 168 '/bin/ls'
167 169
168 check_output(*popenargs, **kwargs): 170 check_output(*popenargs, **kwargs):
169 Run command with arguments and return its output as a byte string. 171 Run command with arguments and return its output.
170 172
171 If the exit code was non-zero it raises a CalledProcessError. The 173 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode 174 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute. 175 attribute and output in the output attribute.
174 176
175 The arguments are the same as for the Popen constructor. Example: 177 The arguments are the same as for the Popen constructor. Example:
176 178
177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"]) 179 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
178 180
181 There is an additional optional argument, "input", allowing you to
182 pass a string to the subprocess's stdin. If you use this argument
183 you may not also use the Popen constructor's "stdin" argument.
179 184
180 Exceptions 185 Exceptions
181 ---------- 186 ----------
182 Exceptions raised in the child process, before the new program has 187 Exceptions raised in the child process, before the new program has
183 started to execute, will be re-raised in the parent. Additionally, 188 started to execute, will be re-raised in the parent. Additionally,
184 the exception object will have one extra attribute called 189 the exception object will have one extra attribute called
185 'child_traceback', which is a string containing traceback information 190 'child_traceback', which is a string containing traceback information
186 from the childs point of view. 191 from the child's point of view.
187 192
188 The most common exception raised is OSError. This occurs, for 193 The most common exception raised is OSError. This occurs, for
189 example, when trying to execute a non-existent file. Applications 194 example, when trying to execute a non-existent file. Applications
190 should prepare for OSErrors. 195 should prepare for OSErrors.
191 196
192 A ValueError will be raised if Popen is called with invalid arguments. 197 A ValueError will be raised if Popen is called with invalid arguments.
193 198
194 Exceptions defined within this module inherit from SubprocessError. 199 Exceptions defined within this module inherit from SubprocessError.
195 check_call() and check_output() will raise CalledProcessError if the 200 check_call() and check_output() will raise CalledProcessError if the
196 called process returns a non-zero return code. TimeoutExpired 201 called process returns a non-zero return code. TimeoutExpired
(...skipping 145 matching lines...) Expand 10 before | Expand all | Expand 10 after
342 347
343 import io 348 import io
344 import os 349 import os
345 import time 350 import time
346 import traceback 351 import traceback
347 import gc 352 import gc
348 import signal 353 import signal
349 import builtins 354 import builtins
350 import warnings 355 import warnings
351 import errno 356 import errno
357 try:
358 from time import monotonic as _time
359 except ImportError:
360 from time import time as _time
352 361
353 # Exception classes used by this module. 362 # Exception classes used by this module.
354 class SubprocessError(Exception): pass 363 class SubprocessError(Exception): pass
355 364
356 365
357 class CalledProcessError(SubprocessError): 366 class CalledProcessError(SubprocessError):
358 """This exception is raised when a process run by check_call() or 367 """This exception is raised when a process run by check_call() or
359 check_output() returns a non-zero exit status. 368 check_output() returns a non-zero exit status.
360 The exit status will be stored in the returncode attribute; 369 The exit status will be stored in the returncode attribute;
361 check_output() will also store the output in the output attribute. 370 check_output() will also store the output in the output attribute.
(...skipping 16 matching lines...) Expand all
378 self.output = output 387 self.output = output
379 388
380 def __str__(self): 389 def __str__(self):
381 return ("Command '%s' timed out after %s seconds" % 390 return ("Command '%s' timed out after %s seconds" %
382 (self.cmd, self.timeout)) 391 (self.cmd, self.timeout))
383 392
384 393
385 if mswindows: 394 if mswindows:
386 import threading 395 import threading
387 import msvcrt 396 import msvcrt
388 import _subprocess 397 import _winapi
389 class STARTUPINFO: 398 class STARTUPINFO:
390 dwFlags = 0 399 dwFlags = 0
391 hStdInput = None 400 hStdInput = None
392 hStdOutput = None 401 hStdOutput = None
393 hStdError = None 402 hStdError = None
394 wShowWindow = 0 403 wShowWindow = 0
395 class pywintypes:
396 error = IOError
397 else: 404 else:
398 import select 405 import select
399 _has_poll = hasattr(select, 'poll') 406 _has_poll = hasattr(select, 'poll')
400 import fcntl 407 import _posixsubprocess
401 import pickle
402
403 try:
404 import _posixsubprocess
405 except ImportError:
406 _posixsubprocess = None
407 warnings.warn("The _posixsubprocess module is not being used. "
408 "Child process reliability may suffer if your "
409 "program uses threads.", RuntimeWarning)
410 408
411 # When select or poll has indicated that the file is writable, 409 # When select or poll has indicated that the file is writable,
412 # we can write up to _PIPE_BUF bytes without risk of blocking. 410 # we can write up to _PIPE_BUF bytes without risk of blocking.
413 # POSIX defines PIPE_BUF as >= 512. 411 # POSIX defines PIPE_BUF as >= 512.
414 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512) 412 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
415 413
416 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
417
418 def _set_cloexec(fd, cloexec):
419 old = fcntl.fcntl(fd, fcntl.F_GETFD)
420 if cloexec:
421 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
422 else:
423 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
424
425 if _posixsubprocess:
426 _create_pipe = _posixsubprocess.cloexec_pipe
427 else:
428 def _create_pipe():
429 fds = os.pipe()
430 _set_cloexec(fds[0], True)
431 _set_cloexec(fds[1], True)
432 return fds
433 414
434 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput", 415 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
435 "getoutput", "check_output", "CalledProcessError", "DEVNULL"] 416 "getoutput", "check_output", "CalledProcessError", "DEVNULL"]
436 417
437 if mswindows: 418 if mswindows:
438 from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP, 419 from _winapi import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
439 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, 420 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
440 STD_ERROR_HANDLE, SW_HIDE, 421 STD_ERROR_HANDLE, SW_HIDE,
441 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW) 422 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
442 423
443 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP", 424 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
444 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE", 425 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
445 "STD_ERROR_HANDLE", "SW_HIDE", 426 "STD_ERROR_HANDLE", "SW_HIDE",
446 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"]) 427 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
428
429 class Handle(int):
430 closed = False
431
432 def Close(self, CloseHandle=_winapi.CloseHandle):
433 if not self.closed:
434 self.closed = True
435 CloseHandle(self)
436
437 def Detach(self):
438 if not self.closed:
439 self.closed = True
440 return int(self)
441 raise ValueError("already closed")
442
443 def __repr__(self):
444 return "Handle(%d)" % int(self)
445
446 __del__ = Close
447 __str__ = __repr__
448
447 try: 449 try:
448 MAXFD = os.sysconf("SC_OPEN_MAX") 450 MAXFD = os.sysconf("SC_OPEN_MAX")
449 except: 451 except:
450 MAXFD = 256 452 MAXFD = 256
451 453
454 # This lists holds Popen instances for which the underlying process had not
455 # exited at the time its __del__ method got called: those processes are wait()ed
456 # for synchronously from _cleanup() when a new Popen object is created, to avoid
457 # zombie processes.
452 _active = [] 458 _active = []
453 459
454 def _cleanup(): 460 def _cleanup():
455 for inst in _active[:]: 461 for inst in _active[:]:
456 res = inst._internal_poll(_deadstate=sys.maxsize) 462 res = inst._internal_poll(_deadstate=sys.maxsize)
457 if res is not None and res >= 0: 463 if res is not None:
458 try: 464 try:
459 _active.remove(inst) 465 _active.remove(inst)
460 except ValueError: 466 except ValueError:
461 # This can happen if two threads create a new Popen instance. 467 # This can happen if two threads create a new Popen instance.
462 # It's harmless that it was already removed, so ignore. 468 # It's harmless that it was already removed, so ignore.
463 pass 469 pass
464 470
465 PIPE = -1 471 PIPE = -1
466 STDOUT = -2 472 STDOUT = -2
467 DEVNULL = -3 473 DEVNULL = -3
468 474
469 475
470 def _eintr_retry_call(func, *args): 476 def _eintr_retry_call(func, *args):
471 while True: 477 while True:
472 try: 478 try:
473 return func(*args) 479 return func(*args)
474 except OSError as e: 480 except InterruptedError:
475 if e.errno == errno.EINTR: 481 continue
476 continue 482
477 raise 483
484 # XXX This function is only used by multiprocessing and the test suite,
485 # but it's here so that it can be imported when Python is compiled without
486 # threads.
487
488 def _args_from_interpreter_flags():
489 """Return a list of command-line arguments reproducing the current
490 settings in sys.flags and sys.warnoptions."""
491 flag_opt_map = {
492 'debug': 'd',
493 # 'inspect': 'i',
494 # 'interactive': 'i',
495 'optimize': 'O',
496 'dont_write_bytecode': 'B',
497 'no_user_site': 's',
498 'no_site': 'S',
499 'ignore_environment': 'E',
500 'verbose': 'v',
501 'bytes_warning': 'b',
502 'quiet': 'q',
503 'hash_randomization': 'R',
504 }
505 args = []
506 for flag, opt in flag_opt_map.items():
507 v = getattr(sys.flags, flag)
508 if v > 0:
509 args.append('-' + opt * v)
510 for opt in sys.warnoptions:
511 args.append('-W' + opt)
512 return args
478 513
479 514
480 def call(*popenargs, timeout=None, **kwargs): 515 def call(*popenargs, timeout=None, **kwargs):
481 """Run command with arguments. Wait for command to complete or 516 """Run command with arguments. Wait for command to complete or
482 timeout, then return the returncode attribute. 517 timeout, then return the returncode attribute.
483 518
484 The arguments are the same as for the Popen constructor. Example: 519 The arguments are the same as for the Popen constructor. Example:
485 520
486 retcode = call(["ls", "-l"]) 521 retcode = call(["ls", "-l"])
487 """ 522 """
488 p = Popen(*popenargs, **kwargs) 523 with Popen(*popenargs, **kwargs) as p:
489 try: 524 try:
490 return p.wait(timeout=timeout) 525 return p.wait(timeout=timeout)
491 except TimeoutExpired: 526 except:
492 p.kill() 527 p.kill()
493 p.wait() 528 p.wait()
494 raise 529 raise
495 530
496 531
497 def check_call(*popenargs, **kwargs): 532 def check_call(*popenargs, **kwargs):
498 """Run command with arguments. Wait for command to complete. If 533 """Run command with arguments. Wait for command to complete. If
499 the exit code was zero then return, otherwise raise 534 the exit code was zero then return, otherwise raise
500 CalledProcessError. The CalledProcessError object will have the 535 CalledProcessError. The CalledProcessError object will have the
501 return code in the returncode attribute. 536 return code in the returncode attribute.
502 537
503 The arguments are the same as for the call function. Example: 538 The arguments are the same as for the call function. Example:
504 539
505 check_call(["ls", "-l"]) 540 check_call(["ls", "-l"])
506 """ 541 """
507 retcode = call(*popenargs, **kwargs) 542 retcode = call(*popenargs, **kwargs)
508 if retcode: 543 if retcode:
509 cmd = kwargs.get("args") 544 cmd = kwargs.get("args")
510 if cmd is None: 545 if cmd is None:
511 cmd = popenargs[0] 546 cmd = popenargs[0]
512 raise CalledProcessError(retcode, cmd) 547 raise CalledProcessError(retcode, cmd)
513 return 0 548 return 0
514 549
515 550
516 def check_output(*popenargs, timeout=None, **kwargs): 551 def check_output(*popenargs, timeout=None, **kwargs):
517 r"""Run command with arguments and return its output as a byte string. 552 r"""Run command with arguments and return its output.
518 553
519 If the exit code was non-zero it raises a CalledProcessError. The 554 If the exit code was non-zero it raises a CalledProcessError. The
520 CalledProcessError object will have the return code in the returncode 555 CalledProcessError object will have the return code in the returncode
521 attribute and output in the output attribute. 556 attribute and output in the output attribute.
522 557
523 The arguments are the same as for the Popen constructor. Example: 558 The arguments are the same as for the Popen constructor. Example:
524 559
525 >>> check_output(["ls", "-l", "/dev/null"]) 560 >>> check_output(["ls", "-l", "/dev/null"])
526 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n' 561 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
527 562
528 The stdout argument is not allowed as it is used internally. 563 The stdout argument is not allowed as it is used internally.
529 To capture standard error in the result, use stderr=STDOUT. 564 To capture standard error in the result, use stderr=STDOUT.
530 565
531 >>> check_output(["/bin/sh", "-c", 566 >>> check_output(["/bin/sh", "-c",
532 ... "ls -l non_existent_file ; exit 0"], 567 ... "ls -l non_existent_file ; exit 0"],
533 ... stderr=STDOUT) 568 ... stderr=STDOUT)
534 b'ls: non_existent_file: No such file or directory\n' 569 b'ls: non_existent_file: No such file or directory\n'
570
571 There is an additional optional argument, "input", allowing you to
572 pass a string to the subprocess's stdin. If you use this argument
573 you may not also use the Popen constructor's "stdin" argument, as
574 it too will be used internally. Example:
575
576 >>> check_output(["sed", "-e", "s/foo/bar/"],
577 ... input=b"when in the course of fooman events\n")
578 b'when in the course of barman events\n'
579
580 If universal_newlines=True is passed, the return value will be a
581 string rather than bytes.
535 """ 582 """
536 if 'stdout' in kwargs: 583 if 'stdout' in kwargs:
537 raise ValueError('stdout argument not allowed, it will be overridden.') 584 raise ValueError('stdout argument not allowed, it will be overridden.')
538 process = Popen(*popenargs, stdout=PIPE, **kwargs) 585 if 'input' in kwargs:
539 try: 586 if 'stdin' in kwargs:
540 output, unused_err = process.communicate(timeout=timeout) 587 raise ValueError('stdin and input arguments may not both be used.')
541 except TimeoutExpired: 588 inputdata = kwargs['input']
542 process.kill() 589 del kwargs['input']
543 output, unused_err = process.communicate() 590 kwargs['stdin'] = PIPE
544 raise TimeoutExpired(process.args, timeout, output=output) 591 else:
545 retcode = process.poll() 592 inputdata = None
546 if retcode: 593 with Popen(*popenargs, stdout=PIPE, **kwargs) as process:
547 raise CalledProcessError(retcode, process.args, output=output) 594 try:
595 output, unused_err = process.communicate(inputdata, timeout=timeout)
596 except TimeoutExpired:
597 process.kill()
598 output, unused_err = process.communicate()
599 raise TimeoutExpired(process.args, timeout, output=output)
600 except:
601 process.kill()
602 process.wait()
603 raise
604 retcode = process.poll()
605 if retcode:
606 raise CalledProcessError(retcode, process.args, output=output)
548 return output 607 return output
549 608
550 609
551 def list2cmdline(seq): 610 def list2cmdline(seq):
552 """ 611 """
553 Translate a sequence of arguments into a command line 612 Translate a sequence of arguments into a command line
554 string, using the same rules as the MS C runtime: 613 string, using the same rules as the MS C runtime:
555 614
556 1) Arguments are delimited by white space, which is either a 615 1) Arguments are delimited by white space, which is either a
557 space or a tab. 616 space or a tab.
(...skipping 74 matching lines...) Expand 10 before | Expand all | Expand 10 after
632 interpreted according to the rules for the C function wait(). Example: 691 interpreted according to the rules for the C function wait(). Example:
633 692
634 >>> import subprocess 693 >>> import subprocess
635 >>> subprocess.getstatusoutput('ls /bin/ls') 694 >>> subprocess.getstatusoutput('ls /bin/ls')
636 (0, '/bin/ls') 695 (0, '/bin/ls')
637 >>> subprocess.getstatusoutput('cat /bin/junk') 696 >>> subprocess.getstatusoutput('cat /bin/junk')
638 (256, 'cat: /bin/junk: No such file or directory') 697 (256, 'cat: /bin/junk: No such file or directory')
639 >>> subprocess.getstatusoutput('/bin/junk') 698 >>> subprocess.getstatusoutput('/bin/junk')
640 (256, 'sh: /bin/junk: not found') 699 (256, 'sh: /bin/junk: not found')
641 """ 700 """
642 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r') 701 with os.popen('{ ' + cmd + '; } 2>&1', 'r') as pipe:
643 text = pipe.read() 702 try:
644 sts = pipe.close() 703 text = pipe.read()
645 if sts is None: sts = 0 704 sts = pipe.close()
646 if text[-1:] == '\n': text = text[:-1] 705 except:
706 process = pipe._proc
707 process.kill()
708 process.wait()
709 raise
710 if sts is None:
711 sts = 0
712 if text[-1:] == '\n':
713 text = text[:-1]
647 return sts, text 714 return sts, text
648 715
649 716
650 def getoutput(cmd): 717 def getoutput(cmd):
651 """Return output (stdout or stderr) of executing cmd in a shell. 718 """Return output (stdout or stderr) of executing cmd in a shell.
652 719
653 Like getstatusoutput(), except the exit status is ignored and the return 720 Like getstatusoutput(), except the exit status is ignored and the return
654 value is a string containing the command's output. Example: 721 value is a string containing the command's output. Example:
655 722
656 >>> import subprocess 723 >>> import subprocess
657 >>> subprocess.getoutput('ls /bin/ls') 724 >>> subprocess.getoutput('ls /bin/ls')
658 '/bin/ls' 725 '/bin/ls'
659 """ 726 """
660 return getstatusoutput(cmd)[1] 727 return getstatusoutput(cmd)[1]
661 728
662 729
663 _PLATFORM_DEFAULT_CLOSE_FDS = object() 730 _PLATFORM_DEFAULT_CLOSE_FDS = object()
664 731
665 732
666 class Popen(object): 733 class Popen(object):
667 def __init__(self, args, bufsize=0, executable=None, 734
735 _child_created = False # Set here since __del__ checks it
736
737 def __init__(self, args, bufsize=-1, executable=None,
668 stdin=None, stdout=None, stderr=None, 738 stdin=None, stdout=None, stderr=None,
669 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, 739 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
670 shell=False, cwd=None, env=None, universal_newlines=False, 740 shell=False, cwd=None, env=None, universal_newlines=False,
671 startupinfo=None, creationflags=0, 741 startupinfo=None, creationflags=0,
672 restore_signals=True, start_new_session=False, 742 restore_signals=True, start_new_session=False,
673 pass_fds=()): 743 pass_fds=()):
674 """Create new Popen instance.""" 744 """Create new Popen instance."""
675 _cleanup() 745 _cleanup()
676 746
677 self._child_created = False
678 self._input = None 747 self._input = None
679 self._communication_started = False 748 self._communication_started = False
680 if bufsize is None: 749 if bufsize is None:
681 bufsize = 0 # Restore default 750 bufsize = -1 # Restore default
682 if not isinstance(bufsize, int): 751 if not isinstance(bufsize, int):
683 raise TypeError("bufsize must be an integer") 752 raise TypeError("bufsize must be an integer")
684 753
685 if mswindows: 754 if mswindows:
686 if preexec_fn is not None: 755 if preexec_fn is not None:
687 raise ValueError("preexec_fn is not supported on Windows " 756 raise ValueError("preexec_fn is not supported on Windows "
688 "platforms") 757 "platforms")
689 any_stdio_set = (stdin is not None or stdout is not None or 758 any_stdio_set = (stdin is not None or stdout is not None or
690 stderr is not None) 759 stderr is not None)
691 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: 760 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
(...skipping 53 matching lines...) Expand 10 before | Expand all | Expand 10 after
745 if mswindows: 814 if mswindows:
746 if p2cwrite != -1: 815 if p2cwrite != -1:
747 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) 816 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
748 if c2pread != -1: 817 if c2pread != -1:
749 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) 818 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
750 if errread != -1: 819 if errread != -1:
751 errread = msvcrt.open_osfhandle(errread.Detach(), 0) 820 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
752 821
753 if p2cwrite != -1: 822 if p2cwrite != -1:
754 self.stdin = io.open(p2cwrite, 'wb', bufsize) 823 self.stdin = io.open(p2cwrite, 'wb', bufsize)
755 if self.universal_newlines: 824 if universal_newlines:
756 self.stdin = io.TextIOWrapper(self.stdin) 825 self.stdin = io.TextIOWrapper(self.stdin, write_through=True)
757 if c2pread != -1: 826 if c2pread != -1:
758 self.stdout = io.open(c2pread, 'rb', bufsize) 827 self.stdout = io.open(c2pread, 'rb', bufsize)
759 if universal_newlines: 828 if universal_newlines:
760 self.stdout = io.TextIOWrapper(self.stdout) 829 self.stdout = io.TextIOWrapper(self.stdout)
761 if errread != -1: 830 if errread != -1:
762 self.stderr = io.open(errread, 'rb', bufsize) 831 self.stderr = io.open(errread, 'rb', bufsize)
763 if universal_newlines: 832 if universal_newlines:
764 self.stderr = io.TextIOWrapper(self.stderr) 833 self.stderr = io.TextIOWrapper(self.stderr)
765 834
835 self._closed_child_pipe_fds = False
766 try: 836 try:
767 self._execute_child(args, executable, preexec_fn, close_fds, 837 self._execute_child(args, executable, preexec_fn, close_fds,
768 pass_fds, cwd, env, universal_newlines, 838 pass_fds, cwd, env,
769 startupinfo, creationflags, shell, 839 startupinfo, creationflags, shell,
770 p2cread, p2cwrite, 840 p2cread, p2cwrite,
771 c2pread, c2pwrite, 841 c2pread, c2pwrite,
772 errread, errwrite, 842 errread, errwrite,
773 restore_signals, start_new_session) 843 restore_signals, start_new_session)
774 except: 844 except:
775 # Cleanup if the child failed starting 845 # Cleanup if the child failed starting.
776 for f in filter(None, [self.stdin, self.stdout, self.stderr]): 846 for f in filter(None, (self.stdin, self.stdout, self.stderr)):
777 try: 847 try:
778 f.close() 848 f.close()
779 except EnvironmentError: 849 except OSError:
780 # Ignore EBADF or other errors 850 pass # Ignore EBADF or other errors.
781 pass 851
852 if not self._closed_child_pipe_fds:
853 to_close = []
854 if stdin == PIPE:
855 to_close.append(p2cread)
856 if stdout == PIPE:
857 to_close.append(c2pwrite)
858 if stderr == PIPE:
859 to_close.append(errwrite)
860 if hasattr(self, '_devnull'):
861 to_close.append(self._devnull)
862 for fd in to_close:
863 try:
864 os.close(fd)
865 except OSError:
866 pass
867
782 raise 868 raise
783 869
784 870
785 def _translate_newlines(self, data, encoding): 871 def _translate_newlines(self, data, encoding):
786 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n") 872 data = data.decode(encoding)
787 return data.decode(encoding) 873 return data.replace("\r\n", "\n").replace("\r", "\n")
788 874
789 def __enter__(self): 875 def __enter__(self):
790 return self 876 return self
791 877
792 def __exit__(self, type, value, traceback): 878 def __exit__(self, type, value, traceback):
793 if self.stdout: 879 if self.stdout:
794 self.stdout.close() 880 self.stdout.close()
795 if self.stderr: 881 if self.stderr:
796 self.stderr.close() 882 self.stderr.close()
797 if self.stdin: 883 if self.stdin:
798 self.stdin.close() 884 self.stdin.close()
799 # Wait for the process to terminate, to avoid zombies. 885 # Wait for the process to terminate, to avoid zombies.
800 self.wait() 886 self.wait()
801 887
802 def __del__(self, _maxsize=sys.maxsize, _active=_active): 888 def __del__(self, _maxsize=sys.maxsize):
803 if not getattr(self, '_child_created', False): 889 if not self._child_created:
804 # We didn't get to successfully create a child process. 890 # We didn't get to successfully create a child process.
805 return 891 return
806 # In case the child hasn't been waited on, check if it's done. 892 # In case the child hasn't been waited on, check if it's done.
807 self._internal_poll(_deadstate=_maxsize) 893 self._internal_poll(_deadstate=_maxsize)
808 if self.returncode is None and _active is not None: 894 if self.returncode is None and _active is not None:
809 # Child is still running, keep us alive until we can wait on it. 895 # Child is still running, keep us alive until we can wait on it.
810 _active.append(self) 896 _active.append(self)
811 897
812 def _get_devnull(self): 898 def _get_devnull(self):
813 if not hasattr(self, '_devnull'): 899 if not hasattr(self, '_devnull'):
814 self._devnull = os.open(os.devnull, os.O_RDWR) 900 self._devnull = os.open(os.devnull, os.O_RDWR)
815 return self._devnull 901 return self._devnull
816 902
817 def communicate(self, input=None, timeout=None): 903 def communicate(self, input=None, timeout=None):
818 """Interact with process: Send data to stdin. Read data from 904 """Interact with process: Send data to stdin. Read data from
819 stdout and stderr, until end-of-file is reached. Wait for 905 stdout and stderr, until end-of-file is reached. Wait for
820 process to terminate. The optional input argument should be a 906 process to terminate. The optional input argument should be
821 string to be sent to the child process, or None, if no data 907 bytes to be sent to the child process, or None, if no data
822 should be sent to the child. 908 should be sent to the child.
823 909
824 communicate() returns a tuple (stdout, stderr).""" 910 communicate() returns a tuple (stdout, stderr)."""
825 911
826 if self._communication_started and input: 912 if self._communication_started and input:
827 raise ValueError("Cannot send input after starting communication") 913 raise ValueError("Cannot send input after starting communication")
828 914
829 # Optimization: If we are not worried about timeouts, we haven't 915 # Optimization: If we are not worried about timeouts, we haven't
830 # started communicating, and we have one or zero pipes, using select() 916 # started communicating, and we have one or zero pipes, using select()
831 # or threads is unnecessary. 917 # or threads is unnecessary.
832 if (timeout is None and not self._communication_started and 918 if (timeout is None and not self._communication_started and
833 [self.stdin, self.stdout, self.stderr].count(None) >= 2): 919 [self.stdin, self.stdout, self.stderr].count(None) >= 2):
834 stdout = None 920 stdout = None
835 stderr = None 921 stderr = None
836 if self.stdin: 922 if self.stdin:
837 if input: 923 if input:
838 try: 924 try:
839 self.stdin.write(input) 925 self.stdin.write(input)
840 except IOError as e: 926 except OSError as e:
841 if e.errno != errno.EPIPE and e.errno != errno.EINVAL: 927 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
842 raise 928 raise
843 self.stdin.close() 929 self.stdin.close()
844 elif self.stdout: 930 elif self.stdout:
845 stdout = self.stdout.read() 931 stdout = _eintr_retry_call(self.stdout.read)
846 self.stdout.close() 932 self.stdout.close()
847 elif self.stderr: 933 elif self.stderr:
848 stderr = self.stderr.read() 934 stderr = _eintr_retry_call(self.stderr.read)
849 self.stderr.close() 935 self.stderr.close()
850 self.wait() 936 self.wait()
851 else: 937 else:
852 if timeout is not None: 938 if timeout is not None:
853 endtime = time.time() + timeout 939 endtime = _time() + timeout
854 else: 940 else:
855 endtime = None 941 endtime = None
856 942
857 try: 943 try:
858 stdout, stderr = self._communicate(input, endtime, timeout) 944 stdout, stderr = self._communicate(input, endtime, timeout)
859 finally: 945 finally:
860 self._communication_started = True 946 self._communication_started = True
861 947
862 sts = self.wait(timeout=self._remaining_time(endtime)) 948 sts = self.wait(timeout=self._remaining_time(endtime))
863 949
864 return (stdout, stderr) 950 return (stdout, stderr)
865 951
866 952
867 def poll(self): 953 def poll(self):
868 return self._internal_poll() 954 return self._internal_poll()
869 955
870 956
871 def _remaining_time(self, endtime): 957 def _remaining_time(self, endtime):
872 """Convenience for _communicate when computing timeouts.""" 958 """Convenience for _communicate when computing timeouts."""
873 if endtime is None: 959 if endtime is None:
874 return None 960 return None
875 else: 961 else:
876 return endtime - time.time() 962 return endtime - _time()
877 963
878 964
879 def _check_timeout(self, endtime, orig_timeout): 965 def _check_timeout(self, endtime, orig_timeout):
880 """Convenience for checking if a timeout has expired.""" 966 """Convenience for checking if a timeout has expired."""
881 if endtime is None: 967 if endtime is None:
882 return 968 return
883 if time.time() > endtime: 969 if _time() > endtime:
884 raise TimeoutExpired(self.args, orig_timeout) 970 raise TimeoutExpired(self.args, orig_timeout)
885 971
886 972
887 if mswindows: 973 if mswindows:
888 # 974 #
889 # Windows methods 975 # Windows methods
890 # 976 #
891 def _get_handles(self, stdin, stdout, stderr): 977 def _get_handles(self, stdin, stdout, stderr):
892 """Construct and return tuple with IO objects: 978 """Construct and return tuple with IO objects:
893 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 979 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
894 """ 980 """
895 if stdin is None and stdout is None and stderr is None: 981 if stdin is None and stdout is None and stderr is None:
896 return (-1, -1, -1, -1, -1, -1) 982 return (-1, -1, -1, -1, -1, -1)
897 983
898 p2cread, p2cwrite = -1, -1 984 p2cread, p2cwrite = -1, -1
899 c2pread, c2pwrite = -1, -1 985 c2pread, c2pwrite = -1, -1
900 errread, errwrite = -1, -1 986 errread, errwrite = -1, -1
901 987
902 if stdin is None: 988 if stdin is None:
903 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE) 989 p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
904 if p2cread is None: 990 if p2cread is None:
905 p2cread, _ = _subprocess.CreatePipe(None, 0) 991 p2cread, _ = _winapi.CreatePipe(None, 0)
992 p2cread = Handle(p2cread)
993 _winapi.CloseHandle(_)
906 elif stdin == PIPE: 994 elif stdin == PIPE:
907 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0) 995 p2cread, p2cwrite = _winapi.CreatePipe(None, 0)
996 p2cread, p2cwrite = Handle(p2cread), Handle(p2cwrite)
908 elif stdin == DEVNULL: 997 elif stdin == DEVNULL:
909 p2cread = msvcrt.get_osfhandle(self._get_devnull()) 998 p2cread = msvcrt.get_osfhandle(self._get_devnull())
910 elif isinstance(stdin, int): 999 elif isinstance(stdin, int):
911 p2cread = msvcrt.get_osfhandle(stdin) 1000 p2cread = msvcrt.get_osfhandle(stdin)
912 else: 1001 else:
913 # Assuming file-like object 1002 # Assuming file-like object
914 p2cread = msvcrt.get_osfhandle(stdin.fileno()) 1003 p2cread = msvcrt.get_osfhandle(stdin.fileno())
915 p2cread = self._make_inheritable(p2cread) 1004 p2cread = self._make_inheritable(p2cread)
916 1005
917 if stdout is None: 1006 if stdout is None:
918 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDL E) 1007 c2pwrite = _winapi.GetStdHandle(_winapi.STD_OUTPUT_HANDLE)
919 if c2pwrite is None: 1008 if c2pwrite is None:
920 _, c2pwrite = _subprocess.CreatePipe(None, 0) 1009 _, c2pwrite = _winapi.CreatePipe(None, 0)
1010 c2pwrite = Handle(c2pwrite)
1011 _winapi.CloseHandle(_)
921 elif stdout == PIPE: 1012 elif stdout == PIPE:
922 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0) 1013 c2pread, c2pwrite = _winapi.CreatePipe(None, 0)
1014 c2pread, c2pwrite = Handle(c2pread), Handle(c2pwrite)
923 elif stdout == DEVNULL: 1015 elif stdout == DEVNULL:
924 c2pwrite = msvcrt.get_osfhandle(self._get_devnull()) 1016 c2pwrite = msvcrt.get_osfhandle(self._get_devnull())
925 elif isinstance(stdout, int): 1017 elif isinstance(stdout, int):
926 c2pwrite = msvcrt.get_osfhandle(stdout) 1018 c2pwrite = msvcrt.get_osfhandle(stdout)
927 else: 1019 else:
928 # Assuming file-like object 1020 # Assuming file-like object
929 c2pwrite = msvcrt.get_osfhandle(stdout.fileno()) 1021 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
930 c2pwrite = self._make_inheritable(c2pwrite) 1022 c2pwrite = self._make_inheritable(c2pwrite)
931 1023
932 if stderr is None: 1024 if stderr is None:
933 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE ) 1025 errwrite = _winapi.GetStdHandle(_winapi.STD_ERROR_HANDLE)
934 if errwrite is None: 1026 if errwrite is None:
935 _, errwrite = _subprocess.CreatePipe(None, 0) 1027 _, errwrite = _winapi.CreatePipe(None, 0)
1028 errwrite = Handle(errwrite)
1029 _winapi.CloseHandle(_)
936 elif stderr == PIPE: 1030 elif stderr == PIPE:
937 errread, errwrite = _subprocess.CreatePipe(None, 0) 1031 errread, errwrite = _winapi.CreatePipe(None, 0)
1032 errread, errwrite = Handle(errread), Handle(errwrite)
938 elif stderr == STDOUT: 1033 elif stderr == STDOUT:
939 errwrite = c2pwrite 1034 errwrite = c2pwrite
940 elif stderr == DEVNULL: 1035 elif stderr == DEVNULL:
941 errwrite = msvcrt.get_osfhandle(self._get_devnull()) 1036 errwrite = msvcrt.get_osfhandle(self._get_devnull())
942 elif isinstance(stderr, int): 1037 elif isinstance(stderr, int):
943 errwrite = msvcrt.get_osfhandle(stderr) 1038 errwrite = msvcrt.get_osfhandle(stderr)
944 else: 1039 else:
945 # Assuming file-like object 1040 # Assuming file-like object
946 errwrite = msvcrt.get_osfhandle(stderr.fileno()) 1041 errwrite = msvcrt.get_osfhandle(stderr.fileno())
947 errwrite = self._make_inheritable(errwrite) 1042 errwrite = self._make_inheritable(errwrite)
948 1043
949 return (p2cread, p2cwrite, 1044 return (p2cread, p2cwrite,
950 c2pread, c2pwrite, 1045 c2pread, c2pwrite,
951 errread, errwrite) 1046 errread, errwrite)
952 1047
953 1048
954 def _make_inheritable(self, handle): 1049 def _make_inheritable(self, handle):
955 """Return a duplicate of handle, which is inheritable""" 1050 """Return a duplicate of handle, which is inheritable"""
956 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(), 1051 h = _winapi.DuplicateHandle(
957 handle, _subprocess.GetCurrentProcess(), 0, 1, 1052 _winapi.GetCurrentProcess(), handle,
958 _subprocess.DUPLICATE_SAME_ACCESS) 1053 _winapi.GetCurrentProcess(), 0, 1,
959 1054 _winapi.DUPLICATE_SAME_ACCESS)
960 1055 return Handle(h)
961 def _find_w9xpopen(self):
962 """Find and return absolut path to w9xpopen.exe"""
963 w9xpopen = os.path.join(
964 os.path.dirname(_subprocess.GetModuleFileName(0)),
965 "w9xpopen.exe")
966 if not os.path.exists(w9xpopen):
967 # Eeek - file-not-found - possibly an embedding
968 # situation - see if we can locate it in sys.exec_prefix
969 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
970 "w9xpopen.exe")
971 if not os.path.exists(w9xpopen):
972 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
973 "needed for Popen to work with your "
974 "shell or platform.")
975 return w9xpopen
976 1056
977 1057
978 def _execute_child(self, args, executable, preexec_fn, close_fds, 1058 def _execute_child(self, args, executable, preexec_fn, close_fds,
979 pass_fds, cwd, env, universal_newlines, 1059 pass_fds, cwd, env,
980 startupinfo, creationflags, shell, 1060 startupinfo, creationflags, shell,
981 p2cread, p2cwrite, 1061 p2cread, p2cwrite,
982 c2pread, c2pwrite, 1062 c2pread, c2pwrite,
983 errread, errwrite, 1063 errread, errwrite,
984 unused_restore_signals, unused_start_new_session): 1064 unused_restore_signals, unused_start_new_session):
985 """Execute program (MS Windows version)""" 1065 """Execute program (MS Windows version)"""
986 1066
987 assert not pass_fds, "pass_fds not supported on Windows." 1067 assert not pass_fds, "pass_fds not supported on Windows."
988 1068
989 if not isinstance(args, str): 1069 if not isinstance(args, str):
990 args = list2cmdline(args) 1070 args = list2cmdline(args)
991 1071
992 # Process startup details 1072 # Process startup details
993 if startupinfo is None: 1073 if startupinfo is None:
994 startupinfo = STARTUPINFO() 1074 startupinfo = STARTUPINFO()
995 if -1 not in (p2cread, c2pwrite, errwrite): 1075 if -1 not in (p2cread, c2pwrite, errwrite):
996 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES 1076 startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES
997 startupinfo.hStdInput = p2cread 1077 startupinfo.hStdInput = p2cread
998 startupinfo.hStdOutput = c2pwrite 1078 startupinfo.hStdOutput = c2pwrite
999 startupinfo.hStdError = errwrite 1079 startupinfo.hStdError = errwrite
1000 1080
1001 if shell: 1081 if shell:
1002 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW 1082 startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
1003 startupinfo.wShowWindow = _subprocess.SW_HIDE 1083 startupinfo.wShowWindow = _winapi.SW_HIDE
1004 comspec = os.environ.get("COMSPEC", "cmd.exe") 1084 comspec = os.environ.get("COMSPEC", "cmd.exe")
1005 args = '{} /c "{}"'.format (comspec, args) 1085 args = '{} /c "{}"'.format (comspec, args)
1006 if (_subprocess.GetVersion() >= 0x80000000 or
1007 os.path.basename(comspec).lower() == "command.com"):
1008 # Win9x, or using command.com on NT. We need to
1009 # use the w9xpopen intermediate program. For more
1010 # information, see KB Q150956
1011 # (http://web.archive.org/web/20011105084002/http://support. microsoft.com/support/kb/articles/Q150/9/56.asp)
1012 w9xpopen = self._find_w9xpopen()
1013 args = '"%s" %s' % (w9xpopen, args)
1014 # Not passing CREATE_NEW_CONSOLE has been known to
1015 # cause random failures on win9x. Specifically a
1016 # dialog: "Your program accessed mem currently in
1017 # use at xxx" and a hopeful warning about the
1018 # stability of your system. Cost is Ctrl+C won't
1019 # kill children.
1020 creationflags |= _subprocess.CREATE_NEW_CONSOLE
1021 1086
1022 # Start the process 1087 # Start the process
1023 try: 1088 try:
1024 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args, 1089 hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
1025 # no special security 1090 # no special security
1026 None, None, 1091 None, None,
1027 int(not close_fds), 1092 int(not close_fds),
1028 creationflags, 1093 creationflags,
1029 env, 1094 env,
1030 cwd, 1095 cwd,
1031 startupinfo) 1096 startupinfo)
1032 except pywintypes.error as e:
1033 # Translate pywintypes.error to WindowsError, which is
1034 # a subclass of OSError. FIXME: We should really
1035 # translate errno using _sys_errlist (or similar), but
1036 # how can this be done from Python?
1037 raise WindowsError(*e.args)
1038 finally: 1097 finally:
1039 # Child is launched. Close the parent's copy of those pipe 1098 # Child is launched. Close the parent's copy of those pipe
1040 # handles that only the child should have open. You need 1099 # handles that only the child should have open. You need
1041 # to make sure that no handles to the write end of the 1100 # to make sure that no handles to the write end of the
1042 # output pipe are maintained in this process or else the 1101 # output pipe are maintained in this process or else the
1043 # pipe will not close when the child process exits and the 1102 # pipe will not close when the child process exits and the
1044 # ReadFile will hang. 1103 # ReadFile will hang.
1045 if p2cread != -1: 1104 if p2cread != -1:
1046 p2cread.Close() 1105 p2cread.Close()
1047 if c2pwrite != -1: 1106 if c2pwrite != -1:
1048 c2pwrite.Close() 1107 c2pwrite.Close()
1049 if errwrite != -1: 1108 if errwrite != -1:
1050 errwrite.Close() 1109 errwrite.Close()
1051 if hasattr(self, '_devnull'): 1110 if hasattr(self, '_devnull'):
1052 os.close(self._devnull) 1111 os.close(self._devnull)
1053 1112
1054 # Retain the process handle, but close the thread handle 1113 # Retain the process handle, but close the thread handle
1055 self._child_created = True 1114 self._child_created = True
1056 self._handle = hp 1115 self._handle = Handle(hp)
1057 self.pid = pid 1116 self.pid = pid
1058 ht.Close() 1117 _winapi.CloseHandle(ht)
1059 1118
1060 def _internal_poll(self, _deadstate=None, 1119 def _internal_poll(self, _deadstate=None,
1061 _WaitForSingleObject=_subprocess.WaitForSingleObject, 1120 _WaitForSingleObject=_winapi.WaitForSingleObject,
1062 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0, 1121 _WAIT_OBJECT_0=_winapi.WAIT_OBJECT_0,
1063 _GetExitCodeProcess=_subprocess.GetExitCodeProcess): 1122 _GetExitCodeProcess=_winapi.GetExitCodeProcess):
1064 """Check if child process has terminated. Returns returncode 1123 """Check if child process has terminated. Returns returncode
1065 attribute. 1124 attribute.
1066 1125
1067 This method is called by __del__, so it can only refer to objects 1126 This method is called by __del__, so it can only refer to objects
1068 in its local scope. 1127 in its local scope.
1069 1128
1070 """ 1129 """
1071 if self.returncode is None: 1130 if self.returncode is None:
1072 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0: 1131 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
1073 self.returncode = _GetExitCodeProcess(self._handle) 1132 self.returncode = _GetExitCodeProcess(self._handle)
1074 return self.returncode 1133 return self.returncode
1075 1134
1076 1135
1077 def wait(self, timeout=None, endtime=None): 1136 def wait(self, timeout=None, endtime=None):
1078 """Wait for child process to terminate. Returns returncode 1137 """Wait for child process to terminate. Returns returncode
1079 attribute.""" 1138 attribute."""
1080 if endtime is not None: 1139 if endtime is not None:
1081 timeout = self._remaining_time(endtime) 1140 timeout = self._remaining_time(endtime)
1082 if timeout is None: 1141 if timeout is None:
1083 timeout_millis = _subprocess.INFINITE 1142 timeout_millis = _winapi.INFINITE
1084 else: 1143 else:
1085 timeout_millis = int(timeout * 1000) 1144 timeout_millis = int(timeout * 1000)
1086 if self.returncode is None: 1145 if self.returncode is None:
1087 result = _subprocess.WaitForSingleObject(self._handle, 1146 result = _winapi.WaitForSingleObject(self._handle,
1088 timeout_millis) 1147 timeout_millis)
1089 if result == _subprocess.WAIT_TIMEOUT: 1148 if result == _winapi.WAIT_TIMEOUT:
1090 raise TimeoutExpired(self.args, timeout) 1149 raise TimeoutExpired(self.args, timeout)
1091 self.returncode = _subprocess.GetExitCodeProcess(self._handle) 1150 self.returncode = _winapi.GetExitCodeProcess(self._handle)
1092 return self.returncode 1151 return self.returncode
1093 1152
1094 1153
1095 def _readerthread(self, fh, buffer): 1154 def _readerthread(self, fh, buffer):
1096 buffer.append(fh.read()) 1155 buffer.append(fh.read())
1097 fh.close() 1156 fh.close()
1098 1157
1099 1158
1100 def _communicate(self, input, endtime, orig_timeout): 1159 def _communicate(self, input, endtime, orig_timeout):
1101 # Start reader threads feeding into a list hanging off of this 1160 # Start reader threads feeding into a list hanging off of this
(...skipping 10 matching lines...) Expand all
1112 self.stderr_thread = \ 1171 self.stderr_thread = \
1113 threading.Thread(target=self._readerthread, 1172 threading.Thread(target=self._readerthread,
1114 args=(self.stderr, self._stderr_buff)) 1173 args=(self.stderr, self._stderr_buff))
1115 self.stderr_thread.daemon = True 1174 self.stderr_thread.daemon = True
1116 self.stderr_thread.start() 1175 self.stderr_thread.start()
1117 1176
1118 if self.stdin: 1177 if self.stdin:
1119 if input is not None: 1178 if input is not None:
1120 try: 1179 try:
1121 self.stdin.write(input) 1180 self.stdin.write(input)
1122 except IOError as e: 1181 except OSError as e:
1123 if e.errno != errno.EPIPE: 1182 if e.errno != errno.EPIPE:
1124 raise 1183 raise
1125 self.stdin.close() 1184 self.stdin.close()
1126 1185
1127 # Wait for the reader threads, or time out. If we time out, the 1186 # Wait for the reader threads, or time out. If we time out, the
1128 # threads remain reading and the fds left open in case the user 1187 # threads remain reading and the fds left open in case the user
1129 # calls communicate again. 1188 # calls communicate again.
1130 if self.stdout is not None: 1189 if self.stdout is not None:
1131 self.stdout_thread.join(self._remaining_time(endtime)) 1190 self.stdout_thread.join(self._remaining_time(endtime))
1132 if self.stdout_thread.isAlive(): 1191 if self.stdout_thread.is_alive():
1133 raise TimeoutExpired(self.args, orig_timeout) 1192 raise TimeoutExpired(self.args, orig_timeout)
1134 if self.stderr is not None: 1193 if self.stderr is not None:
1135 self.stderr_thread.join(self._remaining_time(endtime)) 1194 self.stderr_thread.join(self._remaining_time(endtime))
1136 if self.stderr_thread.isAlive(): 1195 if self.stderr_thread.is_alive():
1137 raise TimeoutExpired(self.args, orig_timeout) 1196 raise TimeoutExpired(self.args, orig_timeout)
1138 1197
1139 # Collect the output from and close both pipes, now that we know 1198 # Collect the output from and close both pipes, now that we know
1140 # both have been read successfully. 1199 # both have been read successfully.
1141 stdout = None 1200 stdout = None
1142 stderr = None 1201 stderr = None
1143 if self.stdout: 1202 if self.stdout:
1144 stdout = self._stdout_buff 1203 stdout = self._stdout_buff
1145 self.stdout.close() 1204 self.stdout.close()
1146 if self.stderr: 1205 if self.stderr:
(...skipping 16 matching lines...) Expand all
1163 elif sig == signal.CTRL_C_EVENT: 1222 elif sig == signal.CTRL_C_EVENT:
1164 os.kill(self.pid, signal.CTRL_C_EVENT) 1223 os.kill(self.pid, signal.CTRL_C_EVENT)
1165 elif sig == signal.CTRL_BREAK_EVENT: 1224 elif sig == signal.CTRL_BREAK_EVENT:
1166 os.kill(self.pid, signal.CTRL_BREAK_EVENT) 1225 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
1167 else: 1226 else:
1168 raise ValueError("Unsupported signal: {}".format(sig)) 1227 raise ValueError("Unsupported signal: {}".format(sig))
1169 1228
1170 def terminate(self): 1229 def terminate(self):
1171 """Terminates the process 1230 """Terminates the process
1172 """ 1231 """
1173 _subprocess.TerminateProcess(self._handle, 1) 1232 try:
1233 _winapi.TerminateProcess(self._handle, 1)
1234 except PermissionError:
1235 # ERROR_ACCESS_DENIED (winerror 5) is received when the
1236 # process already died.
1237 rc = _winapi.GetExitCodeProcess(self._handle)
1238 if rc == _winapi.STILL_ACTIVE:
1239 raise
1240 self.returncode = rc
1174 1241
1175 kill = terminate 1242 kill = terminate
1176 1243
1177 else: 1244 else:
1178 # 1245 #
1179 # POSIX methods 1246 # POSIX methods
1180 # 1247 #
1181 def _get_handles(self, stdin, stdout, stderr): 1248 def _get_handles(self, stdin, stdout, stderr):
1182 """Construct and return tuple with IO objects: 1249 """Construct and return tuple with IO objects:
1183 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 1250 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1184 """ 1251 """
1185 p2cread, p2cwrite = -1, -1 1252 p2cread, p2cwrite = -1, -1
1186 c2pread, c2pwrite = -1, -1 1253 c2pread, c2pwrite = -1, -1
1187 errread, errwrite = -1, -1 1254 errread, errwrite = -1, -1
1188 1255
1189 if stdin is None: 1256 if stdin is None:
1190 pass 1257 pass
1191 elif stdin == PIPE: 1258 elif stdin == PIPE:
1192 p2cread, p2cwrite = _create_pipe() 1259 p2cread, p2cwrite = os.pipe()
1193 elif stdin == DEVNULL: 1260 elif stdin == DEVNULL:
1194 p2cread = self._get_devnull() 1261 p2cread = self._get_devnull()
1195 elif isinstance(stdin, int): 1262 elif isinstance(stdin, int):
1196 p2cread = stdin 1263 p2cread = stdin
1197 else: 1264 else:
1198 # Assuming file-like object 1265 # Assuming file-like object
1199 p2cread = stdin.fileno() 1266 p2cread = stdin.fileno()
1200 1267
1201 if stdout is None: 1268 if stdout is None:
1202 pass 1269 pass
1203 elif stdout == PIPE: 1270 elif stdout == PIPE:
1204 c2pread, c2pwrite = _create_pipe() 1271 c2pread, c2pwrite = os.pipe()
1205 elif stdout == DEVNULL: 1272 elif stdout == DEVNULL:
1206 c2pwrite = self._get_devnull() 1273 c2pwrite = self._get_devnull()
1207 elif isinstance(stdout, int): 1274 elif isinstance(stdout, int):
1208 c2pwrite = stdout 1275 c2pwrite = stdout
1209 else: 1276 else:
1210 # Assuming file-like object 1277 # Assuming file-like object
1211 c2pwrite = stdout.fileno() 1278 c2pwrite = stdout.fileno()
1212 1279
1213 if stderr is None: 1280 if stderr is None:
1214 pass 1281 pass
1215 elif stderr == PIPE: 1282 elif stderr == PIPE:
1216 errread, errwrite = _create_pipe() 1283 errread, errwrite = os.pipe()
1217 elif stderr == STDOUT: 1284 elif stderr == STDOUT:
1218 errwrite = c2pwrite 1285 errwrite = c2pwrite
1219 elif stderr == DEVNULL: 1286 elif stderr == DEVNULL:
1220 errwrite = self._get_devnull() 1287 errwrite = self._get_devnull()
1221 elif isinstance(stderr, int): 1288 elif isinstance(stderr, int):
1222 errwrite = stderr 1289 errwrite = stderr
1223 else: 1290 else:
1224 # Assuming file-like object 1291 # Assuming file-like object
1225 errwrite = stderr.fileno() 1292 errwrite = stderr.fileno()
1226 1293
1227 return (p2cread, p2cwrite, 1294 return (p2cread, p2cwrite,
1228 c2pread, c2pwrite, 1295 c2pread, c2pwrite,
1229 errread, errwrite) 1296 errread, errwrite)
1230 1297
1231 1298
1232 def _close_fds(self, fds_to_keep): 1299 def _close_fds(self, fds_to_keep):
1233 start_fd = 3 1300 start_fd = 3
1234 for fd in sorted(fds_to_keep): 1301 for fd in sorted(fds_to_keep):
1235 if fd >= start_fd: 1302 if fd >= start_fd:
1236 os.closerange(start_fd, fd) 1303 os.closerange(start_fd, fd)
1237 start_fd = fd + 1 1304 start_fd = fd + 1
1238 if start_fd <= MAXFD: 1305 if start_fd <= MAXFD:
1239 os.closerange(start_fd, MAXFD) 1306 os.closerange(start_fd, MAXFD)
1240 1307
1241 1308
1242 def _execute_child(self, args, executable, preexec_fn, close_fds, 1309 def _execute_child(self, args, executable, preexec_fn, close_fds,
1243 pass_fds, cwd, env, universal_newlines, 1310 pass_fds, cwd, env,
1244 startupinfo, creationflags, shell, 1311 startupinfo, creationflags, shell,
1245 p2cread, p2cwrite, 1312 p2cread, p2cwrite,
1246 c2pread, c2pwrite, 1313 c2pread, c2pwrite,
1247 errread, errwrite, 1314 errread, errwrite,
1248 restore_signals, start_new_session): 1315 restore_signals, start_new_session):
1249 """Execute program (POSIX version)""" 1316 """Execute program (POSIX version)"""
1250 1317
1251 if isinstance(args, (str, bytes)): 1318 if isinstance(args, (str, bytes)):
1252 args = [args] 1319 args = [args]
1253 else: 1320 else:
1254 args = list(args) 1321 args = list(args)
1255 1322
1256 if shell: 1323 if shell:
1257 args = ["/bin/sh", "-c"] + args 1324 args = ["/bin/sh", "-c"] + args
1258 if executable: 1325 if executable:
1259 args[0] = executable 1326 args[0] = executable
1260 1327
1261 if executable is None: 1328 if executable is None:
1262 executable = args[0] 1329 executable = args[0]
1330 orig_executable = executable
1263 1331
1264 # For transferring possible exec failure from child to parent. 1332 # For transferring possible exec failure from child to parent.
1265 # Data format: "exception name:hex errno:description" 1333 # Data format: "exception name:hex errno:description"
1266 # Pickle is not used; it is complex and involves memory allocation. 1334 # Pickle is not used; it is complex and involves memory allocation.
1267 errpipe_read, errpipe_write = _create_pipe() 1335 errpipe_read, errpipe_write = os.pipe()
1268 try: 1336 try:
1269 try: 1337 try:
1270 1338 # We must avoid complex work that could involve
1271 if _posixsubprocess: 1339 # malloc or free in the child process to avoid
1272 # We must avoid complex work that could involve 1340 # potential deadlocks, thus we do all this here.
1273 # malloc or free in the child process to avoid 1341 # and pass it to fork_exec()
1274 # potential deadlocks, thus we do all this here. 1342
1275 # and pass it to fork_exec() 1343 if env is not None:
1276 1344 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
1277 if env: 1345 for k, v in env.items()]
1278 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
1279 for k, v in env.items()]
1280 else:
1281 env_list = None # Use execv instead of execve.
1282 executable = os.fsencode(executable)
1283 if os.path.dirname(executable):
1284 executable_list = (executable,)
1285 else:
1286 # This matches the behavior of os._execvpe().
1287 executable_list = tuple(
1288 os.path.join(os.fsencode(dir), executable)
1289 for dir in os.get_exec_path(env))
1290 fds_to_keep = set(pass_fds)
1291 fds_to_keep.add(errpipe_write)
1292 self.pid = _posixsubprocess.fork_exec(
1293 args, executable_list,
1294 close_fds, sorted(fds_to_keep), cwd, env_list,
1295 p2cread, p2cwrite, c2pread, c2pwrite,
1296 errread, errwrite,
1297 errpipe_read, errpipe_write,
1298 restore_signals, start_new_session, preexec_fn)
1299 else: 1346 else:
1300 # Pure Python implementation: It is not thread safe. 1347 env_list = None # Use execv instead of execve.
1301 # This implementation may deadlock in the child if your 1348 executable = os.fsencode(executable)
1302 # parent process has any other threads running. 1349 if os.path.dirname(executable):
1303 1350 executable_list = (executable,)
1304 gc_was_enabled = gc.isenabled() 1351 else:
1305 # Disable gc to avoid bug where gc -> file_dealloc -> 1352 # This matches the behavior of os._execvpe().
1306 # write to stderr -> hang. See issue1336 1353 executable_list = tuple(
1307 gc.disable() 1354 os.path.join(os.fsencode(dir), executable)
1308 try: 1355 for dir in os.get_exec_path(env))
1309 self.pid = os.fork() 1356 fds_to_keep = set(pass_fds)
1310 except: 1357 fds_to_keep.add(errpipe_write)
1311 if gc_was_enabled: 1358 self.pid = _posixsubprocess.fork_exec(
1312 gc.enable() 1359 args, executable_list,
1313 raise 1360 close_fds, sorted(fds_to_keep), cwd, env_list,
1314 self._child_created = True 1361 p2cread, p2cwrite, c2pread, c2pwrite,
1315 if self.pid == 0: 1362 errread, errwrite,
1316 # Child 1363 errpipe_read, errpipe_write,
1317 try: 1364 restore_signals, start_new_session, preexec_fn)
1318 # Close parent's pipe ends 1365 self._child_created = True
1319 if p2cwrite != -1:
1320 os.close(p2cwrite)
1321 if c2pread != -1:
1322 os.close(c2pread)
1323 if errread != -1:
1324 os.close(errread)
1325 os.close(errpipe_read)
1326
1327 # Dup fds for child
1328 def _dup2(a, b):
1329 # dup2() removes the CLOEXEC flag but
1330 # we must do it ourselves if dup2()
1331 # would be a no-op (issue #10806).
1332 if a == b:
1333 _set_cloexec(a, False)
1334 elif a != -1:
1335 os.dup2(a, b)
1336 _dup2(p2cread, 0)
1337 _dup2(c2pwrite, 1)
1338 _dup2(errwrite, 2)
1339
1340 # Close pipe fds. Make sure we don't close the
1341 # same fd more than once, or standard fds.
1342 closed = set()
1343 for fd in [p2cread, c2pwrite, errwrite]:
1344 if fd > 2 and fd not in closed:
1345 os.close(fd)
1346 closed.add(fd)
1347
1348 # Close all other fds, if asked for
1349 if close_fds:
1350 fds_to_keep = set(pass_fds)
1351 fds_to_keep.add(errpipe_write)
1352 self._close_fds(fds_to_keep)
1353
1354
1355 if cwd is not None:
1356 os.chdir(cwd)
1357
1358 # This is a copy of Python/pythonrun.c
1359 # _Py_RestoreSignals(). If that were exposed
1360 # as a sys._py_restoresignals func it would be
1361 # better.. but this pure python implementation
1362 # isn't likely to be used much anymore.
1363 if restore_signals:
1364 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1365 for sig in signals:
1366 if hasattr(signal, sig):
1367 signal.signal(getattr(signal, sig),
1368 signal.SIG_DFL)
1369
1370 if start_new_session and hasattr(os, 'setsid'):
1371 os.setsid()
1372
1373 if preexec_fn:
1374 preexec_fn()
1375
1376 if env is None:
1377 os.execvp(executable, args)
1378 else:
1379 os.execvpe(executable, args, env)
1380
1381 except:
1382 try:
1383 exc_type, exc_value = sys.exc_info()[:2]
1384 if isinstance(exc_value, OSError):
1385 errno_num = exc_value.errno
1386 else:
1387 errno_num = 0
1388 message = '%s:%x:%s' % (exc_type.__name__,
1389 errno_num, exc_value )
1390 message = message.encode(errors="surrogatepa ss")
1391 os.write(errpipe_write, message)
1392 except Exception:
1393 # We MUST not allow anything odd happening
1394 # above to prevent us from exiting below.
1395 pass
1396
1397 # This exitcode won't be reported to applications
1398 # so it really doesn't matter what we return.
1399 os._exit(255)
1400
1401 # Parent
1402 if gc_was_enabled:
1403 gc.enable()
1404 finally: 1366 finally:
1405 # be sure the FD is closed no matter what 1367 # be sure the FD is closed no matter what
1406 os.close(errpipe_write) 1368 os.close(errpipe_write)
1407 1369
1408 if p2cread != -1 and p2cwrite != -1: 1370 # self._devnull is not always defined.
1371 devnull_fd = getattr(self, '_devnull', None)
1372 if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
1409 os.close(p2cread) 1373 os.close(p2cread)
1410 if c2pwrite != -1 and c2pread != -1: 1374 if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
1411 os.close(c2pwrite) 1375 os.close(c2pwrite)
1412 if errwrite != -1 and errread != -1: 1376 if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
1413 os.close(errwrite) 1377 os.close(errwrite)
1414 if hasattr(self, '_devnull'): 1378 if devnull_fd is not None:
1415 os.close(self._devnull) 1379 os.close(devnull_fd)
1380 # Prevent a double close of these fds from __init__ on error.
1381 self._closed_child_pipe_fds = True
1416 1382
1417 # Wait for exec to fail or succeed; possibly raising an 1383 # Wait for exec to fail or succeed; possibly raising an
1418 # exception (limited in size) 1384 # exception (limited in size)
1419 data = bytearray() 1385 errpipe_data = bytearray()
1420 while True: 1386 while True:
1421 part = _eintr_retry_call(os.read, errpipe_read, 50000) 1387 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1422 data += part 1388 errpipe_data += part
1423 if not part or len(data) > 50000: 1389 if not part or len(errpipe_data) > 50000:
1424 break 1390 break
1425 finally: 1391 finally:
1426 # be sure the FD is closed no matter what 1392 # be sure the FD is closed no matter what
1427 os.close(errpipe_read) 1393 os.close(errpipe_read)
1428 1394
1429 if data: 1395 if errpipe_data:
1430 try: 1396 try:
1431 _eintr_retry_call(os.waitpid, self.pid, 0) 1397 _eintr_retry_call(os.waitpid, self.pid, 0)
1432 except OSError as e: 1398 except OSError as e:
1433 if e.errno != errno.ECHILD: 1399 if e.errno != errno.ECHILD:
1434 raise 1400 raise
1435 try: 1401 try:
1436 exception_name, hex_errno, err_msg = data.split(b':', 2) 1402 exception_name, hex_errno, err_msg = (
1403 errpipe_data.split(b':', 2))
1437 except ValueError: 1404 except ValueError:
1438 print('Bad exception data:', repr(data)) 1405 exception_name = b'SubprocessError'
1439 exception_name = b'RuntimeError'
1440 hex_errno = b'0' 1406 hex_errno = b'0'
1441 err_msg = b'Unknown' 1407 err_msg = (b'Bad exception data from child: ' +
1408 repr(errpipe_data))
1442 child_exception_type = getattr( 1409 child_exception_type = getattr(
1443 builtins, exception_name.decode('ascii'), 1410 builtins, exception_name.decode('ascii'),
1444 RuntimeError) 1411 SubprocessError)
1445 for fd in (p2cwrite, c2pread, errread):
1446 if fd != -1:
1447 os.close(fd)
1448 err_msg = err_msg.decode(errors="surrogatepass") 1412 err_msg = err_msg.decode(errors="surrogatepass")
1449 if issubclass(child_exception_type, OSError) and hex_errno: 1413 if issubclass(child_exception_type, OSError) and hex_errno:
1450 errno_num = int(hex_errno, 16) 1414 errno_num = int(hex_errno, 16)
1415 child_exec_never_called = (err_msg == "noexec")
1416 if child_exec_never_called:
1417 err_msg = ""
1451 if errno_num != 0: 1418 if errno_num != 0:
1452 err_msg = os.strerror(errno_num) 1419 err_msg = os.strerror(errno_num)
1453 if errno_num == errno.ENOENT: 1420 if errno_num == errno.ENOENT:
1454 err_msg += ': ' + repr(args[0]) 1421 if child_exec_never_called:
1422 # The error must be from chdir(cwd).
1423 err_msg += ': ' + repr(cwd)
1424 else:
1425 err_msg += ': ' + repr(orig_executable)
1455 raise child_exception_type(errno_num, err_msg) 1426 raise child_exception_type(errno_num, err_msg)
1456 raise child_exception_type(err_msg) 1427 raise child_exception_type(err_msg)
1457 1428
1458 1429
1459 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED, 1430 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1460 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED, 1431 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1461 _WEXITSTATUS=os.WEXITSTATUS): 1432 _WEXITSTATUS=os.WEXITSTATUS, _SubprocessError=SubprocessError):
1462 # This method is called (indirectly) by __del__, so it cannot 1433 # This method is called (indirectly) by __del__, so it cannot
1463 # refer to anything outside of its local scope.""" 1434 # refer to anything outside of its local scope."""
1464 if _WIFSIGNALED(sts): 1435 if _WIFSIGNALED(sts):
1465 self.returncode = -_WTERMSIG(sts) 1436 self.returncode = -_WTERMSIG(sts)
1466 elif _WIFEXITED(sts): 1437 elif _WIFEXITED(sts):
1467 self.returncode = _WEXITSTATUS(sts) 1438 self.returncode = _WEXITSTATUS(sts)
1468 else: 1439 else:
1469 # Should never happen 1440 # Should never happen
1470 raise RuntimeError("Unknown child exit status!") 1441 raise _SubprocessError("Unknown child exit status!")
1471 1442
1472 1443
1473 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid, 1444 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1474 _WNOHANG=os.WNOHANG, _os_error=os.error): 1445 _WNOHANG=os.WNOHANG, _ECHILD=errno.ECHILD, _OSError=OSError):
1475 """Check if child process has terminated. Returns returncode 1446 """Check if child process has terminated. Returns returncode
1476 attribute. 1447 attribute.
1477 1448
1478 This method is called by __del__, so it cannot reference anything 1449 This method is called by __del__, so it cannot reference anything
1479 outside of the local scope (nor can any methods it calls). 1450 outside of the local scope (nor can any methods it calls).
1480 1451
1481 """ 1452 """
1482 if self.returncode is None: 1453 if self.returncode is None:
1483 try: 1454 try:
1484 pid, sts = _waitpid(self.pid, _WNOHANG) 1455 pid, sts = _waitpid(self.pid, _WNOHANG)
1485 if pid == self.pid: 1456 if pid == self.pid:
1486 self._handle_exitstatus(sts) 1457 self._handle_exitstatus(sts)
1487 except _os_error: 1458 except _OSError as e:
1488 if _deadstate is not None: 1459 if _deadstate is not None:
1489 self.returncode = _deadstate 1460 self.returncode = _deadstate
1461 elif e.errno == _ECHILD:
1462 # This happens if SIGCLD is set to be ignored or
1463 # waiting for child processes has otherwise been
1464 # disabled for our process. This child is dead, we
1465 # can't get the status.
1466 # http://bugs.python.org/issue15756
1467 self.returncode = 0
1490 return self.returncode 1468 return self.returncode
1491 1469
1492 1470
1493 def _try_wait(self, wait_flags): 1471 def _try_wait(self, wait_flags):
1494 try: 1472 try:
1495 (pid, sts) = _eintr_retry_call(os.waitpid, self.pid, wait_flags) 1473 (pid, sts) = _eintr_retry_call(os.waitpid, self.pid, wait_flags)
1496 except OSError as e: 1474 except OSError as e:
1497 if e.errno != errno.ECHILD: 1475 if e.errno != errno.ECHILD:
1498 raise 1476 raise
1499 # This happens if SIGCLD is set to be ignored or waiting 1477 # This happens if SIGCLD is set to be ignored or waiting
1500 # for child processes has otherwise been disabled for our 1478 # for child processes has otherwise been disabled for our
1501 # process. This child is dead, we can't get the status. 1479 # process. This child is dead, we can't get the status.
1502 pid = self.pid 1480 pid = self.pid
1503 sts = 0 1481 sts = 0
1504 return (pid, sts) 1482 return (pid, sts)
1505 1483
1506 1484
1507 def wait(self, timeout=None, endtime=None): 1485 def wait(self, timeout=None, endtime=None):
1508 """Wait for child process to terminate. Returns returncode 1486 """Wait for child process to terminate. Returns returncode
1509 attribute.""" 1487 attribute."""
1510 if self.returncode is not None: 1488 if self.returncode is not None:
1511 return self.returncode 1489 return self.returncode
1512 1490
1513 # endtime is preferred to timeout. timeout is only used for 1491 # endtime is preferred to timeout. timeout is only used for
1514 # printing. 1492 # printing.
1515 if endtime is not None or timeout is not None: 1493 if endtime is not None or timeout is not None:
1516 if endtime is None: 1494 if endtime is None:
1517 endtime = time.time() + timeout 1495 endtime = _time() + timeout
1518 elif timeout is None: 1496 elif timeout is None:
1519 timeout = self._remaining_time(endtime) 1497 timeout = self._remaining_time(endtime)
1520 1498
1521 if endtime is not None: 1499 if endtime is not None:
1522 # Enter a busy loop if we have a timeout. This busy loop was 1500 # Enter a busy loop if we have a timeout. This busy loop was
1523 # cribbed from Lib/threading.py in Thread.wait() at r71065. 1501 # cribbed from Lib/threading.py in Thread.wait() at r71065.
1524 delay = 0.0005 # 500 us -> initial delay of 1 ms 1502 delay = 0.0005 # 500 us -> initial delay of 1 ms
1525 while True: 1503 while True:
1526 (pid, sts) = self._try_wait(os.WNOHANG) 1504 (pid, sts) = self._try_wait(os.WNOHANG)
1527 assert pid == self.pid or pid == 0 1505 assert pid == self.pid or pid == 0
1528 if pid == self.pid: 1506 if pid == self.pid:
1529 self._handle_exitstatus(sts) 1507 self._handle_exitstatus(sts)
1530 break 1508 break
1531 remaining = self._remaining_time(endtime) 1509 remaining = self._remaining_time(endtime)
1532 if remaining <= 0: 1510 if remaining <= 0:
1533 raise TimeoutExpired(self.args, timeout) 1511 raise TimeoutExpired(self.args, timeout)
1534 delay = min(delay * 2, remaining, .05) 1512 delay = min(delay * 2, remaining, .05)
1535 time.sleep(delay) 1513 time.sleep(delay)
1536 elif self.returncode is None: 1514 else:
1537 (pid, sts) = self._try_wait(0) 1515 while self.returncode is None:
1538 self._handle_exitstatus(sts) 1516 (pid, sts) = self._try_wait(0)
1517 # Check the pid and loop as waitpid has been known to return
1518 # 0 even without WNOHANG in odd situations. issue14396.
1519 if pid == self.pid:
1520 self._handle_exitstatus(sts)
1539 return self.returncode 1521 return self.returncode
1540 1522
1541 1523
1542 def _communicate(self, input, endtime, orig_timeout): 1524 def _communicate(self, input, endtime, orig_timeout):
1543 if self.stdin and not self._communication_started: 1525 if self.stdin and not self._communication_started:
1544 # Flush stdio buffer. This might block, if the user has 1526 # Flush stdio buffer. This might block, if the user has
1545 # been writing to .stdin in an uncontrolled fashion. 1527 # been writing to .stdin in an uncontrolled fashion.
1546 self.stdin.flush() 1528 self.stdin.flush()
1547 if not input: 1529 if not input:
1548 self.stdin.close() 1530 self.stdin.close()
(...skipping 19 matching lines...) Expand all
1568 if stdout is not None: 1550 if stdout is not None:
1569 stdout = self._translate_newlines(stdout, 1551 stdout = self._translate_newlines(stdout,
1570 self.stdout.encoding) 1552 self.stdout.encoding)
1571 if stderr is not None: 1553 if stderr is not None:
1572 stderr = self._translate_newlines(stderr, 1554 stderr = self._translate_newlines(stderr,
1573 self.stderr.encoding) 1555 self.stderr.encoding)
1574 1556
1575 return (stdout, stderr) 1557 return (stdout, stderr)
1576 1558
1577 1559
1560 def _save_input(self, input):
1561 # This method is called from the _communicate_with_*() methods
1562 # so that if we time out while communicating, we can continue
1563 # sending input if we retry.
1564 if self.stdin and self._input is None:
1565 self._input_offset = 0
1566 self._input = input
1567 if self.universal_newlines and input is not None:
1568 self._input = self._input.encode(self.stdin.encoding)
1569
1570
1578 def _communicate_with_poll(self, input, endtime, orig_timeout): 1571 def _communicate_with_poll(self, input, endtime, orig_timeout):
1579 stdout = None # Return 1572 stdout = None # Return
1580 stderr = None # Return 1573 stderr = None # Return
1581 1574
1582 if not self._communication_started: 1575 if not self._communication_started:
1583 self._fd2file = {} 1576 self._fd2file = {}
1584 1577
1585 poller = select.poll() 1578 poller = select.poll()
1586 def register_and_append(file_obj, eventmask): 1579 def register_and_append(file_obj, eventmask):
1587 poller.register(file_obj.fileno(), eventmask) 1580 poller.register(file_obj.fileno(), eventmask)
(...skipping 16 matching lines...) Expand all
1604 self._fd2output[self.stderr.fileno()] = [] 1597 self._fd2output[self.stderr.fileno()] = []
1605 1598
1606 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI 1599 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1607 if self.stdout: 1600 if self.stdout:
1608 register_and_append(self.stdout, select_POLLIN_POLLPRI) 1601 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1609 stdout = self._fd2output[self.stdout.fileno()] 1602 stdout = self._fd2output[self.stdout.fileno()]
1610 if self.stderr: 1603 if self.stderr:
1611 register_and_append(self.stderr, select_POLLIN_POLLPRI) 1604 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1612 stderr = self._fd2output[self.stderr.fileno()] 1605 stderr = self._fd2output[self.stderr.fileno()]
1613 1606
1614 # Save the input here so that if we time out while communicating, 1607 self._save_input(input)
1615 # we can continue sending input if we retry.
1616 if self.stdin and self._input is None:
1617 self._input_offset = 0
1618 self._input = input
1619 if self.universal_newlines:
1620 self._input = self._input.encode(self.stdin.encoding)
1621 1608
1622 while self._fd2file: 1609 while self._fd2file:
1623 timeout = self._remaining_time(endtime) 1610 timeout = self._remaining_time(endtime)
1624 if timeout is not None and timeout < 0: 1611 if timeout is not None and timeout < 0:
1625 raise TimeoutExpired(self.args, orig_timeout) 1612 raise TimeoutExpired(self.args, orig_timeout)
1626 try: 1613 try:
1627 ready = poller.poll(timeout) 1614 ready = poller.poll(timeout)
1628 except select.error as e: 1615 except OSError as e:
1629 if e.args[0] == errno.EINTR: 1616 if e.args[0] == errno.EINTR:
1630 continue 1617 continue
1631 raise 1618 raise
1632 self._check_timeout(endtime, orig_timeout) 1619 self._check_timeout(endtime, orig_timeout)
1633 1620
1634 # XXX Rewrite these to use non-blocking I/O on the 1621 # XXX Rewrite these to use non-blocking I/O on the
1635 # file objects; they are no longer using C stdio! 1622 # file objects; they are no longer using C stdio!
1636 1623
1637 for fd, mode in ready: 1624 for fd, mode in ready:
1638 if mode & select.POLLOUT: 1625 if mode & select.POLLOUT:
(...skipping 25 matching lines...) Expand all
1664 if not self._communication_started: 1651 if not self._communication_started:
1665 self._read_set = [] 1652 self._read_set = []
1666 self._write_set = [] 1653 self._write_set = []
1667 if self.stdin and input: 1654 if self.stdin and input:
1668 self._write_set.append(self.stdin) 1655 self._write_set.append(self.stdin)
1669 if self.stdout: 1656 if self.stdout:
1670 self._read_set.append(self.stdout) 1657 self._read_set.append(self.stdout)
1671 if self.stderr: 1658 if self.stderr:
1672 self._read_set.append(self.stderr) 1659 self._read_set.append(self.stderr)
1673 1660
1674 if self.stdin and self._input is None: 1661 self._save_input(input)
1675 self._input_offset = 0
1676 self._input = input
1677 if self.universal_newlines:
1678 self._input = self._input.encode(self.stdin.encoding)
1679 1662
1680 stdout = None # Return 1663 stdout = None # Return
1681 stderr = None # Return 1664 stderr = None # Return
1682 1665
1683 if self.stdout: 1666 if self.stdout:
1684 if not self._communication_started: 1667 if not self._communication_started:
1685 self._stdout_buff = [] 1668 self._stdout_buff = []
1686 stdout = self._stdout_buff 1669 stdout = self._stdout_buff
1687 if self.stderr: 1670 if self.stderr:
1688 if not self._communication_started: 1671 if not self._communication_started:
1689 self._stderr_buff = [] 1672 self._stderr_buff = []
1690 stderr = self._stderr_buff 1673 stderr = self._stderr_buff
1691 1674
1692 while self._read_set or self._write_set: 1675 while self._read_set or self._write_set:
1693 timeout = self._remaining_time(endtime) 1676 timeout = self._remaining_time(endtime)
1694 if timeout is not None and timeout < 0: 1677 if timeout is not None and timeout < 0:
1695 raise TimeoutExpired(self.args, orig_timeout) 1678 raise TimeoutExpired(self.args, orig_timeout)
1696 try: 1679 try:
1697 (rlist, wlist, xlist) = \ 1680 (rlist, wlist, xlist) = \
1698 select.select(self._read_set, self._write_set, [], 1681 select.select(self._read_set, self._write_set, [],
1699 timeout) 1682 timeout)
1700 except select.error as e: 1683 except OSError as e:
1701 if e.args[0] == errno.EINTR: 1684 if e.args[0] == errno.EINTR:
1702 continue 1685 continue
1703 raise 1686 raise
1704 1687
1705 # According to the docs, returning three empty lists indicates 1688 # According to the docs, returning three empty lists indicates
1706 # that the timeout expired. 1689 # that the timeout expired.
1707 if not (rlist or wlist or xlist): 1690 if not (rlist or wlist or xlist):
1708 raise TimeoutExpired(self.args, orig_timeout) 1691 raise TimeoutExpired(self.args, orig_timeout)
1709 # We also check what time it is ourselves for good measure. 1692 # We also check what time it is ourselves for good measure.
1710 self._check_timeout(endtime, orig_timeout) 1693 self._check_timeout(endtime, orig_timeout)
(...skipping 42 matching lines...) Expand 10 before | Expand all | Expand 10 after
1753 1736
1754 def terminate(self): 1737 def terminate(self):
1755 """Terminate the process with SIGTERM 1738 """Terminate the process with SIGTERM
1756 """ 1739 """
1757 self.send_signal(signal.SIGTERM) 1740 self.send_signal(signal.SIGTERM)
1758 1741
1759 def kill(self): 1742 def kill(self):
1760 """Kill the process with SIGKILL 1743 """Kill the process with SIGKILL
1761 """ 1744 """
1762 self.send_signal(signal.SIGKILL) 1745 self.send_signal(signal.SIGKILL)
LEFTRIGHT
« no previous file | no next file » | Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Toggle Comments ('s')

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7