Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(500)

Delta Between Two Patch Sets: Lib/subprocess.py

Issue 12085: subprocess.Popen.__del__ raises AttributeError if __init__ was called with an invalid argument list
Left Patch Set: Created 6 years, 5 months ago
Right Patch Set: Created 4 years, 1 month ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
Left: Side by side diff | Download
Right: Side by side diff | Download
« no previous file with change/comment | « no previous file | no next file » | no next file with change/comment »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
LEFTRIGHT
1 # subprocess - Subprocesses with accessible I/O streams 1 # subprocess - Subprocesses with accessible I/O streams
2 # 2 #
3 # For more information about this module, see PEP 324. 3 # For more information about this module, see PEP 324.
4 # 4 #
5 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se> 5 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
6 # 6 #
7 # Licensed to PSF under a Contributor Agreement. 7 # Licensed to PSF under a Contributor Agreement.
8 # See http://www.python.org/2.4/license for licensing details. 8 # See http://www.python.org/2.4/license for licensing details.
9 9
10 r"""subprocess - Subprocesses with accessible I/O streams 10 r"""subprocess - Subprocesses with accessible I/O streams
11 11
12 This module allows you to spawn processes, connect to their 12 This module allows you to spawn processes, connect to their
13 input/output/error pipes, and obtain their return codes. This module 13 input/output/error pipes, and obtain their return codes. This module
14 intends to replace several other, older modules and functions, like: 14 intends to replace several other, older modules and functions, like:
15 15
16 os.system 16 os.system
17 os.spawn* 17 os.spawn*
18 18
19 Information about how the subprocess module can be used to replace these 19 Information about how the subprocess module can be used to replace these
20 modules and functions can be found below. 20 modules and functions can be found below.
21 21
22 22
23 23
24 Using the subprocess module 24 Using the subprocess module
25 =========================== 25 ===========================
26 This module defines one class called Popen: 26 This module defines one class called Popen:
27 27
28 class Popen(args, bufsize=0, executable=None, 28 class Popen(args, bufsize=-1, executable=None,
29 stdin=None, stdout=None, stderr=None, 29 stdin=None, stdout=None, stderr=None,
30 preexec_fn=None, close_fds=True, shell=False, 30 preexec_fn=None, close_fds=True, shell=False,
31 cwd=None, env=None, universal_newlines=False, 31 cwd=None, env=None, universal_newlines=False,
32 startupinfo=None, creationflags=0, 32 startupinfo=None, creationflags=0,
33 restore_signals=True, start_new_session=False, pass_fds=()): 33 restore_signals=True, start_new_session=False, pass_fds=()):
34 34
35 35
36 Arguments are: 36 Arguments are:
37 37
38 args should be a string, or a sequence of program arguments. The 38 args should be a string, or a sequence of program arguments. The
(...skipping 10 matching lines...) Expand all
49 the first item specifies the command string, and any additional items 49 the first item specifies the command string, and any additional items
50 will be treated as additional shell arguments. 50 will be treated as additional shell arguments.
51 51
52 On Windows: the Popen class uses CreateProcess() to execute the child 52 On Windows: the Popen class uses CreateProcess() to execute the child
53 program, which operates on strings. If args is a sequence, it will be 53 program, which operates on strings. If args is a sequence, it will be
54 converted to a string using the list2cmdline method. Please note that 54 converted to a string using the list2cmdline method. Please note that
55 not all MS Windows applications interpret the command line the same 55 not all MS Windows applications interpret the command line the same
56 way: The list2cmdline is designed for applications using the same 56 way: The list2cmdline is designed for applications using the same
57 rules as the MS C runtime. 57 rules as the MS C runtime.
58 58
59 bufsize, if given, has the same meaning as the corresponding argument 59 bufsize will be supplied as the corresponding argument to the io.open()
60 to the built-in open() function: 0 means unbuffered, 1 means line 60 function when creating the stdin/stdout/stderr pipe file objects:
61 buffered, any other positive value means use a buffer of 61 0 means unbuffered (read & write are one system call and can return short),
62 (approximately) that size. A negative bufsize means to use the system 62 1 means line buffered, any other positive value means use a buffer of
63 default, which usually means fully buffered. The default value for 63 approximately that size. A negative bufsize, the default, means the system
64 bufsize is 0 (unbuffered). 64 default of io.DEFAULT_BUFFER_SIZE will be used.
65 65
66 stdin, stdout and stderr specify the executed programs' standard 66 stdin, stdout and stderr specify the executed programs' standard
67 input, standard output and standard error file handles, respectively. 67 input, standard output and standard error file handles, respectively.
68 Valid values are PIPE, an existing file descriptor (a positive 68 Valid values are PIPE, an existing file descriptor (a positive
69 integer), an existing file object, and None. PIPE indicates that a 69 integer), an existing file object, and None. PIPE indicates that a
70 new pipe to the child should be created. With None, no redirection 70 new pipe to the child should be created. With None, no redirection
71 will occur; the child's file handles will be inherited from the 71 will occur; the child's file handles will be inherited from the
72 parent. Additionally, stderr can be STDOUT, which indicates that the 72 parent. Additionally, stderr can be STDOUT, which indicates that the
73 stderr data from the applications should be captured into the same 73 stderr data from the applications should be captured into the same
74 file handle as for stdout. 74 file handle as for stdout.
(...skipping 22 matching lines...) Expand all
97 SIG_IGN are restored to SIG_DFL in the child process before the exec. 97 SIG_IGN are restored to SIG_DFL in the child process before the exec.
98 Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This 98 Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99 parameter does nothing on Windows. 99 parameter does nothing on Windows.
100 100
101 On POSIX, if start_new_session is True, the setsid() system call will be made 101 On POSIX, if start_new_session is True, the setsid() system call will be made
102 in the child process prior to executing the command. 102 in the child process prior to executing the command.
103 103
104 If env is not None, it defines the environment variables for the new 104 If env is not None, it defines the environment variables for the new
105 process. 105 process.
106 106
107 If universal_newlines is false, the file objects stdin, stdout and stderr
108 are opened as binary files, and no line ending conversion is done.
109
107 If universal_newlines is true, the file objects stdout and stderr are 110 If universal_newlines is true, the file objects stdout and stderr are
108 opened as a text files, but lines may be terminated by any of '\n', 111 opened as a text files, but lines may be terminated by any of '\n',
109 the Unix end-of-line convention, '\r', the old Macintosh convention or 112 the Unix end-of-line convention, '\r', the old Macintosh convention or
110 '\r\n', the Windows convention. All of these external representations 113 '\r\n', the Windows convention. All of these external representations
111 are seen as '\n' by the Python program. Note: This feature is only 114 are seen as '\n' by the Python program. Also, the newlines attribute
112 available if Python is built with universal newline support (the 115 of the file objects stdout, stdin and stderr are not updated by the
113 default). Also, the newlines attribute of the file objects stdout, 116 communicate() method.
114 stdin and stderr are not updated by the communicate() method.
115 117
116 The startupinfo and creationflags, if given, will be passed to the 118 The startupinfo and creationflags, if given, will be passed to the
117 underlying CreateProcess() function. They can specify things such as 119 underlying CreateProcess() function. They can specify things such as
118 appearance of the main window and priority for the new process. 120 appearance of the main window and priority for the new process.
119 (Windows only) 121 (Windows only)
120 122
121 123
122 This module also defines some shortcut functions: 124 This module also defines some shortcut functions:
123 125
124 call(*popenargs, **kwargs): 126 call(*popenargs, **kwargs):
(...skipping 34 matching lines...) Expand 10 before | Expand all | Expand 10 after
159 getoutput(cmd): 161 getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell. 162 Return output (stdout or stderr) of executing cmd in a shell.
161 163
162 Like getstatusoutput(), except the exit status is ignored and the return 164 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example: 165 value is a string containing the command's output. Example:
164 166
165 >>> subprocess.getoutput('ls /bin/ls') 167 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls' 168 '/bin/ls'
167 169
168 check_output(*popenargs, **kwargs): 170 check_output(*popenargs, **kwargs):
169 Run command with arguments and return its output as a byte string. 171 Run command with arguments and return its output.
170 172
171 If the exit code was non-zero it raises a CalledProcessError. The 173 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode 174 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute. 175 attribute and output in the output attribute.
174 176
175 The arguments are the same as for the Popen constructor. Example: 177 The arguments are the same as for the Popen constructor. Example:
176 178
177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"]) 179 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
178 180
181 There is an additional optional argument, "input", allowing you to
182 pass a string to the subprocess's stdin. If you use this argument
183 you may not also use the Popen constructor's "stdin" argument.
179 184
180 Exceptions 185 Exceptions
181 ---------- 186 ----------
182 Exceptions raised in the child process, before the new program has 187 Exceptions raised in the child process, before the new program has
183 started to execute, will be re-raised in the parent. Additionally, 188 started to execute, will be re-raised in the parent. Additionally,
184 the exception object will have one extra attribute called 189 the exception object will have one extra attribute called
185 'child_traceback', which is a string containing traceback information 190 'child_traceback', which is a string containing traceback information
186 from the childs point of view. 191 from the child's point of view.
187 192
188 The most common exception raised is OSError. This occurs, for 193 The most common exception raised is OSError. This occurs, for
189 example, when trying to execute a non-existent file. Applications 194 example, when trying to execute a non-existent file. Applications
190 should prepare for OSErrors. 195 should prepare for OSErrors.
191 196
192 A ValueError will be raised if Popen is called with invalid arguments. 197 A ValueError will be raised if Popen is called with invalid arguments.
193 198
194 Exceptions defined within this module inherit from SubprocessError. 199 Exceptions defined within this module inherit from SubprocessError.
195 check_call() and check_output() will raise CalledProcessError if the 200 check_call() and check_output() will raise CalledProcessError if the
196 called process returns a non-zero return code. TimeoutExpired 201 called process returns a non-zero return code. TimeoutExpired
(...skipping 145 matching lines...) Expand 10 before | Expand all | Expand 10 after
342 347
343 import io 348 import io
344 import os 349 import os
345 import time 350 import time
346 import traceback 351 import traceback
347 import gc 352 import gc
348 import signal 353 import signal
349 import builtins 354 import builtins
350 import warnings 355 import warnings
351 import errno 356 import errno
357 try:
358 from time import monotonic as _time
359 except ImportError:
360 from time import time as _time
352 361
353 # Exception classes used by this module. 362 # Exception classes used by this module.
354 class SubprocessError(Exception): pass 363 class SubprocessError(Exception): pass
355 364
356 365
357 class CalledProcessError(SubprocessError): 366 class CalledProcessError(SubprocessError):
358 """This exception is raised when a process run by check_call() or 367 """This exception is raised when a process run by check_call() or
359 check_output() returns a non-zero exit status. 368 check_output() returns a non-zero exit status.
360 The exit status will be stored in the returncode attribute; 369 The exit status will be stored in the returncode attribute;
361 check_output() will also store the output in the output attribute. 370 check_output() will also store the output in the output attribute.
(...skipping 16 matching lines...) Expand all
378 self.output = output 387 self.output = output
379 388
380 def __str__(self): 389 def __str__(self):
381 return ("Command '%s' timed out after %s seconds" % 390 return ("Command '%s' timed out after %s seconds" %
382 (self.cmd, self.timeout)) 391 (self.cmd, self.timeout))
383 392
384 393
385 if mswindows: 394 if mswindows:
386 import threading 395 import threading
387 import msvcrt 396 import msvcrt
388 import _subprocess 397 import _winapi
389 class STARTUPINFO: 398 class STARTUPINFO:
390 dwFlags = 0 399 dwFlags = 0
391 hStdInput = None 400 hStdInput = None
392 hStdOutput = None 401 hStdOutput = None
393 hStdError = None 402 hStdError = None
394 wShowWindow = 0 403 wShowWindow = 0
395 class pywintypes:
396 error = IOError
397 else: 404 else:
398 import select 405 import select
399 _has_poll = hasattr(select, 'poll') 406 _has_poll = hasattr(select, 'poll')
400 import fcntl 407 import _posixsubprocess
401 import pickle
402
403 try:
404 import _posixsubprocess
405 except ImportError:
406 _posixsubprocess = None
407 warnings.warn("The _posixsubprocess module is not being used. "
408 "Child process reliability may suffer if your "
409 "program uses threads.", RuntimeWarning)
410 408
411 # When select or poll has indicated that the file is writable, 409 # When select or poll has indicated that the file is writable,
412 # we can write up to _PIPE_BUF bytes without risk of blocking. 410 # we can write up to _PIPE_BUF bytes without risk of blocking.
413 # POSIX defines PIPE_BUF as >= 512. 411 # POSIX defines PIPE_BUF as >= 512.
414 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512) 412 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
415 413
416 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
417
418 def _set_cloexec(fd, cloexec):
419 old = fcntl.fcntl(fd, fcntl.F_GETFD)
420 if cloexec:
421 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
422 else:
423 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
424
425 if _posixsubprocess:
426 _create_pipe = _posixsubprocess.cloexec_pipe
427 else:
428 def _create_pipe():
429 fds = os.pipe()
430 _set_cloexec(fds[0], True)
431 _set_cloexec(fds[1], True)
432 return fds
433 414
434 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput", 415 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
435 "getoutput", "check_output", "CalledProcessError", "DEVNULL"] 416 "getoutput", "check_output", "CalledProcessError", "DEVNULL"]
436 417
437 if mswindows: 418 if mswindows:
438 from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP, 419 from _winapi import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
439 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, 420 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
440 STD_ERROR_HANDLE, SW_HIDE, 421 STD_ERROR_HANDLE, SW_HIDE,
441 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW) 422 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
442 423
443 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP", 424 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
444 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE", 425 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
445 "STD_ERROR_HANDLE", "SW_HIDE", 426 "STD_ERROR_HANDLE", "SW_HIDE",
446 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"]) 427 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
428
429 class Handle(int):
430 closed = False
431
432 def Close(self, CloseHandle=_winapi.CloseHandle):
433 if not self.closed:
434 self.closed = True
435 CloseHandle(self)
436
437 def Detach(self):
438 if not self.closed:
439 self.closed = True
440 return int(self)
441 raise ValueError("already closed")
442
443 def __repr__(self):
444 return "Handle(%d)" % int(self)
445
446 __del__ = Close
447 __str__ = __repr__
448
447 try: 449 try:
448 MAXFD = os.sysconf("SC_OPEN_MAX") 450 MAXFD = os.sysconf("SC_OPEN_MAX")
449 except: 451 except:
450 MAXFD = 256 452 MAXFD = 256
451 453
454 # This lists holds Popen instances for which the underlying process had not
455 # exited at the time its __del__ method got called: those processes are wait()ed
456 # for synchronously from _cleanup() when a new Popen object is created, to avoid
457 # zombie processes.
452 _active = [] 458 _active = []
453 459
454 def _cleanup(): 460 def _cleanup():
455 for inst in _active[:]: 461 for inst in _active[:]:
456 res = inst._internal_poll(_deadstate=sys.maxsize) 462 res = inst._internal_poll(_deadstate=sys.maxsize)
457 if res is not None and res >= 0: 463 if res is not None:
458 try: 464 try:
459 _active.remove(inst) 465 _active.remove(inst)
460 except ValueError: 466 except ValueError:
461 # This can happen if two threads create a new Popen instance. 467 # This can happen if two threads create a new Popen instance.
462 # It's harmless that it was already removed, so ignore. 468 # It's harmless that it was already removed, so ignore.
463 pass 469 pass
464 470
465 PIPE = -1 471 PIPE = -1
466 STDOUT = -2 472 STDOUT = -2
467 DEVNULL = -3 473 DEVNULL = -3
468 474
469 475
470 def _eintr_retry_call(func, *args): 476 def _eintr_retry_call(func, *args):
471 while True: 477 while True:
472 try: 478 try:
473 return func(*args) 479 return func(*args)
474 except OSError as e: 480 except InterruptedError:
475 if e.errno == errno.EINTR: 481 continue
476 continue 482
477 raise 483
484 # XXX This function is only used by multiprocessing and the test suite,
485 # but it's here so that it can be imported when Python is compiled without
486 # threads.
487
488 def _args_from_interpreter_flags():
489 """Return a list of command-line arguments reproducing the current
490 settings in sys.flags and sys.warnoptions."""
491 flag_opt_map = {
492 'debug': 'd',
493 # 'inspect': 'i',
494 # 'interactive': 'i',
495 'optimize': 'O',
496 'dont_write_bytecode': 'B',
497 'no_user_site': 's',
498 'no_site': 'S',
499 'ignore_environment': 'E',
500 'verbose': 'v',
501 'bytes_warning': 'b',
502 'quiet': 'q',
503 'hash_randomization': 'R',
504 }
505 args = []
506 for flag, opt in flag_opt_map.items():
507 v = getattr(sys.flags, flag)
508 if v > 0:
509 args.append('-' + opt * v)
510 for opt in sys.warnoptions:
511 args.append('-W' + opt)
512 return args
478 513
479 514
480 def call(*popenargs, timeout=None, **kwargs): 515 def call(*popenargs, timeout=None, **kwargs):
481 """Run command with arguments. Wait for command to complete or 516 """Run command with arguments. Wait for command to complete or
482 timeout, then return the returncode attribute. 517 timeout, then return the returncode attribute.
483 518
484 The arguments are the same as for the Popen constructor. Example: 519 The arguments are the same as for the Popen constructor. Example:
485 520
486 retcode = call(["ls", "-l"]) 521 retcode = call(["ls", "-l"])
487 """ 522 """
488 p = Popen(*popenargs, **kwargs) 523 with Popen(*popenargs, **kwargs) as p:
489 try: 524 try:
490 return p.wait(timeout=timeout) 525 return p.wait(timeout=timeout)
491 except TimeoutExpired: 526 except:
492 p.kill() 527 p.kill()
493 p.wait() 528 p.wait()
494 raise 529 raise
495 530
496 531
497 def check_call(*popenargs, **kwargs): 532 def check_call(*popenargs, **kwargs):
498 """Run command with arguments. Wait for command to complete. If 533 """Run command with arguments. Wait for command to complete. If
499 the exit code was zero then return, otherwise raise 534 the exit code was zero then return, otherwise raise
500 CalledProcessError. The CalledProcessError object will have the 535 CalledProcessError. The CalledProcessError object will have the
501 return code in the returncode attribute. 536 return code in the returncode attribute.
502 537
503 The arguments are the same as for the call function. Example: 538 The arguments are the same as for the call function. Example:
504 539
505 check_call(["ls", "-l"]) 540 check_call(["ls", "-l"])
506 """ 541 """
507 retcode = call(*popenargs, **kwargs) 542 retcode = call(*popenargs, **kwargs)
508 if retcode: 543 if retcode:
509 cmd = kwargs.get("args") 544 cmd = kwargs.get("args")
510 if cmd is None: 545 if cmd is None:
511 cmd = popenargs[0] 546 cmd = popenargs[0]
512 raise CalledProcessError(retcode, cmd) 547 raise CalledProcessError(retcode, cmd)
513 return 0 548 return 0
514 549
515 550
516 def check_output(*popenargs, timeout=None, **kwargs): 551 def check_output(*popenargs, timeout=None, **kwargs):
517 r"""Run command with arguments and return its output as a byte string. 552 r"""Run command with arguments and return its output.
518 553
519 If the exit code was non-zero it raises a CalledProcessError. The 554 If the exit code was non-zero it raises a CalledProcessError. The
520 CalledProcessError object will have the return code in the returncode 555 CalledProcessError object will have the return code in the returncode
521 attribute and output in the output attribute. 556 attribute and output in the output attribute.
522 557
523 The arguments are the same as for the Popen constructor. Example: 558 The arguments are the same as for the Popen constructor. Example:
524 559
525 >>> check_output(["ls", "-l", "/dev/null"]) 560 >>> check_output(["ls", "-l", "/dev/null"])
526 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n' 561 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
527 562
528 The stdout argument is not allowed as it is used internally. 563 The stdout argument is not allowed as it is used internally.
529 To capture standard error in the result, use stderr=STDOUT. 564 To capture standard error in the result, use stderr=STDOUT.
530 565
531 >>> check_output(["/bin/sh", "-c", 566 >>> check_output(["/bin/sh", "-c",
532 ... "ls -l non_existent_file ; exit 0"], 567 ... "ls -l non_existent_file ; exit 0"],
533 ... stderr=STDOUT) 568 ... stderr=STDOUT)
534 b'ls: non_existent_file: No such file or directory\n' 569 b'ls: non_existent_file: No such file or directory\n'
570
571 There is an additional optional argument, "input", allowing you to
572 pass a string to the subprocess's stdin. If you use this argument
573 you may not also use the Popen constructor's "stdin" argument, as
574 it too will be used internally. Example:
575
576 >>> check_output(["sed", "-e", "s/foo/bar/"],
577 ... input=b"when in the course of fooman events\n")
578 b'when in the course of barman events\n'
579
580 If universal_newlines=True is passed, the return value will be a
581 string rather than bytes.
535 """ 582 """
536 if 'stdout' in kwargs: 583 if 'stdout' in kwargs:
537 raise ValueError('stdout argument not allowed, it will be overridden.') 584 raise ValueError('stdout argument not allowed, it will be overridden.')
538 process = Popen(*popenargs, stdout=PIPE, **kwargs) 585 if 'input' in kwargs:
539 try: 586 if 'stdin' in kwargs:
540 output, unused_err = process.communicate(timeout=timeout) 587 raise ValueError('stdin and input arguments may not both be used.')
541 except TimeoutExpired: 588 inputdata = kwargs['input']
542 process.kill() 589 del kwargs['input']
543 output, unused_err = process.communicate() 590 kwargs['stdin'] = PIPE
544 raise TimeoutExpired(process.args, timeout, output=output) 591 else:
545 retcode = process.poll() 592 inputdata = None
546 if retcode: 593 with Popen(*popenargs, stdout=PIPE, **kwargs) as process:
547 raise CalledProcessError(retcode, process.args, output=output) 594 try:
595 output, unused_err = process.communicate(inputdata, timeout=timeout)
596 except TimeoutExpired:
597 process.kill()
598 output, unused_err = process.communicate()
599 raise TimeoutExpired(process.args, timeout, output=output)
600 except:
601 process.kill()
602 process.wait()
603 raise
604 retcode = process.poll()
605 if retcode:
606 raise CalledProcessError(retcode, process.args, output=output)
548 return output 607 return output
549 608
550 609
551 def list2cmdline(seq): 610 def list2cmdline(seq):
552 """ 611 """
553 Translate a sequence of arguments into a command line 612 Translate a sequence of arguments into a command line
554 string, using the same rules as the MS C runtime: 613 string, using the same rules as the MS C runtime:
555 614
556 1) Arguments are delimited by white space, which is either a 615 1) Arguments are delimited by white space, which is either a
557 space or a tab. 616 space or a tab.
(...skipping 74 matching lines...) Expand 10 before | Expand all | Expand 10 after
632 interpreted according to the rules for the C function wait(). Example: 691 interpreted according to the rules for the C function wait(). Example:
633 692
634 >>> import subprocess 693 >>> import subprocess
635 >>> subprocess.getstatusoutput('ls /bin/ls') 694 >>> subprocess.getstatusoutput('ls /bin/ls')
636 (0, '/bin/ls') 695 (0, '/bin/ls')
637 >>> subprocess.getstatusoutput('cat /bin/junk') 696 >>> subprocess.getstatusoutput('cat /bin/junk')
638 (256, 'cat: /bin/junk: No such file or directory') 697 (256, 'cat: /bin/junk: No such file or directory')
639 >>> subprocess.getstatusoutput('/bin/junk') 698 >>> subprocess.getstatusoutput('/bin/junk')
640 (256, 'sh: /bin/junk: not found') 699 (256, 'sh: /bin/junk: not found')
641 """ 700 """
642 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r') 701 with os.popen('{ ' + cmd + '; } 2>&1', 'r') as pipe:
643 text = pipe.read() 702 try:
644 sts = pipe.close() 703 text = pipe.read()
645 if sts is None: sts = 0 704 sts = pipe.close()
646 if text[-1:] == '\n': text = text[:-1] 705 except:
706 process = pipe._proc
707 process.kill()
708 process.wait()
709 raise
710 if sts is None:
711 sts = 0
712 if text[-1:] == '\n':
713 text = text[:-1]
647 return sts, text 714 return sts, text
648 715
649 716
650 def getoutput(cmd): 717 def getoutput(cmd):
651 """Return output (stdout or stderr) of executing cmd in a shell. 718 """Return output (stdout or stderr) of executing cmd in a shell.
652 719
653 Like getstatusoutput(), except the exit status is ignored and the return 720 Like getstatusoutput(), except the exit status is ignored and the return
654 value is a string containing the command's output. Example: 721 value is a string containing the command's output. Example:
655 722
656 >>> import subprocess 723 >>> import subprocess
657 >>> subprocess.getoutput('ls /bin/ls') 724 >>> subprocess.getoutput('ls /bin/ls')
658 '/bin/ls' 725 '/bin/ls'
659 """ 726 """
660 return getstatusoutput(cmd)[1] 727 return getstatusoutput(cmd)[1]
661 728
662 729
663 _PLATFORM_DEFAULT_CLOSE_FDS = object() 730 _PLATFORM_DEFAULT_CLOSE_FDS = object()
664 731
665 732
666 class Popen(object): 733 class Popen(object):
667 def __init__(self, args, bufsize=0, executable=None, 734
735 _child_created = False # Set here since __del__ checks it
736
737 def __init__(self, args, bufsize=-1, executable=None,
668 stdin=None, stdout=None, stderr=None, 738 stdin=None, stdout=None, stderr=None,
669 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, 739 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
670 shell=False, cwd=None, env=None, universal_newlines=False, 740 shell=False, cwd=None, env=None, universal_newlines=False,
671 startupinfo=None, creationflags=0, 741 startupinfo=None, creationflags=0,
672 restore_signals=True, start_new_session=False, 742 restore_signals=True, start_new_session=False,
673 pass_fds=()): 743 pass_fds=()):
674 """Create new Popen instance.""" 744 """Create new Popen instance."""
675 _cleanup() 745 _cleanup()
676 746
677 self._child_created = False
678 self._input = None 747 self._input = None
679 self._communication_started = False 748 self._communication_started = False
680 if bufsize is None: 749 if bufsize is None:
681 bufsize = 0 # Restore default 750 bufsize = -1 # Restore default
682 if not isinstance(bufsize, int): 751 if not isinstance(bufsize, int):
683 raise TypeError("bufsize must be an integer") 752 raise TypeError("bufsize must be an integer")
684 753
685 if mswindows: 754 if mswindows:
686 if preexec_fn is not None: 755 if preexec_fn is not None:
687 raise ValueError("preexec_fn is not supported on Windows " 756 raise ValueError("preexec_fn is not supported on Windows "
688 "platforms") 757 "platforms")
689 any_stdio_set = (stdin is not None or stdout is not None or 758 any_stdio_set = (stdin is not None or stdout is not None or
690 stderr is not None) 759 stderr is not None)
691 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: 760 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
(...skipping 53 matching lines...) Expand 10 before | Expand all | Expand 10 after
745 if mswindows: 814 if mswindows:
746 if p2cwrite != -1: 815 if p2cwrite != -1:
747 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) 816 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
748 if c2pread != -1: 817 if c2pread != -1:
749 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) 818 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
750 if errread != -1: 819 if errread != -1:
751 errread = msvcrt.open_osfhandle(errread.Detach(), 0) 820 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
752 821
753 if p2cwrite != -1: 822 if p2cwrite != -1:
754 self.stdin = io.open(p2cwrite, 'wb', bufsize) 823 self.stdin = io.open(p2cwrite, 'wb', bufsize)
755 if self.universal_newlines: 824 if universal_newlines:
756 self.stdin = io.TextIOWrapper(self.stdin) 825 self.stdin = io.TextIOWrapper(self.stdin, write_through=True)
757 if c2pread != -1: 826 if c2pread != -1:
758 self.stdout = io.open(c2pread, 'rb', bufsize) 827 self.stdout = io.open(c2pread, 'rb', bufsize)
759 if universal_newlines: 828 if universal_newlines:
760 self.stdout = io.TextIOWrapper(self.stdout) 829 self.stdout = io.TextIOWrapper(self.stdout)
761 if errread != -1: 830 if errread != -1:
762 self.stderr = io.open(errread, 'rb', bufsize) 831 self.stderr = io.open(errread, 'rb', bufsize)
763 if universal_newlines: 832 if universal_newlines:
764 self.stderr = io.TextIOWrapper(self.stderr) 833 self.stderr = io.TextIOWrapper(self.stderr)
765 834
835 self._closed_child_pipe_fds = False
766 try: 836 try:
767 self._execute_child(args, executable, preexec_fn, close_fds, 837 self._execute_child(args, executable, preexec_fn, close_fds,
768 pass_fds, cwd, env, universal_newlines, 838 pass_fds, cwd, env,
769 startupinfo, creationflags, shell, 839 startupinfo, creationflags, shell,
770 p2cread, p2cwrite, 840 p2cread, p2cwrite,
771 c2pread, c2pwrite, 841 c2pread, c2pwrite,
772 errread, errwrite, 842 errread, errwrite,
773 restore_signals, start_new_session) 843 restore_signals, start_new_session)
774 except: 844 except:
775 # Cleanup if the child failed starting 845 # Cleanup if the child failed starting.
776 for f in filter(None, [self.stdin, self.stdout, self.stderr]): 846 for f in filter(None, (self.stdin, self.stdout, self.stderr)):
777 try: 847 try:
778 f.close() 848 f.close()
779 except EnvironmentError: 849 except OSError:
780 # Ignore EBADF or other errors 850 pass # Ignore EBADF or other errors.
781 pass 851
852 if not self._closed_child_pipe_fds:
853 to_close = []
854 if stdin == PIPE:
855 to_close.append(p2cread)
856 if stdout == PIPE:
857 to_close.append(c2pwrite)
858 if stderr == PIPE:
859 to_close.append(errwrite)
860 if hasattr(self, '_devnull'):
861 to_close.append(self._devnull)
862 for fd in to_close:
863 try:
864 os.close(fd)
865 except OSError:
866 pass
867
782 raise 868 raise
783 869
784 870
785 def _translate_newlines(self, data, encoding): 871 def _translate_newlines(self, data, encoding):
786 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n") 872 data = data.decode(encoding)
787 return data.decode(encoding) 873 return data.replace("\r\n", "\n").replace("\r", "\n")
788 874
789 def __enter__(self): 875 def __enter__(self):
790 return self 876 return self
791 877
792 def __exit__(self, type, value, traceback): 878 def __exit__(self, type, value, traceback):
793 if self.stdout: 879 if self.stdout:
794 self.stdout.close() 880 self.stdout.close()
795 if self.stderr: 881 if self.stderr:
796 self.stderr.close() 882 self.stderr.close()
797 if self.stdin: 883 if self.stdin:
798 self.stdin.close() 884 self.stdin.close()
799 # Wait for the process to terminate, to avoid zombies. 885 # Wait for the process to terminate, to avoid zombies.
800 self.wait() 886 self.wait()
801 887
802 def __del__(self, _maxsize=sys.maxsize, _active=_active): 888 def __del__(self, _maxsize=sys.maxsize):
803 # If anything goes wrong before __init__ executes, 889 if not self._child_created:
804 # we don't have a _child_created attribute at all
805 if not getattr(self, '_child_created', False):
806 # We didn't get to successfully create a child process. 890 # We didn't get to successfully create a child process.
807 return 891 return
808 # In case the child hasn't been waited on, check if it's done. 892 # In case the child hasn't been waited on, check if it's done.
809 self._internal_poll(_deadstate=_maxsize) 893 self._internal_poll(_deadstate=_maxsize)
810 if self.returncode is None and _active is not None: 894 if self.returncode is None and _active is not None:
811 # Child is still running, keep us alive until we can wait on it. 895 # Child is still running, keep us alive until we can wait on it.
812 _active.append(self) 896 _active.append(self)
813 897
814 def _get_devnull(self): 898 def _get_devnull(self):
815 if not hasattr(self, '_devnull'): 899 if not hasattr(self, '_devnull'):
(...skipping 16 matching lines...) Expand all
832 # started communicating, and we have one or zero pipes, using select() 916 # started communicating, and we have one or zero pipes, using select()
833 # or threads is unnecessary. 917 # or threads is unnecessary.
834 if (timeout is None and not self._communication_started and 918 if (timeout is None and not self._communication_started and
835 [self.stdin, self.stdout, self.stderr].count(None) >= 2): 919 [self.stdin, self.stdout, self.stderr].count(None) >= 2):
836 stdout = None 920 stdout = None
837 stderr = None 921 stderr = None
838 if self.stdin: 922 if self.stdin:
839 if input: 923 if input:
840 try: 924 try:
841 self.stdin.write(input) 925 self.stdin.write(input)
842 except IOError as e: 926 except OSError as e:
843 if e.errno != errno.EPIPE and e.errno != errno.EINVAL: 927 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
844 raise 928 raise
845 self.stdin.close() 929 self.stdin.close()
846 elif self.stdout: 930 elif self.stdout:
847 stdout = self.stdout.read() 931 stdout = _eintr_retry_call(self.stdout.read)
848 self.stdout.close() 932 self.stdout.close()
849 elif self.stderr: 933 elif self.stderr:
850 stderr = self.stderr.read() 934 stderr = _eintr_retry_call(self.stderr.read)
851 self.stderr.close() 935 self.stderr.close()
852 self.wait() 936 self.wait()
853 else: 937 else:
854 if timeout is not None: 938 if timeout is not None:
855 endtime = time.time() + timeout 939 endtime = _time() + timeout
856 else: 940 else:
857 endtime = None 941 endtime = None
858 942
859 try: 943 try:
860 stdout, stderr = self._communicate(input, endtime, timeout) 944 stdout, stderr = self._communicate(input, endtime, timeout)
861 finally: 945 finally:
862 self._communication_started = True 946 self._communication_started = True
863 947
864 sts = self.wait(timeout=self._remaining_time(endtime)) 948 sts = self.wait(timeout=self._remaining_time(endtime))
865 949
866 return (stdout, stderr) 950 return (stdout, stderr)
867 951
868 952
869 def poll(self): 953 def poll(self):
870 return self._internal_poll() 954 return self._internal_poll()
871 955
872 956
873 def _remaining_time(self, endtime): 957 def _remaining_time(self, endtime):
874 """Convenience for _communicate when computing timeouts.""" 958 """Convenience for _communicate when computing timeouts."""
875 if endtime is None: 959 if endtime is None:
876 return None 960 return None
877 else: 961 else:
878 return endtime - time.time() 962 return endtime - _time()
879 963
880 964
881 def _check_timeout(self, endtime, orig_timeout): 965 def _check_timeout(self, endtime, orig_timeout):
882 """Convenience for checking if a timeout has expired.""" 966 """Convenience for checking if a timeout has expired."""
883 if endtime is None: 967 if endtime is None:
884 return 968 return
885 if time.time() > endtime: 969 if _time() > endtime:
886 raise TimeoutExpired(self.args, orig_timeout) 970 raise TimeoutExpired(self.args, orig_timeout)
887 971
888 972
889 if mswindows: 973 if mswindows:
890 # 974 #
891 # Windows methods 975 # Windows methods
892 # 976 #
893 def _get_handles(self, stdin, stdout, stderr): 977 def _get_handles(self, stdin, stdout, stderr):
894 """Construct and return tuple with IO objects: 978 """Construct and return tuple with IO objects:
895 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 979 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
896 """ 980 """
897 if stdin is None and stdout is None and stderr is None: 981 if stdin is None and stdout is None and stderr is None:
898 return (-1, -1, -1, -1, -1, -1) 982 return (-1, -1, -1, -1, -1, -1)
899 983
900 p2cread, p2cwrite = -1, -1 984 p2cread, p2cwrite = -1, -1
901 c2pread, c2pwrite = -1, -1 985 c2pread, c2pwrite = -1, -1
902 errread, errwrite = -1, -1 986 errread, errwrite = -1, -1
903 987
904 if stdin is None: 988 if stdin is None:
905 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE) 989 p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
906 if p2cread is None: 990 if p2cread is None:
907 p2cread, _ = _subprocess.CreatePipe(None, 0) 991 p2cread, _ = _winapi.CreatePipe(None, 0)
992 p2cread = Handle(p2cread)
993 _winapi.CloseHandle(_)
908 elif stdin == PIPE: 994 elif stdin == PIPE:
909 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0) 995 p2cread, p2cwrite = _winapi.CreatePipe(None, 0)
996 p2cread, p2cwrite = Handle(p2cread), Handle(p2cwrite)
910 elif stdin == DEVNULL: 997 elif stdin == DEVNULL:
911 p2cread = msvcrt.get_osfhandle(self._get_devnull()) 998 p2cread = msvcrt.get_osfhandle(self._get_devnull())
912 elif isinstance(stdin, int): 999 elif isinstance(stdin, int):
913 p2cread = msvcrt.get_osfhandle(stdin) 1000 p2cread = msvcrt.get_osfhandle(stdin)
914 else: 1001 else:
915 # Assuming file-like object 1002 # Assuming file-like object
916 p2cread = msvcrt.get_osfhandle(stdin.fileno()) 1003 p2cread = msvcrt.get_osfhandle(stdin.fileno())
917 p2cread = self._make_inheritable(p2cread) 1004 p2cread = self._make_inheritable(p2cread)
918 1005
919 if stdout is None: 1006 if stdout is None:
920 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDL E) 1007 c2pwrite = _winapi.GetStdHandle(_winapi.STD_OUTPUT_HANDLE)
921 if c2pwrite is None: 1008 if c2pwrite is None:
922 _, c2pwrite = _subprocess.CreatePipe(None, 0) 1009 _, c2pwrite = _winapi.CreatePipe(None, 0)
1010 c2pwrite = Handle(c2pwrite)
1011 _winapi.CloseHandle(_)
923 elif stdout == PIPE: 1012 elif stdout == PIPE:
924 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0) 1013 c2pread, c2pwrite = _winapi.CreatePipe(None, 0)
1014 c2pread, c2pwrite = Handle(c2pread), Handle(c2pwrite)
925 elif stdout == DEVNULL: 1015 elif stdout == DEVNULL:
926 c2pwrite = msvcrt.get_osfhandle(self._get_devnull()) 1016 c2pwrite = msvcrt.get_osfhandle(self._get_devnull())
927 elif isinstance(stdout, int): 1017 elif isinstance(stdout, int):
928 c2pwrite = msvcrt.get_osfhandle(stdout) 1018 c2pwrite = msvcrt.get_osfhandle(stdout)
929 else: 1019 else:
930 # Assuming file-like object 1020 # Assuming file-like object
931 c2pwrite = msvcrt.get_osfhandle(stdout.fileno()) 1021 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
932 c2pwrite = self._make_inheritable(c2pwrite) 1022 c2pwrite = self._make_inheritable(c2pwrite)
933 1023
934 if stderr is None: 1024 if stderr is None:
935 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE ) 1025 errwrite = _winapi.GetStdHandle(_winapi.STD_ERROR_HANDLE)
936 if errwrite is None: 1026 if errwrite is None:
937 _, errwrite = _subprocess.CreatePipe(None, 0) 1027 _, errwrite = _winapi.CreatePipe(None, 0)
1028 errwrite = Handle(errwrite)
1029 _winapi.CloseHandle(_)
938 elif stderr == PIPE: 1030 elif stderr == PIPE:
939 errread, errwrite = _subprocess.CreatePipe(None, 0) 1031 errread, errwrite = _winapi.CreatePipe(None, 0)
1032 errread, errwrite = Handle(errread), Handle(errwrite)
940 elif stderr == STDOUT: 1033 elif stderr == STDOUT:
941 errwrite = c2pwrite 1034 errwrite = c2pwrite
942 elif stderr == DEVNULL: 1035 elif stderr == DEVNULL:
943 errwrite = msvcrt.get_osfhandle(self._get_devnull()) 1036 errwrite = msvcrt.get_osfhandle(self._get_devnull())
944 elif isinstance(stderr, int): 1037 elif isinstance(stderr, int):
945 errwrite = msvcrt.get_osfhandle(stderr) 1038 errwrite = msvcrt.get_osfhandle(stderr)
946 else: 1039 else:
947 # Assuming file-like object 1040 # Assuming file-like object
948 errwrite = msvcrt.get_osfhandle(stderr.fileno()) 1041 errwrite = msvcrt.get_osfhandle(stderr.fileno())
949 errwrite = self._make_inheritable(errwrite) 1042 errwrite = self._make_inheritable(errwrite)
950 1043
951 return (p2cread, p2cwrite, 1044 return (p2cread, p2cwrite,
952 c2pread, c2pwrite, 1045 c2pread, c2pwrite,
953 errread, errwrite) 1046 errread, errwrite)
954 1047
955 1048
956 def _make_inheritable(self, handle): 1049 def _make_inheritable(self, handle):
957 """Return a duplicate of handle, which is inheritable""" 1050 """Return a duplicate of handle, which is inheritable"""
958 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(), 1051 h = _winapi.DuplicateHandle(
959 handle, _subprocess.GetCurrentProcess(), 0, 1, 1052 _winapi.GetCurrentProcess(), handle,
960 _subprocess.DUPLICATE_SAME_ACCESS) 1053 _winapi.GetCurrentProcess(), 0, 1,
961 1054 _winapi.DUPLICATE_SAME_ACCESS)
962 1055 return Handle(h)
963 def _find_w9xpopen(self):
964 """Find and return absolut path to w9xpopen.exe"""
965 w9xpopen = os.path.join(
966 os.path.dirname(_subprocess.GetModuleFileName(0)),
967 "w9xpopen.exe")
968 if not os.path.exists(w9xpopen):
969 # Eeek - file-not-found - possibly an embedding
970 # situation - see if we can locate it in sys.exec_prefix
971 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
972 "w9xpopen.exe")
973 if not os.path.exists(w9xpopen):
974 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
975 "needed for Popen to work with your "
976 "shell or platform.")
977 return w9xpopen
978 1056
979 1057
980 def _execute_child(self, args, executable, preexec_fn, close_fds, 1058 def _execute_child(self, args, executable, preexec_fn, close_fds,
981 pass_fds, cwd, env, universal_newlines, 1059 pass_fds, cwd, env,
982 startupinfo, creationflags, shell, 1060 startupinfo, creationflags, shell,
983 p2cread, p2cwrite, 1061 p2cread, p2cwrite,
984 c2pread, c2pwrite, 1062 c2pread, c2pwrite,
985 errread, errwrite, 1063 errread, errwrite,
986 unused_restore_signals, unused_start_new_session): 1064 unused_restore_signals, unused_start_new_session):
987 """Execute program (MS Windows version)""" 1065 """Execute program (MS Windows version)"""
988 1066
989 assert not pass_fds, "pass_fds not supported on Windows." 1067 assert not pass_fds, "pass_fds not supported on Windows."
990 1068
991 if not isinstance(args, str): 1069 if not isinstance(args, str):
992 args = list2cmdline(args) 1070 args = list2cmdline(args)
993 1071
994 # Process startup details 1072 # Process startup details
995 if startupinfo is None: 1073 if startupinfo is None:
996 startupinfo = STARTUPINFO() 1074 startupinfo = STARTUPINFO()
997 if -1 not in (p2cread, c2pwrite, errwrite): 1075 if -1 not in (p2cread, c2pwrite, errwrite):
998 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES 1076 startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES
999 startupinfo.hStdInput = p2cread 1077 startupinfo.hStdInput = p2cread
1000 startupinfo.hStdOutput = c2pwrite 1078 startupinfo.hStdOutput = c2pwrite
1001 startupinfo.hStdError = errwrite 1079 startupinfo.hStdError = errwrite
1002 1080
1003 if shell: 1081 if shell:
1004 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW 1082 startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
1005 startupinfo.wShowWindow = _subprocess.SW_HIDE 1083 startupinfo.wShowWindow = _winapi.SW_HIDE
1006 comspec = os.environ.get("COMSPEC", "cmd.exe") 1084 comspec = os.environ.get("COMSPEC", "cmd.exe")
1007 args = '{} /c "{}"'.format (comspec, args) 1085 args = '{} /c "{}"'.format (comspec, args)
1008 if (_subprocess.GetVersion() >= 0x80000000 or
1009 os.path.basename(comspec).lower() == "command.com"):
1010 # Win9x, or using command.com on NT. We need to
1011 # use the w9xpopen intermediate program. For more
1012 # information, see KB Q150956
1013 # (http://web.archive.org/web/20011105084002/http://support. microsoft.com/support/kb/articles/Q150/9/56.asp)
1014 w9xpopen = self._find_w9xpopen()
1015 args = '"%s" %s' % (w9xpopen, args)
1016 # Not passing CREATE_NEW_CONSOLE has been known to
1017 # cause random failures on win9x. Specifically a
1018 # dialog: "Your program accessed mem currently in
1019 # use at xxx" and a hopeful warning about the
1020 # stability of your system. Cost is Ctrl+C won't
1021 # kill children.
1022 creationflags |= _subprocess.CREATE_NEW_CONSOLE
1023 1086
1024 # Start the process 1087 # Start the process
1025 try: 1088 try:
1026 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args, 1089 hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
1027 # no special security 1090 # no special security
1028 None, None, 1091 None, None,
1029 int(not close_fds), 1092 int(not close_fds),
1030 creationflags, 1093 creationflags,
1031 env, 1094 env,
1032 cwd, 1095 cwd,
1033 startupinfo) 1096 startupinfo)
1034 except pywintypes.error as e:
1035 # Translate pywintypes.error to WindowsError, which is
1036 # a subclass of OSError. FIXME: We should really
1037 # translate errno using _sys_errlist (or similar), but
1038 # how can this be done from Python?
1039 raise WindowsError(*e.args)
1040 finally: 1097 finally:
1041 # Child is launched. Close the parent's copy of those pipe 1098 # Child is launched. Close the parent's copy of those pipe
1042 # handles that only the child should have open. You need 1099 # handles that only the child should have open. You need
1043 # to make sure that no handles to the write end of the 1100 # to make sure that no handles to the write end of the
1044 # output pipe are maintained in this process or else the 1101 # output pipe are maintained in this process or else the
1045 # pipe will not close when the child process exits and the 1102 # pipe will not close when the child process exits and the
1046 # ReadFile will hang. 1103 # ReadFile will hang.
1047 if p2cread != -1: 1104 if p2cread != -1:
1048 p2cread.Close() 1105 p2cread.Close()
1049 if c2pwrite != -1: 1106 if c2pwrite != -1:
1050 c2pwrite.Close() 1107 c2pwrite.Close()
1051 if errwrite != -1: 1108 if errwrite != -1:
1052 errwrite.Close() 1109 errwrite.Close()
1053 if hasattr(self, '_devnull'): 1110 if hasattr(self, '_devnull'):
1054 os.close(self._devnull) 1111 os.close(self._devnull)
1055 1112
1056 # Retain the process handle, but close the thread handle 1113 # Retain the process handle, but close the thread handle
1057 self._child_created = True 1114 self._child_created = True
1058 self._handle = hp 1115 self._handle = Handle(hp)
1059 self.pid = pid 1116 self.pid = pid
1060 ht.Close() 1117 _winapi.CloseHandle(ht)
1061 1118
1062 def _internal_poll(self, _deadstate=None, 1119 def _internal_poll(self, _deadstate=None,
1063 _WaitForSingleObject=_subprocess.WaitForSingleObject, 1120 _WaitForSingleObject=_winapi.WaitForSingleObject,
1064 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0, 1121 _WAIT_OBJECT_0=_winapi.WAIT_OBJECT_0,
1065 _GetExitCodeProcess=_subprocess.GetExitCodeProcess): 1122 _GetExitCodeProcess=_winapi.GetExitCodeProcess):
1066 """Check if child process has terminated. Returns returncode 1123 """Check if child process has terminated. Returns returncode
1067 attribute. 1124 attribute.
1068 1125
1069 This method is called by __del__, so it can only refer to objects 1126 This method is called by __del__, so it can only refer to objects
1070 in its local scope. 1127 in its local scope.
1071 1128
1072 """ 1129 """
1073 if self.returncode is None: 1130 if self.returncode is None:
1074 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0: 1131 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
1075 self.returncode = _GetExitCodeProcess(self._handle) 1132 self.returncode = _GetExitCodeProcess(self._handle)
1076 return self.returncode 1133 return self.returncode
1077 1134
1078 1135
1079 def wait(self, timeout=None, endtime=None): 1136 def wait(self, timeout=None, endtime=None):
1080 """Wait for child process to terminate. Returns returncode 1137 """Wait for child process to terminate. Returns returncode
1081 attribute.""" 1138 attribute."""
1082 if endtime is not None: 1139 if endtime is not None:
1083 timeout = self._remaining_time(endtime) 1140 timeout = self._remaining_time(endtime)
1084 if timeout is None: 1141 if timeout is None:
1085 timeout_millis = _subprocess.INFINITE 1142 timeout_millis = _winapi.INFINITE
1086 else: 1143 else:
1087 timeout_millis = int(timeout * 1000) 1144 timeout_millis = int(timeout * 1000)
1088 if self.returncode is None: 1145 if self.returncode is None:
1089 result = _subprocess.WaitForSingleObject(self._handle, 1146 result = _winapi.WaitForSingleObject(self._handle,
1090 timeout_millis) 1147 timeout_millis)
1091 if result == _subprocess.WAIT_TIMEOUT: 1148 if result == _winapi.WAIT_TIMEOUT:
1092 raise TimeoutExpired(self.args, timeout) 1149 raise TimeoutExpired(self.args, timeout)
1093 self.returncode = _subprocess.GetExitCodeProcess(self._handle) 1150 self.returncode = _winapi.GetExitCodeProcess(self._handle)
1094 return self.returncode 1151 return self.returncode
1095 1152
1096 1153
1097 def _readerthread(self, fh, buffer): 1154 def _readerthread(self, fh, buffer):
1098 buffer.append(fh.read()) 1155 buffer.append(fh.read())
1099 fh.close() 1156 fh.close()
1100 1157
1101 1158
1102 def _communicate(self, input, endtime, orig_timeout): 1159 def _communicate(self, input, endtime, orig_timeout):
1103 # Start reader threads feeding into a list hanging off of this 1160 # Start reader threads feeding into a list hanging off of this
(...skipping 10 matching lines...) Expand all
1114 self.stderr_thread = \ 1171 self.stderr_thread = \
1115 threading.Thread(target=self._readerthread, 1172 threading.Thread(target=self._readerthread,
1116 args=(self.stderr, self._stderr_buff)) 1173 args=(self.stderr, self._stderr_buff))
1117 self.stderr_thread.daemon = True 1174 self.stderr_thread.daemon = True
1118 self.stderr_thread.start() 1175 self.stderr_thread.start()
1119 1176
1120 if self.stdin: 1177 if self.stdin:
1121 if input is not None: 1178 if input is not None:
1122 try: 1179 try:
1123 self.stdin.write(input) 1180 self.stdin.write(input)
1124 except IOError as e: 1181 except OSError as e:
1125 if e.errno != errno.EPIPE: 1182 if e.errno != errno.EPIPE:
1126 raise 1183 raise
1127 self.stdin.close() 1184 self.stdin.close()
1128 1185
1129 # Wait for the reader threads, or time out. If we time out, the 1186 # Wait for the reader threads, or time out. If we time out, the
1130 # threads remain reading and the fds left open in case the user 1187 # threads remain reading and the fds left open in case the user
1131 # calls communicate again. 1188 # calls communicate again.
1132 if self.stdout is not None: 1189 if self.stdout is not None:
1133 self.stdout_thread.join(self._remaining_time(endtime)) 1190 self.stdout_thread.join(self._remaining_time(endtime))
1134 if self.stdout_thread.isAlive(): 1191 if self.stdout_thread.is_alive():
1135 raise TimeoutExpired(self.args, orig_timeout) 1192 raise TimeoutExpired(self.args, orig_timeout)
1136 if self.stderr is not None: 1193 if self.stderr is not None:
1137 self.stderr_thread.join(self._remaining_time(endtime)) 1194 self.stderr_thread.join(self._remaining_time(endtime))
1138 if self.stderr_thread.isAlive(): 1195 if self.stderr_thread.is_alive():
1139 raise TimeoutExpired(self.args, orig_timeout) 1196 raise TimeoutExpired(self.args, orig_timeout)
1140 1197
1141 # Collect the output from and close both pipes, now that we know 1198 # Collect the output from and close both pipes, now that we know
1142 # both have been read successfully. 1199 # both have been read successfully.
1143 stdout = None 1200 stdout = None
1144 stderr = None 1201 stderr = None
1145 if self.stdout: 1202 if self.stdout:
1146 stdout = self._stdout_buff 1203 stdout = self._stdout_buff
1147 self.stdout.close() 1204 self.stdout.close()
1148 if self.stderr: 1205 if self.stderr:
(...skipping 16 matching lines...) Expand all
1165 elif sig == signal.CTRL_C_EVENT: 1222 elif sig == signal.CTRL_C_EVENT:
1166 os.kill(self.pid, signal.CTRL_C_EVENT) 1223 os.kill(self.pid, signal.CTRL_C_EVENT)
1167 elif sig == signal.CTRL_BREAK_EVENT: 1224 elif sig == signal.CTRL_BREAK_EVENT:
1168 os.kill(self.pid, signal.CTRL_BREAK_EVENT) 1225 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
1169 else: 1226 else:
1170 raise ValueError("Unsupported signal: {}".format(sig)) 1227 raise ValueError("Unsupported signal: {}".format(sig))
1171 1228
1172 def terminate(self): 1229 def terminate(self):
1173 """Terminates the process 1230 """Terminates the process
1174 """ 1231 """
1175 _subprocess.TerminateProcess(self._handle, 1) 1232 try:
1233 _winapi.TerminateProcess(self._handle, 1)
1234 except PermissionError:
1235 # ERROR_ACCESS_DENIED (winerror 5) is received when the
1236 # process already died.
1237 rc = _winapi.GetExitCodeProcess(self._handle)
1238 if rc == _winapi.STILL_ACTIVE:
1239 raise
1240 self.returncode = rc
1176 1241
1177 kill = terminate 1242 kill = terminate
1178 1243
1179 else: 1244 else:
1180 # 1245 #
1181 # POSIX methods 1246 # POSIX methods
1182 # 1247 #
1183 def _get_handles(self, stdin, stdout, stderr): 1248 def _get_handles(self, stdin, stdout, stderr):
1184 """Construct and return tuple with IO objects: 1249 """Construct and return tuple with IO objects:
1185 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 1250 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1186 """ 1251 """
1187 p2cread, p2cwrite = -1, -1 1252 p2cread, p2cwrite = -1, -1
1188 c2pread, c2pwrite = -1, -1 1253 c2pread, c2pwrite = -1, -1
1189 errread, errwrite = -1, -1 1254 errread, errwrite = -1, -1
1190 1255
1191 if stdin is None: 1256 if stdin is None:
1192 pass 1257 pass
1193 elif stdin == PIPE: 1258 elif stdin == PIPE:
1194 p2cread, p2cwrite = _create_pipe() 1259 p2cread, p2cwrite = os.pipe()
1195 elif stdin == DEVNULL: 1260 elif stdin == DEVNULL:
1196 p2cread = self._get_devnull() 1261 p2cread = self._get_devnull()
1197 elif isinstance(stdin, int): 1262 elif isinstance(stdin, int):
1198 p2cread = stdin 1263 p2cread = stdin
1199 else: 1264 else:
1200 # Assuming file-like object 1265 # Assuming file-like object
1201 p2cread = stdin.fileno() 1266 p2cread = stdin.fileno()
1202 1267
1203 if stdout is None: 1268 if stdout is None:
1204 pass 1269 pass
1205 elif stdout == PIPE: 1270 elif stdout == PIPE:
1206 c2pread, c2pwrite = _create_pipe() 1271 c2pread, c2pwrite = os.pipe()
1207 elif stdout == DEVNULL: 1272 elif stdout == DEVNULL:
1208 c2pwrite = self._get_devnull() 1273 c2pwrite = self._get_devnull()
1209 elif isinstance(stdout, int): 1274 elif isinstance(stdout, int):
1210 c2pwrite = stdout 1275 c2pwrite = stdout
1211 else: 1276 else:
1212 # Assuming file-like object 1277 # Assuming file-like object
1213 c2pwrite = stdout.fileno() 1278 c2pwrite = stdout.fileno()
1214 1279
1215 if stderr is None: 1280 if stderr is None:
1216 pass 1281 pass
1217 elif stderr == PIPE: 1282 elif stderr == PIPE:
1218 errread, errwrite = _create_pipe() 1283 errread, errwrite = os.pipe()
1219 elif stderr == STDOUT: 1284 elif stderr == STDOUT:
1220 errwrite = c2pwrite 1285 errwrite = c2pwrite
1221 elif stderr == DEVNULL: 1286 elif stderr == DEVNULL:
1222 errwrite = self._get_devnull() 1287 errwrite = self._get_devnull()
1223 elif isinstance(stderr, int): 1288 elif isinstance(stderr, int):
1224 errwrite = stderr 1289 errwrite = stderr
1225 else: 1290 else:
1226 # Assuming file-like object 1291 # Assuming file-like object
1227 errwrite = stderr.fileno() 1292 errwrite = stderr.fileno()
1228 1293
1229 return (p2cread, p2cwrite, 1294 return (p2cread, p2cwrite,
1230 c2pread, c2pwrite, 1295 c2pread, c2pwrite,
1231 errread, errwrite) 1296 errread, errwrite)
1232 1297
1233 1298
1234 def _close_fds(self, fds_to_keep): 1299 def _close_fds(self, fds_to_keep):
1235 start_fd = 3 1300 start_fd = 3
1236 for fd in sorted(fds_to_keep): 1301 for fd in sorted(fds_to_keep):
1237 if fd >= start_fd: 1302 if fd >= start_fd:
1238 os.closerange(start_fd, fd) 1303 os.closerange(start_fd, fd)
1239 start_fd = fd + 1 1304 start_fd = fd + 1
1240 if start_fd <= MAXFD: 1305 if start_fd <= MAXFD:
1241 os.closerange(start_fd, MAXFD) 1306 os.closerange(start_fd, MAXFD)
1242 1307
1243 1308
1244 def _execute_child(self, args, executable, preexec_fn, close_fds, 1309 def _execute_child(self, args, executable, preexec_fn, close_fds,
1245 pass_fds, cwd, env, universal_newlines, 1310 pass_fds, cwd, env,
1246 startupinfo, creationflags, shell, 1311 startupinfo, creationflags, shell,
1247 p2cread, p2cwrite, 1312 p2cread, p2cwrite,
1248 c2pread, c2pwrite, 1313 c2pread, c2pwrite,
1249 errread, errwrite, 1314 errread, errwrite,
1250 restore_signals, start_new_session): 1315 restore_signals, start_new_session):
1251 """Execute program (POSIX version)""" 1316 """Execute program (POSIX version)"""
1252 1317
1253 if isinstance(args, (str, bytes)): 1318 if isinstance(args, (str, bytes)):
1254 args = [args] 1319 args = [args]
1255 else: 1320 else:
1256 args = list(args) 1321 args = list(args)
1257 1322
1258 if shell: 1323 if shell:
1259 args = ["/bin/sh", "-c"] + args 1324 args = ["/bin/sh", "-c"] + args
1260 if executable: 1325 if executable:
1261 args[0] = executable 1326 args[0] = executable
1262 1327
1263 if executable is None: 1328 if executable is None:
1264 executable = args[0] 1329 executable = args[0]
1330 orig_executable = executable
1265 1331
1266 # For transferring possible exec failure from child to parent. 1332 # For transferring possible exec failure from child to parent.
1267 # Data format: "exception name:hex errno:description" 1333 # Data format: "exception name:hex errno:description"
1268 # Pickle is not used; it is complex and involves memory allocation. 1334 # Pickle is not used; it is complex and involves memory allocation.
1269 errpipe_read, errpipe_write = _create_pipe() 1335 errpipe_read, errpipe_write = os.pipe()
1270 try: 1336 try:
1271 try: 1337 try:
1272 1338 # We must avoid complex work that could involve
1273 if _posixsubprocess: 1339 # malloc or free in the child process to avoid
1274 # We must avoid complex work that could involve 1340 # potential deadlocks, thus we do all this here.
1275 # malloc or free in the child process to avoid 1341 # and pass it to fork_exec()
1276 # potential deadlocks, thus we do all this here. 1342
1277 # and pass it to fork_exec() 1343 if env is not None:
1278 1344 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
1279 if env: 1345 for k, v in env.items()]
1280 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
1281 for k, v in env.items()]
1282 else:
1283 env_list = None # Use execv instead of execve.
1284 executable = os.fsencode(executable)
1285 if os.path.dirname(executable):
1286 executable_list = (executable,)
1287 else:
1288 # This matches the behavior of os._execvpe().
1289 executable_list = tuple(
1290 os.path.join(os.fsencode(dir), executable)
1291 for dir in os.get_exec_path(env))
1292 fds_to_keep = set(pass_fds)
1293 fds_to_keep.add(errpipe_write)
1294 self.pid = _posixsubprocess.fork_exec(
1295 args, executable_list,
1296 close_fds, sorted(fds_to_keep), cwd, env_list,
1297 p2cread, p2cwrite, c2pread, c2pwrite,
1298 errread, errwrite,
1299 errpipe_read, errpipe_write,
1300 restore_signals, start_new_session, preexec_fn)
1301 else: 1346 else:
1302 # Pure Python implementation: It is not thread safe. 1347 env_list = None # Use execv instead of execve.
1303 # This implementation may deadlock in the child if your 1348 executable = os.fsencode(executable)
1304 # parent process has any other threads running. 1349 if os.path.dirname(executable):
1305 1350 executable_list = (executable,)
1306 gc_was_enabled = gc.isenabled() 1351 else:
1307 # Disable gc to avoid bug where gc -> file_dealloc -> 1352 # This matches the behavior of os._execvpe().
1308 # write to stderr -> hang. See issue1336 1353 executable_list = tuple(
1309 gc.disable() 1354 os.path.join(os.fsencode(dir), executable)
1310 try: 1355 for dir in os.get_exec_path(env))
1311 self.pid = os.fork() 1356 fds_to_keep = set(pass_fds)
1312 except: 1357 fds_to_keep.add(errpipe_write)
1313 if gc_was_enabled: 1358 self.pid = _posixsubprocess.fork_exec(
1314 gc.enable() 1359 args, executable_list,
1315 raise 1360 close_fds, sorted(fds_to_keep), cwd, env_list,
1316 self._child_created = True 1361 p2cread, p2cwrite, c2pread, c2pwrite,
1317 if self.pid == 0: 1362 errread, errwrite,
1318 # Child 1363 errpipe_read, errpipe_write,
1319 try: 1364 restore_signals, start_new_session, preexec_fn)
1320 # Close parent's pipe ends 1365 self._child_created = True
1321 if p2cwrite != -1:
1322 os.close(p2cwrite)
1323 if c2pread != -1:
1324 os.close(c2pread)
1325 if errread != -1:
1326 os.close(errread)
1327 os.close(errpipe_read)
1328
1329 # Dup fds for child
1330 def _dup2(a, b):
1331 # dup2() removes the CLOEXEC flag but
1332 # we must do it ourselves if dup2()
1333 # would be a no-op (issue #10806).
1334 if a == b:
1335 _set_cloexec(a, False)
1336 elif a != -1:
1337 os.dup2(a, b)
1338 _dup2(p2cread, 0)
1339 _dup2(c2pwrite, 1)
1340 _dup2(errwrite, 2)
1341
1342 # Close pipe fds. Make sure we don't close the
1343 # same fd more than once, or standard fds.
1344 closed = set()
1345 for fd in [p2cread, c2pwrite, errwrite]:
1346 if fd > 2 and fd not in closed:
1347 os.close(fd)
1348 closed.add(fd)
1349
1350 # Close all other fds, if asked for
1351 if close_fds:
1352 fds_to_keep = set(pass_fds)
1353 fds_to_keep.add(errpipe_write)
1354 self._close_fds(fds_to_keep)
1355
1356
1357 if cwd is not None:
1358 os.chdir(cwd)
1359
1360 # This is a copy of Python/pythonrun.c
1361 # _Py_RestoreSignals(). If that were exposed
1362 # as a sys._py_restoresignals func it would be
1363 # better.. but this pure python implementation
1364 # isn't likely to be used much anymore.
1365 if restore_signals:
1366 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1367 for sig in signals:
1368 if hasattr(signal, sig):
1369 signal.signal(getattr(signal, sig),
1370 signal.SIG_DFL)
1371
1372 if start_new_session and hasattr(os, 'setsid'):
1373 os.setsid()
1374
1375 if preexec_fn:
1376 preexec_fn()
1377
1378 if env is None:
1379 os.execvp(executable, args)
1380 else:
1381 os.execvpe(executable, args, env)
1382
1383 except:
1384 try:
1385 exc_type, exc_value = sys.exc_info()[:2]
1386 if isinstance(exc_value, OSError):
1387 errno_num = exc_value.errno
1388 else:
1389 errno_num = 0
1390 message = '%s:%x:%s' % (exc_type.__name__,
1391 errno_num, exc_value )
1392 message = message.encode(errors="surrogatepa ss")
1393 os.write(errpipe_write, message)
1394 except Exception:
1395 # We MUST not allow anything odd happening
1396 # above to prevent us from exiting below.
1397 pass
1398
1399 # This exitcode won't be reported to applications
1400 # so it really doesn't matter what we return.
1401 os._exit(255)
1402
1403 # Parent
1404 if gc_was_enabled:
1405 gc.enable()
1406 finally: 1366 finally:
1407 # be sure the FD is closed no matter what 1367 # be sure the FD is closed no matter what
1408 os.close(errpipe_write) 1368 os.close(errpipe_write)
1409 1369
1410 if p2cread != -1 and p2cwrite != -1: 1370 # self._devnull is not always defined.
1371 devnull_fd = getattr(self, '_devnull', None)
1372 if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
1411 os.close(p2cread) 1373 os.close(p2cread)
1412 if c2pwrite != -1 and c2pread != -1: 1374 if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
1413 os.close(c2pwrite) 1375 os.close(c2pwrite)
1414 if errwrite != -1 and errread != -1: 1376 if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
1415 os.close(errwrite) 1377 os.close(errwrite)
1416 if hasattr(self, '_devnull'): 1378 if devnull_fd is not None:
1417 os.close(self._devnull) 1379 os.close(devnull_fd)
1380 # Prevent a double close of these fds from __init__ on error.
1381 self._closed_child_pipe_fds = True
1418 1382
1419 # Wait for exec to fail or succeed; possibly raising an 1383 # Wait for exec to fail or succeed; possibly raising an
1420 # exception (limited in size) 1384 # exception (limited in size)
1421 data = bytearray() 1385 errpipe_data = bytearray()
1422 while True: 1386 while True:
1423 part = _eintr_retry_call(os.read, errpipe_read, 50000) 1387 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1424 data += part 1388 errpipe_data += part
1425 if not part or len(data) > 50000: 1389 if not part or len(errpipe_data) > 50000:
1426 break 1390 break
1427 finally: 1391 finally:
1428 # be sure the FD is closed no matter what 1392 # be sure the FD is closed no matter what
1429 os.close(errpipe_read) 1393 os.close(errpipe_read)
1430 1394
1431 if data: 1395 if errpipe_data:
1432 try: 1396 try:
1433 _eintr_retry_call(os.waitpid, self.pid, 0) 1397 _eintr_retry_call(os.waitpid, self.pid, 0)
1434 except OSError as e: 1398 except OSError as e:
1435 if e.errno != errno.ECHILD: 1399 if e.errno != errno.ECHILD:
1436 raise 1400 raise
1437 try: 1401 try:
1438 exception_name, hex_errno, err_msg = data.split(b':', 2) 1402 exception_name, hex_errno, err_msg = (
1403 errpipe_data.split(b':', 2))
1439 except ValueError: 1404 except ValueError:
1440 print('Bad exception data:', repr(data)) 1405 exception_name = b'SubprocessError'
1441 exception_name = b'RuntimeError'
1442 hex_errno = b'0' 1406 hex_errno = b'0'
1443 err_msg = b'Unknown' 1407 err_msg = (b'Bad exception data from child: ' +
1408 repr(errpipe_data))
1444 child_exception_type = getattr( 1409 child_exception_type = getattr(
1445 builtins, exception_name.decode('ascii'), 1410 builtins, exception_name.decode('ascii'),
1446 RuntimeError) 1411 SubprocessError)
1447 for fd in (p2cwrite, c2pread, errread):
1448 if fd != -1:
1449 os.close(fd)
1450 err_msg = err_msg.decode(errors="surrogatepass") 1412 err_msg = err_msg.decode(errors="surrogatepass")
1451 if issubclass(child_exception_type, OSError) and hex_errno: 1413 if issubclass(child_exception_type, OSError) and hex_errno:
1452 errno_num = int(hex_errno, 16) 1414 errno_num = int(hex_errno, 16)
1415 child_exec_never_called = (err_msg == "noexec")
1416 if child_exec_never_called:
1417 err_msg = ""
1453 if errno_num != 0: 1418 if errno_num != 0:
1454 err_msg = os.strerror(errno_num) 1419 err_msg = os.strerror(errno_num)
1455 if errno_num == errno.ENOENT: 1420 if errno_num == errno.ENOENT:
1456 err_msg += ': ' + repr(args[0]) 1421 if child_exec_never_called:
1422 # The error must be from chdir(cwd).
1423 err_msg += ': ' + repr(cwd)
1424 else:
1425 err_msg += ': ' + repr(orig_executable)
1457 raise child_exception_type(errno_num, err_msg) 1426 raise child_exception_type(errno_num, err_msg)
1458 raise child_exception_type(err_msg) 1427 raise child_exception_type(err_msg)
1459 1428
1460 1429
1461 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED, 1430 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1462 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED, 1431 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1463 _WEXITSTATUS=os.WEXITSTATUS): 1432 _WEXITSTATUS=os.WEXITSTATUS, _SubprocessError=SubprocessError):
1464 # This method is called (indirectly) by __del__, so it cannot 1433 # This method is called (indirectly) by __del__, so it cannot
1465 # refer to anything outside of its local scope.""" 1434 # refer to anything outside of its local scope."""
1466 if _WIFSIGNALED(sts): 1435 if _WIFSIGNALED(sts):
1467 self.returncode = -_WTERMSIG(sts) 1436 self.returncode = -_WTERMSIG(sts)
1468 elif _WIFEXITED(sts): 1437 elif _WIFEXITED(sts):
1469 self.returncode = _WEXITSTATUS(sts) 1438 self.returncode = _WEXITSTATUS(sts)
1470 else: 1439 else:
1471 # Should never happen 1440 # Should never happen
1472 raise RuntimeError("Unknown child exit status!") 1441 raise _SubprocessError("Unknown child exit status!")
1473 1442
1474 1443
1475 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid, 1444 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1476 _WNOHANG=os.WNOHANG, _os_error=os.error): 1445 _WNOHANG=os.WNOHANG, _ECHILD=errno.ECHILD, _OSError=OSError):
1477 """Check if child process has terminated. Returns returncode 1446 """Check if child process has terminated. Returns returncode
1478 attribute. 1447 attribute.
1479 1448
1480 This method is called by __del__, so it cannot reference anything 1449 This method is called by __del__, so it cannot reference anything
1481 outside of the local scope (nor can any methods it calls). 1450 outside of the local scope (nor can any methods it calls).
1482 1451
1483 """ 1452 """
1484 if self.returncode is None: 1453 if self.returncode is None:
1485 try: 1454 try:
1486 pid, sts = _waitpid(self.pid, _WNOHANG) 1455 pid, sts = _waitpid(self.pid, _WNOHANG)
1487 if pid == self.pid: 1456 if pid == self.pid:
1488 self._handle_exitstatus(sts) 1457 self._handle_exitstatus(sts)
1489 except _os_error: 1458 except _OSError as e:
1490 if _deadstate is not None: 1459 if _deadstate is not None:
1491 self.returncode = _deadstate 1460 self.returncode = _deadstate
1461 elif e.errno == _ECHILD:
1462 # This happens if SIGCLD is set to be ignored or
1463 # waiting for child processes has otherwise been
1464 # disabled for our process. This child is dead, we
1465 # can't get the status.
1466 # http://bugs.python.org/issue15756
1467 self.returncode = 0
1492 return self.returncode 1468 return self.returncode
1493 1469
1494 1470
1495 def _try_wait(self, wait_flags): 1471 def _try_wait(self, wait_flags):
1496 try: 1472 try:
1497 (pid, sts) = _eintr_retry_call(os.waitpid, self.pid, wait_flags) 1473 (pid, sts) = _eintr_retry_call(os.waitpid, self.pid, wait_flags)
1498 except OSError as e: 1474 except OSError as e:
1499 if e.errno != errno.ECHILD: 1475 if e.errno != errno.ECHILD:
1500 raise 1476 raise
1501 # This happens if SIGCLD is set to be ignored or waiting 1477 # This happens if SIGCLD is set to be ignored or waiting
1502 # for child processes has otherwise been disabled for our 1478 # for child processes has otherwise been disabled for our
1503 # process. This child is dead, we can't get the status. 1479 # process. This child is dead, we can't get the status.
1504 pid = self.pid 1480 pid = self.pid
1505 sts = 0 1481 sts = 0
1506 return (pid, sts) 1482 return (pid, sts)
1507 1483
1508 1484
1509 def wait(self, timeout=None, endtime=None): 1485 def wait(self, timeout=None, endtime=None):
1510 """Wait for child process to terminate. Returns returncode 1486 """Wait for child process to terminate. Returns returncode
1511 attribute.""" 1487 attribute."""
1512 if self.returncode is not None: 1488 if self.returncode is not None:
1513 return self.returncode 1489 return self.returncode
1514 1490
1515 # endtime is preferred to timeout. timeout is only used for 1491 # endtime is preferred to timeout. timeout is only used for
1516 # printing. 1492 # printing.
1517 if endtime is not None or timeout is not None: 1493 if endtime is not None or timeout is not None:
1518 if endtime is None: 1494 if endtime is None:
1519 endtime = time.time() + timeout 1495 endtime = _time() + timeout
1520 elif timeout is None: 1496 elif timeout is None:
1521 timeout = self._remaining_time(endtime) 1497 timeout = self._remaining_time(endtime)
1522 1498
1523 if endtime is not None: 1499 if endtime is not None:
1524 # Enter a busy loop if we have a timeout. This busy loop was 1500 # Enter a busy loop if we have a timeout. This busy loop was
1525 # cribbed from Lib/threading.py in Thread.wait() at r71065. 1501 # cribbed from Lib/threading.py in Thread.wait() at r71065.
1526 delay = 0.0005 # 500 us -> initial delay of 1 ms 1502 delay = 0.0005 # 500 us -> initial delay of 1 ms
1527 while True: 1503 while True:
1528 (pid, sts) = self._try_wait(os.WNOHANG) 1504 (pid, sts) = self._try_wait(os.WNOHANG)
1529 assert pid == self.pid or pid == 0 1505 assert pid == self.pid or pid == 0
1530 if pid == self.pid: 1506 if pid == self.pid:
1531 self._handle_exitstatus(sts) 1507 self._handle_exitstatus(sts)
1532 break 1508 break
1533 remaining = self._remaining_time(endtime) 1509 remaining = self._remaining_time(endtime)
1534 if remaining <= 0: 1510 if remaining <= 0:
1535 raise TimeoutExpired(self.args, timeout) 1511 raise TimeoutExpired(self.args, timeout)
1536 delay = min(delay * 2, remaining, .05) 1512 delay = min(delay * 2, remaining, .05)
1537 time.sleep(delay) 1513 time.sleep(delay)
1538 elif self.returncode is None: 1514 else:
1539 (pid, sts) = self._try_wait(0) 1515 while self.returncode is None:
1540 self._handle_exitstatus(sts) 1516 (pid, sts) = self._try_wait(0)
1517 # Check the pid and loop as waitpid has been known to return
1518 # 0 even without WNOHANG in odd situations. issue14396.
1519 if pid == self.pid:
1520 self._handle_exitstatus(sts)
1541 return self.returncode 1521 return self.returncode
1542 1522
1543 1523
1544 def _communicate(self, input, endtime, orig_timeout): 1524 def _communicate(self, input, endtime, orig_timeout):
1545 if self.stdin and not self._communication_started: 1525 if self.stdin and not self._communication_started:
1546 # Flush stdio buffer. This might block, if the user has 1526 # Flush stdio buffer. This might block, if the user has
1547 # been writing to .stdin in an uncontrolled fashion. 1527 # been writing to .stdin in an uncontrolled fashion.
1548 self.stdin.flush() 1528 self.stdin.flush()
1549 if not input: 1529 if not input:
1550 self.stdin.close() 1530 self.stdin.close()
(...skipping 19 matching lines...) Expand all
1570 if stdout is not None: 1550 if stdout is not None:
1571 stdout = self._translate_newlines(stdout, 1551 stdout = self._translate_newlines(stdout,
1572 self.stdout.encoding) 1552 self.stdout.encoding)
1573 if stderr is not None: 1553 if stderr is not None:
1574 stderr = self._translate_newlines(stderr, 1554 stderr = self._translate_newlines(stderr,
1575 self.stderr.encoding) 1555 self.stderr.encoding)
1576 1556
1577 return (stdout, stderr) 1557 return (stdout, stderr)
1578 1558
1579 1559
1560 def _save_input(self, input):
1561 # This method is called from the _communicate_with_*() methods
1562 # so that if we time out while communicating, we can continue
1563 # sending input if we retry.
1564 if self.stdin and self._input is None:
1565 self._input_offset = 0
1566 self._input = input
1567 if self.universal_newlines and input is not None:
1568 self._input = self._input.encode(self.stdin.encoding)
1569
1570
1580 def _communicate_with_poll(self, input, endtime, orig_timeout): 1571 def _communicate_with_poll(self, input, endtime, orig_timeout):
1581 stdout = None # Return 1572 stdout = None # Return
1582 stderr = None # Return 1573 stderr = None # Return
1583 1574
1584 if not self._communication_started: 1575 if not self._communication_started:
1585 self._fd2file = {} 1576 self._fd2file = {}
1586 1577
1587 poller = select.poll() 1578 poller = select.poll()
1588 def register_and_append(file_obj, eventmask): 1579 def register_and_append(file_obj, eventmask):
1589 poller.register(file_obj.fileno(), eventmask) 1580 poller.register(file_obj.fileno(), eventmask)
(...skipping 16 matching lines...) Expand all
1606 self._fd2output[self.stderr.fileno()] = [] 1597 self._fd2output[self.stderr.fileno()] = []
1607 1598
1608 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI 1599 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1609 if self.stdout: 1600 if self.stdout:
1610 register_and_append(self.stdout, select_POLLIN_POLLPRI) 1601 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1611 stdout = self._fd2output[self.stdout.fileno()] 1602 stdout = self._fd2output[self.stdout.fileno()]
1612 if self.stderr: 1603 if self.stderr:
1613 register_and_append(self.stderr, select_POLLIN_POLLPRI) 1604 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1614 stderr = self._fd2output[self.stderr.fileno()] 1605 stderr = self._fd2output[self.stderr.fileno()]
1615 1606
1616 # Save the input here so that if we time out while communicating, 1607 self._save_input(input)
1617 # we can continue sending input if we retry.
1618 if self.stdin and self._input is None:
1619 self._input_offset = 0
1620 self._input = input
1621 if self.universal_newlines:
1622 self._input = self._input.encode(self.stdin.encoding)
1623 1608
1624 while self._fd2file: 1609 while self._fd2file:
1625 timeout = self._remaining_time(endtime) 1610 timeout = self._remaining_time(endtime)
1626 if timeout is not None and timeout < 0: 1611 if timeout is not None and timeout < 0:
1627 raise TimeoutExpired(self.args, orig_timeout) 1612 raise TimeoutExpired(self.args, orig_timeout)
1628 try: 1613 try:
1629 ready = poller.poll(timeout) 1614 ready = poller.poll(timeout)
1630 except select.error as e: 1615 except OSError as e:
1631 if e.args[0] == errno.EINTR: 1616 if e.args[0] == errno.EINTR:
1632 continue 1617 continue
1633 raise 1618 raise
1634 self._check_timeout(endtime, orig_timeout) 1619 self._check_timeout(endtime, orig_timeout)
1635 1620
1636 # XXX Rewrite these to use non-blocking I/O on the 1621 # XXX Rewrite these to use non-blocking I/O on the
1637 # file objects; they are no longer using C stdio! 1622 # file objects; they are no longer using C stdio!
1638 1623
1639 for fd, mode in ready: 1624 for fd, mode in ready:
1640 if mode & select.POLLOUT: 1625 if mode & select.POLLOUT:
(...skipping 25 matching lines...) Expand all
1666 if not self._communication_started: 1651 if not self._communication_started:
1667 self._read_set = [] 1652 self._read_set = []
1668 self._write_set = [] 1653 self._write_set = []
1669 if self.stdin and input: 1654 if self.stdin and input:
1670 self._write_set.append(self.stdin) 1655 self._write_set.append(self.stdin)
1671 if self.stdout: 1656 if self.stdout:
1672 self._read_set.append(self.stdout) 1657 self._read_set.append(self.stdout)
1673 if self.stderr: 1658 if self.stderr:
1674 self._read_set.append(self.stderr) 1659 self._read_set.append(self.stderr)
1675 1660
1676 if self.stdin and self._input is None: 1661 self._save_input(input)
1677 self._input_offset = 0
1678 self._input = input
1679 if self.universal_newlines:
1680 self._input = self._input.encode(self.stdin.encoding)
1681 1662
1682 stdout = None # Return 1663 stdout = None # Return
1683 stderr = None # Return 1664 stderr = None # Return
1684 1665
1685 if self.stdout: 1666 if self.stdout:
1686 if not self._communication_started: 1667 if not self._communication_started:
1687 self._stdout_buff = [] 1668 self._stdout_buff = []
1688 stdout = self._stdout_buff 1669 stdout = self._stdout_buff
1689 if self.stderr: 1670 if self.stderr:
1690 if not self._communication_started: 1671 if not self._communication_started:
1691 self._stderr_buff = [] 1672 self._stderr_buff = []
1692 stderr = self._stderr_buff 1673 stderr = self._stderr_buff
1693 1674
1694 while self._read_set or self._write_set: 1675 while self._read_set or self._write_set:
1695 timeout = self._remaining_time(endtime) 1676 timeout = self._remaining_time(endtime)
1696 if timeout is not None and timeout < 0: 1677 if timeout is not None and timeout < 0:
1697 raise TimeoutExpired(self.args, orig_timeout) 1678 raise TimeoutExpired(self.args, orig_timeout)
1698 try: 1679 try:
1699 (rlist, wlist, xlist) = \ 1680 (rlist, wlist, xlist) = \
1700 select.select(self._read_set, self._write_set, [], 1681 select.select(self._read_set, self._write_set, [],
1701 timeout) 1682 timeout)
1702 except select.error as e: 1683 except OSError as e:
1703 if e.args[0] == errno.EINTR: 1684 if e.args[0] == errno.EINTR:
1704 continue 1685 continue
1705 raise 1686 raise
1706 1687
1707 # According to the docs, returning three empty lists indicates 1688 # According to the docs, returning three empty lists indicates
1708 # that the timeout expired. 1689 # that the timeout expired.
1709 if not (rlist or wlist or xlist): 1690 if not (rlist or wlist or xlist):
1710 raise TimeoutExpired(self.args, orig_timeout) 1691 raise TimeoutExpired(self.args, orig_timeout)
1711 # We also check what time it is ourselves for good measure. 1692 # We also check what time it is ourselves for good measure.
1712 self._check_timeout(endtime, orig_timeout) 1693 self._check_timeout(endtime, orig_timeout)
(...skipping 42 matching lines...) Expand 10 before | Expand all | Expand 10 after
1755 1736
1756 def terminate(self): 1737 def terminate(self):
1757 """Terminate the process with SIGTERM 1738 """Terminate the process with SIGTERM
1758 """ 1739 """
1759 self.send_signal(signal.SIGTERM) 1740 self.send_signal(signal.SIGTERM)
1760 1741
1761 def kill(self): 1742 def kill(self):
1762 """Kill the process with SIGKILL 1743 """Kill the process with SIGKILL
1763 """ 1744 """
1764 self.send_signal(signal.SIGKILL) 1745 self.send_signal(signal.SIGKILL)
LEFTRIGHT
« no previous file | no next file » | Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Toggle Comments ('s')

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7