diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/c-api/memory.rst --- a/Doc/c-api/memory.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/c-api/memory.rst Thu Jun 05 21:36:59 2014 +0200 @@ -232,7 +232,7 @@ .. versionadded:: 3.4 -.. c:type:: PyMemAllocatorEx +.. c:type:: PyMemAllocator Structure used to describe a memory block allocator. The structure has four fields: @@ -253,9 +253,7 @@ +----------------------------------------------------------+---------------------------------------+ .. versionchanged:: 3.5 - The :c:type:`PyMemAllocator` structure was renamed to - :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. - + Add a new field ``calloc``. .. c:type:: PyMemAllocatorDomain @@ -269,12 +267,12 @@ :c:func:`PyObject_Realloc` and :c:func:`PyObject_Free` -.. c:function:: void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator) +.. c:function:: void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator) Get the memory block allocator of the specified domain. -.. c:function:: void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator) +.. c:function:: void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator) Set the memory block allocator of the specified domain. diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/c-api/unicode.rst --- a/Doc/c-api/unicode.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/c-api/unicode.rst Thu Jun 05 21:36:59 2014 +0200 @@ -1624,7 +1624,7 @@ Compare a unicode object, *uni*, with *string* and return -1, 0, 1 for less than, equal, and greater than, respectively. It is best to pass only ASCII-encoded strings, but the function interprets the input string as - ISO-8859-1 if it contains non-ASCII characters. + ISO-8859-1 if it contains non-ASCII characters". .. c:function:: PyObject* PyUnicode_RichCompare(PyObject *left, PyObject *right, int op) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/distributing/index.rst --- a/Doc/distributing/index.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/distributing/index.rst Thu Jun 05 21:36:59 2014 +0200 @@ -49,7 +49,7 @@ of the mailing list used to coordinate Python packaging standards development). * ``setuptools`` is a (largely) drop-in replacement for ``distutils`` first - published in 2004. Its most notable addition over the unmodified + published in 2004. It's most notable addition over the unmodified ``distutils`` tools was the ability to declare dependencies on other packages. It is currently recommended as a more regularly updated alternative to ``distutils`` that offers consistent support for more diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/howto/logging.rst --- a/Doc/howto/logging.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/howto/logging.rst Thu Jun 05 21:36:59 2014 +0200 @@ -1027,15 +1027,6 @@ so that if the logger's threshold is set above ``DEBUG``, the calls to :func:`expensive_func1` and :func:`expensive_func2` are never made. -.. note:: In some cases, :meth:`~Logger.isEnabledFor` can iself be more - expensive than you'd like (e.g. for deeply nested loggers where an explicit - level is only set high up in the logger hierarchy). In such cases (or if you - want to avoid calling a method in tight loops), you can cache the result of a - call to :meth:`~Logger.isEnabledFor` in a local or instance variable, and use - that instead of calling the method each time. Such a cached value would only - need to be recomputed when the logging configuration changes dynamically - while the application is running (which is not all that common). - There are other optimizations which can be made for specific applications which need more precise control over what logging information is collected. Here's a list of things you can do to avoid processing during logging which you don't @@ -1045,12 +1036,6 @@ | What you don't want to collect | How to avoid collecting it | +===============================================+========================================+ | Information about where calls were made from. | Set ``logging._srcfile`` to ``None``. | -| | This avoids calling | -| | :func:`sys._getframe`, which may help | -| | to speed up your code in environments | -| | like PyPy (which can't speed up code | -| | that uses :func:`sys._getframe`), if | -| | and when PyPy supports Python 3.x. | +-----------------------------------------------+----------------------------------------+ | Threading information. | Set ``logging.logThreads`` to ``0``. | +-----------------------------------------------+----------------------------------------+ diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/howto/webservers.rst --- a/Doc/howto/webservers.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/howto/webservers.rst Thu Jun 05 21:36:59 2014 +0200 @@ -687,7 +687,7 @@ The newest version of TurboGears, version 2.0, moves even further in direction of WSGI support and a component-based architecture. TurboGears 2 is based on the WSGI stack of another popular component-based web framework, `Pylons -`_. +`_. Zope diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/asyncio-eventloop.rst Thu Jun 05 21:36:59 2014 +0200 @@ -64,19 +64,19 @@ .. method:: get_event_loop() - Get the event loop for the current context. Returns an event loop object - implementing the :class:`BaseEventLoop` interface, or raises an exception in case + Get the event loop for current context. Returns an event loop object + implementing :class:`BaseEventLoop` interface, or raises an exception in case no event loop has been set for the current context and the current policy does not specify to create one. It should never return ``None``. .. method:: set_event_loop(loop) - Set the event loop for the current context to *loop*. + Set the event loop of the current context to *loop*. .. method:: new_event_loop() Create and return a new event loop object according to this policy's rules. - If there's need to set this loop as the event loop for the current context, + If there's need to set this loop as the event loop of the current context, :meth:`set_event_loop` must be called explicitly. Access to the global loop policy @@ -119,12 +119,6 @@ Callback scheduled after :meth:`stop` is called won't. However, those callbacks will run if :meth:`run_forever` is called again later. -.. method:: BaseEventLoop.is_closed() - - Returns ``True`` if the event loop was closed. - - .. versionadded:: 3.4.2 - .. method:: BaseEventLoop.close() Close the event loop. The loop should not be running. diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/asyncio-protocol.rst Thu Jun 05 21:36:59 2014 +0200 @@ -272,8 +272,8 @@ Connection callbacks -------------------- -These callbacks may be called on :class:`Protocol`, :class:`DatagramProtocol` -and :class:`SubprocessProtocol` instances: +These callbacks may be called on :class:`Protocol` and +:class:`SubprocessProtocol` instances: .. method:: BaseProtocol.connection_made(transport) @@ -291,10 +291,10 @@ The latter means a regular EOF is received, or the connection was aborted or closed by this side of the connection. -:meth:`~BaseProtocol.connection_made` and :meth:`~BaseProtocol.connection_lost` -are called exactly once per successful connection. All other callbacks will be -called between those two methods, which allows for easier resource management -in your protocol implementation. +:meth:`connection_made` and :meth:`connection_lost` are called exactly once +per successful connection. All other callbacks will be called between those +two methods, which allows for easier resource management in your protocol +implementation. The following callbacks may be called only on :class:`SubprocessProtocol` instances: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/asyncio-task.rst Thu Jun 05 21:36:59 2014 +0200 @@ -327,27 +327,7 @@ ``None`` is returned when called not in the context of a :class:`Task`. - .. method:: cancel() - - Request this task to cancel itself. - - This arranges for a :exc:`~concurrent.futures.CancelledError` to be - thrown into the wrapped coroutine on the next cycle through the event - loop. The coroutine then has a chance to clean up or even deny the - request using try/except/finally. - - Contrary to :meth:`Future.cancel`, this does not guarantee that the task - will be cancelled: the exception might be caught and acted upon, delaying - cancellation of the task or preventing it completely. The task may also - return a value or raise a different exception. - - Immediately after this method is called, :meth:`~Future.cancelled` will - not return ``True`` (unless the task was already cancelled). A task will - be marked as cancelled when the wrapped coroutine terminates with a - :exc:`~concurrent.futures.CancelledError` exception (even if - :meth:`cancel` was not called). - - .. method:: get_stack(\*, limit=None) + .. method:: get_stack(self, \*, limit=None) Return the list of stack frames for this task's coroutine. @@ -521,8 +501,6 @@ to complete. Coroutines will be wrapped in Tasks. Returns two sets of :class:`Future`: (done, pending). - The sequence *futures* must not be empty. - *timeout* can be used to control the maximum number of seconds to wait before returning. *timeout* can be an int or float. If *timeout* is not specified or ``None``, there is no limit to the wait time. diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/dbm.rst --- a/Doc/library/dbm.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/dbm.rst Thu Jun 05 21:36:59 2014 +0200 @@ -316,18 +316,13 @@ dumbdbm database is created, files with :file:`.dat` and :file:`.dir` extensions are created. - The optional *flag* argument supports only the semantics of ``'c'`` - and ``'n'`` values. Other values will default to database being always - opened for update, and will be created if it does not exist. + The optional *flag* argument is currently ignored; the database is always opened + for update, and will be created if it does not exist. The optional *mode* argument is the Unix mode of the file, used only when the database has to be created. It defaults to octal ``0o666`` (and will be modified by the prevailing umask). - .. versionchanged:: 3.5 - :func:`.open` always creates a new database when the flag has the value - ``'n'``. - In addition to the methods provided by the :class:`collections.abc.MutableMapping` class, :class:`dumbdbm` objects provide the following method: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/os.rst --- a/Doc/library/os.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/os.rst Thu Jun 05 21:36:59 2014 +0200 @@ -379,7 +379,7 @@ .. index:: single: user; id - Return the current process's real user id. + Return the current process's user id. Availability: Unix. @@ -1092,10 +1092,6 @@ Availability: Unix. - .. note:: - - For a higher-level version of this see :mod:`socket.socket.sendfile`. - .. versionadded:: 3.3 diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/smtpd.rst --- a/Doc/library/smtpd.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/smtpd.rst Thu Jun 05 21:36:59 2014 +0200 @@ -20,7 +20,8 @@ Additionally the SMTPChannel may be extended to implement very specific interaction behaviour with SMTP clients. -The code supports :RFC:`5321`, plus the :rfc:`1870` SIZE extension. +The code supports :RFC:`5321`, plus the :rfc:`1870` SIZE and :rfc:`6531` +SMTPUTF8 extensions. SMTPServer Objects @@ -28,7 +29,7 @@ .. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432,\ - map=None, decode_data=True) + map=None, enable_SMTPUTF8=False) Create a new :class:`SMTPServer` object, which binds to local address *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. It @@ -39,13 +40,11 @@ accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no limit. + *enable_SMTPUTF8* determins whether the SMTPUTF8 extension (as defined in + :RFC:`6531`) should be enabled. Default: ``False``. + A dictionary can be specified in *map* to avoid using a global socket map. - *decode_data* specifies whether the data portion of the SMTP transaction - should be decoded using UTF-8. The default is ``True`` for backward - compatibility reasons, but will change to ``False`` in Python 3.6. Specify - the keyword value explicitly to avoid the :exc:`DeprecationWarning`. - .. method:: process_message(peer, mailfrom, rcpttos, data) Raise :exc:`NotImplementedError` exception. Override this in subclasses to @@ -56,10 +55,6 @@ containing the contents of the e-mail (which should be in :rfc:`2822` format). - If the *decode_data* constructor keyword is set to ``True``, the *data* - argument will be a unicode string. If it is set to ``False``, it - will be a bytes object. - .. attribute:: channel_class Override this in subclasses to use a custom :class:`SMTPChannel` for @@ -68,8 +63,8 @@ .. versionchanged:: 3.4 The *map* argument was added. - .. versionchanged:: 3.5 - the *decode_data* argument was added. + .. versionchanged:: 3.6 + The *enable_SMTPUTF8* argument was added. DebuggingServer Objects @@ -109,7 +104,7 @@ ------------------- .. class:: SMTPChannel(server, conn, addr, data_size_limit=33554432,\ - map=None, decode_data=True) + map=None), enable_SMTPUTF8=False) Create a new :class:`SMTPChannel` object which manages the communication between the server and a single SMTP client. @@ -120,19 +115,14 @@ accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no limit. + *enable_SMTPUTF8* determins whether the SMTPUTF8 extension (as defined in + :RFC:`6531`) should be enabled. Default: ``False``. + A dictionary can be specified in *map* to avoid using a global socket map. - *decode_data* specifies whether the data portion of the SMTP transaction - should be decoded using UTF-8. The default is ``True`` for backward - compatibility reasons, but will change to ``False`` in Python 3.6. Specify - the keyword value explicitly to avoid the :exc:`DeprecationWarning`. - To use a custom SMTPChannel implementation you need to override the :attr:`SMTPServer.channel_class` of your :class:`SMTPServer`. - .. versionchanged:: 3.5 - the *decode_data* argument was added. - The :class:`SMTPChannel` has the following instance variables: .. attribute:: smtp_server diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/smtplib.rst --- a/Doc/library/smtplib.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/smtplib.rst Thu Jun 05 21:36:59 2014 +0200 @@ -32,8 +32,7 @@ than a success code, an :exc:`SMTPConnectError` is raised. The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, the global default timeout - setting will be used). If the timeout expires, :exc:`socket.timeout` is - raised. The optional source_address parameter allows to bind + setting will be used). The optional source_address parameter allows to bind to some specific source address in a machine with multiple network interfaces, and/or to some specific source TCP port. It takes a 2-tuple (host, port), for the socket to bind to as its source address before diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/socket.rst --- a/Doc/library/socket.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/socket.rst Thu Jun 05 21:36:59 2014 +0200 @@ -1148,21 +1148,6 @@ .. versionadded:: 3.3 -.. method:: socket.sendfile(file, offset=0, count=None) - - Send a file until EOF is reached by using high-performance - :mod:`os.sendfile` and return the total number of bytes which were sent. - *file* must be a regular file object opened in binary mode. If - :mod:`os.sendfile` is not available (e.g. Windows) or *file* is not a - regular file :meth:`send` will be used instead. *offset* tells from where to - start reading the file. If specified, *count* is the total number of bytes - to transmit as opposed to sending the file until EOF is reached. File - position is updated on return or also in case of error in which case - :meth:`file.tell() ` can be used to figure out the number of - bytes which were sent. The socket must be of :const:`SOCK_STREAM` type. Non- - blocking sockets are not supported. - - .. versionadded:: 3.5 .. method:: socket.set_inheritable(inheritable) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/sqlite3.rst Thu Jun 05 21:36:59 2014 +0200 @@ -646,7 +646,7 @@ .. method:: keys - This method returns a list of column names. Immediately after a query, + This method returns a tuple of column names. Immediately after a query, it is the first member of each tuple in :attr:`Cursor.description`. Let's assume we initialize a table as in the example given above:: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/ssl.rst --- a/Doc/library/ssl.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/ssl.rst Thu Jun 05 21:36:59 2014 +0200 @@ -789,9 +789,6 @@ (but passing a non-zero ``flags`` argument is not allowed) - :meth:`~socket.socket.send()`, :meth:`~socket.socket.sendall()` (with the same limitation) -- :meth:`~socket.socket.sendfile()` (but :mod:`os.sendfile` will be used - for plain-text sockets only, else :meth:`~socket.socket.send()` will be used) - .. versionadded:: 3.5 - :meth:`~socket.socket.shutdown()` However, since the SSL (and TLS) protocol has its own framing atop diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/stdtypes.rst Thu Jun 05 21:36:59 2014 +0200 @@ -3031,8 +3031,8 @@ If no positional argument is given, an empty dictionary is created. If a positional argument is given and it is a mapping object, a dictionary is created with the same key-value pairs as the mapping object. Otherwise, - the positional argument must be an :term:`iterable` object. Each item in - the iterable must itself be an iterable with exactly two objects. The + the positional argument must be an :term:`iterator` object. Each item in + the iterable must itself be an iterator with exactly two objects. The first object of each item becomes a key in the new dictionary, and the second object the corresponding value. If a key occurs more than once, the last value for that key becomes the corresponding value in the new diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/tarfile.rst --- a/Doc/library/tarfile.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/tarfile.rst Thu Jun 05 21:36:59 2014 +0200 @@ -81,10 +81,6 @@ If *fileobj* is specified, it is used as an alternative to a :term:`file object` opened in binary mode for *name*. It is supposed to be at position 0. - For modes ``'w:gz'``, ``'r:gz'``, ``'w:bz2'``, ``'r:bz2'``, :func:`tarfile.open` - accepts the keyword argument *compresslevel* to specify the compression level of - the file. - For special purposes, there is a second format for *mode*: ``'filemode|[compression]'``. :func:`tarfile.open` will return a :class:`TarFile` object that processes its data as a stream of blocks. No random seeking will diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/tempfile.rst --- a/Doc/library/tempfile.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/tempfile.rst Thu Jun 05 21:36:59 2014 +0200 @@ -54,13 +54,6 @@ underlying true file object. This file-like object can be used in a :keyword:`with` statement, just like a normal file. - The :py:data:`os.O_TMPFILE` flag is used if it is available and works - (Linux-specific, require Linux kernel 3.11 or later). - - .. versionchanged:: 3.5 - - The :py:data:`os.O_TMPFILE` flag is now used if available. - .. function:: NamedTemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix='', prefix='tmp', dir=None, delete=True) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/library/tokenize.rst --- a/Doc/library/tokenize.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/library/tokenize.rst Thu Jun 05 21:36:59 2014 +0200 @@ -131,24 +131,6 @@ .. versionadded:: 3.2 -.. exception:: TokenError - - Raised when either a docstring or expression that may be split over several - lines is not completed anywhere in the file, for example:: - - """Beginning of - docstring - - or:: - - [1, - 2, - 3 - -Note that unclosed single-quoted strings do not cause an error to be -raised. They are tokenized as ``ERRORTOKEN``, followed by the tokenization of -their contents. - .. _tokenize-cli: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/make.bat --- a/Doc/make.bat Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/make.bat Thu Jun 05 21:36:59 2014 +0200 @@ -76,15 +76,6 @@ cmd /C %SPHINXBUILD% %SPHINXOPTS% -b%1 -dbuild\doctrees . %BUILDDIR%\%* if "%1" EQU "htmlhelp" ( - if not exist "%HTMLHELP%" ( - echo. - echo.The HTML Help Workshop was not found. Set the HTMLHELP variable - echo.to the path to hhc.exe or download and install it from - echo.http://msdn.microsoft.com/en-us/library/ms669985 - rem Set errorlevel to 1 and exit - cmd /C exit /b 1 - goto end - ) cmd /C "%HTMLHELP%" build\htmlhelp\python%DISTVERSION:.=%.hhp rem hhc.exe seems to always exit with code 1, reset to 0 for less than 2 if not errorlevel 2 cmd /C exit /b 0 diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/reference/compound_stmts.rst Thu Jun 05 21:36:59 2014 +0200 @@ -180,7 +180,7 @@ for i in range(10): print(i) i = 5 # this will not affect the for-loop - # because i will be overwritten with the next + # be i will be overwritten with the next # index in the range diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/reference/datamodel.rst Thu Jun 05 21:36:59 2014 +0200 @@ -285,17 +285,16 @@ single: integer single: Unicode - A string is a sequence of values that represent Unicode code points. - All the code points in the range ``U+0000 - U+10FFFF`` can be - represented in a string. Python doesn't have a :c:type:`char` type; - instead, every code point in the string is represented as a string - object with length ``1``. The built-in function :func:`ord` - converts a code point from its string form to an integer in the - range ``0 - 10FFFF``; :func:`chr` converts an integer in the range - ``0 - 10FFFF`` to the corresponding length ``1`` string object. + A string is a sequence of values that represent Unicode codepoints. + All the codepoints in range ``U+0000 - U+10FFFF`` can be represented + in a string. Python doesn't have a :c:type:`chr` type, and + every character in the string is represented as a string object + with length ``1``. The built-in function :func:`ord` converts a + character to its codepoint (as an integer); :func:`chr` converts + an integer in range ``0 - 10FFFF`` to the corresponding character. :meth:`str.encode` can be used to convert a :class:`str` to - :class:`bytes` using the given text encoding, and - :meth:`bytes.decode` can be used to achieve the opposite. + :class:`bytes` using the given encoding, and :meth:`bytes.decode` can + be used to achieve the opposite. Tuples .. index:: @@ -2107,9 +2106,9 @@ .. note:: - In order to have a coherent integer type class, when :meth:`__index__` is - defined :meth:`__int__` should also be defined, and both should return - the same value. + When :meth:`__index__` is defined, :meth:`__int__` should also be defined, + and both shuld return the same value, in order to have a coherent integer + type class. .. _context-managers: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/reference/expressions.rst Thu Jun 05 21:36:59 2014 +0200 @@ -522,7 +522,7 @@ The primary must evaluate to an object of a type that supports attribute references, which most objects do. This object is then asked to produce the attribute whose name is the identifier. This production can be customized by -overriding the :meth:`__getattr__` method. If this attribute is not available, +overriding the :meth:`__getattr__` method). If this attribute is not available, the exception :exc:`AttributeError` is raised. Otherwise, the type and value of the object produced is determined by the object. Multiple evaluations of the same attribute reference may yield different objects. @@ -1253,7 +1253,7 @@ lambda_expr: "lambda" [`parameter_list`]: `expression` lambda_expr_nocond: "lambda" [`parameter_list`]: `expression_nocond` -Lambda expressions (sometimes called lambda forms) are used to create anonymous +Lambda expressions (sometimes called lambda forms) are create anonymous functions. The expression ``lambda arguments: expression`` yields a function object. The unnamed object behaves like a function object defined with :: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/tools/sphinxext/indexsidebar.html --- a/Doc/tools/sphinxext/indexsidebar.html Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/tools/sphinxext/indexsidebar.html Thu Jun 05 21:36:59 2014 +0200 @@ -1,17 +1,17 @@ -

Download

-

Download these documents

-

Docs for other versions

- +

Download

+

Download these documents

+

Docs for other versions

+ -

Other resources

- +

Other resources

+ diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/whatsnew/2.7.rst Thu Jun 05 21:36:59 2014 +0200 @@ -7,6 +7,7 @@ .. hyperlink all the methods & functions. .. T_STRING_INPLACE not described in main docs +.. "Format String Syntax" in string.rst could use many more examples. .. $Id$ Rules for maintenance: @@ -49,16 +50,17 @@ This saves the maintainer some effort going through the SVN logs when researching a change. -This article explains the new features in Python 2.7. Python 2.7 was released -on July 3, 2010. +This article explains the new features in Python 2.7. The final +release of 2.7 is currently scheduled for July 2010; the detailed +schedule is described in :pep:`373`. Numeric handling has been improved in many ways, for both -floating-point numbers and for the :class:`~decimal.Decimal` class. -There are some useful additions to the standard library, such as a -greatly enhanced :mod:`unittest` module, the :mod:`argparse` module -for parsing command-line options, convenient :class:`~collections.OrderedDict` -and :class:`~collections.Counter` classes in the :mod:`collections` module, -and many other improvements. +floating-point numbers and for the :class:`Decimal` class. There are +some useful additions to the standard library, such as a greatly +enhanced :mod:`unittest` module, the :mod:`argparse` module for +parsing command-line options, convenient ordered-dictionary and +:class:`Counter` classes in the :mod:`collections` module, and many +other improvements. Python 2.7 is planned to be the last of the 2.x releases, so we worked on making it a good release for the long term. To help with porting @@ -79,91 +81,45 @@ The Future for Python 2.x ========================= -Python 2.7 is the last major release in the 2.x series, as the Python -maintainers have shifted the focus of their new feature development efforts -to the Python 3.x series. This means that while Python 2 continues to -receive bug fixes, and to be updated to build correctly on new hardware and -versions of supported operated systems, there will be no new full feature -releases for the language or standard library. - -However, while there is a large common subset between Python 2.7 and Python -3, and many of the changes involved in migrating to that common subset, or -directly to Python 3, can be safely automated, some other changes (notably -those associated with Unicode handling) may require careful consideration, -and preferably robust automated regression test suites, to migrate -effectively. - -This means that Python 2.7 will remain in place for a long time, providing a -stable and supported base platform for production systems that have not yet -been ported to Python 3. The full expected lifecycle of the Python 2.7 -series is detailed in :pep:`373`. - -Some key consequences of the long-term significance of 2.7 are: - -* As noted above, the 2.7 release has a much longer period of maintenance - when compared to earlier 2.x versions. Python 2.7 is currently expected to - remain supported by the core development team (receiving security updates - and other bug fixes) until at least 2020 (10 years after its initial - release, compared to the more typical support period of 18-24 months). - -* As the Python 2.7 standard library ages, making effective use of the - Python Package Index (either directly or via a redistributor) becomes - more important for Python 2 users. In addition to a wide variety of third - party packages for various tasks, the available packages include backports - of new modules and features from the Python 3 standard library that are - compatible with Python 2, as well as various tools and libraries that can - make it easier to migrate to Python 3. The `Python Packaging User Guide - `__ provides guidance on downloading and - installing software from the Python Package Index. - -* While the preferred approach to enhancing Python 2 is now the publication - of new packages on the Python Package Index, this approach doesn't - necessarily work in all cases, especially those related to network - security. In exceptional cases that cannot be handled adequately by - publishing new or updated packages on PyPI, the Python Enhancement - Proposal process may be used to make the case for adding new features - directly to the Python 2 standard library. Any such additions, and the - maintenance releases where they were added, will be noted in the - :ref:`py27-maintenance-enhancements` section below. - -For projects wishing to migrate from Python 2 to Python 3, or for library -and framework developers wishing to support users on both Python 2 and -Python 3, there are a variety of tools and guides available to help decide -on a suitable approach and manage some of the technical details involved. -The recommended starting point is the :ref:`pyporting-howto` HOWTO guide. - - -Changes to the Handling of Deprecation Warnings -=============================================== - -For Python 2.7, a policy decision was made to silence warnings only of -interest to developers by default. :exc:`DeprecationWarning` and its -descendants are now ignored unless otherwise requested, preventing -users from seeing warnings triggered by an application. This change -was also made in the branch that became Python 3.2. (Discussed -on stdlib-sig and carried out in :issue:`7319`.) - -In previous releases, :exc:`DeprecationWarning` messages were -enabled by default, providing Python developers with a clear -indication of where their code may break in a future major version -of Python. - -However, there are increasingly many users of Python-based -applications who are not directly involved in the development of -those applications. :exc:`DeprecationWarning` messages are -irrelevant to such users, making them worry about an application -that's actually working correctly and burdening application developers -with responding to these concerns. - -You can re-enable display of :exc:`DeprecationWarning` messages by -running Python with the :option:`-Wdefault <-W>` (short form: -:option:`-Wd <-W>`) switch, or by setting the :envvar:`PYTHONWARNINGS` -environment variable to ``"default"`` (or ``"d"``) before running -Python. Python code can also re-enable them -by calling ``warnings.simplefilter('default')``. - -The ``unittest`` module also automatically reenables deprecation warnings -when running tests. +Python 2.7 is intended to be the last major release in the 2.x series. +The Python maintainers are planning to focus their future efforts on +the Python 3.x series. + +This means that 2.7 will remain in place for a long time, running +production systems that have not been ported to Python 3.x. +Two consequences of the long-term significance of 2.7 are: + +* It's very likely the 2.7 release will have a longer period of + maintenance compared to earlier 2.x versions. Python 2.7 will + continue to be maintained while the transition to 3.x continues, and + the developers are planning to support Python 2.7 with bug-fix + releases beyond the typical two years. + +* A policy decision was made to silence warnings only of interest to + developers. :exc:`DeprecationWarning` and its + descendants are now ignored unless otherwise requested, preventing + users from seeing warnings triggered by an application. This change + was also made in the branch that will become Python 3.2. (Discussed + on stdlib-sig and carried out in :issue:`7319`.) + + In previous releases, :exc:`DeprecationWarning` messages were + enabled by default, providing Python developers with a clear + indication of where their code may break in a future major version + of Python. + + However, there are increasingly many users of Python-based + applications who are not directly involved in the development of + those applications. :exc:`DeprecationWarning` messages are + irrelevant to such users, making them worry about an application + that's actually working correctly and burdening application developers + with responding to these concerns. + + You can re-enable display of :exc:`DeprecationWarning` messages by + running Python with the :option:`-Wdefault` (short form: + :option:`-Wd`) switch, or by setting the :envvar:`PYTHONWARNINGS` + environment variable to ``"default"`` (or ``"d"``) before running + Python. Python code can also re-enable them + by calling ``warnings.simplefilter('default')``. Python 3.1 Features @@ -177,7 +133,7 @@ A partial list of 3.1 features that were backported to 2.7: * The syntax for set literals (``{1,2,3}`` is a mutable set). -* Dictionary and set comprehensions (``{i: i*2 for i in range(3)}``). +* Dictionary and set comprehensions (``{ i: i*2 for i in range(3)}``). * Multiple context managers in a single :keyword:`with` statement. * A new version of the :mod:`io` library, rewritten in C for performance. * The ordered-dictionary type described in :ref:`pep-0372`. @@ -199,7 +155,7 @@ * :func:`operator.isCallable` and :func:`operator.sequenceIncludes`, which are not supported in 3.x, now trigger warnings. * The :option:`-3` switch now automatically - enables the :option:`-Qwarn <-Q>` switch that causes warnings + enables the :option:`-Qwarn` switch that causes warnings about using classic division with integers and long integers. @@ -434,10 +390,9 @@ .. seealso:: - :mod:`argparse` documentation - The documentation page of the argparse module. - - :ref:`upgrading-optparse-code` + `argparse module documentation `__ + + `Upgrading optparse code to use argparse `__ Part of the Python documentation, describing how to convert code that uses :mod:`optparse`. @@ -447,6 +402,8 @@ PEP 391: Dictionary-Based Configuration For Logging ==================================================== +.. XXX not documented in library reference yet; add link here once it's added. + The :mod:`logging` module is very flexible; applications can define a tree of logging subsystems, and each logger in this tree can filter out certain messages, format them differently, and direct messages to @@ -455,21 +412,21 @@ All this flexibility can require a lot of configuration. You can write Python statements to create objects and set their properties, but a complex set-up requires verbose but boring code. -:mod:`logging` also supports a :func:`~logging.fileConfig` +:mod:`logging` also supports a :func:`~logging.config.fileConfig` function that parses a file, but the file format doesn't support configuring filters, and it's messier to generate programmatically. -Python 2.7 adds a :func:`~logging.dictConfig` function that +Python 2.7 adds a :func:`~logging.config.dictConfig` function that uses a dictionary to configure logging. There are many ways to produce a dictionary from different sources: construct one with code; parse a file containing JSON; or use a YAML parsing library if one is -installed. For more information see :ref:`logging-config-api`. +installed. The following example configures two loggers, the root logger and a -logger named "network". Messages sent to the root logger will be +logger named "network". Messages sent to the root logger will be sent to the system log using the syslog protocol, and messages to the "network" logger will be written to a :file:`network.log` file -that will be rotated once the log reaches 1MB. +that will be rotated once the log reaches 1Mb. :: @@ -488,7 +445,7 @@ 'filename': '/logs/network.log', 'formatter': 'standard', 'level': 'INFO', - 'maxBytes': 1000000}, + 'maxBytes': 1024*1024}, 'syslog': {'class': 'logging.handlers.SysLogHandler', 'formatter': 'standard', 'level': 'ERROR'}}, @@ -526,19 +483,16 @@ for UDP or :const:`socket.SOCK_STREAM` for TCP. The default protocol remains UDP. -* :class:`~logging.Logger` instances gained a :meth:`~logging.Logger.getChild` - method that retrieves a descendant logger using a relative path. - For example, once you retrieve a logger by doing ``log = getLogger('app')``, +* :class:`Logger` instances gained a :meth:`getChild` method that retrieves a + descendant logger using a relative path. For example, + once you retrieve a logger by doing ``log = getLogger('app')``, calling ``log.getChild('network.listen')`` is equivalent to ``getLogger('app.network.listen')``. -* The :class:`~logging.LoggerAdapter` class gained a - :meth:`~logging.LoggerAdapter.isEnabledFor` method that takes a - *level* and returns whether the underlying logger would +* The :class:`LoggerAdapter` class gained a :meth:`isEnabledFor` method + that takes a *level* and returns whether the underlying logger would process a message of that level of importance. -.. XXX: Logger objects don't have a class declaration so the link don't work - .. seealso:: :pep:`391` - Dictionary-Based Configuration For Logging @@ -547,15 +501,14 @@ PEP 3106: Dictionary Views ==================================================== -The dictionary methods :meth:`~dict.keys`, :meth:`~dict.values`, and -:meth:`~dict.items` are different in Python 3.x. They return an object -called a :dfn:`view` instead of a fully materialized list. - -It's not possible to change the return values of :meth:`~dict.keys`, -:meth:`~dict.values`, and :meth:`~dict.items` in Python 2.7 because -too much code would break. Instead the 3.x versions were added -under the new names :meth:`~dict.viewkeys`, :meth:`~dict.viewvalues`, -and :meth:`~dict.viewitems`. +The dictionary methods :meth:`keys`, :meth:`values`, and :meth:`items` +are different in Python 3.x. They return an object called a :dfn:`view` +instead of a fully materialized list. + +It's not possible to change the return values of :meth:`keys`, +:meth:`values`, and :meth:`items` in Python 2.7 because too much code +would break. Instead the 3.x versions were added under the new names +:meth:`viewkeys`, :meth:`viewvalues`, and :meth:`viewitems`. :: @@ -597,8 +550,8 @@ RuntimeError: dictionary changed size during iteration You can use the view methods in Python 2.x code, and the 2to3 -converter will change them to the standard :meth:`~dict.keys`, -:meth:`~dict.values`, and :meth:`~dict.items` methods. +converter will change them to the standard :meth:`keys`, +:meth:`values`, and :meth:`items` methods. .. seealso:: @@ -671,7 +624,7 @@ ``{}`` continues to represent an empty dictionary; use ``set()`` for an empty set. - >>> {1, 2, 3, 4, 5} + >>> {1,2,3,4,5} set([1, 2, 3, 4, 5]) >>> set() # empty set set([]) @@ -841,7 +794,7 @@ ``None`` as its first argument. (Fixed by Georg Brandl; :issue:`4759`.) - .. XXX bytearray doesn't seem to be documented + .. bytearray doesn't seem to be documented * When using ``@classmethod`` and ``@staticmethod`` to wrap methods as class or static methods, the wrapper object now @@ -914,6 +867,12 @@ Several performance enhancements have been added: +.. * A new :program:`configure` option, :option:`--with-computed-gotos`, + compiles the main bytecode interpreter loop using a new dispatch + mechanism that gives speedups of up to 20%, depending on the system + and benchmark. The new mechanism is only supported on certain + compilers, such as gcc, SunPro, and icc. + * A new opcode was added to perform the initial setup for :keyword:`with` statements, looking up the :meth:`__enter__` and :meth:`__exit__` methods. (Contributed by Benjamin Peterson.) @@ -1095,7 +1054,7 @@ :meth:`~collections.deque.count` method that returns the number of contained elements equal to the supplied argument *x*, and a :meth:`~collections.deque.reverse` method that reverses the elements - of the deque in-place. :class:`~collections.deque` also exposes its maximum + of the deque in-place. :class:`deque` also exposes its maximum length as the read-only :attr:`~collections.deque.maxlen` attribute. (Both features added by Raymond Hettinger.) @@ -1176,14 +1135,15 @@ ``Decimal('0.1000000000000000055511151231257827021181583404541015625')``. (Implemented by Raymond Hettinger; :issue:`4796`.) - Comparing instances of :class:`~decimal.Decimal` with floating-point + Comparing instances of :class:`Decimal` with floating-point numbers now produces sensible results based on the numeric values of the operands. Previously such comparisons would fall back to Python's default rules for comparing objects, which produced arbitrary results based on their type. Note that you still cannot combine :class:`Decimal` and floating-point in other operations such as addition, since you should be explicitly choosing how to convert between float and - :class:`~decimal.Decimal`. (Fixed by Mark Dickinson; :issue:`2531`.) + :class:`Decimal`. + (Fixed by Mark Dickinson; :issue:`2531`.) The constructor for :class:`~decimal.Decimal` now accepts floating-point numbers (added by Raymond Hettinger; :issue:`8257`) @@ -1235,8 +1195,8 @@ Ordering comparisons (``<``, ``<=``, ``>``, ``>=``) between fractions and complex numbers now raise a :exc:`TypeError`. - This fixes an oversight, making the :class:`~fractions.Fraction` - match the other numeric types. + This fixes an oversight, making the :class:`Fraction` match the other + numeric types. .. revision 79455 @@ -1250,7 +1210,7 @@ uploads thanks to an added *rest* parameter (patch by Pablo Mouzo; :issue:`6845`.) -* New class decorator: :func:`~functools.total_ordering` in the :mod:`functools` +* New class decorator: :func:`total_ordering` in the :mod:`functools` module takes a class that defines an :meth:`__eq__` method and one of :meth:`__lt__`, :meth:`__le__`, :meth:`__gt__`, or :meth:`__ge__`, and generates the missing comparison methods. Since the @@ -1258,7 +1218,7 @@ this decorator makes it easier to define ordered classes. (Added by Raymond Hettinger; :issue:`5479`.) - New function: :func:`~functools.cmp_to_key` will take an old-style comparison + New function: :func:`cmp_to_key` will take an old-style comparison function that expects two arguments and return a new callable that can be used as the *key* parameter to functions such as :func:`sorted`, :func:`min` and :func:`max`, etc. The primary @@ -1385,7 +1345,7 @@ with any object literal that decodes to a list of pairs. (Contributed by Raymond Hettinger; :issue:`5381`.) -* The :mod:`mailbox` module's :class:`~mailbox.Maildir` class now records the +* The :mod:`mailbox` module's :class:`Maildir` class now records the timestamp on the directories it reads, and only re-reads them if the modification time has subsequently changed. This improves performance by avoiding unneeded directory scans. (Fixed by @@ -1472,7 +1432,7 @@ * The :mod:`signal` module no longer re-installs the signal handler unless this is truly necessary, which fixes a bug that could make it impossible to catch the EINTR signal robustly. (Fixed by - Charles-Francois Natali; :issue:`8354`.) + Charles-François Natali; :issue:`8354`.) * New functions: in the :mod:`site` module, three new functions return various site- and user-specific paths. @@ -1506,10 +1466,10 @@ defaults to False; if overridden to be True, new request connections will have the TCP_NODELAY option set to prevent buffering many small sends into a single TCP packet. - The :attr:`~SocketServer.BaseServer.timeout` class attribute can hold + The :attr:`~SocketServer.TCPServer.timeout` class attribute can hold a timeout in seconds that will be applied to the request socket; if - no request is received within that time, :meth:`~SocketServer.BaseServer.handle_timeout` - will be called and :meth:`~SocketServer.BaseServer.handle_request` will return. + no request is received within that time, :meth:`handle_timeout` + will be called and :meth:`handle_request` will return. (Contributed by Kristján Valur Jónsson; :issue:`6192` and :issue:`6267`.) * Updated module: the :mod:`sqlite3` module has been updated to @@ -1519,7 +1479,7 @@ and then call :meth:`~sqlite3.Connection.load_extension` to load a particular shared library. (Updated by Gerhard Häring.) -* The :mod:`ssl` module's :class:`~ssl.SSLSocket` objects now support the +* The :mod:`ssl` module's :class:`ssl.SSLSocket` objects now support the buffer API, which fixed a test suite failure (fix by Antoine Pitrou; :issue:`7133`) and automatically set OpenSSL's :c:macro:`SSL_MODE_AUTO_RETRY`, which will prevent an error @@ -1575,7 +1535,7 @@ on receiving an :const:`EINTR` signal. (Reported by several people; final patch by Gregory P. Smith in :issue:`1068268`.) -* New function: :func:`~symtable.Symbol.is_declared_global` in the :mod:`symtable` module +* New function: :func:`~symtable.is_declared_global` in the :mod:`symtable` module returns true for variables that are explicitly declared to be global, false for ones that are implicitly global. (Contributed by Jeremy Hylton.) @@ -1756,7 +1716,7 @@ Makefile and the :file:`pyconfig.h` file. * :func:`~sysconfig.get_config_vars` returns a dictionary containing all of the configuration variables. -* :func:`~sysconfig.get_path` returns the configured path for +* :func:`~sysconfig.getpath` returns the configured path for a particular type of module: the standard library, site-specific modules, platform-specific modules, etc. * :func:`~sysconfig.is_python_build` returns true if you're running a @@ -1818,7 +1778,7 @@ Consult the :mod:`unittest` module documentation for more details. (Developed in :issue:`6001`.) -The :func:`~unittest.main` function supports some other new options: +The :func:`main` function supports some other new options: * :option:`-b` or :option:`--buffer` will buffer the standard output and standard error streams during each test. If the test passes, @@ -1836,7 +1796,7 @@ being tested or the tests being run have defined a signal handler of their own, by noticing that a signal handler was already set and calling it. If this doesn't work for you, there's a - :func:`~unittest.removeHandler` decorator that can be used to mark tests that + :func:`removeHandler` decorator that can be used to mark tests that should have the control-C handling disabled. * :option:`-f` or :option:`--failfast` makes @@ -1963,7 +1923,7 @@ :func:`unittest.main` now takes an optional ``exit`` argument. If False, :func:`~unittest.main` doesn't call :func:`sys.exit`, allowing -:func:`~unittest.main` to be used from the interactive interpreter. +:func:`main` to be used from the interactive interpreter. (Contributed by J. Pablo Fernández; :issue:`3379`.) :class:`~unittest.TestResult` has new :meth:`~unittest.TestResult.startTestRun` and @@ -2160,7 +2120,7 @@ :c:macro:`Py_ISSPACE`, :c:macro:`Py_ISUPPER`, :c:macro:`Py_ISXDIGIT`, - :c:macro:`Py_TOLOWER`, and :c:macro:`Py_TOUPPER`. + and :c:macro:`Py_TOLOWER`, :c:macro:`Py_TOUPPER`. All of these functions are analogous to the C standard macros for classifying characters, but ignore the current locale setting, because in @@ -2306,11 +2266,11 @@ (Contributed by David Cournapeau; :issue:`4365`.) * The :mod:`_winreg` module for accessing the registry now implements - the :func:`~_winreg.CreateKeyEx` and :func:`~_winreg.DeleteKeyEx` - functions, extended versions of previously-supported functions that - take several extra arguments. The :func:`~_winreg.DisableReflectionKey`, - :func:`~_winreg.EnableReflectionKey`, and :func:`~_winreg.QueryReflectionKey` - were also tested and documented. + the :func:`CreateKeyEx` and :func:`DeleteKeyEx` functions, extended + versions of previously-supported functions that take several extra + arguments. The :func:`DisableReflectionKey`, + :func:`EnableReflectionKey`, and :func:`QueryReflectionKey` were also + tested and documented. (Implemented by Brian Curtin: :issue:`7347`.) * The new :c:func:`_beginthreadex` API is used to start threads, and @@ -2369,7 +2329,7 @@ attributes of the resulting code objects are overwritten when the original filename is obsolete. This can happen if the file has been renamed, moved, or is accessed through different paths. (Patch by - Ziga Seilnacht and Jean-Paul Calderone; :issue:`1180193`.) + Žiga Seilnacht and Jean-Paul Calderone; :issue:`1180193`.) * The :file:`regrtest.py` script now takes a :option:`--randseed=` switch that takes an integer that will be used as the random seed @@ -2427,20 +2387,20 @@ In the standard library: -* Operations with :class:`~datetime.datetime` instances that resulted in a year +* Operations with :class:`datetime` instances that resulted in a year falling outside the supported range didn't always raise :exc:`OverflowError`. Such errors are now checked more carefully and will now raise the exception. (Reported by Mark Leander, patch by Anand B. Pillai and Alexander Belopolsky; :issue:`7150`.) -* When using :class:`~decimal.Decimal` instances with a string's +* When using :class:`Decimal` instances with a string's :meth:`format` method, the default alignment was previously left-alignment. This has been changed to right-alignment, which might change the output of your programs. (Changed by Mark Dickinson; :issue:`6857`.) Comparisons involving a signaling NaN value (or ``sNAN``) now signal - :const:`~decimal.InvalidOperation` instead of silently returning a true or + :const:`InvalidOperation` instead of silently returning a true or false value depending on the comparison operator. Quiet NaN values (or ``NaN``) are now hashable. (Fixed by Mark Dickinson; :issue:`7279`.) @@ -2451,7 +2411,7 @@ or comment (which looks like ``). (Patch by Neil Muller; :issue:`2746`.) -* The :meth:`~StringIO.StringIO.readline` method of :class:`~StringIO.StringIO` objects now does +* The :meth:`readline` method of :class:`StringIO` objects now does nothing when a negative length is requested, as other file-like objects do. (:issue:`7348`). @@ -2510,54 +2470,6 @@ .. ====================================================================== -.. _py27-maintenance-enhancements: - -New Features Added to Python 2.7 Maintenance Releases -===================================================== - -New features may be added to Python 2.7 maintenance releases when the -situation genuinely calls for it. Any such additions must go through -the Python Enhancement Proposal process, and make a compelling case for why -they can't be adequately addressed by either adding the new feature solely to -Python 3, or else by publishing it on the Python Package Index. - -In addition to the specific proposals listed below, there is a general -exemption allowing new ``-3`` warnings to be added in any Python 2.7 -maintenance release. - - -PEP 434: IDLE Enhancement Exception for All Branches ----------------------------------------------------- - -:pep:`434` describes a general exemption for changes made to the IDLE -development environment shipped along with Python. This exemption makes it -possible for the IDLE developers to provide a more consistent user -experience across all supported versions of Python 2 and 3. - -For details of any IDLE changes, refer to the NEWS file for the specific -release. - - -PEP 466: Network Security Enhancements for Python 2.7 ------------------------------------------------------ - -:pep:`466` describes a number of network security enhancement proposals -that have been approved for inclusion in Python 2.7 maintenance releases, -with the first of those changes appearing in the Python 2.7.7 release. - -:pep:`466` related features added in Python 2.7.7: - -* :func:`hmac.compare_digest` was added to make a timing attack resistant - comparison operation broadly available to Python 2 applications - (backported by Alex Gaynor in :issue:`21306`) - -* the version of OpenSSL linked with the prebuilt Windows installers - published on python.org was updated to 1.0.1g (contributed by - Zachary Ware in :issue:`21462`) - - -.. ====================================================================== - .. _acks27: Acknowledgements diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst Wed Jun 11 12:27:40 2014 -0400 +++ b/Doc/whatsnew/3.5.rst Thu Jun 05 21:36:59 2014 +0200 @@ -181,28 +181,9 @@ * Different constants of :mod:`signal` module are now enumeration values using the :mod:`enum` module. This allows meaningful names to be printed during - debugging, instead of integer “magic numbers”. (contributed by Giampaolo + debugging, instead of integer “magic numbers”. (contribute by Giampaolo Rodola' in :issue:`21076`) -smtpd ------ - -* Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a - *decode_data* keyword to determine if the DATA portion of the SMTP - transaction is decoded using the ``utf-8`` codec or is instead provided to - :meth:`~smtpd.SMTPServer.process_message` as a byte string. The default - is ``True`` for backward compatibility reasons, but will change to ``False`` - in Python 3.6. (Contributed by Maciej Szulik in :issue:`19662`.) - -socket ------- - -* New :meth:`socket.socket.sendfile` method allows to send a file over a socket - by using high-performance :func:`os.sendfile` function on UNIX resulting in - uploads being from 2x to 3x faster than when using plain - :meth:`socket.socket.send`. - (contributed by Giampaolo Rodola' in :issue:`17552`) - xmlrpc ------ @@ -215,8 +196,9 @@ The following performance enhancements have been added: -* Construction of ``bytes(int)`` (filled by zero bytes) is faster and use less - memory for large objects. ``calloc()`` is used instead of ``malloc()`` to +* Construction of ``bytes(int)`` and ``bytearray(int)`` (filled by zero bytes) + is faster and use less memory (until the bytearray buffer is filled with + data) for large objects. ``calloc()`` is used instead of ``malloc()`` to allocate memory for these objects. * Some operations on :class:`~ipaddress.IPv4Network` and @@ -255,12 +237,6 @@ * The :mod:`formatter` module has now graduated to full deprecation and is still slated for removal in Python 3.6. -* :mod:`smtpd` has in the past always decoded the DATA portion of email - messages using the ``utf-8`` codec. This can now be controlled by the new - *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is - ``True``, but this default is deprecated. Specify the *decode_data* keyword - with an appropriate value to avoid the deprecation warning. - Deprecated functions and types of the C API ------------------------------------------- @@ -295,5 +271,4 @@ Changes in the C API -------------------- -* The :c:type:`PyMemAllocator` structure was renamed to - :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. +* The :c:type:`PyMemAllocator` structure has a new ``calloc`` field. diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Include/pymem.h --- a/Include/pymem.h Wed Jun 11 12:27:40 2014 -0400 +++ b/Include/pymem.h Thu Jun 05 21:36:59 2014 +0200 @@ -128,7 +128,7 @@ } PyMemAllocatorDomain; typedef struct { - /* user context passed as the first argument to the 4 functions */ + /* user context passed as the first argument to the 3 functions */ void *ctx; /* allocate a memory block */ @@ -142,11 +142,11 @@ /* release a memory block */ void (*free) (void *ctx, void *ptr); -} PyMemAllocatorEx; +} PyMemAllocator; /* Get the memory block allocator of the specified domain. */ PyAPI_FUNC(void) PyMem_GetAllocator(PyMemAllocatorDomain domain, - PyMemAllocatorEx *allocator); + PyMemAllocator *allocator); /* Set the memory block allocator of the specified domain. @@ -160,7 +160,7 @@ PyMem_SetupDebugHooks() function must be called to reinstall the debug hooks on top on the new allocator. */ PyAPI_FUNC(void) PyMem_SetAllocator(PyMemAllocatorDomain domain, - PyMemAllocatorEx *allocator); + PyMemAllocator *allocator); /* Setup hooks to detect bugs in the following Python memory allocator functions: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/_pyio.py --- a/Lib/_pyio.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/_pyio.py Thu Jun 05 21:36:59 2014 +0200 @@ -200,45 +200,38 @@ (appending and "a" or "") + (updating and "+" or ""), closefd, opener=opener) - result = raw - try: - line_buffering = False - if buffering == 1 or buffering < 0 and raw.isatty(): - buffering = -1 - line_buffering = True - if buffering < 0: - buffering = DEFAULT_BUFFER_SIZE - try: - bs = os.fstat(raw.fileno()).st_blksize - except (OSError, AttributeError): - pass - else: - if bs > 1: - buffering = bs - if buffering < 0: - raise ValueError("invalid buffering size") - if buffering == 0: - if binary: - return result - raise ValueError("can't have unbuffered text I/O") - if updating: - buffer = BufferedRandom(raw, buffering) - elif creating or writing or appending: - buffer = BufferedWriter(raw, buffering) - elif reading: - buffer = BufferedReader(raw, buffering) + line_buffering = False + if buffering == 1 or buffering < 0 and raw.isatty(): + buffering = -1 + line_buffering = True + if buffering < 0: + buffering = DEFAULT_BUFFER_SIZE + try: + bs = os.fstat(raw.fileno()).st_blksize + except (OSError, AttributeError): + pass else: - raise ValueError("unknown mode: %r" % mode) - result = buffer + if bs > 1: + buffering = bs + if buffering < 0: + raise ValueError("invalid buffering size") + if buffering == 0: if binary: - return result - text = TextIOWrapper(buffer, encoding, errors, newline, line_buffering) - result = text - text.mode = mode - return result - except: - result.close() - raise + return raw + raise ValueError("can't have unbuffered text I/O") + if updating: + buffer = BufferedRandom(raw, buffering) + elif creating or writing or appending: + buffer = BufferedWriter(raw, buffering) + elif reading: + buffer = BufferedReader(raw, buffering) + else: + raise ValueError("unknown mode: %r" % mode) + if binary: + return buffer + text = TextIOWrapper(buffer, encoding, errors, newline, line_buffering) + text.mode = mode + return text class DocDescriptor: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/asyncio/base_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -119,7 +119,6 @@ class BaseEventLoop(events.AbstractEventLoop): def __init__(self): - self._closed = False self._ready = collections.deque() self._scheduled = [] self._default_executor = None @@ -129,11 +128,6 @@ self._exception_handler = None self._debug = False - def __repr__(self): - return ('<%s running=%s closed=%s debug=%s>' - % (self.__class__.__name__, self.is_running(), - self.is_closed(), self.get_debug())) - def _make_socket_transport(self, sock, protocol, waiter=None, *, extra=None, server=None): """Create socket transport.""" @@ -179,13 +173,8 @@ """Process selector events.""" raise NotImplementedError - def _check_closed(self): - if self._closed: - raise RuntimeError('Event loop is closed') - def run_forever(self): """Run until stop() is called.""" - self._check_closed() if self._running: raise RuntimeError('Event loop is running.') self._running = True @@ -209,7 +198,6 @@ Return the Future's result, or raise its exception. """ - self._check_closed() future = tasks.async(future, loop=self) future.add_done_callback(_raise_stop_error) self.run_forever() @@ -234,9 +222,6 @@ This clears the queues and shuts down the executor, but does not wait for the executor to finish. """ - if self._closed: - return - self._closed = True self._ready.clear() self._scheduled.clear() executor = self._default_executor @@ -244,10 +229,6 @@ self._default_executor = None executor.shutdown(wait=False) - def is_closed(self): - """Returns True if the event loop was closed.""" - return self._closed - def is_running(self): """Returns running status of event loop.""" return self._running @@ -431,10 +412,6 @@ if sock is not None: sock.close() exceptions.append(exc) - except: - if sock is not None: - sock.close() - raise else: break else: @@ -535,10 +512,6 @@ if sock is not None: sock.close() exceptions.append(exc) - except: - if sock is not None: - sock.close() - raise else: break else: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/asyncio/events.py --- a/Lib/asyncio/events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/asyncio/events.py Thu Jun 05 21:36:59 2014 +0200 @@ -355,33 +355,25 @@ """Abstract policy for accessing the event loop.""" def get_event_loop(self): - """Get the event loop for the current context. - - Returns an event loop object implementing the BaseEventLoop interface, - or raises an exception in case no event loop has been set for the - current context and the current policy does not specify to create one. - - It should never return None.""" + """XXX""" raise NotImplementedError def set_event_loop(self, loop): - """Set the event loop for the current context to loop.""" + """XXX""" raise NotImplementedError def new_event_loop(self): - """Create and return a new event loop object according to this - policy's rules. If there's need to set this loop as the event loop for - the current context, set_event_loop must be called explicitly.""" + """XXX""" raise NotImplementedError # Child processes handling (Unix only). def get_child_watcher(self): - "Get the watcher for child processes." + """XXX""" raise NotImplementedError def set_child_watcher(self, watcher): - """Set the watcher for child processes.""" + """XXX""" raise NotImplementedError @@ -455,42 +447,39 @@ def get_event_loop_policy(): - """Get the current event loop policy.""" + """XXX""" if _event_loop_policy is None: _init_event_loop_policy() return _event_loop_policy def set_event_loop_policy(policy): - """Set the current event loop policy. - - If policy is None, the default policy is restored.""" + """XXX""" global _event_loop_policy assert policy is None or isinstance(policy, AbstractEventLoopPolicy) _event_loop_policy = policy def get_event_loop(): - """Equivalent to calling get_event_loop_policy().get_event_loop().""" + """XXX""" return get_event_loop_policy().get_event_loop() def set_event_loop(loop): - """Equivalent to calling get_event_loop_policy().set_event_loop(loop).""" + """XXX""" get_event_loop_policy().set_event_loop(loop) def new_event_loop(): - """Equivalent to calling get_event_loop_policy().new_event_loop().""" + """XXX""" return get_event_loop_policy().new_event_loop() def get_child_watcher(): - """Equivalent to calling get_event_loop_policy().get_child_watcher().""" + """XXX""" return get_event_loop_policy().get_child_watcher() def set_child_watcher(watcher): - """Equivalent to calling - get_event_loop_policy().set_child_watcher(watcher).""" + """XXX""" return get_event_loop_policy().set_child_watcher(watcher) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/asyncio/proactor_events.py --- a/Lib/asyncio/proactor_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/asyncio/proactor_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -353,14 +353,13 @@ sock, protocol, waiter, extra) def close(self): - if self.is_closed(): - return - self._stop_accept_futures() - self._close_self_pipe() - self._proactor.close() - self._proactor = None - self._selector = None - super().close() + if self._proactor is not None: + self._close_self_pipe() + self._proactor.close() + self._proactor = None + self._selector = None + super().close() + self._accept_futures.clear() def sock_recv(self, sock, n): return self._proactor.recv(sock, n) @@ -429,8 +428,6 @@ self._make_socket_transport( conn, protocol, extra={'peername': addr}, server=server) - if self.is_closed(): - return f = self._proactor.accept(sock) except OSError as exc: if sock.fileno() != -1: @@ -451,12 +448,8 @@ def _process_events(self, event_list): pass # XXX hard work currently done in poll - def _stop_accept_futures(self): + def _stop_serving(self, sock): for future in self._accept_futures.values(): future.cancel() - self._accept_futures.clear() - - def _stop_serving(self, sock): - self._stop_accept_futures() self._proactor._stop_serving(sock) sock.close() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/asyncio/selector_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -55,13 +55,11 @@ return _SelectorDatagramTransport(self, sock, protocol, address, extra) def close(self): - if self.is_closed(): - return - self._close_self_pipe() if self._selector is not None: + self._close_self_pipe() self._selector.close() self._selector = None - super().close() + super().close() def _socketpair(self): raise NotImplementedError @@ -145,7 +143,8 @@ def add_reader(self, fd, callback, *args): """Add a reader callback.""" - self._check_closed() + if self._selector is None: + raise RuntimeError('Event loop is closed') handle = events.Handle(callback, args, self) try: key = self._selector.get_key(fd) @@ -161,7 +160,7 @@ def remove_reader(self, fd): """Remove a reader callback.""" - if self.is_closed(): + if self._selector is None: return False try: key = self._selector.get_key(fd) @@ -183,7 +182,8 @@ def add_writer(self, fd, callback, *args): """Add a writer callback..""" - self._check_closed() + if self._selector is None: + raise RuntimeError('Event loop is closed') handle = events.Handle(callback, args, self) try: key = self._selector.get_key(fd) @@ -199,7 +199,7 @@ def remove_writer(self, fd): """Remove a writer callback.""" - if self.is_closed(): + if self._selector is None: return False try: key = self._selector.get_key(fd) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/asyncio/tasks.py Thu Jun 05 21:36:59 2014 +0200 @@ -269,9 +269,9 @@ print(line, file=file, end='') def cancel(self): - """Request this task to cancel itself. + """Request that a task to cancel itself. - This arranges for a CancelledError to be thrown into the + This arranges for a CancellationError to be thrown into the wrapped coroutine on the next cycle through the event loop. The coroutine then has a chance to clean up or even deny the request using try/except/finally. @@ -387,8 +387,6 @@ def wait(fs, *, loop=None, timeout=None, return_when=ALL_COMPLETED): """Wait for the Futures and coroutines given by fs to complete. - The sequence futures must not be empty. - Coroutines will be wrapped in Tasks. Returns two sets of Future: (done, pending). diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/asyncio/unix_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -223,9 +223,6 @@ raise OSError(errno.EADDRINUSE, msg) from None else: raise - except: - sock.close() - raise else: if sock is None: raise ValueError( @@ -822,7 +819,7 @@ self._watcher.attach_loop(loop) def get_child_watcher(self): - """Get the watcher for child processes. + """Get the child watcher If not yet set, a SafeChildWatcher object is automatically created. """ @@ -832,7 +829,7 @@ return self._watcher def set_child_watcher(self, watcher): - """Set the watcher for child processes.""" + """Set the child watcher""" assert watcher is None or isinstance(watcher, AbstractChildWatcher) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/asyncio/windows_utils.py --- a/Lib/asyncio/windows_utils.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/asyncio/windows_utils.py Thu Jun 05 21:36:59 2014 +0200 @@ -51,25 +51,23 @@ # We create a connected TCP socket. Note the trick with setblocking(0) # that prevents us from having to create a thread. lsock = socket.socket(family, type, proto) + lsock.bind((host, 0)) + lsock.listen(1) + # On IPv6, ignore flow_info and scope_id + addr, port = lsock.getsockname()[:2] + csock = socket.socket(family, type, proto) + csock.setblocking(False) try: - lsock.bind((host, 0)) - lsock.listen(1) - # On IPv6, ignore flow_info and scope_id - addr, port = lsock.getsockname()[:2] - csock = socket.socket(family, type, proto) - try: - csock.setblocking(False) - try: - csock.connect((addr, port)) - except (BlockingIOError, InterruptedError): - pass - ssock, _ = lsock.accept() - csock.setblocking(True) - except: - csock.close() - raise - finally: + csock.connect((addr, port)) + except (BlockingIOError, InterruptedError): + pass + except Exception: lsock.close() + csock.close() + raise + ssock, _ = lsock.accept() + csock.setblocking(True) + lsock.close() return (ssock, csock) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/dbm/dumb.py --- a/Lib/dbm/dumb.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/dbm/dumb.py Thu Jun 05 21:36:59 2014 +0200 @@ -44,7 +44,7 @@ _os = _os # for _commit() _io = _io # for _commit() - def __init__(self, filebasename, mode, flag='c'): + def __init__(self, filebasename, mode): self._mode = mode # The directory file is a text file. Each line looks like @@ -64,17 +64,6 @@ # The index is an in-memory dict, mirroring the directory file. self._index = None # maps keys to (pos, siz) pairs - # Handle the creation - self._create(flag) - self._update() - - def _create(self, flag): - if flag == 'n': - for filename in (self._datfile, self._bakfile, self._dirfile): - try: - _os.remove(filename) - except OSError: - pass # Mod by Jack: create data file if needed try: f = _io.open(self._datfile, 'r', encoding="Latin-1") @@ -82,6 +71,7 @@ f = _io.open(self._datfile, 'w', encoding="Latin-1") self._chmod(self._datfile) f.close() + self._update() # Read directory file into the in-memory index dict. def _update(self): @@ -276,20 +266,20 @@ self.close() -def open(file, flag='c', mode=0o666): +def open(file, flag=None, mode=0o666): """Open the database file, filename, and return corresponding object. The flag argument, used to control how the database is opened in the - other DBM implementations, supports only the semantics of 'c' and 'n' - values. Other values will default to the semantics of 'c' value: - the database will always opened for update and will be created if it - does not exist. + other DBM implementations, is ignored in the dbm.dumb module; the + database is always opened for update, and will be created if it does + not exist. The optional mode argument is the UNIX mode of the file, used only when the database has to be created. It defaults to octal code 0o666 (and will be modified by the prevailing umask). """ + # flag argument is currently ignored # Modify mode depending on the umask try: @@ -300,4 +290,5 @@ else: # Turn off any bits that are set in the umask mode = mode & (~um) - return _Database(file, mode, flag=flag) + + return _Database(file, mode) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/heapq.py --- a/Lib/heapq.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/heapq.py Thu Jun 05 21:36:59 2014 +0200 @@ -397,7 +397,7 @@ except StopIteration: _heappop(h) if h: - key_value, order, value, next = h[0] + key_value, order, value, next = h[0] yield value yield from next.__self__ @@ -413,7 +413,7 @@ # number of comparisons # n inputs k-extreme values (average of 5 trials) % more than min() # ------------- ---------------- --------------------- ----------------- -# 1,000 100 3,317 231.7% +# 1,000 100 3,317 133.2% # 10,000 100 14,046 40.5% # 100,000 100 105,749 5.7% # 1,000,000 100 1,007,751 0.8% @@ -496,8 +496,6 @@ # When key is none, use simpler decoration if key is None: it = iter(iterable) - # put the range(n) first so that zip() doesn't - # consume one too many elements from the iterator result = [(elem, i) for i, elem in zip(range(n), it)] if not result: return result @@ -596,4 +594,4 @@ if __name__ == "__main__": import doctest - print(doctest.testmod()) + doctest.testmod() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/AutoComplete.py --- a/Lib/idlelib/AutoComplete.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/AutoComplete.py Thu Jun 05 21:36:59 2014 +0200 @@ -226,8 +226,3 @@ namespace = sys.modules.copy() namespace.update(__main__.__dict__) return eval(name, namespace) - - -if __name__ == '__main__': - from unittest import main - main('idlelib.idle_test.test_autocomplete', verbosity=2) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/AutoExpand.py --- a/Lib/idlelib/AutoExpand.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/AutoExpand.py Thu Jun 05 21:36:59 2014 +0200 @@ -1,17 +1,3 @@ -'''Complete the current word before the cursor with words in the editor. - -Each menu selection or shortcut key selection replaces the word with a -different word with the same prefix. The search for matches begins -before the target and moves toward the top of the editor. It then starts -after the cursor and moves down. It then returns to the original word and -the cycle starts again. - -Changing the current text line or leaving the cursor in a different -place before requesting the next selection causes AutoExpand to reset -its state. - -This is an extension file and there is only one instance of AutoExpand. -''' import string import re @@ -34,7 +20,6 @@ self.state = None def expand_word_event(self, event): - "Replace the current word with the next expansion." curinsert = self.text.index("insert") curline = self.text.get("insert linestart", "insert lineend") if not self.state: @@ -61,7 +46,6 @@ return "break" def getwords(self): - "Return a list of words that match the prefix before the cursor." word = self.getprevword() if not word: return [] @@ -92,13 +76,8 @@ return words def getprevword(self): - "Return the word prefix before the cursor." line = self.text.get("insert linestart", "insert") i = len(line) while i > 0 and line[i-1] in self.wordchars: i = i-1 return line[i:] - -if __name__ == '__main__': - import unittest - unittest.main('idlelib.idle_test.test_autoexpand', verbosity=2) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/FormatParagraph.py --- a/Lib/idlelib/FormatParagraph.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/FormatParagraph.py Thu Jun 05 21:36:59 2014 +0200 @@ -188,6 +188,7 @@ return m.group(1) if __name__ == "__main__": + from test import support; support.use_resources = ['gui'] import unittest unittest.main('idlelib.idle_test.test_formatparagraph', verbosity=2, exit=False) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/GrepDialog.py --- a/Lib/idlelib/GrepDialog.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/GrepDialog.py Thu Jun 05 21:36:59 2014 +0200 @@ -1,14 +1,9 @@ import os import fnmatch -import re # for htest import sys -from tkinter import StringVar, BooleanVar, Checkbutton # for GrepDialog -from tkinter import Tk, Text, Button, SEL, END # for htest +from tkinter import * from idlelib import SearchEngine -import itertools from idlelib.SearchDialogBase import SearchDialogBase -# Importing OutputWindow fails due to import loop -# EditorWindow -> GrepDialop -> OutputWindow -> EditorWindow def grep(text, io=None, flist=None): root = text._root() @@ -68,7 +63,7 @@ if not path: self.top.bell() return - from idlelib.OutputWindow import OutputWindow # leave here! + from idlelib.OutputWindow import OutputWindow save = sys.stdout try: sys.stdout = OutputWindow(self.flist) @@ -84,26 +79,21 @@ pat = self.engine.getpat() print("Searching %r in %s ..." % (pat, path)) hits = 0 - try: - for fn in list: - try: - with open(fn, errors='replace') as f: - for lineno, line in enumerate(f, 1): - if line[-1:] == '\n': - line = line[:-1] - if prog.search(line): - sys.stdout.write("%s: %s: %s\n" % - (fn, lineno, line)) - hits += 1 - except OSError as msg: - print(msg) - print(("Hits found: %s\n" - "(Hint: right-click to open locations.)" - % hits) if hits else "No hits.") - except AttributeError: - # Tk window has been closed, OutputWindow.text = None, - # so in OW.write, OW.text.insert fails. - pass + for fn in list: + try: + with open(fn, errors='replace') as f: + for lineno, line in enumerate(f, 1): + if line[-1:] == '\n': + line = line[:-1] + if prog.search(line): + sys.stdout.write("%s: %s: %s\n" % + (fn, lineno, line)) + hits += 1 + except OSError as msg: + print(msg) + print(("Hits found: %s\n" + "(Hint: right-click to open locations.)" + % hits) if hits else "No hits.") def findfiles(self, dir, base, rec): try: @@ -130,8 +120,7 @@ self.top.grab_release() self.top.withdraw() - -def _grep_dialog(parent): # for htest +def _grep_dialog(parent): from idlelib.PyShell import PyShellFileList root = Tk() root.title("Test GrepDialog") @@ -152,6 +141,8 @@ root.mainloop() if __name__ == "__main__": + # A human test is a bit tricky since EditorWindow() imports this module. + # Hence Idle must be restarted after editing this file for a live test. import unittest unittest.main('idlelib.idle_test.test_grep', verbosity=2, exit=False) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/IdleHistory.py --- a/Lib/idlelib/IdleHistory.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/IdleHistory.py Thu Jun 05 21:36:59 2014 +0200 @@ -100,5 +100,7 @@ self.prefix = None if __name__ == "__main__": + from test import support + support.use_resources = ['gui'] from unittest import main main('idlelib.idle_test.test_idlehistory', verbosity=2, exit=False) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/SearchEngine.py --- a/Lib/idlelib/SearchEngine.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/SearchEngine.py Thu Jun 05 21:36:59 2014 +0200 @@ -229,5 +229,6 @@ return line, col if __name__ == "__main__": + from test import support; support.use_resources = ['gui'] import unittest unittest.main('idlelib.idle_test.test_searchengine', verbosity=2, exit=False) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/idle_test/README.txt --- a/Lib/idlelib/idle_test/README.txt Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/idle_test/README.txt Thu Jun 05 21:36:59 2014 +0200 @@ -26,6 +26,7 @@ with xyz (lowercased) added after 'test_'. --- if __name__ == "__main__": + from test import support; support.use_resources = ['gui'] import unittest unittest.main('idlelib.idle_test.test_', verbosity=2, exit=False) --- @@ -33,12 +34,12 @@ 2. Gui Tests -Gui tests need 'requires' from test.support (test.test_support in 2.7). A -test is a gui test if it creates a Tk root or master object either directly -or indirectly by instantiating a tkinter or idle class. For the benefit of -test processes that either have no graphical environment available or are not -allowed to use it, gui tests must be 'guarded' by "requires('gui')" in a -setUp function or method. This will typically be setUpClass. +Gui tests need 'requires' and 'use_resources' from test.support +(test.test_support in 2.7). A test is a gui test if it creates a Tk root or +master object either directly or indirectly by instantiating a tkinter or +idle class. For the benefit of buildbot machines that do not have a graphics +screen, gui tests must be 'guarded' by "requires('gui')" in a setUp +function or method. This will typically be setUpClass. To avoid interfering with other gui tests, all gui objects must be destroyed and deleted by the end of the test. If a widget, such as a Tk root, is created @@ -56,17 +57,11 @@ del cls.root --- -Support.requires('gui') causes the test(s) it guards to be skipped if any of -a few conditions are met: - - The tests are being run by regrtest.py, and it was started without - enabling the "gui" resource with the "-u" command line option. - - The tests are being run on Windows by a service that is not allowed to - interact with the graphical environment. - - The tests are being run on Mac OSX in a process that cannot make a window - manager connection. - - tkinter.Tk cannot be successfully instantiated for some reason. - - test.support.use_resources has been set by something other than - regrtest.py and does not contain "gui". +Support.requires('gui') returns true if it is either called in a main module +(which never happens on buildbots) or if use_resources contains 'gui'. +Use_resources is set by test.regrtest but not by unittest. So when running +tests in another module with unittest, we set it ourselves, as in the xyz.py +template above. Since non-gui tests always run, but gui tests only sometimes, tests of non-gui operations should best avoid needing a gui. Methods that make incidental use of @@ -93,8 +88,8 @@ To run all idle_test/test_*.py tests, either interactively ('>>>', with unittest imported) or from a command line, use one of the -following. (Notes: in 2.7, 'test ' (with the space) is 'test.regrtest '; -where present, -v and -ugui can be omitted.) +following. (Notes: unittest does not run gui tests; in 2.7, 'test ' (with the +space) is 'test.regrtest '; where present, -v and -ugui can be omitted.) >>> unittest.main('idlelib.idle_test', verbosity=2, exit=False) python -m unittest -v idlelib.idle_test @@ -103,13 +98,13 @@ The idle tests are 'discovered' by idlelib.idle_test.__init__.load_tests, which is also imported into test.test_idle. Normally, neither file should be -changed when working on individual test modules. The third command runs +changed when working on individual test modules. The third command runs runs unittest indirectly through regrtest. The same happens when the entire test suite is run with 'python -m test'. So that command must work for buildbots to stay green. Idle tests must not disturb the environment in a way that makes other tests fail (issue 18081). To run an individual Testcase or test method, extend the dotted name given to -unittest on the command line. +unittest on the command line. (But gui tests will not this way.) python -m unittest -v idlelib.idle_test.test_xyz.Test_case.test_meth diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/idle_test/mock_idle.py --- a/Lib/idlelib/idle_test/mock_idle.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/idle_test/mock_idle.py Thu Jun 05 21:36:59 2014 +0200 @@ -5,30 +5,6 @@ from idlelib.idle_test.mock_tk import Text -class Func: - '''Mock function captures args and returns result set by test. - - Attributes: - self.called - records call even if no args, kwds passed. - self.result - set by init, returned by call. - self.args - captures positional arguments. - self.kwds - captures keyword arguments. - - Most common use will probably be to mock methods. - Mock_tk.Var and Mbox_func are special variants of this. - ''' - def __init__(self, result=None): - self.called = False - self.result = result - self.args = None - self.kwds = None - def __call__(self, *args, **kwds): - self.called = True - self.args = args - self.kwds = kwds - return self.result - - class Editor: '''Minimally imitate EditorWindow.EditorWindow class. ''' @@ -41,7 +17,6 @@ last = self.text.index('end') return first, last - class UndoDelegator: '''Minimally imitate UndoDelegator,UndoDelegator class. ''' diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/idle_test/mock_tk.py --- a/Lib/idlelib/idle_test/mock_tk.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/idle_test/mock_tk.py Thu Jun 05 21:36:59 2014 +0200 @@ -1,27 +1,9 @@ """Classes that replace tkinter gui objects used by an object being tested. -A gui object is anything with a master or parent paramenter, which is -typically required in spite of what the doc strings say. +A gui object is anything with a master or parent paramenter, which is typically +required in spite of what the doc strings say. """ -class Event: - '''Minimal mock with attributes for testing event handlers. - - This is not a gui object, but is used as an argument for callbacks - that access attributes of the event passed. If a callback ignores - the event, other than the fact that is happened, pass 'event'. - - Keyboard, mouse, window, and other sources generate Event instances. - Event instances have the following attributes: serial (number of - event), time (of event), type (of event as number), widget (in which - event occurred), and x,y (position of mouse). There are other - attributes for specific events, such as keycode for key events. - tkinter.Event.__doc__ has more but is still not complete. - ''' - def __init__(self, **kwds): - "Create event with attributes needed for test" - self.__dict__.update(kwds) - class Var: "Use for String/Int/BooleanVar: incomplete" def __init__(self, master=None, value=None, name=None): @@ -38,10 +20,9 @@ Instead of displaying a message box, the mock's call method saves the arguments as instance attributes, which test functions can then examime. - The test can set the result returned to ask function """ - def __init__(self, result=None): - self.result = result # Return None for all show funcs + def __init__(self): + self.result = None # The return for all show funcs def __call__(self, title, message, *args, **kwds): # Save all args for possible examination by tester self.title = title diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/idle_test/test_autocomplete.py --- a/Lib/idlelib/idle_test/test_autocomplete.py Wed Jun 11 12:27:40 2014 -0400 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,143 +0,0 @@ -import unittest -from test.support import requires -from tkinter import Tk, Text, TclError - -import idlelib.AutoComplete as ac -import idlelib.AutoCompleteWindow as acw -import idlelib.macosxSupport as mac -from idlelib.idle_test.mock_idle import Func -from idlelib.idle_test.mock_tk import Event - -class AutoCompleteWindow: - def complete(): - return - -class DummyEditwin: - def __init__(self, root, text): - self.root = root - self.text = text - self.indentwidth = 8 - self.tabwidth = 8 - self.context_use_ps1 = True - - -class AutoCompleteTest(unittest.TestCase): - - @classmethod - def setUpClass(cls): - requires('gui') - cls.root = Tk() - mac.setupApp(cls.root, None) - cls.text = Text(cls.root) - cls.editor = DummyEditwin(cls.root, cls.text) - - @classmethod - def tearDownClass(cls): - cls.root.destroy() - del cls.text - del cls.editor - del cls.root - - def setUp(self): - self.editor.text.delete('1.0', 'end') - self.autocomplete = ac.AutoComplete(self.editor) - - def test_init(self): - self.assertEqual(self.autocomplete.editwin, self.editor) - - def test_make_autocomplete_window(self): - testwin = self.autocomplete._make_autocomplete_window() - self.assertIsInstance(testwin, acw.AutoCompleteWindow) - - def test_remove_autocomplete_window(self): - self.autocomplete.autocompletewindow = ( - self.autocomplete._make_autocomplete_window()) - self.autocomplete._remove_autocomplete_window() - self.assertIsNone(self.autocomplete.autocompletewindow) - - def test_force_open_completions_event(self): - # Test that force_open_completions_event calls _open_completions - o_cs = Func() - self.autocomplete.open_completions = o_cs - self.autocomplete.force_open_completions_event('event') - self.assertEqual(o_cs.args, (True, False, True)) - - def test_try_open_completions_event(self): - Equal = self.assertEqual - autocomplete = self.autocomplete - trycompletions = self.autocomplete.try_open_completions_event - o_c_l = Func() - autocomplete._open_completions_later = o_c_l - - # _open_completions_later should not be called with no text in editor - trycompletions('event') - Equal(o_c_l.args, None) - - # _open_completions_later should be called with COMPLETE_ATTRIBUTES (1) - self.text.insert('1.0', 're.') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 1)) - - # _open_completions_later should be called with COMPLETE_FILES (2) - self.text.delete('1.0', 'end') - self.text.insert('1.0', '"./Lib/') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 2)) - - def test_autocomplete_event(self): - Equal = self.assertEqual - autocomplete = self.autocomplete - - # Test that the autocomplete event is ignored if user is pressing a - # modifier key in addition to the tab key - ev = Event(mc_state=True) - self.assertIsNone(autocomplete.autocomplete_event(ev)) - del ev.mc_state - - # If autocomplete window is open, complete() method is called - testwin = self.autocomplete._make_autocomplete_window() - self.text.insert('1.0', 're.') - Equal(self.autocomplete.autocomplete_event(ev), 'break') - - # If autocomplete window is not active or does not exist, - # open_completions is called. Return depends on its return. - autocomplete._remove_autocomplete_window() - o_cs = Func() # .result = None - autocomplete.open_completions = o_cs - Equal(self.autocomplete.autocomplete_event(ev), None) - Equal(o_cs.args, (False, True, True)) - o_cs.result = True - Equal(self.autocomplete.autocomplete_event(ev), 'break') - Equal(o_cs.args, (False, True, True)) - - def test_open_completions_later(self): - # Test that autocomplete._delayed_completion_id is set - pass - - def test_delayed_open_completions(self): - # Test that autocomplete._delayed_completion_id set to None and that - # open_completions only called if insertion index is the same as - # _delayed_completion_index - pass - - def test_open_completions(self): - # Test completions of files and attributes as well as non-completion - # of errors - pass - - def test_fetch_completions(self): - # Test that fetch_completions returns 2 lists: - # For attribute completion, a large list containing all variables, and - # a small list containing non-private variables. - # For file completion, a large list containing all files in the path, - # and a small list containing files that do not start with '.' - pass - - def test_get_entity(self): - # Test that a name is in the namespace of sys.modules and - # __main__.__dict__ - pass - - -if __name__ == '__main__': - unittest.main(verbosity=2) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/idle_test/test_autoexpand.py --- a/Lib/idlelib/idle_test/test_autoexpand.py Wed Jun 11 12:27:40 2014 -0400 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,141 +0,0 @@ -"""Unit tests for idlelib.AutoExpand""" -import unittest -from test.support import requires -from tkinter import Text, Tk -#from idlelib.idle_test.mock_tk import Text -from idlelib.AutoExpand import AutoExpand - - -class Dummy_Editwin: - # AutoExpand.__init__ only needs .text - def __init__(self, text): - self.text = text - -class AutoExpandTest(unittest.TestCase): - - @classmethod - def setUpClass(cls): - if 'tkinter' in str(Text): - requires('gui') - cls.tk = Tk() - cls.text = Text(cls.tk) - else: - cls.text = Text() - cls.auto_expand = AutoExpand(Dummy_Editwin(cls.text)) - - @classmethod - def tearDownClass(cls): - if hasattr(cls, 'tk'): - cls.tk.destroy() - del cls.tk - del cls.text, cls.auto_expand - - def tearDown(self): - self.text.delete('1.0', 'end') - - def test_get_prevword(self): - text = self.text - previous = self.auto_expand.getprevword - equal = self.assertEqual - - equal(previous(), '') - - text.insert('insert', 't') - equal(previous(), 't') - - text.insert('insert', 'his') - equal(previous(), 'this') - - text.insert('insert', ' ') - equal(previous(), '') - - text.insert('insert', 'is') - equal(previous(), 'is') - - text.insert('insert', '\nsample\nstring') - equal(previous(), 'string') - - text.delete('3.0', 'insert') - equal(previous(), '') - - text.delete('1.0', 'end') - equal(previous(), '') - - def test_before_only(self): - previous = self.auto_expand.getprevword - expand = self.auto_expand.expand_word_event - equal = self.assertEqual - - self.text.insert('insert', 'ab ac bx ad ab a') - equal(self.auto_expand.getwords(), ['ab', 'ad', 'ac', 'a']) - expand('event') - equal(previous(), 'ab') - expand('event') - equal(previous(), 'ad') - expand('event') - equal(previous(), 'ac') - expand('event') - equal(previous(), 'a') - - def test_after_only(self): - # Also add punctuation 'noise' that shoud be ignored. - text = self.text - previous = self.auto_expand.getprevword - expand = self.auto_expand.expand_word_event - equal = self.assertEqual - - text.insert('insert', 'a, [ab] ac: () bx"" cd ac= ad ya') - text.mark_set('insert', '1.1') - equal(self.auto_expand.getwords(), ['ab', 'ac', 'ad', 'a']) - expand('event') - equal(previous(), 'ab') - expand('event') - equal(previous(), 'ac') - expand('event') - equal(previous(), 'ad') - expand('event') - equal(previous(), 'a') - - def test_both_before_after(self): - text = self.text - previous = self.auto_expand.getprevword - expand = self.auto_expand.expand_word_event - equal = self.assertEqual - - text.insert('insert', 'ab xy yz\n') - text.insert('insert', 'a ac by ac') - - text.mark_set('insert', '2.1') - equal(self.auto_expand.getwords(), ['ab', 'ac', 'a']) - expand('event') - equal(previous(), 'ab') - expand('event') - equal(previous(), 'ac') - expand('event') - equal(previous(), 'a') - - def test_other_expand_cases(self): - text = self.text - expand = self.auto_expand.expand_word_event - equal = self.assertEqual - - # no expansion candidate found - equal(self.auto_expand.getwords(), []) - equal(expand('event'), 'break') - - text.insert('insert', 'bx cy dz a') - equal(self.auto_expand.getwords(), []) - - # reset state by successfully expanding once - # move cursor to another position and expand again - text.insert('insert', 'ac xy a ac ad a') - text.mark_set('insert', '1.7') - expand('event') - initial_state = self.auto_expand.state - text.mark_set('insert', '1.end') - expand('event') - new_state = self.auto_expand.state - self.assertNotEqual(initial_state, new_state) - -if __name__ == '__main__': - unittest.main(verbosity=2) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/idle_test/test_textview.py --- a/Lib/idlelib/idle_test/test_textview.py Wed Jun 11 12:27:40 2014 -0400 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,97 +0,0 @@ -'''Test the functions and main class method of textView.py. - -Since all methods and functions create (or destroy) a TextViewer, which -is a widget containing multiple widgets, all tests must be gui tests. -Using mock Text would not change this. Other mocks are used to retrieve -information about calls. - -The coverage is essentially 100%. -''' -from test.support import requires -requires('gui') - -import unittest -import os -from tkinter import Tk, Text, TclError -from idlelib import textView as tv -from idlelib.idle_test.mock_idle import Func -from idlelib.idle_test.mock_tk import Mbox - -def setUpModule(): - global root - root = Tk() - -def tearDownModule(): - global root - root.destroy() - del root - - -class TV(tv.TextViewer): # used by TextViewTest - transient = Func() - grab_set = Func() - wait_window = Func() - -class TextViewTest(unittest.TestCase): - - def setUp(self): - TV.transient.__init__() - TV.grab_set.__init__() - TV.wait_window.__init__() - - def test_init_modal(self): - view = TV(root, 'Title', 'test text') - self.assertTrue(TV.transient.called) - self.assertTrue(TV.grab_set.called) - self.assertTrue(TV.wait_window.called) - view.Ok() - - def test_init_nonmodal(self): - view = TV(root, 'Title', 'test text', modal=False) - self.assertFalse(TV.transient.called) - self.assertFalse(TV.grab_set.called) - self.assertFalse(TV.wait_window.called) - view.Ok() - - def test_ok(self): - view = TV(root, 'Title', 'test text', modal=False) - view.destroy = Func() - view.Ok() - self.assertTrue(view.destroy.called) - del view.destroy # unmask real function - view.destroy - - -class textviewTest(unittest.TestCase): - - @classmethod - def setUpClass(cls): - cls.orig_mbox = tv.tkMessageBox - tv.tkMessageBox = Mbox - - @classmethod - def tearDownClass(cls): - tv.tkMessageBox = cls.orig_mbox - del cls.orig_mbox - - def test_view_text(self): - # If modal True, tkinter will error with 'can't invoke "event" command' - view = tv.view_text(root, 'Title', 'test text', modal=False) - self.assertIsInstance(view, tv.TextViewer) - - def test_view_file(self): - test_dir = os.path.dirname(__file__) - testfile = os.path.join(test_dir, 'test_textview.py') - view = tv.view_file(root, 'Title', testfile, modal=False) - self.assertIsInstance(view, tv.TextViewer) - self.assertIn('Test', view.textView.get('1.0', '1.end')) - view.Ok() - - # Mock messagebox will be used and view_file will not return anything - testfile = os.path.join(test_dir, '../notthere.py') - view = tv.view_file(root, 'Title', testfile, modal=False) - self.assertIsNone(view) - - -if __name__ == '__main__': - unittest.main(verbosity=2) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/idlelib/textView.py --- a/Lib/idlelib/textView.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/idlelib/textView.py Thu Jun 05 21:36:59 2014 +0200 @@ -12,11 +12,7 @@ def __init__(self, parent, title, text, modal=True, _htest=False): """Show the given text in a scrollable window with a 'close' button - If modal option set to False, user can interact with other windows, - otherwise they will be unable to interact with other windows until - the textview window is closed. - - _htest - bool; change box location when running htest. + _htest - bool, change box location when running htest """ Toplevel.__init__(self, parent) self.configure(borderwidth=5) @@ -72,7 +68,8 @@ try: with open(filename, 'r', encoding=encoding) as file: contents = file.read() - except IOError: + except OSError: + import tkinter.messagebox as tkMessageBox tkMessageBox.showerror(title='File Load Error', message='Unable to load file %r .' % filename, parent=parent) @@ -80,7 +77,5 @@ return view_text(parent, title, contents, modal) if __name__ == '__main__': - import unittest - unittest.main('idlelib.idle_test.test_textview', verbosity=2, exit=False) from idlelib.idle_test.htest import run run(TextViewer) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/logging/__init__.py --- a/Lib/logging/__init__.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/logging/__init__.py Thu Jun 05 21:36:59 2014 +0200 @@ -55,8 +55,8 @@ # _srcfile is used when walking the stack to check when we've got the first # caller stack frame. # -if hasattr(sys, 'frozen'): - _srcfile = os.path.join('logging', '__init__.py') +if hasattr(sys, 'frozen'): #support for py2exe + _srcfile = "logging%s__init__%s" % (os.sep, __file__[-4:]) else: _srcfile = __file__ _srcfile = os.path.normcase(_srcfile) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/pydoc.py --- a/Lib/pydoc.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/pydoc.py Thu Jun 05 21:36:59 2014 +0200 @@ -593,15 +593,10 @@ elif pep: url = 'http://www.python.org/dev/peps/pep-%04d/' % int(pep) results.append('%s' % (url, escape(all))) - elif selfdot: - # Create a link for methods like 'self.method(...)' - # and use for attributes like 'self.attr' - if text[end:end+1] == '(': - results.append('self.' + self.namelink(name, methods)) - else: - results.append('self.%s' % name) elif text[end:end+1] == '(': results.append(self.namelink(name, methods, funcs, classes)) + elif selfdot: + results.append('self.%s' % name) else: results.append(self.namelink(name, classes)) here = end @@ -1415,8 +1410,6 @@ def getpager(): """Decide what method to use for paging through text.""" - if not hasattr(sys.stdin, "isatty"): - return plainpager if not hasattr(sys.stdout, "isatty"): return plainpager if not sys.stdin.isatty() or not sys.stdout.isatty(): diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/smtpd.py --- a/Lib/smtpd.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/smtpd.py Thu Jun 05 21:36:59 2014 +0200 @@ -1,5 +1,5 @@ #! /usr/bin/env python3 -"""An RFC 5321 smtp proxy. +"""An RFC 5321 (opt. RFC 6532) smtp proxy. Usage: %(program)s [options] [localhost:localport [remotehost:remoteport]] @@ -25,6 +25,10 @@ Restrict the total size of the incoming message to "limit" number of bytes via the RFC 1870 SIZE extension. Defaults to 33554432 bytes. + --smtputf8 + -u + Enable the SMTPUTF8 extension and behave as RFC 6532 smtp proxy. + --debug -d Turn on debugging prints. @@ -98,6 +102,7 @@ DEBUGSTREAM = Devnull() NEWLINE = '\n' +EMPTYSTRING = '' COMMASPACE = ', ' DATA_SIZE_DEFAULT = 33554432 @@ -113,36 +118,25 @@ COMMAND = 0 DATA = 1 - command_size_limit = 512 - command_size_limits = collections.defaultdict(lambda x=command_size_limit: x) - command_size_limits.update({ - 'MAIL': command_size_limit + 26, - }) - max_command_size_limit = max(command_size_limits.values()) + command_size_limits = collections.defaultdict(lambda : 512) + + @property + def max_command_size_limit(self): + try: + return max(self.command_size_limits.values()) + except ValueError: + # raised if command_size_limits has no values. 'MAIL' is just a + # random string here (wich would be added anyway). + return self.command_size_limits['MAIL'] def __init__(self, server, conn, addr, data_size_limit=DATA_SIZE_DEFAULT, - map=None, decode_data=None): + map=None, enable_SMTPUTF8=False): asynchat.async_chat.__init__(self, conn, map=map) self.smtp_server = server self.conn = conn self.addr = addr self.data_size_limit = data_size_limit - if decode_data is None: - warn("The decode_data default of True will change to False in 3.6;" - " specify an explicit value for this keyword", - DeprecationWarning, 2) - decode_data = True - self._decode_data = decode_data - if decode_data: - self._emptystring = '' - self._linesep = '\r\n' - self._dotsep = '.' - self._newline = NEWLINE - else: - self._emptystring = b'' - self._linesep = b'\r\n' - self._dotsep = b'.' - self._newline = b'\n' + self.enable_SMTPUTF8 = enable_SMTPUTF8 self.received_lines = [] self.smtp_state = self.COMMAND self.seen_greeting = '' @@ -287,9 +281,10 @@ "set 'addr' instead", DeprecationWarning, 2) self.addr = value - # Overrides base class for convenience + # Overrides base class for convenience. def push(self, msg): - asynchat.async_chat.push(self, bytes(msg + '\r\n', 'ascii')) + asynchat.async_chat.push(self, bytes( + msg + '\r\n', 'utf-8' if self.enable_SMTPUTF8 else 'ascii')) # Implementation of base class abstract method def collect_incoming_data(self, data): @@ -302,14 +297,11 @@ return elif limit: self.num_bytes += len(data) - if self._decode_data: - self.received_lines.append(str(data, 'utf-8')) - else: - self.received_lines.append(data) + self.received_lines.append(str(data, "utf-8")) # Implementation of base class abstract method def found_terminator(self): - line = self._emptystring.join(self.received_lines) + line = EMPTYSTRING.join(self.received_lines) print('Data:', repr(line), file=DEBUGSTREAM) self.received_lines = [] if self.smtp_state == self.COMMAND: @@ -317,9 +309,14 @@ if not line: self.push('500 Error: bad syntax') return + #XXX: Should we check this? (breaks bug compatibility) + try: + line.encode('utf-8' if self.enable_SMTPUTF8 else 'ascii') + except UnicodeEncodeError: + self.push( + '500 Error: unknown characters in input.') + return method = None - if not self._decode_data: - line = str(line, 'utf-8') i = line.find(' ') if i < 0: command = line.upper() @@ -327,8 +324,7 @@ else: command = line[:i].upper() arg = line[i+1:].strip() - max_sz = (self.command_size_limits[command] - if self.extended_smtp else self.command_size_limit) + max_sz = self.command_size_limits[command] if sz > max_sz: self.push('500 Error: line too long') return @@ -350,12 +346,12 @@ # Remove extraneous carriage returns and de-transparency according # to RFC 5321, Section 4.5.2. data = [] - for text in line.split(self._linesep): - if text and text[0] == self._dotsep: + for text in line.split('\r\n'): + if text and text[0] == '.': data.append(text[1:]) else: data.append(text) - self.received_data = self._newline.join(data) + self.received_data = NEWLINE.join(data) status = self.smtp_server.process_message(self.peer, self.mailfrom, self.rcpttos, @@ -391,9 +387,16 @@ else: self.seen_greeting = arg self.extended_smtp = True + self.command_size_limits.pop('MAIL') self.push('250-%s' % self.fqdn) if self.data_size_limit: self.push('250-SIZE %s' % self.data_size_limit) + self.command_size_limits['MAIL'] += 26 + if self.enable_SMTPUTF8: + self.push('250-8BITMIME') + self.push('250-SMTPUTF8') + self.command_size_limits['MAIL'] += 10 + self.push('250 HELP') def smtp_NOOP(self, arg): @@ -427,8 +430,8 @@ def _getparams(self, params): # Return any parameters that appear to be syntactically valid according # to RFC 1869, ignore all others. (Postel rule: accept what we can.) - params = [param.split('=', 1) for param in params.split() - if '=' in param] + params = [param.split('=', 1) if '=' in param else (param, True) for + param in params.split()] return {k: v for k, v in params if k.isalnum()} def smtp_HELP(self, arg): @@ -506,6 +509,22 @@ if params is None: self.push(syntaxerr) return + # XXX: I assume that the 8BITMIME parameter can be ignored since we are + # using 8-bit clean streams anyway. (The variable is used to validate + # SMTPUTF8 later.) + body = params.pop('BODY', '7BIT') + if params.pop('SMTPUTF8', False): + if not self.enable_SMTPUTF8: + self.push( + '555 MAIL FROM parameter SMTPUTF8 is not enabled.' + ) + return + elif body != '8BITMIME': + self.push( + '501 Syntax: MAIL FROM:
BODY=8BITMIME SMTPUTF8') + return + else: + self.require_SMTPUTF8 = True size = params.pop('SIZE', None) if size: if not size.isdigit(): @@ -598,16 +617,11 @@ def __init__(self, localaddr, remoteaddr, data_size_limit=DATA_SIZE_DEFAULT, map=None, - decode_data=None): + enable_SMTPUTF8=False): self._localaddr = localaddr self._remoteaddr = remoteaddr self.data_size_limit = data_size_limit - if decode_data is None: - warn("The decode_data default of True will change to False in 3.6;" - " specify an explicit value for this keyword", - DeprecationWarning, 2) - decode_data = True - self._decode_data = decode_data + self.enable_SMTPUTF8 = enable_SMTPUTF8 asyncore.dispatcher.__init__(self, map=map) try: self.create_socket(socket.AF_INET, socket.SOCK_STREAM) @@ -626,7 +640,8 @@ def handle_accepted(self, conn, addr): print('Incoming connection from %s' % repr(addr), file=DEBUGSTREAM) channel = self.channel_class(self, conn, addr, self.data_size_limit, - self._map, self._decode_data) + self._map, + enable_SMTPUTF8=self.enable_SMTPUTF8) # API for "doing something useful with the message" def process_message(self, peer, mailfrom, rcpttos, data): @@ -788,17 +803,19 @@ class Options: - setuid = 1 + setuid = True classname = 'PureProxy' size_limit = None + enable_SMTPUTF8 = False def parseargs(): global DEBUGSTREAM try: opts, args = getopt.getopt( - sys.argv[1:], 'nVhc:s:d', - ['class=', 'nosetuid', 'version', 'help', 'size=', 'debug']) + sys.argv[1:], 'nVhc:s:du', + ['class=', 'nosetuid', 'version', 'help', 'size=', 'debug', + 'smtputf8']) except getopt.error as e: usage(1, e) @@ -810,11 +827,13 @@ print(__version__) sys.exit(0) elif opt in ('-n', '--nosetuid'): - options.setuid = 0 + options.setuid = False elif opt in ('-c', '--class'): options.classname = arg elif opt in ('-d', '--debug'): DEBUGSTREAM = sys.stderr + elif opt in ('-u', '--smtputf8'): + options.enable_SMTPUTF8 = True elif opt in ('-s', '--size'): try: int_size = int(arg) @@ -869,7 +888,7 @@ class_ = getattr(mod, classname) proxy = class_((options.localhost, options.localport), (options.remotehost, options.remoteport), - options.size_limit) + options.size_limit, enable_SMTPUTF8=options.enable_SMTPUTF8) if options.setuid: try: import pwd diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/smtplib.py --- a/Lib/smtplib.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/smtplib.py Thu Jun 05 21:36:59 2014 +0200 @@ -236,6 +236,7 @@ self._host = host self.timeout = timeout self.esmtp_features = {} + self.encoding = 'ascii' self.source_address = source_address if host: @@ -331,7 +332,7 @@ print('send:', repr(s), file=stderr) if hasattr(self, 'sock') and self.sock: if isinstance(s, str): - s = s.encode("ascii") + s = s.encode(self.encoding) try: self.sock.sendall(s) except OSError: @@ -377,7 +378,6 @@ if self.debuglevel > 0: print('reply:', repr(line), file=stderr) if len(line) > _MAXLINE: - self.close() raise SMTPResponseException(500, "Line too long.") resp.append(line[4:].strip(b' \t\r\n')) code = line[:3] @@ -434,7 +434,7 @@ self.does_esmtp = 1 #parse the ehlo response -ddm assert isinstance(self.ehlo_resp, bytes), repr(self.ehlo_resp) - resp = self.ehlo_resp.decode("latin-1").split('\n') + resp = self.ehlo_resp.decode("utf-8").split('\n') del resp[0] for each in resp: # To be able to communicate with as many SMTP servers as possible, @@ -499,7 +499,12 @@ """SMTP 'mail' command -- begins mail xfer session.""" optionlist = '' if options and self.does_esmtp: - optionlist = ' ' + ' '.join(options) + optionlist = ' ' + ' '.join(options).upper() + # Setting 'SMTPUTF8' when not announced by the server is invalid, but + # its the servers job to tell that. We MUST NOT send non-ASCII in that + # case however. + if 'SMTPUTF8' in options: + self.encoding = 'utf-8' self.putcmd("mail", "FROM:%s%s" % (quoteaddr(sender), optionlist)) return self.getreply() @@ -529,7 +534,7 @@ raise SMTPDataError(code, repl) else: if isinstance(msg, str): - msg = _fix_eols(msg).encode('ascii') + msg = _fix_eols(msg).encode(self.encoding) q = _quote_periods(msg) if q[-2:] != bCRLF: q = q + bCRLF @@ -762,7 +767,9 @@ self.ehlo_or_helo_if_needed() esmtp_opts = [] if isinstance(msg, str): - msg = _fix_eols(msg).encode('ascii') + if 'SMTPUTF8' in ' '.join(mail_options).upper(): + self.encoding = 'utf-8' + msg = _fix_eols(msg).encode(self.encoding) if self.does_esmtp: # Hmmm? what's this? -ddm # self.esmtp_features['7bit']="" diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/socket.py --- a/Lib/socket.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/socket.py Thu Jun 05 21:36:59 2014 +0200 @@ -47,7 +47,7 @@ import _socket from _socket import * -import os, sys, io, selectors +import os, sys, io from enum import IntEnum try: @@ -109,9 +109,6 @@ __all__.append("errorTab") -class _GiveupOnSendfile(Exception): pass - - class socket(_socket.socket): """A subclass of _socket.socket adding the makefile() method.""" @@ -236,149 +233,6 @@ text.mode = mode return text - if hasattr(os, 'sendfile'): - - def _sendfile_use_sendfile(self, file, offset=0, count=None): - self._check_sendfile_params(file, offset, count) - sockno = self.fileno() - try: - fileno = file.fileno() - except (AttributeError, io.UnsupportedOperation) as err: - raise _GiveupOnSendfile(err) # not a regular file - try: - fsize = os.fstat(fileno).st_size - except OSError: - raise _GiveupOnSendfile(err) # not a regular file - if not fsize: - return 0 # empty file - blocksize = fsize if not count else count - - timeout = self.gettimeout() - if timeout == 0: - raise ValueError("non-blocking sockets are not supported") - # poll/select have the advantage of not requiring any - # extra file descriptor, contrarily to epoll/kqueue - # (also, they require a single syscall). - if hasattr(selectors, 'PollSelector'): - selector = selectors.PollSelector() - else: - selector = selectors.SelectSelector() - selector.register(sockno, selectors.EVENT_WRITE) - - total_sent = 0 - # localize variable access to minimize overhead - selector_select = selector.select - os_sendfile = os.sendfile - try: - while True: - if timeout and not selector_select(timeout): - raise _socket.timeout('timed out') - if count: - blocksize = count - total_sent - if blocksize <= 0: - break - try: - sent = os_sendfile(sockno, fileno, offset, blocksize) - except BlockingIOError: - if not timeout: - # Block until the socket is ready to send some - # data; avoids hogging CPU resources. - selector_select() - continue - except OSError as err: - if total_sent == 0: - # We can get here for different reasons, the main - # one being 'file' is not a regular mmap(2)-like - # file, in which case we'll fall back on using - # plain send(). - raise _GiveupOnSendfile(err) - raise err from None - else: - if sent == 0: - break # EOF - offset += sent - total_sent += sent - return total_sent - finally: - if total_sent > 0 and hasattr(file, 'seek'): - file.seek(offset) - else: - def _sendfile_use_sendfile(self, file, offset=0, count=None): - raise _GiveupOnSendfile( - "os.sendfile() not available on this platform") - - def _sendfile_use_send(self, file, offset=0, count=None): - self._check_sendfile_params(file, offset, count) - if self.gettimeout() == 0: - raise ValueError("non-blocking sockets are not supported") - if offset: - file.seek(offset) - blocksize = min(count, 8192) if count else 8192 - total_sent = 0 - # localize variable access to minimize overhead - file_read = file.read - sock_send = self.send - try: - while True: - if count: - blocksize = min(count - total_sent, blocksize) - if blocksize <= 0: - break - data = memoryview(file_read(blocksize)) - if not data: - break # EOF - while True: - try: - sent = sock_send(data) - except BlockingIOError: - continue - else: - total_sent += sent - if sent < len(data): - data = data[sent:] - else: - break - return total_sent - finally: - if total_sent > 0 and hasattr(file, 'seek'): - file.seek(offset + total_sent) - - def _check_sendfile_params(self, file, offset, count): - if 'b' not in getattr(file, 'mode', 'b'): - raise ValueError("file should be opened in binary mode") - if not self.type & SOCK_STREAM: - raise ValueError("only SOCK_STREAM type sockets are supported") - if count is not None: - if not isinstance(count, int): - raise TypeError( - "count must be a positive integer (got {!r})".format(count)) - if count <= 0: - raise ValueError( - "count must be a positive integer (got {!r})".format(count)) - - def sendfile(self, file, offset=0, count=None): - """sendfile(file[, offset[, count]]) -> sent - - Send a file until EOF is reached by using high-performance - os.sendfile() and return the total number of bytes which - were sent. - *file* must be a regular file object opened in binary mode. - If os.sendfile() is not available (e.g. Windows) or file is - not a regular file socket.send() will be used instead. - *offset* tells from where to start reading the file. - If specified, *count* is the total number of bytes to transmit - as opposed to sending the file until EOF is reached. - File position is updated on return or also in case of error in - which case file.tell() can be used to figure out the number of - bytes which were sent. - The socket must be of SOCK_STREAM type. - Non-blocking sockets are not supported. - """ - try: - return self._sendfile_use_sendfile(file, offset, count) - except _GiveupOnSendfile: - return self._sendfile_use_send(file, offset, count) - def _decref_socketios(self): if self._io_refs > 0: self._io_refs -= 1 diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/ssl.py --- a/Lib/ssl.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/ssl.py Thu Jun 05 21:36:59 2014 +0200 @@ -700,16 +700,6 @@ else: return socket.sendall(self, data, flags) - def sendfile(self, file, offset=0, count=None): - """Send a file, possibly by using os.sendfile() if this is a - clear-text socket. Return the total number of bytes sent. - """ - if self._sslobj is None: - # os.sendfile() works with plain sockets only - return super().sendfile(file, offset, count) - else: - return self._sendfile_use_send(file, offset, count) - def recv(self, buflen=1024, flags=0): self._checkClosed() if self._sslobj: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/tempfile.py --- a/Lib/tempfile.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/tempfile.py Thu Jun 05 21:36:59 2014 +0200 @@ -473,11 +473,6 @@ TemporaryFile = NamedTemporaryFile else: - # Is the O_TMPFILE flag available and does it work? - # The flag is set to False if os.open(dir, os.O_TMPFILE) raises an - # IsADirectoryError exception - _O_TMPFILE_WORKS = hasattr(_os, 'O_TMPFILE') - def TemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix="", prefix=template, dir=None): @@ -493,32 +488,11 @@ Returns an object with a file-like interface. The file has no name, and will cease to exist when it is closed. """ - global _O_TMPFILE_WORKS if dir is None: dir = gettempdir() flags = _bin_openflags - if _O_TMPFILE_WORKS: - try: - flags2 = (flags | _os.O_TMPFILE) & ~_os.O_CREAT - fd = _os.open(dir, flags2, 0o600) - except IsADirectoryError: - # Linux kernel older than 3.11 ignores O_TMPFILE flag. - # Set flag to False to not try again. - _O_TMPFILE_WORKS = False - except OSError: - # The filesystem of the directory does not support O_TMPFILE. - # For example, OSError(95, 'Operation not supported'). - pass - else: - try: - return _io.open(fd, mode, buffering=buffering, - newline=newline, encoding=encoding) - except: - _os.close(fd) - raise - # Fallback to _mkstemp_inner(). (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags) try: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/pydoc_mod.py --- a/Lib/test/pydoc_mod.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/pydoc_mod.py Thu Jun 05 21:36:59 2014 +0200 @@ -15,16 +15,6 @@ NO_MEANING = "eggs" pass -class C(object): - def say_no(self): - return "no" - def get_answer(self): - """ Return say_no() """ - return self.say_no() - def is_it_true(self): - """ Return self.get_answer() """ - return self.get_answer() - def doc_func(): """ This function solves all of the world's problems: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/pystone.py --- a/Lib/test/pystone.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/pystone.py Thu Jun 05 21:36:59 2014 +0200 @@ -3,7 +3,7 @@ """ "PYSTONE" Benchmark Program -Version: Python/1.2 (corresponds to C/1.1 plus 3 Pystone fixes) +Version: Python/1.1 (corresponds to C/1.1 plus 2 Pystone fixes) Author: Reinhold P. Weicker, CACM Vol 27, No 10, 10/84 pg. 1013. @@ -30,20 +30,13 @@ percent faster than version 1.0, so benchmark figures of different versions can't be compared directly. - Version 1.2 changes the division to floor division. - - Under Python 3 version 1.1 would use the normal division - operator, resulting in some of the operations mistakenly - yielding floats. Version 1.2 instead uses floor division - making the benchmark a integer benchmark again. - """ LOOPS = 50000 -from time import time +from time import clock -__version__ = "1.2" +__version__ = "1.1" [Ident1, Ident2, Ident3, Ident4, Ident5] = range(1, 6) @@ -93,10 +86,10 @@ global PtrGlb global PtrGlbNext - starttime = time() + starttime = clock() for i in range(loops): pass - nulltime = time() - starttime + nulltime = clock() - starttime PtrGlbNext = Record() PtrGlb = Record() @@ -108,7 +101,7 @@ String1Loc = "DHRYSTONE PROGRAM, 1'ST STRING" Array2Glob[8][7] = 10 - starttime = time() + starttime = clock() for i in range(loops): Proc5() @@ -130,11 +123,11 @@ EnumLoc = Proc6(Ident1) CharIndex = chr(ord(CharIndex)+1) IntLoc3 = IntLoc2 * IntLoc1 - IntLoc2 = IntLoc3 // IntLoc1 + IntLoc2 = IntLoc3 / IntLoc1 IntLoc2 = 7 * (IntLoc3 - IntLoc2) - IntLoc1 IntLoc1 = Proc2(IntLoc1) - benchtime = time() - starttime - nulltime + benchtime = clock() - starttime - nulltime if benchtime == 0.0: loopsPerBenchtime = 0.0 else: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/support/__init__.py --- a/Lib/test/support/__init__.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/support/__init__.py Thu Jun 05 21:36:59 2014 +0200 @@ -454,17 +454,23 @@ return _is_gui_available.result def is_resource_enabled(resource): - """Test whether a resource is enabled. - - Known resources are set by regrtest.py. If not running under regrtest.py, - all resources are assumed enabled unless use_resources has been set. - """ - return use_resources is None or resource in use_resources + """Test whether a resource is enabled. Known resources are set by + regrtest.py.""" + return use_resources is not None and resource in use_resources def requires(resource, msg=None): - """Raise ResourceDenied if the specified resource is not available.""" + """Raise ResourceDenied if the specified resource is not available. + + If the caller's module is __main__ then automatically return True. The + possibility of False being returned occurs when regrtest.py is + executing. + """ if resource == 'gui' and not _is_gui_available(): raise ResourceDenied(_is_gui_available.reason) + # see if the caller's module is __main__ - if so, treat as if + # the resource was set + if sys._getframe(1).f_globals.get("__name__") == "__main__": + return if not is_resource_enabled(resource): if msg is None: msg = "Use of the %r resource not enabled" % resource diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_asyncio/test_base_events.py --- a/Lib/test/test_asyncio/test_base_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_asyncio/test_base_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -52,20 +52,6 @@ gen = self.loop._make_subprocess_transport(m, m, m, m, m, m, m) self.assertRaises(NotImplementedError, next, iter(gen)) - def test_close(self): - self.assertFalse(self.loop.is_closed()) - self.loop.close() - self.assertTrue(self.loop.is_closed()) - - # it should be possible to call close() more than once - self.loop.close() - self.loop.close() - - # operation blocked when the loop is closed - f = asyncio.Future(loop=self.loop) - self.assertRaises(RuntimeError, self.loop.run_forever) - self.assertRaises(RuntimeError, self.loop.run_until_complete, f) - def test__add_callback_handle(self): h = asyncio.Handle(lambda: False, (), self.loop) @@ -597,27 +583,6 @@ self.assertEqual(str(cm.exception), 'Multiple exceptions: err1, err2') - @mock.patch('asyncio.base_events.socket') - def test_create_connection_timeout(self, m_socket): - # Ensure that the socket is closed on timeout - sock = mock.Mock() - m_socket.socket.return_value = sock - - def getaddrinfo(*args, **kw): - fut = asyncio.Future(loop=self.loop) - addr = (socket.AF_INET, socket.SOCK_STREAM, 0, '', - ('127.0.0.1', 80)) - fut.set_result([addr]) - return fut - self.loop.getaddrinfo = getaddrinfo - - with mock.patch.object(self.loop, 'sock_connect', - side_effect=asyncio.TimeoutError): - coro = self.loop.create_connection(MyProto, '127.0.0.1', 80) - with self.assertRaises(asyncio.TimeoutError): - self.loop.run_until_complete(coro) - self.assertTrue(sock.close.called) - def test_create_connection_host_port_sock(self): coro = self.loop.create_connection( MyProto, 'example.com', 80, sock=object()) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_asyncio/test_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -1070,7 +1070,6 @@ def test_internal_fds(self): loop = self.create_event_loop() if not isinstance(loop, selector_events.BaseSelectorEventLoop): - loop.close() self.skipTest('loop is not a BaseSelectorEventLoop') self.assertEqual(1, loop._internal_fds) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_asyncio/test_selector_events.py --- a/Lib/test/test_asyncio/test_selector_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_asyncio/test_selector_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -80,10 +80,7 @@ self.loop._selector.close() self.loop._selector = selector = mock.Mock() - self.assertFalse(self.loop.is_closed()) - self.loop.close() - self.assertTrue(self.loop.is_closed()) self.assertIsNone(self.loop._selector) self.assertIsNone(self.loop._csock) self.assertIsNone(self.loop._ssock) @@ -92,20 +89,9 @@ csock.close.assert_called_with() remove_reader.assert_called_with(7) - # it should be possible to call close() more than once self.loop.close() self.loop.close() - # operation blocked when the loop is closed - f = asyncio.Future(loop=self.loop) - self.assertRaises(RuntimeError, self.loop.run_forever) - self.assertRaises(RuntimeError, self.loop.run_until_complete, f) - fd = 0 - def callback(): - pass - self.assertRaises(RuntimeError, self.loop.add_reader, fd, callback) - self.assertRaises(RuntimeError, self.loop.add_writer, fd, callback) - def test_close_no_selector(self): ssock = self.loop._ssock csock = self.loop._csock @@ -115,6 +101,9 @@ self.loop._selector = None self.loop.close() self.assertIsNone(self.loop._selector) + self.assertFalse(ssock.close.called) + self.assertFalse(csock.close.called) + self.assertFalse(remove_reader.called) def test_socketpair(self): self.assertRaises(NotImplementedError, self.loop._socketpair) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_asyncio/test_unix_events.py --- a/Lib/test/test_asyncio/test_unix_events.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_asyncio/test_unix_events.py Thu Jun 05 21:36:59 2014 +0200 @@ -256,24 +256,6 @@ 'A UNIX Domain Socket was expected'): self.loop.run_until_complete(coro) - @mock.patch('asyncio.unix_events.socket') - def test_create_unix_server_bind_error(self, m_socket): - # Ensure that the socket is closed on any bind error - sock = mock.Mock() - m_socket.socket.return_value = sock - - sock.bind.side_effect = OSError - coro = self.loop.create_unix_server(lambda: None, path="/test") - with self.assertRaises(OSError): - self.loop.run_until_complete(coro) - self.assertTrue(sock.close.called) - - sock.bind.side_effect = MemoryError - coro = self.loop.create_unix_server(lambda: None, path="/test") - with self.assertRaises(MemoryError): - self.loop.run_until_complete(coro) - self.assertTrue(sock.close.called) - def test_create_unix_connection_path_sock(self): coro = self.loop.create_unix_connection( lambda: None, '/dev/null', sock=object()) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_asyncio/test_windows_utils.py --- a/Lib/test/test_asyncio/test_windows_utils.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_asyncio/test_windows_utils.py Thu Jun 05 21:36:59 2014 +0200 @@ -51,15 +51,6 @@ self.assertRaises(ValueError, windows_utils.socketpair, proto=1) - @mock.patch('asyncio.windows_utils.socket') - def test_winsocketpair_close(self, m_socket): - m_socket.AF_INET = socket.AF_INET - m_socket.SOCK_STREAM = socket.SOCK_STREAM - sock = mock.Mock() - m_socket.socket.return_value = sock - sock.bind.side_effect = OSError - self.assertRaises(OSError, windows_utils.socketpair) - self.assertTrue(sock.close.called) class PipeTests(unittest.TestCase): @@ -164,8 +155,6 @@ self.assertTrue(msg.upper().rstrip().startswith(out)) self.assertTrue(b"stderr".startswith(err)) - p.wait() - if __name__ == '__main__': unittest.main() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_codecmaps_hk.py --- a/Lib/test/test_codecmaps_hk.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_codecmaps_hk.py Thu Jun 05 21:36:59 2014 +0200 @@ -16,4 +16,5 @@ support.run_unittest(__name__) if __name__ == "__main__": + support.use_resources = ['urlfetch'] test_main() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_dbm_dumb.py --- a/Lib/test/test_dbm_dumb.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_dbm_dumb.py Thu Jun 05 21:36:59 2014 +0200 @@ -217,14 +217,6 @@ self.assertEqual(str(cm.exception), "DBM object has already been closed") - def test_create_new(self): - with dumbdbm.open(_fname, 'n') as f: - for k in self._dict: - f[k] = self._dict[k] - - with dumbdbm.open(_fname, 'n') as f: - self.assertEqual(f.keys(), []) - def tearDown(self): _delete_files() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_decimal.py --- a/Lib/test/test_decimal.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_decimal.py Thu Jun 05 21:36:59 2014 +0200 @@ -5556,7 +5556,7 @@ all_tests.insert(1, SignatureTest) -def test_main(arith=None, verbose=None, todo_tests=None, debug=None): +def test_main(arith=False, verbose=None, todo_tests=None, debug=None): """ Execute the tests. Runs all arithmetic tests if arith is True or if the "decimal" resource @@ -5566,7 +5566,7 @@ init(C) init(P) global TEST_ALL, DEBUG - TEST_ALL = arith if arith is not None else is_resource_enabled('decimal') + TEST_ALL = arith or is_resource_enabled('decimal') DEBUG = debug if todo_tests is None: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_grammar.py --- a/Lib/test/test_grammar.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_grammar.py Thu Jun 05 21:36:59 2014 +0200 @@ -80,12 +80,6 @@ x = .3e14 x = 3.1e4 - def test_float_exponent_tokenization(self): - # See issue 21642. - self.assertEqual(1 if 1else 0, 1) - self.assertEqual(1 if 0else 0, 0) - self.assertRaises(SyntaxError, eval, "0 if 1Else 0") - def test_string_literals(self): x = ''; y = ""; self.assertTrue(len(x) == 0 and x == y) x = '\''; y = "'"; self.assertTrue(len(x) == 1 and x == y and ord(x) == 39) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_idle.py --- a/Lib/test/test_idle.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_idle.py Thu Jun 05 21:36:59 2014 +0200 @@ -13,4 +13,8 @@ load_tests = idletest.load_tests if __name__ == '__main__': + # Until unittest supports resources, we emulate regrtest's -ugui + # so loaded tests run the same as if textually present here. + # If any Idle test ever needs another resource, add it to the list. + support.use_resources = ['gui'] # use_resources is initially None unittest.main(verbosity=2, exit=False) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_imaplib.py --- a/Lib/test/test_imaplib.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_imaplib.py Thu Jun 05 21:36:59 2014 +0200 @@ -501,4 +501,5 @@ if __name__ == "__main__": + support.use_resources = ['network'] unittest.main() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_io.py --- a/Lib/test/test_io.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_io.py Thu Jun 05 21:36:59 2014 +0200 @@ -653,20 +653,6 @@ fileio.close() f2.readline() - def test_nonbuffered_textio(self): - with warnings.catch_warnings(record=True) as recorded: - with self.assertRaises(ValueError): - self.open(support.TESTFN, 'w', buffering=0) - support.gc_collect() - self.assertEqual(recorded, []) - - def test_invalid_newline(self): - with warnings.catch_warnings(record=True) as recorded: - with self.assertRaises(ValueError): - self.open(support.TESTFN, 'w', newline='invalid') - support.gc_collect() - self.assertEqual(recorded, []) - class CIOTest(IOTest): @@ -806,27 +792,9 @@ with self.assertRaises(OSError) as err: # exception not swallowed b.close() self.assertEqual(err.exception.args, ('close',)) - self.assertIsInstance(err.exception.__context__, OSError) self.assertEqual(err.exception.__context__.args, ('flush',)) self.assertFalse(b.closed) - def test_nonnormalized_close_error_on_close(self): - # Issue #21677 - raw = self.MockRawIO() - def bad_flush(): - raise non_existing_flush - def bad_close(): - raise non_existing_close - raw.close = bad_close - b = self.tp(raw) - b.flush = bad_flush - with self.assertRaises(NameError) as err: # exception not swallowed - b.close() - self.assertIn('non_existing_close', str(err.exception)) - self.assertIsInstance(err.exception.__context__, NameError) - self.assertIn('non_existing_flush', str(err.exception.__context__)) - self.assertFalse(b.closed) - def test_multi_close(self): raw = self.MockRawIO() b = self.tp(raw) @@ -2608,39 +2576,6 @@ self.assertRaises(OSError, txt.close) # exception not swallowed self.assertTrue(txt.closed) - def test_close_error_on_close(self): - buffer = self.BytesIO(self.testdata) - def bad_flush(): - raise OSError('flush') - def bad_close(): - raise OSError('close') - buffer.close = bad_close - txt = self.TextIOWrapper(buffer, encoding="ascii") - txt.flush = bad_flush - with self.assertRaises(OSError) as err: # exception not swallowed - txt.close() - self.assertEqual(err.exception.args, ('close',)) - self.assertIsInstance(err.exception.__context__, OSError) - self.assertEqual(err.exception.__context__.args, ('flush',)) - self.assertFalse(txt.closed) - - def test_nonnormalized_close_error_on_close(self): - # Issue #21677 - buffer = self.BytesIO(self.testdata) - def bad_flush(): - raise non_existing_flush - def bad_close(): - raise non_existing_close - buffer.close = bad_close - txt = self.TextIOWrapper(buffer, encoding="ascii") - txt.flush = bad_flush - with self.assertRaises(NameError) as err: # exception not swallowed - txt.close() - self.assertIn('non_existing_close', str(err.exception)) - self.assertIsInstance(err.exception.__context__, NameError) - self.assertIn('non_existing_flush', str(err.exception.__context__)) - self.assertFalse(txt.closed) - def test_multi_close(self): txt = self.TextIOWrapper(self.BytesIO(self.testdata), encoding="ascii") txt.close() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_logging.py --- a/Lib/test/test_logging.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_logging.py Thu Jun 05 21:36:59 2014 +0200 @@ -679,8 +679,7 @@ """ def __init__(self, addr, handler, poll_interval, sockmap): - smtpd.SMTPServer.__init__(self, addr, None, map=sockmap, - decode_data=True) + smtpd.SMTPServer.__init__(self, addr, None, map=sockmap) self.port = self.socket.getsockname()[1] self._handler = handler self._thread = None @@ -861,6 +860,9 @@ super(TestTCPServer, self).server_bind() self.port = self.socket.getsockname()[1] + class TestUnixStreamServer(TestTCPServer): + address_family = socket.AF_UNIX + class TestUDPServer(ControlMixin, ThreadingUDPServer): """ A UDP server which is controllable using :class:`ControlMixin`. @@ -908,12 +910,8 @@ super(TestUDPServer, self).server_close() self._closed = True - if hasattr(socket, "AF_UNIX"): - class TestUnixStreamServer(TestTCPServer): - address_family = socket.AF_UNIX - - class TestUnixDatagramServer(TestUDPServer): - address_family = socket.AF_UNIX + class TestUnixDatagramServer(TestUDPServer): + address_family = socket.AF_UNIX # - end of server_helper section @@ -1454,13 +1452,12 @@ os.remove(fn) return fn -@unittest.skipUnless(hasattr(socket, "AF_UNIX"), "Unix sockets required") @unittest.skipUnless(threading, 'Threading required for this test.') class UnixSocketHandlerTest(SocketHandlerTest): """Test for SocketHandler with unix sockets.""" - if threading and hasattr(socket, "AF_UNIX"): + if threading: server_class = TestUnixStreamServer def setUp(self): @@ -1526,13 +1523,13 @@ self.handled.wait() self.assertEqual(self.log_output, "spam\neggs\n") -@unittest.skipUnless(hasattr(socket, "AF_UNIX"), "Unix sockets required") + @unittest.skipUnless(threading, 'Threading required for this test.') class UnixDatagramHandlerTest(DatagramHandlerTest): """Test for DatagramHandler using Unix sockets.""" - if threading and hasattr(socket, "AF_UNIX"): + if threading: server_class = TestUnixDatagramServer def setUp(self): @@ -1601,13 +1598,13 @@ self.handled.wait() self.assertEqual(self.log_output, b'<11>h\xc3\xa4m-sp\xc3\xa4m') -@unittest.skipUnless(hasattr(socket, "AF_UNIX"), "Unix sockets required") + @unittest.skipUnless(threading, 'Threading required for this test.') class UnixSysLogHandlerTest(SysLogHandlerTest): """Test for SysLogHandler with Unix sockets.""" - if threading and hasattr(socket, "AF_UNIX"): + if threading: server_class = TestUnixDatagramServer def setUp(self): diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_pydoc.py --- a/Lib/test/test_pydoc.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_pydoc.py Thu Jun 05 21:36:59 2014 +0200 @@ -47,7 +47,6 @@ builtins.object A B - C \x20\x20\x20\x20 class A(builtins.object) | Hello and goodbye @@ -75,26 +74,6 @@ | Data and other attributes defined here: |\x20\x20 | NO_MEANING = 'eggs' -\x20\x20\x20\x20 - class C(builtins.object) - | Methods defined here: - |\x20\x20 - | get_answer(self) - | Return say_no() - |\x20\x20 - | is_it_true(self) - | Return self.get_answer() - |\x20\x20 - | say_no(self) - |\x20\x20 - | ---------------------------------------------------------------------- - | Data descriptors defined here: - |\x20\x20 - | __dict__ - | dictionary for instance variables (if defined) - |\x20\x20 - | __weakref__ - | list of weak references to the object (if defined) FUNCTIONS doc_func() @@ -145,7 +124,6 @@
A
B -
C
@@ -187,28 +165,6 @@ Data and other attributes defined here:
NO_MEANING = 'eggs'
-

- - - -\x20\x20\x20\x20 - -
 
-class C(builtins.object)
    Methods defined here:
-
get_answer(self)
Return say_no()
- -
is_it_true(self)
Return self.get_answer()
- -
say_no(self)
- -
-Data descriptors defined here:
-
__dict__
-
dictionary for instance variables (if defined)
-
-
__weakref__
-
list of weak references to the object (if defined)
-

@@ -446,14 +402,6 @@ result, doc_loc = get_pydoc_text(xml.etree) self.assertEqual(doc_loc, "", "MODULE DOCS incorrectly includes a link") - def test_getpager_with_stdin_none(self): - previous_stdin = sys.stdin - try: - sys.stdin = None - pydoc.getpager() # Shouldn't fail. - finally: - sys.stdin = previous_stdin - def test_non_str_name(self): # issue14638 # Treat illegal (non-str) name like no name diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_robotparser.py Thu Jun 05 21:36:59 2014 +0200 @@ -291,4 +291,5 @@ return suite if __name__=='__main__': + support.use_resources = ['network'] unittest.main() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_smtpd.py --- a/Lib/test/test_smtpd.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_smtpd.py Thu Jun 05 21:36:59 2014 +0200 @@ -7,18 +7,13 @@ class DummyServer(smtpd.SMTPServer): - def __init__(self, localaddr, remoteaddr, decode_data=True): - smtpd.SMTPServer.__init__(self, localaddr, remoteaddr, - decode_data=decode_data) + def __init__(self, localaddr, remoteaddr): + smtpd.SMTPServer.__init__(self, localaddr, remoteaddr) self.messages = [] - if decode_data: - self.return_status = 'return status' - else: - self.return_status = b'return status' def process_message(self, peer, mailfrom, rcpttos, data): self.messages.append((peer, mailfrom, rcpttos, data)) - if data == self.return_status: + if data == 'return status': return '250 Okish' @@ -36,9 +31,9 @@ smtpd.socket = asyncore.socket = mock_socket def test_process_message_unimplemented(self): - server = smtpd.SMTPServer('a', 'b', decode_data=True) + server = smtpd.SMTPServer('a', 'b') conn, addr = server.accept() - channel = smtpd.SMTPChannel(server, conn, addr, decode_data=True) + channel = smtpd.SMTPChannel(server, conn, addr) def write_line(line): channel.socket.queue_recv(line) @@ -50,10 +45,6 @@ write_line(b'DATA') self.assertRaises(NotImplementedError, write_line, b'spam\r\n.\r\n') - def test_decode_data_default_warns(self): - with self.assertWarns(DeprecationWarning): - smtpd.SMTPServer('a', 'b') - def tearDown(self): asyncore.close_all() asyncore.socket = smtpd.socket = socket @@ -66,8 +57,7 @@ self.debug = smtpd.DEBUGSTREAM = io.StringIO() self.server = DummyServer('a', 'b') conn, addr = self.server.accept() - self.channel = smtpd.SMTPChannel(self.server, conn, addr, - decode_data=True) + self.channel = smtpd.SMTPChannel(self.server, conn, addr) def tearDown(self): asyncore.close_all() @@ -205,24 +195,47 @@ def test_command_too_long(self): self.write_line(b'HELO example') self.write_line(b'MAIL from: ' + - b'a' * self.channel.command_size_limit + + b'a' * self.channel.command_size_limits['MAIL'] + b'@example') self.assertEqual(self.channel.socket.last, b'500 Error: line too long\r\n') def test_MAIL_command_limit_extended_with_SIZE(self): self.write_line(b'EHLO example') - fill_len = self.channel.command_size_limit - len('MAIL from:<@example>') + fill_len = 512 - len('MAIL from:<@example>') + self.write_line(b'MAIL from:<' + + b'a' * (fill_len + 26) + + b'@example> SIZE=1234') + self.assertEqual(self.channel.socket.last, + b'500 Error: line too long\r\n') self.write_line(b'MAIL from:<' + b'a' * fill_len + b'@example> SIZE=1234') self.assertEqual(self.channel.socket.last, b'250 OK\r\n') - self.write_line(b'MAIL from:<' + - b'a' * (fill_len + 26) + - b'@example> SIZE=1234') - self.assertEqual(self.channel.socket.last, - b'500 Error: line too long\r\n') + def test_unicode_output_raises_error_by_default(self): + self.write_line('EHLO éxämplé'.encode('utf-8')) + self.assertEqual( + self.channel.socket.last, + b'500 Error: unknown characters in input.\r\n') + + def test_MAIL_command_rejects_SMTPUTF8_by_default(self): + self.write_line(b'EHLO example') + self.write_line( + b'MAIL from: BODY=8BITMIME SMTPUTF8') + self.assertEqual( + self.channel.socket.last, + b'555 MAIL FROM parameter SMTPUTF8 is not enabled.\r\n') + + def test_MAIL_command_accepts_SMTPUTF8_when_announced(self): + self.channel.enable_SMTPUTF8 = True + self.write_line(b'EHLO example') + self.write_line( + 'MAIL from: BODY=8BITMIME SMTPUTF8'.encode( + 'utf-8') + ) + self.assertEqual(self.channel.socket.last, b'250 OK\r\n') + self.channel.enable_SMTPUTF8 = False def test_data_longer_than_default_data_size_limit(self): # Hack the default so we don't have to generate so much data. @@ -512,12 +525,6 @@ with support.check_warnings(('', DeprecationWarning)): self.channel._SMTPChannel__addr = 'spam' - def test_decode_data_default_warning(self): - server = DummyServer('a', 'b') - conn, addr = self.server.accept() - with self.assertWarns(DeprecationWarning): - smtpd.SMTPChannel(server, conn, addr) - class SMTPDChannelWithDataSizeLimitTest(unittest.TestCase): @@ -528,8 +535,7 @@ self.server = DummyServer('a', 'b') conn, addr = self.server.accept() # Set DATA size limit to 32 bytes for easy testing - self.channel = smtpd.SMTPChannel(self.server, conn, addr, 32, - decode_data=True) + self.channel = smtpd.SMTPChannel(self.server, conn, addr, 32) def tearDown(self): asyncore.close_all() @@ -570,92 +576,5 @@ b'552 Error: Too much mail data\r\n') -class SMTPDChannelWithDecodeDataFalse(unittest.TestCase): - - def setUp(self): - smtpd.socket = asyncore.socket = mock_socket - self.old_debugstream = smtpd.DEBUGSTREAM - self.debug = smtpd.DEBUGSTREAM = io.StringIO() - self.server = DummyServer('a', 'b', decode_data=False) - conn, addr = self.server.accept() - # Set decode_data to False - self.channel = smtpd.SMTPChannel(self.server, conn, addr, - decode_data=False) - - def tearDown(self): - asyncore.close_all() - asyncore.socket = smtpd.socket = socket - smtpd.DEBUGSTREAM = self.old_debugstream - - def write_line(self, line): - self.channel.socket.queue_recv(line) - self.channel.handle_read() - - def test_ascii_data(self): - self.write_line(b'HELO example') - self.write_line(b'MAIL From:eggs@example') - self.write_line(b'RCPT To:spam@example') - self.write_line(b'DATA') - self.write_line(b'plain ascii text') - self.write_line(b'.') - self.assertEqual(self.channel.received_data, b'plain ascii text') - - def test_utf8_data(self): - self.write_line(b'HELO example') - self.write_line(b'MAIL From:eggs@example') - self.write_line(b'RCPT To:spam@example') - self.write_line(b'DATA') - self.write_line(b'utf8 enriched text: \xc5\xbc\xc5\xba\xc4\x87') - self.write_line(b'and some plain ascii') - self.write_line(b'.') - self.assertEqual( - self.channel.received_data, - b'utf8 enriched text: \xc5\xbc\xc5\xba\xc4\x87\n' - b'and some plain ascii') - - -class SMTPDChannelWithDecodeDataTrue(unittest.TestCase): - - def setUp(self): - smtpd.socket = asyncore.socket = mock_socket - self.old_debugstream = smtpd.DEBUGSTREAM - self.debug = smtpd.DEBUGSTREAM = io.StringIO() - self.server = DummyServer('a', 'b') - conn, addr = self.server.accept() - # Set decode_data to True - self.channel = smtpd.SMTPChannel(self.server, conn, addr, - decode_data=True) - - def tearDown(self): - asyncore.close_all() - asyncore.socket = smtpd.socket = socket - smtpd.DEBUGSTREAM = self.old_debugstream - - def write_line(self, line): - self.channel.socket.queue_recv(line) - self.channel.handle_read() - - def test_ascii_data(self): - self.write_line(b'HELO example') - self.write_line(b'MAIL From:eggs@example') - self.write_line(b'RCPT To:spam@example') - self.write_line(b'DATA') - self.write_line(b'plain ascii text') - self.write_line(b'.') - self.assertEqual(self.channel.received_data, 'plain ascii text') - - def test_utf8_data(self): - self.write_line(b'HELO example') - self.write_line(b'MAIL From:eggs@example') - self.write_line(b'RCPT To:spam@example') - self.write_line(b'DATA') - self.write_line(b'utf8 enriched text: \xc5\xbc\xc5\xba\xc4\x87') - self.write_line(b'and some plain ascii') - self.write_line(b'.') - self.assertEqual( - self.channel.received_data, - 'utf8 enriched text: żźć\nand some plain ascii') - - if __name__ == "__main__": unittest.main() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_smtplib.py --- a/Lib/test/test_smtplib.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_smtplib.py Thu Jun 05 21:36:59 2014 +0200 @@ -184,8 +184,7 @@ self.old_DEBUGSTREAM = smtpd.DEBUGSTREAM smtpd.DEBUGSTREAM = io.StringIO() # Pick a random unused port by passing 0 for the port number - self.serv = smtpd.DebuggingServer((HOST, 0), ('nowhere', -1), - decode_data=True) + self.serv = smtpd.DebuggingServer((HOST, 0), ('nowhere', -1)) # Keep a note of what port was assigned self.port = self.serv.socket.getsockname()[1] serv_args = (self.serv, self.serv_evt, self.client_evt) @@ -309,6 +308,13 @@ mexpect = '%s%s\n%s' % (MSG_BEGIN, m.decode('ascii'), MSG_END) self.assertEqual(self.output.getvalue(), mexpect) + self.client_evt.set() + self.serv_evt.wait() + self.output.flush() + mexpect = '%s%s\n%s' % (MSG_BEGIN, m.decode('ascii'), MSG_END) + self.assertEqual(self.output.getvalue(), mexpect) + + def testSendNeedingDotQuote(self): # Issue 12283 m = '.A test\n.mes.sage.' @@ -626,13 +632,17 @@ self._extrafeatures = ''.join( [ "250-{0}\r\n".format(x) for x in extra_features ]) super(SimSMTPChannel, self).__init__(*args, **kw) + self.enable_SMTPUTF8 = True def smtp_EHLO(self, arg): - resp = ('250-testhost\r\n' - '250-EXPN\r\n' - '250-SIZE 20000000\r\n' - '250-STARTTLS\r\n' - '250-DELIVERBY\r\n') + resp = ( + '250-testhost\r\n' + '250-EXPN\r\n' + '250-SIZE 20000000\r\n' + '250-STARTTLS\r\n' + '250-DELIVERBY\r\n' + '250-SMTPUTF8\r\n' + '250-8BITMIME\r\n') resp = resp + self._extrafeatures + '250 HELP' self.push(resp) self.seen_greeting = arg @@ -720,8 +730,7 @@ def handle_accepted(self, conn, addr): self._SMTPchannel = self.channel_class( - self._extra_features, self, conn, addr, - decode_data=self._decode_data) + self._extra_features, self, conn, addr) def process_message(self, peer, mailfrom, rcpttos, data): pass @@ -744,7 +753,7 @@ self.serv_evt = threading.Event() self.client_evt = threading.Event() # Pick a random unused port by passing 0 for the port number - self.serv = SimSMTPServer((HOST, 0), ('nowhere', -1), decode_data=True) + self.serv = SimSMTPServer((HOST, 0), ('nowhere', -1)) # Keep a note of what port was assigned self.port = self.serv.socket.getsockname()[1] serv_args = (self.serv, self.serv_evt, self.client_evt) @@ -775,12 +784,15 @@ self.assertEqual(smtp.esmtp_features, {}) # features expected from the test server - expected_features = {'expn':'', - 'size': '20000000', - 'starttls': '', - 'deliverby': '', - 'help': '', - } + expected_features = { + 'expn':'', + 'size': '20000000', + 'starttls': '', + 'deliverby': '', + 'help': '', + 'smtputf8': '', + '8bitmime': '', + } smtp.ehlo() self.assertEqual(smtp.esmtp_features, expected_features) @@ -789,6 +801,21 @@ self.assertFalse(smtp.has_extn('unsupported-feature')) smtp.quit() + def test_send_unicode_without_SMTPUTF8(self): + m = '¡a test message containing unicode!' + smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + self.assertRaises(UnicodeEncodeError, smtp.sendmail, 'Alice', 'Bob', m) + + def test_send_unicode_with_SMTPUTF8(self): + m = '¡a test message containing unicode!' + smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3) + smtp.ehlo() + self.assertTrue(smtp.does_esmtp) + self.assertTrue(smtp.has_extn('smtputf8')) + smtp.sendmail('John', 'Sally', m, + mail_options=['BODY=8BITMIME', 'SMTPUTF8']) + + def testVRFY(self): smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_socket.py --- a/Lib/test/test_socket.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_socket.py Thu Jun 05 21:36:59 2014 +0200 @@ -19,8 +19,6 @@ import math import pickle import struct -import random -import string try: import multiprocessing except ImportError: @@ -5079,275 +5077,6 @@ source.close() -@unittest.skipUnless(thread, 'Threading required for this test.') -class SendfileUsingSendTest(ThreadedTCPSocketTest): - """ - Test the send() implementation of socket.sendfile(). - """ - - FILESIZE = (10 * 1024 * 1024) # 10MB - BUFSIZE = 8192 - FILEDATA = b"" - TIMEOUT = 2 - - @classmethod - def setUpClass(cls): - def chunks(total, step): - assert total >= step - while total > step: - yield step - total -= step - if total: - yield total - - chunk = b"".join([random.choice(string.ascii_letters).encode() - for i in range(cls.BUFSIZE)]) - with open(support.TESTFN, 'wb') as f: - for csize in chunks(cls.FILESIZE, cls.BUFSIZE): - f.write(chunk) - with open(support.TESTFN, 'rb') as f: - cls.FILEDATA = f.read() - assert len(cls.FILEDATA) == cls.FILESIZE - - @classmethod - def tearDownClass(cls): - support.unlink(support.TESTFN) - - def accept_conn(self): - self.serv.settimeout(self.TIMEOUT) - conn, addr = self.serv.accept() - conn.settimeout(self.TIMEOUT) - self.addCleanup(conn.close) - return conn - - def recv_data(self, conn): - received = [] - while True: - chunk = conn.recv(self.BUFSIZE) - if not chunk: - break - received.append(chunk) - return b''.join(received) - - def meth_from_sock(self, sock): - # Depending on the mixin class being run return either send() - # or sendfile() method implementation. - return getattr(sock, "_sendfile_use_send") - - # regular file - - def _testRegularFile(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address) as sock, file as file: - meth = self.meth_from_sock(sock) - sent = meth(file) - self.assertEqual(sent, self.FILESIZE) - self.assertEqual(file.tell(), self.FILESIZE) - - def testRegularFile(self): - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(len(data), self.FILESIZE) - self.assertEqual(data, self.FILEDATA) - - # non regular file - - def _testNonRegularFile(self): - address = self.serv.getsockname() - file = io.BytesIO(self.FILEDATA) - with socket.create_connection(address) as sock, file as file: - sent = sock.sendfile(file) - self.assertEqual(sent, self.FILESIZE) - self.assertEqual(file.tell(), self.FILESIZE) - self.assertRaises(socket._GiveupOnSendfile, - sock._sendfile_use_sendfile, file) - - def testNonRegularFile(self): - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(len(data), self.FILESIZE) - self.assertEqual(data, self.FILEDATA) - - # empty file - - def _testEmptyFileSend(self): - address = self.serv.getsockname() - filename = support.TESTFN + "2" - with open(filename, 'wb'): - self.addCleanup(support.unlink, filename) - file = open(filename, 'rb') - with socket.create_connection(address) as sock, file as file: - meth = self.meth_from_sock(sock) - sent = meth(file) - self.assertEqual(sent, 0) - self.assertEqual(file.tell(), 0) - - def testEmptyFileSend(self): - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(data, b"") - - # offset - - def _testOffset(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address) as sock, file as file: - meth = self.meth_from_sock(sock) - sent = meth(file, offset=5000) - self.assertEqual(sent, self.FILESIZE - 5000) - self.assertEqual(file.tell(), self.FILESIZE) - - def testOffset(self): - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(len(data), self.FILESIZE - 5000) - self.assertEqual(data, self.FILEDATA[5000:]) - - # count - - def _testCount(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address, timeout=2) as sock, file as file: - count = 5000007 - meth = self.meth_from_sock(sock) - sent = meth(file, count=count) - self.assertEqual(sent, count) - self.assertEqual(file.tell(), count) - - def testCount(self): - count = 5000007 - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(len(data), count) - self.assertEqual(data, self.FILEDATA[:count]) - - # count small - - def _testCountSmall(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address, timeout=2) as sock, file as file: - count = 1 - meth = self.meth_from_sock(sock) - sent = meth(file, count=count) - self.assertEqual(sent, count) - self.assertEqual(file.tell(), count) - - def testCountSmall(self): - count = 1 - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(len(data), count) - self.assertEqual(data, self.FILEDATA[:count]) - - # count + offset - - def _testCountWithOffset(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address, timeout=2) as sock, file as file: - count = 100007 - meth = self.meth_from_sock(sock) - sent = meth(file, offset=2007, count=count) - self.assertEqual(sent, count) - self.assertEqual(file.tell(), count + 2007) - - def testCountWithOffset(self): - count = 100007 - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(len(data), count) - self.assertEqual(data, self.FILEDATA[2007:count+2007]) - - # non blocking sockets are not supposed to work - - def _testNonBlocking(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address) as sock, file as file: - sock.setblocking(False) - meth = self.meth_from_sock(sock) - self.assertRaises(ValueError, meth, file) - self.assertRaises(ValueError, sock.sendfile, file) - - def testNonBlocking(self): - conn = self.accept_conn() - if conn.recv(8192): - self.fail('was not supposed to receive any data') - - # timeout (non-triggered) - - def _testWithTimeout(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address, timeout=2) as sock, file as file: - meth = self.meth_from_sock(sock) - sent = meth(file) - self.assertEqual(sent, self.FILESIZE) - - def testWithTimeout(self): - conn = self.accept_conn() - data = self.recv_data(conn) - self.assertEqual(len(data), self.FILESIZE) - self.assertEqual(data, self.FILEDATA) - - # timeout (triggered) - - def _testWithTimeoutTriggeredSend(self): - address = self.serv.getsockname() - file = open(support.TESTFN, 'rb') - with socket.create_connection(address, timeout=0.01) as sock, \ - file as file: - meth = self.meth_from_sock(sock) - self.assertRaises(socket.timeout, meth, file) - - def testWithTimeoutTriggeredSend(self): - conn = self.accept_conn() - conn.recv(88192) - - # errors - - def _test_errors(self): - pass - - def test_errors(self): - with open(support.TESTFN, 'rb') as file: - with socket.socket(type=socket.SOCK_DGRAM) as s: - meth = self.meth_from_sock(s) - self.assertRaisesRegex( - ValueError, "SOCK_STREAM", meth, file) - with open(support.TESTFN, 'rt') as file: - with socket.socket() as s: - meth = self.meth_from_sock(s) - self.assertRaisesRegex( - ValueError, "binary mode", meth, file) - with open(support.TESTFN, 'rb') as file: - with socket.socket() as s: - meth = self.meth_from_sock(s) - self.assertRaisesRegex(TypeError, "positive integer", - meth, file, count='2') - self.assertRaisesRegex(TypeError, "positive integer", - meth, file, count=0.1) - self.assertRaisesRegex(ValueError, "positive integer", - meth, file, count=0) - self.assertRaisesRegex(ValueError, "positive integer", - meth, file, count=-1) - - -@unittest.skipUnless(thread, 'Threading required for this test.') -@unittest.skipUnless(hasattr(os, "sendfile"), - 'os.sendfile() required for this test.') -class SendfileUsingSendfileTest(SendfileUsingSendTest): - """ - Test the sendfile() implementation of socket.sendfile(). - """ - def meth_from_sock(self, sock): - return getattr(sock, "_sendfile_use_sendfile") - - def test_main(): tests = [GeneralModuleTests, BasicTCPTest, TCPCloserTest, TCPTimeoutTest, TestExceptions, BufferIOTest, BasicTCPTest2, BasicUDPTest, UDPTimeoutTest ] @@ -5400,8 +5129,6 @@ InterruptedRecvTimeoutTest, InterruptedSendTimeoutTest, TestSocketSharing, - SendfileUsingSendTest, - SendfileUsingSendfileTest, ]) thread_info = support.threading_setup() diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_ssl.py Thu Jun 05 21:36:59 2014 +0200 @@ -2957,23 +2957,6 @@ self.assertRaises(ValueError, s.read, 1024) self.assertRaises(ValueError, s.write, b'hello') - def test_sendfile(self): - TEST_DATA = b"x" * 512 - with open(support.TESTFN, 'wb') as f: - f.write(TEST_DATA) - self.addCleanup(support.unlink, support.TESTFN) - context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) - context.verify_mode = ssl.CERT_REQUIRED - context.load_verify_locations(CERTFILE) - context.load_cert_chain(CERTFILE) - server = ThreadedEchoServer(context=context, chatty=False) - with server: - with context.wrap_socket(socket.socket()) as s: - s.connect((HOST, server.port)) - with open(support.TESTFN, 'rb') as file: - s.sendfile(file) - self.assertEqual(s.recv(1024), TEST_DATA) - def test_main(verbose=False): if support.verbose: diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_subprocess.py Thu Jun 05 21:36:59 2014 +0200 @@ -1926,64 +1926,6 @@ "Some fds not in pass_fds were left open") self.assertIn(1, remaining_fds, "Subprocess failed") - - @unittest.skipIf(sys.platform.startswith("freebsd") and - os.stat("/dev").st_dev == os.stat("/dev/fd").st_dev, - "Requires fdescfs mounted on /dev/fd on FreeBSD.") - def test_close_fds_when_max_fd_is_lowered(self): - """Confirm that issue21618 is fixed (may fail under valgrind).""" - fd_status = support.findfile("fd_status.py", subdir="subprocessdata") - - open_fds = set() - # Add a bunch more fds to pass down. - for _ in range(40): - fd = os.open("/dev/null", os.O_RDONLY) - open_fds.add(fd) - - # Leave a two pairs of low ones available for use by the - # internal child error pipe and the stdout pipe. - # We also leave 10 more open as some Python buildbots run into - # "too many open files" errors during the test if we do not. - for fd in sorted(open_fds)[:14]: - os.close(fd) - open_fds.remove(fd) - - for fd in open_fds: - self.addCleanup(os.close, fd) - os.set_inheritable(fd, True) - - max_fd_open = max(open_fds) - - import resource - rlim_cur, rlim_max = resource.getrlimit(resource.RLIMIT_NOFILE) - try: - # 29 is lower than the highest fds we are leaving open. - resource.setrlimit(resource.RLIMIT_NOFILE, (29, rlim_max)) - # Launch a new Python interpreter with our low fd rlim_cur that - # inherits open fds above that limit. It then uses subprocess - # with close_fds=True to get a report of open fds in the child. - # An explicit list of fds to check is passed to fd_status.py as - # letting fd_status rely on its default logic would miss the - # fds above rlim_cur as it normally only checks up to that limit. - p = subprocess.Popen( - [sys.executable, '-c', - textwrap.dedent(""" - import subprocess, sys - subprocess.Popen([sys.executable, {fd_status!r}] + - [str(x) for x in range({max_fd})], - close_fds=True).wait() - """.format(fd_status=fd_status, max_fd=max_fd_open+1))], - stdout=subprocess.PIPE, close_fds=False) - finally: - resource.setrlimit(resource.RLIMIT_NOFILE, (rlim_cur, rlim_max)) - - output, unused_stderr = p.communicate() - remaining_fds = set(map(int, output.strip().split(b','))) - - self.assertFalse(remaining_fds & open_fds, - msg="Some fds were left open.") - - # Mac OS X Tiger (10.4) has a kernel bug: sometimes, the file # descriptor of a pipe closed in the parent process is valid in the # child process according to fstat(), but the mode of the file diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_tk.py --- a/Lib/test/test_tk.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_tk.py Thu Jun 05 21:36:59 2014 +0200 @@ -10,9 +10,15 @@ from tkinter.test import runtktests -def test_main(): +def test_main(enable_gui=False): + if enable_gui: + if support.use_resources is None: + support.use_resources = ['gui'] + elif 'gui' not in support.use_resources: + support.use_resources.append('gui') + support.run_unittest( *runtktests.get_tests(text=False, packages=['test_tkinter'])) if __name__ == '__main__': - test_main() + test_main(enable_gui=True) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_tracemalloc.py --- a/Lib/test/test_tracemalloc.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_tracemalloc.py Thu Jun 05 21:36:59 2014 +0200 @@ -807,12 +807,6 @@ b'number of frames', stderr) - def test_pymem_alloc0(self): - # Issue #21639: Check that PyMem_Malloc(0) with tracemalloc enabled - # does not crash. - code = 'import _testcapi; _testcapi.test_pymem_alloc0(); 1' - assert_python_ok('-X', 'tracemalloc', '-c', code) - def test_main(): support.run_unittest( diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_ttk_guionly.py --- a/Lib/test/test_ttk_guionly.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_ttk_guionly.py Thu Jun 05 21:36:59 2014 +0200 @@ -22,7 +22,13 @@ # assuming ttk is not available raise unittest.SkipTest("ttk not available: %s" % msg) -def test_main(): +def test_main(enable_gui=False): + if enable_gui: + if support.use_resources is None: + support.use_resources = ['gui'] + elif 'gui' not in support.use_resources: + support.use_resources.append('gui') + try: support.run_unittest( *runtktests.get_tests(text=False, packages=['test_ttk'])) @@ -30,4 +36,4 @@ get_tk_root().destroy() if __name__ == '__main__': - test_main() + test_main(enable_gui=True) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_urllib.py --- a/Lib/test/test_urllib.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_urllib.py Thu Jun 05 21:36:59 2014 +0200 @@ -7,7 +7,6 @@ import email.message import io import unittest -from unittest.mock import patch from test import support import os import sys @@ -90,26 +89,6 @@ http.client.HTTPConnection = self._connection_class -class FakeFTPMixin(object): - def fakeftp(self): - class FakeFtpWrapper(object): - def __init__(self, user, passwd, host, port, dirs, timeout=None, - persistent=True): - pass - - def retrfile(self, file, type): - return io.BytesIO(), 0 - - def close(self): - pass - - self._ftpwrapper_class = urllib.request.ftpwrapper - urllib.request.ftpwrapper = FakeFtpWrapper - - def unfakeftp(self): - urllib.request.ftpwrapper = self._ftpwrapper_class - - class urlopen_FileTests(unittest.TestCase): """Test urlopen() opening a temporary file. @@ -216,7 +195,7 @@ self.env.set('NO_PROXY', 'localhost, anotherdomain.com, newdomain.com') self.assertTrue(urllib.request.proxy_bypass_environment('anotherdomain.com')) -class urlopen_HttpTests(unittest.TestCase, FakeHTTPMixin, FakeFTPMixin): +class urlopen_HttpTests(unittest.TestCase, FakeHTTPMixin): """Test urlopen() opening a fake http connection.""" def check_read(self, ver): @@ -330,15 +309,6 @@ self.assertFalse(e.exception.filename) self.assertTrue(e.exception.reason) - @patch.object(urllib.request, 'MAXFTPCACHE', 0) - def test_ftp_cache_pruning(self): - self.fakeftp() - try: - urllib.request.ftpcache['test'] = urllib.request.ftpwrapper('user', 'pass', 'localhost', 21, []) - urlopen('ftp://localhost') - finally: - self.unfakeftp() - def test_userpass_inurl(self): self.fakehttp(b"HTTP/1.0 200 OK\r\n\r\nHello!") diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/test/test_venv.py --- a/Lib/test/test_venv.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/test/test_venv.py Thu Jun 05 21:36:59 2014 +0200 @@ -203,22 +203,17 @@ """ Test upgrading an existing environment directory. """ - # See Issue #21643: the loop needs to run twice to ensure - # that everything works on the upgrade (the first run just creates - # the venv). - for upgrade in (False, True): - builder = venv.EnvBuilder(upgrade=upgrade) - self.run_with_capture(builder.create, self.env_dir) - self.isdir(self.bindir) - self.isdir(self.include) - self.isdir(*self.lib) - fn = self.get_env_file(self.bindir, self.exe) - if not os.path.exists(fn): - # diagnostics for Windows buildbot failures - bd = self.get_env_file(self.bindir) - print('Contents of %r:' % bd) - print(' %r' % os.listdir(bd)) - self.assertTrue(os.path.exists(fn), 'File %r should exist.' % fn) + builder = venv.EnvBuilder(upgrade=True) + self.run_with_capture(builder.create, self.env_dir) + self.isdir(self.bindir) + self.isdir(self.include) + self.isdir(*self.lib) + fn = self.get_env_file(self.bindir, self.exe) + if not os.path.exists(fn): # diagnostics for Windows buildbot failures + bd = self.get_env_file(self.bindir) + print('Contents of %r:' % bd) + print(' %r' % os.listdir(bd)) + self.assertTrue(os.path.exists(fn), 'File %r should exist.' % fn) def test_isolation(self): """ diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/tkinter/__init__.py --- a/Lib/tkinter/__init__.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/tkinter/__init__.py Thu Jun 05 21:36:59 2014 +0200 @@ -2591,19 +2591,22 @@ def activate(self, index): """Activate item identified by INDEX.""" self.tk.call(self._w, 'activate', index) - def bbox(self, index): + def bbox(self, *args): """Return a tuple of X1,Y1,X2,Y2 coordinates for a rectangle - which encloses the item identified by the given index.""" - return self._getints(self.tk.call(self._w, 'bbox', index)) or None + which encloses the item identified by index in ARGS.""" + return self._getints( + self.tk.call((self._w, 'bbox') + args)) or None def curselection(self): - """Return the indices of currently selected item.""" - return self._getints(self.tk.call(self._w, 'curselection')) or () + """Return list of indices of currently selected item.""" + # XXX Ought to apply self._getints()... + return self.tk.splitlist(self.tk.call( + self._w, 'curselection')) def delete(self, first, last=None): - """Delete items from FIRST to LAST (included).""" + """Delete items from FIRST to LAST (not included).""" self.tk.call(self._w, 'delete', first, last) def get(self, first, last=None): - """Get list of items from FIRST to LAST (included).""" - if last is not None: + """Get list of items from FIRST to LAST (not included).""" + if last: return self.tk.splitlist(self.tk.call( self._w, 'get', first, last)) else: @@ -2636,7 +2639,7 @@ self.tk.call(self._w, 'selection', 'anchor', index) select_anchor = selection_anchor def selection_clear(self, first, last=None): - """Clear the selection from FIRST to LAST (included).""" + """Clear the selection from FIRST to LAST (not included).""" self.tk.call(self._w, 'selection', 'clear', first, last) select_clear = selection_clear @@ -2646,7 +2649,7 @@ self._w, 'selection', 'includes', index)) select_includes = selection_includes def selection_set(self, first, last=None): - """Set the selection from FIRST to LAST (included) without + """Set the selection from FIRST to LAST (not included) without changing the currently selected elements.""" self.tk.call(self._w, 'selection', 'set', first, last) select_set = selection_set diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/tkinter/test/runtktests.py --- a/Lib/tkinter/test/runtktests.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/tkinter/test/runtktests.py Thu Jun 05 21:36:59 2014 +0200 @@ -68,4 +68,5 @@ yield test if __name__ == "__main__": + test.support.use_resources = ['gui'] test.support.run_unittest(*get_tests()) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/tkinter/test/test_tkinter/test_widgets.py --- a/Lib/tkinter/test/test_tkinter/test_widgets.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/tkinter/test/test_tkinter/test_widgets.py Thu Jun 05 21:36:59 2014 +0200 @@ -467,7 +467,11 @@ def test_bbox(self): widget = self.create() - self.assertIsBoundingBox(widget.bbox(0)) + bbox = widget.bbox(0) + self.assertEqual(len(bbox), 4) + for item in bbox: + self.assertIsInstance(item, int) + self.assertRaises(tkinter.TclError, widget.bbox, 'noindex') self.assertRaises(tkinter.TclError, widget.bbox, None) self.assertRaises(TypeError, widget.bbox) @@ -620,7 +624,11 @@ def test_bbox(self): widget = self.create() - self.assertIsBoundingBox(widget.bbox('1.1')) + bbox = widget.bbox('1.1') + self.assertEqual(len(bbox), 4) + for item in bbox: + self.assertIsInstance(item, int) + self.assertIsNone(widget.bbox('end')) self.assertRaises(tkinter.TclError, widget.bbox, 'noindex') self.assertRaises(tkinter.TclError, widget.bbox, None) @@ -777,46 +785,6 @@ def test_itemconfigure_selectforeground(self): self.check_itemconfigure('selectforeground', '#654321') - def test_box(self): - lb = self.create() - lb.insert(0, *('el%d' % i for i in range(8))) - lb.pack() - self.assertIsBoundingBox(lb.bbox(0)) - self.assertIsNone(lb.bbox(-1)) - self.assertIsNone(lb.bbox(10)) - self.assertRaises(TclError, lb.bbox, 'noindex') - self.assertRaises(TclError, lb.bbox, None) - self.assertRaises(TypeError, lb.bbox) - self.assertRaises(TypeError, lb.bbox, 0, 1) - - def test_curselection(self): - lb = self.create() - lb.insert(0, *('el%d' % i for i in range(8))) - lb.selection_clear(0, tkinter.END) - lb.selection_set(2, 4) - lb.selection_set(6) - self.assertEqual(lb.curselection(), (2, 3, 4, 6)) - self.assertRaises(TypeError, lb.curselection, 0) - - def test_get(self): - lb = self.create() - lb.insert(0, *('el%d' % i for i in range(8))) - self.assertEqual(lb.get(0), 'el0') - self.assertEqual(lb.get(3), 'el3') - self.assertEqual(lb.get('end'), 'el7') - self.assertEqual(lb.get(8), '') - self.assertEqual(lb.get(-1), '') - self.assertEqual(lb.get(3, 5), ('el3', 'el4', 'el5')) - self.assertEqual(lb.get(5, 'end'), ('el5', 'el6', 'el7')) - self.assertEqual(lb.get(5, 0), ()) - self.assertEqual(lb.get(0, 0), ('el0',)) - self.assertRaises(TclError, lb.get, 'noindex') - self.assertRaises(TclError, lb.get, None) - self.assertRaises(TypeError, lb.get) - self.assertRaises(TclError, lb.get, 'end', 'noindex') - self.assertRaises(TypeError, lb.get, 1, 2, 3) - self.assertRaises(TclError, lb.get, 2.4) - @add_standard_options(PixelSizeTests, StandardOptionsTests) class ScaleTest(AbstractWidgetTest, unittest.TestCase): diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/tkinter/test/test_ttk/test_widgets.py --- a/Lib/tkinter/test/test_ttk/test_widgets.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/tkinter/test/test_ttk/test_widgets.py Thu Jun 05 21:36:59 2014 +0200 @@ -460,7 +460,10 @@ def test_bbox(self): - self.assertIsBoundingBox(self.entry.bbox(0)) + self.assertEqual(len(self.entry.bbox(0)), 4) + for item in self.entry.bbox(0): + self.assertIsInstance(item, int) + self.assertRaises(tkinter.TclError, self.entry.bbox, 'noindex') self.assertRaises(tkinter.TclError, self.entry.bbox, None) @@ -1213,7 +1216,12 @@ self.assertTrue(children) bbox = self.tv.bbox(children[0]) - self.assertIsBoundingBox(bbox) + self.assertEqual(len(bbox), 4) + self.assertIsInstance(bbox, tuple) + for item in bbox: + if not isinstance(item, int): + self.fail("Invalid bounding box: %s" % bbox) + break # compare width in bboxes self.tv['columns'] = ['test'] diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/tkinter/test/widget_tests.py --- a/Lib/tkinter/test/widget_tests.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/tkinter/test/widget_tests.py Thu Jun 05 21:36:59 2014 +0200 @@ -202,16 +202,6 @@ def checkVariableParam(self, widget, name, var): self.checkParam(widget, name, var, conv=str) - def assertIsBoundingBox(self, bbox): - self.assertIsNotNone(bbox) - self.assertIsInstance(bbox, tuple) - if len(bbox) != 4: - self.fail('Invalid bounding box: %r' % (bbox,)) - for item in bbox: - if not isinstance(item, int): - self.fail('Invalid bounding box: %r' % (bbox,)) - break - class StandardOptionsTests: STANDARD_OPTIONS = ( diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/unittest/mock.py --- a/Lib/unittest/mock.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/unittest/mock.py Thu Jun 05 21:36:59 2014 +0200 @@ -1894,7 +1894,7 @@ formatted_args = '' args_string = ', '.join([repr(arg) for arg in args]) kwargs_string = ', '.join([ - '%s=%r' % (key, value) for key, value in sorted(kwargs.items()) + '%s=%r' % (key, value) for key, value in kwargs.items() ]) if args_string: formatted_args = args_string diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/unittest/test/testmock/testmock.py Thu Jun 05 21:36:59 2014 +0200 @@ -1206,12 +1206,6 @@ with self.assertRaises(AssertionError): m.hello.assert_not_called() - #Issue21256 printout of keyword args should be in deterministic order - def test_sorted_call_signature(self): - m = Mock() - m.hello(name='hello', daddy='hero') - text = "call(daddy='hero', name='hello')" - self.assertEquals(repr(m.hello.call_args), text) def test_mock_add_spec(self): class _One(object): diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/urllib/request.py --- a/Lib/urllib/request.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/urllib/request.py Thu Jun 05 21:36:59 2014 +0200 @@ -1911,7 +1911,7 @@ # XXX thread unsafe! if len(self.ftpcache) > MAXFTPCACHE: # Prune the cache, rather arbitrarily - for k in list(self.ftpcache): + for k in self.ftpcache.keys(): if k != key: v = self.ftpcache[k] del self.ftpcache[k] diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Lib/venv/__init__.py --- a/Lib/venv/__init__.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Lib/venv/__init__.py Thu Jun 05 21:36:59 2014 +0200 @@ -30,6 +30,7 @@ import logging import os import shutil +import struct import subprocess import sys import types @@ -139,12 +140,11 @@ create_if_needed(path) create_if_needed(libpath) # Issue 21197: create lib64 as a symlink to lib on 64-bit non-OS X POSIX - if ((sys.maxsize > 2**32) and (os.name == 'posix') and + if ((struct.calcsize('P') == 8) and (os.name == 'posix') and (sys.platform != 'darwin')): p = os.path.join(env_dir, 'lib') link_path = os.path.join(env_dir, 'lib64') - if not os.path.exists(link_path): # Issue #21643 - os.symlink(p, link_path) + os.symlink(p, link_path) context.bin_path = binpath = os.path.join(env_dir, binname) context.bin_name = binname context.env_exe = os.path.join(binpath, exename) @@ -215,8 +215,7 @@ # Issue 18807: make copies if # symlinks are not wanted copier(context.env_exe, path) - if not os.path.islink(path): - os.chmod(path, 0o755) + os.chmod(path, 0o755) else: subdir = 'DLLs' include = self.include_binary @@ -238,8 +237,7 @@ if 'init.tcl' in files: tcldir = os.path.basename(root) tcldir = os.path.join(context.env_dir, 'Lib', tcldir) - if not os.path.exists(tcldir): - os.makedirs(tcldir) + os.makedirs(tcldir) src = os.path.join(root, 'init.tcl') dst = os.path.join(tcldir, 'init.tcl') shutil.copyfile(src, dst) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Misc/ACKS --- a/Misc/ACKS Wed Jun 11 12:27:40 2014 -0400 +++ b/Misc/ACKS Thu Jun 05 21:36:59 2014 +0200 @@ -32,7 +32,6 @@ Ray Allen Billy G. Allie Kevin Altis -Skyler Leigh Amador Joe Amenta A. Amoroso Mark Anacker @@ -197,7 +196,6 @@ Alastair Burt Tarn Weisner Burton Lee Busby -Katherine Busch Ralph Butler Nicolas Cadou Jp Calderone @@ -755,7 +753,6 @@ Chris Lawrence Brian Leair Mathieu Leduc-Hamel -Amandine Lee Antony Lee Christopher Lee Inyeol Lee @@ -827,7 +824,6 @@ Nick Maclaren Don MacMillen Tomasz Maćkowiak -Wolfgang Maier Steve Majewski Marek Majkowski Grzegorz Makarewicz diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Misc/NEWS --- a/Misc/NEWS Wed Jun 11 12:27:40 2014 -0400 +++ b/Misc/NEWS Thu Jun 05 21:36:59 2014 +0200 @@ -10,10 +10,6 @@ Core and Builtins ----------------- -- Issue #21642: If the conditional if-else expression, allow an integer written - with no space between itself and the ``else`` keyword (e.g. ``True if 42else - False``) to be valid syntax. - - Issue #21523: Fix over-pessimistic computation of the stack effect of some opcodes in the compiler. This also fixes a quadratic compilation time issue noticeable when compiling code with a large number of "and" @@ -30,9 +26,10 @@ internal iteration logic. - Issue #21233: Add new C functions: PyMem_RawCalloc(), PyMem_Calloc(), - PyObject_Calloc(), _PyObject_GC_Calloc(). bytes(int) is now using - ``calloc()`` instead of ``malloc()`` for large objects which is faster and - use less memory. + PyObject_Calloc(), _PyObject_GC_Calloc(). bytes(int) and bytearray(int) + are now using ``calloc()`` instead of ``malloc()`` for large objects which + is faster and use less memory (until the bytearray buffer is filled with + data). - Issue #21377: PyBytes_Concat() now tries to concatenate in-place when the first argument has a reference count of 1. Patch by Nikolaus Rath. @@ -92,39 +89,6 @@ Library ------- -- Issue 17552: new socket.sendfile() method allowing to send a file over a - socket by using high-performance os.sendfile() on UNIX. - Patch by Giampaolo Rodola'. - -- Issue #18039: dbm.dump.open() now always creates a new database when the - flag has the value 'n'. Patch by Claudiu Popa. - -- Issue #21326: Add a new is_closed() method to asyncio.BaseEventLoop. - run_forever() and run_until_complete() methods of asyncio.BaseEventLoop now - raise an exception if the event loop was closed. - -- Issue #21310: Fixed possible resource leak in failed open(). - -- Issue #21256: Printout of keyword args should be in deterministic order in - a mock function call. This will help to write better doctests. - -- Issue #21677: Fixed chaining nonnormalized exceptions in io close() methods. - -- Issue #11709: Fix the pydoc.help function to not fail when sys.stdin is not a - valid file. - -- Issue #21515: tempfile.TemporaryFile now uses os.O_TMPFILE flag is available. - -- Issue #13223: Fix pydoc.writedoc so that the HTML documentation for methods - that use 'self' in the example code is generated correctly. - -- Issue #21463: In urllib.request, fix pruning of the FTP cache. - -- Issue #21618: The subprocess module could fail to close open fds that were - inherited by the calling process and already higher than POSIX resource - limits would otherwise allow. On systems with a functioning /proc/self/fd - or /dev/fd interface the max is now ignored and all fds are closed. - - Issue #20383: Introduce importlib.util.module_from_spec() as the preferred way to create a new module. @@ -482,15 +446,6 @@ IDLE ---- -- Issue #18910: Add unittest for textView. Patch by Phil Webster. - -- Issue #18292: Add unittest for AutoExpand. Patch by Saihadhav Heblikar. - -- Issue #18409: Add unittest for AutoComplete. Patch by Phil Webster. - -- Issue #21477: htest.py - Improve framework, complete set of tests. - Patches by Saimadhav Heblikar - - Issue #18104: Add idlelib/idle_test/htest.py with a few sample tests to begin consolidating and improving human-validated tests of Idle. Change other files as needed to work with htest. Running the module as __main__ runs all tests. @@ -514,6 +469,8 @@ - Issue #21037: Add a build option to enable AddressSanitizer support. +- The Windows build now includes OpenSSL 1.0.1g + - Issue #19962: The Windows build process now creates "python.bat" in the root of the source tree, which passes all arguments through to the most recently built interpreter. @@ -536,8 +493,6 @@ - Issue #15968: Incorporated Tcl, Tk, and Tix builds into the Windows build solution. -- Issue #17095: Fix Modules/Setup *shared* support. - C API ----- - Issue #20942: PyImport_ImportFrozenModuleObject() no longer sets __file__ to @@ -570,13 +525,6 @@ Tests ----- -- Issue #18492: All resources are now allowed when tests are not run by - regrtest.py. - -- Issue #21634: Fix pystone micro-benchmark: use floor division instead of true - division to benchmark integers instead of floating point numbers. Set pystone - version to 1.2. Patch written by Lennart Regebro. - - Issue #21605: Added tests for Tkinter images. - Issue #21493: Added test for ntpath.expanduser(). Original patch by @@ -623,8 +571,6 @@ Tools/Demos ----------- -- Issue #21629: Fix Argument Clinic's "--converters" feature. - - Add support for ``yield from`` to 2to3. - Add support for the PEP 465 matrix multiplication operator to 2to3. @@ -637,12 +583,6 @@ - Issue #20535: PYTHONWARNING no longer affects the run_tests.py script. Patch by Arfrever Frehtes Taifersar Arahesis. -Windows -------- - -- Issue #21671, CVE-2014-0224: The bundled version of OpenSSL has been - updated to 1.0.1h. - What's New in Python 3.4.0? =========================== diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/_io/_iomodule.c --- a/Modules/_io/_iomodule.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/_io/_iomodule.c Thu Jun 05 21:36:59 2014 +0200 @@ -235,12 +235,11 @@ char rawmode[6], *m; int line_buffering, isatty; - PyObject *raw, *modeobj = NULL, *buffer, *wrapper, *result = NULL; + PyObject *raw, *modeobj = NULL, *buffer = NULL, *wrapper = NULL; _Py_IDENTIFIER(isatty); _Py_IDENTIFIER(fileno); _Py_IDENTIFIER(mode); - _Py_IDENTIFIER(close); if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|sizzziO:open", kwlist, &file, &mode, &buffering, @@ -355,7 +354,6 @@ "OsiO", file, rawmode, closefd, opener); if (raw == NULL) return NULL; - result = raw; modeobj = PyUnicode_FromString(mode); if (modeobj == NULL) @@ -414,7 +412,7 @@ } Py_DECREF(modeobj); - return result; + return raw; } /* wraps into a buffered file */ @@ -435,16 +433,15 @@ buffer = PyObject_CallFunction(Buffered_class, "Oi", raw, buffering); } + Py_CLEAR(raw); if (buffer == NULL) goto error; - result = buffer; - Py_DECREF(raw); /* if binary, returns the buffered file */ if (binary) { Py_DECREF(modeobj); - return result; + return buffer; } /* wraps into a TextIOWrapper */ @@ -453,35 +450,20 @@ buffer, encoding, errors, newline, line_buffering); + Py_CLEAR(buffer); if (wrapper == NULL) goto error; - result = wrapper; - Py_DECREF(buffer); if (_PyObject_SetAttrId(wrapper, &PyId_mode, modeobj) < 0) goto error; Py_DECREF(modeobj); - return result; + return wrapper; error: - if (result != NULL) { - PyObject *exc, *val, *tb; - PyErr_Fetch(&exc, &val, &tb); - if (_PyObject_CallMethodId(result, &PyId_close, NULL) != NULL) - PyErr_Restore(exc, val, tb); - else { - PyObject *exc2, *val2, *tb2; - PyErr_Fetch(&exc2, &val2, &tb2); - PyErr_NormalizeException(&exc, &val, &tb); - Py_XDECREF(exc); - Py_XDECREF(tb); - PyErr_NormalizeException(&exc2, &val2, &tb2); - PyException_SetContext(val2, val); - PyErr_Restore(exc2, val2, tb2); - } - Py_DECREF(result); - } + Py_XDECREF(raw); Py_XDECREF(modeobj); + Py_XDECREF(buffer); + Py_XDECREF(wrapper); return NULL; } diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/_io/bufferedio.c --- a/Modules/_io/bufferedio.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/_io/bufferedio.c Thu Jun 05 21:36:59 2014 +0200 @@ -548,14 +548,13 @@ PyErr_Restore(exc, val, tb); } else { - PyObject *exc2, *val2, *tb2; - PyErr_Fetch(&exc2, &val2, &tb2); - PyErr_NormalizeException(&exc, &val, &tb); + PyObject *val2; Py_DECREF(exc); Py_XDECREF(tb); - PyErr_NormalizeException(&exc2, &val2, &tb2); + PyErr_Fetch(&exc, &val2, &tb); + PyErr_NormalizeException(&exc, &val2, &tb); PyException_SetContext(val2, val); - PyErr_Restore(exc2, val2, tb2); + PyErr_Restore(exc, val2, tb); } } diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/_io/textio.c --- a/Modules/_io/textio.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/_io/textio.c Thu Jun 05 21:36:59 2014 +0200 @@ -2619,14 +2619,13 @@ PyErr_Restore(exc, val, tb); } else { - PyObject *exc2, *val2, *tb2; - PyErr_Fetch(&exc2, &val2, &tb2); - PyErr_NormalizeException(&exc, &val, &tb); + PyObject *val2; Py_DECREF(exc); Py_XDECREF(tb); - PyErr_NormalizeException(&exc2, &val2, &tb2); + PyErr_Fetch(&exc, &val2, &tb); + PyErr_NormalizeException(&exc, &val2, &tb); PyException_SetContext(val2, val); - PyErr_Restore(exc2, val2, tb2); + PyErr_Restore(exc, val2, tb); } } return res; diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/_posixsubprocess.c --- a/Modules/_posixsubprocess.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/_posixsubprocess.c Thu Jun 05 21:36:59 2014 +0200 @@ -44,6 +44,10 @@ #define POSIX_CALL(call) do { if ((call) == -1) goto error; } while (0) +/* Maximum file descriptor, initialized on module load. */ +static long max_fd; + + /* Given the gc module call gc.enable() and return 0 on success. */ static int _enable_gc(PyObject *gc_module) @@ -162,39 +166,14 @@ } -/* Get the maximum file descriptor that could be opened by this process. - * This function is async signal safe for use between fork() and exec(). - */ -static long -safe_get_max_fd(void) -{ - long local_max_fd; -#if defined(__NetBSD__) - local_max_fd = fcntl(0, F_MAXFD); - if (local_max_fd >= 0) - return local_max_fd; -#endif -#ifdef _SC_OPEN_MAX - local_max_fd = sysconf(_SC_OPEN_MAX); - if (local_max_fd == -1) -#endif - local_max_fd = 256; /* Matches legacy Lib/subprocess.py behavior. */ - return local_max_fd; -} - - -/* Close all file descriptors in the range from start_fd and higher - * except for those in py_fds_to_keep. If the range defined by - * [start_fd, safe_get_max_fd()) is large this will take a long - * time as it calls close() on EVERY possible fd. - * - * It isn't possible to know for sure what the max fd to go up to - * is for processes with the capability of raising their maximum. +/* Close all file descriptors in the range start_fd inclusive to + * end_fd exclusive except for those in py_fds_to_keep. If the + * range defined by [start_fd, end_fd) is large this will take a + * long time as it calls close() on EVERY possible fd. */ static void -_close_fds_by_brute_force(long start_fd, PyObject *py_fds_to_keep) +_close_fds_by_brute_force(int start_fd, int end_fd, PyObject *py_fds_to_keep) { - long end_fd = safe_get_max_fd(); Py_ssize_t num_fds_to_keep = PySequence_Length(py_fds_to_keep); Py_ssize_t keep_seq_idx; int fd_num; @@ -234,8 +213,8 @@ char d_name[256]; /* Filename (null-terminated) */ }; -/* Close all open file descriptors in the range from start_fd and higher - * Do not close any in the sorted py_fds_to_keep list. +/* Close all open file descriptors in the range start_fd inclusive to end_fd + * exclusive. Do not close any in the sorted py_fds_to_keep list. * * This version is async signal safe as it does not make any unsafe C library * calls, malloc calls or handle any locks. It is _unfortunate_ to be forced @@ -250,14 +229,16 @@ * it with some cpp #define magic to work on other OSes as well if you want. */ static void -_close_open_fds_safe(int start_fd, PyObject* py_fds_to_keep) +_close_open_fd_range_safe(int start_fd, int end_fd, PyObject* py_fds_to_keep) { int fd_dir_fd; + if (start_fd >= end_fd) + return; fd_dir_fd = _Py_open(FD_DIR, O_RDONLY); if (fd_dir_fd == -1) { /* No way to get a list of open fds. */ - _close_fds_by_brute_force(start_fd, py_fds_to_keep); + _close_fds_by_brute_force(start_fd, end_fd, py_fds_to_keep); return; } else { char buffer[sizeof(struct linux_dirent64)]; @@ -272,23 +253,23 @@ entry = (struct linux_dirent64 *)(buffer + offset); if ((fd = _pos_int_from_ascii(entry->d_name)) < 0) continue; /* Not a number. */ - if (fd != fd_dir_fd && fd >= start_fd && + if (fd != fd_dir_fd && fd >= start_fd && fd < end_fd && !_is_fd_in_sorted_fd_sequence(fd, py_fds_to_keep)) { while (close(fd) < 0 && errno == EINTR); } } } - while (close(fd_dir_fd) < 0 && errno == EINTR); + close(fd_dir_fd); } } -#define _close_open_fds _close_open_fds_safe +#define _close_open_fd_range _close_open_fd_range_safe #else /* NOT (defined(__linux__) && defined(HAVE_SYS_SYSCALL_H)) */ -/* Close all open file descriptors from start_fd and higher. - * Do not close any in the sorted py_fds_to_keep list. +/* Close all open file descriptors in the range start_fd inclusive to end_fd + * exclusive. Do not close any in the sorted py_fds_to_keep list. * * This function violates the strict use of async signal safe functions. :( * It calls opendir(), readdir() and closedir(). Of these, the one most @@ -301,13 +282,17 @@ * http://womble.decadent.org.uk/readdir_r-advisory.html */ static void -_close_open_fds_maybe_unsafe(long start_fd, PyObject* py_fds_to_keep) +_close_open_fd_range_maybe_unsafe(int start_fd, int end_fd, + PyObject* py_fds_to_keep) { DIR *proc_fd_dir; #ifndef HAVE_DIRFD - while (_is_fd_in_sorted_fd_sequence(start_fd, py_fds_to_keep)) { + while (_is_fd_in_sorted_fd_sequence(start_fd, py_fds_to_keep) && + (start_fd < end_fd)) { ++start_fd; } + if (start_fd >= end_fd) + return; /* Close our lowest fd before we call opendir so that it is likely to * reuse that fd otherwise we might close opendir's file descriptor in * our loop. This trick assumes that fd's are allocated on a lowest @@ -315,6 +300,8 @@ while (close(start_fd) < 0 && errno == EINTR); ++start_fd; #endif + if (start_fd >= end_fd) + return; #if defined(__FreeBSD__) if (!_is_fdescfs_mounted_on_dev_fd()) @@ -324,7 +311,7 @@ proc_fd_dir = opendir(FD_DIR); if (!proc_fd_dir) { /* No way to get a list of open fds. */ - _close_fds_by_brute_force(start_fd, py_fds_to_keep); + _close_fds_by_brute_force(start_fd, end_fd, py_fds_to_keep); } else { struct dirent *dir_entry; #ifdef HAVE_DIRFD @@ -337,7 +324,7 @@ int fd; if ((fd = _pos_int_from_ascii(dir_entry->d_name)) < 0) continue; /* Not a number. */ - if (fd != fd_used_by_opendir && fd >= start_fd && + if (fd != fd_used_by_opendir && fd >= start_fd && fd < end_fd && !_is_fd_in_sorted_fd_sequence(fd, py_fds_to_keep)) { while (close(fd) < 0 && errno == EINTR); } @@ -345,13 +332,13 @@ } if (errno) { /* readdir error, revert behavior. Highly Unlikely. */ - _close_fds_by_brute_force(start_fd, py_fds_to_keep); + _close_fds_by_brute_force(start_fd, end_fd, py_fds_to_keep); } closedir(proc_fd_dir); } } -#define _close_open_fds _close_open_fds_maybe_unsafe +#define _close_open_fd_range _close_open_fd_range_maybe_unsafe #endif /* else NOT (defined(__linux__) && defined(HAVE_SYS_SYSCALL_H)) */ @@ -470,8 +457,14 @@ /* close FDs after executing preexec_fn, which might open FDs */ if (close_fds) { + int local_max_fd = max_fd; +#if defined(__NetBSD__) + local_max_fd = fcntl(0, F_MAXFD); + if (local_max_fd < 0) + local_max_fd = max_fd; +#endif /* TODO HP-UX could use pstat_getproc() if anyone cares about it. */ - _close_open_fds(3, py_fds_to_keep); + _close_open_fd_range(3, local_max_fd, py_fds_to_keep); } /* This loop matches the Lib/os.py _execvpe()'s PATH search when */ @@ -766,5 +759,11 @@ PyMODINIT_FUNC PyInit__posixsubprocess(void) { +#ifdef _SC_OPEN_MAX + max_fd = sysconf(_SC_OPEN_MAX); + if (max_fd == -1) +#endif + max_fd = 256; /* Matches Lib/subprocess.py */ + return PyModule_Create(&_posixsubprocessmodule); } diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/_testcapimodule.c Thu Jun 05 21:36:59 2014 +0200 @@ -2756,7 +2756,7 @@ } typedef struct { - PyMemAllocatorEx alloc; + PyMemAllocator alloc; size_t malloc_size; size_t calloc_nelem; @@ -2802,7 +2802,7 @@ PyObject *res = NULL; const char *error_msg; alloc_hook_t hook; - PyMemAllocatorEx alloc; + PyMemAllocator alloc; size_t size, size2, nelem, elsize; void *ptr, *ptr2; @@ -3167,9 +3167,9 @@ {"pytime_object_to_timeval", test_pytime_object_to_timeval, METH_VARARGS}, {"pytime_object_to_timespec", test_pytime_object_to_timespec, METH_VARARGS}, {"with_tp_del", with_tp_del, METH_VARARGS}, + {"test_pymem", + (PyCFunction)test_pymem_alloc0, METH_NOARGS}, {"test_pymem_alloc0", - (PyCFunction)test_pymem_alloc0, METH_NOARGS}, - {"test_pymem_setrawallocators", (PyCFunction)test_pymem_setrawallocators, METH_NOARGS}, {"test_pymem_setallocators", (PyCFunction)test_pymem_setallocators, METH_NOARGS}, diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/_tracemalloc.c --- a/Modules/_tracemalloc.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/_tracemalloc.c Thu Jun 05 21:36:59 2014 +0200 @@ -18,9 +18,9 @@ /* Protected by the GIL */ static struct { - PyMemAllocatorEx mem; - PyMemAllocatorEx raw; - PyMemAllocatorEx obj; + PyMemAllocator mem; + PyMemAllocator raw; + PyMemAllocator obj; } allocators; static struct { @@ -475,10 +475,10 @@ static void* tracemalloc_alloc(int use_calloc, void *ctx, size_t nelem, size_t elsize) { - PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; + PyMemAllocator *alloc = (PyMemAllocator *)ctx; void *ptr; - assert(elsize == 0 || nelem <= PY_SIZE_MAX / elsize); + assert(nelem <= PY_SIZE_MAX / elsize); if (use_calloc) ptr = alloc->calloc(alloc->ctx, nelem, elsize); @@ -501,7 +501,7 @@ static void* tracemalloc_realloc(void *ctx, void *ptr, size_t new_size) { - PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; + PyMemAllocator *alloc = (PyMemAllocator *)ctx; void *ptr2; ptr2 = alloc->realloc(alloc->ctx, ptr, new_size); @@ -546,7 +546,7 @@ static void tracemalloc_free(void *ctx, void *ptr) { - PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; + PyMemAllocator *alloc = (PyMemAllocator *)ctx; if (ptr == NULL) return; @@ -567,7 +567,7 @@ void *ptr; if (get_reentrant()) { - PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; + PyMemAllocator *alloc = (PyMemAllocator *)ctx; if (use_calloc) return alloc->calloc(alloc->ctx, nelem, elsize); else @@ -607,7 +607,7 @@ Example: PyMem_RawRealloc() is called internally by pymalloc (_PyObject_Malloc() and _PyObject_Realloc()) to allocate a new arena (new_arena()). */ - PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; + PyMemAllocator *alloc = (PyMemAllocator *)ctx; ptr2 = alloc->realloc(alloc->ctx, ptr, new_size); if (ptr2 != NULL && ptr != NULL) { @@ -639,7 +639,7 @@ void *ptr; if (get_reentrant()) { - PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; + PyMemAllocator *alloc = (PyMemAllocator *)ctx; if (use_calloc) return alloc->calloc(alloc->ctx, nelem, elsize); else @@ -685,7 +685,7 @@ if (get_reentrant()) { /* Reentrant call to PyMem_RawRealloc(). */ - PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; + PyMemAllocator *alloc = (PyMemAllocator *)ctx; ptr2 = alloc->realloc(alloc->ctx, ptr, new_size); @@ -863,7 +863,7 @@ static int tracemalloc_start(int max_nframe) { - PyMemAllocatorEx alloc; + PyMemAllocator alloc; size_t size; if (tracemalloc_init() < 0) diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/getpath.c --- a/Modules/getpath.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/getpath.c Thu Jun 05 21:36:59 2014 +0200 @@ -734,11 +734,6 @@ bufsz += wcslen(zip_path) + 1; bufsz += wcslen(exec_prefix) + 1; - /* When running from the build directory, add room for the Modules - * subdirectory too. - */ - if (efound == -1) - bufsz += wcslen(argv0_path) + wcslen(L"Modules") + 2; buf = (wchar_t *)PyMem_Malloc(bufsz * sizeof(wchar_t)); if (buf == NULL) { @@ -786,15 +781,6 @@ /* Finally, on goes the directory for dynamic-load modules */ wcscat(buf, exec_prefix); - /* And, if we run from a build directory, the Modules directory (for - * modules built with Modules/Setup.) - */ - if (efound == -1) { - wcscat(buf, delimiter); - wcscat(buf, argv0_path); - wcscat(buf, separator); - wcscat(buf, L"Modules"); - } /* And publish the results */ module_search_path = buf; diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/makesetup --- a/Modules/makesetup Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/makesetup Thu Jun 05 21:36:59 2014 +0200 @@ -217,7 +217,7 @@ *) src='$(srcdir)/'"$srcdir/$src";; esac case $doconfig in - no) cc="$cc \$(CCSHARED) \$(PY_CFLAGS) \$(PY_CPPFLAGS)";; + no) cc="$cc \$(CCSHARED) \$(CFLAGS) \$(CPPFLAGS)";; *) cc="$cc \$(PY_CORE_CFLAGS)";; esac @@ -229,7 +229,11 @@ esac for mod in $mods do - file="$srcdir/$mod\$(EXT_SUFFIX)" + case $objs in + *$mod.o*) base=$mod;; + *) base=${mod}module;; + esac + file="$srcdir/$base\$(SO)" case $doconfig in no) SHAREDMODS="$SHAREDMODS $file";; esac diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Modules/socketmodule.c --- a/Modules/socketmodule.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Modules/socketmodule.c Thu Jun 05 21:36:59 2014 +0200 @@ -288,6 +288,10 @@ #include +#ifndef offsetof +# define offsetof(type, member) ((size_t)(&((type *)0)->member)) +#endif + #ifndef O_NONBLOCK # define O_NONBLOCK O_NDELAY #endif diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Objects/bytearrayobject.c Thu Jun 05 21:36:59 2014 +0200 @@ -813,9 +813,21 @@ } else { if (count > 0) { - if (PyByteArray_Resize((PyObject *)self, count)) + void *sval; + Py_ssize_t alloc; + + assert (Py_SIZE(self) == 0); + + alloc = count + 1; + sval = PyObject_Calloc(1, alloc); + if (sval == NULL) return -1; - memset(PyByteArray_AS_STRING(self), 0, count); + + PyObject_Free(self->ob_bytes); + + self->ob_bytes = self->ob_start = sval; + Py_SIZE(self) = count; + self->ob_alloc = alloc; } return 0; } diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Objects/object.c --- a/Objects/object.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Objects/object.c Thu Jun 05 21:36:59 2014 +0200 @@ -1543,9 +1543,6 @@ void _Py_ReadyTypes(void) { - if (PyType_Ready(&PyBaseObject_Type) < 0) - Py_FatalError("Can't initialize object type"); - if (PyType_Ready(&PyType_Type) < 0) Py_FatalError("Can't initialize type type"); @@ -1558,9 +1555,6 @@ if (PyType_Ready(&_PyWeakref_ProxyType) < 0) Py_FatalError("Can't initialize weakref proxy type"); - if (PyType_Ready(&PyLong_Type) < 0) - Py_FatalError("Can't initialize int type"); - if (PyType_Ready(&PyBool_Type) < 0) Py_FatalError("Can't initialize bool type"); @@ -1585,6 +1579,9 @@ if (PyType_Ready(&PySuper_Type) < 0) Py_FatalError("Can't initialize super type"); + if (PyType_Ready(&PyBaseObject_Type) < 0) + Py_FatalError("Can't initialize object type"); + if (PyType_Ready(&PyRange_Type) < 0) Py_FatalError("Can't initialize range type"); @@ -1609,6 +1606,9 @@ if (PyType_Ready(&PyFloat_Type) < 0) Py_FatalError("Can't initialize float type"); + if (PyType_Ready(&PyLong_Type) < 0) + Py_FatalError("Can't initialize int type"); + if (PyType_Ready(&PyFrozenSet_Type) < 0) Py_FatalError("Can't initialize frozenset type"); diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Objects/obmalloc.c --- a/Objects/obmalloc.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Objects/obmalloc.c Thu Jun 05 21:36:59 2014 +0200 @@ -151,7 +151,7 @@ typedef struct { /* We tag each block with an API ID in order to tag API violations */ char api_id; - PyMemAllocatorEx alloc; + PyMemAllocator alloc; } debug_alloc_api_t; static struct { debug_alloc_api_t raw; @@ -166,7 +166,7 @@ #define PYDBG_FUNCS _PyMem_DebugMalloc, _PyMem_DebugCalloc, _PyMem_DebugRealloc, _PyMem_DebugFree #endif -static PyMemAllocatorEx _PyMem_Raw = { +static PyMemAllocator _PyMem_Raw = { #ifdef PYMALLOC_DEBUG &_PyMem_Debug.raw, PYDBG_FUNCS #else @@ -174,7 +174,7 @@ #endif }; -static PyMemAllocatorEx _PyMem = { +static PyMemAllocator _PyMem = { #ifdef PYMALLOC_DEBUG &_PyMem_Debug.mem, PYDBG_FUNCS #else @@ -182,7 +182,7 @@ #endif }; -static PyMemAllocatorEx _PyObject = { +static PyMemAllocator _PyObject = { #ifdef PYMALLOC_DEBUG &_PyMem_Debug.obj, PYDBG_FUNCS #else @@ -209,7 +209,7 @@ PyMem_SetupDebugHooks(void) { #ifdef PYMALLOC_DEBUG - PyMemAllocatorEx alloc; + PyMemAllocator alloc; alloc.malloc = _PyMem_DebugMalloc; alloc.calloc = _PyMem_DebugCalloc; @@ -237,7 +237,7 @@ } void -PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator) +PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator) { switch(domain) { @@ -255,7 +255,7 @@ } void -PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator) +PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator) { switch(domain) { diff -r a6c846ec5fd3 -r 80ea1cdf2b23 PCbuild/pyproject.props --- a/PCbuild/pyproject.props Wed Jun 11 12:27:40 2014 -0400 +++ b/PCbuild/pyproject.props Thu Jun 05 21:36:59 2014 +0200 @@ -20,7 +20,7 @@ $(externalsDir)\sqlite-3.8.3.1 $(externalsDir)\bzip2-1.0.6 $(externalsDir)\xz-5.0.5 - $(externalsDir)\openssl-1.0.1h + $(externalsDir)\openssl-1.0.1g $(externalsDir)\tcl-8.6.1.0 $(externalsDir)\tk-8.6.1.0 $(externalsDir)\tix-8.4.3.4 diff -r a6c846ec5fd3 -r 80ea1cdf2b23 PCbuild/readme.txt --- a/PCbuild/readme.txt Wed Jun 11 12:27:40 2014 -0400 +++ b/PCbuild/readme.txt Thu Jun 05 21:36:59 2014 +0200 @@ -168,7 +168,7 @@ Homepage: http://tukaani.org/xz/ _ssl - Python wrapper for version 1.0.1h of the OpenSSL secure sockets + Python wrapper for version 1.0.1g of the OpenSSL secure sockets library, which is built by ssl.vcxproj Homepage: http://www.openssl.org/ diff -r a6c846ec5fd3 -r 80ea1cdf2b23 PCbuild/tk.vcxproj --- a/PCbuild/tk.vcxproj Wed Jun 11 12:27:40 2014 -0400 +++ b/PCbuild/tk.vcxproj Thu Jun 05 21:36:59 2014 +0200 @@ -153,9 +153,9 @@ IF "$(Platform)" EQU "x64" set TclMachine=AMD64 IF "$(Configuration)" EQU "Debug" ( - set TclOpts=symbols + set TclOpts=symbols,noxp ) ELSE ( - set TclOpts= + set TclOpts=noxp ) cd $(tkDir)\win diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Parser/tokenizer.c --- a/Parser/tokenizer.c Wed Jun 11 12:27:40 2014 -0400 +++ b/Parser/tokenizer.c Thu Jun 05 21:36:59 2014 +0200 @@ -1603,24 +1603,15 @@ } while (isdigit(c)); } if (c == 'e' || c == 'E') { - int e; - exponent: - e = c; + exponent: /* Exponent part */ c = tok_nextc(tok); - if (c == '+' || c == '-') { + if (c == '+' || c == '-') c = tok_nextc(tok); - if (!isdigit(c)) { - tok->done = E_TOKEN; - tok_backup(tok, c); - return ERRORTOKEN; - } - } else if (!isdigit(c)) { + if (!isdigit(c)) { + tok->done = E_TOKEN; tok_backup(tok, c); - tok_backup(tok, e); - *p_start = tok->start; - *p_end = tok->cur; - return NUMBER; + return ERRORTOKEN; } do { c = tok_nextc(tok); diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Tools/buildbot/external-common.bat --- a/Tools/buildbot/external-common.bat Wed Jun 11 12:27:40 2014 -0400 +++ b/Tools/buildbot/external-common.bat Thu Jun 05 21:36:59 2014 +0200 @@ -7,11 +7,14 @@ @rem if exist bzip2-1.0.6 rd /s/q bzip2-1.0.6 @rem if exist tcltk rd /s/q tcltk @rem if exist tcltk64 rd /s/q tcltk64 -@rem if exist tcl-8.6.1.0 rd /s/q tcl-8.6.1.0 -@rem if exist tk-8.6.1.0 rd /s/q tk-8.6.1.0 -@rem if exist tix-8.4.3.4 rd /s/q tix-8.4.3.4 +@rem if exist tcl8.4.12 rd /s/q tcl8.4.12 +@rem if exist tcl8.4.16 rd /s/q tcl8.4.16 +@rem if exist tcl-8.4.18.1 rd /s/q tcl-8.4.18.1 +@rem if exist tk8.4.12 rd /s/q tk8.4.12 +@rem if exist tk8.4.16 rd /s/q tk8.4.16 +@rem if exist tk-8.4.18.1 rd /s/q tk-8.4.18.1 @rem if exist db-4.4.20 rd /s/q db-4.4.20 -@rem if exist openssl-1.0.1h rd /s/q openssl-1.0.1h +@rem if exist openssl-1.0.1e rd /s/q openssl-1.0.1g @rem if exist sqlite-3.7.12 rd /s/q sqlite-3.7.12 @rem bzip @@ -21,9 +24,9 @@ ) @rem OpenSSL -if not exist openssl-1.0.1h ( - rd /s/q openssl-1.0.1g - svn export http://svn.python.org/projects/external/openssl-1.0.1h +if not exist openssl-1.0.1g ( + rd /s/q openssl-1.0.1e + svn export http://svn.python.org/projects/external/openssl-1.0.1g ) @rem tcl/tk/tix diff -r a6c846ec5fd3 -r 80ea1cdf2b23 Tools/clinic/clinic.py --- a/Tools/clinic/clinic.py Wed Jun 11 12:27:40 2014 -0400 +++ b/Tools/clinic/clinic.py Thu Jun 05 21:36:59 2014 +0200 @@ -2044,9 +2044,11 @@ # automatically add converter for default format unit # (but without stomping on the existing one if it's already # set, in case you subclass) - if ((cls.format_unit not in ('O&', '')) and + if ((cls.format_unit != 'O&') and (cls.format_unit not in legacy_converters)): legacy_converters[cls.format_unit] = cls + if cls.format_unit: + legacy_converters[cls.format_unit] = cls return cls def add_legacy_c_converter(format_unit, **kwargs):