Hash collision security issue

Doc/using/cmdline.rst

 .. highlightlang:: none
-
-.. ATTENTION: You probably should update Misc/python.man, too, if you modify
-.. this file.
 
 .. _using-on-general:
 
 Command line and environment
 ============================
 
 The CPython interpreter scans the command line and the environment for various
 settings.
 
 .. impl-detail::
 
    Other implementations' command line schemes may differ.  See
    :ref:`implementations` for further resources.
 
 
 .. _using-on-cmdline:
 
 Command line
 ------------
 
 When invoking Python, you may specify any of these options::
 
-    python [-BdEiOQsStuUvVWxX3?] [-c command | -m module-name | script | - ] [args]
+    python [-bBdEhiORsSuvVWx?] [-c command | -m module-name | script | - ] [args]
 
 The most common use case is, of course, a simple invocation of a script::
 
     python myscript.py
 
 
 .. _using-on-interface-options:
 
 Interface options
 ~~~~~~~~~~~~~~~~~
@@ skipping 65 matching lines @@
 
        python -mtimeit -s 'setup here' 'benchmarked code here'
        python -mtimeit -h # for details
 
    .. seealso::
       :func:`runpy.run_module`
          Equivalent functionality directly available to Python code
 
       :pep:`338` -- Executing modules as scripts
 
-   .. versionadded:: 2.4
-
-   .. versionchanged:: 2.5
-      The named module can now be located inside a package.
-
-   .. versionchanged:: 2.7
+
+   .. versionchanged:: 3.1
       Supply the package name to run a ``__main__`` submodule.
-      sys.argv[0] is now set to ``"-m"`` while searching for the module
-      (it was previously incorrectly set to ``"-c"``)
-
 
 .. describe:: -
 
    Read commands from standard input (:data:`sys.stdin`).  If standard input is
    a terminal, :option:`-i` is implied.
 
    If this option is given, the first element of :data:`sys.argv` will be
    ``"-"`` and the current directory will be added to the start of
    :data:`sys.path`.
 
 
 .. describe:: <script>
 
    Execute the Python code contained in *script*, which must be a filesystem
    path (absolute or relative) referring to either a Python file, a directory
    containing a ``__main__.py`` file, or a zipfile containing a
    ``__main__.py`` file.
 
    If this option is given, the first element of :data:`sys.argv` will be the
    script name as given on the command line.
 
    If the script name refers directly to a Python file, the directory
    containing that file is added to the start of :data:`sys.path`, and the
    file is executed as the :mod:`__main__` module.
 
    If the script name refers to a directory or zipfile, the script name is
    added to the start of :data:`sys.path` and the ``__main__.py`` file in
    that location is executed as the :mod:`__main__` module.
 
-   .. versionchanged:: 2.5
-      Directories and zipfiles containing a ``__main__.py`` file at the top
-      level are now considered valid Python scripts.
 
 If no interface option is given, :option:`-i` is implied, ``sys.argv[0]`` is
 an empty string (``""``) and the current directory will be added to the
 start of :data:`sys.path`.
 
 .. seealso::  :ref:`tut-invoking`
 
 
 Generic options
 ~~~~~~~~~~~~~~~
 
 .. cmdoption:: -?
                -h
                --help
 
    Print a short description of all command line options.
 
-   .. versionchanged:: 2.5
-      The ``--help`` variant.
-
 
 .. cmdoption:: -V
                --version
 
    Print the Python version number and exit.  Example output could be::
 
-       Python 2.5.1
-
-   .. versionchanged:: 2.5
-      The ``--version`` variant.
+       Python 3.0
 
 
 Miscellaneous options
 ~~~~~~~~~~~~~~~~~~~~~
 
+.. cmdoption:: -b
+
+   Issue a warning when comparing str and bytes. Issue an error when the
+   option is given twice (:option:`-bb`).
+
+
 .. cmdoption:: -B
 
    If given, Python won't try to write ``.pyc`` or ``.pyo`` files on the
    import of source modules.  See also :envvar:`PYTHONDONTWRITEBYTECODE`.
 
-   .. versionadded:: 2.6
-
 
 .. cmdoption:: -d
 
    Turn on parser debugging output (for wizards only, depending on compilation
    options).  See also :envvar:`PYTHONDEBUG`.
 
 
 .. cmdoption:: -E
 
    Ignore all :envvar:`PYTHON*` environment variables, e.g.
    :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`, that might be set.
-
-   .. versionadded:: 2.2
 
 
 .. cmdoption:: -i
 
    When a script is passed as first argument or the :option:`-c` option is used,
    enter interactive mode after executing the script or the command, even when
    :data:`sys.stdin` does not appear to be a terminal.  The
    :envvar:`PYTHONSTARTUP` file is not read.
 
    This can be useful to inspect global variables or a stack trace when a script
    raises an exception.  See also :envvar:`PYTHONINSPECT`.
 
 
 .. cmdoption:: -O
 
    Turn on basic optimizations.  This changes the filename extension for
    compiled (:term:`bytecode`) files from ``.pyc`` to ``.pyo``.  See also
    :envvar:`PYTHONOPTIMIZE`.
 
 
 .. cmdoption:: -OO
 
    Discard docstrings in addition to the :option:`-O` optimizations.
 
 
-.. cmdoption:: -Q <arg>
-
-   Division control. The argument must be one of the following:
-
-   ``old``
-     division of int/int and long/long return an int or long (*default*)
-   ``new``
-     new division semantics, i.e. division of int/int and long/long returns a
-     float
-   ``warn``
-     old division semantics with a warning for int/int and long/long
-   ``warnall``
-     old division semantics with a warning for all uses of the division operator
-
-   .. seealso::
-      :file:`Tools/scripts/fixdiv.py`
-         for a use of ``warnall``
-
-      :pep:`238` -- Changing the division operator
+.. cmdoption:: -R
+
+   Turn on hash randomization, so that the :meth:`__hash__` values of str, bytes
+   and datetime objects are "salted" with an unpredictable random value.
+   Although they remain constant within an individual Python process, they are
+   not predictable between repeated invocations of Python.
+
+   This is intended to provide protection against a denial-of-service caused by
+   carefully-chosen inputs that exploit the worst case performance of a dict
+   insertion, O(n^2) complexity.  See
+   http://www.ocert.org/advisories/ocert-2011-003.html for details.
+
+   Changing hash values affects the order in which keys are retrieved from a
+   dict.  Although Python has never made guarantees about this ordering (and it
+   typically varies between 32-bit and 64-bit builds), enough real-world code
+   implicitly relies on this non-guaranteed behavior that the randomization is
+   disabled by default.
+
+   See also :envvar:`PYTHONHASHSEED`.
+
+   .. versionadded:: 3.1.5
 
 
 .. cmdoption:: -s
 
-   Don't add the :data:`user site-packages directory <site.USER_SITE>` to
-   :data:`sys.path`.
-
-   .. versionadded:: 2.6
+   Don't add user site directory to sys.path
 
    .. seealso::
 
       :pep:`370` -- Per user site-packages directory
 
 
 .. cmdoption:: -S
 
    Disable the import of the module :mod:`site` and the site-dependent
    manipulations of :data:`sys.path` that it entails.
 
 
-.. cmdoption:: -t
-
-   Issue a warning when a source file mixes tabs and spaces for indentation in a
-   way that makes it depend on the worth of a tab expressed in spaces.  Issue an
-   error when the option is given twice (:option:`-tt`).
-
-
 .. cmdoption:: -u
 
-   Force stdin, stdout and stderr to be totally unbuffered.  On systems where it
-   matters, also put stdin, stdout and stderr in binary mode.
-
-   Note that there is internal buffering in :meth:`file.readlines` and
-   :ref:`bltin-file-objects` (``for line in sys.stdin``) which is not influenced
-   by this option.  To work around this, you will want to use
-   :meth:`file.readline` inside a ``while 1:`` loop.
+   Force the binary layer of the stdin, stdout and stderr streams (which is
+   available as their ``buffer`` attribute) to be unbuffered.  The text I/O
+   layer will still be line-buffered.
 
    See also :envvar:`PYTHONUNBUFFERED`.
 
 
 .. cmdoption:: -v
 
    Print a message each time a module is initialized, showing the place
    (filename or built-in module) from which it is loaded.  When given twice
    (:option:`-vv`), print a message for each file that is checked for when
    searching for a module.  Also provides information on module cleanup at exit.
    See also :envvar:`PYTHONVERBOSE`.
 
 
 .. cmdoption:: -W arg
 
    Warning control.  Python's warning machinery by default prints warning
    messages to :data:`sys.stderr`.  A typical warning message has the following
    form::
 
        file:line: category: message
 
    By default, each warning is printed once for each source line where it
    occurs.  This option controls how often warnings are printed.
 
    Multiple :option:`-W` options may be given; when a warning matches more than
    one option, the action for the last matching option is performed.  Invalid
    :option:`-W` options are ignored (though, a warning message is printed about
    invalid options when the first warning is issued).
 
-   Starting from Python 2.7, :exc:`DeprecationWarning` and its descendants
-   are ignored by default.  The :option:`-Wd` option can be used to re-enable
-   them.
-
    Warnings can also be controlled from within a Python program using the
    :mod:`warnings` module.
 
    The simplest form of argument is one of the following action strings (or a
-   unique abbreviation) by themselves:
+   unique abbreviation):
 
    ``ignore``
       Ignore all warnings.
    ``default``
       Explicitly request the default behavior (printing each warning once per
       source line).
    ``all``
       Print a warning each time it occurs (this may generate many messages if a
       warning is triggered repeatedly for the same source line, such as inside a
       loop).
    ``module``
       Print each warning only the first time it occurs in each module.
    ``once``
       Print each warning only the first time it occurs in the program.
    ``error``
       Raise an exception instead of printing a warning message.
 
    The full form of argument is::
 
        action:message:category:module:line
 
    Here, *action* is as explained above but only applies to messages that match
    the remaining fields.  Empty fields match all values; trailing empty fields
    may be omitted.  The *message* field matches the start of the warning message
    printed; this match is case-insensitive.  The *category* field matches the
-   warning category.  This must be a class name; the match tests whether the
+   warning category.  This must be a class name; the match test whether the
    actual warning category of the message is a subclass of the specified warning
    category.  The full class name must be given.  The *module* field matches the
    (fully-qualified) module name; this match is case-sensitive.  The *line*
    field matches the line number, where zero matches all line numbers and is
    thus equivalent to an omitted line number.
 
    .. seealso::
       :mod:`warnings` -- the warnings module
 
       :pep:`230` -- Warning framework
 
-      :envvar:`PYTHONWARNINGS`
-
 
 .. cmdoption:: -x
 
    Skip the first line of the source, allowing use of non-Unix forms of
    ``#!cmd``.  This is intended for a DOS specific hack only.
 
    .. note:: The line numbers in error messages will be off by one.
 
-.. cmdoption:: -3
-
-   Warn about Python 3.x incompatibilities which cannot be fixed trivially by
-   :ref:`2to3 <2to3-reference>`. Among these are:
-
-   * :meth:`dict.has_key`
-   * :func:`apply`
-   * :func:`callable`
-   * :func:`coerce`
-   * :func:`execfile`
-   * :func:`reduce`
-   * :func:`reload`
-
-   Using these will emit a :exc:`DeprecationWarning`.
-
-   .. versionadded:: 2.6
 
 Options you shouldn't use
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. cmdoption:: -J
 
    Reserved for use by Jython_.
 
 .. _Jython: http://jython.org
 
-.. cmdoption:: -U
-
-   Turns all string literals into unicodes globally.  Do not be tempted to use
-   this option as it will probably break your world.  It also produces
-   ``.pyc`` files with a different magic number than normal.  Instead, you can
-   enable unicode literals on a per-module basis by using::
-
-        from __future__ import unicode_literals
-
-   at the top of the file.  See :mod:`__future__` for details.
-
 .. cmdoption:: -X
 
     Reserved for alternative implementations of Python to use for their own
     purposes.
+
 
 .. _using-on-envvars:
 
 Environment variables
 ---------------------
 
 These environment variables influence Python's behavior.
 
 .. envvar:: PYTHONHOME
 
@@ skipping 87 matching lines @@
 
    If this is set, Python ignores case in :keyword:`import` statements.  This
    only works on Windows.
 
 
 .. envvar:: PYTHONDONTWRITEBYTECODE
 
    If this is set, Python won't try to write ``.pyc`` or ``.pyo`` files on the
    import of source modules.
 
-   .. versionadded:: 2.6
-
-.. envvar:: PYTHONHASHRANDOMIZATION
-
-   If this is set to a non-empty string, the hash() values of str, unicode
-   and buffer objects are randomized.  Although they remain constant within
-   an individual Python process, they are not predictable between repeated
-   invocations of Python.
-
-   This is intended to provide protection against a denial-of-service
-   caused by carefully-chosen inputs that exploit the worst case performance
-   of a dict lookup, O(n^2) complexity.  See:
-   
-       http://www.ocert.org/advisories/ocert-2011-003.html
-
-   for details.
-
-   Changing hash values affects the order in which keys are retrieved from
-   a dict.  Although Python has never made guarantees about this ordering
-   (and it typically varies between 32-bit and 64-bit builds), enough
-   real-world code implicitly relies on this non-guaranteed behavior that
-   the randomization is disabled by default.
 
 .. envvar:: PYTHONHASHSEED
 
-   If this is set, it is used as a fixed seed for generating the hash() of
-   the types covererd by PYTHONHASHRANDOMIZATION.  It should be a number in
-   the range [0; 4294967295].  The value 0 overrides the other variable and
-   disables the hash randomization.
+   If this variable is set to ``random``, the effect is the same as specifying
+   the :option:`-R` option: a random value is used to seed the hashes of str,
+   bytes and datetime objects.
+
+   If :envvar:`PYTHONHASHSEED` is set to an integer value, it is used as a fixed
+   seed for generating the hash() of the types covered by the hash
+   randomization.
+
+   Its purpose is to allow repeatable hashing, such as for selftests for the
+   interpreter itself, or to allow a cluster of python processes to share hash
+   values.
+
+   The integer must be a decimal number in the range [0,4294967295].  Specifying
+   the value 0 will lead to the same hash values as when hash randomization is
+   disabled.
+
+   .. versionadded:: 3.1.5
+
 
 .. envvar:: PYTHONIOENCODING
 
    Overrides the encoding used for stdin/stdout/stderr, in the syntax
    ``encodingname:errorhandler``.  The ``:errorhandler`` part is optional and
    has the same meaning as in :func:`str.encode`.
 
-   .. versionadded:: 2.6
+   For stderr, the ``:errorhandler`` part is ignored; the handler will always be
+   ``'backslashreplace'``.
 
 
 .. envvar:: PYTHONNOUSERSITE
 
-   If this is set, Python won't add the :data:`user site-packages directory
-   <site.USER_SITE>` to :data:`sys.path`.
-
-   .. versionadded:: 2.6
+   If this is set, Python won't add the user site directory to sys.path
 
    .. seealso::
 
       :pep:`370` -- Per user site-packages directory
 
 
 .. envvar:: PYTHONUSERBASE
 
-   Defines the :data:`user base directory <site.USER_BASE>`, which is used to
-   compute the path of the :data:`user site-packages directory <site.USER_SITE>`
-   and :ref:`Distutils installation paths <inst-alt-install-user>` for ``python
-   setup.py install --user``.
-
-   .. versionadded:: 2.6
+   Sets the base directory for the user site directory
 
    .. seealso::
 
       :pep:`370` -- Per user site-packages directory
 
 
 .. envvar:: PYTHONEXECUTABLE
 
    If this environment variable is set, ``sys.argv[0]`` will be set to its
    value instead of the value got through the C runtime.  Only works on
    Mac OS X.
 
-.. envvar:: PYTHONWARNINGS
-
-   This is equivalent to the :option:`-W` option. If set to a comma
-   separated string, it is equivalent to specifying :option:`-W` multiple
-   times.
-
 
 Debug-mode variables
 ~~~~~~~~~~~~~~~~~~~~
 
 Setting these variables only has an effect in a debug build of Python, that is,
-if Python was configured with the ``--with-pydebug`` build option.
+if Python was configured with the :option:`--with-pydebug` build option.
 
 .. envvar:: PYTHONTHREADDEBUG
 
    If set, Python will print threading debug info.
 
-   .. versionchanged:: 2.6
-      Previously, this variable was called ``THREADDEBUG``.
 
 .. envvar:: PYTHONDUMPREFS
 
    If set, Python will dump objects and reference counts still alive after
    shutting down the interpreter.
 
 
 .. envvar:: PYTHONMALLOCSTATS
 
    If set, Python will print memory allocation statistics every time a new
    object arena is created, and on shutdown.