Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(67386)

Delta Between Two Patch Sets: Doc/library/os.rst

Issue 25994: File descriptor leaks in os.scandir()
Left Patch Set: Created 3 years, 8 months ago
Right Patch Set: Created 3 years, 8 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
Left: Side by side diff | Download
Right: Side by side diff | Download
« no previous file with change/comment | « no previous file | Doc/whatsnew/3.6.rst » ('j') | Doc/whatsnew/3.6.rst » ('J')
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
LEFTRIGHT
1 :mod:`os` --- Miscellaneous operating system interfaces 1 :mod:`os` --- Miscellaneous operating system interfaces
2 ======================================================= 2 =======================================================
3 3
4 .. module:: os 4 .. module:: os
5 :synopsis: Miscellaneous operating system interfaces. 5 :synopsis: Miscellaneous operating system interfaces.
6 6
7 7
8 This module provides a portable way of using operating system dependent 8 This module provides a portable way of using operating system dependent
9 functionality. If you just want to read or write a file see :func:`open`, if 9 functionality. If you just want to read or write a file see :func:`open`, if
10 you want to manipulate paths, see the :mod:`os.path` module, and if you want to 10 you want to manipulate paths, see the :mod:`os.path` module, and if you want to
(...skipping 1564 matching lines...) Expand 10 before | Expand all | Expand 10 after
1575 in all other circumstances, they will be of type ``str``. 1575 in all other circumstances, they will be of type ``str``.
1576 1576
1577 This function can also support :ref:`specifying a file descriptor 1577 This function can also support :ref:`specifying a file descriptor
1578 <path_fd>`; the file descriptor must refer to a directory. 1578 <path_fd>`; the file descriptor must refer to a directory.
1579 1579
1580 .. note:: 1580 .. note::
1581 To encode ``str`` filenames to ``bytes``, use :func:`~os.fsencode`. 1581 To encode ``str`` filenames to ``bytes``, use :func:`~os.fsencode`.
1582 1582
1583 .. seealso:: 1583 .. seealso::
1584 1584
1585 The :class:`scandir` function returns directory entries along with 1585 The :func:`scandir` function returns directory entries along with
Martin Panter 2016/02/09 02:45:01 Maybe change function → class. But on further tho
storchaka 2016/02/09 16:11:45 Reverted :class: to :func:. Actually os.scandir is
1586 file attribute information, giving better performance for many 1586 file attribute information, giving better performance for many
1587 common use cases. 1587 common use cases.
1588 1588
1589 .. versionchanged:: 3.2 1589 .. versionchanged:: 3.2
1590 The *path* parameter became optional. 1590 The *path* parameter became optional.
1591 1591
1592 .. versionadded:: 3.3 1592 .. versionadded:: 3.3
1593 Added support for specifying an open file descriptor for *path*. 1593 Added support for specifying an open file descriptor for *path*.
1594 1594
1595 1595
(...skipping 265 matching lines...) Expand 10 before | Expand all | Expand 10 after
1861 empty, otherwise, :exc:`OSError` is raised. In order to remove whole 1861 empty, otherwise, :exc:`OSError` is raised. In order to remove whole
1862 directory trees, :func:`shutil.rmtree` can be used. 1862 directory trees, :func:`shutil.rmtree` can be used.
1863 1863
1864 This function can support :ref:`paths relative to directory descriptors 1864 This function can support :ref:`paths relative to directory descriptors
1865 <dir_fd>`. 1865 <dir_fd>`.
1866 1866
1867 .. versionadded:: 3.3 1867 .. versionadded:: 3.3
1868 The *dir_fd* parameter. 1868 The *dir_fd* parameter.
1869 1869
1870 1870
1871 .. class:: scandir(path='.') 1871 .. function:: scandir(path='.')
1872 1872
1873 Return an iterator of :class:`DirEntry` objects corresponding to the entries 1873 Return an iterator of :class:`DirEntry` objects corresponding to the entries
1874 in the directory given by *path*. The entries are yielded in arbitrary 1874 in the directory given by *path*. The entries are yielded in arbitrary
1875 order, and the special entries ``'.'`` and ``'..'`` are not included. 1875 order, and the special entries ``'.'`` and ``'..'`` are not included.
1876 1876
1877 Using :class:`scandir` instead of :func:`listdir` can significantly 1877 Using :func:`scandir` instead of :func:`listdir` can significantly
1878 increase the performance of code that also needs file type or file 1878 increase the performance of code that also needs file type or file
1879 attribute information, because :class:`DirEntry` objects expose this 1879 attribute information, because :class:`DirEntry` objects expose this
1880 information if the operating system provides it when scanning a directory. 1880 information if the operating system provides it when scanning a directory.
1881 All :class:`DirEntry` methods may perform a system call, but 1881 All :class:`DirEntry` methods may perform a system call, but
1882 :func:`~DirEntry.is_dir` and :func:`~DirEntry.is_file` usually only 1882 :func:`~DirEntry.is_dir` and :func:`~DirEntry.is_file` usually only
1883 require a system call for symbolic links; :func:`DirEntry.stat` 1883 require a system call for symbolic links; :func:`DirEntry.stat`
1884 always requires a system call on Unix but only requires one for 1884 always requires a system call on Unix but only requires one for
1885 symbolic links on Windows. 1885 symbolic links on Windows.
1886 1886
1887 On Unix, *path* can be of type :class:`str` or :class:`bytes` (use 1887 On Unix, *path* can be of type :class:`str` or :class:`bytes` (use
1888 :func:`~os.fsencode` and :func:`~os.fsdecode` to encode and decode 1888 :func:`~os.fsencode` and :func:`~os.fsdecode` to encode and decode
1889 :class:`bytes` paths). On Windows, *path* must be of type :class:`str`. 1889 :class:`bytes` paths). On Windows, *path* must be of type :class:`str`.
1890 On both sytems, the type of the :attr:`~DirEntry.name` and 1890 On both sytems, the type of the :attr:`~DirEntry.name` and
1891 :attr:`~DirEntry.path` attributes of each :class:`DirEntry` will be of 1891 :attr:`~DirEntry.path` attributes of each :class:`DirEntry` will be of
1892 the same type as *path*. 1892 the same type as *path*.
1893 1893
1894 :class:`scandir` supports the :term:`context manager` protocol and has 1894 The :func:`scandir` iterator supports the :term:`context manager` protocol
1895 the following method: 1895 and has the following method:
1896 1896
1897 .. method:: close() 1897 .. method:: scandir.close()
1898 1898
1899 Close the iterator and free acquired resources. 1899 Close the iterator and free acquired resources.
1900 1900
1901 This is called automatically when the iterator is exhausted or garbage 1901 This is called automatically when the iterator is exhausted or garbage
1902 collected, or when an error happened during iterating. However it 1902 collected, or when an error happened during iterating. However it
Jim.J.Jewett 2016/02/10 23:20:41 happened -> happens
storchaka 2016/02/11 12:20:36 Done.
1903 is advisable to call it explicitly or use the :keyword:`with` 1903 is advisable to call it explicitly or use the :keyword:`with`
1904 statement. 1904 statement.
1905 1905
1906 .. versionadded:: 3.6 1906 .. versionadded:: 3.6
1907 1907
1908 The following example shows a simple use of :class:`scandir` to display all 1908 The following example shows a simple use of :func:`scandir` to display all
1909 the files (excluding directories) in the given *path* that don't start with 1909 the files (excluding directories) in the given *path* that don't start with
1910 ``'.'``. The ``entry.is_file()`` call will generally not make an additional 1910 ``'.'``. The ``entry.is_file()`` call will generally not make an additional
1911 system call:: 1911 system call::
1912 1912
1913 with os.scandir(path) as it: 1913 with os.scandir(path) as it:
1914 for entry in it: 1914 for entry in it:
1915 if not entry.name.startswith('.') and entry.is_file(): 1915 if not entry.name.startswith('.') and entry.is_file():
1916 print(entry.name) 1916 print(entry.name)
1917 1917
1918 .. note:: 1918 .. note::
1919 1919
1920 On Unix-based systems, :class:`scandir` uses the system's 1920 On Unix-based systems, :func:`scandir` uses the system's
1921 `opendir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/opend ir.html>`_ 1921 `opendir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/opend ir.html>`_
1922 and 1922 and
1923 `readdir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/readd ir_r.html>`_ 1923 `readdir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/readd ir_r.html>`_
1924 functions. On Windows, it uses the Win32 1924 functions. On Windows, it uses the Win32
1925 `FindFirstFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/a a364418(v=vs.85).aspx>`_ 1925 `FindFirstFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/a a364418(v=vs.85).aspx>`_
1926 and 1926 and
1927 `FindNextFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/aa 364428(v=vs.85).aspx>`_ 1927 `FindNextFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/aa 364428(v=vs.85).aspx>`_
1928 functions. 1928 functions.
1929 1929
1930 .. versionadded:: 3.5 1930 .. versionadded:: 3.5
1931 1931
1932 .. versionadded:: 3.6
1933 Added support for the :term:`context manager` protocol and the
1934 :func:`~scandir.close()` method. If a :func:`scandir` iterator is not
Jim.J.Jewett 2016/02/10 23:20:41 "is not ... nor" -> "is neither ... nor"
Martin Panter 2016/02/11 02:22:20 Yes, even better. (This started off as “is not . .
storchaka 2016/02/11 12:20:36 Done.
1935 exhausted nor explicitly closed a :exc:`ResourceWarning` will be emitted
1936 in its destructor.
1937
1932 1938
1933 .. class:: DirEntry 1939 .. class:: DirEntry
1934 1940
1935 Object yielded by :class:`scandir` to expose the file path and other file 1941 Object yielded by :func:`scandir` to expose the file path and other file
1936 attributes of a directory entry. 1942 attributes of a directory entry.
1937 1943
1938 :class:`scandir` will provide as much of this information as possible without 1944 :func:`scandir` will provide as much of this information as possible without
1939 making additional system calls. When a ``stat()`` or ``lstat()`` system call 1945 making additional system calls. When a ``stat()`` or ``lstat()`` system call
1940 is made, the ``DirEntry`` object will cache the result. 1946 is made, the ``DirEntry`` object will cache the result.
1941 1947
1942 ``DirEntry`` instances are not intended to be stored in long-lived data 1948 ``DirEntry`` instances are not intended to be stored in long-lived data
1943 structures; if you know the file metadata has changed or if a long time has 1949 structures; if you know the file metadata has changed or if a long time has
1944 elapsed since calling :class:`scandir`, call ``os.stat(entry.path)`` to fetch 1950 elapsed since calling :func:`scandir`, call ``os.stat(entry.path)`` to fetch
1945 up-to-date information. 1951 up-to-date information.
1946 1952
1947 Because the ``DirEntry`` methods can make operating system calls, they may 1953 Because the ``DirEntry`` methods can make operating system calls, they may
1948 also raise :exc:`OSError`. If you need very fine-grained 1954 also raise :exc:`OSError`. If you need very fine-grained
1949 control over errors, you can catch :exc:`OSError` when calling one of the 1955 control over errors, you can catch :exc:`OSError` when calling one of the
1950 ``DirEntry`` methods and handle as appropriate. 1956 ``DirEntry`` methods and handle as appropriate.
1951 1957
1952 Attributes and methods on a ``DirEntry`` instance are as follows: 1958 Attributes and methods on a ``DirEntry`` instance are as follows:
1953 1959
1954 .. attribute:: name 1960 .. attribute:: name
1955 1961
1956 The entry's base filename, relative to the :class:`scandir` *path* 1962 The entry's base filename, relative to the :func:`scandir` *path*
1957 argument. 1963 argument.
1958 1964
1959 The :attr:`name` attribute will be of the same type (``str`` or 1965 The :attr:`name` attribute will be of the same type (``str`` or
1960 ``bytes``) as the :class:`scandir` *path* argument. Use 1966 ``bytes``) as the :func:`scandir` *path* argument. Use
1961 :func:`~os.fsdecode` to decode byte filenames. 1967 :func:`~os.fsdecode` to decode byte filenames.
1962 1968
1963 .. attribute:: path 1969 .. attribute:: path
1964 1970
1965 The entry's full path name: equivalent to ``os.path.join(scandir_path, 1971 The entry's full path name: equivalent to ``os.path.join(scandir_path,
1966 entry.name)`` where *scandir_path* is the :class:`scandir` *path* 1972 entry.name)`` where *scandir_path* is the :func:`scandir` *path*
1967 argument. The path is only absolute if the :class:`scandir` *path* 1973 argument. The path is only absolute if the :func:`scandir` *path*
1968 argument was absolute. 1974 argument was absolute.
1969 1975
1970 The :attr:`path` attribute will be of the same type (``str`` or 1976 The :attr:`path` attribute will be of the same type (``str`` or
1971 ``bytes``) as the :class:`scandir` *path* argument. Use 1977 ``bytes``) as the :func:`scandir` *path* argument. Use
1972 :func:`~os.fsdecode` to decode byte filenames. 1978 :func:`~os.fsdecode` to decode byte filenames.
1973 1979
1974 .. method:: inode() 1980 .. method:: inode()
1975 1981
1976 Return the inode number of the entry. 1982 Return the inode number of the entry.
1977 1983
1978 The result is cached on the ``DirEntry`` object. Use ``os.stat(entry.path, 1984 The result is cached on the ``DirEntry`` object. Use ``os.stat(entry.path,
1979 follow_symlinks=False).st_ino`` to fetch up-to-date information. 1985 follow_symlinks=False).st_ino`` to fetch up-to-date information.
1980 1986
1981 On the first, uncached call, a system call is required on Windows but 1987 On the first, uncached call, a system call is required on Windows but
(...skipping 1777 matching lines...) Expand 10 before | Expand all | Expand 10 after
3759 :exc:`NotImplementedError` will be raised. 3765 :exc:`NotImplementedError` will be raised.
3760 3766
3761 For an easy-to-use interface to the random number generator 3767 For an easy-to-use interface to the random number generator
3762 provided by your platform, please see :class:`random.SystemRandom`. 3768 provided by your platform, please see :class:`random.SystemRandom`.
3763 3769
3764 .. versionchanged:: 3.5 3770 .. versionchanged:: 3.5
3765 On Linux 3.17 and newer, the ``getrandom()`` syscall is now used 3771 On Linux 3.17 and newer, the ``getrandom()`` syscall is now used
3766 when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` 3772 when available. On OpenBSD 5.6 and newer, the C ``getentropy()``
3767 function is now used. These functions avoid the usage of an internal file 3773 function is now used. These functions avoid the usage of an internal file
3768 descriptor. 3774 descriptor.
LEFTRIGHT

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+