Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(14538)

Side by Side Diff: Doc/library/os.rst

Issue 25994: File descriptor leaks in os.scandir()
Patch Set: Created 3 years, 7 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | Doc/whatsnew/3.5.rst » ('j') | Doc/whatsnew/3.5.rst » ('J')
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 :mod:`os` --- Miscellaneous operating system interfaces 1 :mod:`os` --- Miscellaneous operating system interfaces
2 ======================================================= 2 =======================================================
3 3
4 .. module:: os 4 .. module:: os
5 :synopsis: Miscellaneous operating system interfaces. 5 :synopsis: Miscellaneous operating system interfaces.
6 6
7 7
8 This module provides a portable way of using operating system dependent 8 This module provides a portable way of using operating system dependent
9 functionality. If you just want to read or write a file see :func:`open`, if 9 functionality. If you just want to read or write a file see :func:`open`, if
10 you want to manipulate paths, see the :mod:`os.path` module, and if you want to 10 you want to manipulate paths, see the :mod:`os.path` module, and if you want to
(...skipping 1564 matching lines...) Expand 10 before | Expand all | Expand 10 after
1575 in all other circumstances, they will be of type ``str``. 1575 in all other circumstances, they will be of type ``str``.
1576 1576
1577 This function can also support :ref:`specifying a file descriptor 1577 This function can also support :ref:`specifying a file descriptor
1578 <path_fd>`; the file descriptor must refer to a directory. 1578 <path_fd>`; the file descriptor must refer to a directory.
1579 1579
1580 .. note:: 1580 .. note::
1581 To encode ``str`` filenames to ``bytes``, use :func:`~os.fsencode`. 1581 To encode ``str`` filenames to ``bytes``, use :func:`~os.fsencode`.
1582 1582
1583 .. seealso:: 1583 .. seealso::
1584 1584
1585 The :func:`scandir` function returns directory entries along with 1585 The :class:`scandir` function returns directory entries along with
Martin Panter 2016/02/09 02:45:01 Maybe change function → class. But on further tho
storchaka 2016/02/09 16:11:45 Reverted :class: to :func:. Actually os.scandir is
1586 file attribute information, giving better performance for many 1586 file attribute information, giving better performance for many
1587 common use cases. 1587 common use cases.
1588 1588
1589 .. versionchanged:: 3.2 1589 .. versionchanged:: 3.2
1590 The *path* parameter became optional. 1590 The *path* parameter became optional.
1591 1591
1592 .. versionadded:: 3.3 1592 .. versionadded:: 3.3
1593 Added support for specifying an open file descriptor for *path*. 1593 Added support for specifying an open file descriptor for *path*.
1594 1594
1595 1595
(...skipping 265 matching lines...) Expand 10 before | Expand all | Expand 10 after
1861 empty, otherwise, :exc:`OSError` is raised. In order to remove whole 1861 empty, otherwise, :exc:`OSError` is raised. In order to remove whole
1862 directory trees, :func:`shutil.rmtree` can be used. 1862 directory trees, :func:`shutil.rmtree` can be used.
1863 1863
1864 This function can support :ref:`paths relative to directory descriptors 1864 This function can support :ref:`paths relative to directory descriptors
1865 <dir_fd>`. 1865 <dir_fd>`.
1866 1866
1867 .. versionadded:: 3.3 1867 .. versionadded:: 3.3
1868 The *dir_fd* parameter. 1868 The *dir_fd* parameter.
1869 1869
1870 1870
1871 .. function:: scandir(path='.') 1871 .. class:: scandir(path='.')
1872 1872
1873 Return an iterator of :class:`DirEntry` objects corresponding to the entries 1873 Return an iterator of :class:`DirEntry` objects corresponding to the entries
1874 in the directory given by *path*. The entries are yielded in arbitrary 1874 in the directory given by *path*. The entries are yielded in arbitrary
1875 order, and the special entries ``'.'`` and ``'..'`` are not included. 1875 order, and the special entries ``'.'`` and ``'..'`` are not included.
1876 1876
1877 Using :func:`scandir` instead of :func:`listdir` can significantly 1877 Using :class:`scandir` instead of :func:`listdir` can significantly
1878 increase the performance of code that also needs file type or file 1878 increase the performance of code that also needs file type or file
1879 attribute information, because :class:`DirEntry` objects expose this 1879 attribute information, because :class:`DirEntry` objects expose this
1880 information if the operating system provides it when scanning a directory. 1880 information if the operating system provides it when scanning a directory.
1881 All :class:`DirEntry` methods may perform a system call, but 1881 All :class:`DirEntry` methods may perform a system call, but
1882 :func:`~DirEntry.is_dir` and :func:`~DirEntry.is_file` usually only 1882 :func:`~DirEntry.is_dir` and :func:`~DirEntry.is_file` usually only
1883 require a system call for symbolic links; :func:`DirEntry.stat` 1883 require a system call for symbolic links; :func:`DirEntry.stat`
1884 always requires a system call on Unix but only requires one for 1884 always requires a system call on Unix but only requires one for
1885 symbolic links on Windows. 1885 symbolic links on Windows.
1886 1886
1887 On Unix, *path* can be of type :class:`str` or :class:`bytes` (use 1887 On Unix, *path* can be of type :class:`str` or :class:`bytes` (use
1888 :func:`~os.fsencode` and :func:`~os.fsdecode` to encode and decode 1888 :func:`~os.fsencode` and :func:`~os.fsdecode` to encode and decode
1889 :class:`bytes` paths). On Windows, *path* must be of type :class:`str`. 1889 :class:`bytes` paths). On Windows, *path* must be of type :class:`str`.
1890 On both sytems, the type of the :attr:`~DirEntry.name` and 1890 On both sytems, the type of the :attr:`~DirEntry.name` and
1891 :attr:`~DirEntry.path` attributes of each :class:`DirEntry` will be of 1891 :attr:`~DirEntry.path` attributes of each :class:`DirEntry` will be of
1892 the same type as *path*. 1892 the same type as *path*.
1893 1893
1894 The following example shows a simple use of :func:`scandir` to display all 1894 :class:`scandir` supports the :term:`context manager` protocol and has
1895 the following method:
1896
1897 .. method:: close()
1898
1899 Close the iterator and free acquired resources.
1900
1901 This is called automatically when the iterator is exhausted or garbage
1902 collected, or when an error happened during iterating. However it
1903 is advisable to call it explicitly or use the :keyword:`with`
1904 statement.
1905
1906 .. versionadded:: 3.6
1907
1908 The following example shows a simple use of :class:`scandir` to display all
1895 the files (excluding directories) in the given *path* that don't start with 1909 the files (excluding directories) in the given *path* that don't start with
1896 ``'.'``. The ``entry.is_file()`` call will generally not make an additional 1910 ``'.'``. The ``entry.is_file()`` call will generally not make an additional
1897 system call:: 1911 system call::
1898 1912
1899 for entry in os.scandir(path): 1913 with os.scandir(path) as it:
1900 if not entry.name.startswith('.') and entry.is_file(): 1914 for entry in it:
1901 print(entry.name) 1915 if not entry.name.startswith('.') and entry.is_file():
1916 print(entry.name)
1902 1917
1903 .. note:: 1918 .. note::
1904 1919
1905 On Unix-based systems, :func:`scandir` uses the system's 1920 On Unix-based systems, :class:`scandir` uses the system's
1906 `opendir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/opend ir.html>`_ 1921 `opendir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/opend ir.html>`_
1907 and 1922 and
1908 `readdir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/readd ir_r.html>`_ 1923 `readdir() <http://pubs.opengroup.org/onlinepubs/009695399/functions/readd ir_r.html>`_
1909 functions. On Windows, it uses the Win32 1924 functions. On Windows, it uses the Win32
1910 `FindFirstFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/a a364418(v=vs.85).aspx>`_ 1925 `FindFirstFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/a a364418(v=vs.85).aspx>`_
1911 and 1926 and
1912 `FindNextFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/aa 364428(v=vs.85).aspx>`_ 1927 `FindNextFileW <http://msdn.microsoft.com/en-us/library/windows/desktop/aa 364428(v=vs.85).aspx>`_
1913 functions. 1928 functions.
1914 1929
1915 .. versionadded:: 3.5 1930 .. versionadded:: 3.5
1916 1931
1917 1932
1918 .. class:: DirEntry 1933 .. class:: DirEntry
1919 1934
1920 Object yielded by :func:`scandir` to expose the file path and other file 1935 Object yielded by :class:`scandir` to expose the file path and other file
1921 attributes of a directory entry. 1936 attributes of a directory entry.
1922 1937
1923 :func:`scandir` will provide as much of this information as possible without 1938 :class:`scandir` will provide as much of this information as possible without
1924 making additional system calls. When a ``stat()`` or ``lstat()`` system call 1939 making additional system calls. When a ``stat()`` or ``lstat()`` system call
1925 is made, the ``DirEntry`` object will cache the result. 1940 is made, the ``DirEntry`` object will cache the result.
1926 1941
1927 ``DirEntry`` instances are not intended to be stored in long-lived data 1942 ``DirEntry`` instances are not intended to be stored in long-lived data
1928 structures; if you know the file metadata has changed or if a long time has 1943 structures; if you know the file metadata has changed or if a long time has
1929 elapsed since calling :func:`scandir`, call ``os.stat(entry.path)`` to fetch 1944 elapsed since calling :class:`scandir`, call ``os.stat(entry.path)`` to fetch
1930 up-to-date information. 1945 up-to-date information.
1931 1946
1932 Because the ``DirEntry`` methods can make operating system calls, they may 1947 Because the ``DirEntry`` methods can make operating system calls, they may
1933 also raise :exc:`OSError`. If you need very fine-grained 1948 also raise :exc:`OSError`. If you need very fine-grained
1934 control over errors, you can catch :exc:`OSError` when calling one of the 1949 control over errors, you can catch :exc:`OSError` when calling one of the
1935 ``DirEntry`` methods and handle as appropriate. 1950 ``DirEntry`` methods and handle as appropriate.
1936 1951
1937 Attributes and methods on a ``DirEntry`` instance are as follows: 1952 Attributes and methods on a ``DirEntry`` instance are as follows:
1938 1953
1939 .. attribute:: name 1954 .. attribute:: name
1940 1955
1941 The entry's base filename, relative to the :func:`scandir` *path* 1956 The entry's base filename, relative to the :class:`scandir` *path*
1942 argument. 1957 argument.
1943 1958
1944 The :attr:`name` attribute will be of the same type (``str`` or 1959 The :attr:`name` attribute will be of the same type (``str`` or
1945 ``bytes``) as the :func:`scandir` *path* argument. Use 1960 ``bytes``) as the :class:`scandir` *path* argument. Use
1946 :func:`~os.fsdecode` to decode byte filenames. 1961 :func:`~os.fsdecode` to decode byte filenames.
1947 1962
1948 .. attribute:: path 1963 .. attribute:: path
1949 1964
1950 The entry's full path name: equivalent to ``os.path.join(scandir_path, 1965 The entry's full path name: equivalent to ``os.path.join(scandir_path,
1951 entry.name)`` where *scandir_path* is the :func:`scandir` *path* 1966 entry.name)`` where *scandir_path* is the :class:`scandir` *path*
1952 argument. The path is only absolute if the :func:`scandir` *path* 1967 argument. The path is only absolute if the :class:`scandir` *path*
1953 argument was absolute. 1968 argument was absolute.
1954 1969
1955 The :attr:`path` attribute will be of the same type (``str`` or 1970 The :attr:`path` attribute will be of the same type (``str`` or
1956 ``bytes``) as the :func:`scandir` *path* argument. Use 1971 ``bytes``) as the :class:`scandir` *path* argument. Use
1957 :func:`~os.fsdecode` to decode byte filenames. 1972 :func:`~os.fsdecode` to decode byte filenames.
1958 1973
1959 .. method:: inode() 1974 .. method:: inode()
1960 1975
1961 Return the inode number of the entry. 1976 Return the inode number of the entry.
1962 1977
1963 The result is cached on the ``DirEntry`` object. Use ``os.stat(entry.path, 1978 The result is cached on the ``DirEntry`` object. Use ``os.stat(entry.path,
1964 follow_symlinks=False).st_ino`` to fetch up-to-date information. 1979 follow_symlinks=False).st_ino`` to fetch up-to-date information.
1965 1980
1966 On the first, uncached call, a system call is required on Windows but 1981 On the first, uncached call, a system call is required on Windows but
(...skipping 1777 matching lines...) Expand 10 before | Expand all | Expand 10 after
3744 :exc:`NotImplementedError` will be raised. 3759 :exc:`NotImplementedError` will be raised.
3745 3760
3746 For an easy-to-use interface to the random number generator 3761 For an easy-to-use interface to the random number generator
3747 provided by your platform, please see :class:`random.SystemRandom`. 3762 provided by your platform, please see :class:`random.SystemRandom`.
3748 3763
3749 .. versionchanged:: 3.5 3764 .. versionchanged:: 3.5
3750 On Linux 3.17 and newer, the ``getrandom()`` syscall is now used 3765 On Linux 3.17 and newer, the ``getrandom()`` syscall is now used
3751 when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` 3766 when available. On OpenBSD 5.6 and newer, the C ``getentropy()``
3752 function is now used. These functions avoid the usage of an internal file 3767 function is now used. These functions avoid the usage of an internal file
3753 descriptor. 3768 descriptor.
OLDNEW
« no previous file with comments | « no previous file | Doc/whatsnew/3.5.rst » ('j') | Doc/whatsnew/3.5.rst » ('J')

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+