Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(6607)

Delta Between Two Patch Sets: Doc/library/importlib.rst

Issue 18864: Implementation for PEP 451 (importlib.machinery.ModuleSpec)
Left Patch Set: Created 5 years, 6 months ago
Right Patch Set: Created 5 years, 5 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
Left: Side by side diff | Download
Right: Side by side diff | Download
« no previous file with change/comment | « no previous file | Lib/importlib/_bootstrap.py » ('j') | no next file with change/comment »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
LEFTRIGHT
1 :mod:`importlib` -- An implementation of :keyword:`import` 1 :mod:`importlib` -- An implementation of :keyword:`import`
2 ========================================================== 2 ==========================================================
3 3
4 .. module:: importlib 4 .. module:: importlib
5 :synopsis: An implementation of the import machinery. 5 :synopsis: An implementation of the import machinery.
6 6
7 .. moduleauthor:: Brett Cannon <brett@python.org> 7 .. moduleauthor:: Brett Cannon <brett@python.org>
8 .. sectionauthor:: Brett Cannon <brett@python.org> 8 .. sectionauthor:: Brett Cannon <brett@python.org>
9 9
10 .. versionadded:: 3.1 10 .. versionadded:: 3.1
(...skipping 292 matching lines...) Expand 10 before | Expand all | Expand 10 after
303 loaded, :exc:`ImportError` is raised, otherwise the loaded module is 303 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
304 returned. 304 returned.
305 305
306 If the requested module already exists in :data:`sys.modules`, that 306 If the requested module already exists in :data:`sys.modules`, that
307 module should be used and reloaded. 307 module should be used and reloaded.
308 Otherwise the loader should create a new module and insert it into 308 Otherwise the loader should create a new module and insert it into
309 :data:`sys.modules` before any loading begins, to prevent recursion 309 :data:`sys.modules` before any loading begins, to prevent recursion
310 from the import. If the loader inserted a module and the load fails, it 310 from the import. If the loader inserted a module and the load fails, it
311 must be removed by the loader from :data:`sys.modules`; modules already 311 must be removed by the loader from :data:`sys.modules`; modules already
312 in :data:`sys.modules` before the loader began execution should be left 312 in :data:`sys.modules` before the loader began execution should be left
313 alone (see :func:`importlib.util.module_to_load`). 313 alone (see :func:`importlib.util.module_for_loader`).
314 314
315 The loader should set several attributes on the module. 315 The loader should set several attributes on the module.
316 (Note that some of these attributes can change when a module is 316 (Note that some of these attributes can change when a module is
317 reloaded; see :meth:`init_module_attrs`): 317 reloaded):
318 318
319 - :attr:`__name__` 319 - :attr:`__name__`
320 The name of the module. 320 The name of the module.
321 321
322 - :attr:`__file__` 322 - :attr:`__file__`
323 The path to where the module data is stored (not set for built-in 323 The path to where the module data is stored (not set for built-in
324 modules). 324 modules).
325 325
326 - :attr:`__cached__` 326 - :attr:`__cached__`
327 The path to where a compiled version of the module is/should be 327 The path to where a compiled version of the module is/should be
(...skipping 21 matching lines...) Expand all
349 .. method:: module_repr(module) 349 .. method:: module_repr(module)
350 350
351 An optional method which when implemented calculates and returns the 351 An optional method which when implemented calculates and returns the
352 given module's repr, as a string. The module type's default repr() will 352 given module's repr, as a string. The module type's default repr() will
353 use the result of this method as appropriate. 353 use the result of this method as appropriate.
354 354
355 .. versionadded:: 3.3 355 .. versionadded:: 3.3
356 356
357 .. versionchanged:: 3.4 357 .. versionchanged:: 3.4
358 Made optional instead of an abstractmethod. 358 Made optional instead of an abstractmethod.
359
360 .. method:: init_module_attrs(module)
361
362 Set the :attr:`__loader__` attribute on the module.
363
364 Subclasses overriding this method should set whatever appropriate
365 attributes it can, getting the module's name from :attr:`__name__` when
366 needed. All values should also be overridden so that reloading works as
367 expected.
368
369 .. versionadded:: 3.4
370 359
371 360
372 .. class:: ResourceLoader 361 .. class:: ResourceLoader
373 362
374 An abstract base class for a :term:`loader` which implements the optional 363 An abstract base class for a :term:`loader` which implements the optional
375 :pep:`302` protocol for loading arbitrary resources from the storage 364 :pep:`302` protocol for loading arbitrary resources from the storage
376 back-end. 365 back-end.
377 366
378 .. method:: get_data(path) 367 .. method:: get_data(path)
379 368
(...skipping 55 matching lines...) Expand 10 before | Expand all | Expand 10 after
435 424
436 Create a code object from Python source. 425 Create a code object from Python source.
437 426
438 The *data* argument can be whatever the :func:`compile` function 427 The *data* argument can be whatever the :func:`compile` function
439 supports (i.e. string or bytes). The *path* argument should be 428 supports (i.e. string or bytes). The *path* argument should be
440 the "path" to where the source code originated from, which can be an 429 the "path" to where the source code originated from, which can be an
441 abstract concept (e.g. location in a zip file). 430 abstract concept (e.g. location in a zip file).
442 431
443 .. versionadded:: 3.4 432 .. versionadded:: 3.4
444 433
445 .. method:: init_module_attrs(module)
446
447 Set the :attr:`__package__` attribute and :attr:`__path__` attribute to
448 the empty list if appropriate along with what
449 :meth:`importlib.abc.Loader.init_module_attrs` sets.
450
451 .. versionadded:: 3.4
452
453 .. method:: load_module(fullname) 434 .. method:: load_module(fullname)
454 435
455 Implementation of :meth:`Loader.load_module`. 436 Implementation of :meth:`Loader.load_module`.
456 437
457 438
458 .. class:: ExecutionLoader 439 .. class:: ExecutionLoader
459 440
460 An abstract base class which inherits from :class:`InspectLoader` that, 441 An abstract base class which inherits from :class:`InspectLoader` that,
461 when implemented, helps a module to be executed as a script. The ABC 442 when implemented, helps a module to be executed as a script. The ABC
462 represents an optional :pep:`302` protocol. 443 represents an optional :pep:`302` protocol.
463 444
464 .. method:: get_filename(fullname) 445 .. method:: get_filename(fullname)
465 446
466 An abstract method that is to return the value of :attr:`__file__` for 447 An abstract method that is to return the value of :attr:`__file__` for
467 the specified module. If no path is available, :exc:`ImportError` is 448 the specified module. If no path is available, :exc:`ImportError` is
468 raised. 449 raised.
469 450
470 If source code is available, then the method should return the path to 451 If source code is available, then the method should return the path to
471 the source file, regardless of whether a bytecode was used to load the 452 the source file, regardless of whether a bytecode was used to load the
472 module. 453 module.
473 454
474 .. versionchanged:: 3.4 455 .. versionchanged:: 3.4
475 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`. 456 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
476
477 .. method:: init_module_attrs(module)
478
479 Set :attr:`__file__` and if initializing a package then set
480 :attr:`__path__` to ``[os.path.dirname(__file__)]`` along with
481 all attributes set by
482 :meth:`importlib.abc.InspectLoader.init_module_attrs`.
483
484 .. versionadded:: 3.4
485 457
486 458
487 .. class:: FileLoader(fullname, path) 459 .. class:: FileLoader(fullname, path)
488 460
489 An abstract base class which inherits from :class:`ResourceLoader` and 461 An abstract base class which inherits from :class:`ResourceLoader` and
490 :class:`ExecutionLoader`, providing concrete implementations of 462 :class:`ExecutionLoader`, providing concrete implementations of
491 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`. 463 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
492 464
493 The *fullname* argument is a fully resolved name of the module the loader is 465 The *fullname* argument is a fully resolved name of the module the loader is
494 to handle. The *path* argument is the path to the file for the module. 466 to handle. The *path* argument is the path to the file for the module.
(...skipping 96 matching lines...) Expand 10 before | Expand all | Expand 10 after
591 563
592 Concrete implementation of :meth:`InspectLoader.get_source`. 564 Concrete implementation of :meth:`InspectLoader.get_source`.
593 565
594 .. method:: is_package(fullname) 566 .. method:: is_package(fullname)
595 567
596 Concrete implementation of :meth:`InspectLoader.is_package`. A module 568 Concrete implementation of :meth:`InspectLoader.is_package`. A module
597 is determined to be a package if its file path (as provided by 569 is determined to be a package if its file path (as provided by
598 :meth:`ExecutionLoader.get_filename`) is a file named 570 :meth:`ExecutionLoader.get_filename`) is a file named
599 ``__init__`` when the file extension is removed **and** the module name 571 ``__init__`` when the file extension is removed **and** the module name
600 itself does not end in ``__init__``. 572 itself does not end in ``__init__``.
601
602 .. method:: init_module_attr(module)
603
604 Set :attr:`__cached__` using :func:`imp.cache_from_source`. Other
605 attributes set by
606 :meth:`importlib.abc.ExecutionLoader.init_module_attrs`.
607
608 .. versionadded:: 3.4
609 573
610 574
611 :mod:`importlib.machinery` -- Importers and path hooks 575 :mod:`importlib.machinery` -- Importers and path hooks
612 ------------------------------------------------------ 576 ------------------------------------------------------
613 577
614 .. module:: importlib.machinery 578 .. module:: importlib.machinery
615 :synopsis: Importers and path hooks 579 :synopsis: Importers and path hooks
616 580
617 This module contains the various objects that help :keyword:`import` 581 This module contains the various objects that help :keyword:`import`
618 find and load modules. 582 find and load modules.
(...skipping 127 matching lines...) Expand 10 before | Expand all | Expand 10 after
746 staleness relies upon the granularity of the operating system's state 710 staleness relies upon the granularity of the operating system's state
747 information of the file system, there is a potential race condition of 711 information of the file system, there is a potential race condition of
748 searching for a module, creating a new file, and then searching for the 712 searching for a module, creating a new file, and then searching for the
749 module the new file represents. If the operations happen fast enough to fit 713 module the new file represents. If the operations happen fast enough to fit
750 within the granularity of stat calls, then the module search will fail. To 714 within the granularity of stat calls, then the module search will fail. To
751 prevent this from happening, when you create a module dynamically, make sure 715 prevent this from happening, when you create a module dynamically, make sure
752 to call :func:`importlib.invalidate_caches`. 716 to call :func:`importlib.invalidate_caches`.
753 717
754 .. versionadded:: 3.3 718 .. versionadded:: 3.3
755 719
756 .. versionchanged:: 3.4
757 The empty string is no longer special-cased to be changed into ``'.'``.
758
759 .. attribute:: path 720 .. attribute:: path
760 721
761 The path the finder will search in. 722 The path the finder will search in.
762 723
763 .. method:: find_loader(fullname) 724 .. method:: find_loader(fullname)
764 725
765 Attempt to find the loader to handle *fullname* within :attr:`path`. 726 Attempt to find the loader to handle *fullname* within :attr:`path`.
766 727
767 .. method:: invalidate_caches() 728 .. method:: invalidate_caches()
768 729
(...skipping 123 matching lines...) Expand 10 before | Expand all | Expand 10 after
892 .. versionadded:: 3.4 853 .. versionadded:: 3.4
893 854
894 .. attribute:: name 855 .. attribute:: name
895 856
896 (``__name__``) 857 (``__name__``)
897 858
898 A string for the fully-qualified name of the module. 859 A string for the fully-qualified name of the module.
899 860
900 .. attribute:: loader 861 .. attribute:: loader
901 862
902 (``__name__``) 863 (``__loader__``)
903 864
904 The loader to use for loading. For namespace packages this will be 865 The loader to use for loading. For namespace packages this should be
905 set to None. 866 set to None.
906 867
907 .. attribute:: origin 868 .. attribute:: origin
908 869
909 (``__name__``) 870 (``__file__``)
910 871
911 Name of the place from which the module is loaded, e.g. "builtin" for 872 Name of the place from which the module is loaded, e.g. "builtin" for
912 built-in modules and the filename for modules loaded from source. 873 built-in modules and the filename for modules loaded from source.
913 "origin" may be None, but normally will be set. 874 Normally "origin" should be set, but it may be None (the default)
875 which indicates it is unspecified.
914 876
915 .. attribute:: submodule_search_locations 877 .. attribute:: submodule_search_locations
916 878
917 (``__name__``) 879 (``__path__``)
918 880
919 List of strings for where to find submodules, if a package (None 881 List of strings for where to find submodules, if a package (None
920 otherwise). 882 otherwise).
921 883
922 .. attribute:: loader_state 884 .. attribute:: loader_state
923 885
924 (``__name__``)
925
926 Container of extra module-specific data for use during loading (or 886 Container of extra module-specific data for use during loading (or
927 None). 887 None).
928 888
929 .. attribute:: cached 889 .. attribute:: cached
930 890
931 (``__name__``) 891 (``__cached__``)
932 892
933 String for where the compiled module should be stored (or None). 893 String for where the compiled module should be stored (or None).
934 894
935 .. attribute:: parent 895 .. attribute:: parent
936 896
937 (``__name__``) 897 (``__package__``)
938 898
939 (Read-only) Fully-qualified name of the package to which the module 899 (Read-only) Fully-qualified name of the package to which the module
940 belongs as a submodule (or None). 900 belongs as a submodule (or None).
941 901
942 .. attribute:: has_location 902 .. attribute:: has_location
943 903
944 (``__name__``) 904 Boolean indicating whether or not the module's "origin"
945
946 (Read-only) Boolean indicating whether or not the module's "origin"
947 attribute refers to a loadable location. 905 attribute refers to a loadable location.
948 906
949 :mod:`importlib.util` -- Utility code for importers 907 :mod:`importlib.util` -- Utility code for importers
950 --------------------------------------------------- 908 ---------------------------------------------------
951 909
952 .. module:: importlib.util 910 .. module:: importlib.util
953 :synopsis: Utility code for importers 911 :synopsis: Utility code for importers
954 912
955 This module contains the various objects that help in the construction of 913 This module contains the various objects that help in the construction of
956 an :term:`importer`. 914 an :term:`importer`.
957 915
958 .. attribute:: MAGIC_NUMBER 916 .. attribute:: MAGIC_NUMBER
959 917
960 The bytes which represent the bytecode version number. If you need help with 918 The bytes which represent the bytecode version number. If you need help with
961 loading/writing bytecode then consider :class:`importlib.abc.SourceLoader`. 919 loading/writing bytecode then consider :class:`importlib.abc.SourceLoader`.
962 920
963 .. versionadded:: 3.4 921 .. versionadded:: 3.4
964 922
965 .. function:: cache_from_source(path, debug_override=None) 923 .. function:: cache_from_source(path, debug_override=None)
966 924
967 Return the :pep:`3147` path to the byte-compiled file associated with the 925 Return the :pep:`3147` path to the byte-compiled file associated with the
968 source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return 926 source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return
969 value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2. 927 value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
970 The ``cpython-32`` string comes from the current magic tag (see 928 The ``cpython-32`` string comes from the current magic tag (see
971 :func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then 929 :func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then
972 :exc:`NotImplementedError` will be raised). The returned path will end in 930 :exc:`NotImplementedError` will be raised). The returned path will end in
973 ``.pyc`` when ``__debug__`` is True or ``.pyo`` for an optimized Python 931 ``.pyc`` when ``__debug__`` is ``True`` or ``.pyo`` for an optimized Python
974 (i.e. ``__debug__`` is False). By passing in True or False for 932 (i.e. ``__debug__`` is ``False``). By passing in ``True`` or ``False`` for
975 *debug_override* you can override the system's value for ``__debug__`` for 933 *debug_override* you can override the system's value for ``__debug__`` for
976 extension selection. 934 extension selection.
977 935
978 *path* need not exist. 936 *path* need not exist.
979 937
980 .. versionadded:: 3.4 938 .. versionadded:: 3.4
981 939
982 940
983 .. function:: source_from_cache(path) 941 .. function:: source_from_cache(path)
984 942
(...skipping 23 matching lines...) Expand all
1008 allows for usage such as 966 allows for usage such as
1009 ``importlib.util.resolve_name('sys', __package__)`` without doing a 967 ``importlib.util.resolve_name('sys', __package__)`` without doing a
1010 check to see if the **package** argument is needed. 968 check to see if the **package** argument is needed.
1011 969
1012 :exc:`ValueError` is raised if **name** is a relative module name but 970 :exc:`ValueError` is raised if **name** is a relative module name but
1013 package is a false value (e.g. ``None`` or the empty string). 971 package is a false value (e.g. ``None`` or the empty string).
1014 :exc:`ValueError` is also raised a relative name would escape its containing 972 :exc:`ValueError` is also raised a relative name would escape its containing
1015 package (e.g. requesting ``..bacon`` from within the ``spam`` package). 973 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
1016 974
1017 .. versionadded:: 3.3 975 .. versionadded:: 3.3
1018
1019 .. function:: module_to_load(name, *, reset_name=True)
1020
1021 Returns a :term:`context manager` which provides the module to load. The
1022 module will either come from :attr:`sys.modules` in the case of reloading or
1023 a fresh module if loading a new module. Proper cleanup of
1024 :attr:`sys.modules` occurs if the module was new and an exception was
1025 raised.
1026
1027 If **reset_name** is true and the module requested is being reloaded then
1028 the module's :attr:`__name__` attribute will
1029 be reset to **name**, else it will be left untouched.
1030
1031 .. versionadded:: 3.4
1032 976
1033 .. decorator:: module_for_loader 977 .. decorator:: module_for_loader
1034 978
1035 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` 979 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
1036 to handle selecting the proper 980 to handle selecting the proper
1037 module object to load with. The decorated method is expected to have a call 981 module object to load with. The decorated method is expected to have a call
1038 signature taking two positional arguments 982 signature taking two positional arguments
1039 (e.g. ``load_module(self, module)``) for which the second argument 983 (e.g. ``load_module(self, module)``) for which the second argument
1040 will be the module **object** to be used by the loader. 984 will be the module **object** to be used by the loader.
1041 Note that the decorator will not work on static methods because of the 985 Note that the decorator will not work on static methods because of the
(...skipping 14 matching lines...) Expand all
1056 1000
1057 .. versionchanged:: 3.3 1001 .. versionchanged:: 3.3
1058 :attr:`__loader__` and :attr:`__package__` are automatically set 1002 :attr:`__loader__` and :attr:`__package__` are automatically set
1059 (when possible). 1003 (when possible).
1060 1004
1061 .. versionchanged:: 3.4 1005 .. versionchanged:: 3.4
1062 Set :attr:`__name__`, :attr:`__loader__` :attr:`__package__` 1006 Set :attr:`__name__`, :attr:`__loader__` :attr:`__package__`
1063 unconditionally to support reloading. 1007 unconditionally to support reloading.
1064 1008
1065 .. deprecated:: 3.4 1009 .. deprecated:: 3.4
1066 For the benefit of :term:`loader` subclasses, please use 1010 The import machinery now directly performs all the functionality
1067 :func:`module_to_load` and 1011 provided by this function.
1068 :meth:`importlib.abc.Loader.init_module_attrs` instead.
1069 1012
1070 .. decorator:: set_loader 1013 .. decorator:: set_loader
1071 1014
1072 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` 1015 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
1073 to set the :attr:`__loader__` 1016 to set the :attr:`__loader__`
1074 attribute on the returned module. If the attribute is already set the 1017 attribute on the returned module. If the attribute is already set the
1075 decorator does nothing. It is assumed that the first positional argument to 1018 decorator does nothing. It is assumed that the first positional argument to
1076 the wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set 1019 the wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set
1077 to. 1020 to.
1078 1021
1079 .. note::
1080 As this decorator sets :attr:`__loader__` after loading the module, it is
1081 recommended to use :meth:`importlib.abc.Loader.init_module_attrs` instead
1082 when appropriate.
1083
1084 .. versionchanged:: 3.4 1022 .. versionchanged:: 3.4
1085 Set ``__loader__`` if set to ``None``, as if the attribute does not 1023 Set ``__loader__`` if set to ``None``, as if the attribute does not
1086 exist. 1024 exist.
1087 1025
1088 .. decorator:: set_package 1026 .. decorator:: set_package
1089 1027
1090 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the : attr:`__package__` attribute on the returned module. If :attr:`__package__` 1028 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the : attr:`__package__` attribute on the returned module. If :attr:`__package__`
1091 is set and has a value other than ``None`` it will not be changed. 1029 is set and has a value other than ``None`` it will not be changed.
1092 1030
1093 .. note::
1094 As this decorator sets :attr:`__package__` after loading the module, it is
1095 recommended to use :meth:`importlib.abc.Loader.init_module_attrs` instead
1096 when appropriate.
1097
1098 .. function:: spec_from_loader(name, loader, *, origin=None, is_package=None) 1031 .. function:: spec_from_loader(name, loader, *, origin=None, is_package=None)
1099 1032
1100 A factory function for creating a :class:`ModuleSpec` instance based 1033 A factory function for creating a :class:`ModuleSpec` instance based
1101 on a loader. The parameters have the same meaning as they do for 1034 on a loader. The parameters have the same meaning as they do for
1102 ModuleSpec. The function uses available :term:`loader` APIS, such as 1035 ModuleSpec. The function uses available :term:`loader` APIs, such as
1103 :meth:`InspectLoader.is_package`, to fill in any missing 1036 :meth:`InspectLoader.is_package`, to fill in any missing
1104 information on the spec. 1037 information on the spec.
1105 1038
1106 .. versionadded:: 3.4 1039 .. versionadded:: 3.4
1107 1040
1108 .. function:: spec_from_file_location(name, location, *, loader=None, submodule_ search_locations=None) 1041 .. function:: spec_from_file_location(name, location, *, loader=None, submodule_ search_locations=None)
1109 1042
1110 A factory function for creating a :class:`ModuleSpec` instance based 1043 A factory function for creating a :class:`ModuleSpec` instance based
1111 on the path to a file. Missing information will be filled in on the 1044 on the path to a file. Missing information will be filled in on the
1112 spec by making use of loader APIs and by the implication that the 1045 spec by making use of loader APIs and by the implication that the
1113 module will be file-based. 1046 module will be file-based.
1114 1047
1115 .. versionadded:: 3.4 1048 .. versionadded:: 3.4
LEFTRIGHT

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+