This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Deprecation warning in zoneinfo module
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: p-ganssle Nosy List: jaraco, miss-islington, p-ganssle, xtreak
Priority: normal Keywords: patch

Created on 2021-12-18 14:04 by xtreak, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 30190 merged jaraco, 2021-12-18 23:41
Messages (10)
msg408851 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2021-12-18 14:04
zoneinfo module currently emits deprecation warnings due to API changes in importlib.resources

./python -Wonce -m test test_zoneinfo
0:00:00 load avg: 0.73 Run tests sequentially
0:00:00 load avg: 0.73 [1/1] test_zoneinfo
/home/karthikeyan/stuff/python/cpython/Lib/zoneinfo/_tzpath.py:121: DeprecationWarning: open_text is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  with resources.open_text("tzdata", "zones") as f:
/home/karthikeyan/stuff/python/cpython/Lib/zoneinfo/_common.py:12: DeprecationWarning: open_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  return importlib.resources.open_binary(package_name, resource_name)
/home/karthikeyan/stuff/python/cpython/Lib/zoneinfo/_tzpath.py:121: DeprecationWarning: open_text is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  with resources.open_text("tzdata", "zones") as f:
/home/karthikeyan/stuff/python/cpython/Lib/zoneinfo/_common.py:12: DeprecationWarning: open_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  return importlib.resources.open_binary(package_name, resource_name)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 376 ms
Tests result: SUCCESS

A fix would be to adapt the _legacy module changes inline like below patch though normalize_path is not public API and need to be inlined too.

diff --git a/Lib/zoneinfo/_common.py b/Lib/zoneinfo/_common.py
index 4c24f01bd7..bfe3fe4c3c 100644
--- a/Lib/zoneinfo/_common.py
+++ b/Lib/zoneinfo/_common.py
@@ -2,14 +2,15 @@
 
 
 def load_tzdata(key):
-    import importlib.resources
+    from importlib.resources import files
+    from importlib._common import normalize_path
 
     components = key.split("/")
     package_name = ".".join(["tzdata.zoneinfo"] + components[:-1])
     resource_name = components[-1]
 
     try:
-        return importlib.resources.open_binary(package_name, resource_name)
+        return (files(package_name) / normalize_path(resource_name)).open('rb')
     except (ImportError, FileNotFoundError, UnicodeEncodeError):
         # There are three types of exception that can be raised that all amount
         # to "we cannot find this key":
diff --git a/Lib/zoneinfo/_tzpath.py b/Lib/zoneinfo/_tzpath.py
index 672560b951..b1efe5d99e 100644
--- a/Lib/zoneinfo/_tzpath.py
+++ b/Lib/zoneinfo/_tzpath.py
@@ -111,14 +111,15 @@ def available_timezones():
         determine if a given file on the time zone search path is to open it
         and check for the "magic string" at the beginning.
     """
-    from importlib import resources
+    from importlib.resources import files
+    from importlib._common import normalize_path
 
     valid_zones = set()
 
     # Start with loading from the tzdata package if it exists: this has a
     # pre-assembled list of zones that only requires opening one file.
     try:
-        with resources.open_text("tzdata", "zones") as f:
+        with (files("tzdata") / normalize_path("zones")).open('r') as f:
             for zone in f:
                 zone = zone.strip()
                 if zone:
msg408870 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2021-12-18 22:22
I would hope that normalize_path would not be needed. It might be for drop-in compatibility. If it's needed for zoneinfo, I'd like to consider why and what implications that has for the deprecation of the legacy API.
msg408871 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2021-12-18 22:25
I just confirmed, and `normalize_path` has been moved to the _legacy module for importlib_resources, so I'd expect that change to land in CPython soon too.
msg408873 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2021-12-18 22:31
I filed issue46125 to track that issue separately.
msg409259 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2021-12-28 15:09
Jason's patch looks good to me, but I don't understand why Karthikeyan originally suggested using `normalize_path`. Trying to dig through exactly how `files().joinpath().open` is implemented has so many layers of indirection and abstract classes that I can't quite figure out if the two things are equivalent or not. Seems like `joinpath()` is the right thing to do, but does it have less validation than the `normalize_path` method?
msg409260 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2021-12-28 16:33
I just copied the implementation and normalize_path function was part of it. Looking into the implementation of normalize_path it validates the given argument to be a filename without any separator. I will leave it to Jason for a better understanding of the API and compatibility here.
msg409272 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2021-12-28 21:15
> Does `joinpath` have less validation?

Yes. Previously, resources.* would perform some validation on the path to ensure that it didn't contain path separators (to avoid users attempting to get resources in subdirectories or perhaps manipulating the path in other ways).

So no, they're not equivalent. If `resource_name` or "zones" ever contained path separators, the former implementation would raise an error whereas this implementation would attempt to join those characters to the path. Since "zones" is a static string, it's clearly not affected. And `resource_name` can't have posixpath.sep. If `key` had an ntpath.sep, that might behave differently, but that seemed like an edge case not worth exploring.

If it is worth exploring, I would recommend not to use normalize_path, but instead to implement the validation in zoneinfo._common. That is, wrap key in `_validate_key()` that protects against invalid paths. But in practice, it's better to do that closer to where the unsanitized data would be encountered (if at all).
msg409273 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2021-12-28 21:16
Normalize_path from legacy implementation: https://github.com/python/importlib_resources/blob/3beb2fd5831e65f7b45033e1ec276c4a6b4ca973/importlib_resources/_legacy.py#L30-L40
msg411185 - (view) Author: miss-islington (miss-islington) Date: 2022-01-21 21:18
New changeset 00b2b578bd9e516d601063a086b03177f546bcdd by Jason R. Coombs in branch 'main':
bpo-46124: Update zoneinfo to rely on importlib.resources traversable API. (GH-30190)
https://github.com/python/cpython/commit/00b2b578bd9e516d601063a086b03177f546bcdd
msg411194 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2022-01-21 21:59
Closing, presumed fixed. Please re-open if not.
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90282
2022-01-21 21:59:16jaracosetstatus: open -> closed
resolution: fixed
messages: + msg411194

stage: patch review -> resolved
2022-01-21 21:18:35miss-islingtonsetnosy: + miss-islington
messages: + msg411185
2021-12-28 21:16:34jaracosetmessages: + msg409273
2021-12-28 21:15:41jaracosetmessages: + msg409272
2021-12-28 16:33:11xtreaksetmessages: + msg409260
2021-12-28 15:09:09p-gansslesetmessages: + msg409259
2021-12-19 03:41:41jaracosetassignee: jaraco -> p-ganssle
2021-12-18 23:41:42jaracosetkeywords: + patch
stage: patch review
pull_requests: + pull_request28410
2021-12-18 22:31:48jaracosetmessages: + msg408873
2021-12-18 22:25:22jaracosetmessages: + msg408871
2021-12-18 22:22:25jaracosetassignee: jaraco
2021-12-18 22:22:16jaracosetmessages: + msg408870
2021-12-18 14:04:21xtreakcreate