classification
Title: PEP 615: Add zoneinfo module
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: p-ganssle Nosy List: belopolsky, lemburg, p-ganssle, serhiy.storchaka, steve.dower, twouters, vstinner
Priority: high Keywords: patch

Created on 2020-05-04 18:51 by p-ganssle, last changed 2020-05-18 12:32 by vstinner.

Pull Requests
URL Status Linked Edit
PR 19909 merged p-ganssle, 2020-05-04 18:54
PR 20006 merged p-ganssle, 2020-05-08 19:24
PR 20034 closed p-ganssle, 2020-05-11 16:57
Messages (11)
msg368076 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2020-05-04 18:51
This is an issue to track the implementation of PEP 615: https://www.python.org/dev/peps/pep-0615/

It should mostly involve migrating from the reference implementation: https://github.com/pganssle/zoneinfo/
msg368461 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2020-05-08 20:42
I've separated this into two separate PRs, one for docs and one for tests/implementation.

I have not yet implemented the logic for the ability to configure the TZPATH at compile time because I'm not quite sure how to start on that. How are other compile-time options implemented? Do I need to do something special to handle both Windows and POSIX systems? Is there already a configuration file somewhere that I should use for this? Adding Thomas to the nosy list because he's the only one listed for "autoconf/makefiles" – don't know if that extends to the Windows build as well.
msg368619 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2020-05-11 08:23
The normal way to do this (for make/autoconf) is to add a --with-tzpath argument to configure.ac, and a make variable to pass it to the compilation of anything that needs it. You can then access it from Python code with sysconfig.get_config_var().

In configure.ac, AC_SUBST(TZPATH) makes configure replace @TZPATH@ in the Makefile with the value you set to $TZPATH in configure.ac.  You then either add that to the global PY_CFLAGS_NODIST, or modify the build rule for the module that needs it to pass it along. (See for example how GITTAG/GITVERSION/GITBRANCH are passed to Modules/getbuildinfo.o.)

AC_ARG_WITH() is how you add a new --with-* argument to configure. The usual way people do this is by copying one of the other AC_ARG_WITH blocks and modifying it to suit their needs. It's a mixture of m4 and shell that can be a bit annoying to get right, but it's pretty flexible. Run autoreconf to regenerate configure. You can manually check that the shell in configure makes sense.

Something will have to be done on the Windows side as well, but I'm not sure what. Adding Steve Dower for that.
msg368634 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2020-05-11 17:01
Thanks Thomas, that was super helpful. I've created GH-20034 to add in the compile-time arguments on POSIX systems at least, do you mind having a look?

For the moment I have made it non-configurable on Windows, but I think the right thing to do is to add an argument to PCbuild\build.bat. Hopefully Steve can point me in the right direction for mapping that argument (or something else) to a new config variable in sysconfig.
msg368649 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-05-11 21:00
Do we need the C implementation if there is the Python implementation?
msg368656 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2020-05-11 21:26
I mean, theoretically we don't "need" it, but it's much, much faster, and without it nearly every operation that needs time zone offsets will be slower than pytz (which has a mechanism for caching).

Also, I've already written it, so I see no reason why not use it.
msg368658 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2020-05-11 21:38
Here are some benchmarks run using the latest implementation. The pure Python code is pretty optimized, but the C code is still ~4-5x faster.

Running from_utc in zone Europe/Paris
c_zoneinfo: mean: 494.82 ns ± 3.80 ns; min: 489.23 ns (k=5, N=500000)
py_zoneinfo: mean: 2.48 µs ± 79.17 ns; min: 2.42 µs (k=5, N=100000)
dateutil: mean: 10.41 µs ± 209.97 ns; min: 10.17 µs (k=5, N=50000)
pytz: mean: 4.69 µs ± 252.70 ns; min: 4.39 µs (k=5, N=50000)

Running to_utc in zone Europe/Paris
c_zoneinfo: mean: 539.61 ns ± 25.68 ns; min: 514.39 ns (k=5, N=500000)
py_zoneinfo: mean: 2.01 µs ± 61.69 ns; min: 1.94 µs (k=5, N=100000)
dateutil: mean: 7.88 µs ± 506.89 ns; min: 7.25 µs (k=5, N=50000)
pytz: mean: 773.02 ns ± 14.11 ns; min: 759.56 ns (k=5, N=500000)

Running utcoffset in zone Europe/Paris
c_zoneinfo: mean: 329.34 ns ± 36.31 ns; min: 302.88 ns (k=5, N=1000000)
py_zoneinfo: mean: 1.57 µs ± 9.58 ns; min: 1.55 µs (k=5, N=200000)
dateutil: mean: 6.28 µs ± 86.61 ns; min: 6.16 µs (k=5, N=50000)
pytz: mean: 461.47 ns ± 2.07 ns; min: 458.91 ns (k=5, N=500000)


`utcoffset()` is very likely to be called possibly many times in certain hot loops (including implicitly as it's part of hash and equality checks).
msg368800 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2020-05-13 20:05
Talked to Steve Dower in a sidebar about the issue with compile-time configuration, he is convinced that compile-time configuration is not something that would be useful or worth doing on Windows. I am indifferent on the matter, so I am fine with calling the compile-time configuration part of this done at this point. If anyone building Python for Windows shows up needing support for this, we can re-visit the issue — I don't believe it's technically infeasible, just that the usage patterns of Python on Windows mean that it's not worth doing.

So, once we've got a critical mass of reviews done for zoneinfo, I think this feature is done. 🎉
msg369021 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-16 08:20
New changeset 62972d9d73e83d6eea157617cc69500ffec9e3f0 by Paul Ganssle in branch 'master':
bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
https://github.com/python/cpython/commit/62972d9d73e83d6eea157617cc69500ffec9e3f0
msg369059 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2020-05-16 16:15
New changeset b17e49e0def23238b9e7f48c8a02e2d7bbf1f653 by Paul Ganssle in branch 'master':
bpo-40503: Add documentation and what's new entry for zoneinfo (GH-20006)
https://github.com/python/cpython/commit/b17e49e0def23238b9e7f48c8a02e2d7bbf1f653
msg369206 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-18 12:32
I suggest to open a separated issue to discuss how tzdata can be installed on Travis CI, Azure Pipelines, Buildbots, etc. when running tests.
History
Date User Action Args
2020-05-18 12:32:37vstinnersetmessages: + msg369206
2020-05-16 16:15:06p-gansslesetmessages: + msg369059
2020-05-16 08:20:09vstinnersetnosy: + vstinner
messages: + msg369021
2020-05-13 20:05:35p-gansslesetmessages: + msg368800
2020-05-11 21:38:53p-gansslesetmessages: + msg368658
2020-05-11 21:26:02p-gansslesetmessages: + msg368656
2020-05-11 21:00:28serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg368649
2020-05-11 17:01:42p-gansslesetmessages: + msg368634
2020-05-11 16:57:33p-gansslesetpull_requests: + pull_request19343
2020-05-11 08:23:55twouterssetnosy: + steve.dower
messages: + msg368619
2020-05-08 20:42:43p-gansslesetnosy: + twouters
messages: + msg368461
2020-05-08 19:24:39p-gansslesetpull_requests: + pull_request19318
2020-05-04 18:54:44p-gansslesetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request19224
2020-05-04 18:51:21p-gansslecreate