classification
Title: test_TimeRE_recreation_timezone failure on systems with non-default posixrules
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, izbyshev, miss-islington, p-ganssle, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2018-11-05 22:53 by izbyshev, last changed 2018-11-06 04:59 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 10347 merged izbyshev, 2018-11-05 22:55
PR 10357 merged miss-islington, 2018-11-06 04:29
PR 10358 merged miss-islington, 2018-11-06 04:29
PR 10359 merged miss-islington, 2018-11-06 04:29
Messages (5)
msg329320 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-11-05 22:53
I've got the following on OpenSUSE Tumbleweed (glibc 2.27):

======================================================================
FAIL: test_TimeRE_recreation_timezone (test.test_strptime.CacheTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/scratch2/izbyshev/cpython/Lib/test/support/__init__.py", line 1683, in inner
    return func(*args, **kwds)
  File "/scratch2/izbyshev/cpython/Lib/test/test_strptime.py", line 702, in test_TimeRE_recreation_timezone
    self.assertEqual(tm.tm_isdst, 1)
AssertionError: 0 != 1

This test sets TZ environment variable to 'STD-1DST' and calls time.tzset() (inside support.run_with_tz decorator). This TZ value lacks rules that specify when DST transitions happen. POSIX seems to be silent what should be done if the rule is missing, thought it does specify that the rule is optional[1].

What actually happens at least on Linux/glibc[2] and FreeBSD[3] is that those rules are taken from 'posixrules' file. On OpenSUSE, it's linked to '/etc/localtime', which is linked to 'Europe/Moscow' in my case. DST transitions were cancelled in Russia in 2011, so when Python tries to get timezone names for two points of the year (https://github.com/python/cpython/blob/570e371fd6e8615ece9b9e21fbe77149ebeb172e/Modules/timemodule.c#L1603), it gets unexpected values instead. The actual values depend on the bugginess of the libc used:

* Glibc seems to be confused by having 'posixrules' file with DST cancelled but a specification which includes the DST timezone part, so localtime() returns 'MSK' in tm_zone (both for January and July).

* musl doesn't implement 'posixrules' fallback, so it uses "all-zero" rules instead which cause it return 'DST' in both cases.

* FreeBSD 11.1 (with 'posixrules' linked to 'Europe/Moscow') returns 'STD' in both cases.

With any of the above, the test fails because strptime is called with the same timezone name two times.

Note that even if PyInit_timezone() didn't probe January/July and used 'tzname' global instead, it wouldn't work as expected at least on FreeBSD: it sets tzname[1] to the empty string in the test above.

ISTM the best way to fix this is to remove dependence on 'posixrules' file by specifying the rules in TZ.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08
[2] http://man7.org/linux/man-pages/man3/tzset.3.html
[3] https://www.freebsd.org/cgi/man.cgi?query=tzset&sektion=3
msg329338 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2018-11-06 04:29
New changeset f1b9ad3d38c11676b45edcbf2369239bae436e56 by Benjamin Peterson (Alexey Izbyshev) in branch 'master':
closes bpo-35171: Fix test_TimeRE_recreation_timezone failure on some systems. (GH-10347)
https://github.com/python/cpython/commit/f1b9ad3d38c11676b45edcbf2369239bae436e56
msg329340 - (view) Author: miss-islington (miss-islington) Date: 2018-11-06 04:47
New changeset ca592bcf56fca93210331327777fe20fdeff985a by Miss Islington (bot) in branch '3.7':
closes bpo-35171: Fix test_TimeRE_recreation_timezone failure on some systems. (GH-10347)
https://github.com/python/cpython/commit/ca592bcf56fca93210331327777fe20fdeff985a
msg329341 - (view) Author: miss-islington (miss-islington) Date: 2018-11-06 04:50
New changeset d0e3105f7ca3fc54b167edc756ce545cbab0ce95 by Miss Islington (bot) in branch '3.6':
closes bpo-35171: Fix test_TimeRE_recreation_timezone failure on some systems. (GH-10347)
https://github.com/python/cpython/commit/d0e3105f7ca3fc54b167edc756ce545cbab0ce95
msg329342 - (view) Author: miss-islington (miss-islington) Date: 2018-11-06 04:59
New changeset ffbce43c1aa457b5976665eb2e47771198c7af06 by Miss Islington (bot) in branch '2.7':
closes bpo-35171: Fix test_TimeRE_recreation_timezone failure on some systems. (GH-10347)
https://github.com/python/cpython/commit/ffbce43c1aa457b5976665eb2e47771198c7af06
History
Date User Action Args
2018-11-06 04:59:10miss-islingtonsetmessages: + msg329342
2018-11-06 04:50:06miss-islingtonsetmessages: + msg329341
2018-11-06 04:47:14miss-islingtonsetnosy: + miss-islington
messages: + msg329340
2018-11-06 04:29:48miss-islingtonsetpull_requests: + pull_request9659
2018-11-06 04:29:37miss-islingtonsetpull_requests: + pull_request9658
2018-11-06 04:29:26miss-islingtonsetpull_requests: + pull_request9657
2018-11-06 04:29:11benjamin.petersonsetstatus: open -> closed
resolution: fixed
messages: + msg329338

stage: patch review -> resolved
2018-11-06 01:19:59p-gansslesetnosy: + p-ganssle
2018-11-05 22:55:17izbyshevsetkeywords: + patch
stage: patch review
pull_requests: + pull_request9647
2018-11-05 22:53:29izbyshevcreate