classification
Title: test_strptime failure on OpenBSD
Type: behavior Stage:
Components: Library (Lib) Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, davin, lemburg, p-ganssle, serhiy.storchaka, xtreak
Priority: normal Keywords:

Created on 2017-09-29 09:32 by serhiy.storchaka, last changed 2018-09-18 14:37 by p-ganssle.

Messages (7)
msg303319 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-29 09:32
$ ./python -m test -vuall test_strptime
...
======================================================================
FAIL: test_week_of_year_and_day_of_week_calculation (test.test_strptime.CalculationTests) [Jan 1 on Sunday] (date=(1905, 1, 1), format='%G %V %w')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/serhiy/py/cpython3.7/Lib/test/test_strptime.py", line 508, in test_helper
    self.assertEqual(strp_output[:3], ymd_tuple, msg)
AssertionError: Tuples differ: (1905, 1, 8) != (1905, 1, 1)

First differing element 2:
8
1

- (1905, 1, 8)
?           ^

+ (1905, 1, 1)
?           ^
 : '1904 53 0': 374 != 1

======================================================================
...
FAILED (failures=16, skipped=1)
msg325496 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2018-09-16 18:01
I think this is the relevant test: https://github.com/python/cpython/blob/3.7/Lib/test/test_strptime.py#L536

The issue seems to be in `strftime` rather than in `strptime`. According to Serhiy's paste, it seems that `datetime.datetime.strftime("%G %V %w")` on BSD results in "1904 53 0", whereas on Linux it results in "1904 52 0".

I suspect the confusion *may* be related to the fact that ISO weekdays go Monday (1) to Sunday (7), where as %w goes Sunday (0) to Saturday (6).
msg325497 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2018-09-16 18:04
This appears to be a duplicate of issue #13414, which was resolved as fixed in BSD. @serhiy Are you using a very old version of BSD?
msg325514 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-17 05:57
I used OpenBSD 6.1. I'll check on more modern versions when have chance.
msg325545 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2018-09-17 13:56
@Serhiy Looking more closely at #13414, I think there's a good chance that we've just discovered a different bug in OpenBSD's implementation of wcsftime (or strftime). Looks like OpenBSD 6.1 was released in 2017, which should have been years after the fix from #13414 was released.

It seems that OpenBSD does not have a bug tracker per se, and I can't see if this bug has been reported. I can try to install OpenBSD in a VM later and try to reproduce it, but if it's a bug in OpenBSD, I see some possible options:

1. Add an exception to the test suite on OpenBSD (to be removed or pinned when it's fixed in OpenBSD)

2. If the bug is just in wcsftime and not in strftime, it may be fixed by switching to using strftime as is done on Windows: https://github.com/python/cpython/blob/master/Modules/timemodule.c#L673

I have never tested it, but it's possible this will impact performance, and if it's a bad enough hit it may not be worth sacrificing general performance to avoid a bug in a corner case.
msg325640 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2018-09-18 13:19
I do think this is an active bug in OpenBSD's ISO calculations, here: https://github.com/openbsd/src/blob/b66614995ab119f75167daaa7755b34001836821/lib/libc/time/wcsftime.c#L326

I re-created this algorithm in Python and at the beginning of the year it gives the wrong answer at the beginning of the year in about 50% of years. I have spent some time trying to understand the way the algorithm is *supposed* to work, but I don't totally understand what it's doing. I'll report this to OpenBSD.
msg325653 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2018-09-18 14:37
Bug reported to BSD here: https://marc.info/?l=openbsd-bugs&m=153728102618747&w=2
History
Date User Action Args
2018-09-18 14:37:36p-gansslesetmessages: + msg325653
2018-09-18 13:19:45p-gansslesetmessages: + msg325640
2018-09-17 13:56:06p-gansslesetmessages: + msg325545
2018-09-17 05:57:17serhiy.storchakasetmessages: + msg325514
2018-09-16 18:04:54p-gansslesetmessages: + msg325497
2018-09-16 18:01:48p-gansslesetmessages: + msg325496
2018-09-14 19:09:40p-gansslesetnosy: + p-ganssle
2018-09-14 17:24:35xtreaksetnosy: + xtreak
2017-09-29 09:44:23serhiy.storchakasetnosy: + davin
2017-09-29 09:32:17serhiy.storchakacreate