classification
Title: test_time error on AIX
Type: behavior Stage:
Components: Extension Modules Versions: Python 3.2
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, alef, belopolsky, haypo, python-dev, sable
Priority: normal Keywords: patch

Created on 2011-02-11 14:51 by sable, last changed 2014-02-21 22:54 by python-dev.

Files
File name Uploaded Description Edit
strftime_aix.patch haypo, 2011-02-14 17:23
Messages (15)
msg128396 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-11 14:51
I have the following errors on test_time on AIX:

======================================================================
ERROR: test_mktime (test.test_time.TestAsctime4dyear)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (test.test_time.TestStrftime4dyear)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (test.test_time.Test4dyearBool)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
FAIL: test_negative (test.test_time.TestStrftime4dyear)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py", line 337, in test_negative
    self.assertIn(text, ('-1', '-001'))
AssertionError: '000/' not found in ('-1', '-001')

----------------------------------------------------------------------
Ran 42 tests in 1.217s

FAILED (failures=1, errors=3)

Haven't investigated yet.
msg128541 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-02-14 12:28
>  File ".../Lib/test/test_time.py", line 351, in test_mktime
>    self.assertEqual(time.mktime(tt), t)
> OverflowError: mktime argument out of range

I don't know which values are "out of range". But I guess that the test fails because of t=-1, which would mean that tm_wday field is not modified by mktime().

I don't know if mktime() does really not support t=-1, or if we should use another sentinel. The original patch to fix this issue (support t=-1, issue #1726687) used tm_wday=42 instead of tm_wday=-1:

+	/* invalid value that will not be changed if there is an error. */
+	buf.tm_wday = 42;
 	tt = mktime(&buf);
-	if (tt == (time_t)(-1)) {
+	if ((tt == (time_t)(-1)) && (buf.tm_wday == 42)) {
 		PyErr_SetString(PyExc_OverflowError,
 				"mktime argument out of range");
 		return NULL;

The current code uses:

    buf.tm_wday = -1;  /* sentinel; original value ignored */                   
    tt = mktime(&buf);                                                          
    /* Return value of -1 does not necessarily mean an error, but tm_wday       
     * cannot remain set to -1 if mktime succedded. */                          
    if (tt == (time_t)(-1) && buf.tm_wday == -1) {                              
        PyErr_SetString(PyExc_OverflowError,                                    
                        "mktime argument out of range");                        
        return NULL;                                                            
    }                 


>  File ".../Lib/test/test_time.py", line 337, in test_negative
>    self.assertIn(text, ('-1', '-001'))
> AssertionError: '000/' not found in ('-1', '-001')

AIX doesn't support negative tm_year value. It should be added to the timemodule blacklist (#ifdef):

#if defined(_MSC_VER) || defined(sun)
    if (buf.tm_year + 1900 < 1 || 9999 < buf.tm_year + 1900) {
        PyErr_Format(PyExc_ValueError,
                     "strftime() requires year in [1; 9999]",
                     buf.tm_year + 1900);
        return NULL;
    }
#endif

I don't know if there is a reliable C define to check if the current OS is AIX (something like #ifdef sun). Python configure script checks if "uname -s" output starts with "AIX". We should maybe add a Python define in pyconfig.h using the configure script.
msg128556 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-14 16:48
I tried the following patch (_AIX is defined on AIX platforms):

Index: Modules/timemodule.c
===================================================================
--- Modules/timemodule.c	(révision 88420)
+++ Modules/timemodule.c	(copie de travail)
@@ -474,7 +474,7 @@
     else if (!gettmarg(tup, &buf) || !checktm(&buf))
         return NULL;
 
-#if defined(_MSC_VER) || defined(sun)
+#if defined(_MSC_VER) || defined(sun) || defined(_AIX)
     if (buf.tm_year + 1900 < 1 || 9999 < buf.tm_year + 1900) {
         PyErr_Format(PyExc_ValueError,
                      "strftime() requires year in [1; 9999]",
@@ -694,11 +694,12 @@
     time_t tt;
     if (!gettmarg(tup, &buf))
         return NULL;
-    buf.tm_wday = -1;  /* sentinel; original value ignored */
+    /* invalid value that will not be changed if there is an error. */
+    buf.tm_wday = 42;
     tt = mktime(&buf);
     /* Return value of -1 does not necessarily mean an error, but tm_wday
      * cannot remain set to -1 if mktime succedded. */
-    if (tt == (time_t)(-1) && buf.tm_wday == -1) {
+    if ((tt == (time_t)(-1)) && (buf.tm_wday == 42)) {
         PyErr_SetString(PyExc_OverflowError,
                         "mktime argument out of range");
         return NULL;

This resulted in the following:
======================================================================
ERROR: test_mktime (__main__.TestAsctime4dyear)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (__main__.TestStrftime4dyear)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

======================================================================
ERROR: test_mktime (__main__.Test4dyearBool)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
    self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

----------------------------------------------------------------------
Ran 42 tests in 1.395s

FAILED (errors=3)

So test_negative is now OK, but tm_wday = 42 did not solve the problem it seems.
msg128561 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-02-14 17:23
> So test_negative is now OK

Ok, I converted your comment to a patch: strftime_aix.patch.
msg128562 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-02-14 17:29
> but tm_wday = 42 did not solve the problem it seems.

Can you try with tm_yday=-1 or tm_isdst=-2?
msg128565 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-02-14 17:42
I have come across the following post:

http://code.google.com/p/y2038/wiki/AmazingDiscoveries

I think some of the findings there may be relevant as we push the
boundaries of supported time values.
msg128577 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-02-14 23:01
> http://code.google.com/p/y2038/wiki/AmazingDiscoveries

<<< AIX again

Merijn informs me that before year 0 AIX gets very, very slow. >>>
msg128601 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-15 14:57
Here is what I could find in the "man mktime":
"""
       Upon successful completion, the mktime subroutine sets the values of the tm_wday and tm_yday fields appropriately. Other fields are set to
       represent the specified time since January 1, 1970. However, the values are forced to the ranges specified in the /usr/include/time.h file.
       The final value of the tm_mday field is not set until the values of the tm_mon and tm_year fields are determined. Note: The mktime
       subroutine cannot convert time values before 00:00:00 UTC, January 1, 1970 and after 03:14:07 UTC, January 19, 2038.
"""

I tried tm_yday=-1 then tm_isdst=-2 and both gave the same errors as before.
msg128604 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-02-15 16:13
Sébastien,

Can you tell us what time.localtime(t) produces for t in (-2, -1, 0, 1)?  Apparently time.mktime() fails on values produced by time.localtime() and this sounds like a platform bug.  It is OK to restrict time_t to positive values, but in this case time.localtime(t) should reject t < 0.

If there is a Python issue here, it is likely to be in error detection in time.localtime().

Also, what is your timezone?
msg128605 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-02-15 16:29
It looks like different standards have different requirements for
localtime() error handling compare:

http://pubs.opengroup.org/onlinepubs/009695399/functions/localtime.html

and

http://pubs.opengroup.org/onlinepubs/7990989775/xsh/localtime.html

The later does not require that  localtime() returns NULL or sets
errno on error.  AIX documentation seems to be silent on this issue as
well:

http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf1/ctime.htm
msg128609 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-15 17:23
Python 3.2rc3+ (py3k:88422M, Feb 15 2011, 16:57:29) [C] on aix6
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> for t in (-2, -1, 0, 1):
...     print(time.localtime(t))
... 
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, tm_sec=58, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, tm_sec=59, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, tm_sec=1, tm_wday=3, tm_yday=1, tm_isdst=0)

I am in Paris. On the system, I get:
$ date
Tue Feb 15 18:18:58 NFT 2011
$ env | grep TZ
TZ=NFT-1DFT,M3.5.0,M10.5.0

which is strange since NFT seems to be in Australia.

I will check that tomorrow with the sysadmin.
msg128610 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-02-15 17:43
2011/2/15 Sébastien Sablé <report@bugs.python.org>:
..
>>>> for t in (-2, -1, 0, 1):
> ...     print(time.localtime(t))
> ...
> time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, tm_sec=58, tm_wday=3, tm_yday=1, tm_isdst=0)
..

This looks right. (For time.timezone = -3600.)   I actually suspected
that you were east of Greenwich.   My hypothesis is that AIX localtime
implementation adjusts t -> t - timezone before computing broken down
time and thus fails to detect that localtime() is given negative
argument.  If my hypothesis is correct, time.gmtime(-1) should fail on
your system.

..
> TZ=NFT-1DFT,M3.5.0,M10.5.0
>
> which is strange since NFT seems to be in Australia.

In your TZ setting, the UTC offset and DST rules are specified
explicitly, so it does not matter what names are given to the
timezones for most time calculations.
msg128637 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-16 09:55
gmtime(-1) worked fine :/

>>> import time
>>> time.gmtime(-1)
time.struct_time(tm_year=1969, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, tm_sec=59, tm_wday=2, tm_yday=365, tm_isdst=0)
msg131592 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-03-21 01:13
strftime_aix.patch is a little bit too strict: it looks like strftime() supports large year values (year > 9999). We may only raise an error if the year is smaller than 1.
msg211886 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-21 22:54
New changeset 00e94e454813 by Victor Stinner in branch 'default':
Issue #11188, #19748: mktime() returns -1 on error. On Linux, the tm_wday field
http://hg.python.org/cpython/rev/00e94e454813
History
Date User Action Args
2014-02-21 22:54:59python-devsetmessages: + msg211886
2013-06-19 21:26:04David.Edelsohnsetnosy: + David.Edelsohn
2013-06-19 21:25:56David.Edelsohnsettype: behavior
components: + Extension Modules
2013-03-14 13:35:29alefsetnosy: + alef
2011-05-09 13:19:22mark.dickinsonsetmessages: - msg135584
2011-05-09 13:02:50python-devsetnosy: + python-dev
messages: + msg135584
2011-03-21 01:13:44hayposetnosy: belopolsky, haypo, sable
messages: + msg131592
2011-02-16 09:55:18sablesetnosy: belopolsky, haypo, sable
messages: + msg128637
2011-02-15 17:43:18belopolskysetnosy: belopolsky, haypo, sable
messages: + msg128610
2011-02-15 17:23:12sablesetnosy: belopolsky, haypo, sable
messages: + msg128609
2011-02-15 16:29:31belopolskysetnosy: belopolsky, haypo, sable
messages: + msg128605
2011-02-15 16:13:23belopolskysetnosy: belopolsky, haypo, sable
messages: + msg128604
2011-02-15 14:57:45sablesetnosy: belopolsky, haypo, sable
messages: + msg128601
2011-02-14 23:01:02hayposetnosy: belopolsky, haypo, sable
messages: + msg128577
2011-02-14 17:42:19belopolskysetnosy: belopolsky, haypo, sable
messages: + msg128565
2011-02-14 17:29:29hayposetnosy: belopolsky, haypo, sable
messages: + msg128562
2011-02-14 17:23:36hayposetfiles: + strftime_aix.patch

messages: + msg128561
keywords: + patch
nosy: belopolsky, haypo, sable
2011-02-14 16:48:00sablesetnosy: belopolsky, haypo, sable
messages: + msg128556
2011-02-14 12:28:57hayposetnosy: belopolsky, haypo, sable
messages: + msg128541
2011-02-11 19:56:07belopolskysetnosy: + haypo
2011-02-11 15:43:06pitrousetnosy: + belopolsky
2011-02-11 14:51:10sablecreate