classification
Title: test__xxsubinterpreters.ShareableTypeTests._assert_values fails on AIX - 32-bit mode
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Michael.Felt, eric.snow
Priority: normal Keywords: patch

Created on 2018-09-03 13:47 by Michael.Felt, last changed 2019-01-11 18:18 by eric.snow. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 9127 merged Michael.Felt, 2018-09-09 14:45
Messages (5)
msg324519 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-09-03 13:47
+364      def _assert_values(self, values):
  +365          for obj in values:
  +366              with self.subTest(obj):
  +367                  interpreters.channel_send(self.cid, obj)
  +368                  got = interpreters.channel_recv(self.cid)
  +369
  +370                  self.assertEqual(got, obj)
  +371                  self.assertIs(type(got), type(obj))
  +372                  # XXX Check the following in the channel tests?
  +373                  #self.assertIsNot(got, obj)
  +374

  +395      def test_int(self):
  +396          self._assert_values(range(-1, 258))
  +397

The assert fails on -1 with:

======================================================================
FAIL: test_int (test.test__xxsubinterpreters.ShareableTypeTests) [-1]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/python3-3.8.0/Lib/test/test__xxsubinterpreters.py", line 371, in _assert_values
    self.assertEqual(got, obj)
AssertionError: 4294967295 != -1

Note that this value is the unsigned value for 32-bit int as -1

root@x066:[/data/prj/python/python3-3.8.0]grep 4294967295 /usr/include/sys/*.h
/usr/include/sys/limits.h:#define ULONG_MAX     (4294967295UL)
/usr/include/sys/limits.h:#define UINT_MAX      (4294967295U)
/usr/include/sys/stdint.h:#define UINT32_MAX    (4294967295U)

After quite a lot of "learning", I narrow the issue to:

 +1432  static int
 +1433  _long_shared(PyObject *obj, _PyCrossInterpreterData *data)
 +1434  {
 +1435      int64_t value = PyLong_AsLongLong(obj);
 +1436      if (value == -1 && PyErr_Occurred()) {
 +1437          if (PyErr_ExceptionMatches(PyExc_OverflowError)) {
 +1438              PyErr_SetString(PyExc_OverflowError, "try sending as bytes");
 +1439          }
 +1440          return -1;
 +1441      }
 +1442      data->data = (void *)value;
 +1443      data->obj = NULL;
 +1444      data->new_object = _new_long_object;
 +1445      data->free = NULL;
 +1446      return 0;
 +1447  }
 +1448

 +1426  static PyObject *
 +1427  _new_long_object(_PyCrossInterpreterData *data)
 +1428  {
 +1429      return PyLong_FromLongLong((int64_t)(data->data));
 +1430  }

The "value" is stored as a void data type, and the high-order 64-bits are zero. When it gets returned as a PyLong... it goes positive.

I do not dare touch anything here without some "mentoring".

Or, we change the test so that it knows it is in 32-bit mode, and compares with something else.

In short, "mentoring" requested.

p.s. not had time to test in 64-bit mode. Will post on that later.
msg324560 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-09-04 06:53
64-bit mode, no error.

root@x066:[/data/prj/python/python3-3.8.0]./python -m test -v test__xxsubinterpreters
== CPython 3.8.0a0 (heads/master-dirty:d500e5307a, Sep 3 2018, 13:55:44) [C]
== AIX-1-00C291F54C00-powerpc-64bit-COFF big-endian
== cwd: /data/prj/python/python3-3.8.0/build/test_python_16908532
== CPU count: 8
== encodings: locale=ISO8859-1, FS=iso8859-1
Run tests sequentially
0:00:00 [1/1] test__xxsubinterpreters
test_bad_id (test.test__xxsubinterpreters.ChannelIDTests) ... ok
...
test_int (test.test__xxsubinterpreters.ShareableTypeTests) ... ok
test_singletons (test.test__xxsubinterpreters.ShareableTypeTests) ... ok
test_types (test.test__xxsubinterpreters.ShareableTypeTests) ... ok

----------------------------------------------------------------------
Ran 111 tests in 4.572s

OK (skipped=5)

== Tests result: SUCCESS ==
msg325013 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2018-09-11 14:52
Thanks for bringing this up, Michael.  I'll give you a review on the PR sometime this week (while at the core sprint).
msg333496 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-01-11 18:17
New changeset a909460a09cca79bd051c45b02e650862a57dbd9 by Eric Snow (Michael Felt) in branch 'master':
bpo-34569: Fix subinterpreter 32-bit  ABI, pystate.c/_new_long_object() (gh-9127)
https://github.com/python/cpython/commit/a909460a09cca79bd051c45b02e650862a57dbd9
msg333498 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-01-11 18:18
Thanks, Michael.
History
Date User Action Args
2019-01-11 18:18:13eric.snowsetstatus: open -> closed
resolution: fixed
messages: + msg333498

stage: patch review -> resolved
2019-01-11 18:17:12eric.snowsetmessages: + msg333496
2018-09-11 14:52:40eric.snowsetmessages: + msg325013
2018-09-11 05:44:42Michael.Feltsettype: behavior
2018-09-09 14:45:17Michael.Feltsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8580
2018-09-04 06:54:08Michael.Feltsetcomponents: + Tests
2018-09-04 06:53:58Michael.Feltsetmessages: + msg324560
versions: + Python 3.8
2018-09-03 13:50:12vstinnersetnosy: + eric.snow
2018-09-03 13:47:59Michael.Feltcreate