tcl test change crashes AIX #66150

DavidEdelsohn · 2014-07-10T17:38:47Z

BPO	21951
Nosy	@vstinner, @serhiy-storchaka
Files	tkinter_nomemory.patch tkinter_debug.patch asobj_empty_sequence.patch asobj_empty_sequence_2.patch tkinter_ckallock.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-09-11.11:07:28.538>
created_at = <Date 2014-07-10.17:38:46.705>
labels = ['interpreter-core', 'expert-tkinter', 'type-crash']
title = 'tcl test change crashes AIX'
updated_at = <Date 2014-09-11.11:35:34.916>
user = 'https://bugs.python.org/DavidEdelsohn'

bugs.python.org fields:

activity = <Date 2014-09-11.11:35:34.916>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-09-11.11:07:28.538>
closer = 'vstinner'
components = ['Interpreter Core', 'Tkinter']
creation = <Date 2014-07-10.17:38:46.705>
creator = 'David.Edelsohn'
dependencies = []
files = ['35944', '36530', '36535', '36544', '36582']
hgrepos = []
issue_num = 21951
keywords = ['patch']
message_count = 26.0
messages = ['222680', '222837', '222897', '222950', '222965', '222968', '222970', '222977', '224290', '224291', '226345', '226359', '226361', '226362', '226364', '226367', '226374', '226400', '226637', '226721', '226723', '226751', '226754', '226762', '226763', '226765']
nosy_count = 4.0
nosy_names = ['vstinner', 'python-dev', 'serhiy.storchaka', 'David.Edelsohn']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue21951'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

DavidEdelsohn · 2014-07-10T17:38:46Z

The patch for Issue bpo-21881 causes CPython test_tcl to crash on AIX.

$ ./python -m test -v test_tcl
== CPython 3.5.0a0 (default:d1f89eb9ea1e+, Jul 10 2014, 10:21:22) [GCC 4.8.1]
==   AIX-1-00F84C0C4C00-powerpc-32bit big-endian
==   hash algorithm: siphash24 32bit
==   /home/dje/src/cpython/build/test_python_27984522
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_tcl
patchlevel = 8.4.7
testCall (test.test_tcl.TclTest) ... ok
testCallException (test.test_tcl.TclTest) ... ok
testCallException2 (test.test_tcl.TclTest) ... ok
testEval (test.test_tcl.TclTest) ... ok
testEvalException (test.test_tcl.TclTest) ... ok
testEvalException2 (test.test_tcl.TclTest) ... ok
testEvalFile (test.test_tcl.TclTest) ... ok
testEvalFileException (test.test_tcl.TclTest) ... ok
testGetVar (test.test_tcl.TclTest) ... ok
testGetVarArray (test.test_tcl.TclTest) ... ok
testGetVarArrayException (test.test_tcl.TclTest) ... ok
testGetVarException (test.test_tcl.TclTest) ... ok
testLoadWithUNC (test.test_tcl.TclTest) ... skipped 'Requires Windows'
testPackageRequireException (test.test_tcl.TclTest) ... ok
testSetVar (test.test_tcl.TclTest) ... ok
testSetVarArray (test.test_tcl.TclTest) ... ok
testUnsetVar (test.test_tcl.TclTest) ... ok
testUnsetVarArray (test.test_tcl.TclTest) ... ok
testUnsetVarException (test.test_tcl.TclTest) ... ok
test_eval_null_in_result (test.test_tcl.TclTest) ... ok
test_evalfile_null_in_result (test.test_tcl.TclTest) ... ok
test_exprboolean (test.test_tcl.TclTest) ... ok
test_exprdouble (test.test_tcl.TclTest) ... ok
test_exprlong (test.test_tcl.TclTest) ... ok
test_exprstring (test.test_tcl.TclTest) ... ok
test_getboolean (test.test_tcl.TclTest) ... ok
test_getdouble (test.test_tcl.TclTest) ... ok
test_getint (test.test_tcl.TclTest) ... ok
test_passing_values (test.test_tcl.TclTest) ... ok
test_split (test.test_tcl.TclTest) ... ok
test_splitlist (test.test_tcl.TclTest) ... ok
test_user_command (test.test_tcl.TclTest) ... Assertion failed: __EX, file  Objects/methodobject.c, line 94
Fatal Python error: Aborted

Current thread 0x00000001 (most recent call first):
File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 426 in check
File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 458 in test_user_command
File "/home/dje/src/cpython/Lib/unittest/case.py", line 577 in run
File "/home/dje/src/cpython/Lib/unittest/case.py", line 625 in __call__
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__
File "/home/dje/src/cpython/Lib/unittest/runner.py", line 168 in run
File "/home/dje/src/cpython/Lib/test/support/init.py", line 1724 in _run_suite
File "/home/dje/src/cpython/Lib/test/support/init.py", line 1758 in run_unittest
File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 601 in test_main
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1278 in runtest_inner
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 978 in runtest
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 763 in main
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1562 in main_in_temp_cwd
File "/home/dje/src/cpython/Lib/test/main.py", line 3 in <module>
File "/home/dje/src/cpython/Lib/runpy.py", line 85 in _run_code
File "/home/dje/src/cpython/Lib/runpy.py", line 170 in _run_module_as_main
IOT/Abort trap (core dumped)

serhiy-storchaka · 2014-07-12T12:34:14Z

Is it crashes when comment out "check((), '')"? Is it crashes when comment out two previous checks (for inf and -inf)?

DavidEdelsohn · 2014-07-13T00:02:05Z

If I comment out all three tests, it runs.

--- a/Lib/test/test_tcl.py      Thu Jul 10 01:17:11 2014 -0400
+++ b/Lib/test/test_tcl.py      Sat Jul 12 16:59:33 2014 -0700
@@ -455,9 +455,9 @@
         check(float('inf'), 'Inf', eq=float_eq)
         check(-float('inf'), '-Inf', eq=float_eq)
         # XXX NaN representation can be not parsable by float()
-        check((), '')
-        check((1, (2,), (3, 4), '5 6', ()), '1 2 {3 4} {5 6} {}')
-        check([1, [2,], [3, 4], '5 6', []], '1 2 {3 4} {5 6} {}')
+        #check((), '')
+        #check((1, (2,), (3, 4), '5 6', ()), '1 2 {3 4} {5 6} {}')
+        #check([1, [2,], [3, 4], '5 6', []], '1 2 {3 4} {5 6} {}')
 
     def test_splitlist(self):
         splitlist = self.interp.tk.splitlist

serhiy-storchaka · 2014-07-13T16:23:27Z

Is following test passed?

    check((1,), '1')

Is CPython crashes when change signature of testfunc?

--- a/Lib/test/test_tcl.py      Sat Jul 12 18:26:03 2014 +0300
+++ b/Lib/test/test_tcl.py      Sun Jul 13 19:21:55 2014 +0300
@@ -416,7 +416,7 @@
 
     def test_user_command(self):
         result = None
-        def testfunc(arg):
+        def testfunc(arg=None, *args):
             nonlocal result
             result = arg
             return arg

DavidEdelsohn · 2014-07-13T18:53:04Z

Is following test passed?

    check((1,), '1')

That test succeeds.

Is CPython crashes when change signature of testfunc?

--- a/Lib/test/test_tcl.py      Sat Jul 12 18:26:03 2014 +0300
+++ b/Lib/test/test_tcl.py      Sun Jul 13 19:21:55 2014 +0300
@@ -416,7 +416,7 @@
 
     def test_user_command(self):
         result = None
-        def testfunc(arg):
+        def testfunc(arg=None, *args):
             nonlocal result
             result = arg
             return arg

This change does not have any effect on the original failure. The assertion failure and crash continues to occur.

DavidEdelsohn · 2014-07-13T19:26:51Z

#5 0x1019bb90 in PyCFunction_Call (func=0x30533b5c, arg=0x305d8ab4, kw=0x0)
at Objects/methodobject.c:94
#6 0x1012534c in call_function (pp_stack=0x2ff16144, oparg=2)
at Python/ceval.c:4269

94 CHECK_RESULT(res);

(gdb) print res
$9 = (PyObject *) 0x0
(gdb) print meth
$10 = (PyCFunction) @0x20380888: 0xd8dd57dc <Tkapp_Call>
(gdb) print self
$11 = (PyObject *) 0x305d44d8

serhiy-storchaka · 2014-07-13T19:53:52Z

May be this patch will help.

DavidEdelsohn · 2014-07-13T21:49:05Z

No difference with the patch. The problem is not a a malloc() failure.

python-dev · 2014-07-30T08:02:15Z

New changeset 7de64def6565 by Serhiy Storchaka in branch '2.7':
Issue bpo-21951: Temporary skip crashing test_user_command on AIX.
http://hg.python.org/cpython/rev/7de64def6565

New changeset 31f4cb1fede9 by Serhiy Storchaka in branch '3.4':
Issue bpo-21951: Temporary skip crashing test_user_command on AIX.
http://hg.python.org/cpython/rev/31f4cb1fede9

New changeset de32cd419174 by Serhiy Storchaka in branch 'default':
Issue bpo-21951: Temporary skip crashing test_user_command on AIX.
http://hg.python.org/cpython/rev/de32cd419174

serhiy-storchaka · 2014-07-30T08:07:23Z

Link to log with a crash:

http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2442/steps/test/logs/stdio

vstinner · 2014-09-04T07:59:05Z

The failing assertion means that the method raised an exception and returned a result, or failed but didn't raised an exception. According to the gdb trace, the function failed (res is NULL). You can display the current exception in gdb using: "print PyErr_Occurred()" (NULL if there is no exception).

Please run test_tcl on AIX with attached test_tkinter.patch applied.

serhiy-storchaka · 2014-09-04T12:00:46Z

As far as I remember, both res and PyErr_Occurred() were NULL.

DavidEdelsohn · 2014-09-04T14:22:11Z

When the failing test is re-enabled, the _tkinter patch produces the following output:

test_user_command (test.test_tcl.TclTest) ... Assertion failed: __EX, file /home/dje/src/cpython/Modules/_tkinter.c, line 1277
Fatal Python error: Aborted

Current thread 0x00000001 (most recent call first):
File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 435 in check
File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 468 in test_user_command
File "/home/dje/src/cpython/Lib/unittest/case.py", line 577 in run
File "/home/dje/src/cpython/Lib/unittest/case.py", line 625 in __call__
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run
File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in __call__
File "/home/dje/src/cpython/Lib/unittest/runner.py", line 168 in run
File "/home/dje/src/cpython/Lib/test/support/init.py", line 1750 in _run_suite
File "/home/dje/src/cpython/Lib/test/support/init.py", line 1784 in run_unittest
File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 615 in test_main
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1280 in runtest_inner
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 978 in runtest
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 763 in main
File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1564 in main_in_temp_cwd
File "/home/dje/src/cpython/Lib/test/main.py", line 3 in <module>
File "/home/dje/src/cpython/Lib/runpy.py", line 85 in _run_code
File "/home/dje/src/cpython/Lib/runpy.py", line 170 in _run_module_as_main
IOT/Abort trap (core dumped)

python-dev · 2014-09-04T15:30:33Z

New changeset 9ab404cdcaa1 by Victor Stinner in branch 'default':
Issue bpo-21951: Fix AsObj() of the _tkinter module: raise MemoryError on memory
http://hg.python.org/cpython/rev/9ab404cdcaa1

vstinner · 2014-09-04T15:47:22Z

When the failing test is re-enabled, the _tkinter patch produces the following output:

Thanks. I found and fixed a bug in AsObj() for tuple and list. It looks like ckalloc(size) returns NULL if size is 0. Please update Python, compile and run test_tcl to see if you get a MemoryError exception instead of a crash.

I just created the issue bpo-22336 which may fix this issue if my analysis was correct. It uses PyMem_Malloc(size) which has a well-defined behaviour when size is 0.

DavidEdelsohn · 2014-09-04T17:19:41Z

$ ./python -m test -v test_tcl
== CPython 3.5.0a0 (default:9ab404cdcaa1+, Sep 4 2014, 10:06:33) [GCC 4.8.1]
==   AIX-1-00F84C0C4C00-powerpc-32bit big-endian
==   hash algorithm: siphash24 32bit
==   /home/dje/src/cpython/build/test_python_48694044
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_tcl
patchlevel = 8.4.7
testCall (test.test_tcl.TclTest) ... ok
testCallException (test.test_tcl.TclTest) ... ok
testCallException2 (test.test_tcl.TclTest) ... ok
testEval (test.test_tcl.TclTest) ... ok
testEvalException (test.test_tcl.TclTest) ... ok
testEvalException2 (test.test_tcl.TclTest) ... ok
testEvalFile (test.test_tcl.TclTest) ... ok
testEvalFileException (test.test_tcl.TclTest) ... ok
testGetVar (test.test_tcl.TclTest) ... ok
testGetVarArray (test.test_tcl.TclTest) ... ok
testGetVarArrayException (test.test_tcl.TclTest) ... ok
testGetVarException (test.test_tcl.TclTest) ... ok
testLoadWithUNC (test.test_tcl.TclTest) ... skipped 'Requires Windows'
testPackageRequireException (test.test_tcl.TclTest) ... ok
testSetVar (test.test_tcl.TclTest) ... ok
testSetVarArray (test.test_tcl.TclTest) ... ok
testUnsetVar (test.test_tcl.TclTest) ... ok
testUnsetVarArray (test.test_tcl.TclTest) ... ok
testUnsetVarException (test.test_tcl.TclTest) ... ok
test_eval_null_in_result (test.test_tcl.TclTest) ... ok
test_evalfile_null_in_result (test.test_tcl.TclTest) ... ok
test_exprboolean (test.test_tcl.TclTest) ... ok
test_exprdouble (test.test_tcl.TclTest) ... ok
test_exprlong (test.test_tcl.TclTest) ... ok
test_exprstring (test.test_tcl.TclTest) ... ok
test_getboolean (test.test_tcl.TclTest) ... ok
test_getdouble (test.test_tcl.TclTest) ... ok
test_getint (test.test_tcl.TclTest) ... ok
test_passing_values (test.test_tcl.TclTest) ... ok
test_split (test.test_tcl.TclTest) ... ok
test_splitlist (test.test_tcl.TclTest) ... ok
test_user_command (test.test_tcl.TclTest) ... ERROR
testFlattenLen (test.test_tcl.TkinterTest) ... ok
test_huge_string_builtins (test.test_tcl.BigmemTclTest) ... skipped 'needs UINT_MAX < SIZE_MAX'
test_huge_string_call (test.test_tcl.BigmemTclTest) ... skipped 'needs UINT_MAX < SIZE_MAX'

======================================================================
ERROR: test_user_command (test.test_tcl.TclTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 468, in test_user_command
    check((), '')
  File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 435, in check
    r = self.interp.call('testfunc', value)
MemoryError

Ran 35 tests in 0.133s

FAILED (errors=1, skipped=3)
test test_tcl failed
1 test failed:
test_tcl

vstinner · 2014-09-04T19:41:28Z

Thanks David! Your ouput confirms my analysis. asobj_empty_sequence.patch should fix the test on AIX.

I read Tcl source code, Tcl_NewListObj() doesn't use the second parameter if the first parameter (length) is zero.

serhiy-storchaka · 2014-09-05T06:15:51Z

The patch can be simpler. And there is yet one similar bug, for unicode strings.

serhiy-storchaka · 2014-09-09T14:59:55Z

As Victor noted on IRC, ckalloc() panics and doesn't returns NULL in case of error. We should use attemptckalloc() instead.

vstinner · 2014-09-10T21:24:04Z

I'm not sure that tkinter_ckallock.patch is correct. Extract of attemptckalloc() manual page:
"If the allocation fails, these functions will return NULL. Note that on some platforms, but not all, attempting to allocate a zero-sized block of memory will also cause these functions to return NULL."

http://linux.die.net/man/3/attemptckalloc

It looks like you are fixing two different issues in the same patch: fix AIX and enhance handling of memory allocation failure.

Can you please split your patch in two parts? For AIX, adding these two lines for list/tuple should be enough:

if (size == 0)
return Tcl_NewListObj(0, NULL);

vstinner · 2014-09-10T21:26:55Z

"I'm not sure that tkinter_ckallock.patch is correct."

Oh, when I read my message, I realized that I was not explicit enough. It looks like attemptckalloc() can be called with 0 at some places with tkinter_ckallock.patch. But I didn't check carefully, the patch is maybe correct.

python-dev · 2014-09-11T08:04:31Z

New changeset ee969a717cb5 by Serhiy Storchaka in branch '2.7':
Issue bpo-21951: Use attemptckalloc() instead of ckalloc() in Tkinter.
http://hg.python.org/cpython/rev/ee969a717cb5

New changeset 1223c882253f by Serhiy Storchaka in branch '3.4':
Issue bpo-21951: Use attemptckalloc() instead of ckalloc() in Tkinter.
http://hg.python.org/cpython/rev/1223c882253f

New changeset 499b60b7d067 by Serhiy Storchaka in branch 'default':
Issue bpo-21951: Use attemptckalloc() instead of ckalloc() in Tkinter.
http://hg.python.org/cpython/rev/499b60b7d067

New changeset d6c7ab5a2065 by Serhiy Storchaka in branch '2.7':
Issue bpo-21951: Fixed a crash in Tkinter on AIX when called Tcl command with
http://hg.python.org/cpython/rev/d6c7ab5a2065

New changeset 6a96c28f9474 by Serhiy Storchaka in branch '3.4':
Issue bpo-21951: Fixed a crash in Tkinter on AIX when called Tcl command with
http://hg.python.org/cpython/rev/6a96c28f9474

New changeset 7b7bae546959 by Serhiy Storchaka in branch 'default':
Issue bpo-21951: Fixed a crash in Tkinter on AIX when called Tcl command with
http://hg.python.org/cpython/rev/7b7bae546959

serhiy-storchaka · 2014-09-11T08:24:27Z

Committed these two changes as separate patches.

vstinner · 2014-09-11T11:05:12Z

Committed these two changes as separate patches.

Thanks, it's clearer like that.

vstinner · 2014-09-11T11:07:28Z

test_tcl now pass on AIX:
http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2592/steps/test/logs/stdio

Thanks Serhiy for the fix.

serhiy-storchaka · 2014-09-11T11:35:35Z

Thank you Victor for great investigation of this issue.

DavidEdelsohn mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-tkinter type-crash A hard crash of the interpreter, possibly with a core dump labels Jul 10, 2014

serhiy-storchaka self-assigned this Sep 9, 2014

vstinner closed this as completed Sep 11, 2014

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tcl test change crashes AIX #66150

tcl test change crashes AIX #66150

DavidEdelsohn mannequin commented Jul 10, 2014

DavidEdelsohn mannequin commented Jul 10, 2014

serhiy-storchaka commented Jul 12, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

serhiy-storchaka commented Jul 13, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

serhiy-storchaka commented Jul 13, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

python-dev mannequin commented Jul 30, 2014

serhiy-storchaka commented Jul 30, 2014

vstinner commented Sep 4, 2014

serhiy-storchaka commented Sep 4, 2014

DavidEdelsohn mannequin commented Sep 4, 2014

python-dev mannequin commented Sep 4, 2014

vstinner commented Sep 4, 2014

DavidEdelsohn mannequin commented Sep 4, 2014

vstinner commented Sep 4, 2014

serhiy-storchaka commented Sep 5, 2014

serhiy-storchaka commented Sep 9, 2014

vstinner commented Sep 10, 2014

vstinner commented Sep 10, 2014

python-dev mannequin commented Sep 11, 2014

serhiy-storchaka commented Sep 11, 2014

vstinner commented Sep 11, 2014

vstinner commented Sep 11, 2014

serhiy-storchaka commented Sep 11, 2014

tcl test change crashes AIX #66150

tcl test change crashes AIX #66150

Comments

DavidEdelsohn mannequin commented Jul 10, 2014

DavidEdelsohn mannequin commented Jul 10, 2014

serhiy-storchaka commented Jul 12, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

serhiy-storchaka commented Jul 13, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

serhiy-storchaka commented Jul 13, 2014

DavidEdelsohn mannequin commented Jul 13, 2014

python-dev mannequin commented Jul 30, 2014

serhiy-storchaka commented Jul 30, 2014

vstinner commented Sep 4, 2014

serhiy-storchaka commented Sep 4, 2014

DavidEdelsohn mannequin commented Sep 4, 2014

python-dev mannequin commented Sep 4, 2014

vstinner commented Sep 4, 2014

DavidEdelsohn mannequin commented Sep 4, 2014

vstinner commented Sep 4, 2014

serhiy-storchaka commented Sep 5, 2014

serhiy-storchaka commented Sep 9, 2014

vstinner commented Sep 10, 2014

vstinner commented Sep 10, 2014

python-dev mannequin commented Sep 11, 2014

serhiy-storchaka commented Sep 11, 2014

vstinner commented Sep 11, 2014

vstinner commented Sep 11, 2014

serhiy-storchaka commented Sep 11, 2014