classification
Title: struct.pack() + numpy int raises SystemError
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.0, Python 3.1, Python 2.7, Python 2.6
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: docs@python Nosy List: georg.brandl, grubert, jvr, mark.dickinson, pitrou, vstinner, zaytsev
Priority: normal Keywords:

Created on 2008-03-10 07:39 by jvr, last changed 2017-03-07 19:19 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
issue2263-numpy.py grubert, 2009-01-29 01:05 produce test report.
Messages (8)
msg63435 - (view) Author: Just van Rossum (jvr) * (Python triager) Date: 2008-03-10 07:39
struct.pack() raises SystemError when fed a numpy integer in some cases. 
The following was run on MacOSX 10.4, little endian (I can only 
reproduce the error if I specify big endian for the struct format). Not 
sure if this could be a numpy bug.

Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import struct
>>> import numpy
>>> i = numpy.int16(1)
>>> struct.pack(">B", i)
__main__:1: DeprecationWarning: struct integer overflow masking is 
deprecated
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File 
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/struct.
py", line 63, in pack
    return o.pack(*args)
SystemError: /Users/ronald/r252/Objects/longobject.c:322: bad argument 
to internal function
>>> struct.pack(">H", i)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File 
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/struct.
py", line 63, in pack
    return o.pack(*args)
SystemError: /Users/ronald/r252/Objects/longobject.c:322: bad argument 
to internal function
>>> struct.pack(">h", i)
'\x00\x01'
>>> struct.pack(">b", i)
'\x01'
>>> struct.pack("B", i)
'\x01'
>>> struct.pack("h", i)
'\x01\x00'
>>> numpy.__version__
'1.0.4'
>>>
msg80736 - (view) Author: engelbert gruber (grubert) * Date: 2009-01-29 01:05
on ubuntu 8.04, Python 2.7a0 (trunk:69044) and numpy 1.2.1
("." is OK, "e" means SystemError)

* signed always works, longlong also.
* unsigned native works half of the time
* unsigned little/big endian never works
* the numpy type does seam to have any effect.

signed char          b   'int16'   .  <.  >.  'uint32'   .  <.  >. 
sys:1: DeprecationWarning: struct integer overflow masking is deprecated
unsigned char        B   'int16'   .  <e  >e  'uint32'   .  <e  >e 
signed short         h   'int16'   .  <.  >.  'uint32'   .  <.  >. 
unsigned short       H   'int16'   .  <e  >e  'uint32'   .  <e  >e 
signed int           i   'int16'   .  <.  >.  'uint32'   .  <.  >. 
unsigned int         I   'int16'   e  <e  >e  'uint32'   e  <e  >e 
signed long          l   'int16'   .  <.  >.  'uint32'   .  <.  >. 
unsigned long        L   'int16'   e  <e  >e  'uint32'   e  <e  >e 
signed long long     q   'int16'   .  <.  >.  'uint32'   .  <.  >. 
unsigned long long   Q   'int16'   .  <.  >.  'uint32'   .  <.  >.
msg81515 - (view) Author: engelbert gruber (grubert) * Date: 2009-02-09 23:37
in 2.7 svn _struct.c 

    formatdef native_table[] = {
      {'B',   sizeof(char),   0,      nu_ubyte,   np_ubyte},
    formatdef bigendian_table[]
      {'B',   1,      0,      nu_ubyte,   bp_uint},
    formatdef lilendian_table[]
        {'B',   1,      0,      nu_ubyte,   lp_uint},

np_ubyte calls get_long b/lp_uint call get_wrapped_ulong which calls
#define PyInt_Check(op) \
  PyType_FastSubclass((op)->ob_type, Py_TPFLAGS_INT_SUBCLASS)

but ob_type is set to BASE flags INT_SUBCLASS i snot set.
msg81782 - (view) Author: engelbert gruber (grubert) * Date: 2009-02-12 17:02
Including Py_TPFLAGS_INT_SUBCLASS in numpy's BASEFLAGS cures this.
This is an external problem then.
But I could not find any reference to Py_TPFLAGS_*_SUBCLASS in the
documentation, that is a python problem.
msg81783 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-12 17:19
Ok, so Python has to improve its C-API documentation, and numpy to fix
its int types :)
msg88034 - (view) Author: engelbert gruber (grubert) * Date: 2009-05-18 16:29
issue5476 has a problem with timedelta(microseconds = int32(36))

interestingly 0 to 35 work , if the patch for this issue2263 is applied.
msg173062 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-10-16 17:13
FYI Py_TPFLAGS_INT_SUBCLASS already not used in Python 3.x.
msg193181 - (view) Author: Yury V. Zaytsev (zaytsev) Date: 2013-07-16 16:10
As noted in issue5476, I've submitted a pull request for NumPy: ttps://github.com/numpy/numpy/pull/3526 .

I hope that this fixes this problem too: on Py2, I've added Py_TPFLAGS_INT_SUBCLASS, on Py3, NumPy doesn't inherit from int anymore, because it's not a fixed-width integer type.

I guess it makes sense to close this bug now.
History
Date User Action Args
2017-03-07 19:19:43serhiy.storchakasetstatus: pending -> closed
resolution: third party
stage: resolved
2015-07-25 06:42:17serhiy.storchakasetstatus: open -> pending
2014-01-07 13:09:41vstinnersetnosy: + vstinner
2013-07-16 16:10:10zaytsevsetnosy: + zaytsev
messages: + msg193181
2013-01-07 17:42:56serhiy.storchakasetnosy: - serhiy.storchaka
2012-10-16 17:13:35serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg173062
2010-10-29 10:07:21adminsetassignee: georg.brandl -> docs@python
2010-06-14 14:36:20mark.dickinsonsetnosy: + mark.dickinson
2009-05-18 16:29:50grubertsetmessages: + msg88034
2009-02-12 17:19:14pitrousetversions: + Python 2.6, Python 3.0, Python 3.1, - Python 2.5
nosy: + georg.brandl, pitrou
messages: + msg81783
priority: normal
assignee: georg.brandl
components: + Documentation, - Library (Lib)
2009-02-12 17:02:36grubertsetmessages: + msg81782
2009-02-09 23:37:39grubertsetmessages: + msg81515
2009-01-29 01:05:52grubertsetfiles: + issue2263-numpy.py
nosy: + grubert
messages: + msg80736
versions: + Python 2.7
2008-03-10 07:39:24jvrcreate