This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: struct.pack: Unable to pack more than 256 bytes at a time
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Dave Lotton, mark.dickinson
Priority: normal Keywords:

Created on 2019-11-18 17:14 by Dave Lotton, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg356886 - (view) Author: Dave Lotton (Dave Lotton) Date: 2019-11-18 17:14
Using struct.pack it is not possible (Python 3.6.8 and 2.7.15) to pack more than 256 bytes at a time.  

This seems like an arbitrarily small number, and seems to be inconsistent with the capabilities of the unpack function, which is able to unpack a larger number of bytes.

Below demonstrates the issue...

>>> # Create large data set to be packed
>>> data = range(0,512)

>>> # Demonstrate format string for packing
>>> '<%dB' % len(data)
'<512B'

>>> # Try to pack large data set
>>> packed = pack('<%dB' % len(data), *data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: ubyte format requires 0 <= number <= 255

>>> # Make data set <= 256 bytes
>>> data = range(0,256)
>>> packed = pack('<%dB' % len(data), *data)
>>> # Data successfully packed


>>> # Append another 256 bytes to packed data
>>> packed += pack('<%dB' % len(data), *data)

>>> # Unpack all 512 bytes
>>> unpacked = unpack('<%dB' % len(packed), packed)
>>> 
>>> unpacked
(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255)




The 256 byte limit on packing seems arbitrarily small and is inconsistent with the unpack function.  I'm wondering if there is a rationale for this limit, or it has simply been an oversight in the implementation.

I am using Mint Linux 19.2 64-bit (Ubuntu Bionic - based distro).  The problem is manifested on both Python 3.6.8 and 2.7.15 included in the distro.
msg356948 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2019-11-19 07:34
The error you're getting is because you're trying to pack a *value* that's larger than 255, not because you're trying to pack more than 256 bytes. Packing more than 256 bytes at a time works fine for me.

>>> packed = struct.pack("<512B", *[n//2 for n in range(512)])  # fine
>>> packed = struct.pack("2B", *[255, 256])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: ubyte format requires 0 <= number <= 255
msg356967 - (view) Author: Dave Lotton (Dave Lotton) Date: 2019-11-19 13:49
Mark, you are absolutely correct.  I'm an idiot.  

Focused on wrong thing.  Thank you.
History
Date User Action Args
2022-04-11 14:59:23adminsetgithub: 83018
2019-11-19 13:49:26Dave Lottonsetstatus: open -> closed
resolution: not a bug
messages: + msg356967

stage: resolved
2019-11-19 07:34:10mark.dickinsonsetnosy: + mark.dickinson
messages: + msg356948
2019-11-18 17:14:51Dave Lottoncreate