This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Improve struct.pack out of range error messages
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: corona10, mark.dickinson, meador.inge, sobolevn, steven.daprano, terry.reedy
Priority: normal Keywords: patch

Created on 2021-08-28 01:02 by steven.daprano, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 28178 merged sobolevn, 2021-09-05 17:36
Messages (6)
msg400452 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2021-08-28 01:02
Packing errors using struct in 3.9 seem to be unnecessarily obfuscated to me.

    >>> import struct
    >>> struct.pack('H', 70000)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    struct.error: ushort format requires 0 <= number <= (0x7fff * 2 + 1)


Why "0x7fff * 2 + 1"? Why not the more straightforward "0xffff" or 65536? (I have a slight preference for hex.)

Compare that to:

    >>> struct.pack('I', 4300000000)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    struct.error: 'I' format requires 0 <= number <= 4294967295

which at least gives the actual value, but it would perhaps be a bit more useful in hex 0xffffffff.

For the long-long format, the error message just gives up:

    >>> struct.pack('Q', 2**65)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    struct.error: argument out of range

Could be improved by:

    'Q' format requires 0 <= number <= 0xffff_ffff_ffff_ffff
msg401020 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-09-04 00:27
I agree, including use of hex, if possible, for unsigned (non-negative) values.

Grepping 'format requires' returns
F:\dev\3x\Modules\_struct.c: 365:             "'%c' format requires 0 <= number <= %zu",
F:\dev\3x\Modules\_struct.c: 371:             "'%c' format requires %zd <= number <= %zd",
F:\dev\3x\Modules\_struct.c: 550:                         "byte format requires -128 <= number <= 127");
F:\dev\3x\Modules\_struct.c: 565:                         "ubyte format requires 0 <= number <= 255");
F:\dev\3x\Modules\_struct.c: 577:                         "char format requires a bytes object of length 1");
F:\dev\3x\Modules\_struct.c: 593:                         "short format requires " Py_STRINGIFY(SHRT_MIN)
F:\dev\3x\Modules\_struct.c: 611:                         "ushort format requires 0 <= number <= "
                                                           Py_STRINGIFY(USHRT_MAX));

I believe l365 is the source for the 2nd example.  AFAIK, 'zu' is not valid for either C printf or Python % formating.
Lines 611 and 612 are the source for the 1st example.  From comments before line 365, there can be issues with lefts shifts, but '0xffff', '0xffff_ffff', and '0xffff_ffff_ffff_ffff' could be hard-coded strings that would cover all 'normal' systems.

Grepping "argument out of range" returns
F:\dev\3x\Modules\_struct.c: 168:                             "argument out of range");
F:\dev\3x\Modules\_struct.c: 192:                             "argument out of range");
F:\dev\3x\Modules\_struct.c: 215:                             "argument out of range");
F:\dev\3x\Modules\_struct.c: 238:                             "argument out of range");
F:\dev\3x\Modules\_struct.c: 261:                             "argument out of range");
F:\dev\3x\Modules\_struct.c: 284:                             "argument out of range");

It is nnclear to me without more reading why some codes lead to this less helpful message.
msg401043 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2021-09-04 12:53
I would like to work on this if nobody else has already started :)
msg401053 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-09-04 18:26
Go ahead, with the understanding that there is no guaranteed acceptance of the change, even with a good patch.  There may be reasons for the status quo that neither Steven nor I are aware of.  I don't think that either of the listed experts, added as nosy, are very active.  In any case,  revised or new tests will be needed.
msg401251 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-09-07 12:18
New changeset 8ca6b61e3fd7f1e2876126cee82da8d812c8462f by Nikita Sobolev in branch 'main':
bpo-45034: Fix how upper limit is formatted for `struct.pack("H", ...)` (GH-28178)
https://github.com/python/cpython/commit/8ca6b61e3fd7f1e2876126cee82da8d812c8462f
msg401254 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-09-07 12:25
The out-of-range error messages for unsigned short and short have been fixed, thanks to Nikita Sobolev. They resulted from a rather odd use of the Py_STRINGIFY macro, which meant that not only were the messages obscure, but they differed from system to system: e.g., on my machine I get: "struct.error: ushort format requires 0 <= number <= (32767 *2 +1)", and "struct.error: short format requires (-32767 -1) <= number <= 32767", complete with the weird spacing.

There's still room for normalising the other messages and/or converting the limits to hex if anyone wants to provide a PR.

I'm unsure about using hex universally: while `0xffffffffffffffff` (or even better, `0xffff_ffff_ffff_ffff`) is definitely more useful than `18446744073709551615`, I think I'd still find `-128` more immediately readable than `-0x80`.
History
Date User Action Args
2022-04-11 14:59:49adminsetgithub: 89197
2021-09-07 12:25:15mark.dickinsonsetmessages: + msg401254
2021-09-07 12:18:50mark.dickinsonsetmessages: + msg401251
2021-09-05 17:36:04sobolevnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26605
2021-09-04 18:26:46terry.reedysetnosy: + mark.dickinson, meador.inge
messages: + msg401053
2021-09-04 12:53:30sobolevnsetnosy: + sobolevn
messages: + msg401043
2021-09-04 12:19:01corona10setnosy: + corona10
2021-09-04 00:27:10terry.reedysetnosy: + terry.reedy
messages: + msg401020
2021-08-28 01:02:05steven.dapranocreate