classification
Title: format error messages should provide context information
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.smith, ezio.melotti, iritkatriel, r.david.murray, sobolevn
Priority: normal Keywords: patch

Created on 2014-02-05 23:23 by r.david.murray, last changed 2021-09-13 12:04 by sobolevn.

Pull Requests
URL Status Linked Edit
PR 28310 open sobolevn, 2021-09-13 12:00
Messages (5)
msg210351 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-02-05 23:23
Consider the following:

   '{run_time:%H:%M:%S}, ,COM,DA{id},"{title:.43}",{id},{length:%M:%S}'.format(**mydict)

The error message I got was:

   Invalid format specifier

The problem turned out to be that the value of the 'length' key was an integer instead of a datetime.time(), but it sure wasn't easy to figure out which bit of the format string or which variable was the problem.

It would be nice for the format error message to include the pattern that it is parsing when it hits the error.  The type of the value being substituted would also be nice.  Perhaps something like:

   The format specifier in {length:%HH:%MM} is not valid for type int()
msg210354 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2014-02-06 00:32
That would be a great improvement. It's in Python/formatter_unicode.c, line 245, in parse_internal_render_format_spec().

That code knows about the format spec, but not the type being formatted. That would be easy enough to pass in.

This fix would only work for the built-in types: int, float, and str, I think. Maybe complex. But that's probably good enough.
msg210359 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2014-02-06 01:08
int, float, str, and complex are the types formatted by that code.

Notice that Decimal already has a better message:

>>> format(Decimal(42), 'tx')
Traceback (most recent call last):
  ...
ValueError: Invalid format specifier: tx

>>> format(42, 'tx')
Traceback (most recent call last):
  ...
ValueError: Invalid conversion specification

But, look at this:
>>> format(3, '--')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code '-' for object of type 'int'

This is generated in unknown_presentation_type, also in formatter_unicode.c. It almost does what you want, but just handles the presentation type, not the whole format specifier.

Error handling could be cleaned up in that module. I'd say that the string should be:
"<specific error> with format specifier <specifier> for object of type '<type>'"

<specific error> might be "Unknown presentation type '-'", or "Cannot specify ','".

I think that would require some major surgery to the code, but would be worth it.

Note that in your original example, you want the error to contain "{length:%HH:%MM}". By the time the error is detected, the only thing the code knows is the format specifier "%HH:%MM". It doesn't know the "length" part. The error is basically in int.__format__. By the time that gets called, the format specifier has already been extracted and the argument selection (by indexing, by name, including attribute access) has already taken place.
msg401606 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-09-10 18:14
Reproduced on 3.11:

>>> dd = {'length': 12, 'id': 4, 'title': "t", 'run_time': datetime.time()}
>>> '{run_time:%H:%M:%S}, ,COM,DA{id},"{title:.43}",{id},{length:%M:%S}'.format(**dd)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier
msg401700 - (view) Author: Nikita Sobolev (sobolevn) * Date: 2021-09-13 12:04
> I think that would require some major surgery to the code, but would be worth it.

I've decided to take an easy path: https://github.com/python/cpython/pull/28310

```
PyErr_Format(PyExc_ValueError,
                     "Invalid format specifier: '%s' for object of type '%s'",
                     PyUnicode_AsUTF8AndSize(format_spec, NULL),
                     Py_TYPE(obj)->tp_name);
```

This worked for me as the starting point. It covered: int, float, complex, and str types.

> Note that in your original example, you want the error to contain "{length:%HH:%MM}". By the time the error is detected, the only thing the code knows is the format specifier "%HH:%MM". It doesn't know the "length" part. 

I guess it has changed since the 2014.

I would love to hear any feedback on my proposal and improve it to the level when it can be merged :)
History
Date User Action Args
2021-09-13 12:04:47sobolevnsetmessages: + msg401700
2021-09-13 12:00:52sobolevnsetkeywords: + patch
nosy: + sobolevn

pull_requests: + pull_request26723
stage: test needed -> patch review
2021-09-10 18:14:27iritkatrielsetnosy: + iritkatriel

messages: + msg401606
versions: + Python 3.11
2014-02-06 01:08:15eric.smithsetmessages: + msg210359
2014-02-06 00:32:46eric.smithsetmessages: + msg210354
2014-02-05 23:36:05ezio.melottisetnosy: + ezio.melotti

type: behavior
stage: test needed
2014-02-05 23:23:15r.david.murraycreate