int, float, str, and complex are the types formatted by that code.

Notice that Decimal already has a better message:

>>> format(Decimal(42), 'tx')
Traceback (most recent call last):
  ...
ValueError: Invalid format specifier: tx

>>> format(42, 'tx')
Traceback (most recent call last):
  ...
ValueError: Invalid conversion specification

But, look at this:
>>> format(3, '--')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code '-' for object of type 'int'

This is generated in unknown_presentation_type, also in formatter_unicode.c. It almost does what you want, but just handles the presentation type, not the whole format specifier.

Error handling could be cleaned up in that module. I'd say that the string should be:
"<specific error> with format specifier <specifier> for object of type '<type>'"

<specific error> might be "Unknown presentation type '-'", or "Cannot specify ','".

I think that would require some major surgery to the code, but would be worth it.

Note that in your original example, you want the error to contain "{length:%HH:%MM}". By the time the error is detected, the only thing the code knows is the format specifier "%HH:%MM". It doesn't know the "length" part. The error is basically in int.__format__. By the time that gets called, the format specifier has already been extracted and the argument selection (by indexing, by name, including attribute access) has already taken place.