I agree it's not the best error message. What's happening is that these types (list, tuple, etc.) do not implement __format__, so object.__format__ is used. It returns str(self). Then the resulting string is formatted with the given format_spec. Since str does not support the 'd' format type, the error you see is raised.

I'm open to suggestions on how to improve this, but I don't see how it's possible given what str.__format__ knows when it generates the error.

also strange(unobvious) behavior:
>>> '{0:.3s}'.format((i for i in (1, 2, 3)))
'<ge'
>>> '{0:.3s}'.format(range(10))
'ran'
>>> '{0:.3s}'.format(None)
'Non'
>>>

it would be better to print an error:
ValueError: Unknown format code 's' for object of type 'generator'

like in this:
>>> '{0:d}'.format(4.5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'float'
>>>

in the documentation there is nothing about it

No, it wouldn't. I expect

"{}".format(x)

to produce something for an arbitrary x. Breaking that would break a fundamental Python contract.

Improving the error message for 'd' is more possible. Perhaps "the format code 'd' is not implemented by objects of type <type>"?

Oh, and when you say there is nothing in the documentation about the 's' case for arbitrary objects, it is made clear in various places that every object has an str, which defaults to its repr if it has no specific __str__. Combine that with the description of what happens when you use a fixed field length for 's', and you get the results you see. There should be nothing surprising about this to anyone who has read the tutorial, I think. (But specific suggestions for improving the docs are always welcome.)

R. David Murray wrote:

it is made clear in various places that every object has an str

here:
http://docs.python.org/py3k/library/string.html#format-specification-mini-language

3rd paragraph:
"A general convention is that an empty format string ("") produces the same result as if you had called str() on the value. A non-empty format string typically modifies the result."

"an empty format string ("")" what does it mean ?

"".format(value) or "{}".format(value) or "{0}".format(value) ?

also here:
http://docs.python.org/py3k/library/string.html#format-examples

there is no example with list or tuple to know exactly how they are formatted

"an empty format string" is exactly what I was talking about. Putting nothing between the {}'s is an empty format string. I can't think of any way to make that wording clearer.

The format docs should not contains examples of the repr of all possible python objects. The examples of what tuples and lists and dicts &c look like are shown in the docs for those objects.

R. David Murray wrote:

Putting nothing between the {}'s is an empty format string.

this is an empty replacement field

here:
http://docs.python.org/py3k/library/string.html#format-string-syntax

the definition of format string:
"Format strings contain “replacement fields” surrounded by curly braces {}. Anything that is not contained in braces is considered literal text, which is copied unchanged to the output."

"The grammar for a replacement field is as follows:"
replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] "}"

Good point. That should be fixed. It should be "empty format specification".

Changing to a documentation issue.

Doc patch attached to make sure correct. Should {} be quoted?

Eric, do you want to close off the idea of changing :d errors, or switch back after the doc fix?

I don't think "{}" is the correct way to document this. These all have an empty format specifier:

"{}".format(foo)
"{:}".format(foo)
"{0}".format(foo)
"{0:}".format(foo)
"{name}".format(name=foo)
format(foo, "")
format(foo)

That is, they all call foo.__format__(""). If foo.__format__ (well, really type(foo).__format__) doesn't exist, then object.__format__(foo, "") gets called. It's object.__format__ that's checking for the empty format string, and if so it returns str(foo).

What would you suggest changing the ':d' error message to, for objects that don't support a format type of 'd'? This makes sense to me:

>>> format('', 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

The problem, if there is one, is:
>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

The problem is that the str that's producing this error doesn't know that it exists because object.__format__ returned str([]).

Oh, I see. Yes, that is a problem.

object.__format__ knows the type of the object it was called on, right? Couldn't it catch the error and re-raise it with the correct type? (If the type isn't str, of course, we don't want to get into an infinite recursion.)

Oh, never mind that comment about recursion, I wasn't thinking it through.

OK, the example of an empty format spec should be dropped. Let people figure it out ;-).

>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

One possibility is to give (str of) the object instead of the type:

ValueError: Unknown format code 'd' for object '[]'

The downside is a long message for long strings. It would need to be limited (as is done in test error reports).

While looking at object.__format__, I recall that we've already addressed this, sort of. For a different reason, this is already deprecated in 3.3 and will become an error in 3.4. See issues 9856 and 7994.

$ ./python -Wd
Python 3.3.0a0 (default:40e1be1e0707, Jan 15 2012, 00:58:51) 
[GCC 4.6.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> format([], 'd')
__main__:1: DeprecationWarning: object.__format__ with a non-empty format string is deprecated
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'
[67288 refs]
>>> 

We could still have object.__format__ catch and re-throw the ValueError with a better message. I'd have to think it through if we could catch all ValueErrors, or if it's possible for another ValueError to be thrown and we'd only catch and rethrow this specific ValueError.

But since this is deprecated, I'm not sure it's worth the hassle. I'd advocate closing this issue as "won't fix".

So the error is going to be something about the source type not supporting '__format__'?

That change will also address the OP's concern about truncated reprs when a fixed string length is specified, so I agree that the title issue can be closed. Terry's patch with the ("{}") removed should be committed, though.

The error message will be: "non-empty format string passed to object.__format__".

I agree with your comment about Terry's patch.

Looking further, I noticed that 'string' needed to be changed to 'specification' in the following sentence also. Then I decided that the preceding sentence

"Most built-in types implement the following options for format specifications, although some of the formatting options are only supported by the numeric types."

should really follow the one about non-empty format specs. This positioning should make it more obvious that most of the options affect the string representation of the object after, not before, the string is produced, and are therefore applicable to all objects and not just string and number objects. I also propose to modify it so it is shorter and no longer contradictory, to read

"Most built-in types implement various options for such modifications, although some are only supported by the numeric types."

Further on, under "The available string presentation types are:"
I think "'s' String format. This is the default type for strings and may be omitted." should have 'and other non-numeric types ' inserted after strings. New patch i13790b.diff attached

The point of these additional changes is to make it clearer that the default formatting of non-number, non-string objects is to call str() and then apply the options to the resulting string. That makes something like
>>> format(range(5), '-^20s') # same with object.__format__(), 3.3.0a0
'----range(0, 5)-----'
predictable and comprehensible.

I agree with not making a temporary change (but see below ;-).

But it seems that the 3.4 message should at least be
"numeric format string passed to object.__format__" or
"format string with number-only options passed to object.__format__" or
"object.__format__ cannot handle number-only options"
as string formats work fine and, I presume, are not deprecated (?).

However, if the new ValueError message did not specify object.__format__ (which could still be confusing, even if more accurate), the change could be make now. For instance
'Numeric option 'd' for non-number object'.
It would not really matter if it is later raised in object.__format__ instead of str.__format__. I believe *all* of the format codes 'unknown' to str (and by extension, by default, to all other non-number types) *are* number codes.

Serhiy: I'm not sure what you're saying. At the point that str.format() is producing its error message, it doesn't know as much as %-formatting does about the original arguments, so it can't produce a similar message.

Serhiy: I'm not sure what you're saying. At the point that str.format() is producing its error message, it doesn't know as much as %-formatting does about the original arguments, so it can't produce a similar message.

I'm surprised that the code of the classic and the modern formatting is
so different. Looking deeper, I saw that the issue will go away in 3.4.
I agree with you in msg151728.

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> '{0:d}'.format('a')
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    '{0:d}'.format('a')
ValueError: Unknown format code 'd' for object of type 'str'

Nothing appears to have changed despite "the issue will go away in 3.4" in msg164373. What should have happened here?

I believe that comment was referring to the subject of this bug:

$ ./python
Python 3.4.1+ (3.4:bec6f18dd636, Jun 12 2014, 20:23:30)
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__
>>> format((), 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__
>>> format({}, 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__
>>> format(set(), 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: non-empty format string passed to object.__format__

With the possible exception of listing the type in this error message, I think these are all fine.

I'm not sure what you'd expect format('a', 'd') to produce other than the error you're seeing. 'd' is in fact an unknown "format code" for str.

>>> format('a', 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

Python 2.7.7 is still printing.

>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'
>>>

Yes, the deprecation in 3.3 did not apply to 2.7.

From the examples in msg220401, bpo-28385 changed it to print the object type in the message.

>>> format([], 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to list.__format__
>>> format((), 'd')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported format string passed to tuple.__format__

Would the change left on this issue be to create a PR for Terry's documetation patch?

Thanks!

In msg151730, R. David Murry said "Terry's [first] patch with the ("{}") removed should be committed, though."
In msg151738, Eric V. Smith said "I agree with your comment about Terry's patch."

My second patch removed "{}" but also made more text changes, explained in msg151757. Someone should re-review

PR-18690 makes the approved change of 'string' to 'specification'. After merging, I will re-review the other changes in i13790b.diff and possibly make another PR for review.

New changeset 916895f by Terry Jan Reedy in branch 'master':
bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
916895f

New changeset 5157506 by Miss Islington (bot) in branch '3.7':
bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
5157506

New changeset 445152e by Miss Islington (bot) in branch '3.8':
bpo-13790: Change 'string' to 'specification' in format doc (GH-18690)
445152e

This seems complete.