Consider this simple error:

>>> class foo(object):
...   def __init__(self,bar):
...    pass
... 
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() takes exactly 2 positional arguments (1 given)
>>> 

The problem is that if that "foo" call is through a variable (or
anything else that obscures which class I'm actually calling) there's no
good way to figure this from the traceback.

The last line should read

TypeError: foo.__init__() takes exactly 2 positional arguments (1 given)

or similar.

This is similar to bpo-2516.

Perhaps __qualname__ could be used in the traceback.

Talked with David Murray (r.david.murray) at @pycon2013 sprints - will try to address this.

The issue is not that easy to address - because PyFunctionObject is not available from PyEval_EvalCodeEx.

Another note, is that the issue of reporting only function name without class name is observed in many places, not just for example above.

Still, will try to make a patch that will address the issue.

Made a straightforward patch for this. Probably not that pretty, so suggestions are welcome.

Note that some function names will become pretty long in exceptions:

>>> class A:
...     def __init__(self):
...         def f():
...             pass
...         f(1,2,3)
...
>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in __init__
TypeError: A.__init__.<locals>.f() takes 0 positional arguments but 3 were given

Passing UTs from lastest snapshot.

I wonder if you could add this to the new code in http://bugs.python.org/issue17911 which I'm hoping to commit this week.

As long as you think It fits into your issue, I'm ok with adding this patch to it (: it's different files, so no conflicts. Should I just add this patch to your ticket?

Please do.

bpo-17911 was submitted. I pulled latest updates and rechecked that "./python -m test -uall" passing with the same patch.

Addrressed feedback: splitted long line in multiple in tests.

Ok, so this is an API and ABI change. I'm going to do a variant without that downside.

And herewith.

Forgot docs - I'll do before committing, but not worried about review of them so much.

Moving target version to 3.6 (since we're discussing adding a new public C API).

This is an interesting problem, as seeing an expanding API like this (PyEval_EvalCode -> PyEval_EvalCodeEx -> PyEval_EvalCodeEx2) suggests to me that we actually have a missing abstraction layer trying to get out.

My first reaction was to say "Hey, let's just make the eval loop function aware, and add PyEval_EvalFunction", but that creates a conceptual dependency loop we don't really want.

So I asked myself a different question: why aren't we storing the qualname as a code object attribute, even though it's a known constant at compile time?

Looking at PEP-3155 (https://www.python.org/dev/peps/pep-3155/ ) it doesn't look like the question came up, and I don't recall it coming up either.

However, looking at https://hg.python.org/cpython/file/default/Python/compile.c I also don't see any reason why we *couldn't* provide a qualname field on code objects, such that f.__code__.co_qualname would give the same answer as f.__qualname__. The compiler already knows the qualname at compile time, we're just currently storing it as a separate constant that gets composed together with the code object at runtime.

My suspicion is that the reason this *wasn't* done is because it's a slightly more intrusive change to the code generation pipeline, but I currently suspect following up with that increase in compiler complexity would be a better option than making the PyEval_* API more complicated.

Moving this back to patch review while we consider the alternative approach of moving qualname storage into code objects, rather than continuing to transport qualnames separately.

Code objects currently have no mutable fields.

So what are you planning to do about things like:

Foo = namedtuple('Foo', 'x y')
def frob(self):
    return self.x + self.y
# probably actually done via a @member(Foo) decorator
# so adding more code here is not a problem.
Foo.frob.__qualname__ = 'Foo.frob'
Foo.frob = frob
del frob

I made a minimal gist of my motivating code:

https://gist.github.com/o11c/ce0c2ff74b87ea71ad46

The reason I didn't put __qualname__ on code objects is that code objects don't have a __module__ either. That information, up to now, belongs on the module.

Conceptually, a code object could very well be used by multiple functions living in different modules. Now, in practice, I'm not sure that happens (except when constructing function objects on your own, which is not a terribly supported usecase I think).

Also, Ben makes a very point about being able to change a __qualname__. In particular, the user should *still* be able to change a function's __qualname__ even if it's technically stored on the code object.

Correcting myself:

That information, up to now, belongs on the module.

on the function, of course.

Note to self: this is about changing the name in an exception message, not in the back-trace of call sites that triggered an exception :)

With __qualname__ being mutable, I agree that adding __code__.co_qualname wouldn't be a substitute for that. Instead, similar to the relationship between __name__ and __code__.co_name, they would start out the same, but the function attribute may change later (e.g. through the use of functools.wraps).

However, that also highlights why we need to use the compile time qualname in the traceback, rather than the runtime one: because we currently use the compile time name from the code object rather than the runtime name from the function.

A test case to demonstrate that:

>>> def f():
...     1/0
... 
>>> import functools
>>> @functools.wraps(f)
... def g():
...     return f()
... 
>>> g()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in g
  File "<stdin>", line 2, in f
ZeroDivisionError: division by zero
>>> g.__qualname__
'f'
>>> g.__name__
'f'

Note that "g" still appears in the relevant traceback line, even though both __name__ and __qualname__ have been updated to refer to "f". For a traceback we want to know where the source of the function actually lives, not where it claims to be from for human introspection purposes.

As far as the comparison to __module__ goes, I think that's a different case - we couldn't add that to the code object even if we wanted to, because the compiler doesn't know the module name, it only knows the file name. It's also possible to take an already compiled python file, move it to a different directory, and have the module name change due to the new location in the package hierarchy. __qualname__ is different as, like __name__, it's entirely local to the module and hence available to the compiler at compile time.

Hmm, I'd made the same mistake Martin did - I was looking at the tracebacks moreso than at the exception message itself. Looking at the argument unpacking exception message in the context of the test case above, that also uses __code__.co_name rather than the runtime function name:

>>> g(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: g() takes 0 positional arguments but 1 was given

In this case though, I think that's arguably a problem, as we probably want error messages (unlike tracebacks) to include the "intended for human consumption" name, but they currently don't, they expose the name of the wrapper function if it's actually constraining its arguments rather than passing them through for subsequent validation:

>>> def make_wrapper(f):
...     @functools.wraps(f)
...     def wrapper():
...         return f()
...     return wrapper
... 
>>> @make_wrapper
... def g():
...     return f()
... 
>>> g(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: wrapper() takes 0 positional arguments but 1 was given

That makes me more inclined towards a solution like Daniil's patch, but the growing signature of PyEval_Code* still bothers me. Perhaps we could collect the various runtime state arguments up into a "PyEvalScope" struct and make the new API PyEval_EvalCodeInScope(PyObject *code, PyObject *scope)?

Here are some options.

a) Don't make the new thing public - instead export within Python.exe the existing private symbol _...withNames. Pros: no change to public API. Cons: extension modules using the public API cannot make these sorts of errors clearer.

b) Add a new API. Either narrow (add the parameter) or big (add a struct). Pros: everyone treated equally. Cons: More API surface area, and more complex calling convention.

c) use symbol versioning to change the existing API. Cons: more complex build-time detection for users. Pros: API surface area held constant.

d) Don't fix it. :)

I don't have a particular preference, though the struct thing is a wash IMO - it saves adding a serial to the API, at the cost of having a versioned datastructure which needs an embedded serial number. [Except perhaps that you can use symbol versioning to deal with evolutions of the struct - but thats fairly complex to carry off well, and I think this would be the first example w/in Python].

I think a) is worth doing regardless - in many cases, third party libraries will be dealing with function, generator or coroutine objects, rather than with code objects directly, so they'll benefit even if the updated API remains private to the interpreter.

So let's narrow the scope of this particular issue to just that, and defer the idea of a new public API for the time being.

See also bpo-4322.

A few weeks ago I reported bpo-27389 which looks to be a duplicate of this issue. Has any progress been made on this issue?

This seems to have been fixed by now, on 3.11 I get this:

>>> class foo:
...   def __init__(self, bar):
...      pass
... 
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo.__init__() missing 1 required positional argument: 'bar'