Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent behaviours with explicit and implicit inheritance from object #75464

Closed
gone mannequin opened this issue Aug 26, 2017 · 21 comments
Closed

Inconsistent behaviours with explicit and implicit inheritance from object #75464

gone mannequin opened this issue Aug 26, 2017 · 21 comments
Labels
docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error

Comments

@gone
Copy link
Mannequin

gone mannequin commented Aug 26, 2017

BPO 31283
Nosy @rhettinger, @stevendaprano, @bitdancer, @vedgar

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-08-26.19:49:09.195>
created_at = <Date 2017-08-26.09:14:22.961>
labels = ['type-bug', 'invalid', 'docs']
title = 'Inconsistent behaviours with explicit and implicit inheritance from object'
updated_at = <Date 2017-08-26.19:49:09.193>
user = 'https://bugs.python.org/gone'

bugs.python.org fields:

activity = <Date 2017-08-26.19:49:09.193>
actor = 'r.david.murray'
assignee = 'docs@python'
closed = True
closed_date = <Date 2017-08-26.19:49:09.195>
closer = 'r.david.murray'
components = ['Documentation']
creation = <Date 2017-08-26.09:14:22.961>
creator = 'gone'
dependencies = []
files = []
hgrepos = []
issue_num = 31283
keywords = []
message_count = 21.0
messages = ['300865', '300866', '300867', '300868', '300869', '300871', '300872', '300873', '300874', '300878', '300879', '300880', '300881', '300882', '300883', '300888', '300891', '300894', '300897', '300899', '300901']
nosy_count = 6.0
nosy_names = ['rhettinger', 'steven.daprano', 'r.david.murray', 'docs@python', 'veky', 'gone']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31283'
versions = ['Python 3.5', 'Python 3.6']

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

I discovered this while messing about with an unrelated idea, but the issue is that if you inherit explicitly from object, you get different behaviour than when you inherit implicitly. This is duplicated from my SO answer here: https://stackoverflow.com/questions/1238606/is-it-necessary-or-useful-to-inherit-from-pythons-object-in-python-3-x/45893772#45893772

If you explicitly inherit from object, what you are actually doing is inheriting from builtins.object regardless of what that points to at the time.

Therefore, I could have some (very wacky) module which overrides object for some reason. We'll call this first module "newobj.py":

import builtins

old_object = builtins.object  # otherwise cyclic dependencies

class new_object(old_object):

    def __init__(self, *args, **kwargs):
        super(new_object, self).__init__(*args, **kwargs)
        self.greeting = "Hello World!" 

builtins.object = new_object  #overrides the default object
Then in some other file ("klasses.py"):

class Greeter(object):
    pass

class NonGreeter:
    pass
Then in a third file (which we can actually run):

import newobj, klasses  # This order matters!

greeter = klasses.Greeter()
print(greeter.greeting)  # prints the greeting in the new __init__

non_greeter = NonGreeter()
print(non_greeter.greeting) # throws an attribute error
So you can see that, in the case where it is explicitly inheriting from object, we get a different behaviour than where you allow the implicit inheritance.

@gone gone mannequin added the type-bug An unexpected behavior, bug, or error label Aug 26, 2017
@vedgar
Copy link
Mannequin

vedgar mannequin commented Aug 26, 2017

To me, it seems like obvious behavior. What exactly do you think is inconsistent here?

Of course, the default inherit-class is "what's usually known as builtins.object", and it's always the same class. You should not be able to change it just by rebinding builtins.object to something else. Just like you shouldn't be able to change the top of exception hierarchy by rebinding builtins.BaseException, for example. :-)

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

I don't necessarily disagree with what you are saying- however, the received wisdom (as per my response to the SO question) is that

class SomeClass(object):
    pass

and

class SomeClass:
    pass

Should do the same thing. They evidently don't. It might be that the correct "solution" is that this is more properly documented that the two are not equivalent.

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

Out of curiosity, given that you can change every other point in the hierarchy by changing the binding of modules, what is your philosophical objection to being able to change the top of the hierarchy?

@vedgar
Copy link
Mannequin

vedgar mannequin commented Aug 26, 2017

Yes, they are obviously not equivalent if you execute

    object = int

before that. :-D

And (like I said) "received wisdom" is also that "except:" is equivalent to "except BaseException:", but of course it isn't equivalent if you execute

    BaseException = NameError

before.

Rebinding the names for fundamental objects in Python doesn't change the semantics of these objects. Python interpreter is an external thing: it doesn't exist "in-universe". It doesn't refer to these objects by names in some currently-executing-Python-instance namespace, it refers to them externally. And, mostly, so do we when explaining how Python works.

Let me give you an analogy: In the old days, you could rebind True and False, for example by saying

True, False = False, True

But of course, you would trivially understand that after such rebinding,

    if 6 > 4:
        print(1)
    else:
        print(2)

would print 1, not 2. Although we might say "(6>4) is True" while giving the semantics of "if" statement, we _don't_ mean "the object referred by the internal name 'True'". We mean the object _externally_ referred by the name 'True'.

Same as with your "hierarchy question": if you _give_ the explicit list of bases, you give it as a list of names. Those names can be rebound and if the class definition is executed in such a changed namespace, the bases will differ. But you can't rebind anything if you haven't given any explicit base names in the first place. It doesn't matter whether a class is "at the top" or somewhere else; it matters whether its name is explicitly given.

An illustration: Let's say you have executed

    class A: pass
    class B(A): pass

so now A refers to some class, and B refers to another class, inherited from A. If you now execute

    class C(B): pass  # *

you will have a new class C, inherited from B. Of course, if you now change B somehow, and then execute () again, you will have another class C, inherited from changed B - because () refers to B explicitly. But if you change _A_ somehow, and then execute () again, you will have C inherited from the same old B - because () doesn't refer to A explicitly. As you see, it doesn't matter whether A is "on the top" (in fact object is on the top, above A), what matters is whether the statement (*) that you re-execute in the changed namespace refers to it by name or not.

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

I accept what you are saying as consistent. Nevertheless, the documentation states that the two examples I gave should yield identical results. They do not, they perform different actions, albeit subtly.

Ergo, this is unexpected behaviour from a documentation point of view, if nothing else. If we are to maintain the fiction that these two things are equivalent, then they should yield the same result. If they are not equivalent, then we should state that this is the case explicitly.

@stevendaprano
Copy link
Member

I don't think this is a bug, I think it is standard behaviour which should be expected if you think about Python's execution model. If you inherit from object implicitly:

    class Spam: ...

then the interpreter gets to pick the base class, and it uses the genuine, builtin object base class. There's no name lookup, it is all built into the guts of the interpreter.

But if you specify the name of a base class:

    class Spam(foo): ...

then foo is looked up at runtime, regardless of whether you type "foo" or "int" or "str" or "object". If you have replaced or shadowed the builtin object with your own class, then you'll get *that* as the base for Spam, not the real built-in object base class.

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

Still not disagreeing with you- I just don't think that this is what the documentation implies.

@stevendaprano
Copy link
Member

the documentation states that the two examples I gave should yield identical results.

Got a link to the specific documentation that says this? And a suggested improvement?

@stevendaprano stevendaprano added the docs Documentation in the Doc dir label Aug 26, 2017
@vedgar
Copy link
Mannequin

vedgar mannequin commented Aug 26, 2017

It's not hard to find a link. https://docs.python.org/3.7/reference/compound_stmts.html#class-definitions

But trying to change that to incorporate what OP is asking is a wild goose chase. There are numerous instances when a documentation is referring to a builtin. E.g.

(print): All non-keyword arguments are converted to strings like str() does
(repr): this function makes an attempt to return a string that would yield an object with the same value when passed to eval()
(type): The isinstance() built-in function is recommended for testing the type of an object

Of course, all of these must be changed, since they don't work as advertised if you rebind str, eval or isinstance.

I claim this is nonsense. If anything, we should educate people that when documentation refers to "the builtin X", it doesn't mean "whatever is current referrent of builtins.X". Never. Only "__import__" kinda works this way (though not in builtins, but in globals), and that's mostly a historic accident.

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

Not at all- what you are talking about is obviously absurd. I am merely asserting that the statement in the docs you point to - that the two statements are equivalent - is untrue, the two statements are not equivalent in their behaviour.

@bitdancer
Copy link
Member

shadowadler, the documentation assumes *throughout* that you have not created any variable that shadows any standard Python entities. There is no other rational way to write the documentation. To change that policy would, as has been pointed out, require disclaimers in thousands of places in the documentation. That's not something that is going to be done :)

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

I really don't see that as a logical extension of what I am saying at all. Sure, shadowing builtins changes what they do, but if you're saying the syntax is equivalent then te effect of the shadowing should be consistent.

@bitdancer
Copy link
Member

I see I didn't specifically address your counter argument ("that would obviously be absurd"). Having thought it it some more, your are right, there *is* a difference between the examples you think it would be absurd to disclaim and your example here. In python, an entity is identified by an id (which is a memory address in CPython, but that's an implementation detail). A name is just a convenience label used to refer to that id. When you rebind a name, you change what id it points to, but any other piece of python that is already using the original id is not affected. But anything using an indirect reference through another object (such as builtins) *will* see the change.

Your argument, then, is that it is not documented that 'class x:' is using a direct reference to object rather than an indirect reference. Our argument is that this is obviously the way Python works (the interpreter itself refers directly to the fundamental entities such as the base of the exception hierarchy and object).

The number of places that would need to be changed to make this explicit is much smaller than I was thinking when I closed the issue, but I'm still not convinced it is something that needs to be explicitly documented. It is just part of the way Python works at a fundamental level, and because it is a *statement* that does not refer to a variable, it is intuitive that it is going to reference the original object, not whatever builtins.object is referring to.

We do say that explicit is better than implicit, but in this case we're talking about a fundamental part of the language, and the specification of how this works probably belongs in some overview section on statements. On the other hand, are there any examples *other* than class and except where this distinction matters?

@bitdancer bitdancer reopened this Aug 26, 2017
@bitdancer bitdancer removed the invalid label Aug 26, 2017
@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

You have put that much more precisely than I could have.

I'm not aware that it is an issie elsewhere, but given that I only ran into this today I may not the person best qualified to answer that question.

@vedgar
Copy link
Mannequin

vedgar mannequin commented Aug 26, 2017

Sorry, I fail to see the big difference.

Let's take print as an example:

All non-keyword arguments are converted to strings like str() does and written to the stream, separated by sep and followed by end. Both sep and end must be strings; they can also be None, which means to use the default values. If no objects are given, print() will just write end. The file argument must be an object with a write(string) method; if it is not present or None, sys.stdout will be used.

Is the above so different than writing:

    print(*args, file=f, sep=s, end=e)

is equivalent to

f.write(s.join(map(str, args))+e)

? In my head, no. It's just that sometimes we use Python, and sometimes English, to describe the semantics.

@gone
Copy link
Mannequin Author

gone mannequin commented Aug 26, 2017

The two phrases you present are significantly different- one draws an equivalence. The other does not. That in essence is what this is all about.

The difference between an indirect and direct reference in the class inheritance syntax is neither implied nor mentioned in the documentation. It led me to expect one behaviour and find another. That is the extent of my issue, and the fix seems to be to be small.

@vedgar
Copy link
Mannequin

vedgar mannequin commented Aug 26, 2017

I don't know whether the fix is small, since there is no fix that I see yet.

I'd just want to draw your attention to the fact that Python is extremely expressive language: almost nothing is "equivalent" to anything else, if you look hard enough.

Surely, in the docs, in various places it is written that some code is equivalent to some other code, where it's obvious that those are not completely equivalent in your sense.

E.g. "a_list += [1, 2, 3] is equivalent to a_list.extend([1, 2, 3])" (https://docs.python.org/3.5/faq/programming.html?highlight=equivalent#why-did-changing-list-y-also-change-list-x)

where it's obvious that the second one is an expression, while the first one is not. Also, the docs are full of "equivalents" to various idioms from _other programming languages_, where again it's obvious that total behavioral equivalence is not what's intended.

@vedgar
Copy link
Mannequin

vedgar mannequin commented Aug 26, 2017

On the other hand, are there any examples *other* than class and except where this distinction matters?

Of course. For example, "for" semantics mentions StopIteration. Of course it doesn't mean "whatever builtins.StopIteration currently refers to".

[And in a lot of places it would be possible to say that some builtin is implicit in the statement itself: e.g.

    while t:   is equivalent to   while bool(t):
    for a in b:     is equivalent to    for a in iter(b):
  • of course, the docs _don't_ currently say so, so maybe this occurance too should just be deleted. But I still think there are lots of places where docs refer to builtins directly.]

@rhettinger
Copy link
Contributor

I agree with the other commenters and recommend this be closed. There doesn't seem to be a a useful issue here. It seems more like a pedantic twisting of words that ignores how Python works (i.e. that you can specify a specific class to inherit from and that it is possible to either shadow or alter builtins). I don't see any magic here beyond changing a variable name to refer to a new object while previous references (or builtin references) to that name continue to refer to old object.

@bitdancer
Copy link
Member

OK, agreed. The general principle is: if you reference the name, it is looked up in the the builtins namespace at runtime (effectively an indirect reference). If the syntax doesn't explicitly mention the name, then it is going to be (the equivalent of) a direct reference, and changing builtins won't change it. So this is really just a specific example of how python namespacing works in general.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants