classification
Title: UnboundLocalError as a result of except statement variable re-assignment
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, christian.heimes, oleksandr.suvorov, pablogsal
Priority: normal Keywords:

Created on 2020-05-22 10:59 by oleksandr.suvorov, last changed 2020-05-22 20:17 by pablogsal. This issue is now closed.

Messages (13)
msg369588 - (view) Author: Oleksandr Suvorov (oleksandr.suvorov) Date: 2020-05-22 10:59
def foo(exc):
   try:
       1/0
   except Exception as exc:
       ...
   finally:
       return exc

foo(1)

Executing the following code results in UnboundLocalError, while exc has been defined and passed to a function explicitly.

I think this is something similar to python2 issue when list comprehensions where overwriting local variable values, expected behaviour in here is that foo(1) returns 1
msg369589 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-05-22 11:04
This is expected behaviour and happens because we need to clean the exception variable outside the `except` block to avoid reference cycles. If you need the variable later, you need to do an assignment:

   try:
       1/0
   except Exception as exc:
       e = exc
   finally:
       return e
msg369590 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-05-22 11:08
This is the equivalent construct that we compile it to:

https://github.com/python/cpython/blob/master/Python/compile.c#L3102-L3110
msg369591 - (view) Author: Oleksandr Suvorov (oleksandr.suvorov) Date: 2020-05-22 11:10
But then why I still can access this variable? Shouldn't it then be
resulting in NameError as it's undefined variable at this point of time?
I think UnboundError creates more confusing here, as it's either the
variable should exist and have a value or it should be unknown and result
in NameError at this point.

On Fri, May 22, 2020 at 1:08 PM Pablo Galindo Salgado <
report@bugs.python.org> wrote:

>
> Change by Pablo Galindo Salgado <pablogsal@gmail.com>:
>
>
> ----------
> nosy: +Mark.Shannon
> resolution:  -> not a bug
> stage:  -> resolved
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40728>
> _______________________________________
>
msg369592 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-05-22 11:15
>But then why I still can access this variable? Shouldn't it then be
resulting in NameError as it's undefined variable at this point of time?

I don't think so, this is the same as if you do:

def f(exc):
   del exc
   return exc

>>> f(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lel.py", line 3, in f
    return exc
UnboundLocalError: local variable 'exc' referenced before assignment
msg369593 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-05-22 11:15
UnboundLocalError is a subclass of NameError. You are getting a more specific error here.

>>> UnboundLocalError.__mro__
(<class 'UnboundLocalError'>, <class 'NameError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>)
msg369594 - (view) Author: Oleksandr Suvorov (oleksandr.suvorov) Date: 2020-05-22 11:19
but if exc variable is only available in except block why then it shadows
the function variable name?

On Fri, May 22, 2020 at 1:15 PM Christian Heimes <report@bugs.python.org>
wrote:

>
> Christian Heimes <lists@cheimes.de> added the comment:
>
> UnboundLocalError is a subclass of NameError. You are getting a more
> specific error here.
>
> >>> UnboundLocalError.__mro__
> (<class 'UnboundLocalError'>, <class 'NameError'>, <class 'Exception'>,
> <class 'BaseException'>, <class 'object'>)
>
> ----------
> nosy: +christian.heimes
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40728>
> _______________________________________
>
msg369595 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-05-22 11:19
> as it's either the
variable should exist and have a value or it should be unknown and result
in NameError at this point.

One thing is the symbol and another thing is its value. The function is always aware of the existence of the symbol. For instance:

def f(x):
   del x
   print(x)

>>> print(f.__code__.co_varnames)
('x',)

But when the function tries to use it, it has no value associated to it because del x removed it, so it emits an UnboundLocalError because there is no value.

raceback (most recent call last):
  File "lel.py", line 7, in <module>
    f(3)
  File "lel.py", line 3, in f
    print(x)
UnboundLocalError: local variable 'x' referenced before assignment

A NameError will occur if the symbol is unknown to the function.
msg369596 - (view) Author: Oleksandr Suvorov (oleksandr.suvorov) Date: 2020-05-22 11:20
What I mean is why would it need to shadow function variable name if after
except block it's always unbound, what is the reason and why this is
expected?

On Fri, May 22, 2020 at 1:19 PM Oleksandr Suvorov <report@bugs.python.org>
wrote:

>
> Oleksandr Suvorov <susliko@gmail.com> added the comment:
>
> but if exc variable is only available in except block why then it shadows
> the function variable name?
>
> On Fri, May 22, 2020 at 1:15 PM Christian Heimes <report@bugs.python.org>
> wrote:
>
> >
> > Christian Heimes <lists@cheimes.de> added the comment:
> >
> > UnboundLocalError is a subclass of NameError. You are getting a more
> > specific error here.
> >
> > >>> UnboundLocalError.__mro__
> > (<class 'UnboundLocalError'>, <class 'NameError'>, <class 'Exception'>,
> > <class 'BaseException'>, <class 'object'>)
> >
> > ----------
> > nosy: +christian.heimes
> >
> > _______________________________________
> > Python tracker <report@bugs.python.org>
> > <https://bugs.python.org/issue40728>
> > _______________________________________
> >
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40728>
> _______________________________________
>
msg369597 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-05-22 11:21
> but if exc variable is only available in except block why then it shadows
the function variable name?

Because the except block does not create a new scope and therefore when you do 'except as x' it does an assignment to x in the same scope as the function.
msg369612 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2020-05-22 15:20
Oleksandr,
Are you saying that the current behaviour is incorrect, in the sense that it disagrees with the documentation?
Or are you saying that the current behaviour is undesirable?
msg369631 - (view) Author: Oleksandr Suvorov (oleksandr.suvorov) Date: 2020-05-22 18:41
Mark, 
I'm not sure if this is even documented, 
I find this a bit inconsistent when compared to the behavior of other statements, 
E.g.: 
* list expressions do have their scope and do not overwrite local variables
* with statement does overwrite the variable value but makes it available even after with block 
* except overwrites variable value, but then the value gets cleaned and no longer accessible. 

I do understand why this happens, but I find this undesirable. At least some consistency will be nice to have, or this should be documented. Otherwise, it confuses.
msg369638 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-05-22 20:17
I don't think the examples you mention is proof of inconsistent behaviour:

> * list expressions do have their scope and do not overwrite local variables

comprehensions have their own scope in Python 3 and this is documented. General blocks (like normal loops, exceptions, with statements...etc don't have their own scope).

* with statement does overwrite the variable value but makes it available even after with block 

Yeah, but with statements do not have the problem of needing to break reference cycles in the exception names, so this does not apply. The same for any other "target" variable such as loop assignment (for x in ...).

* except overwrites variable value, but then the value gets cleaned and no longer accessible.

This is a consequence of the fact that when you do 'as varname' you are voluntary overwriting the previous value, therefore loosing whatever was there before. The semantics here are that the variable is not accessible outside the except block, which is unique to except blocks but this is needed to avoid circular references.
History
Date User Action Args
2020-05-22 20:17:03pablogsalsetmessages: + msg369638
2020-05-22 18:41:03oleksandr.suvorovsetmessages: + msg369631
2020-05-22 15:20:01Mark.Shannonsetmessages: + msg369612
2020-05-22 11:25:41pablogsalsetnosy: + christian.heimes
2020-05-22 11:21:09pablogsalsetmessages: + msg369597
2020-05-22 11:20:23oleksandr.suvorovsetmessages: + msg369596
2020-05-22 11:19:51pablogsalsetnosy: - christian.heimes
messages: + msg369595
2020-05-22 11:19:17oleksandr.suvorovsetmessages: + msg369594
2020-05-22 11:15:35christian.heimessetnosy: + christian.heimes
messages: + msg369593
2020-05-22 11:15:13pablogsalsetmessages: + msg369592
2020-05-22 11:10:40oleksandr.suvorovsetmessages: + msg369591
2020-05-22 11:08:35pablogsalsetstatus: open -> closed
nosy: + Mark.Shannon

resolution: not a bug
stage: resolved
2020-05-22 11:08:12pablogsalsetmessages: + msg369590
2020-05-22 11:04:31pablogsalsetnosy: + pablogsal
messages: + msg369589
2020-05-22 10:59:10oleksandr.suvorovcreate