classification
Title: UnboundLocalError with local variable set by setattr, caused by code run later
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: docs@python Nosy List: cben, docs@python, eric.smith, ssc, terry.reedy
Priority: normal Keywords:

Created on 2010-10-07 10:13 by ssc, last changed 2010-11-20 23:30 by eric.smith. This issue is now closed.

Files
File name Uploaded Description Edit
show_weird_behavior.py ssc, 2010-10-07 10:13 sample code for 'UnboundLocalError with local variable set by setattr' bug
Messages (8)
msg118099 - (view) Author: Steven Samuel Cole (ssc) Date: 2010-10-07 10:13
inside a function, I create a local variable by using setattr with the current module as object, as in
  setattr(modules[__name__], 'name', 'value')

if I _later_ in the code set name to None, the attribute becomes unavailable even for code that is executed _before_ setting name to None.

please also see sample file.
msg118100 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-10-07 11:31
When python sees the assignment "name = None", it assumes that 'name' is a local variable. This happens before any code is executed. It then sees that you're printing 'name' before it is assigned to, which is an error. It does not take into account the dynamic creation of the module level variable.
msg118105 - (view) Author: Steven Samuel Cole (ssc) Date: 2010-10-07 12:08
I'm just a developer, I don't know anything about Python internals or what Python sees or does at what stage.

Looking at the sample file, code executed later has an influence on code executed earlier - and that behavior is unexpected, confusing, bizarre, no matter what expert explanations there are.

It took me a while to isolate this problem, thought I report it here to save others the effort. This should at least be mentioned in the docs somewhere around setattr(...).

I think it is premature to declare this report invalid and close the bug - but hey, if _you_ are not interested in improving Python wherever possible, why would _I_ be ?
msg118106 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-10-07 12:31
It's a well documented behavior.

Surely you ran across this link while researching your problem? http://docs.python.org/reference/executionmodel.html#naming-and-binding
The paragraph beginning "The following constructs bind names ..." describes why setattr does not create a binding, although I'll grant that it doesn't mention setattr by name. The paragraph beginning "If a name binding operation occurs anywhere within a code block ..." explains why the assignment affects code that occurs before the binding.

If you'd like to improve the docs, please suggest a patch.

Also note that you're not creating a local variable with setattr, you're creating a module level (sometimes called global) variable. Perhaps that's part of the misunderstanding? You might want to look into the global (on nonlocal) statement, unless there's some particular reason the name you're creating needs to be dynamically computed.
msg118108 - (view) Author: Steven Samuel Cole (ssc) Date: 2010-10-07 13:03
thank you very much for the clarification.

i did indeed not come across the page you've linked to, mainly because i did not really know what to google for. actually, i do not recall ever looking into the python language reference in 6+ years of python coding.

googling for 'python UnboundLocalError setattr' returns this bug report as top result at the moment, as your link leads to more indepth information, the main objective of saving / alleviating others the research work seems achieved. i will nonetheless try to make time to get get my head around the python doc conventions and processes and submit a patch.

thanks for pointing out the difference between a local variable and one with module scope. however, it is not relevant for my situation: i have a long list of strings coming in (database column names, not under my influence) which i need to access _like_ a local variable. if you can think of a smarter approach to turn parameter strings into variables of whatever scope, i'm all ears, but i have a feeling that's actually what setattr(...) is meant for.
 
as a quick fix for the UnboundLocalError, sticking to working with module attributes worked for me. instead of changing the value of a dynamically created variable in the conventional way
  my_var = None
(which is where the error occurred in my code), it might help to use setattr(...) even if the variable name is known:
  setattr(modules[__name__], 'my_var', None)
msg118237 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-10-08 23:55
The usual way to set a module variable by name, rather than
setattr(modules[__name__], 'name', 'value')
is
>>> globals()['name'] = 2
>>> name
2

Issues of working with external names, such as from database columns, has been discussed several times on python-list and the corresponding newsgroups. Please post there is you want to discuss that.
msg121804 - (view) Author: Cherniavsky Beni (cben) * Date: 2010-11-20 22:41
Hi Steven.
Please confirm if we can mark the bug closed; if you need farther advice, posting your full code (not just the error case) on comp.lang.python or StackOverflow would be more effective.

The documentation is indeed correct but hard to find (you're not the first to be surprised by UnboundLocalError); I'm working on making things more discoverable in issue 4246.
See also http://docs.python.org/py3k/faq/programming.html#why-am-i-getting-an-unboundlocalerror-when-the-variable-has-a-value

First of all, it's important to understand that a Python function has a *fixed* set of local variables, frozen when the function is parsed.  If you assign to it (e.g. ``name = None``), *all* appearances of the name in the function refer to a local variable; if not, they refer to the outer scope.
Therefore, you can't achieve what you want with local variables.

Generally, dynamically creating variables is a bad programming practice.  A dictionary is the cleanest way to hold a set of names/values that is not fixed.  Yes, you'll have to write ``cols['foo']`` instead of ``foo``; OTOH, setting them will not require any ugly magic...
Note also that string formatting can use values from a dictionary with very conveniently: ``"... {foo} ...".format(**cols)``.

The next best thing if ``cols['foo']`` is too verbose for you is ``cols.foo``: create an object which will contain the values as instance variables (that's a good use for setattr()).
This is the most Pythonic solution if a dictionary doesn't suffice - it's what most object-relational mappers do.

The third idea is to (ab)use a class statement.  A class statement in Python creates a temporary namespace *during the class definition* (we'll not be defining any methods or using it object-oriented stuff).
And the nice part is that you can put a class statement anywhere, even inside a function:
    def f():
        cols = {'foo': 42}  # however you fetch them...
	class temp_namespace:
            locals().update(cols)
            print(foo / 6)     # prints 7.0
        assert 'foo' not in locals()  # no effect outside the class!
This works both in CPython 2 and 3.  I'm not 100% sure that being able to change locals() in a class is guaranteed in all other implementations.  (Note again that locals() of a *function* are not a real dictionary and you *can't* change them - as I said these are fixed when the function is defined.)

The fourth idea if you must have code that says just ``foo`` to access columns is to use the exec statement - you can pass it a dictionary that will serve as globals and/or locals.  An upside is that the code will be a string and can be dynamic as well.

(BTW, if the code is not dynamic, how come you don't know the names you're accessing?  If you do, you could just set ``foo = cols['foo']`` etc. for every variable you need - tedious but no magic needed.)

Lastly, as you discovered you can dynamically create global variables.  
(As Terry said, just use the dictionary returned by ``globals()``; no need for setattr).
But this is a very last resort (changing globals for a single function is ugly), and somewhat dangerous - e.g. consider what happens if a column names changes and overwrites a function name you had...
msg121813 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-11-20 23:30
Closing, as I don't think there's any action item here.
History
Date User Action Args
2010-11-20 23:30:37eric.smithsetstatus: open -> closed
resolution: not a bug
messages: + msg121813

stage: needs patch -> resolved
2010-11-20 22:41:13cbensetnosy: + cben
messages: + msg121804
2010-10-08 23:55:21terry.reedysetnosy: + terry.reedy
messages: + msg118237
2010-10-07 13:03:50sscsetmessages: + msg118108
2010-10-07 12:31:14eric.smithsetstatus: closed -> open

assignee: docs@python
components: + Documentation, - Interpreter Core
versions: + Python 3.2, - Python 2.6
nosy: + docs@python

messages: + msg118106
resolution: not a bug -> (no value)
stage: needs patch
2010-10-07 12:08:33sscsetmessages: + msg118105
2010-10-07 11:31:49eric.smithsetstatus: open -> closed

nosy: + eric.smith
messages: + msg118100

resolution: not a bug
2010-10-07 10:13:47ssccreate