classification
Title: Allow mappings as globals (was: Fix dictionary subclass ...)
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: arigo, crutcher_gmail, loewis, methane, ppperry, rhettinger, terry.reedy
Priority: normal Keywords:

Created on 2006-01-11 00:24 by crutcher_gmail, last changed 2019-04-15 00:52 by rhettinger. This issue is now closed.

Files
File name Uploaded Description Edit
dictsubclassexec.patch crutcher_gmail, 2006-01-11 00:24 patch file
dictsubclassexec.42135.patch crutcher_gmail, 2006-01-22 09:19 new version of patch file
mappingglobals.42206.patch crutcher_gmail, 2006-01-30 19:10 updated patch for mappings as globals review
Messages (26)
msg49288 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-11 00:24
There is an inconsistancy in the way that dictionary
subclasses behave
when they are used as as namespaces in execs.

Basically, while python 2.4 permits the usage of
dictionary subclasses
for local environments, it still bypasses the subclass
functions and
uses the C API for global environments. The attached
patch (and
unittest!) addresses this issue.

I'm pretty sure we keep the fast path in this.
msg49289 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2006-01-11 02:03
Logged In: YES 
user_id=80475

Do you have any use cases?  AFAICT, there were no unmet
needs with locals option.

Also, there was a reason that globals weren't included but I
remember what it was.
msg49290 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-11 02:16
Logged In: YES 
user_id=1424288

With the ability to use dictionary subclasses for local and
global environments, it becomes very easy to implement
execution environments with extended variable semantics for
special purposes.

This includes driving interactive processes, config file
processors, type checking, lazy object construction, and
read-only variables.

As it currently stands, the semantics are inconsistent. The
local case should not have been changed if the global case
was not going to be changed.
msg49291 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-11 18:12
Logged In: YES 
user_id=1424288

Here's an example of a little typed environment. It's not
the most robust, but it gets you thinking.

import code
class TypedDictionary(dict):
 def __setitem__(self, key, value):
   if self.has_key(key):
     t = type(self[key])
     if t != type(value):
       try:
         value = t(value)
       except Exception:
         raise TypeError, \
             "illegal assignment to '%s':" \
             " %s cannot be coerced to %s" \
               % (key, type(value), t)
   dict.__setitem__(self, key, value)
code.interact(local=TypedDictionary())
msg49292 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-11 23:11
Logged In: YES 
user_id=1424288

Here's a more interesting example. This works fine, unless
the variables fall through to the global dictionary, or some
code marks them global.

import code

class ManagedVariable:
  def get(self):
    return None

  def set(self, value):
    pass

  def delete(self):
    # Return false to stop the delete.
    return True

class ManagedEnvironment(dict):
 def __setitem__(self, key, value):
   if self.has_key(key):
     if isinstance(dict.__getitem__(self, key),
ManagedVariable):
       dict.__getitem__(self, key).set(value)
       return
   dict.__setitem__(self, key, value)

 def __getitem__(self, key):
   if self.has_key(key):
     if isinstance(dict.__getitem__(self, key),
ManagedVariable):
       return dict.__getitem__(self, key).get()
   return dict.__getitem__(self, key)

 def __delitem__(self, key):
   if self.has_key(key):
     if isinstance(dict.__getitem__(self, key),
ManagedVariable):
       if not dict.__getitem__(self, key).delete():
         return
   dict.__delitem__(self, key)


class RangedInt(ManagedVariable):
  def __init__(self, value, (low, high)):
    self.value = value
    self.low = low
    self.high = high

  def get(self):
    return self.value

  def set(self, value):
    if value < self.low:
      value = self.low
    if value > self.high:
      value = self.high
    self.value = value


class FunctionValue(ManagedVariable):
  def __init__(self, get_func = None, set_func = None,
del_func = None):
    self.get_func = get_func
    self.set_func = set_func
    self.del_func = del_func

  def get(self):
    if self.get_func:
      return self.get_func()
    return None

  def set(self, value):
    if self.set_func:
      self.set_func(value)

  def delete(self):
    if self.del_func:
      return self.del_func()
    return True

class Constant(ManagedVariable):
  def __init__(self, value):
    self.value = value

  def get(self):
    return self.value

  def delete(self):
    return False


import time
d = ManagedEnvironment()
d['ranged'] = RangedInt(1, (0, 100))
d['time'] = FunctionValue(lambda: time.time())
d['constant'] = Constant(42)

code.interact(local=d)
msg49293 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2006-01-12 01:21
Logged In: YES 
user_id=80475

Okay, I will take it under consideration.  Also, I'll try to
remember the reason it wasn't done the first time around.

No promises though -- by itself consistency is a weak
motivation.
msg49294 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-01-12 06:11
Logged In: YES 
user_id=21627

I think the history is this: it originates from
python.org/sf/215126, which requested that you can pass dict
subtypes to eval. Armin noted that eval will always produce
LOAD/STORE_NAME, so just modifying these opcodes is
sufficient to fulfil the feature request.
msg49295 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-12 07:50
Logged In: YES 
user_id=1424288

Well, why fix it for eval but not for exec? I don't think
the time hit is noticeable, I ran 'time make test' twice
each on the trunk with and without the patch. Here are the
results:

Trunk:
real    9m17.117s
user    3m30.930s
sys     0m35.417s

real    9m9.471s
user    3m31.484s
sys     0m34.978s

Patch:
real    9m32.469s
user    3m40.134s
sys     0m36.140s

real    9m6.779s
user    3m27.529s
sys     0m34.716s
msg49296 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2006-01-14 09:54
Logged In: YES 
user_id=4771

Provided this can be done with no measurable performance hit,
I guess that I'm fine with the idea.  The patch needs a bit
more work, though: I don't see why it should accept dict
subclasses as globals but not arbitrary mappings (as it now
does for the locals).  This is mainly an issue of removing
a few checks in various places, like EXEC_STMT and the eval()
and execfile() built-ins.

There is a missing exception check/clear in the part about
LOAD_NAME, after the PyObject_GetItem(f->f_globals, w).

A side note: in the current trunk already, LOAD_GLOBAL
contains a couple of checks, namely PyString_CheckExact()
and hash != -1.  We might be able to prove in advance that
these two conditions are always true.  We could then remove
the checks.  Not sure the difference measurable, though.
msg49297 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-22 05:11
Logged In: YES 
user_id=1424288

I'm going to fix the missed exception test this weekend, and
try to get an updated patch to you.
msg49298 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-22 09:19
Logged In: YES 
user_id=1424288

I've fixed up the exception case, and extended the test case
to check for it. Is there anything else I can do to get this in?
msg49299 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-25 03:38
Logged In: YES 
user_id=1424288

currently chasing through the PyFrame code and the EXEC_STMT
code to make globals generic mappings, will update this
patch yet again.
msg49300 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2006-01-25 18:20
Logged In: YES 
user_id=4771

I will review and check-in your patch, I think it's a good
idea despite the added code complexity.
msg49301 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-30 17:08
Logged In: YES 
user_id=1424288

I have reworked and extended this patch so that arbitrary
mappings are permitted for globals, not just dictionary
subtypes. This touched a good deal more code.

Please use the updated version.
msg49302 - (view) Author: crutcher (crutcher_gmail) Date: 2006-01-30 19:10
Logged In: YES 
user_id=1424288

doh, forgot to check the 'upload' box
msg49303 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2006-02-21 13:28
Logged In: YES 
user_id=80475

Crutcher, this looked good on my first read-through.  Will 
go through it in detail sometime in March.
msg49304 - (view) Author: crutcher (crutcher_gmail) Date: 2006-03-30 18:12
Logged In: YES 
user_id=1424288

Are you going to get to this?
msg49305 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-02-15 09:55
Raymond, do you think you can make it this March? If not, please unassign.
msg49306 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2007-02-15 20:04
I do still expect to work on this patch.  I'm not sure where the March date is coming from.  This is a new feature and accordingly would go into Py2.6.
msg49307 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-02-18 09:06
Raymond, on 2006-02-21, you said "Will go through it in detail sometime in March", hence I suggested that perhaps you could do it March 2007 :-)
msg78882 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-02 21:47
Haven't had a chance to work on this one.  Unassigning.
msg113446 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-09 18:41
I believe this is covered by the PEP3003 3.2 change moratorium.
msg340180 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2019-04-14 03:51
13 years past from this proposed.  Is this still good feature for Python?

Personally, I dislike adding more dynamic flexibility to Python name space.

Python is very dynamic language, and it made difficult to make Python faster.
For example, PHP is not so dynamic as Python, and it is one of reasons why PHP VM is now much faster than Python VM.  They can easily do more optimization like function inlining statically.

I'm interested in optimizing Python.  But I'm not interested in adding more "slow path" we need to maintain forever.
msg340186 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-04-14 06:25
I am willing to close it.
msg340237 - (view) Author: (ppperry) Date: 2019-04-14 23:52
See also issue32615
msg340239 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-04-15 00:52
I concur that this should be closed. We have a decade of evidence that no one really needs this.
History
Date User Action Args
2020-01-17 05:51:12methanelinkissue32615 superseder
2019-04-15 00:52:41rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg340239

stage: patch review -> resolved
2019-04-14 23:52:14ppperrysetnosy: + ppperry
messages: + msg340237
2019-04-14 06:25:32terry.reedysetmessages: + msg340186
2019-04-14 03:51:58methanesetmessages: + msg340180
2019-04-13 23:24:21cheryl.sabellasetkeywords: - after moratorium
nosy: + methane

versions: + Python 3.8, - Python 3.5
2014-01-31 21:49:40yselivanovsetversions: + Python 3.5, - Python 3.3
2010-08-09 18:43:24terry.reedysetkeywords: + after moratorium, - patch
2010-08-09 18:41:43terry.reedysetnosy: + terry.reedy

messages: + msg113446
versions: + Python 3.3, - Python 3.2
2010-06-11 00:30:22belopolskysetstage: patch review
type: enhancement
versions: + Python 3.2, - Python 3.1, Python 2.7
2009-01-02 21:47:36rhettingersetassignee: rhettinger ->
messages: + msg78882
versions: + Python 3.1, Python 2.7, - Python 2.6
2006-01-11 00:24:45crutcher_gmailcreate