classification
Title: Write UserDict fixer for 2to3
Type: behavior Stage:
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 2.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: benjamin.peterson Nosy List: asmodai, benjamin.peterson, brett.cannon, collinwinter, eric.araujo, loewis, nedds, r.david.murray, rhettinger
Priority: normal Keywords: patch

Created on 2008-05-16 04:49 by brett.cannon, last changed 2012-07-02 19:26 by brett.cannon. This issue is now closed.

Files
File name Uploaded Description Edit
fix_userdict.diff nedds, 2008-09-27 16:27 UserDict fixer and test suite
Messages (30)
msg66900 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-05-16 04:49
In Python 3.0, the UserDict module was removed and the UserDict class was 
moved to the collections module. That change-over needs to be backported 
to 2.6 so that the UserDict module can be deprecated.
msg66904 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-05-16 04:59
This doesn't make any sense to me.  The 2.6 code runs fine as-is.  The 
2-to-3 tool can handle switching from UserDict.UserDict to 
collections.UserDict.  What's the issue?  And why is this marked as a 
release blocker?
msg67089 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-05-20 04:18
You only kept the UserDict class, right? Well, that means you need to 
deprecate DictMixin and any other classes in that module.
msg67090 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-05-20 04:19
And it is a release blocker as we are trying to get all deprecations into 
b1.
msg67100 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-05-20 04:46
The DictMixin class gets renamed to collections.MutableMapping.  So, it 
is just another 2-to-3 converter job.
msg67101 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-05-20 05:01
The that needs to be added to 2to3.
msg67357 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-05-26 00:24
Raymond, can you tell me exactly where each module-level thing in UserDict 
went and if it was renamed or not? That way the fixer can get written.
msg67361 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-05-26 03:49
UserDict.UserDict is gone.
UserDict.IterableUserDict class is now collections.UserDict.
UserDict.Mixin is now collections.MutableMapping.

The new classes comform to the new dict API, so they fixer needs to 
also make same method adaptatations as for dicts (i.e. d.keys() --> list
(d.keys() and d.has_key --> d.__contains__).
msg67378 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-05-26 17:34
I don't know if 2to3 can handle the class renames. Up to this point we 
have only been handling module renames.

Collin, how doable is this?
msg68248 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2008-06-15 20:02
It's doable, but there's no existing support similar to what fix_imports
provides.

I won't have time to work on this for the foreseeable future, but if
someone is interested in working on this, I'd add this challenge:
implement this in a way that generalizes well and is better/easier than
what fix_imports does. I never expected fix_imports's mechanism to be so
heavily reused, and as a result it's ass-slow.
msg69112 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-07-02 19:50
There is a possible patch in issue2046.
msg71354 - (view) Author: Nick Edds (nedds) Date: 2008-08-18 17:59
What's the current status of this? If nobody is working on it, I would
be willing to give it a shot. Can somebody just confirm that I have a
correct understanding of the problem.
UserDict.UserDict needs a deprecation warning, UserDict.IterableUserDict
becomes collections.UserDict, and UserDict.Mixin becomes
collections.MutableMapping. Then for keys(), items(), and values(), I
want to replace something like d.keys() to list(d.keys()). 
One added question: based on what I gathered from PEP 3106, this last
part is only needed in a situation such as a = d.keys(), but not a
situation like for key in keys:, so because in the later case wrapping
the call to keys() in list() would presumably be suboptimal, is this
something I should try and avoid? I'm not sure exactly how it could be
done, but I have some idea of how it to do it.
msg71355 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-08-18 18:07
On Mon, Aug 18, 2008 at 10:59 AM, Nick Edds <report@bugs.python.org> wrote:
>
> Nick Edds <nedds@uchicago.edu> added the comment:
>
> What's the current status of this?

I think it is waiting for someone to work on it.

> If nobody is working on it, I would
> be willing to give it a shot.

Great!

> Can somebody just confirm that I have a
> correct understanding of the problem.
> UserDict.UserDict needs a deprecation warning, UserDict.IterableUserDict
> becomes collections.UserDict, and UserDict.Mixin becomes
> collections.MutableMapping.

In theory, yes, but the superclasses have changed, so I don't know if
having a fixer is really the best option in this case since it is not
a simple renaming. Perhaps deprecation warnings stating that the
classes have been moved and given different superclasses would make
more sense?

> Then for keys(), items(), and values(), I
> want to replace something like d.keys() to list(d.keys()).

Sure, but how the heck are you going to get 2to3 to do this properly
just for UserDict instances? Doesn't the keys()/values()/items() fixer
already do this blindly anyway?

> One added question: based on what I gathered from PEP 3106, this last
> part is only needed in a situation such as a = d.keys(), but not a
> situation like for key in keys:, so because in the later case wrapping
> the call to keys() in list() would presumably be suboptimal, is this
> something I should try and avoid? I'm not sure exactly how it could be
> done, but I have some idea of how it to do it.
>

Yes, you are right that wrapping the call in a for loop is not needed
since code will never come across it. But as I said above, doesn't the
fixer for dicts already handle all of this?
msg71378 - (view) Author: Nick Edds (nedds) Date: 2008-08-18 21:08
Ahh right. I totally forgot that there was already a fix_dict.py that
took care of that.
msg71650 - (view) Author: Nick Edds (nedds) Date: 2008-08-21 15:24
I've been thinking about this a bit, and I don't see how handling it
should be all that different from handling module renaming. 

For import UserDict, we can just change UserDict to collections and deal
with UserDict.UserDict with a deprecation warning and
UserDict.IterableUserDict and UserDict.Mixin with transformations to
collections.UserDict and collections.MutableMapping.

For from imports, we can raise the deprecation warning on UserDict, and
otherwise change from UserDict import ... to from collections import ...
and then for non-import appearances of UserDict, IterableUserDict, and
Mixin rename or raise a deprecation warning as appropriate.

For the other import cases, similar things could be done.

I think the old version of fix_imports, which had support like this,
would basically be all that we need. I should be able to mimic that
functionality, but also make it more scalable than the previous version.

Is there something I'm missing here about the difference between modules
and classes? From the use-cases I've observed of UserDict, it seems like
this functionality would be sufficient.
msg71667 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-08-21 18:33
As I said, it isn't using the import filter, it's whether this should be
done through a warning instead because the superclasses have changed.

I really should ask on python-dev about this.
msg71669 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-08-21 18:49
You know what, Nick, go for the fixer. UserDict.UserDict will need a
deprecation warning, but otherwise a fixer for IterableUserDict and
DictMixin is fine.

And I can handle the deprecation if you want.
msg71891 - (view) Author: Nick Edds (nedds) Date: 2008-08-24 22:00
How soon is this needed by? I probably won't have a chance to do it in
the next week, but it shouldn't take that long to make once I get a
chance to work on it.
msg71903 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-08-24 23:41
On Sun, Aug 24, 2008 at 3:00 PM, Nick Edds <report@bugs.python.org> wrote:
>
> Nick Edds <nedds@uchicago.edu> added the comment:
>
> How soon is this needed by? I probably won't have a chance to do it in
> the next week, but it shouldn't take that long to make once I get a
> chance to work on it.
>

rc1 is Sep. 3, with rc3 Sep 17 (at the moment). Hitting either should
be okay, although obviously the sooner the better.
msg72970 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-09-10 15:58
I still don't see why this is a release blocker. Adding a 2to3 fixer
certainly isn't a release blocker - if the fixer isn't available, users
would need to manually fix the code.

Adding a Py3k warning shouldn't be a release blocker, either - I think
such warnings could get added in 2.6.1 also (as they are only issued if
you invoke Python with -3).

Tentatively lowering the priority to normal.
msg73347 - (view) Author: Nick Edds (nedds) Date: 2008-09-17 22:10
Sorry about the delay with this. I've finally found some time to work on
this, so it should be completed within the next couple of days.
msg73915 - (view) Author: Nick Edds (nedds) Date: 2008-09-27 02:25
I have the functionality for this working, and will post it tomorrow
when I complete its test suite.
msg73933 - (view) Author: Nick Edds (nedds) Date: 2008-09-27 16:27
Here is a fixer and test suite for the UserDict. It doesn't do a very
good job handling imports with 'as' right now, but it's core
functionality appears to be working. If I get a chance I will improve
its handling of 'as' cases at some point in the future, but I think this
is an alright start. It passes all tests.
msg74107 - (view) Author: Nick Edds (nedds) Date: 2008-10-01 02:09
Could somebody else take a look at this and check in it if it looks
alright? I think that it works currently but I can't check it in...
msg74278 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-10-03 22:21
The patch looks very good over all. However, I'm not completely sure
that 2to3 needs to give a warning on UserDict.UserDict instances.
Wouldn't it be better to handle that with a py3k warning in UserDict?
msg74279 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-10-03 22:23
Also, I believe the variable initializations at the beginning of the
transform method are unneeded.
msg102620 - (view) Author: Jeroen Ruigrok van der Werven (asmodai) * (Python committer) Date: 2010-04-08 14:44
What's the consensus on this? I ask since I actually ran into code today that uses DictMixin and as such wasn't converted by the 2to3 tool.
msg102623 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-04-08 15:36
UserDict.DictMixin needs to convert to collections.MutableMapping
msg102624 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-04-08 15:53
Converting DictMixin to collections.MutableMapping is not necessarily a complete solution.  MutableMapping does not provide all the methods that DictMixin does.  See for example the problems encountered in issue 7975.
msg164532 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-07-02 19:26
Closing since 2.7 is already out the door.
History
Date User Action Args
2012-07-02 19:26:26brett.cannonsetstatus: open -> closed
resolution: wont fix
messages: + msg164532
2010-11-29 16:25:48eric.araujosetsuperseder: patch to fix_import: UserDict -> collections ->
2010-11-29 16:25:41eric.araujolinkissue2046 superseder
2010-11-29 16:24:49eric.araujosetnosy: + eric.araujo
2010-04-10 22:10:25loewissetnosy: loewis, brett.cannon, collinwinter, rhettinger, benjamin.peterson, asmodai, nedds, r.david.murray
components: + 2to3 (2.x to 3.x conversion tool), - Library (Lib)
2010-04-08 15:53:17r.david.murraysetnosy: + r.david.murray
messages: + msg102624
2010-04-08 15:36:41rhettingersetmessages: + msg102623
2010-04-08 14:44:07asmodaisetnosy: + asmodai
messages: + msg102620
2008-10-03 22:23:20benjamin.petersonsetmessages: + msg74279
2008-10-03 22:21:05benjamin.petersonsetmessages: + msg74278
2008-10-01 03:07:03benjamin.petersonsetassignee: benjamin.peterson
nosy: + benjamin.peterson
2008-10-01 02:09:54neddssetmessages: + msg74107
2008-09-27 16:27:30neddssetfiles: + fix_userdict.diff
keywords: + patch
messages: + msg73933
2008-09-27 02:25:08neddssetmessages: + msg73915
2008-09-17 22:10:47neddssetmessages: + msg73347
2008-09-10 15:58:55loewissetpriority: release blocker -> normal
nosy: + loewis
messages: + msg72970
2008-09-09 13:13:03barrysetpriority: deferred blocker -> release blocker
2008-09-04 01:09:19benjamin.petersonsetpriority: release blocker -> deferred blocker
2008-08-24 23:41:08brett.cannonsetmessages: + msg71903
2008-08-24 22:00:39neddssetmessages: + msg71891
2008-08-21 22:24:06benjamin.petersonlinkissue2012 superseder
2008-08-21 18:49:19brett.cannonsetassignee: collinwinter -> (no value)
messages: + msg71669
2008-08-21 18:33:02brett.cannonsetpriority: high -> release blocker
messages: + msg71667
2008-08-21 15:24:58neddssetmessages: + msg71650
2008-08-18 21:08:01neddssetmessages: + msg71378
2008-08-18 18:07:32brett.cannonsetmessages: + msg71355
2008-08-18 17:59:26neddssetnosy: + nedds
messages: + msg71354
2008-07-02 19:50:45brett.cannonsetsuperseder: patch to fix_import: UserDict -> collections
messages: + msg69112
2008-07-02 19:50:24brett.cannonunlinkissue2046 superseder
2008-07-02 19:49:43brett.cannonlinkissue2046 superseder
2008-06-15 20:02:49collinwintersetmessages: + msg68248
2008-06-08 23:28:19benjamin.petersonunlinkissue2775 dependencies
2008-06-08 23:25:09benjamin.petersonsetpriority: release blocker -> high
2008-05-26 17:34:32brett.cannonsetassignee: brett.cannon -> collinwinter
messages: + msg67378
nosy: + collinwinter
2008-05-26 03:59:16rhettingersetassignee: rhettinger -> brett.cannon
2008-05-26 03:50:21rhettingersetmessages: + msg67361
2008-05-26 00:24:21brett.cannonsetassignee: rhettinger
messages: + msg67357
2008-05-20 05:01:29brett.cannonsetstatus: closed -> open
messages: + msg67101
title: Backport UserDict move in 3.0 -> Write UserDict fixer for 2to3
2008-05-20 04:46:10rhettingersetstatus: open -> closed
messages: + msg67100
2008-05-20 04:19:16brett.cannonsetmessages: + msg67090
2008-05-20 04:18:50brett.cannonsetmessages: + msg67089
2008-05-16 04:59:51rhettingersetnosy: + rhettinger
messages: + msg66904
2008-05-16 04:49:20brett.cannonlinkissue2775 dependencies
2008-05-16 04:49:07brett.cannoncreate