This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: parser support for future import of unicode_strings
Type: Stage:
Components: Interpreter Core Versions: Python 2.6
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, eric.smith, lemburg, ncoghlan, nnorwitz, schmir
Priority: critical Keywords: patch

Created on 2008-03-25 06:15 by nnorwitz, last changed 2022-04-11 14:56 by admin. This issue is now closed.

File name Uploaded Description Edit
uni-strs.diff nnorwitz, 2008-03-25 06:15
unicode_literals2.patch christian.heimes, 2008-03-25 19:37
Messages (9)
msg64458 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-03-25 06:15
This is a patch that modifies the parser to allow getting the future
import flags into the AST.  There are 2 approaches that are embedded
within the patch.  Both approaches can be seen in Python/pythonrun.c.

1) update_flags_from_node() - this pulls the __future__ import out of
the parser nodes.  It is not complete, but should give an idea of how
this approach could be generalized.
2) Add APIS such as PyParser_ParseFileFlagsEx that returns the flags
from the parser

The first approach is somewhat fragile and kinda breaks encapsulation. 
It's nice that all the changes are internal and localized.

The second approach is probably a better long term solution, but adds
even more APIs where there are already too many.
msg64468 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-25 08:44
I'll check it out today.
msg64472 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-25 10:30
I've carefully examined both version. I agree that
update_flags_from_node is too fragile. I haven't verified my theory but
I *think* the current code does neither support

  from __future__ import egg, spam


  from __future__ import egg
  from __future__ import spam

The PyParser_ParseFileFlagsEx() is easier and more stable. I'll go with it.

By the way I also like your name "unicode_strings". Should we stay with
it or use my idea "unicode_literals"?
msg64477 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2008-03-25 11:32
The flag should be named "unicode_literals" since that what changes.
Unicode strings are already available in Python 2.x :-)

BTW: I'm not convinced that these future imports are all that useful -
the ratio between usefulness and added complexity leans a lot towards
the latter.
msg64479 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-25 12:42
Working patch

I had to introduce yet another PyParser Ex method for string parsing. I
also renamed the future feature to "unicode_literals"
msg64486 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-03-25 15:02
The patch actually isn't all that complicated - the main structural
change to make it possible to process the strings correctly is
converting the parser functions to have treat the flags argument as an
I/O variable, rather than just an input. Other than that, there are just
some fairly straightforward updates to the compiler to take advantage of
the additional flag information now available from the parser.

Being able to write significant pieces of code that run on both 2.6 and
3.0 without modification will be a big win in my opinion. While 2to3
will still be a valuable migration tool, especially for one-way
migrations, it will be far far easier to support 2.6 and 3.0 in parallel
if that pseudo-compilation step isn't necessary.

Also Christian - the posted patch accidentally included your hack to
make the version info a bit more non-SVN checkout friendly.
msg64497 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-25 16:54
> Also Christian - the posted patch accidentally included your hack to
make the version info a bit more non-SVN checkout friendly.

I'm still learning bzr. I just have figured out how to merge changes
from the upstream trunk into my branch. It's actually nice ...
msg64508 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-03-25 19:37
Slightly better patch

* removed bzr hack
* simplified parsestr()
* updated NEWS
msg64683 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-03-29 04:48
Christian checked this in a few days ago in r61953 and a few other
Date User Action Args
2022-04-11 14:56:32adminsetgithub: 46729
2008-03-29 04:48:27nnorwitzsetstatus: open -> closed
resolution: accepted
messages: + msg64683
2008-03-25 19:38:01christian.heimessetfiles: + unicode_literals2.patch
messages: + msg64508
2008-03-25 19:37:05christian.heimessetfiles: - unicode_literals.patch
2008-03-25 16:54:48christian.heimessetmessages: + msg64497
2008-03-25 15:02:54ncoghlansetnosy: + ncoghlan
messages: + msg64486
2008-03-25 12:59:24schmirsetnosy: + schmir
2008-03-25 12:42:47christian.heimessetfiles: + unicode_literals.patch
messages: + msg64479
2008-03-25 11:32:55lemburgsetnosy: + lemburg
messages: + msg64477
2008-03-25 10:30:03christian.heimessetmessages: + msg64472
2008-03-25 09:07:01eric.smithsetnosy: + eric.smith
2008-03-25 08:44:44christian.heimessetassignee: christian.heimes
messages: + msg64468
nosy: + christian.heimes
2008-03-25 06:15:47nnorwitzcreate