Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parser support for future import of unicode_strings #46729

Closed
nnorwitz mannequin opened this issue Mar 25, 2008 · 9 comments
Closed

parser support for future import of unicode_strings #46729

nnorwitz mannequin opened this issue Mar 25, 2008 · 9 comments
Assignees
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@nnorwitz
Copy link
Mannequin

nnorwitz mannequin commented Mar 25, 2008

BPO 2477
Nosy @malemburg, @ncoghlan, @ericvsmith, @tiran
Files
  • uni-strs.diff
  • unicode_literals2.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tiran'
    closed_at = <Date 2008-03-29.04:48:27.950>
    created_at = <Date 2008-03-25.06:15:47.401>
    labels = ['interpreter-core']
    title = 'parser support for future import of unicode_strings'
    updated_at = <Date 2008-03-29.04:48:27.939>
    user = 'https://bugs.python.org/nnorwitz'

    bugs.python.org fields:

    activity = <Date 2008-03-29.04:48:27.939>
    actor = 'nnorwitz'
    assignee = 'christian.heimes'
    closed = True
    closed_date = <Date 2008-03-29.04:48:27.950>
    closer = 'nnorwitz'
    components = ['Interpreter Core']
    creation = <Date 2008-03-25.06:15:47.401>
    creator = 'nnorwitz'
    dependencies = []
    files = ['9844', '9855']
    hgrepos = []
    issue_num = 2477
    keywords = ['patch']
    message_count = 9.0
    messages = ['64458', '64468', '64472', '64477', '64479', '64486', '64497', '64508', '64683']
    nosy_count = 6.0
    nosy_names = ['lemburg', 'nnorwitz', 'ncoghlan', 'eric.smith', 'christian.heimes', 'schmir']
    pr_nums = []
    priority = 'critical'
    resolution = 'accepted'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue2477'
    versions = ['Python 2.6']

    @nnorwitz
    Copy link
    Mannequin Author

    nnorwitz mannequin commented Mar 25, 2008

    This is a patch that modifies the parser to allow getting the future
    import flags into the AST. There are 2 approaches that are embedded
    within the patch. Both approaches can be seen in Python/pythonrun.c.

    1. update_flags_from_node() - this pulls the __future__ import out of
      the parser nodes. It is not complete, but should give an idea of how
      this approach could be generalized.
    2. Add APIS such as PyParser_ParseFileFlagsEx that returns the flags
      from the parser

    The first approach is somewhat fragile and kinda breaks encapsulation.
    It's nice that all the changes are internal and localized.

    The second approach is probably a better long term solution, but adds
    even more APIs where there are already too many.

    @nnorwitz nnorwitz mannequin added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Mar 25, 2008
    @tiran
    Copy link
    Member

    tiran commented Mar 25, 2008

    I'll check it out today.

    @tiran tiran self-assigned this Mar 25, 2008
    @tiran
    Copy link
    Member

    tiran commented Mar 25, 2008

    I've carefully examined both version. I agree that
    update_flags_from_node is too fragile. I haven't verified my theory but
    I *think* the current code does neither support

      from __future__ import egg, spam

    nor

      from __future__ import egg
      from __future__ import spam

    The PyParser_ParseFileFlagsEx() is easier and more stable. I'll go with it.

    By the way I also like your name "unicode_strings". Should we stay with
    it or use my idea "unicode_literals"?

    @malemburg
    Copy link
    Member

    The flag should be named "unicode_literals" since that what changes.
    Unicode strings are already available in Python 2.x :-)

    BTW: I'm not convinced that these future imports are all that useful -
    the ratio between usefulness and added complexity leans a lot towards
    the latter.

    @tiran
    Copy link
    Member

    tiran commented Mar 25, 2008

    Working patch

    I had to introduce yet another PyParser Ex method for string parsing. I
    also renamed the future feature to "unicode_literals"

    @ncoghlan
    Copy link
    Contributor

    The patch actually isn't all that complicated - the main structural
    change to make it possible to process the strings correctly is
    converting the parser functions to have treat the flags argument as an
    I/O variable, rather than just an input. Other than that, there are just
    some fairly straightforward updates to the compiler to take advantage of
    the additional flag information now available from the parser.

    Being able to write significant pieces of code that run on both 2.6 and
    3.0 without modification will be a big win in my opinion. While 2to3
    will still be a valuable migration tool, especially for one-way
    migrations, it will be far far easier to support 2.6 and 3.0 in parallel
    if that pseudo-compilation step isn't necessary.

    Also Christian - the posted patch accidentally included your hack to
    make the version info a bit more non-SVN checkout friendly.

    @tiran
    Copy link
    Member

    tiran commented Mar 25, 2008

    Also Christian - the posted patch accidentally included your hack to
    make the version info a bit more non-SVN checkout friendly.

    I'm still learning bzr. I just have figured out how to merge changes
    from the upstream trunk into my branch. It's actually nice ...

    @tiran
    Copy link
    Member

    tiran commented Mar 25, 2008

    Slightly better patch

    • removed bzr hack
    • simplified parsestr()
    • updated NEWS

    @nnorwitz
    Copy link
    Mannequin Author

    nnorwitz mannequin commented Mar 29, 2008

    Christian checked this in a few days ago in r61953 and a few other
    revisions.

    @nnorwitz nnorwitz mannequin closed this as completed Mar 29, 2008
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants