Especially since PEP 263, we badly need a way of
turning a Python string literal into the appropriate
object.

The immediate need is to get the compiler package
working again, but I'm sure there are other places it
could be used.

I thought Martin already had a patch somewhere doing
this, but I can't find it.  Maybe he remembers?

Logged In: YES 
user_id=21627

Not sure what you are asking. eval() would certainly be an
easy way, no?
In any case, I'm not aware of an alternative.

Logged In: YES 
user_id=6656

How do you get eval to take notice of the declared encoding?

What I want is the functionality of compile.c:parsestr
available from Python.

Logged In: YES 
user_id=21627

I see.

For a different reason (interactive mode), I have been
contemplating to recognize a variable source_encoding in
globals(), which would be set in interactive mode to the
terminal's encoding, so that input of non-ASCII Unicode
literals would "work", without having to type an encoding
declaration each time.

I believe such a mechanism could also solve this problem;
callers of eval would have to set the global.

What do you think?

Logged In: YES 
user_id=38388

I believe the correct way to handle this would be to have
eval() accept Unicode and then convert it to UTF-8 for
the parser to process. The PEP only talks about compile()
which should accept Unicode objects, but eval() is certainly
a candidate too.

About the source_encoding idea: wouldn't it be better to
define this for interactive use in the sys module ? A Python
startup script could then be used to set this variable in
interactive mode.

BTW, it's better to discuss these things on python-dev than
in these small textareas... IMHO, at least.

Logged In: YES 
user_id=21627

That would not be correct. If you have a (plain) string
literal, it would end up encoded as UTF-8, whereas the
original source encoding was, say, Latin-1. People will be
confused if string literals don't come out in the expected
encoding in interactive mode.

Logged In: YES 
user_id=6656

MAL: I don't understand your first paragraph at all.

The {sys.,}source_encoding variable might work, but it would
be a dreadful hack for my purposes.  Using eval() is already
a nasty hack, IMHO.

I agree a mailing list is a more civilised interface than a
web browser...

Logged In: YES 
user_id=38388

MWH: My first paragraph was refering to Martin's suggestion
to use eval() for mapping literal strings to objects.

Please take this to the python-dev mailing list.

Logged In: YES 
user_id=6656

For a bunch of reasons, I now care less about this so I'm
going to reduce the priority.

Logged In: YES 
user_id=38388

Has there been any progress on this or can I close it ?

Logged In: YES 
user_id=21627

It still needs to be implemented.

Logged In: YES 
user_id=6656

I'd have to admit to not being sure what I'm actually asking for in 
this report, so I won't object if it's closed.

I do think that, e.g., using the tokenize module has gotten harder 
since PEP 263, but I don't have a pressing problem right now...

Closed as requested by the OP four years ago.