This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: 'exec' does not accept what 'open' returns
Type: Stage:
Components: Interpreter Core Versions: Python 3.0
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: gvanrossum Nosy List: brett.cannon, christian.heimes, georg.brandl, gvanrossum
Priority: normal Keywords:

Created on 2007-07-29 03:24 by brett.cannon, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (14)
msg32561 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2007-07-29 03:24
Since the move over to 'exec' no longer accepts what 'open' returns.  Looks like there is a type check in 'exec' for strings, files, or code object, but 'open' returns a TextIOWrapper that fails that typecheck.
msg32562 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-08-03 20:26
Arguably we don't need this feature any more.  The only reasonable way to implement it would be to slurp the entire contents of the file into a string and then exec that.  The caller might as well do this.  Alternatively one could use execfile() (which takes a filename instead of a stream).  The only advantage of using exec(<stream>) might be that it would deal with encoding declarations transparantly; but we should probably have a way to handle those somewhere in the library anyway, as it's also necessary for other tools (e.g., and  Alternatively I'm not so sure that we will need to support encoding declarations forever -- I'm hoping that at some point UTF-8 and BOM+UTF-16 will take over the world.
msg32563 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2007-08-03 21:09
I am fine with ditching the feature and forcing people to deal with the encoding issues externally to 'exec', although as you pointed out dealing with any -*- encoding marker might be a pain without some library help (although doesn't the parser pick up on this and handle those nasty details?).

As for execfile, PEP 3100 has it listed for removal in favour of moving people over to using 'exec'.

The main thing is trying to come up with an easy solution for replacing 'reload' with a reasonable one-liner when debugging at the interpreter.

And yes, I hope encoding declarations can go away and the world either lives with just UTF-8 or BOM+UTF-16 (I really wish we could just require this for all Python source files and start towards this, but I know some people would flip out over that).
msg32564 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-08-03 21:14
OK, let's ditch it (i.e. status quo) and also ditch execfile.

The one-liner you're looking for is *almost*

exec(open(M.__file__).read(), M.__dict__)

(where M stands for any already imported module), except for the nastiness that if m.__file__ ends with .pyc or .pyo you'd have to strip the last character.  (And to do this right you'd need help from the imp module because those extensions may vary by platform -- e.g. I know of a company that patches their Pythons to use .pyc24.)
msg32565 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2007-08-03 21:42
Hrm.  Well then we can either add something like __source_file__ to modules, put 'reload' into the imp module (or on module objects as a method), or have a function in imp (or as a method on modules) that returns the source path (if it exists).

But having to do::

  M.__file__.rsplit('.', 1) + filter((lambda x : x[2] == imp.PY_SOURCE), imp.get_suffixes())[0]

seems like a lot to memorize (let alone type in), especially since there is even no error checking that the path is even to a source or bytecode file to start with or that you end up with.
msg32566 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-08-03 21:53
Or we can change the semantics of __file__ to always point to the source file.  I think that's acceptable behavior -- the current behavior seems more of a pain than necessary.

I also think that adding imp.reload() makes sense; I was just saying yesterday to a few fellow-Googler-Python-developers that I was already missing reload().
msg32567 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2007-08-03 22:50
Agreed on both, although what to do when there is no source but only bytecode?  Just say that __file__ will point to the source if available, otherwise to the bytecode?  That seems reasonable to me.

And imp.reload sounds good.  This whole bug report started because 'reload' was gone and I was trying to figure out the best way to replace its loss.

I know I am short on time and thus have no clue if I can help on these.  Should we add these to PEP 3100 so we don't forget, or do you think you might get to this?
msg59465 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-07 16:57
both sounds like a good idea.
msg59467 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-07 17:14
I've readded reload() as imp.reload() in r59825. I've also copied two
unit tests from trunk to py3k.
msg59484 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-07 20:19
I've added the __file__ patch in r59841. Is there another location that
needs to be fixed?
msg59485 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-01-07 20:37
Maybe just a doc update?
msg63890 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-03-18 05:01
Christian, I still see __file__ point to a .pyc file:

guido-macbookpro:~/p3 guido$ ./python.exe 
Python 3.0a3+ (py3k:61464M, Mar 17 2008, 16:36:53) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import string
>>> string.__file__

Please assign back to me if you've fixed this...
msg66800 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-05-13 19:34
This has apparently been fixed now.
msg76993 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-12-05 09:32
All open items in this issue seem to be addressed
Date User Action Args
2022-04-11 14:56:25adminsetgithub: 45253
2008-12-05 09:32:44georg.brandlsetstatus: open -> closed
messages: + msg76993
2008-06-25 17:20:23brett.cannonlinkissue3192 superseder
2008-05-13 19:34:52georg.brandlsetnosy: + georg.brandl
messages: + msg66800
2008-05-13 19:34:43georg.brandlsetassignee: christian.heimes -> gvanrossum
2008-03-18 05:01:29gvanrossumsetassignee: gvanrossum -> christian.heimes
messages: + msg63890
2008-01-07 20:37:51gvanrossumsetmessages: + msg59485
2008-01-07 20:19:29christian.heimessetmessages: + msg59484
2008-01-07 17:14:39christian.heimessetresolution: accepted
messages: + msg59467
2008-01-07 16:57:05christian.heimessetnosy: + christian.heimes
messages: + msg59465
2008-01-06 22:29:46adminsetkeywords: - py3k
versions: Python 3.0
2007-08-30 00:23:01gvanrossumsetversions: + Python 3.0
2007-07-29 03:24:55brett.cannoncreate