classification
Title: import of site.py fails on startup
Type: Stage:
Components: Versions: Python 3.0
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: benjamin.peterson Nosy List: alexandre.vassalotti, amaury.forgeotdarc, benjamin.peterson, brett.cannon, rupole
Priority: deferred blocker Keywords: needs review, patch

Created on 2008-07-04 03:40 by rupole, last changed 2008-09-05 23:27 by benjamin.peterson. This issue is now closed.

Files
File name Uploaded Description Edit
import-site.patch amaury.forgeotdarc, 2008-08-25 09:32
different_order.patch benjamin.peterson, 2008-08-25 14:37
make_modules_builtin.patch benjamin.peterson, 2008-09-05 22:14
make_modules_builtin2.patch benjamin.peterson, 2008-09-05 22:47
Messages (17)
msg69240 - (view) Author: Roger Upole (rupole) Date: 2008-07-04 03:40
In pythonrun.c, initstdio injects 'open' into builtins.  However,
initsite is called before initstdio and site.py uses open.
Running with -v, this traceback is printed:

Traceback (most recent call last):
  File "j:\python30\lib\site.py", line 518, in <module>
main()
  File "j:\python30\lib\site.py", line 501, in main
known_paths = addsitepackages(known_paths)
  File "j:\python30\lib\site.py", line 281, in addsitepackages
addsitedir(sitedir, known_paths)
  File "j:\python30\lib\site.py", line 178, in addsitedir
addpackage(sitedir, name, known_paths)
  File "j:\python30\lib\site.py", line 141, in addpackage
f = open(fullname, "rU")
NameError: global name 'open' is not defined
msg69252 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-07-04 11:47
This happens if the site-packages directory contains a .pth file.

This was unnoticed because the usual message
   'import site' failed; use -v for traceback
is never printed: sys.stderr is empty before the call to initstdio().
msg71913 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-08-25 09:32
Attached patch writes the "'import site' failed" message to libc stderr
instead of sys.stderr, which does not exist at this point.

The initial problem still remains: site.py cannot use the "open" builtin.
msg71922 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-08-25 13:40
We could:

1. Use _fileio._FileIO directly.
2. Use os.open.
3. Initialize the stdio streams before importing site. This seems like
the best option, although I haven't run all the tests yet.
msg71928 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-08-25 14:37
Here's my patch for the problem.
msg71931 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-08-25 14:54
Well, it would be interesting to know why r62778 has exactly the
opposite move.
Alexandre, do you remember doing it?
msg71933 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-08-25 15:02
It was probably done so that importing _bytesio would actually work in
io. IMO, it would be better to just compile _bytesio and _stringio into
the python binary.
msg71935 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-08-25 15:11
Benjamin is right. site.py imports the io module before _bytesio and
_stringio are available for import. Thus the python version of BytesIO
and StringIO is always used. There is an old thread about the issue at
http://mail.python.org/pipermail/python-3000/2007-December/011569.html

As for compiling _bytesio and _stringio into the main binary, you should
ask python-dev or python-3000.
msg71946 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-08-25 17:48
On Mon, Aug 25, 2008 at 6:40 AM, Benjamin Peterson
<report@bugs.python.org> wrote:
>
> Benjamin Peterson <musiccomposition@gmail.com> added the comment:
>
> We could:
>
> 1. Use _fileio._FileIO directly.
> 2. Use os.open.
> 3. Initialize the stdio streams before importing site. This seems like
> the best option, although I haven't run all the tests yet.
>

Or you could use _warnings which is already compiled into the binary.
msg71947 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-08-25 17:49
On Mon, Aug 25, 2008 at 12:48 PM, Brett Cannon <report@bugs.python.org> wrote:
>
> Brett Cannon <brett@python.org> added the comment:
>
> On Mon, Aug 25, 2008 at 6:40 AM, Benjamin Peterson
> <report@bugs.python.org> wrote:
>>
>> Benjamin Peterson <musiccomposition@gmail.com> added the comment:
>>
>> We could:
>>
>> 1. Use _fileio._FileIO directly.
>> 2. Use os.open.
>> 3. Initialize the stdio streams before importing site. This seems like
>> the best option, although I haven't run all the tests yet.
>>
>
> Or you could use _warnings which is already compiled into the binary.

How would that help the problem of site using open() when it can't?
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue3279>
> _______________________________________
>

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
msg71948 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-08-25 17:52
On Mon, Aug 25, 2008 at 10:49 AM, Benjamin Peterson
<report@bugs.python.org> wrote:
>
> Benjamin Peterson <musiccomposition@gmail.com> added the comment:
>
> On Mon, Aug 25, 2008 at 12:48 PM, Brett Cannon <report@bugs.python.org> wrote:
>>
>> Brett Cannon <brett@python.org> added the comment:
>>
>> On Mon, Aug 25, 2008 at 6:40 AM, Benjamin Peterson
>> <report@bugs.python.org> wrote:
>>>
>>> Benjamin Peterson <musiccomposition@gmail.com> added the comment:
>>>
>>> We could:
>>>
>>> 1. Use _fileio._FileIO directly.
>>> 2. Use os.open.
>>> 3. Initialize the stdio streams before importing site. This seems like
>>> the best option, although I haven't run all the tests yet.
>>>
>>
>> Or you could use _warnings which is already compiled into the binary.
>
> How would that help the problem of site using open() when it can't?
>>

Sorry, replied to the wrong email. It was meant as a reply to Amaury's
suggestion of writing out a comment that importing site failed to
stderr.
msg72629 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-09-05 21:20
IMO, it's unacceptable to be running Python code before the stdio
streams are set up. I believe that moving the site initialization after
the setting up the streams isn't as big a problem because the dynlibs
are on sys.path before site is run when python is installed. See
#586680. I will write to Python-dev.
msg72636 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-09-05 22:14
Here's a patch that builds _bytesio and _stringio right into the binary.
I think Windows build files will need to be updated, though.
msg72642 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-09-05 22:44
> Windows build files will need to be updated
Nothing to do here: they are already built-in, linked into Python30.dll,
and listed in PC/config.c

But shouldn't the two extension modules be removed from setup.py?
msg72643 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-09-05 22:47
Yes, indeed.
msg72647 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-09-05 23:17
Look good to me, and python-dev accepted the patch. So, go ahead and
commit it.
msg72650 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-09-05 23:27
Fixed in r66239.
History
Date User Action Args
2008-09-05 23:27:32benjamin.petersonsetstatus: open -> closed
resolution: accepted -> fixed
messages: + msg72650
2008-09-05 23:17:20alexandre.vassalottisetassignee: benjamin.peterson
resolution: accepted
messages: + msg72647
2008-09-05 22:47:44benjamin.petersonsetfiles: + make_modules_builtin2.patch
messages: + msg72643
2008-09-05 22:44:37amaury.forgeotdarcsetmessages: + msg72642
2008-09-05 22:14:51benjamin.petersonsetkeywords: + needs review
files: + make_modules_builtin.patch
messages: + msg72636
2008-09-05 21:20:52benjamin.petersonsetmessages: + msg72629
2008-09-04 01:09:24benjamin.petersonsetpriority: release blocker -> deferred blocker
2008-08-25 17:52:21brett.cannonsetmessages: + msg71948
2008-08-25 17:49:34benjamin.petersonsetmessages: + msg71947
2008-08-25 17:48:27brett.cannonsetmessages: + msg71946
2008-08-25 15:11:47alexandre.vassalottisetmessages: + msg71935
2008-08-25 15:02:36benjamin.petersonsetmessages: + msg71933
2008-08-25 14:54:30amaury.forgeotdarcsetnosy: + alexandre.vassalotti
messages: + msg71931
2008-08-25 14:37:26benjamin.petersonsetfiles: + different_order.patch
messages: + msg71928
2008-08-25 13:40:56benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg71922
2008-08-25 09:32:42amaury.forgeotdarcsetfiles: + import-site.patch
keywords: + patch
messages: + msg71913
2008-08-21 14:51:27benjamin.petersonsetpriority: critical -> release blocker
2008-07-04 16:51:41brett.cannonsetnosy: + brett.cannon
2008-07-04 14:34:03georg.brandlsetpriority: critical
2008-07-04 11:47:27amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg69252
2008-07-04 03:40:26rupolecreate