classification
Title: pkgutil.walk_packages fails on write-only directory in sys.path
Type: behavior Stage: needs patch
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: Quik_Foot, ned.deily, python-dev
Priority: normal Keywords: easy

Created on 2009-11-20 07:31 by Quik_Foot, last changed 2011-10-07 19:03 by python-dev. This issue is now closed.

Messages (9)
msg95533 - (view) Author: Jay Lugo (Quik_Foot) Date: 2009-11-20 07:31
I spend the better part of a day figuring out this relatively simple
problem with the help files in IDLE with python 2.6 on a mac OSX

I came about the solution by following the code errors all over
up and down the program files.

Eventual I came about the public folder ex..
Users/ME/Public

I spent time looking through the properties and settings tabs editing my
firewall settings and file sharing options (I reverted them when i fixed
the problem)
but i ended up removing all file sharing settings anyway.

I simply right clicked on the public folder and my permission was set
to "Write only(Drop Box)"
a simple change to "Reed and Write" fixed the IDLE error.

Hope i just saved some one some time.
msg95534 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2009-11-20 09:19
Can you be more specific about how to reproduce the problem you saw?  
Also, which version of Python were you using?  Please copy the information 
from the the first line of IDLE's Python Shell window, like so:

Python 2.6.1 (r261:67515, Jul  7 2009, 23:51:51) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
msg95535 - (view) Author: Jay Lugo (Quik_Foot) Date: 2009-11-20 11:17
Python 2.6.4
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin

From the best I can tell the error came about
when I updated my firewall/shared folder settings.
(I lost a computer on my home network due to my partner leaving me :( )
so I did a complete revamp of my network to close up any holes
and Bam no IDLE.

The solution as I said came about from simply changing the user access
on the public folder in User/ME/

I initially came across the error when attempting to get a modules list
and was returned a list of errors. One of them was the [errno 13].

Hope that clears things.
msg95544 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-11-20 14:00
A more specific issue title will also help others hitting the same
problem...hopefully I've guessed correctly as to what it should be.
msg95569 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2009-11-20 21:45
>I initially came across the error when attempting to get a modules list
>and was returned a list of errors. One of them was the [errno 13].

OK, now that makes more sense!

The problem is that somehow you had a write-only directory ("~/Public/Drop 
Box") on your python module search path.  If you type the following commands 
in IDLE or in the interpreter:

import sys
sys.path

you will likely see the "Drop Box" directory in the list of paths (assuming 
things haven't changed).

When you run the "modules" help command, the python help module, pydoc, uses 
the pkgutil module to search the entire list of directories in sys.path and 
attempts to import each package it finds in each directory to print the 
module's help information (btw, there are no separate help files).

While the pkgutil.walk_packages function tries to handle most errors, it 
isn't prepared to handle the case of a write-only directory on sys.path and 
that's the root cause of the error of the error you saw.  The following test 
case demonstrates the problem:

import os, pkgutil, sys

dirname_wo = os.tempnam()
os.mkdir(dirname_wo, 0222)  # write permission only
try:
    sys.path.insert(0, dirname_wo)
    def onerror(pkgname):
        print('onerror called for package %s' % pkgname)
    for importer, modname, ispkg in pkgutil.walk_packages(onerror=onerror):
        print modname
finally:
    os.rmdir(dirname_wo)

The long-term solution is to have pkgutil.walk_packages protect itself 
against os.listdir errors.  A patch is needed for that.

In your case, though, you should check for and remove whatever is adding 
"Drop Box" to your Python sys.path.  Perhaps you have an out-of-date export 
of PYTHONPATH in a shell profile (.bash_profile, etc)?  Or something left 
over in a site-packages pth file (perhaps by trying to install a package 
from the "Drop Box" directory)?  By changing the permissions as you did, you 
worked around the problem but that's not the right solution and you could be 
compromising the security of your drop box file.
msg95571 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2009-11-20 21:54
More precisely: you should check for and remove whatever is adding 
"Drop Box" or a directory within it to your Python sys.path.
msg145041 - (view) Author: Roundup Robot (python-dev) Date: 2011-10-06 21:43
New changeset 096b010ae90b by Ned Deily in branch '2.7':
Issue #7367: Add test case to test_pkgutil for walking path with
http://hg.python.org/cpython/rev/096b010ae90b

New changeset 1449095397ae by Ned Deily in branch '2.7':
Issue #7367: Fix pkgutil.walk_paths to skip directories whose
http://hg.python.org/cpython/rev/1449095397ae

New changeset a1e6633ef3f1 by Ned Deily in branch '3.2':
Issue #7367: Add test case to test_pkgutil for walking path with
http://hg.python.org/cpython/rev/a1e6633ef3f1

New changeset 77bac85f610a by Ned Deily in branch '3.2':
Issue #7367: Fix pkgutil.walk_paths to skip directories whose
http://hg.python.org/cpython/rev/77bac85f610a

New changeset 5a4018570a59 by Ned Deily in branch '3.2':
Issue #7367: add NEWS item.
http://hg.python.org/cpython/rev/5a4018570a59

New changeset 0408001e4765 by Ned Deily in branch 'default':
Issue #7367: merge from 3.2
http://hg.python.org/cpython/rev/0408001e4765
msg145043 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-10-06 22:00
The applied changesets correct pkgutil's walk_packages for "classic" imports to ignore unreadable directories the same way that the interpreter's import does.  With this fix to pkgutil, pydoc -k also no longer fails in this case.  Applied in 2.7 (for 2.7.3), 3.2 (for 3.2.3), and default (for 3.3.0).
msg145126 - (view) Author: Roundup Robot (python-dev) Date: 2011-10-07 19:03
New changeset 17b5afd321a8 by Ned Deily in branch '2.7':
Issue #7367: Ensure test directory always gets removed.
http://hg.python.org/cpython/rev/17b5afd321a8

New changeset ff72f76dcf43 by Ned Deily in branch '3.2':
Issue #7367: Ensure test directory always gets removed.
http://hg.python.org/cpython/rev/ff72f76dcf43
History
Date User Action Args
2012-02-15 18:56:21ned.deilylinkissue14022 superseder
2011-10-07 19:03:17python-devsetmessages: + msg145126
2011-10-07 06:23:23ned.deilylinkissue12696 superseder
2011-10-07 06:08:33ned.deilylinkissue11914 superseder
2011-10-06 22:00:32ned.deilysetstatus: open -> closed
resolution: fixed
messages: + msg145043
2011-10-06 21:43:08python-devsetnosy: + python-dev
messages: + msg145041
2011-08-06 03:00:19ned.deilysetversions: + Python 3.3, - Python 3.1
2011-08-06 02:59:46ned.deilysetassignee: ned.deily
2010-11-21 08:14:06ned.deilysetkeywords: + easy
stage: needs patch
versions: - Python 2.6
2009-11-20 21:54:03ned.deilysetmessages: + msg95571
2009-11-20 21:46:00ned.deilysettitle: IDLE OSError [errno 13] permission denied accessing help files -> pkgutil.walk_packages fails on write-only directory in sys.path
messages: + msg95569
components: + Library (Lib), - IDLE
versions: + Python 3.1, Python 2.7, Python 3.2
2009-11-20 14:01:17r.david.murraysetnosy: - r.david.murray
2009-11-20 14:00:36r.david.murraysetpriority: normal
title: OSError [errno 13] permission denied -> IDLE OSError [errno 13] permission denied accessing help files
nosy: + r.david.murray

messages: + msg95544

type: resource usage -> behavior
2009-11-20 11:17:26Quik_Footsetmessages: + msg95535
2009-11-20 09:19:09ned.deilysetnosy: + ned.deily
messages: + msg95534
2009-11-20 07:31:23Quik_Footcreate