This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: urllib2.open failed to access a url when a perent directory of the url is permission denied
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: orsenthil Nosy List: foxkiller, orsenthil, python-dev, terry.reedy
Priority: normal Keywords:

Created on 2013-05-13 05:50 by foxkiller, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (9)
msg189097 - (view) Author: zhaoqifa (foxkiller) Date: 2013-05-13 05:50
urllib2.open failed to access a url when a perent directory of the url is permission denied.

I had a file in a ftp, the address is like ftp://host/home/work/123.txt which can be access by wget.
When I urllib2.open to access the file, it always tell me that:
urllib2.URLError: <urlopen error ftp error: 550 home: No such file or directory>

I found that the path /home is denied for anonymouse access but the path /home/work is not. 
urllib2.open try to call ftp.cwd("home") and then ftp.cwd("work") which failed when ftp.cwd("home").

I tried to fix this by changing "for dir in self.dirs: self.ftp.cwd(dir)" to "self.ftp.cwd(os.path.join(*self.dirs)" in urllib.py. And it works fine for this situation.
msg189310 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-05-15 21:28
Did you run the urllib test module after your code change?
Have you tried 3.3?
msg189423 - (view) Author: zhaoqifa (foxkiller) Date: 2013-05-17 07:15
I had run test by python ./urllib.py -t, test steps for accessing /etc/passwd, file:/etc/passwd, file://localhost/etc/passwd passed, steps for ftp://ftp.gnu.org/pub/README not tested because my work env is not free for internet accessing.

And, I have not tried version 3.3.
msg189754 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2013-05-21 13:38
Hello zhaoqifa, 

Your suggestion for the solution looks good to me. Instead of cd-ing to each dir, you cd to target dir when ftp url is accessed.

Two things to consider.

1) Will the above change have any security implications. It does not seem to me. But I did not write that code originally that way either.

2) On Ubuntu, I am unable to setup a directory based (ACL'ed) anonymous ftp server. The option is either anonymous from ftp root (/) which will map to my operating system folder (/srv/ftp) or non-anonymous. I wonder how your server is configured to allow anonymous from a particular point onwards.
msg190345 - (view) Author: zhaoqifa (foxkiller) Date: 2013-05-30 06:25
Sorry for replying so late.
"1) Will the above change have any security implications."
---It does not seem to me either.

"2) On Ubuntu, I am unable to setup a directory based (ACL'ed) anonymous ftp server. The option is either anonymous from ftp root (/) which will map to my operating system folder (/srv/ftp) or non-anonymous. I wonder how your server is configured to allow anonymous from a particular point onwards."
---I configured proftpd like this:
<Limit WRITE DIRS>
    DenyAll
</Limit>
<Directory /home/*>
        <Limit DIRS LIST NLIST>
        AllowAll
        </Limit>
</Directory>
And get message as bellow when login anonymous:
lftp perf00:~> cd /home
cd: Access failed: 550 /home: No such file or directory
lftp perf00:~> cd /home/work
cd ok, cwd=/home/work
lftp perf00:/home/work> cd /home/work/test
cd ok, cwd=/home/work/test
lftp perf00:/home/work/test> ls
drwxrwxr-x   8 work     work         4096 Jun 15  2011 pb
msg190449 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-06-01 15:27
New changeset 0a544bb539e6 by Senthil Kumaran in branch '2.7':
Fix #17967: For ftp urls CWD to target instead of hopping to each directory
http://hg.python.org/cpython/rev/0a544bb539e6

New changeset dbfbdf2b5c19 by Senthil Kumaran in branch '3.3':
Fix #17967: For ftp urls CWD to target instead of hopping to each directory
http://hg.python.org/cpython/rev/dbfbdf2b5c19

New changeset c1101f0d6c29 by Senthil Kumaran in branch 'default':
merge from 3.3
http://hg.python.org/cpython/rev/c1101f0d6c29
msg190450 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2013-06-01 15:42
Thanks Zhaoqifa for steps to configure the ftp with acl. I could reproduce it manually and fix it by changes to the urllib.request code.
Unfortunately, we dont have rigorous tests for ftp handling in urllib module. That's noted in test_urllib2net module too.

This bug is fixed in all branches.
msg190451 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2013-06-01 17:02
Reopening: This change is seeing problems with Windows Buildbot
http://buildbot.python.org/all/builders/x86%20XP-4%203.x/builds/8643/steps/test/logs/stdio
msg190495 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-06-02 19:00
New changeset e9d0fb934b46 by Senthil Kumaran in branch '2.7':
Fix #17967 - Fix related to regression on Windows.
http://hg.python.org/cpython/rev/e9d0fb934b46

New changeset f5906026a7e9 by Senthil Kumaran in branch '3.3':
Fix #17967 - Fix related to regression on Windows.
http://hg.python.org/cpython/rev/f5906026a7e9

New changeset adfec512fb32 by Senthil Kumaran in branch 'default':
merge from 3.3
http://hg.python.org/cpython/rev/adfec512fb32
History
Date User Action Args
2022-04-11 14:57:45adminsetgithub: 62167
2013-06-02 19:53:13orsenthilsetstatus: open -> closed
2013-06-02 19:00:51python-devsetmessages: + msg190495
2013-06-01 17:02:43orsenthilsetstatus: closed -> open

messages: + msg190451
2013-06-01 15:42:25orsenthilsetstatus: open -> closed
resolution: fixed
messages: + msg190450

stage: test needed -> resolved
2013-06-01 15:27:59python-devsetnosy: + python-dev
messages: + msg190449
2013-05-30 06:25:42foxkillersetmessages: + msg190345
2013-05-21 13:38:27orsenthilsetassignee: orsenthil
messages: + msg189754
versions: + Python 3.3, Python 3.4
2013-05-17 07:15:05foxkillersetmessages: + msg189423
2013-05-15 21:28:10terry.reedysetnosy: + terry.reedy, orsenthil

messages: + msg189310
stage: test needed
2013-05-13 05:50:22foxkillersetcomponents: + Library (Lib)
2013-05-13 05:50:09foxkillercreate