classification
Title: CGI HTTP server not running scripts from subdirectories
Type: behavior Stage: resolved
Components: Versions: Python 3.5, Python 3.4, Python 3.3, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, k.s.solnushkin, ned.deily, python-dev, zbyrne
Priority: high Keywords: easy, patch

Created on 2014-04-21 19:05 by k.s.solnushkin, last changed 2014-07-13 05:31 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
test.py k.s.solnushkin, 2014-04-21 19:05 A trivial CGI script to test HTTP server functionality
21323.patch zbyrne, 2014-07-09 00:54 review
Messages (10)
msg216960 - (view) Author: Konstantin S. Solnushkin (k.s.solnushkin) Date: 2014-04-21 19:05
Somewhere between Python 3.3 and 3.4, a bug was introduced that forbids the "http.server" module, working in CGI server mode, to run scripts residing in subdirectories.

This will break existing software that relies on this feature.

How to reproduce the bug:

1. Create a temporary directory and enter it.
2. Create a directory "cgi-bin", and then directory "test" inside "cgi-bin".
3. Create a file "test.py" in "cgi-bin/test" with the following contents (see also attachment to this bug report):

print("""Content-type: text/plain

CGI script executed successfully!
""")

4. When run, it should print the following:

Content-type: text/plain

CGI script executed successfully!

5. Now, run Python 3.3 in CGI HTTP server mode:

c:\Python33\python.exe -m http.server --cgi 8000

A request to "http://localhost:8000/cgi-bin/test/test.py" then produces the following in the HTTP server log:

Serving HTTP on 0.0.0.0 port 8000 ...
127.0.0.1 - - [21/Apr/2014 22:59:11] "GET /cgi-bin/test/test.py HTTP/1.0" 200 -
127.0.0.1 - - [21/Apr/2014 22:59:11] command: c:\Python33\python.exe -u C:\TMP\cgi-bin\test\test.py ""
127.0.0.1 - - [21/Apr/2014 22:59:11] CGI script exited OK

6. Now, try this with Python 3.4, and the request will fail with the following in the log:

C:\TMP>c:\Python34\python.exe -m http.server --cgi 8000
Serving HTTP on 0.0.0.0 port 8000 ...
127.0.0.1 - - [21/Apr/2014 23:02:38] code 403, message CGI script is not a plain file ('/cgi-bin/test')
127.0.0.1 - - [21/Apr/2014 23:02:38] "GET /cgi-bin/test/test.py HTTP/1.0" 403 -

This _could_ be related to the change introduced by issue 19435, although I am not sure.

Tested with Windows XP SP3.
msg222402 - (view) Author: Konstantin S. Solnushkin (k.s.solnushkin) Date: 2014-07-06 11:53
Hi, I am curious about the fate of this issue -- whether it will be recognised as a bug (possibly a regression bug). Remember, it worked in Python 3.3 but stopped working in 3.4.
msg222421 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-06 22:06
The change in behavior is definitely caused by the changes for Issue19435.  The code added there does not work correctly if there are nested directories in the "cgi-bin" directory; for one thing, the initial os.path.isdir(scriptdir) test in run_cgi is failing because the path in nextdir is missing the dir component: it's testing /path/to/test instead of /path/to/cgi-bin/test.  I didn't look further.  Along with a fix, there should be a test for nested directories.  And decide whether to fix the regression it presumably caused in the security branches as well.

http://hg.python.org/cpython/file/d25ae22cc992/Lib/http/server.py#l1009
msg222422 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-06 22:43
BTW, if someone is looking for an issue to help out with, this might be a good choice.
msg222594 - (view) Author: Zach Byrne (zbyrne) * Date: 2014-07-09 00:54
Hi, I'm new. I wrote a test for nested directories under cgi-bin and got that to pass without failing the test added for 19435 by undoing most of the changes to run_cgi() but building path from the values in self.cgi_info. Thoughts?
msg222603 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-09 07:31
Zach, thanks for the patch and the test. Someone will review it in the coming days. In the meantime, if you haven't already, please review and submit the Python Contributor's Agreement: https://www.python.org/psf/contrib/contrib-form/ otherwise we won't be able to use your contribution.
msg222729 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-11 07:49
The fix and test look good to me.  I'll apply it after Zach submits the contributor agreement.
msg222793 - (view) Author: Zach Byrne (zbyrne) * Date: 2014-07-11 20:46
Done and done.
msg222910 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-13 05:21
New changeset d367ea865ea4 by Ned Deily in branch '2.7':
Issue #21323: Fix CGIHTTPServer to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/d367ea865ea4

New changeset 4de94641ba3e by Ned Deily in branch '3.2':
Issue #21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/4de94641ba3e

New changeset b957f475e41e by Ned Deily in branch '3.3':
Issue #21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/b957f475e41e

New changeset 385f4406dc26 by Ned Deily in branch '3.4':
Issue #21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/385f4406dc26

New changeset 22e5a85ba840 by Ned Deily in branch 'default':
Issue #21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/22e5a85ba840
msg222912 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-07-13 05:31
Since the original changes for Issue19435 were also applied to branches in security fix mode, the fix here is applied to them as well, with the exception of 3.1 which has subsequently moved to end-of-life status.  Applied for release in 2.7.9, 3.2.6, 3.3.6, 3.4.2, and 3.5.0.  Thanks again, Konstantin and Zach.
History
Date User Action Args
2014-07-13 05:31:22ned.deilysetstatus: open -> closed
versions: + Python 3.2, Python 3.3
messages: + msg222912

resolution: fixed
stage: commit review -> resolved
2014-07-13 05:21:15python-devsetnosy: + python-dev
messages: + msg222910
2014-07-11 20:46:16zbyrnesetmessages: + msg222793
2014-07-11 07:49:33ned.deilysetmessages: + msg222729
stage: needs patch -> commit review
2014-07-09 07:31:17ned.deilysetmessages: + msg222603
2014-07-09 00:54:07zbyrnesetfiles: + 21323.patch

nosy: + zbyrne
messages: + msg222594

keywords: + patch
2014-07-06 22:43:10ned.deilysetkeywords: + easy, - buildbot

messages: + msg222422
2014-07-06 22:06:00ned.deilysetpriority: normal -> high

versions: + Python 2.7, Python 3.5
keywords: + buildbot
nosy: + ned.deily, benjamin.peterson

messages: + msg222421
stage: needs patch
2014-07-06 11:53:57k.s.solnushkinsetmessages: + msg222402
2014-04-21 19:05:31k.s.solnushkincreate