classification
Title: os.getcwd fails for long path names on linux
Type: Stage:
Components: Library (Lib) Versions: Python 2.4, Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, boya, christian.heimes, exarkun, facundobatista, georg.brandl, giampaolo.rodola, gregory.p.smith, nbm, smurfix
Priority: normal Keywords: easy, patch

Created on 2008-04-30 00:01 by exarkun, last changed 2009-09-02 02:59 by boya. This issue is now closed.

Files
File name Uploaded Description Edit
python-getcwd-dual-strategy.patch nbm, 2008-05-10 13:38 Patch for stack/malloc dual strategy for getcwd
python-getcwd-malloconly.patch nbm, 2008-05-10 13:41 Patch for using malloc exclusively with getcwd
python-getcwd-test_longpathnames.patch nbm, 2008-05-10 14:54 Patch to add a test for long path names with getcwd
Messages (10)
msg65987 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2008-04-30 00:01
$ python -c "print len('`pwd`'); import os; print os.getcwd()"
1174
Traceback (most recent call last):
  File "<string>", line 1, in ?
OSError: [Errno 34] Numerical result out of range
$

The getcwd man page documents the ERANGE failure and suggests that the
calling application allocate a larger buffer and try again.
msg66021 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-04-30 20:17
Why isn't the buffer length MAX_PATH anyway?
msg66022 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-04-30 20:19
It should be MAX_PATH. All path related functions should use MAX_PATH as
buffer size, maybe even MAX_PATH+1
msg66370 - (view) Author: Matthias Urlichs (smurfix) * Date: 2008-05-07 20:01
MAX_PATH is a compile time constant which, like FD_BITS for select(),
may be too small for the system you're ultimately running on.

Using that as default initial size is OK, but handling ERANGE is still a
very good idea.
msg66519 - (view) Author: Neil Blakey-Milner (nbm) Date: 2008-05-10 13:38
This affects OS X too.

I'm attaching two patches - one uses malloc to build a bigger string if
the existing implementation returns ERANGE, and the other one uses
malloc from the start.  This was done on the theory that stack memory
use would be better/more efficient.  However, based on testing, there's
no real difference - 16.8usec vs. 17usec - on a rarely-used function.
msg66531 - (view) Author: Neil Blakey-Milner (nbm) Date: 2008-05-10 14:54
Here's a patch to add a test that fails with Python 2.5 on OS X, but
passes with either of these patches.
msg68563 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-06-22 13:37
Went for the malloc only patch. Just fixed a small detail (weird corner
case if malloc returned NULL first time, res will be unassigned).

The test could be better (no necessity of using a recursive function, it
could be done with a while), but it works.

Commited in r64452.
msg92145 - (view) Author: Boya Sun (boya) Date: 2009-09-01 22:46
This bug occurred in posix_getcwd() and is being fixed. 
Should the following code in posix_getcwdu() also be fixed the same way?

posix_getcwdu(PyObject *self, PyObject *noargs)
{
	char buf[1026];
        ...
#if defined(PYOS_OS2) && defined(PYCC_GCC)
	res = _getcwd2(buf, sizeof buf);
#else
	res = getcwd(buf, sizeof buf);
#endif
        ...
}

In my opinion, the fixed length buf should be discarded and instead
allocate memory to buf as needed (as the fix code in posix_getcwd()),
since getcwd() does not have a maximum anymore.
msg92150 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-09-02 01:38
Please open a new issue; and a patch is welcome.
msg92152 - (view) Author: Boya Sun (boya) Date: 2009-09-02 02:59
Amaury,

Created issue 6817 with a patch.
History
Date User Action Args
2009-09-02 02:59:47boyasetmessages: + msg92152
2009-09-02 01:38:13amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg92150
2009-09-01 22:46:54boyasetnosy: + boya
messages: + msg92145
2008-06-22 13:37:12facundobatistasetstatus: open -> closed
nosy: + facundobatista
resolution: fixed
messages: + msg68563
2008-05-10 14:54:40nbmsetfiles: + python-getcwd-test_longpathnames.patch
messages: + msg66531
2008-05-10 13:41:21nbmsetfiles: + python-getcwd-malloconly.patch
2008-05-10 13:38:16nbmsetfiles: + python-getcwd-dual-strategy.patch
nosy: + nbm
messages: + msg66519
keywords: + patch
versions: + Python 2.5
2008-05-07 20:01:58smurfixsetnosy: + smurfix
messages: + msg66370
2008-05-02 17:09:42gregory.p.smithsetpriority: normal
nosy: + gregory.p.smith
keywords: + easy
2008-04-30 20:19:51christian.heimessetnosy: + christian.heimes
messages: + msg66022
2008-04-30 20:17:57georg.brandlsetnosy: + georg.brandl
messages: + msg66021
2008-04-30 13:12:00giampaolo.rodolasetnosy: + giampaolo.rodola
2008-04-30 00:01:51exarkuncreate