This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_getcwd_long_pathnames (in test_posix) kills NetBSD
Type: behavior Stage:
Components: Versions: Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: trent Nosy List: neologix, python-dev, skrah, trent, vstinner
Priority: normal Keywords: patch

Created on 2012-08-22 14:12 by trent, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
getcwd_netbsd.patch trent, 2012-08-24 15:25 review
Messages (7)
msg168885 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-08-22 14:12
The 2.7 builds of both my NetBSD slaves keep hanging on test_posix (and eventually time out after an hour).  Running the test manually:


netbsd51-x86-1$ pwd                       
/home/cpython/2.7
netbsd51-x86-1$ b/test/regrtest.py -uall -rwW -v test_posix                   <
== CPython 2.7.3+ (2.7:c1c45755397b, Aug 22 2012, 14:05:02) [GCC 4.1.3 20080704 prerelease (NetBSD nb2 20081120)]
==   NetBSD-5.1.2-i386-32bit-ELF little-endian
==   /home/cpython/2.7/build/test_python_10630
Testing with flags: sys.flags(debug=0, py3k_warning=1, division_warning=1, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, tabcheck=2, verbose=0, unicode=0, bytes_warning=0, hash_randomization=1)
Using random seed 2040018
test_posix
testNoArgFunctions (test.test_posix.PosixTester) ... ok
test_access (test.test_posix.PosixTester) ... ok
test_chdir (test.test_posix.PosixTester) ... ok
test_chflags (test.test_posix.PosixTester) ... ok
test_chown (test.test_posix.PosixTester) ... ok
test_confstr (test.test_posix.PosixTester) ... ok
test_dup (test.test_posix.PosixTester) ... ok
test_dup2 (test.test_posix.PosixTester) ... ok
test_fchown (test.test_posix.PosixTester) ... ok
test_fdopen (test.test_posix.PosixTester) ... ok
test_fstat (test.test_posix.PosixTester) ... ok
test_fstatvfs (test.test_posix.PosixTester) ... ok
test_ftruncate (test.test_posix.PosixTester) ... ok
test_getcwd_long_pathnames (test.test_posix.PosixTester) ... load: 0.50  cmd: python 10630 [runnable] 0.27u 13.07s 46% 10976k
load: 0.87  cmd: python 10630 [runnable] 0.27u 94.24s 98% 10976k
^C^C

Placeholder issue at the moment.

....although this looks pretty dodgy:


                    os.chdir(dirname)
                    try:
                        os.getcwd()
                        if current_path_length < 4099:
                            _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
msg168902 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-08-22 17:51
The timeouts on NetBSD and OpenBSD are very likely related to
bugs in the user-space thread libraries. See also #8714.

I would recommend running these bots --without-threads.
msg168994 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2012-08-24 10:13
This doesn't look thread-related:
"""
load: 0.50  cmd: python 10630 [runnable] 0.27u 13.07s 46% 10976k
load: 0.87  cmd: python 10630 [runnable] 0.27u 94.24s 98% 10976k
"""

See the huge system time?
This probably means that the kernel is struggling with path name
resolution, although that's surprising because 4099 is a reasonable
limit.
Do you have something like strace or truss on NetBSD?
The first thing to try could be to lower the limit, and see what
happens, but it's really likely a FS scalibility limit you're hitting
(altough, once again, that's really surprising given the reasonable
limit). Are you running those under VMs?
msg169012 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-08-24 12:53
Hacked the test to print out info each loop:


test_getcwd_long_pathnames (test.test_posix.PosixTester) ... [0/0] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[1/57] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[2/114] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[3/171] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[4/228] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[5/285] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[6/342] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[7/399] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[8/456] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[9/513] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[10/570] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[11/627] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[12/684] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[13/741] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[14/798] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[15/855] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[16/912] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[17/969] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[18/1026] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[19/1083] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[20/1140] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[21/1197] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[22/1254] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[23/1311] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[24/1368] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[25/1425] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[26/1482] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[27/1539] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[28/1596] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[29/1653] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[30/1710] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[31/1767] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[32/1824] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[33/1881] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[34/1938] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[35/1995] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[36/2052] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[37/2109] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[38/2166] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[39/2223] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[40/2280] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[41/2337] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[42/2394] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[43/2451] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[44/2508] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[45/2565] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[46/2622] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[47/2679] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[48/2736] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[49/2793] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[50/2850] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[51/2907] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[52/2964] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[53/3021] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[54/3078] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[55/3135] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[56/3192] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[57/3249] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[58/3306] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[59/3363] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[60/3420] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[61/3477] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[62/3534] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[63/3591] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[64/3648] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[65/3705] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[66/3762] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[67/3819] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[68/3876] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[69/3933] getcwd-test-directory-0123456789abcdef-01234567890abcdef
[70/3990] getcwd-test-directory-0123456789abcdef-01234567890abcdef
^C^C
load: 1.08  cmd: python 4486 [runnable] 0.07u 14.38s 40% 10012k
load: 1.08  cmd: python 4486 [runnable] 0.07u 14.91s 40% 10012k
load: 1.08  cmd: python 4486 [runnable] 0.07u 15.24s 42% 10012k
load: 1.08  cmd: python 4486 [runnable] 0.07u 15.40s 42% 10012k
Terminated 

(Had to kill it from a different process, mind you; the ctrl-Cs never get caught.)

Yeah, it's a VM.  Also, it looks like 3.x uses 1027 instead of 4099.

It's definitely odd that it ends up spinning in the kernel at ~3990.  I'll do some more digging.
msg169027 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-08-24 15:25
Looks like this is a duplicate of issue 9185.  The fix used for that just needs to be extended to cover NetBSD as well.  See proposed patch.  I'll commit this to 2.7 in a day or two if there are no objections.

Antoine: added you as nosy as you were involved with the original patch.
msg169119 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-08-25 09:21
I completely forgot about that issue. The patch looks good to me.
msg169375 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-08-29 13:21
New changeset bfa5d0ddfbeb by Trent Nelson in branch '2.7':
Issue #15765: Fix quirky NetBSD getcwd() behaviour.
http://hg.python.org/cpython/rev/bfa5d0ddfbeb
History
Date User Action Args
2022-04-11 14:57:35adminsetgithub: 59969
2012-08-30 05:32:25trentsetstatus: open -> closed
2012-08-29 13:21:34python-devsetnosy: + python-dev
messages: + msg169375
2012-08-25 09:21:00skrahsetmessages: + msg169119
2012-08-24 18:07:15neologixlinkissue11225 superseder
2012-08-24 15:25:44trentsetfiles: + getcwd_netbsd.patch
resolution: duplicate
messages: + msg169027

keywords: + patch
2012-08-24 12:53:44trentsetmessages: + msg169012
2012-08-24 10:13:17neologixsetmessages: + msg168994
2012-08-23 16:20:39vstinnersetnosy: + vstinner, neologix
2012-08-22 17:51:50skrahsetnosy: + skrah
messages: + msg168902
2012-08-22 14:12:15trentcreate