classification
Title: shutil.rmtree doesn't work correctly on FreeBSD.
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5, Python 3.4
process
Status: pending Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, koobs, negval, serhiy.storchaka, sumpfralle, vstinner
Priority: normal Keywords:

Created on 2015-01-29 04:52 by negval, last changed 2018-11-02 09:19 by serhiy.storchaka.

Messages (10)
msg234946 - (view) Author: Denis Sukhonin (negval) Date: 2015-01-29 04:52
shutil.rmtree doesn't work correctly on FreeBSD 9.1.

For example if I create a path /tmp/test and try to remove it, I get an exception:

>>> shutil.rmtree('/tmp/test')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/shutil.py", line 463, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/local/lib/python3.4/shutil.py", line 385, in _rmtree_safe_fd
    onerror(os.listdir, path, sys.exc_info())
  File "/usr/local/lib/python3.4/shutil.py", line 382, in _rmtree_safe_fd
    names = os.listdir(topfd)
OSError: [Errno 22] Invalid argument: '/tmp/test'

---

shutil._use_fd_functions has value True. When I change it to False, the shutil.rmtree works perfecty.

Version info:
>>> print(sys.version)
3.4.2 (default, Dec 22 2014, 21:56:20) 
[GCC 4.2.1 20070831 patched [FreeBSD]]
>>> print(sys.platform)
freebsd9

$ uname -r
9.1-RELEASE
msg234959 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-29 09:54
topfd is '/tmp/test'. But it should be file descriptor, an integer. What os.open('/tmp/test', os.O_RDONLY) returns?
msg234960 - (view) Author: Denis Sukhonin (negval) Date: 2015-01-29 10:57
It returns an integer.

>>> import os
>>> os.open('/tmp/test', os.O_RDONLY)
3
msg234964 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-29 12:01
Does os.listdir(os.open('/tmp/test', os.O_RDONLY)) work?
msg234965 - (view) Author: Denis Sukhonin (negval) Date: 2015-01-29 12:50
No, it throws 22.

>>> os.listdir(os.open('/tmp/test', os.O_RDONLY))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
msg234966 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-29 12:54
>> Does os.listdir(os.open('/tmp/test', os.O_RDONLY)) work?

> No, it throws 22.

Oh, too bad. Maybe we can detect this at Python compilation. If not, rmtree() should detect the failure and switch back to the unsafe implementation (which uses paths, not file descriptors).
msg234973 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-29 14:08
What if add a slash at the end of the path?

os.listdir(os.open('/tmp/test/', os.O_RDONLY))
msg234977 - (view) Author: Denis Sukhonin (negval) Date: 2015-01-29 16:06
The same problem.

>>> os.listdir(os.open('/tmp/test/', os.O_RDONLY))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
msg242688 - (view) Author: (sumpfralle) Date: 2015-05-06 16:49
I experience the same issue on a virtualized server (based on Virtuozzo) with Linux kernel 2.6.32.

The following command fails:

 echo "import os; os.listdir(os.open('/tmp', os.O_RDONLY))" | python3
 Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
 OSError: [Errno 22] Invalid argument
msg319384 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-06-12 14:01
I can't reproduce the problem on FreeBSD 11.1. If it is reproducible only on old versions, we can close this issue. If it depends on specific configuration, it is still a Python issue.
History
Date User Action Args
2018-11-02 09:19:24serhiy.storchakasetstatus: open -> pending
2018-06-12 14:01:43serhiy.storchakasetmessages: + msg319384
2018-06-12 10:36:38giampaolo.rodolasetnosy: + giampaolo.rodola
2015-05-06 16:49:30sumpfrallesetnosy: + sumpfralle
messages: + msg242688
2015-01-29 16:06:41negvalsetmessages: + msg234977
2015-01-29 14:08:12serhiy.storchakasetmessages: + msg234973
2015-01-29 13:09:37koobssetnosy: + koobs
2015-01-29 12:55:00vstinnersetversions: + Python 3.5
2015-01-29 12:54:49vstinnersetnosy: + vstinner
messages: + msg234966
2015-01-29 12:50:41negvalsetmessages: + msg234965
2015-01-29 12:01:56serhiy.storchakasetmessages: + msg234964
2015-01-29 10:57:54negvalsetmessages: + msg234960
2015-01-29 09:54:25serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg234959
2015-01-29 04:52:53negvalcreate