This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: '0 -> /dev/null' is lost
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Geass-LL, JelleZijlstra, eryksun, javabrett, shihai1991, sxt1001
Priority: normal Keywords:

Created on 2022-01-21 11:55 by sxt1001, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
fd.py sxt1001, 2022-01-21 12:01
fd2.py sxt1001, 2022-01-21 12:03
python2.7.13-out.png sxt1001, 2022-01-21 12:04
python3.10.0-out.png sxt1001, 2022-01-21 12:05
python3.4.6-out.png sxt1001, 2022-01-21 12:05
Messages (8)
msg411113 - (view) Author: tongxiaoge (sxt1001) Date: 2022-01-21 11:55
I found a problem in the environment of python2 and python3, as follows:

in python2:
```
VM-0-13-suse:~ # python2 -V
Python 2.7.13
VM-0-13-suse:~ #
VM-0-13-suse:~ # ps -aux|grep python2
root     29414  0.1  0.7  37096  6632 pts/3    S+   19:41   0:00 python2 fd2.py
root     29442  0.0  0.1  10540  1656 pts/5    S+   19:41   0:00 grep --color=auto python2
VM-0-13-suse:~ #
VM-0-13-suse:~ # ps -aux|grep sleep
root     29415  0.0  0.1   5760  1256 pts/3    S+   19:41   0:00 sleep 12345
root     29451  0.0  0.1  10540  1648 pts/5    S+   19:41   0:00 grep --color=auto sleep
VM-0-13-suse:~ #
VM-0-13-suse:~ # ls -l /proc/29415/fd
total 0
lrwx------ 1 root root 64 Jan 21 19:41 0 -> /dev/null
lrwx------ 1 root root 64 Jan 21 19:41 1 -> /dev/pts/3
lrwx------ 1 root root 64 Jan 21 19:41 2 -> /dev/pts/3
VM-0-13-suse:~ #
VM-0-13-suse:~ # kill -9 29415 29414
VM-0-13-suse:~ #
VM-0-13-suse:~ # ps -aux|grep python2
root     29551  0.1  0.7  37096  6632 pts/3    S+   19:42   0:00 python2 fd.py
root     29564  0.0  0.1  10540  1608 pts/5    S+   19:42   0:00 grep --color=auto python2
VM-0-13-suse:~ # ps -aux|grep sleep
root     29552  0.0  0.1   5760  1260 pts/3    S+   19:42   0:00 sleep 12345
root     29576  0.0  0.1  10540  1628 pts/5    S+   19:42   0:00 grep --color=auto sleep
VM-0-13-suse:~ #
VM-0-13-suse:~ # ls -l /proc/29552/fd
total 0
lrwx------ 1 root root 64 Jan 21 19:42 0 -> /dev/null
lrwx------ 1 root root 64 Jan 21 19:42 1 -> /dev/pts/3
lrwx------ 1 root root 64 Jan 21 19:42 2 -> /dev/pts/3
lrwx------ 1 root root 64 Jan 21 19:42 3 -> /dev/null
VM-0-13-suse:~ #
```

in python3.4.6:
```
VM-0-13-suse:~ # python3 -V
Python 3.4.6
VM-0-13-suse:~ # ps -aux|grep python3
root     29086  0.1  0.9  33628  8136 pts/3    S+   19:39   0:00 python3 fd2.py
root     29143  0.0  0.1  10540  1616 pts/5    S+   19:39   0:00 grep --color=auto python3
VM-0-13-suse:~ #
VM-0-13-suse:~ # ps -aux|grep sleep
root     29087  0.0  0.1   5760  1284 pts/3    S+   19:39   0:00 sleep 12345
root     29164  0.0  0.1  10540  1620 pts/5    S+   19:39   0:00 grep --color=auto sleep
VM-0-13-suse:~ #
VM-0-13-suse:~ # ls -l /proc/29087/fd
total 0
lrwx------ 1 root root 64 Jan 21 19:39 1 -> /dev/pts/3
lrwx------ 1 root root 64 Jan 21 19:39 2 -> /dev/pts/3
VM-0-13-suse:~ #
VM-0-13-suse:~ # kill -9 29087 29086
VM-0-13-suse:~ #
VM-0-13-suse:~ # ps -aux|grep python3
root     29257  0.3  0.9  33628  8076 pts/3    S+   19:40   0:00 python3 fd.py
root     29270  0.0  0.1  10540  1620 pts/5    S+   19:40   0:00 grep --color=auto python3
VM-0-13-suse:~ #
VM-0-13-suse:~ # ps -aux|grep sleep
root     29258  0.0  0.1   5760  1264 pts/3    S+   19:40   0:00 sleep 12345
root     29281  0.0  0.1  10540  1628 pts/5    S+   19:40   0:00 grep --color=auto sleep
VM-0-13-suse:~ #
VM-0-13-suse:~ # ls -l /proc/29258/fd
total 0
lrwx------ 1 root root 64 Jan 21 19:40 0 -> /dev/null
lrwx------ 1 root root 64 Jan 21 19:40 1 -> /dev/pts/3
lrwx------ 1 root root 64 Jan 21 19:40 2 -> /dev/pts/3
```

in python3.10.0:
```
bash-5.1# python3 -V
Python 3.10.0
bash-5.1#
bash-5.1# ps -aux|grep python3
root     28688  0.0  0.9  11664  8732 ?        S+   11:36   0:00 python3 fd2.py
root     28725  0.0  0.2   6408  2216 ?        S+   11:36   0:00 grep python3
bash-5.1#
bash-5.1# ps -aux|grep sleep
root     28694  0.0  0.1   5524   908 ?        S+   11:36   0:00 sleep 12345
root     28729  0.0  0.2   6408  2276 ?        S+   11:36   0:00 grep sleep
bash-5.1#
bash-5.1# ls -l /proc/28694/fd
total 0
lrwx------ 1 root root 64 Jan 21 11:36 1 -> /dev/pts/3
lrwx------ 1 root root 64 Jan 21 11:36 2 -> /dev/pts/3
bash-5.1#
bash-5.1#
bash-5.1# kill -9 28694 28688
bash-5.1#
bash-5.1# ps -aux|grep python3
root     28846  0.5  0.9  11680  8428 ?        S+   11:37   0:00 python3 fd.py
root     28854  0.0  0.2   6408  2064 ?        S+   11:37   0:00 grep python3
bash-5.1#
bash-5.1# ps -aux|grep sleep
root     28847  0.0  0.0   5524   872 ?        S+   11:37   0:00 sleep 12345
root     28863  0.0  0.2   6408  2220 ?        S+   11:37   0:00 grep sleep
bash-5.1#
bash-5.1# ls -l /proc/28847/fd
total 0
lrwx------ 1 root root 64 Jan 21 11:37 0 -> /dev/null
lrwx------ 1 root root 64 Jan 21 11:37 1 -> /dev/pts/3
lrwx------ 1 root root 64 Jan 21 11:37 2 -> /dev/pts/3
bash-5.1#
```

When we execute the script fd2.py in the python 3 environment, we can find that  '0 -> /dev/null' is lost. I wonder if this is a bug in Python3 or a new feature? If it is a bug, how to fix it? I will look forward to your reply very much.
msg411138 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-01-21 15:43
This sounds like a consequence of PEP 446.
msg411144 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-01-21 16:37
That's not a bug, but a deliberate choice. In Python 3, file descriptors are no longer inherited by default by child processes (fork+exec). Jelle is right, it was changed by my PEP 446.

You must use pass_fds=[fd] parameter of subprocess.

Or at least, make the file descriptor inheritable (worse solution).
msg411224 - (view) Author: licunlong (Geass-LL) Date: 2022-01-22 03:11
But one more question.
Please take a look at fd.py. In this file, we first close fd 0, then open /dev/null as fd_null, and dump fd_null to fd 0. After we fork new process, this fd 0 `is` inherited.
This seems to be a bug?
msg411225 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-01-22 03:19
> this fd 0 `is` inherited.

File descriptors 0, 1, 2 are special: stdin, stdout and stderr file descriptors. You should use stdin, stdout, and stderr parameter of subprocess.

Sorry but the bug tracker is not a good place to ask questions about how to use Python, it's only to report bugs.

Python works as expected.
msg411229 - (view) Author: licunlong (Geass-LL) Date: 2022-01-22 04:01
> File descriptors 0, 1, 2 are special: stdin, stdout and stderr file descriptors. You should use stdin, stdout, and stderr parameter of subprocess.

I finally find why `fd 0` is inherited. The reason is that os.dup2() has a parameter "inheritable" which defaults to True. Not because it's special stdin.

I still think this is a bug. If some one closes fd 0, then he reopens it. it will not be inherited.
msg411572 - (view) Author: Brett Randall (javabrett) Date: 2022-01-25 10:44
For the possible benefit of future readers of this issue, the sequence and behaviour here which may cause confusion:

- say stdin is closed, perhaps along with stdout and stderr via a call to os.closerange(0, 3) or otherwise
- os.open() is called and creates a new fd=0, which will always be 0 since 0 is now available and lowest.  Per PEP 446 this fd is non-inheritable.
- a call to os.dup2(fd, 0) is made but is a noop, since fd=0 and dup2() ignores copy-to-same.  This behaviour may go unnoticed - it may not be noticed that the new fd=0 (it should always be this), or that dup2() has this noop behaviour.  Per PEP 446, dup2() still creates inheritable by-default, so the caller may expect that they have just assigned an inheritable fd to stdin, instead of a noop.
- subprocess() is called and the subprocess does not have stdin/0 fd assigned, since it was not inheritable.

It seems the main way to avoid this is to os.open() the replacement fd _before_ closing stdin/0, in which case it will receive an fd >= 3.  The dup2(fd, 0) call will then work and the resulting fd 0 will be inheritable, including by-default by subprocess() processes.
msg411573 - (view) Author: Brett Randall (javabrett) Date: 2022-01-25 10:56
docker run -i --rm python:3.9 <<-EOF

import os

print(os.get_inheritable(0))

fd_null_before_close = os.open("/dev/null", os.O_RDWR)
os.close(0)
fd_null = os.open("/dev/null", os.O_RDWR)

print(fd_null_before_close)
print(fd_null)

os.dup2(fd_null, 0)
print(os.get_inheritable(0))

os.dup2(fd_null_before_close, 0)
print(os.get_inheritable(0))

EOF
True
3
0
False
True
History
Date User Action Args
2022-04-11 14:59:55adminsetgithub: 90612
2022-01-28 04:15:39eryksunsetmessages: - msg411287
2022-01-28 04:15:13eryksunsetmessages: - msg411776
2022-01-26 16:54:11eryksunsetmessages: + msg411776
2022-01-25 10:56:07javabrettsetmessages: + msg411573
2022-01-25 10:44:17javabrettsetmessages: + msg411572
2022-01-25 09:53:58javabrettsetnosy: + javabrett
2022-01-22 22:57:00vstinnersetnosy: - vstinner
2022-01-22 19:57:02eryksunsetnosy: + eryksun
messages: + msg411287
2022-01-22 04:01:05Geass-LLsetmessages: + msg411229
2022-01-22 03:19:52vstinnersetmessages: + msg411225
2022-01-22 03:11:22Geass-LLsetnosy: + Geass-LL
messages: + msg411224
2022-01-21 16:37:53vstinnersetstatus: open -> closed
resolution: not a bug
messages: + msg411144

stage: resolved
2022-01-21 15:43:05JelleZijlstrasetnosy: + JelleZijlstra
messages: + msg411138
2022-01-21 12:05:31sxt1001setfiles: + python3.4.6-out.png
2022-01-21 12:05:21sxt1001setfiles: + python3.10.0-out.png
2022-01-21 12:04:51sxt1001setfiles: + python2.7.13-out.png
2022-01-21 12:03:11sxt1001setfiles: + fd2.py
2022-01-21 12:01:50sxt1001setfiles: + fd.py
2022-01-21 11:55:53sxt1001create