classification
Title: PEP 475: fnctl functions are not retried if interrupted by a signal (EINTR)
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Michael.Felt, akeskimo, miss-islington, nierob, vstinner
Priority: normal Keywords: patch

Created on 2018-11-08 12:45 by akeskimo, last changed 2019-02-14 18:41 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 10413 merged nierob, 2018-11-08 14:06
PR 10637 merged vstinner, 2018-11-21 14:49
PR 10678 merged vstinner, 2018-11-23 15:48
PR 10685 merged vstinner, 2018-11-23 16:59
PR 10782 merged vstinner, 2018-11-29 00:15
PR 10784 merged miss-islington, 2018-11-29 00:35
PR 10785 merged miss-islington, 2018-11-29 00:35
PR 11424 merged Michael.Felt, 2019-01-03 17:29
PR 11424 merged Michael.Felt, 2019-01-03 17:29
PR 11424 merged Michael.Felt, 2019-01-03 17:29
PR 11424 merged Michael.Felt, 2019-01-03 17:29
PR 11858 merged miss-islington, 2019-02-14 18:22
Messages (20)
msg329469 - (view) Author: Aapo Samuli Keskimolo (akeskimo) Date: 2018-11-08 12:45
According to https://www.python.org/dev/peps/pep-0475/ the EINTR interruption should be retried automatically, but somehow it does not work and the exception is raised:

2018-11-05 05:21:35,257 ERROR:storage(23491): Remote storage operation failed (request: '{  'excludeSubModules': None,
   'storageLocation': 'qt/qtdatavis3d/68faa5b00f73096eb096c6acdfce76b052ca20b9/LinuxUbuntu_18_04x86_64LinuxQEMUarm64GCCqtci-linux-Ubuntu-18.04-x86_64-a6
c9f7Release/ac4280d182ec320eaf0e68efaeeeb6be14b9689f/test_1542834179',
   'type': 3}')
Traceback (most recent call last):
  File "src/storage.py", line 507, in handle
    self.handle_upload_artifact(message)
  File "src/storage.py", line 437, in handle_upload_artifact
    log.info("upload of %s to %s", uploadType, message.storageLocation)
  File "/usr/lib/python3.6/logging/__init__.py", line 1306, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python3.6/logging/__init__.py", line 1442, in _log
    self.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1452, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1514, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 861, in handle
    self.acquire()
  File "/home/vmbuilder/qt-ci/src/application.py", line 151, in acquire
    fcntl.lockf(self._lock_fd, fcntl.LOCK_EX)
InterruptedError: [Errno 4] Interrupted system call
msg329470 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-08 12:48
You're right, it should, but the fcntl module hasn't been patched. Are you interested to work or on patch, or do you want to me to find someone to do it?
msg329471 - (view) Author: Aapo Samuli Keskimolo (akeskimo) Date: 2018-11-08 12:51
My colleague has made a prospective fix:

https://github.com/nierob/cpython/commit/3b76b887b2b339664dc52c5f246bc7cb268ea616
msg329472 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-08 12:53
> https://github.com/nierob/cpython/commit/3b76b887b2b339664dc52c5f246bc7cb268ea616

Oh, nice! Please rebase this change on the master branch and reuse "int async_err = 0;" pattern from Modules/posixmodule.c. You must not raise a new exception if PyErr_CheckSignals() raised an exception, something like:

        return (!async_err) ? posix_error() : NULL;
msg329473 - (view) Author: Nierob (nierob) * Date: 2018-11-08 14:06
PR waits for CLA
msg330208 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-21 15:33
New changeset aac1f81eef971876ba5b1673db9ce6620311c469 by Victor Stinner in branch 'master':
bpo-35189: Fix eintr_tester.py (GH-10637)
https://github.com/python/cpython/commit/aac1f81eef971876ba5b1673db9ce6620311c469
msg330335 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-23 15:46
New changeset b409ffa848b280c1db1b4f450bfae14f263099ac by Victor Stinner (nierob) in branch 'master':
bpo-35189: Retry fnctl calls on EINTR (GH-10413)
https://github.com/python/cpython/commit/b409ffa848b280c1db1b4f450bfae14f263099ac
msg330339 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-23 16:53
New changeset 56742f1eb05401a27499af0ccdcb4e4214859fd1 by Victor Stinner in branch '3.7':
[3.7] bpo-35189: Retry fnctl calls on EINTR (GH-10413) (GH-10678)
https://github.com/python/cpython/commit/56742f1eb05401a27499af0ccdcb4e4214859fd1
msg330348 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-23 18:00
New changeset eef813b1091f4baaaa7411aa58d2746a9761ee99 by Victor Stinner in branch '3.6':
[3.7] bpo-35189: Retry fnctl calls on EINTR (GH-10413) (GH-10678) (GH-10685)
https://github.com/python/cpython/commit/eef813b1091f4baaaa7411aa58d2746a9761ee99
msg330350 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-23 18:01
Thanks Aapo Samuli Keskimolo for the bug report and thanks Nierob for the fix!
msg330645 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-29 00:34
New changeset 2956bffbc00127af65f69e04d7979021a21d1288 by Victor Stinner in branch 'master':
bpo-35189, bpo-35316: Make test_eintr less strict (GH-10782)
https://github.com/python/cpython/commit/2956bffbc00127af65f69e04d7979021a21d1288
msg330649 - (view) Author: miss-islington (miss-islington) Date: 2018-11-29 00:52
New changeset 2fa5b2ac4485c5c9083b4b0459dd9b691daaea28 by Miss Islington (bot) in branch '3.7':
bpo-35189, bpo-35316: Make test_eintr less strict (GH-10782)
https://github.com/python/cpython/commit/2fa5b2ac4485c5c9083b4b0459dd9b691daaea28
msg330652 - (view) Author: miss-islington (miss-islington) Date: 2018-11-29 00:57
New changeset 833a7067a4d5621d024511d9166db7331c4650c9 by Miss Islington (bot) in branch '3.6':
bpo-35189, bpo-35316: Make test_eintr less strict (GH-10782)
https://github.com/python/cpython/commit/833a7067a4d5621d024511d9166db7331c4650c9
msg332580 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-12-27 10:02
I have not looked at 3.6, but I have bisected the 3.7 and 3.8 branches for AIX. I get:

On 3.7 Branch:
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[56742f1eb05401a27499af0ccdcb4e4214859fd1] [3.7] bpo-35189: Retry fnctl calls on EINTR (GH-10413) (GH-10678)

On 3.8 Branch:
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[b409ffa848b280c1db1b4f450bfae14f263099ac] bpo-35189: Retry fnctl calls on EINTR (GH-10413)

So, my assumption is that the PR-10413 is not 100% correct for AIX.

Will look further, but also request - will this issue reopen, or do we need a new issue?
msg332581 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-12-27 10:05
Forgot to include the test failure message:

======================================================================
FAIL: test_all (test.test_eintr.EINTRTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.8/Lib/test/test_eintr.py", line 18, in test_all
    script_helper.assert_python_ok("-u", tester)
  File "/data/prj/python/git/python3-3.8/Lib/test/support/script_helper.py", line 157, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/data/prj/python/git/python3-3.8/Lib/test/support/script_helper.py", line 143, in _assert_python
    res.fail(cmd_line)
  File "/data/prj/python/git/python3-3.8/Lib/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['/data/prj/python/python3-3.8/python', '-X', 'faulthandler', '-I', '-u', '/data/prj/python/git/python3-3.8/Lib/test/eintrdata/eintr_tester.py']

stdout:
---

---

stderr:
---
.E......sss.............
======================================================================
ERROR: test_lockf (__main__.FNTLEINTRTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.8/Lib/test/eintrdata/eintr_tester.py", line 522, in test_lockf
    self._lock(fcntl.lockf, "lockf")
  File "/data/prj/python/git/python3-3.8/Lib/test/eintrdata/eintr_tester.py", line 507, in _lock
    lock_func(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
PermissionError: [Errno 13] Permission denied

----------------------------------------------------------------------
Ran 24 tests in 9.692s

FAILED (errors=1, skipped=3)
---

----------------------------------------------------------------------

Ran 1 test in 10.404s

FAILED (failures=1)
test test_eintr failed
test_eintr failed

== Tests result: FAILURE ==

1 test failed:
    test_eintr

Total duration: 10 sec 645 ms
msg332591 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-12-27 14:48
The "improved" output after getting back to "latest" commit:

== CPython 3.8.0a0 (heads/master-dirty:34ae04f74d, Dec 27 2018, 14:05:08) [C]
== AIX-1-00C291F54C00-powerpc-32bit big-endian
== cwd: /data/prj/python/python3-3.8/build/test_python_13566116
== CPU count: 8
== encodings: locale=ISO8859-1, FS=iso8859-1
Run tests sequentially
0:00:00 [1/1] test_eintr
test_all (test.test_eintr.EINTRTests) ...
--- run eintr_tester.py ---
test_flock (__main__.FNTLEINTRTest) ... ok
test_lockf (__main__.FNTLEINTRTest) ... ERROR
test_read (__main__.OSEINTRTest) ... ok
test_wait (__main__.OSEINTRTest) ... ok
test_wait3 (__main__.OSEINTRTest) ... ok
test_wait4 (__main__.OSEINTRTest) ... ok
test_waitpid (__main__.OSEINTRTest) ... ok
test_write (__main__.OSEINTRTest) ... ok
test_devpoll (__main__.SelectEINTRTest) ... skipped 'need select.devpoll'
test_epoll (__main__.SelectEINTRTest) ... skipped 'need select.epoll'
test_kqueue (__main__.SelectEINTRTest) ... skipped 'need select.kqueue'
test_poll (__main__.SelectEINTRTest) ... ok
test_select (__main__.SelectEINTRTest) ... ok
test_sigtimedwait (__main__.SignalEINTRTest) ... ok
test_sigwaitinfo (__main__.SignalEINTRTest) ... ok
test_accept (__main__.SocketEINTRTest) ... ok
test_open (__main__.SocketEINTRTest) ... ok
test_os_open (__main__.SocketEINTRTest) ... ok
test_recv (__main__.SocketEINTRTest) ... ok
test_recvmsg (__main__.SocketEINTRTest) ... ok
test_send (__main__.SocketEINTRTest) ... ok
test_sendall (__main__.SocketEINTRTest) ... ok
test_sendmsg (__main__.SocketEINTRTest) ... ok
test_sleep (__main__.TimeEINTRTest) ... ok

======================================================================
ERROR: test_lockf (__main__.FNTLEINTRTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.8/Lib/test/eintrdata/eintr_tester.py", line 522, in test_lockf
    self._lock(fcntl.lockf, "lockf")
  File "/data/prj/python/git/python3-3.8/Lib/test/eintrdata/eintr_tester.py", line 507, in _lock
    lock_func(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
PermissionError: [Errno 13] Permission denied

----------------------------------------------------------------------
Ran 24 tests in 8.822s

FAILED (errors=1, skipped=3)
--- eintr_tester.py completed: exit code 1 ---
FAIL

======================================================================
FAIL: test_all (test.test_eintr.EINTRTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.8/Lib/test/test_eintr.py", line 31, in test_all
    self.fail("eintr_tester.py failed")
AssertionError: eintr_tester.py failed

----------------------------------------------------------------------

Ran 1 test in 9.392s

FAILED (failures=1)
test test_eintr failed
test_eintr failed

== Tests result: FAILURE ==

1 test failed:
    test_eintr

Total duration: 9 sec 609 ms
Tests result: FAILURE
msg332624 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-12-27 22:14
On 27/12/2018 15:48, Michael Felt wrote:
> Michael Felt <aixtools@felt.demon.nl> added the comment:
>
> The "improved" output after getting back to "latest" commit:
>
> == CPython 3.8.0a0 (heads/master-dirty:34ae04f74d, Dec 27 2018, 14:05:08) [C]
> == AIX-1-00C291F54C00-powerpc-32bit big-endian
> == cwd: /data/prj/python/python3-3.8/build/test_python_13566116
> == CPU count: 8
> == encodings: locale=ISO8859-1, FS=iso8859-1
> Run tests sequentially
> 0:00:00 [1/1] test_eintr
> test_all (test.test_eintr.EINTRTests) ...
> --- run eintr_tester.py ---
> test_flock (__main__.FNTLEINTRTest) ... ok
> test_lockf (__main__.FNTLEINTRTest) ... ERROR
> test_read (__main__.OSEINTRTest) ... ok
> test_wait (__main__.OSEINTRTest) ... ok
> test_wait3 (__main__.OSEINTRTest) ... ok
> test_wait4 (__main__.OSEINTRTest) ... ok
> test_waitpid (__main__.OSEINTRTest) ... ok
> test_write (__main__.OSEINTRTest) ... ok
> test_devpoll (__main__.SelectEINTRTest) ... skipped 'need select.devpoll'
> test_epoll (__main__.SelectEINTRTest) ... skipped 'need select.epoll'
> test_kqueue (__main__.SelectEINTRTest) ... skipped 'need select.kqueue'
> test_poll (__main__.SelectEINTRTest) ... ok
> test_select (__main__.SelectEINTRTest) ... ok
> test_sigtimedwait (__main__.SignalEINTRTest) ... ok
> test_sigwaitinfo (__main__.SignalEINTRTest) ... ok
> test_accept (__main__.SocketEINTRTest) ... ok
> test_open (__main__.SocketEINTRTest) ... ok
> test_os_open (__main__.SocketEINTRTest) ... ok
> test_recv (__main__.SocketEINTRTest) ... ok
> test_recvmsg (__main__.SocketEINTRTest) ... ok
> test_send (__main__.SocketEINTRTest) ... ok
> test_sendall (__main__.SocketEINTRTest) ... ok
> test_sendmsg (__main__.SocketEINTRTest) ... ok
> test_sleep (__main__.TimeEINTRTest) ... ok
>
> ======================================================================
> ERROR: test_lockf (__main__.FNTLEINTRTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/data/prj/python/git/python3-3.8/Lib/test/eintrdata/eintr_tester.py", line 522, in test_lockf
>     self._lock(fcntl.lockf, "lockf")
>   File "/data/prj/python/git/python3-3.8/Lib/test/eintrdata/eintr_tester.py", line 507, in _lock
>     lock_func(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
> PermissionError: [Errno 13] Permission denied
>
> ----------------------------------------------------------------------
> Ran 24 tests in 8.822s
>
> FAILED (errors=1, skipped=3)
> --- eintr_tester.py completed: exit code 1 ---
> FAIL
>
> ======================================================================
> FAIL: test_all (test.test_eintr.EINTRTests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/data/prj/python/git/python3-3.8/Lib/test/test_eintr.py", line 31, in test_all
>     self.fail("eintr_tester.py failed")
> AssertionError: eintr_tester.py failed
>
> ----------------------------------------------------------------------
>
> Ran 1 test in 9.392s
>
> FAILED (failures=1)
> test test_eintr failed
> test_eintr failed
>
> == Tests result: FAILURE ==
>
> 1 test failed:
>     test_eintr
>
> Total duration: 9 sec 609 ms
> Tests result: FAILURE
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue35189>
> _______________________________________
>
I have been doing reading and debugging.

Question: does mode "wb" imply also open for reading? Both Freebsd and
AIX man pages specify that a file needs to be open for a shared lock to
even be considered.

Further, AIX talks about "enforced" and "advisory" locks, as well as
"read" and "write" locks. In much older documentation I recall the names
"simple" and "complex" locks. From memory, advisory (aka simple) locks
tend to be exclusive in nature. Shared is only for reading, writing is
always exclusive. I'll have to dig for how "wait" is actually handled -
and work to not confuse "in memory" locks (for multi-threaded locking of
variables) with "file-locking".

Regards,

Michael
msg333021 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-01-04 23:03
Michael created bpo-35633: test_eintr: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX.
msg335551 - (view) Author: miss-islington (miss-islington) Date: 2019-02-14 18:22
New changeset b94d4bed67c60834b1a5a0936b8c13934cf3b872 by Miss Islington (bot) (Michael Felt) in branch 'master':
bpo-35633: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX (GH-11424)
https://github.com/python/cpython/commit/b94d4bed67c60834b1a5a0936b8c13934cf3b872
msg335554 - (view) Author: miss-islington (miss-islington) Date: 2019-02-14 18:41
New changeset 7e618f3154404613cd9774fb8ffb5b48e7233a9e by Miss Islington (bot) in branch '3.7':
bpo-35633: test_lockf() fails with "PermissionError: [Errno 13] Permission denied" on AIX (GH-11424)
https://github.com/python/cpython/commit/7e618f3154404613cd9774fb8ffb5b48e7233a9e
History
Date User Action Args
2019-02-14 18:41:41miss-islingtonsetmessages: + msg335554
2019-02-14 18:22:48miss-islingtonsetpull_requests: + pull_request11891
2019-02-14 18:22:39miss-islingtonsetmessages: + msg335551
2019-01-04 23:03:58vstinnersetmessages: + msg333021
2019-01-03 17:30:15Michael.Feltsetpull_requests: + pull_request10844
2019-01-03 17:30:09Michael.Feltsetpull_requests: + pull_request10845
2019-01-03 17:30:03Michael.Feltsetpull_requests: + pull_request10843
2019-01-03 17:29:56Michael.Feltsetpull_requests: + pull_request10842
2018-12-27 22:14:12Michael.Feltsetmessages: + msg332624
2018-12-27 14:48:41Michael.Feltsetmessages: + msg332591
2018-12-27 10:05:58Michael.Feltsetmessages: + msg332581
2018-12-27 10:02:05Michael.Feltsetnosy: + Michael.Felt
messages: + msg332580
2018-11-29 00:57:21miss-islingtonsetmessages: + msg330652
2018-11-29 00:52:15miss-islingtonsetnosy: + miss-islington
messages: + msg330649
2018-11-29 00:35:43miss-islingtonsetpull_requests: + pull_request10028
2018-11-29 00:35:25miss-islingtonsetpull_requests: + pull_request10026
2018-11-29 00:34:55vstinnersetmessages: + msg330645
2018-11-29 00:15:29vstinnersetpull_requests: + pull_request10023
2018-11-23 18:01:48vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg330350

stage: patch review -> resolved
2018-11-23 18:00:18vstinnersetmessages: + msg330348
2018-11-23 16:59:42vstinnersetpull_requests: + pull_request9937
2018-11-23 16:53:16vstinnersetmessages: + msg330339
2018-11-23 15:48:22vstinnersetpull_requests: + pull_request9933
2018-11-23 15:46:14vstinnersetmessages: + msg330335
2018-11-21 15:33:18vstinnersetmessages: + msg330208
2018-11-21 14:49:23vstinnersetpull_requests: + pull_request9884
2018-11-08 14:38:16vstinnersetversions: + Python 3.7, Python 3.8
2018-11-08 14:38:09vstinnersettitle: EINTR is not being retried -> PEP 475: fnctl functions are not retried if interrupted by a signal (EINTR)
2018-11-08 14:06:36nierobsetnosy: + nierob
messages: + msg329473
pull_requests: + pull_request9693

keywords: + patch
stage: patch review
2018-11-08 12:53:42vstinnersetmessages: + msg329472
2018-11-08 12:51:33akeskimosetmessages: + msg329471
2018-11-08 12:48:01vstinnersetnosy: + vstinner
messages: + msg329470
2018-11-08 12:45:30akeskimocreate