classification
Title: PEP 475: Add _Py_read() and _Py_write() functions
Type: Stage:
Components: Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: python-dev, vstinner
Priority: normal Keywords: patch

Created on 2015-03-19 13:33 by vstinner, last changed 2015-03-20 11:24 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
py_read_write.patch vstinner, 2015-03-19 13:33 review
select_write.patch vstinner, 2015-03-19 13:34 review
py_read_write-2.patch vstinner, 2015-03-19 14:28 review
Messages (14)
msg238514 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-19 13:33
To factorize code handling EINTR, I propose add 2 new functions to fileutils.h: _Py_read() and _Py_write().

Attached patch adds these functions and use them in the os and _io modules.

The code of the functions is based on the code from the os and _io modules. The main change is that the functions now truncate code if greater than PY_SSIZE_T_MAX. Other changes are just refactoring.
msg238515 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-19 13:34
select_write.patch: use _Py_write() in the select module.
msg238517 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-19 13:37
I created the issue #23709 for the ossaudiodev module: "Refactor ossaudiodev: use _Py_read and _Py_write with the Py_buffer".
msg238526 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-19 14:11
With py_read_write.patch in debug mode, test_threading crash with an assertion error because of the issue #15751. Currently, it's not possible to call assert(PyGILState_Check()); inside Py_NewInterpreter(), while creating a subinterpreter (ex: _testcapi.run_in_subinterp()).

The workaround is to comment the assertion in _Py_read() and Py_write() until the issue #15751 is fixed.
msg238528 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-19 14:28
Oops, there was a bug in FileIO.readall(), fixed in the new patch py_read_write-2.patch.

I also commented assert(PyGILState_Check()); to workaround the issue #15751.
msg238569 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-19 22:30
New changeset c3c47ea32f72 by Victor Stinner in branch 'default':
Issue #23708: Add _Py_read() and _Py_write() functions to factorize code handle
https://hg.python.org/cpython/rev/c3c47ea32f72
msg238571 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-19 22:33
New changeset e232b57ee784 by Victor Stinner in branch 'default':
Issue #23708: select.devpoll now retries its internal write() when interrupted
https://hg.python.org/cpython/rev/e232b57ee784
msg238572 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-19 22:34
I commited both patches. Thanks Antoine for the review of py_read_write-2.patch, I removed the commented assertion.
msg238576 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-19 22:53
New changeset 116e4c40115f by Victor Stinner in branch 'default':
Issue #23708: Fix _Py_read() compilation error on Windows
https://hg.python.org/cpython/rev/116e4c40115f
msg238638 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-20 10:19
An assertion failed in _Py_read() while running test_signal on AMD64 Snow Leop 3.x:

http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/2779/steps/test/logs/stdio


[321/393/1] test_signal
Assertion failed: (errno == EINTR && PyErr_Occurred()), function _Py_read, file Python/fileutils.c, line 1181.
Fatal Python error: Aborted

Current thread 0x00007fff71296cc0 (most recent call first):
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/subprocess.py", line 1407 in _execute_child
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/subprocess.py", line 855 in __init__
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_signal.py", line 126 in run_test
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_signal.py", line 180 in test_main
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 577 in run
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 625 in __call__
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 122 in run
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 84 in __call__
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 122 in run
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 84 in __call__
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/runner.py", line 176 in run
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/support/__init__.py", line 1772 in _run_suite
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/support/__init__.py", line 1806 in run_unittest
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_signal.py", line 1123 in test_main
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/regrtest.py", line 1284 in runtest_inner
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/regrtest.py", line 967 in runtest
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/regrtest.py", line 532 in main
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/regrtest.py", line 1568 in main_in_temp_cwd
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/regrtest.py", line 1593 in <module>
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/runpy.py", line 85 in _run_code
  File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/runpy.py", line 170 in _run_module_as_main
msg238639 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-20 10:37
New changeset 6dd201b6bb4f by Victor Stinner in branch 'default':
Issue #23708: Split assertion expression in two assertions in _Py_read() and
https://hg.python.org/cpython/rev/6dd201b6bb4f
msg238640 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-20 10:40
Same error on "x86 Tiger 3.x" buildbot:

http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/9381/steps/test/logs/stdio

[345/393] test_signal
Python/fileutils.c:1181: failed assertion `errno == EINTR && PyErr_Occurred()'
Fatal Python error: Aborted

Current thread 0xa000d000 (most recent call first):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/subprocess.py", line 1407 in _execute_child
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/subprocess.py", line 855 in __init__
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_signal.py", line 126 in run_test
  ...
msg238642 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-20 11:02
New changeset 07fd54208434 by Victor Stinner in branch 'default':
Issue #23708: Save/restore errno in _Py_read() and _Py_write()
https://hg.python.org/cpython/rev/07fd54208434
msg238647 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-20 11:24
> New changeset 07fd54208434 by Victor Stinner in branch 'default':
> Issue #23708: Save/restore errno in _Py_read() and _Py_write()

When I wrote _Py_read()/_Py_write(), I added assertions on errno to ensure that errno is not modified. I chose to use assertions instead of save/restore errno because on Linux errno is not modified. Since they *are* platforms where errno is modified, it's safer to always save/restore errno.

_Py_read()/_Py_write() *must* set set errno because some callers uses it. It's more convinient to use errno than having to get the errno attribute of the raised OSError exception.

test_signal pass again on "AMD64 Snow Leop 3.x" buildbot, I close the issue.
History
Date User Action Args
2015-03-20 11:24:27vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg238647
2015-03-20 11:02:40python-devsetmessages: + msg238642
2015-03-20 10:40:51vstinnersetmessages: + msg238640
2015-03-20 10:37:39python-devsetmessages: + msg238639
2015-03-20 10:19:27vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg238638
2015-03-19 22:53:38python-devsetmessages: + msg238576
2015-03-19 22:34:49vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg238572
2015-03-19 22:33:57python-devsetmessages: + msg238571
2015-03-19 22:30:20python-devsetnosy: + python-dev
messages: + msg238569
2015-03-19 14:28:16vstinnersetfiles: + py_read_write-2.patch

messages: + msg238528
2015-03-19 14:11:45vstinnersetmessages: + msg238526
2015-03-19 13:41:20vstinnerlinkissue23648 dependencies
2015-03-19 13:38:08vstinnerlinkissue23707 dependencies
2015-03-19 13:37:29vstinnersetmessages: + msg238517
2015-03-19 13:36:53vstinnerlinkissue23709 dependencies
2015-03-19 13:34:04vstinnersetfiles: + select_write.patch

messages: + msg238515
2015-03-19 13:33:30vstinnercreate