classification
Title: Python 3.5.2 strange-behavior issues (from PyPy)
Type: behavior Stage:
Components: Versions: Python 3.5
process
Status: open Resolution:
Dependencies: 18018 28893 Superseder:
Assigned To: Nosy List: arigo, brett.cannon, gregory.p.smith, haypo, larry, rhettinger, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2016-12-06 12:02 by arigo, last changed 2017-01-10 18:47 by brett.cannon.

Files
File name Uploaded Description Edit
os-chmod-EINTR-retry.gps01.diff gregory.p.smith, 2016-12-07 01:16 review
Messages (20)
msg282537 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2016-12-06 12:02
As discussed on python-dev, I am creating omnibus issues from the lists of crashers, of wrong-according-to-the-docs, and of strange-behavior-only issues that I found while developing Python 3.5.2 support for PyPy.  These occur with CPython 3.5.2 but most of them are likely still here in trunk.

This is the issue containing the "strange behaviors" and some of them, or possibly most, will turn out to be my own feelings only and not python-dev's, which is fine by me.
msg282538 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2016-12-06 12:03
(S1) ceval.c: GET_AITER: calls _PyCoro_GetAwaitableIter(), which might
  get an exception from calling the user-defined __await__() or checking
  what it returns; such an exception is completely eaten.
msg282539 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2016-12-06 12:03
(S2) argument clinic turns the "bool" specifier into
  PyObject_IsTrue(), accepting any argument whatsoever.  This can easily
  get very confusing for the user, e.g. after messing up the number of
  arguments.  For example: os.symlink("/path1", "/path2", "/path3")
  doesn't fail, it just considers the 3rd argument as some true value.
msg282540 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2016-12-06 12:03
(S3) hash({}.values()) works (but hash({}.keys()) correctly gives
  TypeError).  That's a bit confusing and, as far as I can tell, always
  pointless.  Also, related: d.keys()==d.keys() but
  d.values()!=d.values().
msg282541 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2016-12-06 12:03
(S4) if you write ``from .a import b`` inside the Python prompt, or in
  a module not in any package, then you get a SystemError(!) with an
  error message that is unlikely to help newcomers.
msg282542 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2016-12-06 12:03
(S5) pep 475: unclear why 'os.fchmod(fd)' retries automatically when
  it gets EINTR but the otherwise-equivalent 'os.chmod(fd)' does not.
  (The documentation says they are fully equivalent, so someone is
  wrong.)
msg282546 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-06 12:59
(S4) was fixed in 3.6 (issue18018). But the fix was not applied to 3.5.
msg282547 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-06 13:02
(S2) is related not to Argument Clinic itself, but to the 'p' format unit added in 3.3 (issue14705).
msg282562 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-12-06 18:16
Victor & Yury for S1.
msg282566 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-12-06 18:40
I'll take a look at s1 later tonight
msg282579 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2016-12-07 00:52
+larry for (S2) regarding argument clinic's treatment of bool.
msg282584 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2016-12-07 01:15
attaching a proposed fix for (S5).  os.chmod should have the same EINTR retry treatment as everything else.  That it does not appears to have been an oversight.
msg282589 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-12-07 03:36
I assert that Argument Clinic's / PyArg_ParseTuple's handling of bool is correct.

* Python has a well-established concept of "truth":

  https://docs.python.org/3/library/stdtypes.html#truth-value-testing

  The C equivalent is, indeed, PyObject_IsTrue().

* When we added the 'p' format unit to PyArg_ParseTuple, I originally had a second format unit 'P' that would only accept boolean literal values.  But nobody could come up with a use case for it.  So we removed it.  See #14705 for the discussion.

* Existing code often does something half-assed here.  eg. stat_float_times uses the PyArg_ParseTuple "i" format unit for its boolean parameter, which accepts booleans and integers but not strings.  This is strange and inconsistent.  (Of course, you can do this with Argument Clinic, just specify the parameter as an int.  But, yuck.)

If you have a counter-proposal for how it should behave, I'd be interested to hear it.  But as far as I'm concerned, the two legitimate answers for "how to handle integers" are either a) accept literally only True and False--an approach which nobody has a use case for--or b) PyObject_IsTrue(), which behaves exactly like Python.
msg282596 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-12-07 04:42
I created an issue to track S1: http://bugs.python.org/issue28893
msg282601 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2016-12-07 08:48
(S5) gregory: actually there is also fchown/chown in the same situation.
msg283561 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-18 16:37
(S3) is a side effect of implementing __eq__ in dict_keys and dict_items. Since __eq__ is not implemented in dict_values, __hash__ is inherited from object.
msg285101 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2017-01-10 09:16
(S6) 'xxx' % b'foo' == 'xxx'
     b'xxx' % b'foo' raises TypeError

The first case is because PyMapping_Check() is true on b'foo', so it works like 'xxx' % {...}, which always just returns 'xxx'.  The second case is because _PyBytes_Format() contains more special cases, for bytes and bytearray, which are not present in PyUnicode_Format().
msg285102 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-10 09:43
Armin, it would help if you report all cases as separate issues. Some of them are already resolved, some can be closed with "won't fix" resolution, others need special discussions. This issue can be left as a metaissue.
msg285132 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2017-01-10 17:03
> Armin, it would help if you report all cases as separate issues.

I asked on python-dev before creating these three issues, and got the opposite answer.  If you decide it was a bad idea after all, I will open separate issues in the future.
msg285143 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-01-10 18:47
I think if you're up for doing individual issues, Armin, that's preferred. But if it's too much work we will take it all in this single issue instead of risking the loss of the information. And if you want to use this issue as a meta one to track everything you report that's obviously fine.
History
Date User Action Args
2017-01-10 18:47:42brett.cannonsetmessages: + msg285143
2017-01-10 17:03:32arigosetmessages: + msg285132
2017-01-10 09:43:55serhiy.storchakasetmessages: + msg285102
2017-01-10 09:16:10arigosetmessages: + msg285101
2016-12-18 16:37:47serhiy.storchakasetnosy: + rhettinger
messages: + msg283561
2016-12-07 09:32:05serhiy.storchakasetdependencies: + Make sure exceptions raised in __aiter__ are properly chained in ceval
2016-12-07 08:48:16arigosetmessages: + msg282601
2016-12-07 04:42:07yselivanovsetmessages: + msg282596
2016-12-07 03:36:31larrysetmessages: + msg282589
2016-12-07 01:16:05gregory.p.smithsetfiles: + os-chmod-EINTR-retry.gps01.diff
keywords: + patch
messages: + msg282584
2016-12-07 00:52:03gregory.p.smithsetnosy: + larry, gregory.p.smith
messages: + msg282579
2016-12-06 18:40:03yselivanovsetmessages: + msg282566
2016-12-06 18:16:50brett.cannonsetnosy: + haypo, yselivanov
messages: + msg282562
2016-12-06 13:02:59serhiy.storchakasetmessages: + msg282547
2016-12-06 12:59:07serhiy.storchakasetnosy: + serhiy.storchaka, brett.cannon
dependencies: + SystemError: Parent module '' not loaded, cannot perform relative import
messages: + msg282546
2016-12-06 12:09:05arigosettype: behavior
2016-12-06 12:08:13arigosetversions: + Python 3.5
2016-12-06 12:03:44arigosetmessages: + msg282542
2016-12-06 12:03:34arigosetmessages: + msg282541
2016-12-06 12:03:26arigosetmessages: + msg282540
2016-12-06 12:03:16arigosetmessages: + msg282539
2016-12-06 12:03:01arigosetmessages: + msg282538
2016-12-06 12:02:48arigocreate