classification
Title: errors writing to stdout during interpreter exit exit with status 0
Type: Stage:
Components: Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: Nosy List: martin.panter, rbcollins, serhiy.storchaka
Priority: normal Keywords:

Created on 2015-08-14 08:59 by rbcollins, last changed 2015-08-15 00:48 by rbcollins. This issue is now closed.

Messages (9)
msg248570 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-14 08:59
I was trying to demonstrate how testing some code is hard, and I stumbled upon this.

The following code should be debuggable when run with a bad stdout - e.g.

python foo.py > /dev/full

---
import sys
import traceback

import pdb;pdb.Pdb(stdout=sys.stderr).set_trace()
try:
    print("What... is your quest?")
except:
    sys.stderr.write("Exception in program.\n")
    traceback.print_exc(file=sys.stderr)
---

Here is a transcript of a session with it:
---
$ python ./02.py > /dev/full 
> /home/robertc/work/Presentations/reveal.js/effectsnippets/02.py(5)<module>()
-> try:
(Pdb) n
> /home/robertc/work/Presentations/reveal.js/effectsnippets/02.py(6)<module>()
-> print("What... is your quest?")
(Pdb) n
--Return--
> /home/robertc/work/Presentations/reveal.js/effectsnippets/02.py(6)<module>()->None
-> print("What... is your quest?")
(Pdb) n
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
OSError: [Errno 28] No space left on device
$ echo $?
0
---

The same thing done on stderr exits with an exit code of 0 as well.
msg248571 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-14 09:09
Oh, and for added joy sys.last_value is not set here, so I've yet to manage to poke at what is being executed - clearly pdb is still managing to single-step, ish.
msg248573 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-08-14 09:12
See also issue11380.
msg248575 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-14 09:37
FWIW Python 2.7.8 behaves the same way (different message, same behaviour).
msg248578 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-14 10:38
Oh, one nuance - the reason my except isn't triggering is that the write is happening in interpreter shutdown - in flush_std_files.

Adding a a flush there allows that to work, but its fugly:

---
import sys

try:
    print("What... is your quest?")
    # Workaround bug 24864: force the write to be immediate
    sys.stdout.flush()
except:
    sys.stderr.write("Exception in program.\n")
    e = sys.exc_info()[1]
    try:
        # Workaround bug 24864 close the file so flush_std_files doesn't run
        # during interpreter cleanup.
        sys.stdout.close()
    finally:
        raise e from None
        
---
msg248580 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-14 10:58
Updating the title to reflect my deeper understanding: the only issue here is that we don't alter the exit code.
msg248582 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-08-14 11:21
I don’t know much about PDB, but if this is just about setting the exit code, it may be a dupe of Issue 5319. The fix proposed there is a new Py_FinalizeEx() API that returns an exit status, if I remember correctly.
msg248624 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-14 23:57
This patch is a minimal stab at the issue.

We should do this change because as it stands genuine user affecting errors can be masked both in pipelines and non-pipeline cases.

A more comprehensive patch would also change e.g. Py_Exit, and the various other users of Py_Finalize like freeze.

I'm not sure where to add an automated test for this; it does sane things under my manual testing.
msg248627 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-14 23:58
Oh, just saw your comment Martin; yes, this does look like a dupe.
History
Date User Action Args
2015-08-15 00:48:35rbcollinssetstatus: open -> closed
resolution: duplicate
2015-08-14 23:58:37rbcollinssetmessages: + msg248627
2015-08-14 23:57:31rbcollinssetmessages: + msg248624
2015-08-14 11:21:10martin.pantersetnosy: + martin.panter
messages: + msg248582
2015-08-14 10:58:42rbcollinssetmessages: + msg248580
title: errors writing to stdout are uncatchable and exit with status 0 -> errors writing to stdout during interpreter exit exit with status 0
2015-08-14 10:38:07rbcollinssetmessages: + msg248578
2015-08-14 09:37:04rbcollinssetmessages: + msg248575
2015-08-14 09:12:47serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg248573
2015-08-14 09:09:45rbcollinssetmessages: + msg248571
2015-08-14 08:59:02rbcollinscreate