Just noticed this (tested on Python 3.7 and 3.8):

    mkdir /tmp/demo
    cd /tmp/demo
    cat << EOF > abc.py
    raise Exception("Hi")
    EOF
    PYTHONPATH=: python -c ""


This will crash the interpreter with:

    Fatal Python error: init_sys_streams: can't initialize sys standard 
streams
    Traceback (most recent call last):
      File ".../lib/python3.7/io.py", line 52, in <module>
      File "/tmp/demo/abc.py", line 1, in <module>
    Exception: Hi
    Aborted (core dumped)


It seems that the problem is that the io module imports the abc module, which raises an exception, so io fails to load. Evidently the io module is necessary to load the interpreter, so the interpreter crashes.

Here's the backtrace for 3.7.2 on Arch Linux:

(gdb) bt
#0  0x00007f234b3f0d7f in raise () from /usr/lib/libc.so.6
#1  0x00007f234b3db672 in abort () from /usr/lib/libc.so.6
#2  0x00007f234b7db490 in fatal_error (prefix=prefix@entry=0x7f234b9d5fe0 <__func__.16645> "init_sys_streams", 
    msg=msg@entry=0x7f234ba01f60 "can't initialize sys standard streams", status=status@entry=-1)
    at Python/pylifecycle.c:2179
#3  0x00007f234b8460cb in _Py_FatalInitError (err=...) at Python/pylifecycle.c:2198
#4  0x00007f234b8495a9 in pymain_init (pymain=0x7fff971cca70) at Modules/main.c:3019
#5  0x0000555dfa560050 in ?? ()
#6  0x00007fff971ccbc0 in ?? ()
#7  0x0000000000000000 in ?? ()


I'm not sure if anything can or should be done about this. It's very fair for the interpreter to fail to start, though I would guess that it should do that without dumping a core.

Tested with 3.6 and 2.7 - core dump on 3.6, no core dump on 2.7. The crash on 2.7 goes through a different path, it goes site.py > os.py >= UserDict.py > _abcoll.py > abc.py. That may account for why it's not crashing the interpreter itself.

I hope there are certain modules that CPython interpreter expects to load properly. There were some cases in the past where encodings module caused the interpreter to crash. A similar scenario.

➜  cpython git:(master) echo "raise Exception('a')" > encodings.py
➜  cpython git:(master) ✗ PYTHONPATH=: ./python.exe
Fatal Python error: initfsencoding: failed to get the Python codec of the filesystem encoding
Traceback (most recent call last):
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/encodings.py", line 1, in <module>
Exception: a
[1]    29005 abort      PYTHONPATH=: ./python.exe

@Karthikeyan I would certainly consider this a duplicate of the encodings bug/behavior that you demonstrate. I don't see an existing bug on the tracker for it, though.

I think it's pretty clear that the interpreter needs to fail, but it seems to *crash* rather than exit gracefully with a non-zero return code, which I think is the main problem. I'll note that it *does* print some stuff out before it crashes, which makes me think that even though io and/or encodings hasn't loaded, you still are able to do the "print an error message" part of "print an error message and exit with non-zero error code".

Sorry by previous reports I was talking about some of the issues where the installers were not correct though I couldn't find the issue at the moment. I just presented it as a data point where some necessary modules that cannot be imported could lead to crash. But if this can be dealt with in a better way I think it will be a good improvement.

Hi,

The reason of the aborted(core dump) of the interpreter is because when the init process was wrong, the 

```_Py_NO_RETURN fatal_error()```

call and abort()

So, I change it for a exit(status) to avoid break the interpreter (I attach a patch)

What do you think about it? If all it's ok I can prepare the PR

Last year, I reworked Python initialization to introduce a new _PyInitError structure which allows to report a failure to the caller of a function instead of calling directly Py_FatalError() which always call abort() immediately.

_PyInitError allows to distinguish "user error" and "internal error". User error is caused by an user mistake, whereas internal errors are all other errors. User errors don't call abort() to avoid dumping a core dump.

It's not always easy to decide if a bug is caused by the user or not.

For example, a memory allocation failure is now considered as an "user error":

#define _Py_INIT_NO_MEMORY() _Py_INIT_USER_ERR("memory allocation failed")

See Include/coreconfig.h for the definition of _PyInitError and related macros.

One enhancement of this new API is that it now reports the name of the C function which causes the error. The initial bug report says "Fatal Python error: init_sys_streams: can't initialize sys standard 
streams": init_sys_streams() function raised the fatal error.

A workaround is the usage of -I option:

vstinner@apu$ PYTHONPATH=: python3 -c ""
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Traceback (most recent call last):
  File "/usr/lib64/python3.7/io.py", line 52, in <module>
  File "/tmp/demo/abc.py", line 1, in <module>
Exception: Hi
Aborted (core dumped)

vstinner@apu$ PYTHONPATH=: python3 -I -c ""
<it works as expected>

A long term solution would to enable -I behavior by default: don't add the current directory to sys.path when using -c CMD.

> One enhancement of this new API is that it now reports the name of the C function which causes the error. The initial bug report says "Fatal Python error: init_sys_streams: can't initialize sys standard 
streams": init_sys_streams() function raised the fatal error.

I think that's a welcome change, the main question is whether it's necessary to hard-crash the interpreter and dump the core, or if, in most cases, the error message + non-zero return code would be sufficient.

It's not clear to me how useful the traceback itself is, but of course I'm giving a toy example where the Python exception is sufficient to debug what's failing. I would think that *in general* if the interpreter is failing to initialize because of a problem in the Python stack, printing the Python traceback should be sufficient. If there's good reason to think otherwise, then I'm fine with closing this bug. Otherwise I'd say it would be nice to refactor in such a way that avoids the core dump when the problem is on the Python side (leaving in place the error message you added that shows where it failed on the C side).

> A workaround is the usage of -I option:

In this case I don't think we need a workaround, because the issue is really "should it be possible for a pure python crash to cause the interpreter to crash rather than exit with an error code?" Shadowing a standard library module from your PYTHONPATH is just the easiest way to manifest this. (Though probably more people should be using -I, so it's good to bring it up).


> A long term solution would to enable -I behavior by default: don't add the current directory to sys.path when using -c CMD.

If you've been following the recent issues with the PEP 517 rollout in setuptools, you'll see that I'm deeply sympathetic to this view (though in this case I'm not amazingly optimistic about its feasibility). Probably we shouldn't derail this issue too much with a discussion of our revolutionary views on the right default Python path, though.

See also bpo-35971: "Documentation should warn about code injection from current working directory" which discuss changing the default behavior.

On Tue, Feb 12, 2019 at 3:57 AM STINNER Victor <report@bugs.python.org> wrote:
> A long term solution would to enable -I behavior by default: don't add the current directory to sys.path when using -c CMD.

From what I can recall, it was a conscious decision to include CWD in sys.path.  Doing so benefits some scripting use cases.  The downside is the effect when running an app.

related: issue #13475

@Guido, I recall a while back you explained the value of adding CWD to sys.path.  Would you mind providing some insight here?

> @Guido, I recall a while back you explained the value of adding CWD to sys.path.  Would you mind providing some insight here

I think bpo-35971 or bpo-13475 are better places to discuss that, because this issue doesn't require CWD to be in the path. You can trigger this by any mechanism that would raise an exception in the Python code used to initialize the interpreter.

Even if we restrict it to the most common case - shadowing a part of the standard library - you would still be able to trigger this as long as you have *some* mechanism to manipulate the python path in the environment that allows you to put paths on there with higher precedence than the standard library.

I think the core question in this issue is whether errors in the Python code run during the interpreter initialization should crash the interpreter or exit gracefully with an error code.

> Otherwise I'd say it would be nice to refactor in such a way that avoids the core dump when the problem is on the Python side 

I agree with @p-ganssle, If exist some problem on initialize Python interpreter it would be great if it return a non-zero status and avoid an abort()

+ Python3.5. I test it and has the same result.

> I agree with @p-ganssle, If exist some problem on initialize Python interpreter it would be great if it return a non-zero status and avoid an abort()

Since Python 3.8 (PEP 587), Python no longer call abort() on initialization error.

$ echo bug > abc.py

$ PYTHONPATH=$PWD python3.8 -c pass
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Python runtime state: core initialized
Traceback (most recent call last):
  File "/usr/lib64/python3.8/io.py", line 52, in <module>
  File "/home/vstinner/abc.py", line 1, in <module>
NameError: name 'bug' is not defined

I don't think that it's worth it to change Python 3.7.

The initial issue was that abort() was called in this case. It's no longer the case, so I close the issue.

If someone cares abou the more general issue with the current directory being added to sys.path, please open a separated issue (or look for existing issues).