Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'
###########################################
Process:               Python [1889]
Path:                  /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.6.5 (3.6.5)
Code Type:             X86-64 (Native)
Parent Process:        bash [722]
Responsible:           Python [1889]
User ID:               501

Date/Time:             2018-05-22 23:52:40.997 -0700
OS Version:            Mac OS X 10.13.4 (17E202)
Report Version:        12
Anonymous UUID:        F667AFCE-E0F9-BF0F-FB74-372904C44DB5


Time Awake Since Boot: 2000 seconds

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00000001018eeb6e __pthread_kill + 10
1   libsystem_pthread.dylib       	0x0000000101928080 pthread_kill + 333
2   libsystem_c.dylib             	0x00000001015c51ae abort + 127
3   org.python.python             	0x0000000100877611 Py_FatalError + 65
4   org.python.python             	0x00000001008793c1 _Py_InitializeEx_Private + 1281
5   org.python.python             	0x0000000100896b40 Py_Main + 1360
6   org.python.python             	0x0000000100000dfe 0x100000000 + 3582
7   org.python.python             	0x0000000100000c34 0x100000000 + 3124

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x000000010192e380  rcx: 0x00007fff5bffef58  rdx: 0x0000000000000000
  rdi: 0x0000000000000307  rsi: 0x0000000000000006  rbp: 0x00007fff5bffef90  rsp: 0x00007fff5bffef58
   r8: 0x0000000000000000   r9: 0x0000000000000006  r10: 0x0000000000000000  r11: 0x0000000000000202
  r12: 0x0000000000000307  r13: 0x0000000000000002  r14: 0x0000000000000006  r15: 0x000000000000002d
  rip: 0x00000001018eeb6e  rfl: 0x0000000000000202  cr2: 0x0000000101bd7020
  
Logical CPU:     0
Error Code:      0x02000148
Trap Number:     133


Binary Images:
       0x100000000 -        0x100000ff7 +org.python.python (3.6.5 - 3.6.5) <C539EB14-B454-D151-7B93-6A12255582B2> /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
       0x100003000 -        0x10049dff7  com.apple.CoreFoundation (6.9 - 1452.23) <358C547D-E227-3228-8218-62327F4605C8> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
       0x100750000 -        0x100977fef +org.python.python (3.6.5, [c] 2001-2018 Python Software Foundation. - 3.6.5) <EBF198CD-A37E-A0F1-1E7C-19B9F912D34A> /Library/Frameworks/Python.framework/Versions/3.6/Python
       0x100ac0000 -        0x100ac1ffb  libSystem.B.dylib (1252.50.4) <F64430FD-30A6-328D-A476-0AED41D371C6> /usr/lib/libSystem.B.dylib
       0x100ac8000 -        0x100ac9ff3  libDiagnosticMessagesClient.dylib (104) <9712E980-76EE-3A89-AEA6-DF4BAF5C0574> /usr/lib/libDiagnosticMessagesClient.dylib
       0x100acf000 -        0x100cf6ffb  libicucore.A.dylib (59173.0.1) <CECAD5E5-3EFD-3AAC-AD9B-E355B2DD5E9C> /usr/lib/libicucore.A.dylib
       0x100dc8000 -        0x1011b93b7  libobjc.A.dylib (723) <BEEA51E2-1C4D-3672-87F6-F267B4AC6221> /usr/lib/libobjc.A.dylib
       0x10136a000 -        0x10137cffb  libz.1.dylib (70) <48C67CFC-940D-3857-8DAD-857774605352> /usr/lib/libz.1.dylib
       0x101382000 -        0x101386ff7  libcache.dylib (80) <092479CB-1008-3A83-BECF-E115F24D13C1> /usr/lib/system/libcache.dylib
       0x10138c000 -        0x101396ff3  libcommonCrypto.dylib (60118.50.1) <029F5985-9B6E-3DCB-9B96-FD007678C6A7> /usr/lib/system/libcommonCrypto.dylib
       0x1013a4000 -        0x1013abfff  libcompiler_rt.dylib (62) <968B8E3F-3681-3230-9D78-BB8732024F6E> /usr/lib/system/libcompiler_rt.dylib
       0x1013b9000 -        0x1013c2ffb  libcopyfile.dylib (146.50.5) <3885083D-50D8-3EEC-B481-B2E605180D7F> /usr/lib/system/libcopyfile.dylib
       0x1013c9000 -        0x10144efff  libcorecrypto.dylib (562.50.17) <67007279-24E1-3F30-802D-A55CD5C27946> /usr/lib/system/libcorecrypto.dylib
       0x10146b000 -        0x1014a4ff7  libdispatch.dylib (913.50.12) <848EEE57-4235-3A61-9A52-C0097DD2AB5E> /usr/lib/system/libdispatch.dylib
       0x1014df000 -        0x1014fcff7  libdyld.dylib (551.3) <CF59A5A5-288B-30E6-BD42-9056B4E4139A> /usr/lib/system/libdyld.dylib
       0x101517000 -        0x101517ffb  libkeymgr.dylib (28) <E34E283E-90FA-3C59-B48E-1277CDB9CDCE> /usr/lib/system/libkeymgr.dylib
       0x10151c000 -        0x10151cff7  liblaunch.dylib (1205.50.76) <4D52BB64-C568-3A36-8935-2480427EE2A2> /usr/lib/system/liblaunch.dylib
       0x101523000 -        0x101527ffb  libmacho.dylib (906) <1902A611-081A-3452-B11E-EBD1B166E831> /usr/lib/system/libmacho.dylib
       0x10152e000 -        0x101530ff3  libquarantine.dylib (86) <26C0BA22-8F93-3A07-9A4E-C8D53D2CE42E> /usr/lib/system/libquarantine.dylib
       0x101537000 -        0x101538ff3  libremovefile.dylib (45) <711E18B2-5BBE-3211-A916-56740C27D17A> /usr/lib/system/libremovefile.dylib
       0x10153e000 -        0x101555fff  libsystem_asl.dylib (356.50.1) <3B24F2D1-B578-359D-ADB2-0ED19A364C38> /usr/lib/system/libsystem_asl.dylib
       0x101563000 -        0x101563fff  libsystem_blocks.dylib (67) <17303FDF-0D2D-3963-B05E-B4DF63052D47> /usr/lib/system/libsystem_blocks.dylib
       0x101568000 -        0x1015f1ff7  libsystem_c.dylib (1244.50.9) <1187BFE8-4576-3247-8177-481554E1F9E7> /usr/lib/system/libsystem_c.dylib
       0x10161a000 -        0x10161dffb  libsystem_configuration.dylib (963.50.8) <DF6B5287-203E-30CB-9947-78DF446C72B8> /usr/lib/system/libsystem_configuration.dylib
       0x101624000 -        0x101627ffb  libsystem_coreservices.dylib (51) <486000D3-D8CB-3BE7-8EE5-8BF380DE6DF7> /usr/lib/system/libsystem_coreservices.dylib
       0x10162d000 -        0x10162efff  libsystem_darwin.dylib (1244.50.9) <09C21A4A-9EE0-388B-A9D9-DFF8F6758791> /usr/lib/system/libsystem_darwin.dylib
       0x101634000 -        0x10163aff7  libsystem_dnssd.dylib (878.50.17) <9033B909-BCF7-37EB-A040-ADE8081611D6> /usr/lib/system/libsystem_dnssd.dylib
       0x101641000 -        0x10168aff7  libsystem_info.dylib (517.30.1) <AB634A98-B8AA-3804-8436-38261FC8EC4D> /usr/lib/system/libsystem_info.dylib
       0x1016a2000 -        0x1016edfcb  libsystem_m.dylib (3147.50.1) <FC2E58BA-E6D5-3D2A-89CA-67F212294136> /usr/lib/system/libsystem_m.dylib
       0x1016fc000 -        0x10171bfff  libsystem_malloc.dylib (140.50.6) <7FD43735-9DDD-300E-8C4A-F909A74BDF49> /usr/lib/system/libsystem_malloc.dylib
       0x101728000 -        0x101858ff3  libsystem_network.dylib (1229.51.2) <F65ED095-8ABA-3355-93F8-B9D0505932F4> /usr/lib/system/libsystem_network.dylib
       0x101899000 -        0x1018a3ffb  libsystem_networkextension.dylib (767.50.25) <758F1414-796D-3422-83C9-92D2915A06CE> /usr/lib/system/libsystem_networkextension.dylib
       0x1018b0000 -        0x1018b9ff3  libsystem_notify.dylib (172) <08012EC0-2CD2-34BE-BF93-E7F56491299A> /usr/lib/system/libsystem_notify.dylib
       0x1018c1000 -        0x1018c4fff  libsystem_sandbox.dylib (765.50.51) <B48D256E-D7DB-3D4F-BC95-34557170C7AA> /usr/lib/system/libsystem_sandbox.dylib
       0x1018cb000 -        0x1018ccff3  libsystem_secinit.dylib (30) <DE8D14E8-A276-3FF8-AE13-77F7040F33C1> /usr/lib/system/libsystem_secinit.dylib
       0x1018d2000 -        0x1018f8ff7  libsystem_kernel.dylib (4570.51.2) <7FF1E390-2FAC-33A7-A545-60D9BB6692D4> /usr/lib/system/libsystem_kernel.dylib
       0x101912000 -        0x101919ff7  libsystem_platform.dylib (161.50.1) <6355EE2D-5456-3CA8-A227-B96E8F1E2AF8> /usr/lib/system/libsystem_platform.dylib
       0x101922000 -        0x10192dfff  libsystem_pthread.dylib (301.50.1) <0E51CCBA-91F2-34E1-BF2A-FEEFD3D321E4> /usr/lib/system/libsystem_pthread.dylib
       0x10193a000 -        0x101941ff7  libsystem_symptoms.dylib (820.50.37) <ED7350E0-B68B-374A-9AB7-DC1F195A8327> /usr/lib/system/libsystem_symptoms.dylib
       0x101948000 -        0x10195bfff  libsystem_trace.dylib (829.50.17) <6568D68B-1D4C-38EE-90A9-54821D6403C0> /usr/lib/system/libsystem_trace.dylib
       0x10196a000 -        0x10196fff7  libunwind.dylib (35.3) <BEF3FB49-5604-3B5F-82B5-332B80023AC3> /usr/lib/system/libunwind.dylib
       0x101977000 -        0x1019a4fff  libxpc.dylib (1205.50.76) <25DB244E-217D-3CE0-A8F2-0C4255783B42> /usr/lib/system/libxpc.dylib
       0x1019c7000 -        0x1019faff7  libclosured.dylib (551.3) <DC3DA678-9C40-339C-A9C6-32AB74FCC682> /usr/lib/closure/libclosured.dylib
       0x101a16000 -        0x101a3aff7  libc++abi.dylib (400.8.2) <EF5E37D7-11D9-3530-BE45-B986612D13E2> /usr/lib/libc++abi.dylib
       0x101a4a000 -        0x101aa0fff  libc++.1.dylib (400.9) <7D3DACCC-3804-393C-ABC1-1A580FD00CB6> /usr/lib/libc++.1.dylib
    0x7fff61e04000 -     0x7fff61e4e9df  dyld (551.3) <AFAB4EFA-7020-34B1-BBEF-0F26C6D3CA36> /usr/lib/dyld

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 950
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=22.8M resident=0K(0%) swapped_out_or_unallocated=22.8M(100%)
Writable regions: Total=26.4M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=26.4M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        2 
MALLOC                             9.9M        9 
MALLOC guard page                   16K        5 
__DATA                            4332K       61 
__LINKEDIT                        4716K       47 
__TEXT                            18.2M       47 
__UNICODE                          560K        2 
__UNIXSTACK                       16.0M        2 
shared memory                        8K        3 
===========                     =======  ======= 
TOTAL                             53.6M      169

It seems not a bug of Python.
Maybe, your Python installation is broken, or you have wrong PYTHONPATH envvar.

> Maybe, your Python installation is broken, or you have wrong PYTHONPATH envvar.

We should maybe dump sys.path into stderr on the specific "unable to load the file system codec" error.

if it could be broken installation or configuration (like incorrectly set
PYTHONHOME or any other python related environment variables), then this
crash should be expected every time I invoke python executable and attempt
to run python scripts.
However, it is not a case, and some of my scripts are running just fine.
I definitely have some bugs in certain scripts (they are accessing sqlite3)
and this is when I am seeing this crash of python.
...and it is python, who is crashing:
I think whatever bugs in my scripts are - it is not a valid reason for
python (which is wrapper around c code and compiles c code on the fly) to
crash.

I have reinstalled python3 on my laptop with brew and for half of the day
it was running fine...

On Wed, May 23, 2018 at 2:59 AM, STINNER Victor <report@bugs.python.org>
wrote:

>
> STINNER Victor <vstinner@redhat.com> added the comment:
>
> > Maybe, your Python installation is broken, or you have wrong PYTHONPATH
> envvar.
>
> We should maybe dump sys.path into stderr on the specific "unable to load
> the file system codec" error.
>
> ----------
> nosy: +vstinner
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue33611>
> _______________________________________
>

> if it could be broken installation or configuration (like incorrectly set
> PYTHONHOME or any other python related environment variables), then this
> crash should be expected every time I invoke python executable and attempt
> to run python scripts.
> However, it is not a case, and some of my scripts are running just fine.

There are many possibilities.
* Your scripts may have different shebang
* You're using some tools selecting Python (e.g. pyenv)
* You're using some tools modifying env vars
etc...

Anyway, your report doesn't have enough information to specify what's wrong on your environment.
Valid Python environment doesn't raise such error.  I use Python everyday.

> We should maybe dump sys.path into stderr on the specific "unable to load the file system codec" error.

In addition to sys.path, I think "What's current python home (prefix and exec-prefix), and where it come from" is very useful.

I recently fixed Vim's bug. https://github.com/vim/vim/pull/2787

Vim called Py_SetPythonHome with path of Python when vim is compiled.  After reinstalling Python to another directory, vim fails with "no module named encoding" error everytime.  This suffered many homebrew users because homebrew use different directory to install Python every revisions.

Python home will be come from different locations:

* Calculated from Python executable path
* Embedded in Python static/dynamic library
* PYTHONHOME environment variable
* Py_SetPythonHome()

I completed the _PyCoreConfig structure to add home, module_search_path_env, and many other options. It might help to guess how the final configuration has been computed. I also reworked the code to clarify when and how sys.path is computed. sys.path value (not the actual sys.path Python list) is now computed earlier.

Guys,
First of all - thank you for accepting it and trying to fix it.
I lost two days attempting to resurrect python3 on my laptop and ended up
with uninstalling it and restoring python 2.7.10.
I agree (and I think it is separate and well known python's problem) -
there are several tools/techniques  installing python and paths ARE
different between OS. this is what I was observing today comparing Mac OS
and CentOS 7.
and as it seems to me this "complaint" about "encoding" is exactly about it
- as it is such a directory on Linux and there is no such directory on Mac.
In any case, whatever way it all this handled, it should not cause python
to die and more importantly - something is making changes in configuration
of python on the system and user is unable to use python after this error
AT ALL.
I had to reinstall it completely several times yesterday and today because
of this.
About shebangs.
yes, I do have different ones (and I suspect many of us do, as python's
very interesting feature of incompatibility between major versions)
And after all these my attempts to fix python 3 on my laptop I ended up
with total mess on the system - none of the pythons were working.
So, I said - you know... I'll wait for python 3 be more mature...
Perhaps that is the reason Apple is not installing more resent version and
keeps 2.7.10? :)

Regards,
Slava

On Wed, May 23, 2018 at 6:20 PM, STINNER Victor <report@bugs.python.org>
wrote:

>
> STINNER Victor <vstinner@redhat.com> added the comment:
>
> I completed the _PyCoreConfig structure to add home,
> module_search_path_env, and many other options. It might help to guess how
> the final configuration has been computed. I also reworked the code to
> clarify when and how sys.path is computed. sys.path value (not the actual
> sys.path Python list) is now computed earlier.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue33611>
> _______________________________________
>

I feel your message is just a complaint, not issue report.
You didn't report any concrete information about your environment. (e.g. How did you install Python, which bash do you use, `env` output, `which python` output, etc...)

Your message made I feel bad, without any useful information.
If you really want to fix your environment, you can use other place (python-list, stackoverflow, or something else).

I think we can improve error message, but I don't want to keep using this issue.
May I close this issue?

msg317525, s (sm): "Guys, (...)"

In the future please use gender-neutral words such as "folks" and "y'all" instead of "guys". Thanks.

Slava doesn't seem to want to provide any useful information to debug his event, whereas Naoki asked multiple times gently. I also dislike Slava's tone in answers, trolling Python 3 doesn't help to fix your issue...

There are millions of Python 3 users on Windows, and it's the first time that I see this bug on Windows. So it's likely something very specific to your setup.

It's very unlikely a bug in the Windows installer of Python, so I close the issue.

:)
sure. if you want to close it - it's up to you.
if you however, want to fix it, please tell me what information do you need
me to provide.
I don't think my tone was bad/offensive. ...and it's still  a bug,
regardless of my tone or anything else.
and by the way, I don't like yours tone.

I used brew and pip

bash --version
GNU bash, version 4.4.12(1)-release (x86_64-apple-darwin16.3.0)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>

which python
/usr/bin/python

env
_system_type=Darwin
OPT=/opt
LANG=en_US.UTF-8
XPC_FLAGS=0x0
DISPLAY=/private/tmp/com.apple.launchd.XNIGqOXCBz/org.macosforge.xquartz:0
rvm_bin_path=/Users/xxxx/.rvm/bin
TERM_SESSION_ID=781A3902-E266-48B2-8BDF-0114DE0CAC97
rvm_version=1.26.11 (latest)
RUBY_VERSION=ruby-2.2.3
GEM_HOME=/Users/xxxx/.rvm/gems/ruby-2.2.3
JAVA_HOME=/usr
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.tIfMUtKKFM/Listeners
GVM_ROOT=/Users/xxxx/.gvm
STDLIB=/usr/local/lib
USER=xxxx
RBENV_SHELL=bash
GOPATH=/Users/xxxx/working/golang/projects
PWD=/Users/xxxx/working/python/scripts/passman/db
_system_version=10.13
HOME=/Users/xxxx
GOROOT=/usr/local/go
LC_CTYPE=en_US.UTF-8
_system_name=OSX
rvm_user_install_flag=1
TERM_PROGRAM=Apple_Terminal
TERM_PROGRAM_VERSION=404
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.zL9jzzDVzY/Render
_system_arch=x86_64
GEM_PATH=/Users/xxxx/.rvm/gems/ruby-2.2.3:/Users/xxxx/.rvm/gems/ruby-2.2.3@global
TMPDIR=/var/folders/g8/xflwp75s5nv0lc4wmhz2t2700000gn/T/
rvm_stored_umask=0022
rvm_loaded_flag=1
rvm_path=/Users/xxxx/.rvm
XPC_SERVICE_NAME=0
SHELL=/bin/bash
TERM=xterm-256color
SECURITYSESSIONID=186a7
rvm_prefix=/Users/xxxx
SHLVL=2
PYENV_SHELL=bash
PYTHONPATH=/usr/local/lib/python2.7/site-packages
LOGNAME=xxxx
LDFLAGS=/usr/local/opt/bison/lib
GOBIN=/usr/local/go/bin
STDINCLUDE=/usr/local/include
MY_RUBY_HOME=/Users/xxxx/.rvm/rubies/ruby-2.2.3
PATH=/opt/:/usr/local/lib:/usr/local/include:/usr/local/opt/bison/bin:/usr/local/opt/e2fsprogs/sbin:/Users/xxxx/.pyenv/shims:/Users/xxxx/.rvm/gems/ruby-2.2.3/bin:/Users/xxxx/.rvm/gems/ruby-2.2.3@global
/bin:/Users/xxxx/.rvm/rubies/ruby-2.2.3/bin:/Users/xxxx/.rbenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/opt/X11/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS:/Applications/Postgres.app/Contents/Versions/latest/bin/:/Users/xxxx/.rvm/bin:/usr/local/go/bin:/Users/xxxx/working/golang/projects:/usr/local/go/bin:/usr/local/include:/opt:/usr/local/lib:/Users/xxxx/.rvm/bin:/usr/local/go/bin:/usr/local:/opt
IRBRC=/Users/xxxx/.rvm/rubies/ruby-2.2.3/.irbrc
GOROOT_BOOTSTRAP=/usr/local/go
_=/usr/bin/env

On Thu, May 24, 2018 at 7:27 AM, STINNER Victor <report@bugs.python.org>
wrote:

>
> STINNER Victor <vstinner@redhat.com> added the comment:
>
> Slava doesn't seem to want to provide any useful information to debug his
> event, whereas Naoki asked multiple times gently. I also dislike Slava's
> tone in answers, trolling Python 3 doesn't help to fix your issue...
>
> There are millions of Python 3 users on Windows, and it's the first time
> that I see this bug on Windows. So it's likely something very specific to
> your setup.
>
> It's very unlikely a bug in the Windows installer of Python, so I close
> the issue.
>
> ----------
> resolution:  -> not a bug
> stage:  -> resolved
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue33611>
> _______________________________________
>

>  PYTHONPATH=/usr/local/lib/python2.7/site-packages

That looks pretty suspicious right there. It's pulling in 2.7 modules. Normally you should never need to set PYTHONPATH in a shell initialization script, like .profile, rarely elsewhere.