can't make IDLEX work with python._pth and python-3.6.0b2 #72708

BigStone · 2016-10-24T20:45:13Z

BPO	28522
Nosy	@terryjreedy, @pfmoore, @tjguk, @serwy, @zware, @eryksun, @zooba
PRs	[Do Not Merge] Convert `Misc/NEWS` so that it is managed by towncrier #552
Files	Report.wer

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2016-10-28.18:28:03.384>
created_at = <Date 2016-10-24.20:45:13.189>
labels = ['3.7', 'OS-windows', 'type-crash']
title = "can't make IDLEX work with python._pth and python-3.6.0b2"
updated_at = <Date 2017-03-31.16:36:32.396>
user = 'https://bugs.python.org/BigStone'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:32.396>
actor = 'dstufft'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2016-10-28.18:28:03.384>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2016-10-24.20:45:13.189>
creator = 'Big Stone'
dependencies = []
files = ['45303']
hgrepos = []
issue_num = 28522
keywords = []
message_count = 29.0
messages = ['279338', '279342', '279354', '279423', '279425', '279431', '279459', '279468', '279487', '279503', '279504', '279515', '279527', '279539', '279551', '279552', '279563', '279571', '279572', '279616', '279617', '279624', '279626', '279690', '279691', '279703', '279871', '279872', '279873']
nosy_count = 10.0
nosy_names = ['terry.reedy', 'paul.moore', 'tim.golden', 'roger.serwy', 'python-dev', 'zach.ware', 'eryksun', 'steve.dower', 'illagrenan', 'Big Stone']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue28522'
versions = ['Python 3.6', 'Python 3.7']

The text was updated successfully, but these errors were encountered:

BigStone · 2016-10-24T20:45:13Z

on WinPython-64bit-3.6.0.0Zerorc2.exe, python-3.6.0b2 based, I can't get IDLEX working with "python._pth".

If I put "Lib\site-packages\" in python._pth, python.exe dies.
If I put "#Lib\site-packages\", idlexlib is said "unable to located".

Could it be a python-3.6.0b2 bug, or just a wrong-doing from WinPython ?

"Python._pth"=
.
Lib
import site
DLLs
#Lib/site-packages
#python36.zip

zooba · 2016-10-24T21:53:56Z

Might have to use a backslash in the path - I don't remember whether I tried to handle forward slashes or not, but I suspect not. Definitely tried it with site-packages though.

BigStone · 2016-10-25T05:23:25Z

python.exe crashes when I try this python._pth:
.
Lib
import site
DLLs
Lib\site-packages
#python36.zip

all variation I try on Lib\site-packages do fail, when I not commenting # the line...
Nevertheless, jupyter/numpy/bokeh do work with original setting.

I'm lost in thoughts: how adding a line in this file can make Python crash ?

zooba · 2016-10-25T17:29:39Z

Can you tell me more about the crash? It doesn't cause a crash when I try it with my own install.

BigStone · 2016-10-25T17:43:04Z

I just see a windows screen poping up with (translated from french)

"Python has stopped to work"

a problem caused this program to stop working correctly. Windows is going to close this program and will inform you if a solution is available.

zooba · 2016-10-25T17:58:15Z

Check in the Event Log viewer to see if there is an "Application Error" entry for python.exe. Also, if you run python.exe from a command prompt there may be more information displayed in the output.

BigStone · 2016-10-25T21:38:48Z

possible particularities of my PC vs yours:

I have no python entry at all in the regex
I have no py.exe,
I have no Visual Studio (but the compiler)

with Windows 10, I don't know where is the even viewer.

zooba · 2016-10-26T00:15:38Z

If you right-click the Start button, Event Viewer is near the top.

eryksun · 2016-10-26T03:23:37Z

I installed "WinPython-64bit-3.6.0.0Zerorc2.exe" on Windows 10. As you can see below, the included version of IDLEX depends on idlelib implementation details that have changed between 3.5 and 3.6:

    C:\WinPython36\python-3.6.0b2.amd64>.\python
    Python 3.6.0b2 (default, Oct 10 2016, 21:15:32) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>>  import idlexlib
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\WinPython36\python-3.6.0b2.amd64\Lib\site-packages\idlexlib\__init__.py", line 10, in <module>
        from .idlexMain import version as __version__
      File "C:\WinPython36\python-3.6.0b2.amd64\Lib\site-packages\idlexlib\idlexMain.py", line 46, in <module>
        from idlexlib.extensionManager import extensionManager
      File "C:\WinPython36\python-3.6.0b2.amd64\Lib\site-packages\idlexlib\extensionManager.py", line 60, in <module>
        from idlelib.configHandler import idleConf, IdleConfParser
    ModuleNotFoundError: No module named 'idlelib.configHandler'

The "Unable to located" [sic] error is from the idlex.py script due to the above import error.

I couldn't reproduce the crash due to python._pth. Your Windows application log should provide the DLL (module) and exception code for the crash, but a dump file would be even better. With the error reporting dialog still open, look for the crashed python.exe in the task manager details tab (the working set of the crashed process should be small -- about 100K). Right-click it and select the option to create a dump file. Zip the dump file and upload it here.

BigStone · 2016-10-26T17:41:16Z

Event Viewer says, when I put "Lub\site-packages" in python._pth:
""""
Nom de l’application défaillante python.exe, version : 3.6.112.1013, horodatage : 0x57fc0593
Nom du module défaillant : ucrtbase.dll, version : 10.0.14393.0, horodatage : 0x578997b5
Code d’exception : 0xc0000409
Décalage d’erreur : 0x000000000006d5b8
ID du processus défaillant : 0x190c
Heure de début de l’application défaillante : 0x01d22fafd622ed09
Chemin d’accès de l’application défaillante : C:\WinPython\basedir36\build\winpython-64bit-3.6.x.0\python-3.6.0b2.amd64\python.exe
Chemin d’accès du module défaillant: C:\WINDOWS\System32\ucrtbase.dll
ID de rapport : ec44b511-6196-48a0-95ec-dc997b4d0302
Nom complet du package défaillant :
ID de l’application relative au package défaillant :

BigStone · 2016-10-26T17:53:17Z

Thanks Eryk,

So the root cause is that IDLEX is no more compatible with IDLE in python3.6.

==> I can survive this loss... Now, I don't if the "python._pth" crash is a problem, as I can stay with "#Lib\site-packages" for now.

terryjreedy · 2016-10-26T21:54:14Z

I don't know anything about 'python._pth' and whether there is any bug with respect to that.

As far as IDLE goes, there is no bug and this issue should be closed.

In bpo-24225, before the release of 3.6.0a2, most file names within idlelib were changed to shorter or lowercased names in conformance with PEP-8 and as anticipated by PEP-434. In particular, 'configHandler.py' is now 'config.py'. API changes within and between files are and will be much more disruptive to external users. As Nick Coughlin said in msg266409, 3rd party idlelib users are free to bundle or depend on a frozen copy of a past version.

WinPython should test the third party modules it includes with the python it is releasing. Importing a module is as minimal as it gets. Consider reporting the incompatibility to them.

eryksun · 2016-10-27T06:36:21Z

You may have uncovered a bug in Python that's causing the invalid parameter handler to be invoked. It would help if you uploaded the zipped dump file for the crashed process.

The status code you're getting (i.e. STATUS_STACK_BUFFER_OVERRUN, 0xC0000409) is used by the __fastfail intrinsic 1. The exception occurred in ucrtbase.dll at offset 0x6d5b8. In a debugger you can see it's in the CRT's invoke_watson function:

0:000> u ucrtbase + 0x6d5b8 - 0x18 L7
ucrtbase!invoke_watson:
00007ffd`8984d5a0 4883ec28        sub     rsp,28h
00007ffd`8984d5a4 b917000000      mov     ecx,17h
00007ffd`8984d5a9 ff1531310400    call    qword ptr
    [ucrtbase!_imp_IsProcessorFeaturePresent (00007ffd`898906e0)]
00007ffd`8984d5af 85c0            test    eax,eax
00007ffd`8984d5b1 7407            je      ucrtbase!invoke_watson+0x1a
                                            (00007ffd`8984d5ba)
00007ffd`8984d5b3 b905000000      mov     ecx,5
00007ffd`8984d5b8 cd29            int     29h

A fast fail executes an int 29h software interrupt, which gets handled by the following system function:

lkd> !idt 0x29
Dumping IDT: fffff801819a8070
29:     fffff8017fd66680 nt!KiRaiseSecurityCheckFailure

In this case the CRT passes a value of 5 (FAST_FAIL_INVALID_ARG) in register rcx (ecx), so it's not a critical security failure.

zooba · 2016-10-27T14:57:58Z

I suspect there's a .pth file in site-packages that is importing something to trigger the failure. Without a crash dump (or debug build) it's going to be difficult to find it, but it is certainly an unwrapped invalid parameter termination.

To save me some time, where can I get that build of WinPython from?

BigStone · 2016-10-27T17:28:07Z

hi Steve,

You can grab it there https://sourceforge.net/projects/winpython/files/WinPython_3.6/3.6.0.0/betas/WinPython-64bit-3.6.0.0Zerorc2.exe/download
MD5 | SHA-1 | SHA-256 | Binary | Size
---------------------------------|------------------------------------------|------------------------------------------------------------------|---------------------------------|-------------------
dd946ed17ee86ea035361d2e757a1cc1 | f0ec7ffac477a220dd24aea3fb70afaba579df00 | af6536f1922a044ac74300efcd275c9e25c5eb56140ded84a99f11d38ae5ac7b | WinPython-64bit-3.6.0.0Zerorc2.exe | 24 196 083 Bytes
dabae69ad09e1646625d3a8995a75056 | aaece4907096422c1df78f80a42a2268369d7697 | 0d530b84f29481e7f03e4615c9da489711ca6883b49996219d9cd13aa5393330 | WinPython-64bit-3.6.0.0rc2.exe | 208 255 944 Bytes

BigStone · 2016-10-27T17:34:55Z

maybe click on the "WinPython Command Prompt.exe" and do "pip uninstall IDLEX" as a first step. so you should see IDLE working

zooba · 2016-10-27T19:02:18Z

It's a genuine bug in path processing, specifically how we handle buffer resizing. I'll make a fix.

python-dev · 2016-10-27T21:29:38Z

New changeset eea669163131 by Steve Dower in branch '3.6':
Issue bpo-28522: Fixes mishandled buffer reallocation in getpathp.c
https://hg.python.org/cpython/rev/eea669163131

New changeset 72e64fc8746b by Steve Dower in branch 'default':
Issue bpo-28522: Fixes mishandled buffer reallocation in getpathp.c
https://hg.python.org/cpython/rev/72e64fc8746b

zooba · 2016-10-27T21:31:36Z

Fixed and added a test. (Yes I know that it's not the most efficient algorithm for joining the strings together, but I consider correctness to be more important here.)

BigStone · 2016-10-28T17:06:52Z

will it be in python-3.6.0b3 ?

what should be in python._pth, in WinPython particular case ?
(as Lib\site-packages didn't seem needed, for unknown reason)

terryjreedy · 2016-10-28T17:14:11Z

Yes, see the commit to branch 3.6, which will next be released as .0b3.

zooba · 2016-10-28T18:28:03Z

"Lib\site-packages" is probably unnecessary because of "import site", which likely adds it in anyway.

It's very likely that WinPython doesn't actually want to specify this at all, since it also enables isolated mode, which will ignore PYTHONPATH and the current working directory. But if that's okay, you probably want::

python36.zip
DLLs
Lib
.
import site

That should give you the same default sys.path as if the file were omitted, except for the empty entry at the start and anything in PYTHONPATH or the registry. (I figured this out by running "python -S" and looking at sys.path.)

eryksun · 2016-10-28T20:42:55Z

ignore PYTHONPATH and the current working directory

Generally it's the script directory that isolated mode removes from sys.path. It's the working directory when there is no script.

BigStone · 2016-10-29T18:18:24Z

thank you all for the patch

IDLEX was a requirement for a french examination.

I think the reason was to see the line numbers on the left of the editor.

For sure since IDLEX birth, IDLE has made some progress and IDLEX is becoming irrelevant, but this lovely tiny feature seems still missing.

BigStone · 2016-10-29T18:29:03Z

the "show line number on the left" feature is on the "github web editor", on "atom", and on "spyder" and "erik" python IDE, so rather the expected standard for python editing.

terryjreedy · 2016-10-30T02:57:03Z

bpo-17535 is about adding line numbers to IDLE editor. I have approved it in priciple, but not yet the proposed patch, or a revision thereof.

BigStone · 2016-11-01T11:22:38Z

the suggested python._pth change makes python unhappy:

python36.zip
DLLs
Lib
.
import site

Récipient d’erreurs 116251549737, type 5
Nom d’événement : BEX64
Réponse : Non disponible
ID de CAB : 0

Signature du problème :
P1 : python.exe
P2 : 3.6.112.1013
P3 : 57fc0593
P4 : ucrtbase.dll
P5 : 10.0.14393.0
P6 : 578997b5
P7 : 000000000006d5b8
P8 : c0000409
P9 : 0000000000000005
P10 :

Fichiers joints :
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERF455.tmp.WERInternalMetadata.xml
\\?\C:\Users\famille\AppData\Local\Temp\WERFF72.tmp.appcompat.txt

Ces fichiers sont peut-être disponibles ici :
C:\ProgramData\Microsoft\Windows\WER\ReportArchive\AppCrash_python.exe_ec9ba1cd21c08ca3de75f183bf945ce737867af_b3b3e14f_1575008a

Symbole d’analyse :
Nouvelle recherche de la solution : 0
ID de rapport : 66625eb0-e311-4368-be37-4b600bcb8978
Statut du rapport : 1
Récipient avec hachage : 57d09a5cb4a63ab86af1e62bd270b573

BigStone · 2016-11-01T11:24:37Z

oups! I may have test the old one...

BigStone · 2016-11-01T11:26:59Z

it looks ok with 3.6.0b3 ... sorry for the false alarm

BigStone mannequin added the OS-windows label Oct 24, 2016

zooba self-assigned this Oct 27, 2016

zooba added 3.7 (EOL) end of life type-crash A hard crash of the interpreter, possibly with a core dump labels Oct 27, 2016

zooba closed this as completed Oct 28, 2016

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

can't make IDLEX work with python._pth and python-3.6.0b2 #72708

can't make IDLEX work with python._pth and python-3.6.0b2 #72708

BigStone mannequin commented Oct 24, 2016

BigStone mannequin commented Oct 24, 2016

zooba commented Oct 24, 2016

BigStone mannequin commented Oct 25, 2016

zooba commented Oct 25, 2016

BigStone mannequin commented Oct 25, 2016

zooba commented Oct 25, 2016

BigStone mannequin commented Oct 25, 2016

zooba commented Oct 26, 2016

eryksun commented Oct 26, 2016

BigStone mannequin commented Oct 26, 2016

BigStone mannequin commented Oct 26, 2016

terryjreedy commented Oct 26, 2016

eryksun commented Oct 27, 2016

zooba commented Oct 27, 2016

BigStone mannequin commented Oct 27, 2016

BigStone mannequin commented Oct 27, 2016

zooba commented Oct 27, 2016

python-dev mannequin commented Oct 27, 2016

zooba commented Oct 27, 2016

BigStone mannequin commented Oct 28, 2016

terryjreedy commented Oct 28, 2016

zooba commented Oct 28, 2016

eryksun commented Oct 28, 2016

BigStone mannequin commented Oct 29, 2016

BigStone mannequin commented Oct 29, 2016

terryjreedy commented Oct 30, 2016

BigStone mannequin commented Nov 1, 2016

BigStone mannequin commented Nov 1, 2016

BigStone mannequin commented Nov 1, 2016

can't make IDLEX work with python._pth and python-3.6.0b2 #72708

can't make IDLEX work with python._pth and python-3.6.0b2 #72708

Comments

BigStone mannequin commented Oct 24, 2016

BigStone mannequin commented Oct 24, 2016

zooba commented Oct 24, 2016

BigStone mannequin commented Oct 25, 2016

zooba commented Oct 25, 2016

BigStone mannequin commented Oct 25, 2016

zooba commented Oct 25, 2016

BigStone mannequin commented Oct 25, 2016

zooba commented Oct 26, 2016

eryksun commented Oct 26, 2016

BigStone mannequin commented Oct 26, 2016

BigStone mannequin commented Oct 26, 2016

terryjreedy commented Oct 26, 2016

eryksun commented Oct 27, 2016

zooba commented Oct 27, 2016

BigStone mannequin commented Oct 27, 2016

BigStone mannequin commented Oct 27, 2016

zooba commented Oct 27, 2016

python-dev mannequin commented Oct 27, 2016

zooba commented Oct 27, 2016

BigStone mannequin commented Oct 28, 2016

terryjreedy commented Oct 28, 2016

zooba commented Oct 28, 2016

eryksun commented Oct 28, 2016

BigStone mannequin commented Oct 29, 2016

BigStone mannequin commented Oct 29, 2016

terryjreedy commented Oct 30, 2016

BigStone mannequin commented Nov 1, 2016

BigStone mannequin commented Nov 1, 2016

BigStone mannequin commented Nov 1, 2016