classification
Title: help([object]) returns "Not enough memory." on standard Python types, object and object functions
Type: behavior Stage:
Components: Interpreter Core, Windows Versions: Python 3.3
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: brian.curtin, eryksun, hct, r.david.murray, terry.reedy, tim.golden, vstinner
Priority: normal Keywords:

Created on 2013-12-06 23:27 by hct, last changed 2016-03-16 08:03 by BreamoreBoy. This issue is now closed.

Messages (8)
msg205413 - (view) Author: HCT (hct) Date: 2013-12-06 23:27
not sure if this ever worked. first time using help([object]), but these should work according to the documentation. OS is Win7 SP1 32-bit.

Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> help(list)
Not enough memory.

>>> help(list())
Not enough memory.

>>> help([])
Not enough memory.

>>> help(tuple())
Not enough memory.

>>> help(())
Not enough memory.

>>> help(str.format)
Not enough memory.

>>> help(str)
Not enough memory.

>>> help(b'1234')
Not enough memory.

>>> help(str.strip)
Not enough memory.

>>> help(str.count)
Not enough memory.

>>>
msg205437 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-12-07 03:33
This works in general, so there must be something odd going on with your system.
msg205766 - (view) Author: HCT (hct) Date: 2013-12-10 01:55
verified issue is due to code page was set to 65001, but I set PYTHONIOENCODING to utf-8 (tried UTF-8, utf8, utf-8...etc), so I'm not sure why there is problem with code page 65001

setting code page back to ascii (non-Unicode) fixed the issue.
msg205780 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2013-12-10 09:24
I can confirm that code page 65001 is the problem using 3.3.3 on Windows 7.
msg205782 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-12-10 10:32
It looks like the issue comes from the "more" system command, used as a pager for the documentation. When the OEM code page is set to 65001 (ex: type "chcp 65001" in a Windows console), "more document.txt" does nothing (display nothing and exit).

Try commamands:
---
more document.txt
# works fine
chcp 65001
more document.txt
# no output
---

It might be related to this:
http://stackoverflow.com/questions/3401802/codepage-850-works-65001-fails-there-is-no-response-to-call-foo-cmd-interna
msg205861 - (view) Author: HCT (hct) Date: 2013-12-10 21:41
the other issue I'm also seeing is that help() doesn't seem to take exactly an object as an optional argument. perhaps Python manual for help() should be updated according to the actual implementation?

>>> help('hello')
no Python documentation found for 'hello'

>>> type('hello')
<class 'str'>
>>> a='hello'
>>> help(a)
no Python documentation found for 'hello'

>>>
msg206158 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-12-14 01:37
cp65001 fails in many ways quite independent of Python.
Idle:
>>> "ሴé€㑖Ѓ⌴*"
'ሴé€㑖Ѓ⌴*'
Pasting the same string into Command Prompt (Win 7, USA, updated):
C:\Users\Terry>echo "?‚????*"
"?‚????*"

C:\Users\Terry>chcp 65001
Active code page: 65001

C:\Users\Terry>echo "*"
The system cannot write to the specified device.
---

help('xyz') treats string instances specially. Try 'topics' or 'LISTS'.
I opened #19980 for improving the 'not found' message.
msg261838 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-03-16 04:28
For posterity for anyone that finds this old issue, I investigated this problem in the debugger in Windows 7. It turns out that more.com (the pager used by Python's help) calls MultiByteToWideChar [1] with dwFlags passed as MB_PRECOMPOSED (1), which is forbidden for UTF-8. The error message is just a generic error that incorrectly assumes decoding the byte string failed due to running out of memory.

You may be happy to learn that this problem is fixed in Windows 10.

[1]: https://msdn.microsoft.com/en-us/library/dd319072

Here are a few snapshots from the debugger.

more.com calls SetConsoleConversions from its init function, InitializeThings:

    Breakpoint 0 hit
    more!MORE::InitializeThings:
    00000000`ff293058 48895c2408      mov     qword ptr [rsp+8],rbx
                ss:00000000`0024f7a0=0000000000000000
    0:000> g
    Breakpoint 2 hit
    ulib!WSTRING::SetConsoleConversions:
    000007fe`f6498934 8a05d6a80000    mov     al,byte ptr
                [ulib!WSTRING::_UseAnsiConversions
                 (000007fe`f64a3210)] ds:000007f
    e`f64a3210=00

This causes decoding byte strings to use the current console codepage instead of the system ANSI or OEM codepage. The intention here is to allow a user to correctly display a text file that's in a different encoding. The decoded text is written to the console as Unicode via WriteConsoleW.

Here is the bad call where dwFlags (register rdx) is passed as MB_PRECOMPOSED (1), which is invalid for codepage 65001 (register rcx).

    0:000> g
    Breakpoint 1 hit
    KERNELBASE!MultiByteToWideChar:
    000007fe`fd191f00 fff3            push    rbx
    0:000> ? @rcx
    Evaluate expression: 65001 = 00000000`0000fde9
    0:000> r rdx
    rdx=0000000000000001

In Windows 10 this argument is passed as 0, the correct value.

This problem occurs indirectly via a utility library named ulib.dll, which is used by Windows command-line utilities. It should only occur when console conversions are enabled. Otherwise ulib converts using the system OEM and ANSI codepages.  I searched for other utilities that use ulib!WSTRING::SetConsoleConversions:

    C:\>for %f in (C:\Windows\system32\*.exe) do @(^
    More? dumpbin /imports "%f" | ^
    More? findstr SetConsoleConversions && echo %f)
               7FF713B8934   167 ?SetConsoleConversions@WSTRING@@SAXXZ
    C:\Windows\system32\find.exe

I found that find.exe is also subject to this bug in Windows 7. It fails to print the result if the console is using codepage 65001:

    C:\Temp\test>type test
    eggs
    spam

    C:\Temp\test>find /n "spam" *

    ---------- TEST
    [2]spam

    C:\Temp\test>chcp 65001
    Active code page: 65001

    C:\Temp\test>find /n "spam" *

    ---------- TEST

This works correctly in Windows 10.
History
Date User Action Args
2016-03-16 08:03:15BreamoreBoysetnosy: - BreamoreBoy
2016-03-16 04:28:54eryksunsetnosy: + eryksun
messages: + msg261838
2013-12-14 01:37:46terry.reedysetstatus: open -> closed

nosy: + terry.reedy
messages: + msg206158

resolution: not a bug
2013-12-10 21:41:09hctsetmessages: + msg205861
2013-12-10 10:32:12vstinnersetmessages: + msg205782
2013-12-10 09:24:29BreamoreBoysetnosy: + BreamoreBoy
messages: + msg205780
2013-12-10 07:31:26serhiy.storchakasetnosy: + vstinner, tim.golden, brian.curtin
components: + Windows
2013-12-10 01:55:54hctsetmessages: + msg205766
2013-12-07 03:33:55r.david.murraysetnosy: + r.david.murray
messages: + msg205437
2013-12-06 23:27:27hctcreate