Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

help([object]) returns "Not enough memory." on standard Python types, object and object functions #64113

Closed
hct mannequin opened this issue Dec 6, 2013 · 8 comments
Closed
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) OS-windows type-bug An unexpected behavior, bug, or error

Comments

@hct
Copy link
Mannequin

hct mannequin commented Dec 6, 2013

BPO 19914
Nosy @terryjreedy, @vstinner, @tjguk, @bitdancer, @briancurtin, @eryksun

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-12-14.01:37:46.285>
created_at = <Date 2013-12-06.23:27:27.509>
labels = ['interpreter-core', 'type-bug', 'invalid', 'OS-windows']
title = 'help([object]) returns "Not enough memory." on standard Python types, object and object functions'
updated_at = <Date 2016-03-16.08:03:15.250>
user = 'https://bugs.python.org/hct'

bugs.python.org fields:

activity = <Date 2016-03-16.08:03:15.250>
actor = 'BreamoreBoy'
assignee = 'none'
closed = True
closed_date = <Date 2013-12-14.01:37:46.285>
closer = 'terry.reedy'
components = ['Interpreter Core', 'Windows']
creation = <Date 2013-12-06.23:27:27.509>
creator = 'hct'
dependencies = []
files = []
hgrepos = []
issue_num = 19914
keywords = []
message_count = 8.0
messages = ['205413', '205437', '205766', '205780', '205782', '205861', '206158', '261838']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'vstinner', 'tim.golden', 'r.david.murray', 'brian.curtin', 'hct', 'eryksun']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19914'
versions = ['Python 3.3']

@hct
Copy link
Mannequin Author

hct mannequin commented Dec 6, 2013

not sure if this ever worked. first time using help([object]), but these should work according to the documentation. OS is Win7 SP1 32-bit.

Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> help(list)
Not enough memory.

>>> help(list())
Not enough memory.

>>> help([])
Not enough memory.

>>> help(tuple())
Not enough memory.

>>> help(())
Not enough memory.

>>> help(str.format)
Not enough memory.

>>> help(str)
Not enough memory.

>>> help(b'1234')
Not enough memory.

>>> help(str.strip)
Not enough memory.

>>> help(str.count)
Not enough memory.

>>

@hct hct mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Dec 6, 2013
@bitdancer
Copy link
Member

This works in general, so there must be something odd going on with your system.

@hct
Copy link
Mannequin Author

hct mannequin commented Dec 10, 2013

verified issue is due to code page was set to 65001, but I set PYTHONIOENCODING to utf-8 (tried UTF-8, utf8, utf-8...etc), so I'm not sure why there is problem with code page 65001

setting code page back to ascii (non-Unicode) fixed the issue.

@BreamoreBoy
Copy link
Mannequin

BreamoreBoy mannequin commented Dec 10, 2013

I can confirm that code page 65001 is the problem using 3.3.3 on Windows 7.

@vstinner
Copy link
Member

It looks like the issue comes from the "more" system command, used as a pager for the documentation. When the OEM code page is set to 65001 (ex: type "chcp 65001" in a Windows console), "more document.txt" does nothing (display nothing and exit).

Try commamands:
---
more document.txt
# works fine
chcp 65001
more document.txt
# no output
---

It might be related to this:
http://stackoverflow.com/questions/3401802/codepage-850-works-65001-fails-there-is-no-response-to-call-foo-cmd-interna

@hct
Copy link
Mannequin Author

hct mannequin commented Dec 10, 2013

the other issue I'm also seeing is that help() doesn't seem to take exactly an object as an optional argument. perhaps Python manual for help() should be updated according to the actual implementation?

>>> help('hello')
no Python documentation found for 'hello'

>>> type('hello')
<class 'str'>
>>> a='hello'
>>> help(a)
no Python documentation found for 'hello'

>>

@terryjreedy
Copy link
Member

cp65001 fails in many ways quite independent of Python.
Idle:
>>> "ሴé€㑖Ѓ⌴*"
'ሴé€㑖Ѓ⌴*'
Pasting the same string into Command Prompt (Win 7, USA, updated):
C:\Users\Terry>echo "?‚????*"
"?‚????*"

C:\Users\Terry>chcp 65001
Active code page: 65001

C:\Users\Terry>echo "*"
The system cannot write to the specified device.
---

help('xyz') treats string instances specially. Try 'topics' or 'LISTS'.
I opened bpo-19980 for improving the 'not found' message.

@eryksun
Copy link
Contributor

eryksun commented Mar 16, 2016

For posterity for anyone that finds this old issue, I investigated this problem in the debugger in Windows 7. It turns out that more.com (the pager used by Python's help) calls MultiByteToWideChar 1 with dwFlags passed as MB_PRECOMPOSED (1), which is forbidden for UTF-8. The error message is just a generic error that incorrectly assumes decoding the byte string failed due to running out of memory.

You may be happy to learn that this problem is fixed in Windows 10.

Here are a few snapshots from the debugger.

more.com calls SetConsoleConversions from its init function, InitializeThings:

Breakpoint 0 hit
more!MORE::InitializeThings:
00000000`ff293058 48895c2408      mov     qword ptr [rsp+8],rbx
            ss:00000000`0024f7a0=0000000000000000
0:000> g
Breakpoint 2 hit
ulib!WSTRING::SetConsoleConversions:
000007fe`f6498934 8a05d6a80000    mov     al,byte ptr
            [ulib!WSTRING::_UseAnsiConversions
             (000007fe`f64a3210)] ds:000007f
e`f64a3210=00

This causes decoding byte strings to use the current console codepage instead of the system ANSI or OEM codepage. The intention here is to allow a user to correctly display a text file that's in a different encoding. The decoded text is written to the console as Unicode via WriteConsoleW.

Here is the bad call where dwFlags (register rdx) is passed as MB_PRECOMPOSED (1), which is invalid for codepage 65001 (register rcx).

0:000> g
Breakpoint 1 hit
KERNELBASE!MultiByteToWideChar:
000007fe`fd191f00 fff3            push    rbx
0:000> ? @rcx
Evaluate expression: 65001 = 00000000`0000fde9
0:000> r rdx
rdx=0000000000000001

In Windows 10 this argument is passed as 0, the correct value.

This problem occurs indirectly via a utility library named ulib.dll, which is used by Windows command-line utilities. It should only occur when console conversions are enabled. Otherwise ulib converts using the system OEM and ANSI codepages. I searched for other utilities that use ulib!WSTRING::SetConsoleConversions:

C:\>for %f in (C:\Windows\system32\*.exe) do @(^
More? dumpbin /imports "%f" | ^
More? findstr SetConsoleConversions && echo %f)
           7FF713B8934   167 ?SetConsoleConversions@WSTRING@@SAXXZ
C:\Windows\system32\find.exe

I found that find.exe is also subject to this bug in Windows 7. It fails to print the result if the console is using codepage 65001:

C:\Temp\test>type test
eggs
spam

C:\Temp\test>find /n "spam" *

---------- TEST
[2]spam

C:\Temp\test>chcp 65001
Active code page: 65001

C:\Temp\test>find /n "spam" *

---------- TEST

This works correctly in Windows 10.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) OS-windows type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants