Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Malloc errors in test_io #49864

Closed
ronaldoussoren opened this issue Mar 30, 2009 · 12 comments
Closed

Malloc errors in test_io #49864

ronaldoussoren opened this issue Mar 30, 2009 · 12 comments
Labels
3.7 (EOL) end of life OS-mac type-bug An unexpected behavior, bug, or error

Comments

@ronaldoussoren
Copy link
Contributor

BPO 5614
Nosy @ronaldoussoren, @pitrou, @ned-deily, @wkschwartz

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-11-17.16:04:26.233>
created_at = <Date 2009-03-30.22:02:55.707>
labels = ['OS-mac', 'type-bug', '3.7']
title = 'Malloc errors in test_io'
updated_at = <Date 2018-10-10.07:07:36.905>
user = 'https://github.com/ronaldoussoren'

bugs.python.org fields:

activity = <Date 2018-10-10.07:07:36.905>
actor = 'ronaldoussoren'
assignee = 'none'
closed = True
closed_date = <Date 2009-11-17.16:04:26.233>
closer = 'ronaldoussoren'
components = ['macOS']
creation = <Date 2009-03-30.22:02:55.707>
creator = 'ronaldoussoren'
dependencies = []
files = []
hgrepos = []
issue_num = 5614
keywords = []
message_count = 12.0
messages = ['84672', '84674', '84695', '84706', '84710', '84712', '84723', '84724', '84726', '95386', '327446', '327457']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'pitrou', 'ned.deily', 'William.Schwartz']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue5614'
versions = ['Python 3.7']

@ronaldoussoren
Copy link
Contributor Author

The malloc warnings happen on OSX, with a fresh checkout of the python3
branch. Python was build using --enable-universalsdk=/ --with-universal-
archs=all, on a x86_64 capable laptop running Leopard.

The issue goes away when running from the commandline, the "-E -bb"
flags that 'make test' use seem to be involved in the issue.

test_io

Testing large file ops skipped on darwin.
It requires 2147483648 bytes and a long time.
Use 'regrtest.py -u largefile test_io' to run it.

Testing large file ops skipped on darwin.
It requires 2147483648 bytes and a long time.
Use 'regrtest.py -u largefile test_io' to run it.
python.exe(35535,0x7fff701d1720) malloc: *** mmap(size=-
9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
python.exe(35535,0x7fff701d1720) malloc: *** mmap(size=-
9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
python.exe(35535,0x7fff701d1720) malloc: *** mmap(size=-
9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
test_ioctl

@ronaldoussoren
Copy link
Contributor Author

What's more annoying: the error goes away while running in a debugger.

The issue happens at least in tes_constructor:

test_constructor (main.CBufferedWriterTest) ...
python.exe(35957,0x7fff701d1720) malloc: *** mmap(size=-
9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
ok

@ronaldoussoren
Copy link
Contributor Author

Buffer_size is set to 0x7fffffffffffffff in BufferedWriter_init at the end
of test_constructor.

I have no idea why this happens, but this definitly seems wrong to me.
Debugging is rather hard at the moment because the issue goes away in the
debugger.

Setting this as a release blocker because arguments of a wrong magnitude
to memory allocation functions are scary.

@ronaldoussoren ronaldoussoren added release-blocker type-crash A hard crash of the interpreter, possibly with a core dump labels Mar 30, 2009
@ocean-city
Copy link
Mannequin

ocean-city mannequin commented Mar 30, 2009

malloc: *** mmap(size=-9223372036854775808) failed

Does this error occur even if you revert Modules/mmapmodule.c to r69214?
I have touched this module recently (I believe I fixed only SEGV bugs,
not changed behavior) so I want to know it.

@ronaldoussoren
Copy link
Contributor Author

I don't think this is related to Python's mmap module, the message says
that malloc(3) cannot mmap some extra memory space.

I'll test with the older version of mmap just in case.

@ocean-city
Copy link
Mannequin

ocean-city mannequin commented Mar 30, 2009

Sorry, this is not related to mmap module... Forgot my comments.

@pitrou
Copy link
Member

pitrou commented Mar 30, 2009

If you look at test_io, calling malloc() with sys.maxsize is precisely
the purpose of the test, to check that allocation failures are detected
and reported properly. So the warnings can be safely ignored, if they
only happen in test_constructor.

@pitrou
Copy link
Member

pitrou commented Mar 30, 2009

PS: if the warnings are annoying, the test can be disabled selectively
under Mac OS X.

@ronaldoussoren
Copy link
Contributor Author

I can't believe that I completely missed that this is the purpose of the
tests, I honestly believed that I had checked everything there was to
check :-(

Disabling the test won't be necessary, I'm keeping this issue open for a
while longer to check if we can avoid getting a message on stderr during
the test. It's rather annoying that malloc logs error messages during what
turns out to be perfectly fine behaviour.

@ronaldoussoren
Copy link
Contributor Author

Closing the issue as this is not a bug in python (as noted in the
comments) and there doesn't seem to be a workaround for avoiding the
spurious messages on stderr.

@ronaldoussoren ronaldoussoren added type-bug An unexpected behavior, bug, or error and removed type-crash A hard crash of the interpreter, possibly with a core dump labels Nov 17, 2009
@wkschwartz
Copy link
Mannequin

wkschwartz mannequin commented Oct 10, 2018

In Jupyter Notebook, I tried to pass a large amount of data (about 2.3 GB) to Statsmodels's KDEUnivariate.fit (https://www.statsmodels.org/dev/generated/statsmodels.nonparametric.kde.KDEUnivariate.fit.html#statsmodels.nonparametric.kde.KDEUnivariate.fit) with kernel='cos'. In the Notebook, I got a MemoryError exception and traceback (which unfortunately I have since lost).

In the terminal where I was running the Jupyter server, the Python 3.7 kernel spat out:

Python(86244,0x7fff91104380) malloc: *** mach_vm_map(size=465845322670080) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

The MemoryError coming up in the Notebook interface is the correct behavior, but I wanted to bring to your attention that the malloc junk on stderr happens even outside of Python's tests.

I'm on macOS 10.13.5.

@wkschwartz wkschwartz mannequin added OS-mac 3.7 (EOL) end of life labels Oct 10, 2018
@ronaldoussoren
Copy link
Contributor Author

The error is printed by libc, there's nothing we can do about that.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.7 (EOL) end of life OS-mac type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants