This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Access violation when using the C version of the io module
Type: crash Stage: resolved
Components: IO Versions: Python 2.7
Status: closed Resolution: out of date
Dependencies: Superseder: Document that binary IO classes work with bytes-likes objects
View: 20699
Assigned To: Nosy List: OscarL, benjamin.peterson, christian.heimes, pitrou, santoso.wijaya, serhiy.storchaka
Priority: normal Keywords:

Created on 2011-06-15 15:41 by OscarL, last changed 2022-04-11 14:57 by admin. This issue is now closed.

File name Uploaded Description Edit santoso.wijaya, 2011-06-16 20:13 santoso.wijaya, 2011-06-16 20:17
Messages (9)
msg138382 - (view) Author: (OscarL) Date: 2011-06-15 15:41
This is on Windows XP SP2. Using Python 2.7.1 and 2.7.2.

While executing the unit tests of the PySerial package, version 2.5 ( one of the tests fails when using the C version of the io module ("io"), but pass if the pure Python version is used ("").

The specific test case in question is: When executed like:

    > python loop://

the test fails with the following traceback:

Traceback (most recent call last):
  File "C:\pyserial-2.5\test\", line 68, in test_hello_raw

    self.failUnlessEqual(hello, unicode("hello\n"))
AssertionError: u'<memory at 0x00C5D940>' != u'hello\n'

And Windows shows a crash report dialog, with the following information:

AppName: python.exe	 AppVer:	 ModName: python27.dll
ModVer: 2.7.2150.1013	 Offset: 00089f57

Exception Information
Code: 0xc0000005	Flags: 0x00000000
Record: 0x0000000000000000	Address: 0x000000001e089f57

As said above, if in the said test case, instead of "import io" one does "import _pyio as io", the crash does not happens, and the test succeeds.
msg138404 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-06-15 22:46
Regarding the crash,

From what I can see, the `tp_clear` method of bufferedrwpair is called during Py_Finalize() that leads to garbage collection. This NULLs `self->writer` for the rwpair object.

Later, in the same garbage collection routine, the `tp_clear` of textio (the wrapper) is called. This attempts to finalize its wrapped object by first calling the `closed` property of the wrapped object. This property read is propagated down to the wrapped bufferedrwpair, `bufferedrwpair_closed_get()`.

At this point, `self->writer` for the bufferedrwpair object is already NULL from the previous clear. Hence, the crash happens because a NULL pointer dereferencing is attempted.
msg138406 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-06-16 00:27
Can you produce a self-contained test case?
msg138463 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-06-16 20:13
As for the "<memory at %p>" string, when _io.BufferedWriter prepares the byte buffer into a PyMemoryView wrapper and passes it into the raw IO object:

    res = PyObject_CallMethodObjArgs(self->raw, _PyIO_str_write, memobj, NULL);

For some reason, memobj.__repr__ is called before being passed to the raw IO's write method.

I can't reproduce this odd behavior using a dumb raw IO that extends _io.RawIOBase. Mimicking what pyserial does, however, reproduces both issues (see attached).
msg138464 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-06-16 20:16
Note: Removing threading call (lock acquire, release) would remove the crash, but still triggers the "<memory at %p>" conversion of PyMemoryView.
msg138466 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-06-16 20:25
If `write(self, data)` is expected to receive `data` as a memoryview object, then it's up to pyserial to use `data.tobytes()` instead of `bytes(data)`, though this does not seem to be documented in the io module doc.
msg201214 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-10-25 06:52
The issue hasn't seen an update for over two years. Were you able to solve the issue?
msg201239 - (view) Author: (OscarL) Date: 2013-10-25 14:33
I can't reproduce the access violation on Windows, using Python 2.7.3.

Using the script by santa4nt, I can see that the fail still happens.

As Santoso Wijaya (santa4nt) said in a previous comment:

The `write(data)` method is getting a `memoryview` object, thus the test code should be using `data.tobytes()` instead of using `bytes(data)`. If we introduce that logic on the script it passes.

The thing is that if we, instead of doing `import io` (the C version of the module) we do `import _pyio as io` (the pure Python version) the FAIL does not appears at all in the original script!

That difference ("`write` might receive a `memoryview` object if using the C version of this module") is not documented anywhere.

It should, if that's the expected behavior [*], and the code using the io module is responsible for using .tobytes() instead of bytes().

[*] Why "" behaves differently?
msg235775 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-11 23:01
Looks as the crash was fixed in one of previous bugfixes. As for memoryview related issue, this is a duplicate of issue20699.
Date User Action Args
2022-04-11 14:57:18adminsetgithub: 56549
2015-11-26 17:33:38serhiy.storchakasetstatus: pending -> closed
resolution: out of date
stage: resolved
2015-02-11 23:01:51serhiy.storchakasetstatus: languishing -> pending

nosy: + serhiy.storchaka
messages: + msg235775

superseder: Document that binary IO classes work with bytes-likes objects
2013-10-25 14:33:26OscarLsetmessages: + msg201239
2013-10-25 06:52:41christian.heimessetstatus: open -> languishing
nosy: + christian.heimes
messages: + msg201214

2011-06-16 20:25:40santoso.wijayasetmessages: + msg138466
2011-06-16 20:17:50santoso.wijayasetfiles: +
2011-06-16 20:16:24santoso.wijayasetmessages: + msg138464
2011-06-16 20:13:08santoso.wijayasetfiles: +

messages: + msg138463
2011-06-16 00:27:43benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg138406
2011-06-15 22:46:57santoso.wijayasetnosy: + santoso.wijaya
messages: + msg138404
2011-06-15 15:49:22r.david.murraysetnosy: + pitrou
2011-06-15 15:41:54OscarLcreate