Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on test_issue_1395_5 with Python 2.7 and VS2010 #57670

Closed
sable mannequin opened this issue Nov 23, 2011 · 10 comments
Closed

Error on test_issue_1395_5 with Python 2.7 and VS2010 #57670

sable mannequin opened this issue Nov 23, 2011 · 10 comments
Assignees
Labels
easy topic-IO type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@sable
Copy link
Mannequin

sable mannequin commented Nov 23, 2011

BPO 13461
Nosy @loewis, @amauryfa, @pitrou, @briancurtin, @serhiy-storchaka
Files
  • issue13461-27.patch
  • issue13461-3x.patch: similar patch for 3.x to not use numerical literals
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2013-08-20.18:24:04.174>
    created_at = <Date 2011-11-23.14:02:38.396>
    labels = ['easy', 'expert-IO', 'type-crash']
    title = 'Error on test_issue_1395_5 with Python 2.7 and VS2010'
    updated_at = <Date 2013-08-20.18:24:04.172>
    user = 'https://bugs.python.org/sable'

    bugs.python.org fields:

    activity = <Date 2013-08-20.18:24:04.172>
    actor = 'serhiy.storchaka'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2013-08-20.18:24:04.174>
    closer = 'serhiy.storchaka'
    components = ['IO']
    creation = <Date 2011-11-23.14:02:38.396>
    creator = 'sable'
    dependencies = []
    files = ['30238', '30239']
    hgrepos = []
    issue_num = 13461
    keywords = ['patch', 'easy']
    message_count = 10.0
    messages = ['148184', '148186', '148257', '148327', '148328', '182102', '182103', '189061', '195704', '195708']
    nosy_count = 9.0
    nosy_names = ['loewis', 'amaury.forgeotdarc', 'pitrou', 'sable', 'brian.curtin', 'python-dev', 'serhiy.storchaka', 'prlw1', 'Yogesh.Chaudhari']
    pr_nums = []
    priority = 'high'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue13461'
    versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Nov 23, 2011

    I am trying to get Python working when compiled with Visual Studio 2010 (cf bpo-13210).

    When running the tests with the python 2.7 branch compiled with VS2010, the "test_issue_1395_5" in test_io.py will cause Python to eat the whole memory within a few seconds and make the server completely unresponsive.

    @sable sable mannequin added the topic-IO label Nov 23, 2011
    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Nov 23, 2011

    The problem is in CTextIOWrapperTest.test_issue1395_5
    Here is the backtrace:

    msvcr100d.dll!memset()  Line 145	Asm
    

    msvcr100d.dll!_heap_alloc_dbg_impl(unsigned __int64 nSize, int nBlockUse, const char * szFileName, int nLine, int * errno_tmp) Line 498 C++
    msvcr100d.dll!_nh_malloc_dbg_impl(unsigned __int64 nSize, int nhFlag, int nBlockUse, const char * szFileName, int nLine, int * errno_tmp) Line 239 + 0x22 bytes C++
    msvcr100d.dll!_nh_malloc_dbg(unsigned __int64 nSize, int nhFlag, int nBlockUse, const char * szFileName, int nLine) Line 302 + 0x2a bytes C++
    msvcr100d.dll!malloc(unsigned __int64 nSize) Line 56 + 0x21 bytes C++
    python27_d.dll!PyObject_Malloc(unsigned __int64 nbytes) Line 944 C
    python27_d.dll!_PyObject_DebugMallocApi(char id, unsigned __int64 nbytes) Line 1445 + 0xa bytes C
    python27_d.dll!_PyObject_DebugMalloc(unsigned __int64 nbytes) Line 1413 C
    python27_d.dll!PyString_FromStringAndSize(const char * str, __int64 size) Line 88 + 0x11 bytes C
    python27_d.dll!do_mkvalue(const char * * p_format, char * * p_va, int flags) Line 427 + 0xf bytes C
    python27_d.dll!va_build_value(const char * format, char * va, int flags) Line 537 + 0x14 bytes C
    python27_d.dll!_Py_VaBuildValue_SizeT(const char * format, char * va) Line 511 C
    python27_d.dll!_PyObject_CallMethod_SizeT(_object * o, char * name, char * format, ...) Line 2671 + 0xf bytes C
    python27_d.dll!textiowrapper_tell(textio * self, _object * args) Line 2222 + 0x2c bytes C

    So the problem happens when calling in textio.c:
    {{{
    PyObject *decoded = PyObject_CallMethod(
                self->decoder, "decode", "s#", input, 1);
    }}}

    self->decoder is of type "_io.IncrementalNewlineDecoder" and input is "BBB".

    This will result in PyString_FromStringAndSize being called with size = 4294967297, which will cause the server to fall.

    @pitrou
    Copy link
    Member

    pitrou commented Nov 24, 2011

    What if you replace:

    PyObject *decoded = PyObject_CallMethod(
                self->decoder, "decode", "s#", input, 1);

    with:

    PyObject *decoded = PyObject_CallMethod(
                self->decoder, "decode", "s#", input, (Py_ssize_t) 1);

    @sable
    Copy link
    Mannequin Author

    sable mannequin commented Nov 25, 2011

    Thanks Antoine! It solved the issue.

    I will check soon with Python trunk to see if the same thing applies.

    @pitrou pitrou added the type-crash A hard crash of the interpreter, possibly with a core dump label Nov 25, 2011
    @amauryfa
    Copy link
    Member

    I've identified a few other cases where a '#' format is passed a numeric literal:

    Python/codecs.c:514: return Py_BuildValue("(u#n)", &end, 0, end);
    Modules/_io/textio.c:2323: DECODER_DECODE(input, 1, n);

    @serhiy-storchaka
    Copy link
    Member

    Can we fix this easy issue before 2.7.4 release?

    @pitrou
    Copy link
    Member

    pitrou commented Feb 14, 2013

    Ageed, it's probably easy enough.

    @YogeshChaudhari
    Copy link
    Mannequin

    YogeshChaudhari mannequin commented May 12, 2013

    This patch should work for 2.7 branch

    @serhiy-storchaka serhiy-storchaka self-assigned this Aug 19, 2013
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Aug 20, 2013

    New changeset 826233404be8 by Serhiy Storchaka in branch '3.3':
    Issue bpo-13461: Fix a crash in the TextIOWrapper.tell method on 64-bit platforms.
    http://hg.python.org/cpython/rev/826233404be8

    New changeset 6c9d49b8e3ec by Serhiy Storchaka in branch 'default':
    Issue bpo-13461: Fix a crash in the TextIOWrapper.tell method on 64-bit platforms.
    http://hg.python.org/cpython/rev/6c9d49b8e3ec

    New changeset 5e679ef2a55c by Serhiy Storchaka in branch '2.7':
    Issue bpo-13461: Fix a crash in the TextIOWrapper.tell method and in the "replace"
    http://hg.python.org/cpython/rev/5e679ef2a55c

    @serhiy-storchaka
    Copy link
    Member

    Thank you for the report Sébastien. Thank you for the patch Yogesh.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    easy topic-IO type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants