classification
Title: Python's crash in heavy multithreading IO operations
Type: crash Stage:
Components: Interpreter Core, Library (Lib) Versions: Python 2.4, Python 2.3, Python 2.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, sadit
Priority: normal Keywords:

Created on 2008-06-25 17:27 by sadit, last changed 2008-06-25 21:00 by amaury.forgeotdarc. This issue is now closed.

Files
File name Uploaded Description Edit
unnamed sadit, 2008-06-25 17:50
unnamed sadit, 2008-06-25 19:32
unnamed sadit, 2008-06-25 19:36
unnamed sadit, 2008-06-25 19:38
Messages (7)
msg68736 - (view) Author: Eric Sadit (sadit) Date: 2008-06-25 17:27
Hello,

I found a rare bug in heavy multithreading IO operations. The bug arises
under a stupid sequence of I/O operations. The sequence is not a normal
one, but the real problem is that the Python's interpreter crashes. The
correct behavior should be raise an IOError exception (I think).

The bug occurs when some thread closes a file or socket whilst other
thread tries to read or write from the same socket. The failure it's
inside the fileobject.c because tries to read or write the specified
data size inside a loop without check in every iteration if the object
is closed (it checks only outside the loop). The file object is set to
NULL and a SIGSEGV happens.

I'm a python developer (in the language, but not developer of the
language), so i really don't know anything about the best way to solve
this issue, and how the project it's managed. I can develop an script to
raise this bug if anyone it's interested in this problem. I'm already
owns one, but I don't find it, and the real program that arises the
error was rewritten so i don't have it.

Cheers

Eric Sadit
msg68738 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-06-25 17:44
This problem may be a duplicate of issue815646 and issue595601, and was
probably corrected with r62195.

Can you please check with the new python version 2.6b1 ?
msg68740 - (view) Author: Eric Sadit (sadit) Date: 2008-06-25 17:50
Ok, I will try to reproduce it in the python 2.6b1 version, and I report to
you the result

Saludos

On Wed, Jun 25, 2008 at 12:44 PM, Amaury Forgeot d'Arc <
report@bugs.python.org> wrote:

>
> Amaury Forgeot d'Arc <amauryfa@gmail.com> added the comment:
>
> This problem may be a duplicate of issue815646 and issue595601, and was
> probably corrected with r62195.
>
> Can you please check with the new python version 2.6b1 ?
>
> ----------
> nosy: +amaury.forgeotdarc
> resolution:  -> out of date
> status: open -> pending
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue3200>
> _______________________________________
>
msg68747 - (view) Author: Eric Sadit (sadit) Date: 2008-06-25 19:32
Thanks Amaury

I run a test script and Python 2.5.2 crashes, but 2.6b1 runs perfectly
throwing a ValueError exception (Operation of closed file)
in the reader/writer thread and "IOError: close() called during concurrent
operation on the same file object" in the closer thread.
Python doesn't crash anymore.

Cheers

Eric Sadit

On Wed, Jun 25, 2008 at 12:44 PM, Amaury Forgeot d'Arc <
report@bugs.python.org> wrote:

>
> Amaury Forgeot d'Arc <amauryfa@gmail.com> added the comment:
>
> This problem may be a duplicate of issue815646 and issue595601, and was
> probably corrected with r62195.
>
> Can you please check with the new python version 2.6b1 ?
>
> ----------
> nosy: +amaury.forgeotdarc
> resolution:  -> out of date
> status: open -> pending
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue3200>
> _______________________________________
>
msg68748 - (view) Author: Eric Sadit (sadit) Date: 2008-06-25 19:36
The test script

#!/usr/bin/python

import sys
import thread

f = file("o.test","w")
buff = " " * 1024
def run():
    while 1:
        try:
            f.read(100)
            #
f.write(buff)

        except Exception, e:
            print >>sys.stderr, "An exception, that's OK, ", repr(e)

if __name__ == '__main__':
    thread.start_new_thread(run, (), {})
    while 1:
        try:

On Wed, Jun 25, 2008 at 2:32 PM, Eric Sadit <report@bugs.python.org> wrote:

>
> Eric Sadit <donsadit@gmail.com> added the comment:
>
> Thanks Amaury
>
> I run a test script and Python 2.5.2 crashes, but 2.6b1 runs perfectly
> throwing a ValueError exception (Operation of closed file)
> in the reader/writer thread and "IOError: close() called during concurrent
> operation on the same file object" in the closer thread.
> Python doesn't crash anymore.
>
> Cheers
>
> Eric Sadit
>
> On Wed, Jun 25, 2008 at 12:44 PM, Amaury Forgeot d'Arc <
> report@bugs.python.org> wrote:
>
> >
> > Amaury Forgeot d'Arc <amauryfa@gmail.com> added the comment:
> >
> > This problem may be a duplicate of issue815646 and issue595601, and was
> > probably corrected with r62195.
> >
> > Can you please check with the new python version 2.6b1 ?
> >
> > ----------
> > nosy: +amaury.forgeotdarc
> > resolution:  -> out of date
> > status: open -> pending
> >
> > _______________________________________
> > Python tracker <report@bugs.python.org>
> > <http://bugs.python.org/issue3200>
> > _______________________________________
> >
>
> Added file: http://bugs.python.org/file10732/unnamed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue3200>
> _______________________________________
>
msg68749 - (view) Author: Eric Sadit (sadit) Date: 2008-06-25 19:38
#!/usr/bin/python

import sys
import thread

f = file("o.test","w")
buff = " " * 1024
def run():
    while 1:
        try:
            f.read(100)
            #
f.write(buff)

        except Exception, e:
            print >>sys.stderr, "An exception, that's OK, ", repr(e)

if __name__ == '__main__':
    thread.start_new_thread(run, (), {})
    while 1:
        try:
            f.close()
        except Exception, e:
            print >>sys.stderr, "An exception, main thread ", repr(e)

        f = file("o.test","w")

On Wed, Jun 25, 2008 at 2:36 PM, Eric Sadit Téllez Avila <donsadit@gmail.com>
wrote:

> The test script
>
> #!/usr/bin/python
>
> import sys
> import thread
>
> f = file("o.test","w")
> buff = " " * 1024
> def run():
>     while 1:
>         try:
>             f.read(100)
>             #
> f.write(buff)
>
>         except Exception, e:
>             print >>sys.stderr, "An exception, that's OK, ", repr(e)
>
> if __name__ == '__main__':
>     thread.start_new_thread(run, (), {})
>     while 1:
>         try:
>
>
> On Wed, Jun 25, 2008 at 2:32 PM, Eric Sadit <report@bugs.python.org>
> wrote:
>
>>
>> Eric Sadit <donsadit@gmail.com> added the comment:
>>
>>
>> Thanks Amaury
>>
>> I run a test script and Python 2.5.2 crashes, but 2.6b1 runs perfectly
>> throwing a ValueError exception (Operation of closed file)
>> in the reader/writer thread and "IOError: close() called during concurrent
>> operation on the same file object" in the closer thread.
>> Python doesn't crash anymore.
>>
>> Cheers
>>
>> Eric Sadit
>>
>> On Wed, Jun 25, 2008 at 12:44 PM, Amaury Forgeot d'Arc <
>> report@bugs.python.org> wrote:
>>
>> >
>> > Amaury Forgeot d'Arc <amauryfa@gmail.com> added the comment:
>> >
>> > This problem may be a duplicate of issue815646 and issue595601, and was
>> > probably corrected with r62195.
>> >
>> > Can you please check with the new python version 2.6b1 ?
>> >
>> > ----------
>> > nosy: +amaury.forgeotdarc
>> > resolution:  -> out of date
>> > status: open -> pending
>> >
>> > _______________________________________
>> > Python tracker <report@bugs.python.org>
>> > <http://bugs.python.org/issue3200>
>> > _______________________________________
>> >
>>
>> Added file: http://bugs.python.org/file10732/unnamed
>>
>> _______________________________________
>> Python tracker <report@bugs.python.org>
>> <http://bugs.python.org/issue3200>
>> _______________________________________
>>
>
>
msg68758 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-06-25 21:00
Closing as "already resolved in the upcoming release"
History
Date User Action Args
2008-06-25 21:00:14amaury.forgeotdarcsetstatus: pending -> closed
messages: + msg68758
2008-06-25 19:38:39saditsetfiles: + unnamed
messages: + msg68749
2008-06-25 19:36:30saditsetfiles: + unnamed
messages: + msg68748
2008-06-25 19:32:10saditsetfiles: + unnamed
messages: + msg68747
2008-06-25 17:50:41saditsetfiles: + unnamed
messages: + msg68740
2008-06-25 17:44:02amaury.forgeotdarcsetstatus: open -> pending
resolution: out of date
messages: + msg68738
nosy: + amaury.forgeotdarc
2008-06-25 17:27:52saditcreate