classification
Title: itertools.tee causes segfault in a multithreading environment, while the equivalent implementation doesn't
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: itertools.tee not thread-safe; can segfault interpreter when wrapped iterator releases GIL
View: 34410
Assigned To: Nosy List: Dmitry Odzerikho, rhettinger, serhiy.storchaka, xtreak
Priority: normal Keywords:

Created on 2015-06-21 10:46 by Dmitry Odzerikho, last changed 2018-08-28 11:20 by xtreak. This issue is now closed.

Files
File name Uploaded Description Edit
test_segfault.py Dmitry Odzerikho, 2015-06-21 11:02
test_segfault.py serhiy.storchaka, 2015-06-21 17:28
Messages (4)
msg245588 - (view) Author: Dmitry Odzerikho (Dmitry Odzerikho) Date: 2015-06-21 10:59
Hi,

I'm using iterators returned by itertools.tee in different threads. The original iterator passed to itertools.tee is thread-safe, however, it doesn't guarantees that the cloned iterators are thread safe too. However the equivalent implementation of itertools.tee in pure Python seems to work fine in this case. Is it expected or I'm doing something wrong?

I'm attaching a simple example that shows the behaviour.
msg245602 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-06-21 16:56
> Is it expected?

No, it is a bug.
msg245605 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-06-21 17:28
I can't reproduce the crash in 3.3+ (this doesn't matter the bug is gone), but can reproduce in 2.7 and 3.2. Here is a reproducer that compatible with 2.7 and 3.x.
msg324239 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2018-08-28 11:20
I can also reproduce the original script's segfault in 2.7 but not in master. There is a related bug as per the original description by Dmitry that itertools.tee is not thread safe in https://bugs.python.org/msg323817 and a related SO answer to this that will add some context https://stackoverflow.com/a/44638570/2610955

Thanks
History
Date User Action Args
2018-08-28 11:20:10xtreaksetnosy: + xtreak
messages: + msg324239
2018-08-28 11:12:47xiang.zhangsetstatus: open -> closed
superseder: itertools.tee not thread-safe; can segfault interpreter when wrapped iterator releases GIL
resolution: duplicate
stage: needs patch -> resolved
2015-06-21 17:28:03serhiy.storchakasetfiles: + test_segfault.py

messages: + msg245605
2015-06-21 16:56:41rhettingersetmessages: + msg245602
2015-06-21 11:47:05serhiy.storchakasetnosy: + serhiy.storchaka
2015-06-21 11:07:43ezio.melottisetnosy: + rhettinger

stage: needs patch
2015-06-21 11:02:50Dmitry Odzerikhosetfiles: - test_segfault.py
2015-06-21 11:02:43Dmitry Odzerikhosetfiles: + test_segfault.py
2015-06-21 10:59:20Dmitry Odzerikhosetfiles: + test_segfault.py

messages: + msg245588
title: itertools.tee causes segfault in a multithreading environment, while the equivalent implementation does'nt -> itertools.tee causes segfault in a multithreading environment, while the equivalent implementation doesn't
2015-06-21 10:46:00Dmitry Odzerikhocreate