This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: socket._fileobject: read raises AttributeError when closed in another thread
Type: behavior Stage: needs patch
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, martin.panter, panzi, pitrou
Priority: normal Keywords:

Created on 2010-07-05 01:21 by panzi, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
FileObject.py panzi, 2010-07-05 01:21 bugfix by subtyping socket._fileobject and overriding close and closed
Messages (7)
msg109281 - (view) Author: Mathias Panzenböck (panzi) * Date: 2010-07-05 01:21
When you open a socket._fileobject through sock.makefile('rb') or similar and you read blocking in one thread and close the file object from another thread the reading thread gets an AttributeError. This is because the close method sets the underlying fileobject._sock member to None and the read/readline/... methods call recv on this member without checking if its None. I think the _sock member should not be set to None at all but a flag should be set that this file object is closed.

For the time being I use the "bugfix" I attached and therefore do not call sock.makefile('rb') but FileObject(sock, 'rb'). FileObject is a subtype of socket._fileobject that overrides the close method and closed property.

I don't know if this bug persists in 2.7 or 3.x. I still use Fedora 12 and that comes with Python 2.6.2.
msg175247 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-09 19:42
I'm not sure there's really a point in making socket.makefile() objects thread-safe. In any case, making them thread-safe would probably require more than this small fix.
msg175250 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2012-11-09 20:55
I agree with Antoine. IMHO it's common knowledge that file and socket object aren't thread safe. All access to these objects must be synced with a lock.
msg175251 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-09 21:21
The problem is that the other thread is blocked, waiting for new client connections. A common pattern for very simple server applications (like you write during exercises at universities) is:

2 threads:
 1. A server thread waiting for clients, maybe spawning even more threads for each connection.
 2. A small shell thread reading commands from stdin. When "quit" is read it closes the server socket, which shuts down the server (the server thread just notices that the socket was closed and shuts down properly).

This pattern we used all the time when writing exercises in Java. However, it seems to work quite differently in Python 3. I have to check if this problem still exists.
msg175252 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-09 21:28
> 2 threads:
>  1. A server thread waiting for clients, maybe spawning even more
> threads for each connection.
>  2. A small shell thread reading commands from stdin. When "quit" is
> read it closes the server socket, which shuts down the server (the
> server thread just notices that the socket was closed and shuts down
> properly).

You're originally talking about socket._fileobject. I don't see what
that has to do with a listening socket being closed ("a server thread
waiting for clients").
msg175254 - (view) Author: Mathias Panzenböck (panzi) * Date: 2012-11-09 22:35
Yeah, I don't remember anymore. It was so long ago.
msg269001 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-21 14:22
IMO closing an OS-level file descriptor in one thread while it is in use by another thread is a bad idea, full of race conditions and undefined behaviour. An AttributeError sounds like a best-case scenario. It is like freeing a memory allocation in one thread while another thread is accessing the memory. What if the other thread was slow, or a syscall was interrupted, and hasn’t (re-)started the recv() call in time? What if a third thread opens a file and reuses the file descriptor you just closed?

I suggest to close this.
History
Date User Action Args
2022-04-11 14:57:03adminsetgithub: 53402
2016-06-21 14:22:33martin.pantersetstatus: open -> closed

nosy: + martin.panter
messages: + msg269001

resolution: rejected
2012-11-09 22:35:28panzisetmessages: + msg175254
2012-11-09 21:28:13pitrousetmessages: + msg175252
2012-11-09 21:21:02panzisetmessages: + msg175251
2012-11-09 20:55:54christian.heimessetnosy: + christian.heimes
messages: + msg175250
2012-11-09 19:42:01pitrousetmessages: + msg175247
2012-11-09 13:24:54ezio.melottisetnosy: + pitrou
stage: needs patch

versions: + Python 2.7, - Python 2.6
2010-07-05 01:21:33panzicreate