classification
Title: "[Errno 11] Resource temporarily unavailable" while using tracing facility/threads (in linux listdir with unicode path)
Type: crash Stage:
Components: Interpreter Core, Library (Lib) Versions: Python 2.4, Python 2.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, fabioz, terry.reedy
Priority: normal Keywords:

Created on 2008-08-02 19:11 by fabioz, last changed 2010-06-09 23:57 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
listdir_problem.py fabioz, 2008-08-02 19:11 File reproducing problem
Messages (7)
msg70638 - (view) Author: Fabio Zadrozny (fabioz) Date: 2008-08-02 19:11
A bug has been reported against pydev complaining about os.listdir()
failing while debugging (with a unicode path).

The link for that is:
http://sourceforge.net/tracker/index.php?func=detail&aid=2012138&group_id=85796&atid=577329

After trying to find its cause, it appears that pydev is exercising a
python bug (which is reproduced in the file attached). I have no idea
why that's happening. 

I've reproduced it in python 2.4 and python 2.5 under ubuntu 6.10, but I
believe it happens on other versions too -- the original bug was
reported in Ubuntu 8.04 AMD64 Hardy Heron)
msg71330 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-08-18 13:36
I do not reproduce the problem, but in your example, the trace function
is called only when the script has finished, i.e during interpreter
shutdown. Creating a thread from there is not recommended...

Can you provide the complete traceback of your error?
msg71362 - (view) Author: Fabio Zadrozny (fabioz) Date: 2008-08-18 18:55
I've pasted the output below... also, the trace function is called for
each function call after the settrace (not only in interpreter shutdown)
-- and the error happens in the listdir -- which is in the main thread,
so, it must happen before the interpreter shutdown.

Also, one thing: it works if you read an empty folder... And putting:
"print frame.f_code.co_filename, frame.f_lineno" in the 'func', it'll go
on and print
/usr/lib/python2.4/encodings/utf_8.py 15
/usr/lib/python2.4/encodings/utf_8.py 16
/usr/lib/python2.4/encodings/utf_8.py 16

For each file/dir available (so, it seems it's actually able to go on
and get all the contents, but before returning, that exception is thrown)

Output from running it:

-----------------------------------------


/usr/bin/python
Traceback (most recent call last):
  File "/home/fabioz/test workspace with spaces/test
project/src/mod1/mod2/listdir_problem.py", line 23, in ?
    print listdir(dir)
OSError: [Errno 11] Resource temporarily unavailable: '/home/fabioz/jython'
msg71377 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-08-18 21:06
I could reproduce the problem, and it appears that it is the same as
#1608818, corrected by r65037:

on posix platforms, listdir() used to check for errno only at the end of
the list; the problem is that the trace function creates a thread, and
this sets errno (harmlessly) somewhere when playing with locks.

In the python codebase, every function that checks errno should set it
to zero before the system call.
The above fix could be backported.

As a workaround, I suggest to put a line like 
   int('0')
near the end of the trace funtion (at least after the creation of the
thread) because it has the nice side-effect to reset errno.
msg71383 - (view) Author: Fabio Zadrozny (fabioz) Date: 2008-08-18 21:26
Thanks for looking into this... 

Unfortunately, I'm not sure I can use the workaround of the int('0'), as
this could fix the debugger, but if the code that's being debugged
spawned other threads (which is pretty common), it would be pointless,
but at least clients that really want this can get that fix and apply it
to their python versions...

And I think that setting the errno to 0 in the debugger every time would
make other things misbehave, as in each python call it'd clear it (or
not? I'm not really aware of the implications of doing so -- but if no
one used it, it wouldn't be there, right?)
msg71386 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-08-18 21:43
> but if the code that's being debugged
> spawned other threads (which is pretty common), it would be pointless,

No, the problem specifically lies in the implementation of listdir().
This function is not supposed to commonly spawn threads... this can only
happen with a sys.settrace function (or a very special
Py_FileSystemDefaultEncoding)

> And I think that setting the errno to 0 in the debugger every time would
> make other things misbehave, as in each python call it'd clear it

I don't think so. errno is (should be) only used shortly after the C
system call which modified it. There is no room for the trace function
to run.
Remember that the python stack is often displayed on every debugging
step, and this modifies errno more than once (to get the source file
content)
msg107443 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-06-09 23:57
"The above fix could be backported."
Too late now.
History
Date User Action Args
2010-06-09 23:57:12terry.reedysetstatus: open -> closed

nosy: + terry.reedy
messages: + msg107443

resolution: out of date
2008-08-18 21:43:32amaury.forgeotdarcsetmessages: + msg71386
2008-08-18 21:26:26fabiozsetmessages: + msg71383
2008-08-18 21:06:38amaury.forgeotdarcsetmessages: + msg71377
2008-08-18 18:55:20fabiozsetmessages: + msg71362
2008-08-18 13:36:29amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg71330
2008-08-02 19:11:17fabiozcreate