classification
Title: OverflowError in RLock.acquire()
Type: crash Stage:
Components: Library (Lib), Windows Versions: Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, davidar, effbot, ggenellina, pitrou
Priority: normal Keywords:

Created on 2009-07-24 11:26 by davidar, last changed 2009-08-05 02:34 by davidar. This issue is now closed.

Files
File name Uploaded Description Edit
tileproviders.py davidar, 2009-07-25 08:38 snippets from OSMTileProvider, DynamicTileProvider, TileProvider, TileCache
osm.log davidar, 2009-07-26 02:43 logging related to exceptions in os.path.join
tilestore.py davidar, 2009-07-26 02:44 snippet from tilestore.py
Messages (14)
msg90879 - (view) Author: David Roberts (davidar) Date: 2009-07-24 11:26
I'm getting the following error on Windows in an application I've
written (the error does not occur on Linux):

Exception in thread Thread-4:
Traceback (most recent call last):
 File "C:\Python26\lib\threading.py", line 525, in __bootstrap_inner
   self.run()
 File "C:\Documents and Settings\David\My
Documents\pyzui\pyzui\tileprovider.py", line 97, in run
   self.__tilecache[tile_id] = Tile(tile)
 File "C:\Documents and Settings\David\My
Documents\pyzui\pyzui\tilecache.py", line 165, in __setitem__
   with self.__lock:
 File "C:\Python26\lib\threading.py", line 115, in acquire
   me = current_thread()
 File "C:\Python26\lib\threading.py", line 803, in currentThread
   return _active[_get_ident()]
OverflowError: can't convert negative value to unsigned long

Where __lock is an RLock object.

The odd thing is that it only affects a single class (which is derived
from the TileProvider class in tileprovider.py, which in turn is derived
from threading.Thread). This led me to believe there was an error in my
code, but I asked on the mailing list and was told that it is likely a
bug in the threading module.

The Python version is 2.6.2.
msg90883 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-07-24 13:20
This is strange, thread.get_ident() isn't supposed to raise this kind of
error.
Could you call it individually from the same thread to see what it returns?
Also, could you post the code for your derived class for us to see?
msg90911 - (view) Author: David Roberts (davidar) Date: 2009-07-25 08:38
thread.get_ident() returns 1192 and 1560 in the cases where there is no
error, and 1316 in the case where the error is thrown. Doesn't seem
particularly unusual, and shows get_ident() isn't throwing the error itself.

I also noticed that there is a warning right before the exception is raised:

C:\Python26\lib\threading.py:803: RuntimeWarning: tp_compare didn't
return -1 or -2 for exception
  return _active[_get_ident()]

I've attached the relevant snippets of code. As you can see
OSMTileProvider (the class that is triggering the exception) doesn't
actually do all that much, and the two other classes derived from
DynamicTileProvider function correctly so I don't understand what it is
about this specific class.
msg90912 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-07-25 09:36
Could you provide a complete example for us to reproduce the issue?
msg90915 - (view) Author: David Roberts (davidar) Date: 2009-07-25 11:12
I haven't been able to isolate the issue. Could someone provide some
insight into what the error could possibly mean so that I have a better
idea of what I'm trying to isolate?
msg90916 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-07-25 11:38
> I also noticed that there is a warning right before the exception is raised:
> 
> C:\Python26\lib\threading.py:803: RuntimeWarning: tp_compare didn't
> return -1 or -2 for exception
>   return _active[_get_ident()]

Right, it probably means the exception was raised before this very code
line, but wasn't properly signalled to the Python runtime.
Does the error disappear if you comment out the "with self.__lock" line?
msg90919 - (view) Author: David Roberts (davidar) Date: 2009-07-25 12:35
If I comment out all occurrences of "with self.__lock:" I then get the
same error in another part of the code:

C:\Python26\lib\threading.py:803: RuntimeWarning: tp_compare didn't
return -1 or -2 for exception
  return _active[_get_ident()]
Exception in thread Thread-4:
Traceback (most recent call last):
  File "C:\Python26\lib\threading.py", line 525, in __bootstrap_inner
    self.run()
  File "C:\Documents and Settings\David\My
Documents\pyzui\pyzui\tileprovider.py", line 82, in run
    self.__tasks_available.acquire()
  File "C:\Python26\lib\threading.py", line 115, in acquire
    me = current_thread()
  File "C:\Python26\lib\threading.py", line 803, in currentThread
    return _active[_get_ident()]
OverflowError: can't convert negative value to unsigned long

Where self.__tasks_available is a threading.Condition.

If I comment out that bit, I then get the following error:

Traceback (most recent call last):
  File "C:\Documents and Settings\David\My
Documents\pyzui\pyzui\tileprovider.py", line 91, in run
    tile = self._load(tile_id)
  File "C:\Documents and Settings\David\My
Documents\pyzui\pyzui\dynamictileprovider.py", line 55, in _load
    tile_id, True, filext=self.filext)
  File "C:\Documents and Settings\David\My
Documents\pyzui\pyzui\tilestore.py", line 72, in get_tile_path
    prefix = get_media_path(media_id)
  File "C:\Documents and Settings\David\My
Documents\pyzui\pyzui\tilestore.py", line 46, in get_media_path
    media_dir = os.path.join(tile_dir, media_hash)
  File "C:\Python26\lib\ntpath.py", line 97, in join
    if path[-1] in "/\\":
IndexError: cannot fit 'int' into an index-sized integer

I'm not sure if this error is related to the other errors, but similarly
this error only occurs in this specific thread.
msg90920 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-07-25 12:47
Le samedi 25 juillet 2009 à 12:35 +0000, David Roberts a écrit :
>   File "C:\Python26\lib\ntpath.py", line 97, in join
>     if path[-1] in "/\\":
> IndexError: cannot fit 'int' into an index-sized integer

This is definitely fishy and shouldn't happen in such a line of code.
Perhaps there is some deficient exception handling in one of the
extension modules you are using.

What if you remove calls to Image.open() (I assume it is the PIL
library?) and simply return filenames instead of Image objects?
msg90933 - (view) Author: David Roberts (davidar) Date: 2009-07-26 02:43
Yes, it is the PIL library. Removing calls to Image.open() still results
in the same error - the exception is thrown before it even reaches that
bit of the code anyway.

Another odd thing is that the exception is only thrown on some of the
calls to get_tile_path, every second call to be exact - see the attached
log file.

I'm also attaching the relevant code from tilestore.py.
msg91253 - (view) Author: David Roberts (davidar) Date: 2009-08-04 11:07
I think I've narrowed it down to the ImageQt class provided by PIL -
commenting out the reference to this (in the constructor of the Tile
class referenced by TileProvider.run) stops the errors.

So how do I go about determining where the problem with ImageQt lies?
(It's only ~50 lines, so there can't be too many things that could
possibly go wrong)
msg91254 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-08-04 11:26
I don't know, but you should probably contact the author, Fredrik Lundh,
and/or post on the PIL list. Since it is probably not a bug in Python
itself, I'm closing the issue.
(please reopen if you discover a problem in Python itself)
msg91255 - (view) Author: Fredrik Lundh (effbot) * (Python committer) Date: 2009-08-04 12:04
PIL is completely thread-agnostic, so I not sure there's anything PIL can 
do to fix this.

(and ImageQt is of course an interface to PyQt, which is an interface to 
Qt, which consists of a *lot* more than 50 lines...)
msg91266 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-08-04 19:03
This error is certainly due to a C function that sets an exception but
does not return an error value.
When running with a debug build of python, you will see "XXX undetected
error" printed to stderr.
msg91298 - (view) Author: David Roberts (davidar) Date: 2009-08-05 02:34
Ok, so if it's a bug in (Py)Qt then I'm not going to worry about it.

I've managed to fix the issue in my case anyway, by (essentially) replacing:
    image = Image.open(fname)
    image.load()
    tile = ImageQt(image)
with (the much more obvious way to do it):
    tile = QImage(fname)
History
Date User Action Args
2009-08-05 02:34:32davidarsetmessages: + msg91298
2009-08-04 19:03:18amaury.forgeotdarcsetmessages: + msg91266
2009-08-04 12:04:53effbotsetmessages: + msg91255
2009-08-04 11:26:55pitrousetstatus: open -> closed

nosy: + effbot
messages: + msg91254

resolution: not a bug
2009-08-04 11:07:50davidarsetmessages: + msg91253
2009-07-26 02:44:11davidarsetfiles: + tilestore.py
2009-07-26 02:43:22davidarsetfiles: + osm.log

messages: + msg90933
2009-07-25 12:47:15pitrousetmessages: + msg90920
2009-07-25 12:35:38davidarsetmessages: + msg90919
2009-07-25 11:38:56pitrousetmessages: + msg90916
2009-07-25 11:12:35davidarsetmessages: + msg90915
2009-07-25 09:36:21amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg90912
2009-07-25 08:38:24davidarsetfiles: + tileproviders.py

messages: + msg90911
2009-07-25 07:39:46ggenellinasetnosy: + ggenellina
2009-07-24 13:20:42pitrousetnosy: + pitrou
messages: + msg90883
2009-07-24 11:26:34davidarcreate