Actually, in the issue reported, the initial problem occurs in the evaluation of an object in a boolean context (and the subsequent problem occurs with an explicit len invocation):

http://www.selenic.com/pipermail/mercurial/2010-February/030066.html

Presumably (from memory and a brief look at the reference), when "if data:" is evaluated, Python attempts to invoke an instance's __nonzero__ method or its __len__ method. Since the mercurial.httprepo.httpsendfile class only provides a __len__ method, the __len__ method's return value is used to determine truth.

The following demonstrates this particular issue:

>>> class C:
...     def __len__(self):
...             return 2**35
...
>>> c = C()
>>> if c: pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __nonzero__ should return an int
>>> class C(object):
...     def __len__(self):
...             return 2**35
...
>>> c = C()
>>> if c: pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int

Here, I could actually argue that the message mentioning __nonzero__ is obscure: there isn't such a method defined, and __len__ is the misbehaving method. Still, in the context of boolean evaluation, the OverflowError is less helpful than it could be.