This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: left shift operator doesn't accepts long as second argument
Type: enhancement Stage: test needed
Components: Interpreter Core Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: mark.dickinson, owirj, pitrou, r.david.murray
Priority: normal Keywords: patch

Created on 2010-03-29 12:29 by owirj, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
no_outrage.patch mark.dickinson, 2010-04-06 16:26
Messages (8)
msg101887 - (view) Author: (owirj) Date: 2010-03-29 12:29
python -V:
Python 2.6.5

Executing on 32-bit machine.

From documentation ( http://docs.python.org/reference/expressions.html ), "5.7. Shifting operations":
"These operators accept plain or long integers as arguments. The arguments are converted to a common type. They shift the first argument to the left or right by the number of bits given by the second argument."

In interpreter:
>>> x = 2677691728509L << 2147483648L
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int

hint: 2147483648 = (1 << 31)

First argument is long, second is long too, so expected behavior was normal execution. But as seen from the code above interpreter gives overflow error. Looks like he tries to convert second argument to int type, because this code works gracefully:
>>> x = 2677691728509L << 2147483647L
msg101889 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-03-29 13:13
This appears to be working as designed, and so is probably a doc issue.  Python 3, which only has one integer type, has the same behavior:

>>> x = 2677691728509 << 2147483648
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long

Adding Mark Dickinson as nosy to get his opinion on whether this is a doc bug or a feature request :)
msg101891 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-03-29 14:55
I get another error on a 64-bit build:

>>> x = 2677691728509L << 2147483648L
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: outrageous left shift count

(yes, Python is outraged by such a large amount of left shifting)

The reason you get an OverflowError in 32-bit mode is that 2**31 is too large to be represented as a (signed) C long, rather than unsigned.
msg101893 - (view) Author: (owirj) Date: 2010-03-29 15:26
Antoine Pitrou:
The reason you get an OverflowError in 32-bit mode is that 2**31 is too large to be represented as a (signed) C long, rather than unsigned.

I understand that, but after reading documentation, was expecting it to convert second argument to long type, which is not concerned with architecture dependant implementation of int.
msg102159 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-04-02 11:29
The original error, for a 32-bit machine, looks like expected behaviour to me;  at worst, it's a doc bug.  If the right-hand argument doesn't fit into a Py_ssize_t, then I think it's reasonable to refuse to do the shift.

But Antoine's 'outrageous left shift count' comes from the following code, in long_lshift in Objects/longobject.c.

	if ((long)(int)shiftby != shiftby) {
		PyErr_SetString(PyExc_ValueError,
				"outrageous left shift count");
		goto lshift_error;
	}

I think this code dates from the days when the number of 'digits' of a PyLong was held in an int rather than a Py_ssize_t, and I think it's a (minor) bug.  A 64-bit machine with sufficient RAM shouldn't mind shifting something by >= 2**31 places.
msg102473 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-04-06 16:26
Patch that unoutrages Python attached:  it allows shift counts of up to sys.maxsize in both left shift and right shift.  I don't have a test for this, since the only tests I can think of (e.g. actually doing 1 << (2**31)) require > 270Mb of RAM, and that may be too much for some of the buildbots.  (We could add a 'largemem' special resource to regrtest, I suppose, but it doesn't seem worth the effort just for this.)

This patch doesn't solve the OPs problem, though.  The PyLong code *could* be modified to accept larger shift counts (e.g., up to 8*PY_SSIZE_T_MAX;  beyond this you'll get a MemoryError anyway), but this seems like a significant effort for little gain.  It might make more sense for right shifting, though.  Anyway, I'd definitely call that a feature request.
msg102475 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-04-06 16:53
Patch applied in r79843 (and added forgotten Misc/NEWS entry in r79844);  merged to py3k in r79845.
msg102480 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-04-06 18:24
By the way, this has nothing to do with int versus long (unless I'm misunderstanding the original issue);  it's simply that large shift counts cause an OverflowError.

I've added a note about this to the docs in r79852.
History
Date User Action Args
2022-04-11 14:56:59adminsetgithub: 52506
2010-04-06 18:24:38mark.dickinsonsetstatus: open -> closed
resolution: fixed
messages: + msg102480
2010-04-06 16:53:57mark.dickinsonsetassignee: mark.dickinson ->
type: enhancement
messages: + msg102475
2010-04-06 16:26:58mark.dickinsonsetfiles: + no_outrage.patch
assignee: mark.dickinson
messages: + msg102473

keywords: + patch
2010-04-02 11:29:44mark.dickinsonsetmessages: + msg102159
2010-03-29 15:26:24owirjsetmessages: + msg101893
2010-03-29 14:55:28pitrousetnosy: + pitrou
messages: + msg101891
2010-03-29 13:13:07r.david.murraysetpriority: normal

components: + Interpreter Core, - None
versions: + Python 3.1, Python 2.7, Python 3.2
nosy: + mark.dickinson, r.david.murray

messages: + msg101889
stage: test needed
2010-03-29 12:29:20owirjcreate