classification
Title: random.choices does not work with negative weights
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Ted Whalen, aldwinaldwin, ariddell, davecom, mark.dickinson, rhettinger
Priority: normal Keywords:

Created on 2017-10-04 12:53 by ariddell, last changed 2019-07-19 06:59 by aldwinaldwin. This issue is now closed.

Messages (7)
msg303683 - (view) Author: Allen Riddell (ariddell) Date: 2017-10-04 12:53
Code to reproduce problem:

population = list(range(10))
weights = list(-1 * w for w in range(10))
[random.choices(population, weights) for _ in range(1000)]

will raise IndexError:

    358         bisect = _bisect.bisect
    359         total = cum_weights[-1]
--> 360         return [population[bisect(cum_weights, random() * total)] for i in range(k)]
    361 
    362 ## -------------------- real-valued distributions  -------------------

IndexError: list index out of range
msg303685 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2017-10-04 13:18
@ariddell: What behaviour did you want to see here?

It wouldn't have occurred to me to even try using `random.choices` with negative weights; forcing the weights to be nonnegative (with strictly positive sum) sounds like a natural restriction.
msg303697 - (view) Author: Allen Riddell (ariddell) Date: 2017-10-04 14:41
Upon some reflection, I think raising a ValueError is the right thing to do. Negative weights don't have an obvious interpretation.
msg303738 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-10-05 04:09
I'm content with the current exception.
msg323235 - (view) Author: David Kopec (davecom) Date: 2018-08-07 04:49
It's not a bug, but I agree with Allen that it could use a much more clear error message. I think his proposed ValueError makes a lot more sense than just raising an IndexError as currently occurs. This will help people debug their programs who don't even realize they're accidentally using negative weights to begin with.
msg336933 - (view) Author: Ted Whalen (Ted Whalen) Date: 2019-03-01 17:12
I think this should be reopened, as the behavior doesn't always raise an error, and, in fact, does something very unexpected:

Python 3.7.2 (default, Jan 13 2019, 12:50:01)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import Counter
>>> from random import choices
>>> Counter(choices("abcdefg", weights=(1,1,-1,1,1,0,1), k=10000))
Counter({'a': 2569, 'b': 2514, 'e': 2487, 'g': 2430})

It's really not clear to me why supplying a negative weight for "c" should have any effect on "d".
msg348145 - (view) Author: Aldwin Pollefeyt (aldwinaldwin) * Date: 2019-07-19 06:59
issue37624: not adding an extra O(n) step to check for unusual inputs with undefined meaning -- that would just impair the normal use cases for near zero benefit.
History
Date User Action Args
2019-07-19 06:59:20aldwinaldwinsetnosy: + aldwinaldwin
messages: + msg348145
2019-03-01 17:12:56Ted Whalensetnosy: + Ted Whalen
messages: + msg336933
2018-08-07 04:49:59davecomsetnosy: + davecom

messages: + msg323235
versions: + Python 3.7, - Python 3.6
2017-10-05 04:11:29rhettingersetstatus: open -> closed
resolution: not a bug
stage: resolved
2017-10-05 04:09:39rhettingersetassignee: rhettinger
messages: + msg303738
2017-10-04 14:41:15ariddellsetmessages: + msg303697
2017-10-04 13:18:50mark.dickinsonsetnosy: + rhettinger, mark.dickinson
messages: + msg303685
2017-10-04 12:53:19ariddellcreate