Title: itertools.accumulate should have an optional initializer argument
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.6
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: achampion, mark.dickinson, rhettinger
Priority: low Keywords:

Created on 2015-09-20 08:46 by achampion, last changed 2015-11-02 13:26 by achampion. This issue is now closed.

Messages (8)
msg251155 - (view) Author: Alun Champion (achampion) Date: 2015-09-20 08:46
itertools.accumulate should have an initializer with the same semantics as functools.reduce.
These two functions are closely related, reduce only providing you the end result accumulate providing an iterator over all the intermediate results.
However, if you want all the intermediate results to this reduce:

    functools.reduce(operator.mul, range(1, 4), 10)

You currently need to do:

    itertools.accumulate(itertools.chain([10], range(1,4), operator.mul)

Adding an optional initialiser argument would avoid this.
msg251185 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-09-21 01:28
I know this was considered at the beginning but I am not immediately remembering what the reason was for not doing it (however, I do remember looking to APL to see what was done for their well thought-out implementation of accumulate).

AFAIK, the case you sketched-out (computing running totals for four factorial starting from an initial product of 10) just doesn't come-up in the real-world.  I'm reluctant to have API feature-creep without strong use cases (it just makes the tool more complicated to learn, remember, and use).  When I get a chance, I'll go to github and run a code search to see whether people are routinely have to do a chain() operation to prepend a starting point; if it isn't rare, then there would be a better case for API expansion; if it is rate, then it goes in the it-isn't-worth it category.
msg251201 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-09-21 05:14
AFAICT other implementations of accumulators do not have an initializer.  See:



  \+ 1 2 3 4 5
  1 3 6 10 15

msg251220 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2015-09-21 12:09
A couple of observations:

1. Haskell's (rough) equivalents would be `mapAccumL` and `mapAccumR`, which do take an initializer [1].  The signature for both functions is something like:

(acc -> b -> (acc, c)) -> acc -> [b] -> (acc, [c]).

2. In the particular case of NumPy, I've found np.cumsum a pain to use on multiple occasions because of the missing left-hand point in the result.  Grepping through a couple of recent (real-world) projects produces code like:

    summed_widths = cumsum(hstack(([0], widths[:-1])))


    cumulative_segments = np.insert(np.cumsum(segment_lengths), 0, 0.0)

where that extra missing value is having to be inserted either in the input list or the output list. OTOH, none of those projects is natural fit for itertools, so changing itertools wouldn't help them directly.

msg253439 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-10-25 22:42
> OTOH, none of those projects is natural fit for itertools,
> so changing itertools wouldn't help them directly.

Indeed, this may not be a natural fit for itertools problems.

As a side note, the itertools were designed to form the elements of "an iterator algebra" with itertools.chain() as the intended compose-operation, so the OPs original solution was the right way to do it.  

The main question is whether the current solution is common enough to warrant being made a little more convenient (saving 9 characters), and whether the resulting code reads well:

current:  accumulate(chain([10]), range(1,4), operator.mul)
proposed: accumulate(range(1,4), operator.mul, 10) 

The latter spelling saves 9 characters; however, to my eyes, the "10" isn't obvious about what it doing and its placement is odd (the order of execution is 10, 1, 2, 3 eventhough the argument order has the range(1,4) as the first argument and the "10" as the third-argument).  I don't think the proposed spelling is doing us any favors in terms of code clarity -- the current spelling with chain() is more versatile and makes it perfectly clear that the 10 happens before the 1,2,3.

The OP made reference to functools.reduce() for comparison but we should keep in mind that reduce() was banished to functools because Guido found it to be relatively unintelligible and thought most code would be clearer without it.

As an additional data point, I did a GitHub code search.  In the first 20 pages of the search results, I found only three examples.  The first example seems reasonable.  The second example was a toy demonstration of "generator and iterator towers".  And the third example was a olympiad toy puzzle problem.

Github Code Search

def collect(seed, iterable):
    return it.accumulate(it.chain([seed], iterable))

for number, op, fac in itertools.chain(
     zip(itertools.accumulate(itertools.chain((nr,), range(2, 10)),
     operator.mul), itertools.repeat(" * "), range(2, 10)),

if sum(loop) % 2015 == 0 and not list(filter(test,
      itertools.accumulate(itertools.chain(start, loop)))):
msg253447 - (view) Author: Alun Champion (achampion) Date: 2015-10-26 00:57
If you are looking for other examples.
Here's a wheel factorization of an indefinite sieve (primes) that was peer reviewed on, see the final result in the last post by Will Ness: which included two uses of accumulate(chain(...), ...).

Constructing and unpacking the list in a chain seems like an unnecessary overhead when all you are looking for is an initial value. The reason I suggested it as the 3rd argument is it is the "most" optional argument and mirrors reduce (though iterable, function is the reverse of reduce).
msg253454 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-10-26 03:44
To me prime sieve demos and puzzle problems aren't motivating use cases.  Also, the two examples in the single stack-exchange answer become awkward because they would also need a middle argument for the accumulation function:

  cs = accumulate(chain([11], cycle(wh11)))
  wheel = accumulate(chain([psq+x[i]], cycle(x[i+1:] + x[:i+1])))

  cs = accumulate(cycle(wh11), operator.add, 11)
  wheel = accumulate(cycle(x[i+1:] + x[:i+1]), operator.add, psq+x[i])

IMO both of the newer ones are wordier, don't read well, and are slower.  

After more thought, I've decided to stick with the original decision and decline the feature request.  There are other major languages that seem to do fine without the feature, the current approach works fine, my search for possible use cases shows that they are uncommon or contrived, and looking at the revised examples I just don't think that the newer code reads well.
msg253909 - (view) Author: Alun Champion (achampion) Date: 2015-11-02 13:26
Understandable, though it would have also made the first order recurrence relationships more accurate, e.g. annualized interest on an initial balance:

    accumulate(repeat(None), lambda: bal, _: bal*1.05, initializer=1000)

vs. artificially repeating 1000:

    accumulate(repeat(1000), lambda: bal, _: bal*1.05)

Or, if the initialiser is expensive:

    accumulate(chain([1000], repeat(None)), lambda: bal, _: bal*1.05)
Date User Action Args
2015-11-02 13:26:47achampionsetmessages: + msg253909
2015-10-26 03:44:08rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg253454
2015-10-26 00:57:53achampionsetmessages: + msg253447
2015-10-25 22:42:58rhettingersetmessages: + msg253439
2015-09-21 12:09:07mark.dickinsonsetnosy: + mark.dickinson
messages: + msg251220
2015-09-21 05:23:37serhiy.storchakasettitle: itertools.accumlate should have an optional initializer argument -> itertools.accumulate should have an optional initializer argument
2015-09-21 05:14:58rhettingersetmessages: + msg251201
2015-09-21 01:28:33rhettingersetpriority: normal -> low

messages: + msg251185
2015-09-20 08:51:59serhiy.storchakasetassignee: rhettinger

type: enhancement
nosy: + rhettinger
versions: - Python 3.3, Python 3.4, Python 3.5
2015-09-20 08:46:40achampioncreate