I'm running actually millions of tasks, so sending them all at once will
consume much more resources than needed.

The issue happens no only with 2 tasks in parallel but with higher numbers
as well.

Also your proposed solution, has the problem that when you are waiting for
the last tasks to finish you lose some parallelism.

In any case, it seems to me that there's some kind of race condition
preventing the task to finish, so if that's true the same could happen with
as_completed.
On Jun 16, 2014 2:00 PM, "Glenn Langford" <report@bugs.python.org> wrote:

>
> Glenn Langford added the comment:
>
> > Any ideas how to debug this further?
>
> Wherever the cause of the problem might live, and to either work around it
> or gain additional information, here is one idea to consider.
>
> Do you need to submit your Futures just two at a time, and tightly loop
> every 15s? Why not submit a block of a larger number and wait for the block
> with as_completed(), logging for each completion. Then submit another block
> when they are all done. To control how many run at one time, create the
> Executor with max_workers=2 for example. (I had an app that ran > 1,000
> futures in this way, which worked fine).
>
> In general I suggest to only timeout when there is really a problem, not
> as an expected event.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue20319>
> _______________________________________
>