naïve way to return a generator instead of a list (not asynchronously) #585
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements #217 (* but does does implement #79 *)
This is a naïve way to return the results from Parallel.call: simply yield outputs instead of storing it inside a list.
The default behavior is not changed (the parameter
as_generator
is set by default tofalse
), but the__call__
now always use a generator on the result, it only transforms it to a list ifas_generator = False
.All the unitest are still green so it proves that the generator behaves as expected.
Here the generator return the results in the same order as usual, and not in asynchronously (= the first results returned are the fastest to be processed).
BUT since the main thread can still be used for other computations (rather than reading the generator), it could be combined with #582 to offer an API to the user to read the list of asynchronous results. If we change
self.last_async_output
in #582 to a list (let's sayself.async_output_list
) that record all the tasks that have already finished, then at any given time the user could inspect everything.