Skip to content

Conversation

fcharras
Copy link
Contributor

This PR implements #217 (* but does does implement #79 *)

This is a naïve way to return the results from Parallel.call: simply yield outputs instead of storing it inside a list.

The default behavior is not changed (the parameter as_generator is set by default to false), but the __call__ now always use a generator on the result, it only transforms it to a list if as_generator = False.

All the unitest are still green so it proves that the generator behaves as expected.

Here the generator return the results in the same order as usual, and not in asynchronously (= the first results returned are the fastest to be processed).

BUT since the main thread can still be used for other computations (rather than reading the generator), it could be combined with #582 to offer an API to the user to read the list of asynchronous results. If we change self.last_async_output in #582 to a list (let's say self.async_output_list) that record all the tasks that have already finished, then at any given time the user could inspect everything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant