Return backfill.Task from Start phase operations
#924
Conversation
14e56bb to
a65b6e0
Compare
a65b6e0 to
351f414
Compare
351f414 to
5e22a83
Compare
5e22a83 to
6423c1b
Compare
6423c1b to
d1f72a8
Compare
backfill.Task from operationsbackfill.Task from Start phase operations
There was a problem hiding this comment.
Some comments and a question about how this relates to the DBAction refactoring.
AIUI, the DBAction refactoring was going to make the operation Start methods return []DBAction which would then be merged/reordered/combined by a new orchestrator component. This component could also include combining triggers that are intended to act on the same table.
With this PR, we make operation Start methods return backfill jobs, which will allow us to combine backfill triggers, but what happens to the idea to return []DBAction from Start? It seems like returning []DBAction would allow us to solve a greater class of problems, including the one addressed by this PR.
How do you see the changes here fitting in with the broader plan to return []DBAction from operation Start methods?
Merging this branch will increase overall coverage
Coverage by fileChanged files (no unit tests)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. |
Yes. However, I am unhappy with returning 3 things, so I will try to find a way to decrease this number. |
From now on
Startfunctions of each operations return abackfill.Taskinstance. This struct contains information required by the backfill process. The tasks are collected in a singlebackfill.Job.This aggregates all relevant information for the backfill process: list of tables to backfill and all the triggers to load.