-
Notifications
You must be signed in to change notification settings - Fork 44
Description
Question
ALL EXAMPLES ARE RUN LOCALLY
in the example for htcondor (https://github.com/riga/law/blob/master/examples/sequential_htcondor_at_cern/analysis/tasks.py) workload_requires
is being used and results in the following graph:
Scheduled 45 tasks of which:
- 45 ran successfully:
- 32 CreateChars(...)
- 1 CreateFullAlphabet(...)
- 12 CreatePartialAlphabet(...)
But when I just comment it I get the following more clearer graph:
Scheduled 39 tasks of which:
- 39 ran successfully:
- 26 CreateChars(...)
- 1 CreateFullAlphabet(...)
- 12 CreatePartialAlphabet(...)
Does this change anything? Other than number of tasks decreases when no workflow_requirements is not provided from 45 to 39
In addition it is also possible by changing reruires and run to:
def requires(self):
# require CreateChars for each index referred to by the branch_data of _this_ instance
return CreateChars.req(self, branches=self.branch_data, branch=-1)
def run(self):
# gather characters and save them
alphabet = ""
for inp in self.input()['collection'].targets.values():
alphabet += inp.load()["char"]
....
to obtain the following graph:
And finally the result which I was expecting to see:
can be done by changing the CreateFullAlphabet:
def requires(self):
return CreatePartialAlphabet.req(self)
def run(self):
# loop over all targets holding partial alphabet fractions and concat them
inputs = self.input()["collection"].targets
parts = [
inp.load().strip()
for inp in inputs.values()
]
alphabet = "-".join(parts)
I would really appreciate if you could help me with that, struggled a lot with this trying to find the reason for workload_requires
. Thank you for reading